From 23c14aaec16ba5137684fa5c37829ac26dfceb73 Mon Sep 17 00:00:00 2001 From: Sam Date: Wed, 23 Sep 2026 18:47:48 +0700 Subject: [PATCH] the reference tables, the install guard and the vendored forks stop naming the maintainer's analysis The notes and comments say what is true of the 1.12.1 client and cite the byte address; the record numbers, the dates and the private repository's file paths go. Co-Authored-By: Claude Fable 5.1 --- .claude/hooks/guard-wow-install.sh | 47 +++------ .gitignore | 8 +- reference/README.md | 131 ++++++++++-------------- third_party/kira/BENILLA.md | 4 +- third_party/lua-src/BENILLA.md | 116 +++++++++------------ third_party/lua-src/lua-5.1.5/lparser.c | 107 +++++++++---------- third_party/lua-src/lua-5.1.5/luaconf.h | 7 +- third_party/lua-src/lua-5.1.5/lvm.c | 51 ++++----- 8 files changed, 192 insertions(+), 279 deletions(-) diff --git a/.claude/hooks/guard-wow-install.sh b/.claude/hooks/guard-wow-install.sh index a0b05ff0..56f1a17f 100755 --- a/.claude/hooks/guard-wow-install.sh +++ b/.claude/hooks/guard-wow-install.sh @@ -1,33 +1,18 @@ #!/usr/bin/env bash -# PreToolUse(Edit|Write|NotebookEdit) guard (docs/METHOD.md hard rules; decision 1486): refuse to write -# anything into the 1.12.1 install. Exit 2 blocks the call and hands stderr back to the agent. +# PreToolUse(Edit|Write|NotebookEdit) guard: refuse to write anything into the 1.12.1 install. +# Exit 2 blocks the call and hands stderr back to the agent. # -# The rule is the director's, stated 2026-08-21 while B261 was being built: *benilla never changes -# or adds anything in the WoW folder*. It has two halves and this file is the second one. +# benilla reads a WoW install and never writes to one (docs/METHOD.md). The client persists +# everything through `crate::local_state` into `benilla-config/` beside the binary, and +# `scripts/smoke.sh` fails a run that leaves the install changed; this hook is the same rule for +# the tools. The install is the player's own copy of somebody else's game, and "what here is +# benilla's?" is only answerable while nothing of ours is anywhere else. # -# 1 · the CLIENT never writes there — every persisted file resolves through -# `crate::local_state` into `benilla-config/` beside the binary (0954/1175), and -# `scripts/smoke.sh` now measures that a full run leaves the install byte-for-byte as it -# found it; -# 2 · WE never write there either — which is this hook, because we already had. The install -# currently carries `apitrace-stderr.log`, `apitrace-stderr-ring.log` and -# `abbey-standing-frame1501.png`: three files a session dropped in the nearest folder to -# hand. Nobody decided to; that is exactly why prose was never going to be enough (0976's -# argument, one folder over). -# -# WHY IT MATTERS MORE HERE THAN IT LOOKS. The install is not ours to write to at all — it is the -# player's own copy of somebody else's game, and the one thing the repo may never contain -# (docs/METHOD.md). On THIS machine it is also a symlink into the sibling RE repo (`wow-5875-re/WoW`), -# so a stray write lands in another repository's working tree and shows up in their `git status`. -# And it makes "what here is benilla's?" unanswerable, which is the whole reason 0954 put our state -# in one visible folder. -# -# Scoped by PHYSICAL path, not by name: the slot's `WoW` is a symlink, so a write can arrive +# Scoped by physical path, not by name: a checkout's `WoW` may be a symlink, so a write can arrive # spelled either way and only `cd -P` collapses the two. `$WOW_DATA` is honoured because that is -# the resolver's own first step — a machine that points the client elsewhere protects that instead. +# the resolver's own first step. # -# `BENILLA_ALLOW_INSTALL_WRITE=1` overrides, the `WT_ALLOW_PRIMARY` shape: structurally hard rather -# than merely forbidden, and a deliberate exception names itself in the transcript. +# `BENILLA_ALLOW_INSTALL_WRITE=1` overrides: a deliberate exception names itself in the transcript. input=$(cat) path=$(printf '%s' "$input" | jq -r '.tool_input.file_path // .tool_input.notebook_path // empty' 2>/dev/null) @@ -60,20 +45,18 @@ for root in "${WOW_DATA:-}" "$primary/WoW" "$cwd/WoW"; do case "$real/" in "$root"/*) cat >&2 <typeorigin`. +The 1.12.1 client's entire global namespace: 21,555 names, `nametypeorigin`. -**Where it comes from.** wow-5875-re ships an addon (`W5875Capture`) that dumps the in-world `_G` -of the *running* reference client, and its output is committed there as a fixture. This is the -authoritative answer to "what can an addon see in 1.12", and it is strictly better evidence than -the binary's registration table would be: it is the actual runtime table, so it carries -FrameXML-defined globals as well as engine ones — which is exactly the surface an addon sees, and -therefore exactly the surface an addon-compatibility target needs (decision 1189). +Where it comes from: a dump of `_G` inside the running reference client, which is the surface an +addon sees (FrameXML-defined globals as well as engine ones), unioned with every global the shipped +UI defines. The union matters because the twelve `Blizzard_*` addons are LoadOnDemand: a live dump +only carries their globals if their windows were opened first, so the 1,986 rows a dump cannot see +carry type `lod`. The table answers "is this a 1.12 name?", not "was this name live in one session". -**Plus what a capture structurally cannot see** (decision 1200). The twelve `Blizzard_*` addons are -**LoadOnDemand**: unless the player had opened the talent window, the trainer, the auction house and -the rest before the dump, none of their globals is in it — so the capture alone reported -`TalentFrame`, `MacroFrame`, `TradeSkillFrame`, `CraftFrame`, `TrainerFrame` and `InspectFrame` as -absent from 1.12 while listing their `*_LoadUI` stubs. The table is therefore the capture **union** -every global the shipped-UI corpus *defines*, and those 1,986 extra rows carry type **`lod`** so -the two halves stay distinguishable. The question this artifact answers is "is this a 1.12 name?", -not "was this name live at one moment in one session". - -**The `origin` column** answers a second question the capture alone cannot — *whose job is it to -provide this?* — by attributing every name to its definition site in a complete 1.12 shipped-UI -corpus: +The `origin` column says whose job it is to provide the name, attributed by its definition site in +the complete 1.12 shipped-UI corpus: | origin | count | meaning | |---|---|---| -| `engine` | 1,104 | the C client provides it. **benilla implements these in Rust.** | -| `framexml` | 20,427 | the shipped UI defines it. **The stock files run off the player's own install** (`ui_script::reference_ui`). | -| `lua` | 24 | Lua 5.0's own runtime. mlua gives us these. | +| `engine` | 1,104 | the C client provides it; benilla implements these in Rust | +| `framexml` | 20,427 | the shipped UI defines it; the stock files run off the player's own install (`ui_script::reference_ui`) | +| `lua` | 24 | Lua 5.0's own runtime | -Of the `framexml` rows, **1,986 carry type `lod`**: defined by the shipped UI but absent from the -capture, which is what a LoadOnDemand addon looks like from a live dump (decision 1200). +The `framexml` bulk is UI objects, `ContainerFrame1Item16IconTexture` and its 11,000 siblings, +which materialize from the XML and are nobody's to implement. The number that matters for the API +surface is the function split: 1,100 engine functions, 1,075 FrameXML functions. A name defined in +shipped Lua is FrameXML's, and hardcoding it in Rust is a category error even when it works. -The `framexml` bulk is UI objects — `ContainerFrame1Item16IconTexture` and its 11,000 siblings — -which materialize from whatever XML we ship and are nobody's to implement. The number that -matters for the API surface is the function split: **1,100 engine functions, 1,075 FrameXML -functions.** This is the `tContains` distinction (1187 got it right for one name) applied -wholesale: a name defined in shipped Lua is FrameXML's, and hardcoding it in Rust is a category -error even when it works. +Regenerating: `scripts/gen-reference-globals.py`, whose header carries the method and the traps. It +needs a 1.12 install and the maintainer's runtime capture of the reference client's `_G` +(`WOW_RE=`), which is not in this repo, so it is a manual regeneration. The table is +stable: it describes a client that shipped in 2006. -**Regenerating** — `scripts/gen-reference-globals.py`, whose header carries the full method and -the traps. It needs a 1.12 install and the RE repo's runtime capture of the reference client's -`_G` (`WOW_RE=`), so it is a manual regeneration like `genmap.sh`, not something -CI can run. The artifact is stable: it describes a client that shipped in 2006. - -**Reading it** — `scripts/api-coverage.sh` is the instrument. It asks a real `UiScript::new()` -what benilla exposes and reports `have / missing / beyond-1.12`, always with 1178's split. -`crates/benilla-ui/src/script/tests/reference_surface.rs` is the gate: it fails if benilla grows a -global that 1.12 does not have and nobody wrote down why. +Reading it: `scripts/api-coverage.sh` asks a real `UiScript::new()` what benilla exposes and reports +`have / missing / beyond-1.12`. `crates/benilla-ui/src/script/tests/reference_surface.rs` is the +gate: it fails if benilla grows a global that 1.12 does not have and nobody wrote down why. ## `1.12-shapes.tsv` and `1.12-events.tsv` -wow-5875-re's binding-shape table (what each Lua verb takes and returns; decision 1842, gated by -`ui_script/shape_gate.rs`) and the event table with the arguments each producer pushes -(decision 2140, gated by `ui_script/event_shape_gate.rs`). Each file's header carries its column -contract. +The binding-shape table (what each Lua verb takes and returns, gated by `ui_script/shape_gate.rs`) +and the event table with the arguments each producer pushes (gated by +`ui_script/event_shape_gate.rs`). Each file's header carries its column contract. -**Regenerating** — `1.12-shapes.tsv` is vendored as wow-5875-re writes it. +Regenerating: `1.12-shapes.tsv` is vendored as the reference analysis writes it. `scripts/gen-reference-events.py` derives `1.12-events.tsv` from two more vendored tables, -`1.12-event-catalog.tsv` (event id → name, off the client's name-pointer array) and -`1.12-event-firesites.tsv` (every fire site with the format string it pushes), so it runs from -any clone and reproduces the committed file byte for byte. +`1.12-event-catalog.tsv` (event id to name, off the client's name-pointer array) and +`1.12-event-firesites.tsv` (every fire site with the format string it pushes), so it runs from any +clone and reproduces the committed file byte for byte. ## `1.12-message-catalog.tsv` -The client's message registry (`0xb4b498`: id, key, kind, sound, chat type), vendored from -wow-5875-re; `scripts/gen-message-catalog.py` re-shapes it into -`crates/benilla-ui/src/messages/catalog.rs`, the table `CGGameUI::DisplayError` indexes. +The client's message registry (`0xb4b498`: id, key, kind, sound, chat type). +`scripts/gen-message-catalog.py` re-shapes it into `crates/benilla-ui/src/messages/catalog.rs`, +the table `CGGameUI::DisplayError` indexes. ## `1.12-verb-events.tsv` -Every event the reference fires **from inside a Lua verb**, keyed by the verb — the verb's own -body, or a helper that only registered verbs call (`0x4d8c90` is the trainer's). The third -question on the event seam: the two tables above say what an event carries and whether something -fires it; this one says *who*. A stock file that calls a verb for its side effect of an event -repaints nothing when the verb fires nothing, which is how the trainer filter regressed twelve -days after its window went stock (decision 2244), and the class every migrated window shares. +Every event the reference fires from inside a Lua verb, keyed by the verb: the verb's own body, or +a helper that only registered verbs call (`0x4d8c90` is the trainer's). The two tables above say +what an event carries and whether something fires it; this one says who. A stock file that calls a +verb for its side effect of an event repaints nothing when the verb fires nothing. -**Regenerating** — `scripts/gen-reference-verb-events.py`, whose header carries the rule, the -two families it drops (glue-space verbs, the unit-field bridge) and why getters are excluded (a -getter's fire is a cache-miss re-query, a state event benilla's feeds fire off the packet). A fire -site is attributed to a function against wow-re's verified extents (the ledgers' `size=`) plus the -disassembly's padding boundaries, and a site past its candidate's extent is left out rather than -mis-filed (decision 2257: four of the first table's rows were callbacks reached by pointer, or a -site past an extent). The fire sites and the shapes are the vendored tables above; the function -names, the ledgers and the disassembly are the RE repo's (`WOW_RE=`), so this one -is a manual regeneration there. +Regenerating: `scripts/gen-reference-verb-events.py`, whose header carries the rule, the two +families it drops (glue-space verbs, the unit-field bridge) and why getters are excluded (a getter's +fire is a cache-miss re-query, a state event benilla's feeds fire off the packet). A fire site is +attributed to a function by the verified function extents plus the disassembly's padding +boundaries, and a site past its candidate's extent is left out rather than mis-filed. The fire +sites and the shapes are the vendored tables above; the function names and the disassembly are the +maintainer's (`WOW_RE=`), so this one is a manual regeneration. -**Reading it** — `crates/benilla-app/src/ui_script/verb_event_gate.rs` is the gate: for every -pair whose verb benilla registers, the module that registers the verb fires the event, or the -pair is declared there — `ELSEWHERE` (a feed fires it on the state the verb changes, and the -named file is checked to) or `GAP` (nothing does, with the reason). Decision 2251. +Reading it: `crates/benilla-app/src/ui_script/verb_event_gate.rs` is the gate. For every pair whose +verb benilla registers, the module that registers the verb fires the event, or the pair is declared +there: `ELSEWHERE` (a feed fires it on the state the verb changes, and the named file is checked +too) or `GAP` (nothing does, with the reason). diff --git a/third_party/kira/BENILLA.md b/third_party/kira/BENILLA.md index d2fb6c33..c5b7cb65 100644 --- a/third_party/kira/BENILLA.md +++ b/third_party/kira/BENILLA.md @@ -10,8 +10,8 @@ upstream files are CRLF and are left so, except the one file patched. The mixer runs on benilla's own audio render thread (`sound/output`), and with a raid buffing itself — a dozen live voices, most of them spatial — that thread cost **~1.5 ms of every 60 Hz -frame** on the M2 Air (decision 1945: a sampled profile of the crowd rig's Blackrock leg with -sound on, `Track::process` two thirds of it). Upstream's sub-track `process` spatializes **every +frame** on an M2 Air (a sampled profile with a dozen spatial voices playing, `Track::process` +two thirds of it). Upstream's sub-track `process` spatializes **every frame**: two vector normalizes, a length, the attenuation curve and a dB→amplitude `powf` per frame per spatial track, then two more `powf`s per frame for the volume tween and the fade. At 48 kHz × 12 tracks that is ~1.7 M `powf`s a second plus the vector math, for gains that change diff --git a/third_party/lua-src/BENILLA.md b/third_party/lua-src/BENILLA.md index 4e4722db..f5dc6a09 100644 --- a/third_party/lua-src/BENILLA.md +++ b/third_party/lua-src/BENILLA.md @@ -1,82 +1,67 @@ -# benilla's `lua-src` fork — what differs from upstream, and how to check +# benilla's `lua-src` fork: what differs from upstream, and how to check -Upstream: [`mlua-rs/lua-src-rs`](https://github.com/mlua-rs/lua-src-rs), MIT, version `550.0.0` — +Upstream: [`mlua-rs/lua-src-rs`](https://github.com/mlua-rs/lua-src-rs), MIT, version `550.0.0`, the version mlua 0.11 resolves to. Wired in through `[patch.crates-io]` in the workspace root. ## Why a fork exists at all -**benilla's Lua has to accept exactly the grammar the 1.12.1 client's Lua accepts — no more and no -less.** Every hunk below is that one rule: three restore 5.0 constructs 5.1 deleted, three delete -5.1 constructs 5.0 never had. A grammar difference in either direction is invisible to every -instrument we own until an addon trips over it, and the corpus *asks about* the difference -directly (decisions 1208, 2101). +benilla's Lua has to accept exactly the grammar the 1.12.1 client's Lua accepts, no more and no +less. Every hunk below is that one rule: three restore 5.0 constructs 5.1 deleted, three delete 5.1 +constructs 5.0 never had. A grammar difference in either direction is invisible to every instrument +until an addon trips over it, and the addon corpus asks about the difference directly. -The 1.12 addon corpus is Lua **5.0** code. It uses the iterator-less generic-for: +The 1.12 addon corpus is Lua 5.0 code. It uses the iterator-less generic-for: ```lua for k, v in someTable do ... end -- no pairs(), no iterator function ``` -**183 of 218 corpus addons are reached by it** (118 carry it, 65 inherit it through a declared -dependency), across 1,163 sites. On stock Lua 5.1 it raises `attempt to call a table value`, and it -is the **first session-start error for 60 of 218 addons — three quarters of everything that breaks -once the client is actually running**, eleven times the next cause. +183 of 218 corpus addons are reached by it (118 carry it, 65 inherit it through a declared +dependency), across 1,163 sites. On stock Lua 5.1 it raises `attempt to call a table value`, the +first session-start error for 60 of the 218. -Lua 5.1 removed it at the **opcode** level, which is why no layer above the VM can reach it. Every -other dialect gap this project has closed was reachable from inside Lua — `table.setn` is a library -function, `bit` is a library, `print` is a global (decisions 1194, 1196). This one is not: +Lua 5.1 removed it at the opcode level, which is why no layer above the VM can reach it: a `__call` +metamethod would need a per-type default metatable for tables, which 5.1 does not have; rewriting +the chunk source means parsing Lua with a regex; and `lua-src` ships 5.1 through 5.5, every one of +which removed it. -- a `__call` metamethod would work, but the tables are the addon's own and Lua 5.1 has no - per-*type* default metatable for tables; -- rewriting the chunk source means parsing Lua with a regex to tell `in t do` from `in pairs(t) do`, - and getting that wrong changes behaviour silently; -- there is no `lua50` backend — `lua-src` ships 5.1 through 5.5 and every one of them removed it. +## What is not being done -## What is NOT being done - -**benilla is not adopting Lua 5.0.** We stay on 5.1.5 and restore the behaviour of a single opcode +benilla is not adopting Lua 5.0. It stays on 5.1.5 and restores the behaviour of a single opcode that Lua itself shipped and labelled `/* for compatibility only */`. Every other 5.1 fix, and the -whole of mlua's surface, stay exactly as upstream. That is a far smaller commitment than a version -downgrade, and it is the entire reason this is a one-hunk fork rather than a vendored 5.0. +whole of mlua's surface, stay exactly as upstream. ## The delta, and the command that proves it -**Current state: eight hunks, in three files.** `src/lib.rs` additionally differs by having Lua -5.2/5.3/5.4/5.5 stripped from the `Version` enum (with their source trees deleted) — benilla builds -5.1 and only 5.1, and a fork that still *offered* the other four would answer a request for one with -a missing directory at build time instead of a compile error. +Eight hunks in three files. `src/lib.rs` additionally differs by having Lua 5.2/5.3/5.4/5.5 stripped +from the `Version` enum (with their source trees deleted): benilla builds 5.1 and only 5.1, and a +fork that still offered the other four would answer a request for one with a missing directory at +build time instead of a compile error. -**Five hunks restore what 5.1 changed or removed:** +Five hunks restore what 5.1 changed or removed: -| file | hunk | why | record | -|---|---|---|---| -| `lvm.c` | 5.0's `OP_TFORPREP` table→`next` substitution, folded into the top of `OP_TFORLOOP` | `for k, v in someTable do` raised "attempt to call a table value"; it was the first session-start error for **60** corpus addons | 1215 | -| `luaconf.h` | `LUA_COMPAT_LSTR` `1` → `2` | 5.1 kept 5.0's nesting machinery and put an advisory error in front of it; two corpus addons died on "nesting of `[[...]]` is deprecated" | — | -| `luaconf.h` | `LUA_QL(x)` back to 5.0's `` `x' `` from 5.1's `'x'` | every Lua error message quotes a program element, and `WoW.exe`'s own `.rdata` carries all five formats in the 5.0 spelling (`` bad argument #%d to `%s' (%s) ``, `` attempt to %s %s `%s' (a %s value) ``, `` %s:%d: %s near `%s' `` at the client's `0x87217c`, …). ~80 corpus addons ship AceLibrary, whose `argCheck` parses its own caller back out of `debugstack()` with a pattern that requires the backquote | 2122 | -| `lparser.c` | 5.0's compat-semicolon skip restored at the top of `constructor()`'s field loop — 5.0's own line, verbatim, comment included | one extra `;` after a field separator inside a table constructor (`Back_Title = AL["Factions"];;`, ×20 in AtlasLoot's `ButtonRegistry.lua`) parses on the client and died here with "unexpected symbol near `;`"; statement-level `;;` stays rejected by both dialects | 1315 | -| `lparser.c` | `recfield`'s `cc->nh++` moved back INSIDE its `TK_NAME` arm — 5.0's own placement | a `[expr] = value` constructor field must credit **neither** `OP_NEWTABLE` size hint, so the table is born on the dummy node, the first store rehashes, and the dense integer keys land in an ARRAY part that `next` walks ascending. 5.1 pre-sizes a node vector instead and leaves them in the hash, where `next` is slot order — so every list in every saved-variables file came back scrambled (Bagnon's keyring drew first) | 2111 | +| file | hunk | why | +|---|---|---| +| `lvm.c` | 5.0's `OP_TFORPREP` table-to-`next` substitution, folded into the top of `OP_TFORLOOP` | `for k, v in someTable do` raised "attempt to call a table value" | +| `luaconf.h` | `LUA_COMPAT_LSTR` `1` to `2` | 5.1 kept 5.0's `[[...]]` nesting machinery and put an advisory error in front of it; two corpus addons died on "nesting of `[[...]]` is deprecated" | +| `luaconf.h` | `LUA_QL(x)` back to 5.0's `` `x' `` from 5.1's `'x'` | every Lua error message quotes a program element, and `WoW.exe`'s own `.rdata` carries all five formats in the 5.0 spelling | +| `lparser.c` | 5.0's compat-semicolon skip restored at the top of `constructor()`'s field loop, 5.0's own line verbatim | one extra `;` after a field separator inside a table constructor (`Back_Title = AL["Factions"];;`, which AtlasLoot writes) is accepted by the client and was a parse error here | +| `lparser.c` | `recfield`'s `cc->nh++` moved back inside its `TK_NAME` arm, 5.0's own placement | a `[expr] = value` constructor field credits neither `OP_NEWTABLE` size hint in 5.0, so the table is born on the dummy node | -**Three delete what 5.1 added** — all in `lparser.c`, all byte-read out of the client's own parser +Three delete what 5.1 added, all in `lparser.c`, all byte-read out of the client's own parser (`simpleexp 0x6fd240`, `getunopr 0x6fe0a0`, `getbinopr 0x6fe0c0`), all landing on 5.0's own `unexpected symbol` at `prefixexp 0x6fde40`: -| hunk | why | record | -|---|---|---| -| `simpleexp`'s `case TK_DOTS` deleted — `...` is no longer an expression | the Ace2 corpus **asks** which interpreter it is on by compiling `return function(...) return ... end`; 170 `loadstring` sites in the 219-addon corpus are that question and 92 library files gate on it. Answering `true` put Cartographer on its client-2.0 branch, hooking `CloseSpecialWindows` — a name 1.12 does not have. Also restores 5.0's rule that **every** vararg function gets its `arg` table | 1208, 2101 | -| `getunopr`'s `case '#'` deleted — no length operator | the client's `getunopr` tests exactly two tokens (`-`, `not`) and its `OPR_NOUNOPR` is **2**: a three-member enum, 5.0's. 5.0 asks a table with `table.getn` and a string with `string.len` | 2101 | -| `getbinopr`'s `case '%'` deleted — no modulo operator | the client's `getbinopr` switch is based at `'*'` (0x2A), so `%` (0x25) is below its range and reaches `OPR_NOBINOPR` = **14**: a fifteen-member `BinOpr`, 5.0's. 5.0 spells it `math.mod` | 2101 | +| hunk | why | +|---|---| +| `simpleexp`'s `case TK_DOTS` deleted: `...` is not an expression | the Ace2 corpus asks which interpreter it is on by compiling `return function(...) return ... end`; 170 `loadstring` sites in the corpus are that question, and 92 library files in 24 folders branch on the answer | +| `getunopr`'s `case '#'` deleted: no length operator | the client's `getunopr` tests exactly two tokens (`-`, `not`) and its `OPR_NOUNOPR` is 2, a three-member enum; 5.0 asks a table with `table.getn` and a string with `string.len` | +| `getbinopr`'s `case '%'` deleted: no modulo operator | the client's `getbinopr` switch is based at `'*'` (0x2A), so `%` (0x25) is below its range and reaches `OPR_NOBINOPR` = 14, a fifteen-member `BinOpr`; 5.0 spells it `math.mod` | -The ordering was deliberate and it held: the fork landed **inert**, byte-identical to upstream, so -the build pipeline was proven transparent *before* any behavioural payload went into it — and when -the suite then moved, the patch was the only thing that could have moved it. Every hunk is measured -against the corpus rather than argued: the long-string one alone took loaded 134 → 139 and -survivors 99 → 104. - -**The deletions are safe because nothing the reference runs uses those constructs, and that was -scanned rather than assumed** (2101 §4): a comment/string-stripping scan of the extracted 1.12 -FrameXML (177 files), GlueXML, Blizzard's own addons and the director's installed AddOns folder -finds **zero** sites of all three. Our own engine Lua did use them and was rewritten into the 5.0 -spellings; the rule now is that no chunk of ours may use a construct the client's parser rejects. +The deletions are safe because nothing the reference runs uses those constructs: a comment- and +string-stripping scan of the 1.12 FrameXML (177 files), GlueXML, Blizzard's own addons and the +corpus finds zero sites of all three. No chunk of benilla's own Lua may use a construct the client's +parser rejects. To verify the Lua sources against upstream at any time: @@ -86,20 +71,13 @@ diff -r third_party/lua-src/lua-5.1.5 \ ~/.cargo/registry/src/*/lua-src-550.0.0/lua-5.1.5 ``` -That must print **exactly** the eight hunks in the tables above and nothing else. If it ever prints -more, something crept in. +That must print exactly the eight hunks in the tables above and nothing else. -## Status of the payload +## The generic-for, in detail -**Landed.** The reverse-engineering answer this was deliberately blocked on came back from -`wow-5875-re` (`system/ui/scratch/lua-generic-for.md`) and is what the `lvm.c` comment cites, detail -by detail: a **bare type-tag** test that never consults a metatable (so a table carrying `__call` -still gets `next`), the **global** `next` read raw and fetched fresh at every loop entry (so an addon -assigning `next = myfn` changes every later generic-for in the session), and userdata *not* -substituted. - -Waiting was the right call twice over. Checking stock 5.0 first corrected two assumptions the -project had already written down — the substitution happens in `OP_TFORPREP`, an opcode 5.1 deleted, -*not* in `OP_TFORLOOP`; and the test is a bare `ttistable` with no `__call` check. Stock 5.0 was -corroboration, never the authority: the client links 5.0 but may be modified, and only the byte-read -of the client settles what it actually does. +The `lvm.c` comment carries the three details that are the client's own, each byte-read there: the +substitution test is a bare type-tag equality that never consults a metatable (a table carrying +`__call` still gets `next`); the callee is the global `next`, read raw and fetched fresh at every loop +entry (an addon assigning `next = myfn` changes every later generic-for in the session); and userdata +is not substituted. The substitution happens in `OP_TFORPREP`, an opcode 5.1 deleted, so the top of +`OP_TFORLOOP` is where 5.1 can host it. diff --git a/third_party/lua-src/lua-5.1.5/lparser.c b/third_party/lua-src/lua-5.1.5/lparser.c index 13e7d04c..cdaf099b 100644 --- a/third_party/lua-src/lua-5.1.5/lparser.c +++ b/third_party/lua-src/lua-5.1.5/lparser.c @@ -509,22 +509,17 @@ static void constructor (LexState *ls, expdesc *t) { checknext(ls, '{'); do { lua_assert(cc.v.k == VVOID || cc.tostore > 0); - /* BENILLA: Lua 5.0's constructor compat-semicolon, restored (decision 1315; - ** see third_party/lua-src/BENILLA.md). 5.0 opened this loop with exactly the - ** line below -- its own comment read "compatibility only", a Lua 4.0 leftover - ** (4.0 separated a constructor's list part from its record part with `;`). - ** 5.1 deleted the line, so one extra `;` after a field separator -- the - ** `Back_Title = AL["Factions"];;` shape AtlasLoot's ButtonRegistry.lua writes - ** twenty times -- became "unexpected symbol near ';'" here while the 1.12.1 - ** client (Lua 5.0) accepts it. Statement-level `;;` stays rejected by both, - ** which is the half wow-5875-re byte-verified first (system/ui/scratch/ - ** lua-dialect.md 9.1), and byte-verified in the client itself rather than - ** inferred from stock 5.0: constructor is 0x6fd430, and its loop head - ** 0x6fd4a0 calls testnext(ls,';') WITHOUT testing EAX -- a skip, not a - ** separator check -- while the loop's continuation at 0x6fd4f5/0x6fd505 does - ** test it. Of the 16 call sites of testnext 0x6fccf0 image-wide, the only - ** other consumer of `;' is chunk 0x6fcccc, at STATEMENT level, with no such - ** head skip. This line restores the constructor context only. */ + /* BENILLA: Lua 5.0's constructor compat-semicolon, restored (see + ** third_party/lua-src/BENILLA.md). 5.0 opened this loop with exactly the + ** line below, a Lua 4.0 leftover; 5.1 deleted it, so one extra `;' after a + ** field separator (`Back_Title = AL["Factions"];;', which AtlasLoot writes) + ** raised "unexpected symbol near ';'" while the 1.12.1 client accepts it. + ** Statement-level `;;' stays rejected by both. Byte-read in the client: + ** constructor is 0x6fd430, and its loop head 0x6fd4a0 calls testnext(ls,';') + ** without testing EAX (a skip, not a separator check), while the loop's + ** continuation at 0x6fd4f5/0x6fd505 does test it; of testnext 0x6fccf0's 16 + ** call sites, the only other `;' consumer is chunk 0x6fcccc, at statement + ** level, with no such head skip. */ testnext(ls, ';'); /* compatibility only */ if (ls->t.token == '}') break; closelistfield(fs, &cc); @@ -766,39 +761,32 @@ static void simpleexp (LexState *ls, expdesc *v) { init_exp(v, VFALSE, 0); break; } - /* BENILLA: 5.1's vararg EXPRESSION arm, deleted (decision 2101; see + /* BENILLA: 5.1's vararg EXPRESSION arm, deleted (see ** third_party/lua-src/BENILLA.md). Lua 5.0 has no grammar for `...' as a - ** value -- a vararg function reads its extra arguments out of the implicit - ** `arg' table -- and `...' as an expression is the construct the Ace2 corpus - ** uses to ASK which interpreter it is running: + ** value: a vararg function reads its extra arguments out of the implicit + ** `arg' table. `...' as an expression is how the Ace2 corpus asks which + ** interpreter it is on: ** ** local lua51 = loadstring("return function(...) return ... end") and true or false ** - ** 170 loadstring sites across the 219-addon corpus are that question, and 92 - ** library files in 24 folders gate on it (decision 1208). Answering it wrong - ** makes every one of them take a client-2.0 branch on a 1.12 client -- which - ** is how Cartographer came to hook `CloseSpecialWindows', a name 1.12 does - ** not have. + ** and 92 library files in 24 folders branch on the answer, taking a + ** client-2.0 path on a 1.12 client when it is wrong. ** - ** Byte-verified in the client itself, not inferred from stock 5.0. simpleexp - ** is 0x6fd240; it switches on `token - 0x7B' bounded at 0xA3, indexes the - ** byte table at 0x6fd338 and jumps through the 8-entry table at 0x6fd318. - ** TK_DOTS is 0x118 (TK_NAME is 0x116 -- wow-5875-re system/ui/scratch/ - ** lua-dialect.md 9), so its index byte is 0x9D, whose value is 7: the DEFAULT - ** arm, 0x6fd30a, which tail-jumps to primaryexp 0x6fdb00. The seven real arms - ** are `{', TK_FALSE, TK_FUNCTION, TK_NIL, TK_TRUE, TK_NUMBER and TK_STRING -- - ** stock Lua 5.0's simpleexp exactly, with no vararg case. + ** Byte-read in the client: simpleexp is 0x6fd240; it switches on + ** `token - 0x7B' bounded at 0xA3, indexes the byte table at 0x6fd338 and + ** jumps through the 8-entry table at 0x6fd318. TK_DOTS is 0x118, so its + ** index byte is 0x9D, whose value is 7: the DEFAULT arm, 0x6fd30a, which + ** tail-jumps to primaryexp 0x6fdb00. The seven real arms are `{', TK_FALSE, + ** TK_FUNCTION, TK_NIL, TK_TRUE, TK_NUMBER and TK_STRING: stock Lua 5.0's + ** simpleexp exactly, with no vararg case. So `...' as a value reaches + ** prefixexp 0x6fde40, whose head accepts only `(' or TK_NAME and otherwise + ** raises "unexpected symbol" (0x871e80); deleting the arm reproduces that + ** error here, verbatim. ** - ** So `...' as a value reaches prefixexp 0x6fde40, whose head accepts only `(' - ** or TK_NAME and otherwise raises "unexpected symbol" (0x871e80) -- the same - ** reject site lua-dialect.md 9 documents for a stray `;'. Deleting the arm - ** reproduces that error here, verbatim. - ** - ** It also restores 5.0's `arg' rule for free: the deleted line cleared - ** VARARG_NEEDSARG for any function that mentioned `...', so `arg' was not - ** built for it. With the arm gone, NEEDSARG is never cleared and EVERY vararg - ** function gets its `arg' table -- which is what 5.0 does and what all 177 - ** files of the shipped 1.12 FrameXML read. */ + ** It also restores 5.0's `arg' rule: the deleted line cleared + ** VARARG_NEEDSARG for any function that mentioned `...', so with the arm + ** gone every vararg function gets its `arg' table, which is what 5.0 does + ** and what the shipped 1.12 FrameXML reads. */ case '{': { /* constructor */ constructor(ls, v); return; @@ -821,15 +809,14 @@ static UnOpr getunopr (int op) { switch (op) { case TK_NOT: return OPR_NOT; case '-': return OPR_MINUS; - /* BENILLA: 5.1's `#' length operator, deleted (decision 2101). 5.0 asks a - ** table for its size with table.getn/getn and a string with string.len; the - ** operator does not exist and neither does its __len metamethod. - ** Byte-verified: getunopr is 0x6fe0a0, a leaf with exactly TWO token tests - ** -- `cmp ecx,0x2D' (`-') then `cmp ecx,0x10E' (TK_NOT), then setne/inc -- so - ** its OPR_NOUNOPR is 2, i.e. the enum has three members, not 5.1's four. The - ** metamethod-name pool at 0x871896 agrees: no __len (wow-5875-re - ** system/ui/scratch/lua-dialect.md 1, discriminator 4). A `#' therefore - ** reaches simpleexp's default arm and raises "unexpected symbol near `#'". */ + /* BENILLA: 5.1's `#' length operator, deleted. 5.0 asks a table for its + ** size with table.getn and a string with string.len; the operator does not + ** exist and neither does its __len metamethod. Byte-read in the client: + ** getunopr is 0x6fe0a0, a leaf with exactly two token tests, `cmp ecx,0x2D' + ** (`-') then `cmp ecx,0x10E' (TK_NOT), so its OPR_NOUNOPR is 2: a + ** three-member enum, not 5.1's four. The metamethod-name pool at 0x871896 + ** has no __len. A `#' therefore reaches simpleexp's default arm and raises + ** "unexpected symbol near `#'". */ default: return OPR_NOUNOPR; } } @@ -841,16 +828,14 @@ static BinOpr getbinopr (int op) { case '-': return OPR_SUB; case '*': return OPR_MUL; case '/': return OPR_DIV; - /* BENILLA: 5.1's `%' modulo operator, deleted (decision 2101). 5.0 spells it - ** math.mod (and the bare global `mod'), which this VM already publishes. - ** Byte-verified: getbinopr is 0x6fe0c0 and its switch is based at `ecx-0x2A' - ** -- `*', the LOWEST token it handles. `%' is 0x25, below the base, so it is + /* BENILLA: 5.1's `%' modulo operator, deleted. 5.0 spells it math.mod + ** (and the bare global `mod'), which this VM publishes. Byte-read in the + ** client: getbinopr is 0x6fe0c0 and its switch is based at `ecx-0x2A' (`*', + ** the lowest token it handles). `%' is 0x25, below the base, so it is ** outside the range test (`cmp eax,0xF2; ja') and falls to the default, - ** 0x6fe129, `mov eax,0xE' = OPR_NOBINOPR 14 -- a 15-member BinOpr, 5.0's, not - ** 5.1's 16-member one with OPR_MOD. Had 5.1's `%' case been present the - ** switch would have been based at 0x25. The metamethod pool has no __mod - ** either (wow-5875-re system/ui/scratch/lua-dialect.md 1, discriminator 4). - ** priority[] is indexed by the BinOpr enum, which is untouched -- only the + ** 0x6fe129, `mov eax,0xE' = OPR_NOBINOPR 14: a 15-member BinOpr, 5.0's, not + ** 5.1's 16-member one with OPR_MOD. The metamethod pool has no __mod either. + ** priority[] is indexed by the BinOpr enum, which is untouched; only the ** token that reaches OPR_MOD is gone. */ case '^': return OPR_POW; case TK_CONCAT: return OPR_CONCAT; diff --git a/third_party/lua-src/lua-5.1.5/luaconf.h b/third_party/lua-src/lua-5.1.5/luaconf.h index b2e29f17..fc3a12c8 100644 --- a/third_party/lua-src/lua-5.1.5/luaconf.h +++ b/third_party/lua-src/lua-5.1.5/luaconf.h @@ -201,7 +201,7 @@ ** CHANGE it if you want a different appearance. */ /* -** BENILLA (decision 2122): 1.12's Lua is 5.0, which quotes every program element in an error +** BENILLA: 1.12's Lua is 5.0, which quotes every program element in an error ** message with a BACKQUOTE and an apostrophe. 5.1 introduced this macro and made it two ** apostrophes. Read out of WoW.exe's own .rdata, which carries all five of the formats this macro ** feeds, in the 5.0 spelling: @@ -368,9 +368,8 @@ ** off the advisory error when nesting [[...]]. */ /* BENILLA: 2, the 5.0 behaviour. 1.12.1 ships Lua 5.0, where `[[ ... [[ ... ]] ... ]]` -** nests; 5.1 kept the machinery and put an advisory error in front of it. Two corpus -** addons stop loading on "nesting of [[...]] is deprecated" — a 5.1 opinion about a -** dialect we are not targeting. Same family as the OP_TFORPREP restoration (1215). +** nests; 5.1 kept the machinery and put an advisory error in front of it, on which two +** corpus addons stop loading ("nesting of [[...]] is deprecated"). */ #define LUA_COMPAT_LSTR 2 diff --git a/third_party/lua-src/lua-5.1.5/lvm.c b/third_party/lua-src/lua-5.1.5/lvm.c index 50c89e75..b412a29c 100644 --- a/third_party/lua-src/lua-5.1.5/lvm.c +++ b/third_party/lua-src/lua-5.1.5/lvm.c @@ -680,42 +680,35 @@ void luaV_execute (lua_State *L, int nexeccalls) { } case OP_TFORLOOP: { StkId cb = ra + 3; /* call base */ - /* BENILLA: Lua 5.0's OP_TFORPREP, folded into the loop opcode. - ** See third_party/lua-src/BENILLA.md and benilla decision 1215. + /* BENILLA: Lua 5.0's OP_TFORPREP, folded into the loop opcode (see + ** third_party/lua-src/BENILLA.md). ** - ** 5.0 ran this ONCE at loop entry in its own opcode (index 30, 157 bytes, verified in - ** the 1.12.1 client at 0x6f94a2): when a generic-for's generator is a table, the table - ** becomes the STATE and the generator becomes the global `next`. 5.1 deleted the opcode, - ** so `for k, v in someTable do` -- which 183 of 218 corpus addons are reached by, and - ** which is the first session-start error for 60 of them -- raises "attempt to call a - ** table value". + ** 5.0 ran this once at loop entry in its own opcode (index 30, 157 bytes, at + ** 0x6f94a2 in the 1.12.1 client): when a generic-for's generator is a table, the + ** table becomes the STATE and the generator becomes the global `next`. 5.1 deleted + ** the opcode, so `for k, v in someTable do`, which most 1.12 addons use, raised + ** "attempt to call a table value". ** - ** Placing it here is timing-exact rather than approximate: 5.1's parser emits a JMP - ** straight to this instruction, so the top of TFORLOOP IS 5.0's prep point, before the - ** first call. It is self-clearing -- after one pass `ra` holds a function, so the test - ** fails on every later iteration -- which is what makes an entry-only rule safe to - ** express as a per-iteration check. + ** The top of TFORLOOP is 5.0's prep point exactly: 5.1's parser emits a JMP straight + ** to this instruction, before the first call. The check is self-clearing, since after + ** one pass `ra` holds a function, so an entry-only rule is safe as a per-iteration + ** test. ** - ** Three details are the client's, not conveniences, each byte-verified in wow-5875-re - ** system/ui/scratch/lua-generic-for.md: - ** - the test is a BARE TYPE TAG equality. The handler makes no metatable access at - ** all, so a table carrying __call still gets `next` and never reaches tryfuncTM. - ** "not callable" and "no __call" are different conditions and both diverge here. - ** - the callee is the GLOBAL `next`, read RAW (no __index) from the thread's own + ** Three details are the client's, each byte-read there: + ** - the test is a BARE TYPE TAG equality. The handler makes no metatable access, so + ** a table carrying __call still gets `next` and never reaches tryfuncTM. + ** - the callee is the GLOBAL `next`, read raw (no __index) from the thread's own ** globals table, fetched fresh at every loop entry. An addon that assigns - ** `next = myfn` therefore changes every later generic-for in the session. That is - ** observable, and it is reproduced rather than improved on. + ** `next = myfn` therefore changes every later generic-for in the session. ** - userdata is NOT substituted (tag mismatch), so it still reaches __call, and a ** nil/string/number generator still raises on iteration 1, not at entry. ** - ** KNOWN, MEASURED DIVERGENCE THIS DOES NOT CLOSE. 5.0's generic-for has two hidden - ** locals, not three: ra+2 is the first user variable AND the control, so assigning to it - ** inside the body changes what the next iteration is asked for. 5.1 gives the control - ** its own slot, and this patch does not move it -- that is a register-layout property - ** owned by the code generator, not by this handler. Measured before shipping - ** (scripts/generic-for-control-census.py): 95 addons contain the shape, concentrated in - ** two files nearly everything embeds -- Dewdrop-2.0.lua (123 sites) and AceEvent-2.0.lua - ** (113). Recorded rather than hidden; closing it is a separate change. + ** Deviation from 1.12 this does not close: 5.0's generic-for has two hidden locals, + ** not three, so ra+2 is the first user variable AND the control, and assigning to it + ** inside the body changes what the next iteration asks for. 5.1 gives the control its + ** own slot, a register-layout property of the code generator, not of this handler. + ** 95 corpus addons contain the shape, concentrated in Dewdrop-2.0.lua (123 sites) and + ** AceEvent-2.0.lua (113). */ if (ttistable(ra)) { setobjs2s(L, ra+1, ra);