benilla-octo/third_party/lua-src/BENILLA.md

# benilla's `lua-src` fork: what differs from upstream, and how to check

Upstream: [`mlua-rs/lua-src-rs`](https://github.com/mlua-rs/lua-src-rs), MIT, version `550.0.0`,
the version mlua 0.11 resolves to. Wired in through `[patch.crates-io]` in the workspace root.

## Why a fork exists at all

benilla's Lua has to accept exactly the grammar **the OctoWoW client** accepts, no more and no
less — and no OctoWoW client runs stock 1.12.1. `ClassicAPI.dll` ships with the realm's client
bundle and rewrites every chunk at the `luaL_loadbuffer` chokepoint (file scripts, `loadstring`
and XML `OnLoad` alike), adding five constructs Lua 5.0 never had: `#`, `%`, `...` as an
expression, `0x` hex literals and `[=[ ]=]` long brackets. benilla bakes the DLLs in, so that is
the grammar its addon corpus is written against.

So the target is a **union**, and the hunks below are that union: five restore 5.0 constructs 5.1
deleted (which the client takes and stock 5.1 rejects), one takes hex the way ClassicAPI does
without reopening 5.1's word-slurp, and one puts a token boundary back where the client has it.
A grammar difference in either direction is invisible to every instrument until an addon trips
over it, and the addon corpus asks about the difference directly.

The 1.12 addon corpus is Lua 5.0 code. It uses the iterator-less generic-for:

```lua
for k, v in someTable do ... end        -- no pairs(), no iterator function
```

183 of 218 corpus addons are reached by it (118 carry it, 65 inherit it through a declared
dependency), across 1,163 sites. On stock Lua 5.1 it raises `attempt to call a table value`, the
first session-start error for 60 of the 218.

Lua 5.1 removed it at the opcode level, which is why no layer above the VM can reach it: a `__call`
metamethod would need a per-type default metatable for tables, which 5.1 does not have; rewriting
the chunk source means parsing Lua with a regex; and `lua-src` ships 5.1 through 5.5, every one of
which removed it.

## What is not being done

benilla is not adopting Lua 5.0. It stays on 5.1.5 and restores the behaviour of a single opcode
that Lua itself shipped and labelled `/* for compatibility only */`. Every other 5.1 fix, and the
whole of mlua's surface, stay exactly as upstream.

## The delta, and the command that proves it

Nine hunks in four files. `src/lib.rs` additionally differs by having Lua 5.2/5.3/5.4/5.5 stripped
from the `Version` enum (with their source trees deleted): benilla builds 5.1 and only 5.1, and a
fork that still offered the other four would answer a request for one with a missing directory at
build time instead of a compile error.

Six hunks restore what 5.1 changed or removed:

| file | hunk | why |
|---|---|---|
| `lvm.c` | 5.0's `OP_TFORPREP` table-to-`next` substitution, folded into the top of `OP_TFORLOOP` | `for k, v in someTable do` raised "attempt to call a table value" |
| `luaconf.h` | `LUA_COMPAT_LSTR` `1` to `2` | 5.1 kept 5.0's `[[...]]` nesting machinery and put an advisory error in front of it; two corpus addons died on "nesting of `[[...]]` is deprecated" |
| `luaconf.h` | `LUA_QL(x)` back to 5.0's `` `x' `` from 5.1's `'x'` | every Lua error message quotes a program element, and `WoW.exe`'s own `.rdata` carries all five formats in the 5.0 spelling |
| `lparser.c` | 5.0's compat-semicolon skip restored at the top of `constructor()`'s field loop, 5.0's own line verbatim | one extra `;` after a field separator inside a table constructor (`Back_Title = AL["Factions"];;`, which AtlasLoot writes) is accepted by the client and was a parse error here |
| `lparser.c` | `recfield`'s `cc->nh++` moved back inside its `TK_NAME` arm, 5.0's own placement | a `[expr] = value` constructor field credits neither `OP_NEWTABLE` size hint in 5.0, so the table is born on the dummy node |
| `llex.c` | `read_numeral`'s trailing `while (isalnum(c) \|\| c == '_')` slurp replaced by 5.0's own exponent-digit loop | the client ends a numeral at its digits, so `if Item.ReplaceWords == 1then` (no space, `AdvancedTradeSkillWindow2.lua:3878`) compiles there and raised `malformed number` here, costing a 4000-line file and every global it defines |

### The three that used to be deleted, and are not any more

`simpleexp`'s `case TK_DOTS`, `getunopr`'s `case '#'` and `getbinopr`'s `case '%'` were removed
from this fork, because stock 1.12 rejects all three — byte-read out of the client's own parser
(`simpleexp 0x6fd240` has seven arms and no `TK_DOTS`; `getunopr 0x6fe0a0` tests exactly two
tokens, `OPR_NOUNOPR` = 2; `getbinopr 0x6fe0c0` bases its switch at `*` 0x2A so `%` 0x25 falls
below it, `OPR_NOBINOPR` = 14). All three are **restored**, as stock 5.1 writes them, because
ClassicAPI adds all three to the client benilla emulates. The byte-reads stay in the comments:
they are what says *why the question was ever open*, and they are what a strict-1.12 build would
delete again.

`...` is the one with a cost, and it is ClassicAPI's cost too. 170 corpus `loadstring` sites ask
which interpreter they are on with

```lua
local lua51 = loadstring("return function(...) return ... end") and true or false
```

and 92 library files in 24 folders branch on the answer. They now take the 5.1 branch, as they do
on a real OctoWoW client. Two of them branch *wrongly* there, and ClassicAPI carries a
compensation for each; benilla carries the same two, in
`crates/benilla-ui/src/script/compat/classic_api/lua51_probe_fallout.rs`.

What restoring the arm does **not** cost is 1.12's `arg` table: `LUA_COMPAT_VARARG` clears
`VARARG_NEEDSARG` only for a function that actually uses `...` as a value, so the shipped
FrameXML's `function f(...) ... arg.n ... end` still gets `arg`, and only 5.1-era code opts out.
The two dialects coexist one function at a time —
`script::lua50::tests::vararg_takes_both_dialects_one_function_at_a_time`.


To verify the Lua sources against upstream at any time:

```sh
# 550.0.0 is the pinned version; adjust the path if cargo's registry hash differs.
diff -r third_party/lua-src/lua-5.1.5 \
  ~/.cargo/registry/src/*/lua-src-550.0.0/lua-5.1.5
```

That must print exactly the nine hunks in the tables above and nothing else.

## The generic-for, in detail

The `lvm.c` comment carries the three details that are the client's own, each byte-read there: the
substitution test is a bare type-tag equality that never consults a metatable (a table carrying
`__call` still gets `next`); the callee is the global `next`, read raw and fetched fresh at every loop
entry (an addon assigning `next = myfn` changes every later generic-for in the session); and userdata
is not substituted. The substitution happens in `OP_TFORPREP`, an opcode 5.1 deleted, so the top of
`OP_TFORLOOP` is where 5.1 can host it.

## The numeral boundary, in detail

`read_numeral` is one routine in each dialect, and they part at one line. The client's
(`0x6ffee0`) is 5.0's: digits, at most one `.` and its digits, then — if an `e`/`E` is there — the
marker, an optional sign (the `0x2b`/`0x2d` test at `0x6fff04`, reached with the marker already
saved) and its own digit loop. Its only character-class test is the ctype digit mask `0x4` at
`0x6fff47`; past the loop it NUL-terminates the buffer and calls `luaO_str2d` (`0x6f5900`), raising
`malformed number` (`0x6fffa9`, string `0x8721c8`) when that fails. 5.1 instead ends with
`while (isalnum(ls->current) || ls->current == '_') save_and_next(ls);`, which carries the exponent
digits *and* swallows any word glued to the tail.

Two consequences, and both are the client's:

- `1then` is the number 1 and the name `then`. One corpus site —
  `AdvancedTradeSkillWindow2.lua:3878` — and it cost the whole file: a syntax error leaves every
  global it defines undefined, so its own XML then raised four more times calling them.
- 1.12 has **no hex literals**. The digit loop stops at the `x`, so `0xFF` is `0` and the name
  `xFF`, and the statement around it does not parse. A comment- and string-stripping scan of the
  installed corpus finds hex in code in exactly two files, `pfQuest/toolbox/pngLua/deflate.lua`
  and `numberlua.lua`, which no `.toc` row reaches (only `toolbox/extractor.lua` loads them, and it
  is unlisted too) — so zero loaded sites, which is why this deletion is as safe as the other
  three. Our own shipped Lua has none either.

The slurp could not simply be deleted: 5.1 had no exponent-digit loop of its own, so `1e2` would
have lexed as `1e`. The hunk is a replacement, and the test is
`script::lua50::tests::a_numeral_ends_where_the_digits_do_so_a_glued_keyword_is_its_own_token`.

One difference inside the routine is left alone: 5.1's opening `do ... while (isdigit(c) || c ==
'.')` takes *repeated* dots where 5.0 takes at most one, so `1..2` is one malformed token here and
`1.` followed by a `..` there. Both raise; only the message differs, and a comment- and
string-stripping scan of the corpus finds no site, so there is nothing to tell apart.