Files
ClassicAPI/docs
Brues d98d3a9fd5 perf(luasyntax): decide with one allocation-free lex before tokenizing a chunk
The load hook gated on raw bytes — memchr for `#`/`%` and three scalar
scans for `...`, `0x` and `[=` — then tokenized the whole chunk, and
RewriteAll threw the token vector away when no operator token existed.
A `%` in a format string or a `...` in prose was enough, so nearly every
addon file and every SavedVariables file paid for a token stream it did
not need: on the dev install, 723 of 1391 addon files (101.6 of 160 MB
on disk) and 39 of 301 SV files (16.7 of 16.9 MB).

The lexer is now `Lex<Sink>`, one body with two sinks. RunPasses runs it
first with `FlagSink`, which records only whether `#`, `%`, `...`, a hex
number or a leveled long bracket occurred outside a string or comment,
and tokenizes only when one did. The byte gates are gone (the flag lex
replaces all five) and RewriteAll takes the flags instead of rescanning
the tokens. A chunk with no real 5.1 syntax costs one pass and no
allocation, and the string/comment skipping stays single-sourced.

Measured in game after the change: 40,921 chunks / 19.1 MB per login,
112 chunks / 2.8 MB tokenized, 39 ms inside the rewrite in total.
`_classicapi_TranspileStats()` exposes those counters.
2026-09-11 21:05:38 -05:00
..
2026-05-07 15:08:45 -05:00
2026-05-07 15:52:48 -05:00
2026-05-10 00:10:06 -05:00
2026-05-07 12:23:57 -05:00
2026-05-07 16:28:32 -05:00
2026-05-07 12:23:57 -05:00
2026-05-07 15:08:45 -05:00
2026-05-07 15:08:45 -05:00
2026-05-09 14:23:40 -05:00