mirror of
https://github.com/brues-code/ClassicAPI.git
synced 2026-11-16 16:10:36 +00:00
The load hook gated on raw bytes — memchr for `#`/`%` and three scalar scans for `...`, `0x` and `[=` — then tokenized the whole chunk, and RewriteAll threw the token vector away when no operator token existed. A `%` in a format string or a `...` in prose was enough, so nearly every addon file and every SavedVariables file paid for a token stream it did not need: on the dev install, 723 of 1391 addon files (101.6 of 160 MB on disk) and 39 of 301 SV files (16.7 of 16.9 MB). The lexer is now `Lex<Sink>`, one body with two sinks. RunPasses runs it first with `FlagSink`, which records only whether `#`, `%`, `...`, a hex number or a leveled long bracket occurred outside a string or comment, and tokenizes only when one did. The byte gates are gone (the flag lex replaces all five) and RewriteAll takes the flags instead of rescanning the tokens. A chunk with no real 5.1 syntax costs one pass and no allocation, and the string/comment skipping stays single-sourced. Measured in game after the change: 40,921 chunks / 19.1 MB per login, 112 chunks / 2.8 MB tokenized, 39 ms inside the rewrite in total. `_classicapi_TranspileStats()` exposes those counters.