Closures, threads, and protos all go to grayagain for atomic re-traversal.
This covers all upvalue/stack/compiler mutations between mark steps without
needing to hook every individual write site.
Additional fixes this round:
- LUA_TUPVAL handling in markObject (5.1 reallymarkobject pattern)
- isCleared: strings never cleared from weak tables (5.1 lstring.c:337)
- Unified cleartable matching 5.1 (key OR value cleared removes entry)
- Clear stale weak flags when metatable is NULL
- lua_replace barrier hook (0x6F32B0) for upvalue pseudo-index writes
- Compiler barrier hooks (create_constant, finish_function) -- installed
but superseded by proto grayagain
- fullgc abandon walks all objects with makewhite before restart
- Removed old separate clearWeakValues/clearWeakKeys
CHUNK_SIZE=5000, confirmed incremental: mark=46, sweep=184 on large heaps.
No crashes or addon errors through enter world, /reload, logout/login cycles.
Full 5.1 incremental GC port with CHUNK_SIZE=5000. All crash scenarios
resolved: enter world, logout/login, /reload, /gcstress.
Compiler barriers (verified from Ghidra):
- Hook lua_parser_create_constant (0x6FD400) and
lua_parser_finish_function (0x6FCB00)
- ECX = LexState in all callers (not FuncState)
- Proto via: LexState+0x30 -> FuncState, FuncState+0x00 -> Proto
- Barrier-back: if proto is BLACK during mark, gray + grayagain
String/upvalue resurrection:
- Hook luaS_newlstr (0x6F9D00) -- resurrect dead strings from intern
- Hook luaF_findupval (0x6F9F10) -- resurrect dead open upvalues
- Birth mark patches for strings (0x6F9DC1) and upvalues (0x6F9F4A)
Other:
- Volatile reads/writes for all foreign memory access
- Fix upvalue traversal: dereference v pointer not inline value
- Remove bootstrap (luaC_link hook covers all objects except mainthread)
- Mainthread makewhite in markroot
- Remove 5.1 removeentry from traverseTable (5.0 tables don't expect TNONE)
- Clear stale weak flags when metatable is NULL
- fullgc abandon when lua_setgcthreshold forces GC mid-cycle
Remaining: /reload causes addon errors ("attempt to call a number value").
Not a crash -- investigation needed for the specific collection difference.
Critical fix: hook luaS_newlstr (0x6F9D00) for dead string resurrection
during incremental sweep. When the string intern table returns an existing
string with otherwhite (dead but not yet swept), flip its white bits to
resurrect it. Without this, code gets references to strings that sweep
will free, causing hash chain crashes. Matches 5.1 lstring.c:88.
Also:
- Upvalue resurrection: hook luaF_findupval (0x6F9F10), same pattern
- Upvalue birth mark: patch 0x6F9F4A so new open upvalues get currentwhite
- Volatile reads/writes on all memory accessors (foreign memory via @ptrFromInt)
- Fix upvalue traversal: dereference v pointer (+0x08) not inline value (+0x10)
- Fix upvalue birth mark address (0x6F9F4A not 0x6F9F48 -- immediate is last byte)
- Correct hook target: luaS_newlstr does intern lookup, not lua_create_string_object
CHUNK_SIZE=5000 (true incremental). Multi-step mark cycles confirmed working.
Open upvalues have v pointing to the Lua stack. Closed upvalues have v
pointing to the inline copy at upval+0x10. Reading upval+0x10 directly
was wrong for open upvalues -- gave stale/garbage data, causing
markObject to be called with invalid pointers (e.g. 0x3).
Fix: dereference upval->v at +0x08 to get the actual TValue location.
Also: volatile reads on all readU8/readU32/writeU8/writeU32 (correct
for foreign memory access via @ptrFromInt).
Major port of Lua 5.1 incremental GC patterns:
Tri-color system:
- Two-white (WHITE0/WHITE1) with currentwhite flip in atomic phase
- Objects born currentwhite via hooked luaC_link (0x6F7B20)
- String birth mark patched to currentwhite at each flip
- isDead: otherwhite check with FIXED exception for WoW
- makewhite on survivors (preserves KEYWEAK/VALUEWEAK/FIXED)
- Bootstrap: first cycle sets all existing objects to currentwhite
Mark system (5.1 reallymarkobject/propagatemark):
- Strings: stringmark inline (no gray stack)
- Userdata: go black inline, mark metatable
- Tables: gray2black, traverse, black2gray if weak
- Threads: NEVER black, always grayagain (re-traversed in atomic)
- Closures: go black normally
Forward barriers (5.1 luaC_barrierf at every write site):
- OP_SETUPVAL: VM patch at 0x6F8A97 with naked asm trampoline
- lua_setmetatable (0x6F4020): hooked, barrier on new metatable
- lua_setupvalue (0x6F47B0): hooked, barrier grays closure
- lua_setfenv (0x6F40D0): hooked, barrier grays object
Backward barriers (5.1 luaC_barrierback):
- lua_table_set_value (0x6FA840): black2gray + grayagain
- lua_table_set_int_key (0x6FAD80): black2gray + grayagain
Atomic phase (5.1 ordering):
- propagateall, re-traverse weak tables, mark running thread,
grayagain drain, separateudata, marktmu, cleartable, flip white
Status: enters world successfully with CHUNK_SIZE=5000 (incremental).
Crashes on UI reload (different crash pattern -- NULL+8 deref in mark,
not the old hash chain crash). Progress from "crashes on enter world"
to "crashes on UI reload."
Adds the infrastructure for generational GC on top of the incremental sweep:
- luaalloc: per-page age bitmap (1 bit per slot) lazily allocated with each
slab page. Public isOld/setOld/setYoung/clearAllAges API. Segment-indexed
for O(1) lookup in the write barrier.
- luagc: write barrier via Detour on lua_table_set_value. Records touched
old tables in a 4096-entry set. Pure bitmap read on every table write.
- luagc: minor/major cycle distinction. First collection, touched overflow,
and every 32nd minor are forced to major. Major clears the age bitmap and
touched set; minor preserves both. cycleFinish repopulates the bitmap by
walking rootgc after a full sweep completes.
Both cycles still run lua_gc_full_collection for now -- the custom minor
mark that actually skips untouched old objects is the next layer.
Previous crash investigations proved Detours on lua_table_set_value are
safe; earlier failures were from simultaneous marked-byte modifications,
not the Detour itself.
File_FindInArchive has two conditional jumps that skip calling
CheckFileExistence. Without NOPing them, preloadFileWithFlags
never reaches our hook and the game skips Bindings.xml loading
for embedded addons. This was previously only done by the
customassets module, so standalone variant DLLs (without
customassets) failed to load addon keybindings.
A/B results: hash_lookup prefetch/cache both regressed (original is
near-optimal at 59 bytes), VM opcode SSE2 patches were break-even.
Only luaS_newlstr hash pre-check shows consistent 40-45% per-call win.
Drop hash_lookup hook, VM jump table patches, and resize hook. Keep
newlstr Detour with per-call A/B rdtsc and OnWorldUpdate periodic dump.
Add GC_WRITE_BARRIER.md documenting the barrier globals at 0xCEEAC0/C4
found in lua_vm_execute (useful for luagc module).
bench: add Detour overhead micro-benchmark. Simulates zhook's
JMP+trampoline mechanism on mmap'd executable pages. Result: 5 cyc/call
overhead -- negligible vs the ~100 cyc/call newlstr savings.
Rootgc sweep chunked by temporarily NULLing a next pointer and calling
the original lua_gc_remove_objects on the truncated sublist. Swept
survivors are detached to a separate list to prevent re-sweeping.
Lists reconnected after final chunk.
Birth-mark via binary patch of luaC_link immediate byte (0x6F7B37)
ensures new objects born during sweep survive.
Profiling shows sweep chunks at 0-9ms each (was 225ms avg, 5s worst).
Remaining bottleneck: atomic mark (80ms) + udata/string sweep (44-92ms).
Incremental GC: atomic mark + batched rootgc sweep. Crashes from
objects born white during sweep -- needs birth-mark barrier fix.
Link hook (luaC_link) calling convention verified but interaction
with sweep_ptr at list head needs solving. Disabled pending fix.
AllocBench: realistic addon-like data structures (combat events,
player damage breakdowns with metatables, aura trackers with
closures). Cross-references shared spell/unit caches. These
complex structures reveal GC as the true stutter source -- simple
flat objects don't trigger meaningful GC pauses.
Removed arena pre-reservation (was fragmenting 32-bit address space).
Profiled 67M Lua allocations to find hot sizes. Lua table hash nodes
are 40 bytes, so hash tables produce allocs at 80, 160, 320, 640,
1280, 2560 (2..64 nodes). Added classes at these exact sizes.
Fixed size class LUT: was 16-byte granularity (size 22 mapped to
class 32 instead of 24). Now direct 1:1 lookup table, 4KB, one
byte load per alloc with zero computation.
Other changes:
- Removed pool_ctx==0 passthrough and seg_val==0 fallback (dead code)
- Removed pool_ctx param from slabFree/slabRealloc
- Added optional PROFILE mode with size histogram dump to log file
- Added AllocBench addon (/allocbench) for in-game benchmarking
Hook 0x686940 (testAABBFrustum): SSE center/extent P-vertex method replaces
x87 sign-bit corner selection. ~160M cycles in profile, 50% rejection rate.
Hook 0x686180 (FrustumCullBoundingBox 8-corner): SSE replacement transforms
all 8 AABB corners through view-proj, horizon buffer scan with 4-wide test.
Fix behind-camera bug in both FrustumCullBoundingBox variants (0x686000,
0x686180): original returned 0 for behind-camera objects, callers interpret
0 as "visible, render it". Now returns 2 (occluded) when all geometry is
behind the camera (Z < 0 in clip space).
Replace 4-byte per-allocation header with 64KB segment table for O(1)
class lookup. VirtualAlloc guarantees 64KB-aligned pages, so ptr>>16
maps directly to the class index. Zero per-allocation overhead.
- VirtualAlloc for slab pages (guaranteed 64KB alignment)
- VirtualAlloc for large allocs (>4096) with size+magic header
- pool_ctx==0 passthrough for non-Lua callers
WoW's memory_pool_allocate (0x6FAE90) does O(classes * pages) linear
scan on every free/realloc to find which pool owns a pointer. Our slab
stores slot size in a 4-byte header for O(1) lookup.
Baseline 2000ms -> testing with allocator enabled next.
- 15 size classes (16-4096) vs WoW's 6 (16-256)
- @memcpy for cross-class realloc (WoW uses manual dword loop)
- Large allocs (>4092 usable) fall through to game heap
- No C dependencies (replaces mimalloc which couldn't cross-compile)
NPC priority pass now whitelists standard vanilla interaction flags
(0x7FFF: gossip through repair) instead of treating any non-zero
npc_flags as interactable. Fixes void zones and visual-effect NPCs
(e.g. C'Thun Portal, npc_flags=0x2000000) blocking player selection.
Cascade raycast disabled entirely in GM mode (PLAYER_FLAGS_GM) so
GMs get unfiltered targeting.
Remove the core mutex that prevented secondary DLLs from installing
shared infrastructure hooks (Lua registration, file serving, engine
init/shutdown). zhook already has explicit E9-chain detection -- each
DLL's trampoline chains to the previous DLL's detour, and per-module
mutexes still prevent duplicate module hooks. DLL_PROCESS_DETACH fires
in reverse load order so detach unwinds correctly.
weirdperformance's filecache hook (File_FindInArchive) stayed active
during game teardown, returning stale archive/block pointers after
Storm freed its MPQ archives. This corrupted heap metadata, causing
ERROR #124 (SGroupPtr invalid block) on game close.
Set remove_on_shutdown=true so the hook is detached during
logoutDetour before Storm archive teardown begins.
Also wires up superweirdo module and clickthrough lateInit.
The base DLL target was compiled with SSE4.1+FMA+AVX, which meant LLVM
could emit FMA instructions (VFMADD etc.) anywhere in the main module.
This crashes on pre-Haswell CPUs that lack FMA support (e.g. i7-3930K).
Split into two targets: base target (SSE2 only) for all main code, and
sse_target (SSE4.1+FMA+AVX) only for the separately-compiled SSE object
files that replace hot game functions. This ensures FMA/AVX instructions
are confined to the performance modules.
Fixes#6.
Filter ritual (type 18) and mage portal (type 22) GOs whose creator
is a player not in the local player's party or raid. Completely
unclickable, not just deprioritized.
Also:
- Add wow.isInGroup() shared group membership check (party + raid)
- Add party/raid addresses to shared offsets.zig
- Fix dpslog party member GUID address (was 0xBC7600, correct: 0xBC6F48)
Verified from client's is_player_in_allowed_list @ 0x4e7f70
- Restore MSVC ABI (was accidentally GNU since v0.6.0, broke .CRT section)
- Replace game allocator with Windows process heap for filecache and
libdeflate malloc/free - game allocator not initialized during DllMain
when injected via CreateRemoteThread
- Defer timer calibration (Sleep 500ms) to lateInit - blocks under
loader lock during DllMain
- Remove exported malloc/free symbols from DLL
- Eliminate addObject compilation units for SSE files - direct @import
with AVX target instead
- Heap-allocate filecache (was 9.3MB static BSS)
- Strip transform44 of performance/ externs, pure profiling only
- Rename performance/ to weirdperformance/ to match module convention
- Skip default-off modules in all-variants build step
- Remove dead debug vars and stride logging from particle_sse
MarcelineVQ