Untrack TODO.md, RELEASING.md, RELEASE_NOTES.md, ideas/ and
docs/CLAUDE_PROPER_OBJECT_REGISTRATION_PLAN.md -- internal planning and
agent notes with no reason to be published. Files stay on disk locally.
Replace remaining absolute machine paths in docs with relative ones
(one referenced a separate private project), and swap decorative emoji
for ASCII markers.
README: lead with the no-longer-actively-developed notice.
Keeps the source README (a superset - documents every module, not just the
released subset) and include/weirdutils_api.h over the release-trimmed
root copy.
# Conflicts:
# README.md
# include/weirdutils_api.h
build.zig.zon pointed at ../zhook, so a clone of this repo alone could not
build. Pin the Codeberg tarball for f1b252e (current zhook master) with its
hash; zig fetches it automatically. Verified by building with no sibling
zhook checkout present.
Brings in .gitea/ issue templates and preserves the release tag ancestry
(v0.1.0-v0.7.0 were tagged on the distribution branch, not on source).
# Conflicts:
# README.md
The remote was previously a distribution-only point for pre-built DLLs.
This opens the source.
- LICENSE: Unlicense, with a GPL-3.0 carve-out for src/dpslog/WeirdDPSMate
(a DPSMate fork that keeps its own license)
- README.md replaces the stale internal one with the user-facing docs from
DLL_README.md, swapping the 'Why No Source Code?' section for build and
layout notes. DLL_README.md is dropped; one README now serves both.
- RELEASING.md: drop the trim-the-README-per-release dance and the
remote/WeirdUtils/ distribution clone, both obsolete now
- gitignore agent/editor scratch, build caches, the vendored WSBT addon,
and the WeirdThreat/uwu-logs checkouts (separate upstream repos)
- Commit outstanding module work: superweirdo, clickthrough portal visuals,
transform44 decompiles, worldmarkers demo presets, tools/
Adds bone_sse64.zig as an f64-intermediate port of transformMatrix4x4, used as
the active hook. M2 bone matrices are built and multiplied as [16]f64 and only
narrow to f32 on final store into the bone output buffer -- matching the x87
original's rounding profile (wide intermediates, single f32 store) and keeping
M2 vertex positions aligned with the terrain/projected-texture pipeline.
Also fixes, in both bone_sse (f32) and bone_sse64:
- Pre-billboard tx/ty/tz accumulation order (row 0 = pz+px+py; rows 1/2 = pz+py+px)
- Post-billboard pos_y/pos_z accumulation order (py+pz+px)
- Post-billboard scale-recompute accumulation order (row0 + row2 + row1)
- Billboard types 2/4 normalize using f64 intermediates (load-bearing for camera
basis vectors -- pure f32 drifted from x87 by a ULP per axis and caused
particle emitters to jitter on camera motion)
Additional bone_sse64-specific changes:
- Local attachmentRecursion64 that recurses into transformImpl_SSE64 instead of
bone_sse.transformImpl_SSE, so attached child models stay on the f64 path
- child_padding (this+0x84) computed with f64 intermediates
bone_sse remains the reference f32 implementation; its struct fields, inline
helpers, and section-loop fns are now `pub` so bone_sse64 can share them
(types/interpolation helpers/post-loop loops). Artifact size is unchanged.
build.zig adds bench_bone_sse64 object; src/bench/main.zig runs the new variant
through the same warmup/timing harness and prints SSE vs SSE64 vs BASELINE
cycles plus a parity check.
C runtime starts in "C" locale (dot decimals) on all platforms.
WoW doesn't call setlocale. The integer encoding was unnecessary
and would break sync with older builds.
Addon messages now encode coordinates as integers (*1000) to avoid
locale-dependent decimal separators breaking cross-client sync.
DLL detects group leave via LEADER_GUID dropping to zero and clears
all markers locally -- no addon permission checks needed.
Adds World Markers section (new module shipping in 0.7.0). Updates the
Performance section with Lua runtime optimizations (allocator, GC,
string interning, pattern prefilter). Fixes Log Sessions documentation
to reflect per-day rollover. Adds worldmarkers, clickthrough, and
weirdperformance to the DLL auto-discovery list in weirdutils_api.h.
si_frustumCullBBox (0x686000, center+radius variant) was culling any
bounding volume whose center had view-space z < 0, but an object whose
center is behind the camera can still have its front extent inside the
frustum (think of standing next to a big WMO -- the building's center
can be a few units behind you while its front wall is still in frame).
Use the bounding sphere rather than the center point: only cull when
the closest point of the sphere is itself behind the camera, i.e.
center.z + |radius| < 0. Straddling cases now fall through to the
near-plane passthrough and get rendered.
si_frustumCullBBox8 (0x686180, 8-corner variant) already does the
right thing here -- it only returns "all behind" when every transformed
corner has z < 0, which is a true convex-hull test for an AABB.
Drop the top-level luastr and luavm module flags and wire both through
weirdperformance, matching the existing luaalloc/luagc sub-module pattern.
Also removes luavm's A/B rdtsc instrumentation now that the newlstr hash
pre-check is production-only.
- build.zig: remove luastr/luavm from module_list
- DLL_README.md: add Lua Runtime bullet under Performance, swap
em/en-dashes for ASCII
- src/luavm/: delete (files moved into src/weirdperformance/ which was
already the tracked location in HEAD)
Current state CRASHES on load due to pre-alloc hooks with wrong calling
conventions. pushcclosure hook is still enabled with partially disabled
others. Needs all pre-alloc hooks disabled or CCs fixed before use.
Implemented:
- Full 5.1 tri-color incremental GC
- 5.1-style singlestep budget loop (replaces chunk-based approach)
- Per-step rdtsc timing (ZGCStats returns max_step_us, atomic_us)
- Configurable gcstepsize/gcstepmul/gcpause via ZGCTune()
- Pre-alloc GC hooks for 7 API functions (5.1 luaC_checkGC pattern)
- Stress test addon: /gclife, /gcsweep, /gccompare
Broken:
- Pre-alloc hooks have CC mismatches causing immediate crash on load
- pushfstring is cdecl variadic, not fastcall (removed)
- pushstring has ambiguous CC (removed, covered by pushlstring)
- pushcclosure hook still active -- may be causing the crash
- Half-initialized objects cause garbage gcptr under heavy stress
weirdperformance: re-enabled luagc.install() + luastr + luavm
- Per-step rdtsc timing: ZGCStats returns max_step_us and atomic_us
- AllocBench /zgcstats frame shows step/atomic timing with color coding
- /gccompare command for timing forced GC cycles
- Honor lua_setgcthreshold forced collections (collectgarbage() with low
threshold now triggers GC instead of being silently ignored)
- Disable luaalloc in weirdperformance when luagc is active (no conflict)
- Closures go to grayagain (covers all upvalue mutation paths)
Closures, threads, and protos all go to grayagain for atomic re-traversal.
This covers all upvalue/stack/compiler mutations between mark steps without
needing to hook every individual write site.
Additional fixes this round:
- LUA_TUPVAL handling in markObject (5.1 reallymarkobject pattern)
- isCleared: strings never cleared from weak tables (5.1 lstring.c:337)
- Unified cleartable matching 5.1 (key OR value cleared removes entry)
- Clear stale weak flags when metatable is NULL
- lua_replace barrier hook (0x6F32B0) for upvalue pseudo-index writes
- Compiler barrier hooks (create_constant, finish_function) -- installed
but superseded by proto grayagain
- fullgc abandon walks all objects with makewhite before restart
- Removed old separate clearWeakValues/clearWeakKeys
CHUNK_SIZE=5000, confirmed incremental: mark=46, sweep=184 on large heaps.
No crashes or addon errors through enter world, /reload, logout/login cycles.
Full 5.1 incremental GC port with CHUNK_SIZE=5000. All crash scenarios
resolved: enter world, logout/login, /reload, /gcstress.
Compiler barriers (verified from Ghidra):
- Hook lua_parser_create_constant (0x6FD400) and
lua_parser_finish_function (0x6FCB00)
- ECX = LexState in all callers (not FuncState)
- Proto via: LexState+0x30 -> FuncState, FuncState+0x00 -> Proto
- Barrier-back: if proto is BLACK during mark, gray + grayagain
String/upvalue resurrection:
- Hook luaS_newlstr (0x6F9D00) -- resurrect dead strings from intern
- Hook luaF_findupval (0x6F9F10) -- resurrect dead open upvalues
- Birth mark patches for strings (0x6F9DC1) and upvalues (0x6F9F4A)
Other:
- Volatile reads/writes for all foreign memory access
- Fix upvalue traversal: dereference v pointer not inline value
- Remove bootstrap (luaC_link hook covers all objects except mainthread)
- Mainthread makewhite in markroot
- Remove 5.1 removeentry from traverseTable (5.0 tables don't expect TNONE)
- Clear stale weak flags when metatable is NULL
- fullgc abandon when lua_setgcthreshold forces GC mid-cycle
Remaining: /reload causes addon errors ("attempt to call a number value").
Not a crash -- investigation needed for the specific collection difference.
Critical fix: hook luaS_newlstr (0x6F9D00) for dead string resurrection
during incremental sweep. When the string intern table returns an existing
string with otherwhite (dead but not yet swept), flip its white bits to
resurrect it. Without this, code gets references to strings that sweep
will free, causing hash chain crashes. Matches 5.1 lstring.c:88.
Also:
- Upvalue resurrection: hook luaF_findupval (0x6F9F10), same pattern
- Upvalue birth mark: patch 0x6F9F4A so new open upvalues get currentwhite
- Volatile reads/writes on all memory accessors (foreign memory via @ptrFromInt)
- Fix upvalue traversal: dereference v pointer (+0x08) not inline value (+0x10)
- Fix upvalue birth mark address (0x6F9F4A not 0x6F9F48 -- immediate is last byte)
- Correct hook target: luaS_newlstr does intern lookup, not lua_create_string_object
CHUNK_SIZE=5000 (true incremental). Multi-step mark cycles confirmed working.
Open upvalues have v pointing to the Lua stack. Closed upvalues have v
pointing to the inline copy at upval+0x10. Reading upval+0x10 directly
was wrong for open upvalues -- gave stale/garbage data, causing
markObject to be called with invalid pointers (e.g. 0x3).
Fix: dereference upval->v at +0x08 to get the actual TValue location.
Also: volatile reads on all readU8/readU32/writeU8/writeU32 (correct
for foreign memory access via @ptrFromInt).
Major port of Lua 5.1 incremental GC patterns:
Tri-color system:
- Two-white (WHITE0/WHITE1) with currentwhite flip in atomic phase
- Objects born currentwhite via hooked luaC_link (0x6F7B20)
- String birth mark patched to currentwhite at each flip
- isDead: otherwhite check with FIXED exception for WoW
- makewhite on survivors (preserves KEYWEAK/VALUEWEAK/FIXED)
- Bootstrap: first cycle sets all existing objects to currentwhite
Mark system (5.1 reallymarkobject/propagatemark):
- Strings: stringmark inline (no gray stack)
- Userdata: go black inline, mark metatable
- Tables: gray2black, traverse, black2gray if weak
- Threads: NEVER black, always grayagain (re-traversed in atomic)
- Closures: go black normally
Forward barriers (5.1 luaC_barrierf at every write site):
- OP_SETUPVAL: VM patch at 0x6F8A97 with naked asm trampoline
- lua_setmetatable (0x6F4020): hooked, barrier on new metatable
- lua_setupvalue (0x6F47B0): hooked, barrier grays closure
- lua_setfenv (0x6F40D0): hooked, barrier grays object
Backward barriers (5.1 luaC_barrierback):
- lua_table_set_value (0x6FA840): black2gray + grayagain
- lua_table_set_int_key (0x6FAD80): black2gray + grayagain
Atomic phase (5.1 ordering):
- propagateall, re-traverse weak tables, mark running thread,
grayagain drain, separateudata, marktmu, cleartable, flip white
Status: enters world successfully with CHUNK_SIZE=5000 (incremental).
Crashes on UI reload (different crash pattern -- NULL+8 deref in mark,
not the old hash chain crash). Progress from "crashes on enter world"
to "crashes on UI reload."
Adds the infrastructure for generational GC on top of the incremental sweep:
- luaalloc: per-page age bitmap (1 bit per slot) lazily allocated with each
slab page. Public isOld/setOld/setYoung/clearAllAges API. Segment-indexed
for O(1) lookup in the write barrier.
- luagc: write barrier via Detour on lua_table_set_value. Records touched
old tables in a 4096-entry set. Pure bitmap read on every table write.
- luagc: minor/major cycle distinction. First collection, touched overflow,
and every 32nd minor are forced to major. Major clears the age bitmap and
touched set; minor preserves both. cycleFinish repopulates the bitmap by
walking rootgc after a full sweep completes.
Both cycles still run lua_gc_full_collection for now -- the custom minor
mark that actually skips untouched old objects is the next layer.
Previous crash investigations proved Detours on lua_table_set_value are
safe; earlier failures were from simultaneous marked-byte modifications,
not the Detour itself.
File_FindInArchive has two conditional jumps that skip calling
CheckFileExistence. Without NOPing them, preloadFileWithFlags
never reaches our hook and the game skips Bindings.xml loading
for embedded addons. This was previously only done by the
customassets module, so standalone variant DLLs (without
customassets) failed to load addon keybindings.
A/B results: hash_lookup prefetch/cache both regressed (original is
near-optimal at 59 bytes), VM opcode SSE2 patches were break-even.
Only luaS_newlstr hash pre-check shows consistent 40-45% per-call win.
Drop hash_lookup hook, VM jump table patches, and resize hook. Keep
newlstr Detour with per-call A/B rdtsc and OnWorldUpdate periodic dump.
Add GC_WRITE_BARRIER.md documenting the barrier globals at 0xCEEAC0/C4
found in lua_vm_execute (useful for luagc module).
bench: add Detour overhead micro-benchmark. Simulates zhook's
JMP+trampoline mechanism on mmap'd executable pages. Result: 5 cyc/call
overhead -- negligible vs the ~100 cyc/call newlstr savings.
Rootgc sweep chunked by temporarily NULLing a next pointer and calling
the original lua_gc_remove_objects on the truncated sublist. Swept
survivors are detached to a separate list to prevent re-sweeping.
Lists reconnected after final chunk.
Birth-mark via binary patch of luaC_link immediate byte (0x6F7B37)
ensures new objects born during sweep survive.
Profiling shows sweep chunks at 0-9ms each (was 225ms avg, 5s worst).
Remaining bottleneck: atomic mark (80ms) + udata/string sweep (44-92ms).
Incremental GC: atomic mark + batched rootgc sweep. Crashes from
objects born white during sweep -- needs birth-mark barrier fix.
Link hook (luaC_link) calling convention verified but interaction
with sweep_ptr at list head needs solving. Disabled pending fix.
AllocBench: realistic addon-like data structures (combat events,
player damage breakdowns with metatables, aura trackers with
closures). Cross-references shared spell/unit caches. These
complex structures reveal GC as the true stutter source -- simple
flat objects don't trigger meaningful GC pauses.
Removed arena pre-reservation (was fragmenting 32-bit address space).
Profiled 67M Lua allocations to find hot sizes. Lua table hash nodes
are 40 bytes, so hash tables produce allocs at 80, 160, 320, 640,
1280, 2560 (2..64 nodes). Added classes at these exact sizes.
Fixed size class LUT: was 16-byte granularity (size 22 mapped to
class 32 instead of 24). Now direct 1:1 lookup table, 4KB, one
byte load per alloc with zero computation.
Other changes:
- Removed pool_ctx==0 passthrough and seg_val==0 fallback (dead code)
- Removed pool_ctx param from slabFree/slabRealloc
- Added optional PROFILE mode with size histogram dump to log file
- Added AllocBench addon (/allocbench) for in-game benchmarking
Hook 0x686940 (testAABBFrustum): SSE center/extent P-vertex method replaces
x87 sign-bit corner selection. ~160M cycles in profile, 50% rejection rate.
Hook 0x686180 (FrustumCullBoundingBox 8-corner): SSE replacement transforms
all 8 AABB corners through view-proj, horizon buffer scan with 4-wide test.
Fix behind-camera bug in both FrustumCullBoundingBox variants (0x686000,
0x686180): original returned 0 for behind-camera objects, callers interpret
0 as "visible, render it". Now returns 2 (occluded) when all geometry is
behind the camera (Z < 0 in clip space).
Dusk-92