Commit Graph

26 Commits

Author SHA1 Message Date
MarcelineVQ 4b5fcd6fa6 perf: glyph cache A/B testing, fast hash, frustumCull rcpss+cvtss2si
- glyph cache: add A/B toggle so BASELINE/CUSTOM periods alternate
  between original function and shadow cache. Swap Murmur2 hash for
  fast golden-ratio integer mix (3 insns vs multi-step).
- frustumCullBBox: fastRecip (vrcpss+NR) and cvtss2si replace vdivss
  and @round bloat. 2.0x speedup (was 1.6x).
- Fix all Zig operator precedence bugs in silicon_sse: & and | bind
  looser than == in Zig, so (flags & 0x8 == 0) was always false.
  Affected frustumCullBBox and processLinkedListCollision.
2026-03-23 21:15:15 -07:00
MarcelineVQ 7819d6d914 perf: findInterpIdx dedup, processLinkedListCollision SSE, permanent t44
- bone_sse: deduplicate findInterpIdx calls in bone loop — rotation's
  search result reused for scale/translation when tracks share temporal
  structure (canReuseInterp guard). Est. ~23% bone loop cycle reduction.
- silicon: SSE replacement for processLinkedListCollision (0x6ABC40,
  1.57% CPU). V4 AABB overlap test replaces 6 x87 FCOMP/FNSTSW.
  Benched at 3.2x speedup (378→115 cyc/call, 8 nodes).
- transform44: remove A/B toggle, always use SSE path (teardown guard
  kept). A/B infrastructure remains for other hooks.
- bench: add processLinkedListCollision benchmark with fake linked list
  test fixture and stubbed addGeometryToBuffer.
2026-03-23 20:27:26 -07:00
MarcelineVQ 8736e0b3da bone_sse: clean A/B testing, remove diagnostic comparison code
Remove the double-call REF/SSE bone output comparison diagnostic.
Clean detour: REF baseline, SSE custom, simple toggle.
2026-03-16 10:18:35 -07:00
MarcelineVQ 008db88403 bone_sse: replace matMul/ftol/vec3SqMag with pure Zig, fix visual artifacts
Root cause of billboard visual artifacts: callVec3SqMag used inline asm
to call game's x87 vec3SqMag (0x4549F0) with fstps to capture ST0.
With SSE2 codegen, the x87/SSE state interaction caused corrupted float
values in billboard bone matrices (mat[0][2] wildly wrong).

Replaced with pure Zig: x*x + y*y + z*z — no x87, no inline asm.

Also replaced:
- matMul (0x74A7C0): pure Zig f64 scalar matmul, no alignment needs
- callFtol (0x40A2B0): f64 intermediate + @intFromFloat (cvttsd2si)

Architecture: bone_sse.zig is REF code compiled with SSE2, called as
cdecl from thiscall wrapper in transform44.zig (cross-object to prevent
LLVM inlining AND ESP alignment into thiscall frame).

A/B: other hooks gated behind AB_OTHER_HOOKS=false for isolated testing.
2026-03-16 10:10:02 -07:00
MarcelineVQ 3a031803ef bone_sse: pure Zig SSE/FMA reimplementation, zero game function calls
Replace bone_sse.zig with a complete pure Zig implementation compiled
with SSE4.1 + FMA + AVX. All 18 game function calls replaced:

- findInterpIdx (0x713D50): temporal-coherence keyframe search
- interpAnimKF (0x713EA0): CompQuat lerp for rotation keyframes
- extractByte (0x71AE90): byte keyframe extraction
- getInterpolatedFloat (0x71AF20): float track with direct blend read
- callFtol (0x40A2B0): @intFromFloat replaces x87 __ftol
- callVec3SqMag (0x4549F0): inline FMA dot product
- callGetIndexOffset/callSetShortValue (0x71AFF0/0x71B010): direct ri16
- matMul (0x74A7C0): V4 FMA matmul (broadcast + 3 @mulAdd per row)
- buildRotFn (0x74B6B5): inline quat→matrix
- rotateQuat (0x7BDDB0): quat→matrix then FMA matmul
- scaleMat (0x7BDCA0): inline scale from vec3 ptr
- applyTrans (0x7BDC40): inline FMA dot product translation

Only 2 game calls remain:
- 0x409AEF: one-time atexit init (boneKeyframeLoop)
- 0x7B5F60: IsParticleBufferEmpty (reads game particle state)

Child recursion calls transformMatrix4x4_SSE directly instead of
going through the hook at 0x714260.

Detour cleaned up: REF is baseline, SSE activates via ab_use_custom
toggle. Diagnostic/bisect/FPU-comparison scaffolding removed.
build.zig: bone_sse gets dedicated target with sse4_1+fma+avx features.
2026-03-15 18:16:07 -07:00
MarcelineVQ 36ce1a05ce bone_sse_ref: fix M2 black screen — 5 bugs found via asm comparison
Assembly-level comparison of compiled REF against original 0x714260 revealed:

1. Billboard cross product sign error (types 0x10/0x20): computed +cross
   instead of -cross for components 0/1, corrupting billboard bone matrices
2. colorAnimLoop wrong count field: read model_hdr+0x6C instead of +0x64
3. colorAnimLoop wrong gate offset: checked anim_data+0x04 instead of +0x0C
4. Timestamp delta guard inverted: REF guarded on cur_ts!=0 and always
   wrote to this+0x4C; original guards on this+0x4C!=0 first and never
   seeds the field (something else initializes it)
5. Section 5 emitter_ctx cached instead of re-read after matMul call

Also: build REF with x87-only target (subtract SSE/SSE2 features) to
match original's FLD/FMUL/FSTP codegen, and use callVec3SqMag for all
magnitude computations instead of inline SSE math.

Remaining known issues (not yet fixed):
- texAnimLoop alpha track missing crossfade blend
- colorAnimLoop missing crossfade blend
- Missing word animation section (model_hdr+0x6C/0x70)
- Bisect infrastructure and diagnostic code still present (test scaffolding)
2026-03-15 15:04:05 -07:00
MarcelineVQ 89e49cedf2 bone_sse_ref: replace reimplemented game funcs with actual calls, fix runtime constants
- Replace all reimplemented game functions with actual game calls:
  vec3_sqmag (0x4549F0), __ftol (0x40A2B0), getIndexOffset (0x71AFF0),
  setShortValue (0x71B010) — matching assembly exactly
- Fix 3 wrong hardcoded constants that differ at runtime from Ghidra static values:
  SHORT_TO_FLOAT: 0x38000000→0x38000100 (1/32767 not 1/32768)
  BILLBOARD_EPSILON: 0x3727c5ac→0x34800000
  HERMITE_5: 5.0→6.0
  All now read from game memory at runtime
- Fix timestamp delta guard (this+0x4C): was guarding on stored value,
  assembly guards on anim_ctx pointer — prevents first-frame initialization
- Change REF calling convention to thiscall matching original
- Add comprehensive memory comparison diagnostic (original vs REF)
- Disable interpKfDetour hook (was pure passthrough)
2026-03-15 13:20:05 -07:00
MarcelineVQ da7155a404 bone_sse: fix ribbon emitter offsets (#18-19), teardown guard
Bug #18: Ribbon emitter track offsets completely wrong. Position was
entry+0x24, actual tracks from assembly (0x716402-0x716AA9):
  Track 1 (Vec3): gate=+0x1C, AnimData=+0x10, output=+0x00
  Track 2 (Vec3): gate=+0x38, AnimData=+0x2C, output=+0x30
  Track 3 (float): gate=+0x70, AnimData=+0x64, output=+0x80
  Track 4 (Vec3): gate=+0x54, AnimData=+0x48, output=+0x50

Bug #19: Track 4 output was +0xA0 (should be +0x50), and tracks 2/4
were float (should be Vec3). Wrong output offsets corrupted stack data,
causing bone_rt pointer to contain float bit patterns.

Teardown: hook World_HandleLogoutCleanup (0x491180) instead of
CleanupWorldAndEntities (0x66FC40). Fires at START of logout sequence
before any Lua callbacks trigger model processing on freed data.
Prologue: PUSH ESI/EDI, epilogue: POP EDI/ESI, JMP (tail call).

Also: added bone_sse_reference.zig as separate compilation unit for
A/B testing the proven-working version independently.
2026-03-15 02:22:18 -07:00
MarcelineVQ cb5fb46012 bone_sse: fix 3 more bugs from assembly verification (#15-17)
Bug #15: Short-value interpolation read AD+0x0C (nTimestamps count) as
pointer to short array. Should be AD+0x18 (ofsValues). Caused segfault
reading from address ~0x134 (a count value treated as pointer).

Bug #16: boneKeyframeLoop AnimData offsets wrong. Rotation was kf_data+0x10,
should be +0x1C. Scale was kf_data+0x28 with garbled gate, should be +0x38
with gate at +0x44. Translation was correct at +0x00. Entry structure is
3x28-byte AnimBlocks: trans(+0x00), rot(+0x1C), scale(+0x38) = stride 0x54.

Bug #17: particleEmitterLoop (model_hdr+0x124) position AnimData was
entry+0x04, should be entry+0x10 with gate at entry+0x1C. Second track
was entry+0x20, should be entry+0x38/gate +0x44. Third track at
entry+0x60/gate +0x6C was missing entirely.

Also fixed IsParticleBufferEmpty calling convention (bug #14):
was __stdcall with stack param, now __fastcall(ECX=ptr) plain RET.

SSE dispatch enabled for A/B testing.
2026-03-14 23:45:20 -07:00
MarcelineVQ 3c9a95d51e ssemaths: extract UnitXP math polyfill hooks into standalone module
Moved math_sse.zig and all 17 hook declarations + CriticalSection spin
count optimization from transform44 into new ssemaths module. Off by
default (-Dssemaths=true to enable).

transform44 retains its profiling hooks and blit_hub optimization.
ssemaths is a clean standalone module with its own mutex, install/remove
lifecycle, and lateInit for post-UnitXP hook clobbering.

Added performance note to math_sse.zig documenting that hook-based
replacement adds ~5-8 cycles overhead that makes small functions slower,
and that in-place patching is the path to realize the full 2-4x gains
shown in inlined benchmarks.
2026-03-14 22:28:31 -07:00
MarcelineVQ 8307e525d9 math_sse: fix calling conventions from Ghidra disassembly verification
Hooks 1,3 (vecMulMat4, quatMulMat4): thiscall->fastcall. Assembly
confirms EDX is read as a parameter (FMUL [EDX+...]) before any write.
Detour types TC2r->FC3r, AB wrappers abTC2r->abFC3r.

Hook 15 (0x699330): removed entirely. Was misidentified as vectorNormalize
but Ghidra shows it's a vec3 componentwise >= comparison returning u32.
Silicon-only function (not UnitXP), stub already in silicon.zig. Our
normalize implementation would have silently corrupted comparison results.

Also fixed comment accuracy: hook 8 RET 0x4 (not plain RET), hook 14
__thiscall (not __fastcall). Bitmask selection replaces MATH_TEST_HOOK
single-select with proper bit flags. 17 hooks total, mask 0x77FFE.
2026-03-14 18:44:22 -07:00
MarcelineVQ c060f0d469 bone_sse: verify remaining particle sections from assembly
Section 12c (model_hdr+0x134): visibility byte animation pattern verified
from assembly at 0x7176C2-0x717774. Byte array indexing, crossfade output
at +0xCC (not +0xBC). Position track at entry+0x24 with 12-byte keyframes.

Section 12e (model_hdr+0x13C): all 10 tracks verified. Tracks 1-6 use
scalar float interpolation (findInterpIdx + 4-byte keyframes). Tracks 7-10
use getInterpolatedFloat (0x71AF20). Track offsets, gate checks, and output
positions all confirmed from assembly.

SSE dispatch remains disabled pending final testing.
2026-03-14 17:40:50 -07:00
MarcelineVQ 2b9b5bc043 bone_sse: assembly-verified reimplementation of transformMatrix4x4
13 bugs fixed by comparing against full assembly dump (5317 instructions):
- Emitter check: this+0x188 -> this+0x1D8
- Animation time: added FILD*time_scale pattern for both primary (+0xB0)
  and secondary (+0xDC) slots
- Conditional multiply: bone_local *= *(bone_rt+0xF0) was missing
- Billboard post-processing: 4 switch cases (types 8/16/32/64) implemented
- Color animation loop bound: model_hdr+0x64 -> +0x6C
- Bone keyframe data stride: 0x24 -> 0x54
- Ribbon emitter output stride: 0x15C -> 0x170
- Particle data/output strides: 0x1FC/0x17C -> 0x1F8/0x16C
- Child SceneObject offsets: attach_idx +0x184->+0x1D4, next +0x190->+0x1E4
- Root bone parent: identity -> this+0xFC

New files:
- BONE_SSE_PROGRESS.md: section-by-section verification status
- t44_full_asm.txt: complete function assembly (ground truth)
- t44_helpers_asm.txt: all 12 helper function assemblies
- math_sse.zig: 18 x87->SSE polyfill stubs (VanillaFixes integration)

Also: OnWorldUpdate hook for true per-frame counting, DUMP_FRAMES=450.
SSE dispatch currently disabled while particle sections are being verified.
2026-03-14 17:34:37 -07:00
MarcelineVQ 692a02ed2c Move file cache to standalone module, add timer fix, fix refcount crashes
File cache (filecache module):
- Moved from transform44 sub-module to standalone src/filecache/
- 2-way set-associative cache (32768 sets x 2 ways) with FNV-1a hash + finalizer
- Fixed negative cache hit crash: zero output params before returning 0
  (FindFileInArchive reuses filename slot for out_outer_archive)
- Fixed path 2 crash: set out_outer_archive on all cache hit paths
- Fixed stale block_entry crash: cache block index instead of raw pointer,
  recompute from archive+0x290 on each hit
- Fixed archive-freed crash: use game's FindAndIncrementResourceReference
  (0x650780) instead of manual +0x38 increment -- validates archive is alive
- Periodic stats dump with projected time savings (hit=~1000cy vs miss=~30000cy)

Timer fix (transform44 sub-module, ported from VanillaFixes):
- TSC calibration via QPC reference over 500ms
- Enables TSC mode if game was using GetTickCount fallback
- NtSetTimerResolution for 0.5ms OS timer granularity
- SetProcessInformation to disable Windows 11 power throttling
- Always-on (no A/B toggle -- no measurable impact on Wine/Linux)
2026-03-14 14:57:58 -07:00
MarcelineVQ 0833d23e28 WIP: bone_sse SSE reimplementation of transformMatrix4x4 + OnWorldUpdate frame counter
bone_sse.zig: Full standalone SSE reimplementation of the 17703-byte bone
transform engine (0x714260). All helper functions reimplemented inline
(findInterpolationIndices, interpolateAnimationKeyframes, scaleMatrix3x3,
ApplyTranslation, rotateByQuaternion). Currently disabled (A/B dispatch
commented out) due to NULL ofsValues crash under investigation.

transform44.zig: Add OnWorldUpdate (0x482EA0) hook for true per-frame
counting. Previous frame counter used executeSceneRenderPass which fires
multiple times per frame (shadows, reflections, spell effects), causing
A/B periods to be as short as 0.5s during combat instead of ~30s.

SCENEOBJECT_OFFSETS.md: Complete assembly-verified field offset map
(51 offsets) extracted from [EBX+N] patterns in transformMatrix4x4.
Corrects bone_runtime_base from +0x80 to +0x090.
2026-03-14 00:22:36 -07:00
MarcelineVQ 55735d7b9e Add MPQ archive file cache: skip redundant chain walks on repeat file opens
Hook File_FindInArchive (0x6549a0) with a direct-mapped filename-verified
cache. First open does the full MPQ chain walk (~60K cycles), subsequent
opens hit the cache (~300 cycles). 80% hit rate in gameplay testing.

Cache design: 16384 entries, FNV-1a hash for slot index, raw filename
comparison (128 bytes) for collision safety. Stores outer_archive,
inner_archive, and block_entry per file. Negative cache for not-found files.
Refcount at +0x38 incremented on all output archives to match original
File_FindInArchive behavior (verified via Ghidra: FindAndIncrementResourceReference
at 0x650780, DecrementResourceReference at 0x6507e0).

Also: default build changed to ReleaseFast (works around Zig fastcall inreg
bug in Debug mode), logging gate changed to != ReleaseSmall, file cache
integrated as transform44 sub-module with A/B comparison timing in readout.
2026-03-13 23:05:38 -07:00
MarcelineVQ d9701c528d Add glyph shadow cache: direct-mapped O(1) bypass for game's 4-bucket hash table
GetOrCreateCharacterGlyph (0x5ca2d0) is the #2 CPU hotspot at 3.65%.
The game's glyph cache uses only 4 hash buckets for ~95 ASCII chars,
causing ~24-entry chain walks with pointer chasing on every lookup.
Text measurement (99.8% of calls) re-walks these chains per character,
thousands of times per frame during UI updates.

Shadow cache: 4096-entry direct-mapped array with Murmur2 hash,
keyed on (FontObject*, charCode, param2). Cache hit returns the
cached float width via FPU ST(0) inline asm, skipping the chain
walk entirely. Gated behind ab_use_custom for A/B benchmarking.
2026-03-13 11:55:17 -07:00
MarcelineVQ fc7f6d637b Add SSE multiplyMatrix4x4 hook (0x7bc6a0) with A/B benchmark — 4.3x speedup
Standalone SSE 4x4 matrix multiply replacing 542 bytes of x87 FPU.
Refactored rotateMatrixByAxisAngle to call the new function (with temp
buffer to avoid aliasing). All 5 SSE replacements now confirmed winners:
clip (4x), triplane (10x), rotmat (4.3x), raytri (1.3x), matmul (4.3x).
2026-03-12 18:44:02 -07:00
MarcelineVQ 53fb100368 Fix bone count: model container is at this+0x30 not this+0x2C
Ghidra decompiler swapped the two fields. Assembly verification shows:
  +0x2C = animation_context_ptr (sync check at +0x10)
  +0x30 = model_container_ptr (+0x130 = M2 model header)
Bone count chain: *(*(*(this+0x30) + 0x130) + 0x34)
2026-03-12 11:45:35 -07:00
MarcelineVQ f46e5d07f4 Fix bone count read: add missing +0x130 indirection to M2 header 2026-03-12 11:29:42 -07:00
MarcelineVQ 83a55150b9 Fix overflow panic: all profiling counters to u64 with saturating ops 2026-03-12 11:19:48 -07:00
MarcelineVQ 1f4f2b03c4 Increase profiling dump interval to 600 frames (~10s) 2026-03-12 11:10:15 -07:00
MarcelineVQ 67ffaa7315 Add frame time percentages to render pipeline profiling
Track frame-to-frame wall time via rdtsc delta at executeSceneRenderPass.
Stats dump now shows each function's cycles as % of total frame time,
plus rough ms estimate at 3GHz. Helps identify which functions dominate.
2026-03-12 11:06:58 -07:00
MarcelineVQ cfc252f161 Add render pipeline profiling: 5 hooks across render/movement path
Hooks executeSceneRenderPass (0x708900), renderFrame (0x707680),
transformMatrix4x4 (0x714260), RenderTextureQuads (0x76FB00), and
CMovement::ProcessUnitMovementUpdate (0x616620).

Unified stats dump every 180 render passes shows per-frame call counts,
cycle costs, bone counts, recursion depth, and quad item counts.
2026-03-12 11:03:23 -07:00
MarcelineVQ 572635dd9f Add transform44 profiling hook and inner function analysis
Phase 1 profiling: hooks transformMatrix4x4 (0x714260) to measure call
frequency, early-exit rate, cycle cost, bone counts, and recursion depth.
Dumps stats every 500 calls.

Decompiled and analyzed all 11 inner functions. Key findings:
- findInterpolationIndices already has good temporal coherence (linear scan)
- Matrix math (scale, translate) uses x87 FPU — SSE candidates
- Game already has SSE matrix multiply used by rotateMatrixByQuaternion
- interpolateAnimationKeyframes does 4-component lerp — textbook SSE
2026-03-12 10:49:27 -07:00
MarcelineVQ 9965a50aab Add transform44 module stub for M2 bone transform optimization
Wires transform44 into build system and main.zig module table.
Module skeleton with mutex/logger, no hooks yet — analysis in progress.
2026-03-12 10:36:36 -07:00