Commit Graph
33 Commits
Author SHA1 Message Date
MarcelineVQ 112246e687 perf: make all verified optimizations permanent, remove A/B toggles
- Glyph cache: unconditional (was A/B toggled)
- RenderParticleSprites SSE: unconditional (was A/B toggled)
- transform44 bone SSE: already permanent (teardown guard only)
- processLinkedListCollision: already permanent (JMP patch)
- frustumCullBoundingBox: already permanent (JMP patch)
- silicon functions (ftol, normalize, matmul, etc.): already permanent

Also adds decompilation of UpdateEntityAndChunksPositions — analyzed
but not optimizable (game function calls dominate, math is ~50 cycles
of the ~375 cycle total).
2026-03-24 00:44:18 -07:00
MarcelineVQ 3d872fdb1e spritequad: recreation + analysis (disabled, no improvement)
Faithful recreation of RenderSpriteQuads (0x5A0F50) with hoisted
invariant division and inlined DisplayMode_CalculateOffset. No
measurable improvement — cost is dominated by getAdapterInfo (7
sub-calls to D3D device per invocation × 3203 calls/frame) and
DrawPrimitive/DrawIndexedPrimitive virtual dispatch.

Added decompilation and assembly dumps for future reference.
2026-03-24 00:36:37 -07:00
MarcelineVQ 1046c0a4e3 particle: WIP setupParticleRendering recreation (disabled)
Faithful recreation attempt of SetupParticleRendering (0x7B3D20).
All game function CCs verified from assembly. Vertex buffer setup
works (8 verts produced), but D3D draw submission doesn't produce
visible output. Disabled pending investigation of GfxDeviceMethod
param struct layout. The function remains as timing-only pass-through.

Fixes found during work:
- max_particle_sprites global: 0xCF58F4 → 0xCF5B60
- billboard_matrix global: 0xCF5898 → 0xCF5888
- index_buffer_6/12: 0xCF58D0/D4 → 0xCF5BAC/0xCF5AF4
- BuildIndexBuffer takes renders_count not field_28
- Identity matrices must be mutable (game writes to them)
2026-03-24 00:25:12 -07:00
MarcelineVQ 379fcb62eb particle: contiguous vertex writes, skip normals, stride logging
- Detected interleaved 24-byte vertex layout: xyz(12)+color(4)+uv(8).
  Fast path writes 6 sequential u32s instead of scattered stores.
- Normal stride=0 (shared global) — write once in writeback, not 4x.
- Added stride_info export + logging for vertex layout analysis.
- A/B: ~30% peak reduction, baseline also improved due to less overhead.
2026-03-23 22:41:16 -07:00
MarcelineVQ 0d994db4b3 particle: VBState all paths, cached setupRender, @mulAdd — ~25% speedup
- VBState caching on all 5 vertex paths (was only 2D and sin/cos).
  Eliminates pointer re-reads: load once, emit 4 vertices, writeback.
- Cache setupRender() result per-frame via static + resetParticleCache()
  called from worldUpdateDetour. Saves ~7K function calls/frame.
- @mulAdd throughout for FMA codegen on vertex position and texcoord.
- A/B verified: BASELINE ~521ms → CUSTOM ~387ms (~25% reduction).
2026-03-23 22:29:12 -07:00
MarcelineVQ 574f96f661 particle: faithful RenderParticleSprites recreation, A/B verified
Full recreation of RenderParticleSprites (0x7B2A50, 2688 bytes) in
particle_sse.zig. All 5 code paths: 2D billboard, 3D billboard,
2D+rotation (sin/cos), 3D+rotation (axis-angle matrix), tail particles.

Key fixes during verification:
- colorCtx address: removed Ghidra's spurious -0x12 offset
- calcColor arg2: pass raw u32 from emitter+0x1A8, not truncated
- Texture coord lookups: +8 offset to match assembly's eax increment
  between position and texcoord reads in the vertex loop

Verified in-game: particles render identically in CUSTOM vs BASELINE.
Next: optimize with SSE (inline calcColor, V4 vertex math).
2026-03-23 22:12:17 -07:00
MarcelineVQ d205693dbb particle: Ghidra decompilations, particle_sse.zig scaffold, bench
- Ghidra C decompilation of RenderParticleSprites (422 lines) and 5
  helper functions (calculateColorValues, matVec3Transform, etc.)
- particle_sse.zig with calcColorValues_SSE (10.7x bench but cache-miss
  bound in-game — needs inlining into full function replacement)
- Bench harness for calcColorValues with correctness check
- build.zig: particle_sse as separate ReleaseFast compilation unit
- colorDetour reverted to pass-through (SSE has no in-game effect due
  to L1 cache misses on scattered ColorCtx structs)
2026-03-23 21:56:57 -07:00
MarcelineVQ 4b5fcd6fa6 perf: glyph cache A/B testing, fast hash, frustumCull rcpss+cvtss2si
- glyph cache: add A/B toggle so BASELINE/CUSTOM periods alternate
  between original function and shadow cache. Swap Murmur2 hash for
  fast golden-ratio integer mix (3 insns vs multi-step).
- frustumCullBBox: fastRecip (vrcpss+NR) and cvtss2si replace vdivss
  and @round bloat. 2.0x speedup (was 1.6x).
- Fix all Zig operator precedence bugs in silicon_sse: & and | bind
  looser than == in Zig, so (flags & 0x8 == 0) was always false.
  Affected frustumCullBBox and processLinkedListCollision.
2026-03-23 21:15:15 -07:00
MarcelineVQ 7819d6d914 perf: findInterpIdx dedup, processLinkedListCollision SSE, permanent t44
- bone_sse: deduplicate findInterpIdx calls in bone loop — rotation's
  search result reused for scale/translation when tracks share temporal
  structure (canReuseInterp guard). Est. ~23% bone loop cycle reduction.
- silicon: SSE replacement for processLinkedListCollision (0x6ABC40,
  1.57% CPU). V4 AABB overlap test replaces 6 x87 FCOMP/FNSTSW.
  Benched at 3.2x speedup (378→115 cyc/call, 8 nodes).
- transform44: remove A/B toggle, always use SSE path (teardown guard
  kept). A/B infrastructure remains for other hooks.
- bench: add processLinkedListCollision benchmark with fake linked list
  test fixture and stubbed addGeometryToBuffer.
2026-03-23 20:27:26 -07:00
MarcelineVQ 8736e0b3da bone_sse: clean A/B testing, remove diagnostic comparison code
Remove the double-call REF/SSE bone output comparison diagnostic.
Clean detour: REF baseline, SSE custom, simple toggle.
2026-03-16 10:18:35 -07:00
MarcelineVQ 008db88403 bone_sse: replace matMul/ftol/vec3SqMag with pure Zig, fix visual artifacts
Root cause of billboard visual artifacts: callVec3SqMag used inline asm
to call game's x87 vec3SqMag (0x4549F0) with fstps to capture ST0.
With SSE2 codegen, the x87/SSE state interaction caused corrupted float
values in billboard bone matrices (mat[0][2] wildly wrong).

Replaced with pure Zig: x*x + y*y + z*z — no x87, no inline asm.

Also replaced:
- matMul (0x74A7C0): pure Zig f64 scalar matmul, no alignment needs
- callFtol (0x40A2B0): f64 intermediate + @intFromFloat (cvttsd2si)

Architecture: bone_sse.zig is REF code compiled with SSE2, called as
cdecl from thiscall wrapper in transform44.zig (cross-object to prevent
LLVM inlining AND ESP alignment into thiscall frame).

A/B: other hooks gated behind AB_OTHER_HOOKS=false for isolated testing.
2026-03-16 10:10:02 -07:00
MarcelineVQ 3a031803ef bone_sse: pure Zig SSE/FMA reimplementation, zero game function calls
Replace bone_sse.zig with a complete pure Zig implementation compiled
with SSE4.1 + FMA + AVX. All 18 game function calls replaced:

- findInterpIdx (0x713D50): temporal-coherence keyframe search
- interpAnimKF (0x713EA0): CompQuat lerp for rotation keyframes
- extractByte (0x71AE90): byte keyframe extraction
- getInterpolatedFloat (0x71AF20): float track with direct blend read
- callFtol (0x40A2B0): @intFromFloat replaces x87 __ftol
- callVec3SqMag (0x4549F0): inline FMA dot product
- callGetIndexOffset/callSetShortValue (0x71AFF0/0x71B010): direct ri16
- matMul (0x74A7C0): V4 FMA matmul (broadcast + 3 @mulAdd per row)
- buildRotFn (0x74B6B5): inline quat→matrix
- rotateQuat (0x7BDDB0): quat→matrix then FMA matmul
- scaleMat (0x7BDCA0): inline scale from vec3 ptr
- applyTrans (0x7BDC40): inline FMA dot product translation

Only 2 game calls remain:
- 0x409AEF: one-time atexit init (boneKeyframeLoop)
- 0x7B5F60: IsParticleBufferEmpty (reads game particle state)

Child recursion calls transformMatrix4x4_SSE directly instead of
going through the hook at 0x714260.

Detour cleaned up: REF is baseline, SSE activates via ab_use_custom
toggle. Diagnostic/bisect/FPU-comparison scaffolding removed.
build.zig: bone_sse gets dedicated target with sse4_1+fma+avx features.
2026-03-15 18:16:07 -07:00
MarcelineVQ 36ce1a05ce bone_sse_ref: fix M2 black screen — 5 bugs found via asm comparison
Assembly-level comparison of compiled REF against original 0x714260 revealed:

1. Billboard cross product sign error (types 0x10/0x20): computed +cross
   instead of -cross for components 0/1, corrupting billboard bone matrices
2. colorAnimLoop wrong count field: read model_hdr+0x6C instead of +0x64
3. colorAnimLoop wrong gate offset: checked anim_data+0x04 instead of +0x0C
4. Timestamp delta guard inverted: REF guarded on cur_ts!=0 and always
   wrote to this+0x4C; original guards on this+0x4C!=0 first and never
   seeds the field (something else initializes it)
5. Section 5 emitter_ctx cached instead of re-read after matMul call

Also: build REF with x87-only target (subtract SSE/SSE2 features) to
match original's FLD/FMUL/FSTP codegen, and use callVec3SqMag for all
magnitude computations instead of inline SSE math.

Remaining known issues (not yet fixed):
- texAnimLoop alpha track missing crossfade blend
- colorAnimLoop missing crossfade blend
- Missing word animation section (model_hdr+0x6C/0x70)
- Bisect infrastructure and diagnostic code still present (test scaffolding)
2026-03-15 15:04:05 -07:00
MarcelineVQ 89e49cedf2 bone_sse_ref: replace reimplemented game funcs with actual calls, fix runtime constants
- Replace all reimplemented game functions with actual game calls:
  vec3_sqmag (0x4549F0), __ftol (0x40A2B0), getIndexOffset (0x71AFF0),
  setShortValue (0x71B010) — matching assembly exactly
- Fix 3 wrong hardcoded constants that differ at runtime from Ghidra static values:
  SHORT_TO_FLOAT: 0x38000000→0x38000100 (1/32767 not 1/32768)
  BILLBOARD_EPSILON: 0x3727c5ac→0x34800000
  HERMITE_5: 5.0→6.0
  All now read from game memory at runtime
- Fix timestamp delta guard (this+0x4C): was guarding on stored value,
  assembly guards on anim_ctx pointer — prevents first-frame initialization
- Change REF calling convention to thiscall matching original
- Add comprehensive memory comparison diagnostic (original vs REF)
- Disable interpKfDetour hook (was pure passthrough)
2026-03-15 13:20:05 -07:00
MarcelineVQ da7155a404 bone_sse: fix ribbon emitter offsets (#18-19), teardown guard
Bug #18: Ribbon emitter track offsets completely wrong. Position was
entry+0x24, actual tracks from assembly (0x716402-0x716AA9):
  Track 1 (Vec3): gate=+0x1C, AnimData=+0x10, output=+0x00
  Track 2 (Vec3): gate=+0x38, AnimData=+0x2C, output=+0x30
  Track 3 (float): gate=+0x70, AnimData=+0x64, output=+0x80
  Track 4 (Vec3): gate=+0x54, AnimData=+0x48, output=+0x50

Bug #19: Track 4 output was +0xA0 (should be +0x50), and tracks 2/4
were float (should be Vec3). Wrong output offsets corrupted stack data,
causing bone_rt pointer to contain float bit patterns.

Teardown: hook World_HandleLogoutCleanup (0x491180) instead of
CleanupWorldAndEntities (0x66FC40). Fires at START of logout sequence
before any Lua callbacks trigger model processing on freed data.
Prologue: PUSH ESI/EDI, epilogue: POP EDI/ESI, JMP (tail call).

Also: added bone_sse_reference.zig as separate compilation unit for
A/B testing the proven-working version independently.
2026-03-15 02:22:18 -07:00
MarcelineVQ cb5fb46012 bone_sse: fix 3 more bugs from assembly verification (#15-17)
Bug #15: Short-value interpolation read AD+0x0C (nTimestamps count) as
pointer to short array. Should be AD+0x18 (ofsValues). Caused segfault
reading from address ~0x134 (a count value treated as pointer).

Bug #16: boneKeyframeLoop AnimData offsets wrong. Rotation was kf_data+0x10,
should be +0x1C. Scale was kf_data+0x28 with garbled gate, should be +0x38
with gate at +0x44. Translation was correct at +0x00. Entry structure is
3x28-byte AnimBlocks: trans(+0x00), rot(+0x1C), scale(+0x38) = stride 0x54.

Bug #17: particleEmitterLoop (model_hdr+0x124) position AnimData was
entry+0x04, should be entry+0x10 with gate at entry+0x1C. Second track
was entry+0x20, should be entry+0x38/gate +0x44. Third track at
entry+0x60/gate +0x6C was missing entirely.

Also fixed IsParticleBufferEmpty calling convention (bug #14):
was __stdcall with stack param, now __fastcall(ECX=ptr) plain RET.

SSE dispatch enabled for A/B testing.
2026-03-14 23:45:20 -07:00
MarcelineVQ 3c9a95d51e ssemaths: extract UnitXP math polyfill hooks into standalone module
Moved math_sse.zig and all 17 hook declarations + CriticalSection spin
count optimization from transform44 into new ssemaths module. Off by
default (-Dssemaths=true to enable).

transform44 retains its profiling hooks and blit_hub optimization.
ssemaths is a clean standalone module with its own mutex, install/remove
lifecycle, and lateInit for post-UnitXP hook clobbering.

Added performance note to math_sse.zig documenting that hook-based
replacement adds ~5-8 cycles overhead that makes small functions slower,
and that in-place patching is the path to realize the full 2-4x gains
shown in inlined benchmarks.
2026-03-14 22:28:31 -07:00
MarcelineVQ 8307e525d9 math_sse: fix calling conventions from Ghidra disassembly verification
Hooks 1,3 (vecMulMat4, quatMulMat4): thiscall->fastcall. Assembly
confirms EDX is read as a parameter (FMUL [EDX+...]) before any write.
Detour types TC2r->FC3r, AB wrappers abTC2r->abFC3r.

Hook 15 (0x699330): removed entirely. Was misidentified as vectorNormalize
but Ghidra shows it's a vec3 componentwise >= comparison returning u32.
Silicon-only function (not UnitXP), stub already in silicon.zig. Our
normalize implementation would have silently corrupted comparison results.

Also fixed comment accuracy: hook 8 RET 0x4 (not plain RET), hook 14
__thiscall (not __fastcall). Bitmask selection replaces MATH_TEST_HOOK
single-select with proper bit flags. 17 hooks total, mask 0x77FFE.
2026-03-14 18:44:22 -07:00
MarcelineVQ c060f0d469 bone_sse: verify remaining particle sections from assembly
Section 12c (model_hdr+0x134): visibility byte animation pattern verified
from assembly at 0x7176C2-0x717774. Byte array indexing, crossfade output
at +0xCC (not +0xBC). Position track at entry+0x24 with 12-byte keyframes.

Section 12e (model_hdr+0x13C): all 10 tracks verified. Tracks 1-6 use
scalar float interpolation (findInterpIdx + 4-byte keyframes). Tracks 7-10
use getInterpolatedFloat (0x71AF20). Track offsets, gate checks, and output
positions all confirmed from assembly.

SSE dispatch remains disabled pending final testing.
2026-03-14 17:40:50 -07:00
MarcelineVQ 2b9b5bc043 bone_sse: assembly-verified reimplementation of transformMatrix4x4
13 bugs fixed by comparing against full assembly dump (5317 instructions):
- Emitter check: this+0x188 -> this+0x1D8
- Animation time: added FILD*time_scale pattern for both primary (+0xB0)
  and secondary (+0xDC) slots
- Conditional multiply: bone_local *= *(bone_rt+0xF0) was missing
- Billboard post-processing: 4 switch cases (types 8/16/32/64) implemented
- Color animation loop bound: model_hdr+0x64 -> +0x6C
- Bone keyframe data stride: 0x24 -> 0x54
- Ribbon emitter output stride: 0x15C -> 0x170
- Particle data/output strides: 0x1FC/0x17C -> 0x1F8/0x16C
- Child SceneObject offsets: attach_idx +0x184->+0x1D4, next +0x190->+0x1E4
- Root bone parent: identity -> this+0xFC

New files:
- BONE_SSE_PROGRESS.md: section-by-section verification status
- t44_full_asm.txt: complete function assembly (ground truth)
- t44_helpers_asm.txt: all 12 helper function assemblies
- math_sse.zig: 18 x87->SSE polyfill stubs (VanillaFixes integration)

Also: OnWorldUpdate hook for true per-frame counting, DUMP_FRAMES=450.
SSE dispatch currently disabled while particle sections are being verified.
2026-03-14 17:34:37 -07:00
MarcelineVQ 692a02ed2c Move file cache to standalone module, add timer fix, fix refcount crashes
File cache (filecache module):
- Moved from transform44 sub-module to standalone src/filecache/
- 2-way set-associative cache (32768 sets x 2 ways) with FNV-1a hash + finalizer
- Fixed negative cache hit crash: zero output params before returning 0
  (FindFileInArchive reuses filename slot for out_outer_archive)
- Fixed path 2 crash: set out_outer_archive on all cache hit paths
- Fixed stale block_entry crash: cache block index instead of raw pointer,
  recompute from archive+0x290 on each hit
- Fixed archive-freed crash: use game's FindAndIncrementResourceReference
  (0x650780) instead of manual +0x38 increment -- validates archive is alive
- Periodic stats dump with projected time savings (hit=~1000cy vs miss=~30000cy)

Timer fix (transform44 sub-module, ported from VanillaFixes):
- TSC calibration via QPC reference over 500ms
- Enables TSC mode if game was using GetTickCount fallback
- NtSetTimerResolution for 0.5ms OS timer granularity
- SetProcessInformation to disable Windows 11 power throttling
- Always-on (no A/B toggle -- no measurable impact on Wine/Linux)
2026-03-14 14:57:58 -07:00
MarcelineVQ 0833d23e28 WIP: bone_sse SSE reimplementation of transformMatrix4x4 + OnWorldUpdate frame counter
bone_sse.zig: Full standalone SSE reimplementation of the 17703-byte bone
transform engine (0x714260). All helper functions reimplemented inline
(findInterpolationIndices, interpolateAnimationKeyframes, scaleMatrix3x3,
ApplyTranslation, rotateByQuaternion). Currently disabled (A/B dispatch
commented out) due to NULL ofsValues crash under investigation.

transform44.zig: Add OnWorldUpdate (0x482EA0) hook for true per-frame
counting. Previous frame counter used executeSceneRenderPass which fires
multiple times per frame (shadows, reflections, spell effects), causing
A/B periods to be as short as 0.5s during combat instead of ~30s.

SCENEOBJECT_OFFSETS.md: Complete assembly-verified field offset map
(51 offsets) extracted from [EBX+N] patterns in transformMatrix4x4.
Corrects bone_runtime_base from +0x80 to +0x090.
2026-03-14 00:22:36 -07:00
MarcelineVQ 55735d7b9e Add MPQ archive file cache: skip redundant chain walks on repeat file opens
Hook File_FindInArchive (0x6549a0) with a direct-mapped filename-verified
cache. First open does the full MPQ chain walk (~60K cycles), subsequent
opens hit the cache (~300 cycles). 80% hit rate in gameplay testing.

Cache design: 16384 entries, FNV-1a hash for slot index, raw filename
comparison (128 bytes) for collision safety. Stores outer_archive,
inner_archive, and block_entry per file. Negative cache for not-found files.
Refcount at +0x38 incremented on all output archives to match original
File_FindInArchive behavior (verified via Ghidra: FindAndIncrementResourceReference
at 0x650780, DecrementResourceReference at 0x6507e0).

Also: default build changed to ReleaseFast (works around Zig fastcall inreg
bug in Debug mode), logging gate changed to != ReleaseSmall, file cache
integrated as transform44 sub-module with A/B comparison timing in readout.
2026-03-13 23:05:38 -07:00
MarcelineVQ d9701c528d Add glyph shadow cache: direct-mapped O(1) bypass for game's 4-bucket hash table
GetOrCreateCharacterGlyph (0x5ca2d0) is the #2 CPU hotspot at 3.65%.
The game's glyph cache uses only 4 hash buckets for ~95 ASCII chars,
causing ~24-entry chain walks with pointer chasing on every lookup.
Text measurement (99.8% of calls) re-walks these chains per character,
thousands of times per frame during UI updates.

Shadow cache: 4096-entry direct-mapped array with Murmur2 hash,
keyed on (FontObject*, charCode, param2). Cache hit returns the
cached float width via FPU ST(0) inline asm, skipping the chain
walk entirely. Gated behind ab_use_custom for A/B benchmarking.
2026-03-13 11:55:17 -07:00
MarcelineVQ fc7f6d637b Add SSE multiplyMatrix4x4 hook (0x7bc6a0) with A/B benchmark — 4.3x speedup
Standalone SSE 4x4 matrix multiply replacing 542 bytes of x87 FPU.
Refactored rotateMatrixByAxisAngle to call the new function (with temp
buffer to avoid aliasing). All 5 SSE replacements now confirmed winners:
clip (4x), triplane (10x), rotmat (4.3x), raytri (1.3x), matmul (4.3x).
2026-03-12 18:44:02 -07:00
MarcelineVQ 53fb100368 Fix bone count: model container is at this+0x30 not this+0x2C
Ghidra decompiler swapped the two fields. Assembly verification shows:
  +0x2C = animation_context_ptr (sync check at +0x10)
  +0x30 = model_container_ptr (+0x130 = M2 model header)
Bone count chain: *(*(*(this+0x30) + 0x130) + 0x34)
2026-03-12 11:45:35 -07:00
MarcelineVQ f46e5d07f4 Fix bone count read: add missing +0x130 indirection to M2 header 2026-03-12 11:29:42 -07:00
MarcelineVQ 83a55150b9 Fix overflow panic: all profiling counters to u64 with saturating ops 2026-03-12 11:19:48 -07:00
MarcelineVQ 1f4f2b03c4 Increase profiling dump interval to 600 frames (~10s) 2026-03-12 11:10:15 -07:00
MarcelineVQ 67ffaa7315 Add frame time percentages to render pipeline profiling
Track frame-to-frame wall time via rdtsc delta at executeSceneRenderPass.
Stats dump now shows each function's cycles as % of total frame time,
plus rough ms estimate at 3GHz. Helps identify which functions dominate.
2026-03-12 11:06:58 -07:00
MarcelineVQ cfc252f161 Add render pipeline profiling: 5 hooks across render/movement path
Hooks executeSceneRenderPass (0x708900), renderFrame (0x707680),
transformMatrix4x4 (0x714260), RenderTextureQuads (0x76FB00), and
CMovement::ProcessUnitMovementUpdate (0x616620).

Unified stats dump every 180 render passes shows per-frame call counts,
cycle costs, bone counts, recursion depth, and quad item counts.
2026-03-12 11:03:23 -07:00
MarcelineVQ 572635dd9f Add transform44 profiling hook and inner function analysis
Phase 1 profiling: hooks transformMatrix4x4 (0x714260) to measure call
frequency, early-exit rate, cycle cost, bone counts, and recursion depth.
Dumps stats every 500 calls.

Decompiled and analyzed all 11 inner functions. Key findings:
- findInterpolationIndices already has good temporal coherence (linear scan)
- Matrix math (scale, translate) uses x87 FPU — SSE candidates
- Game already has SSE matrix multiply used by rotateMatrixByQuaternion
- interpolateAnimationKeyframes does 4-component lerp — textbook SSE
2026-03-12 10:49:27 -07:00
MarcelineVQ 9965a50aab Add transform44 module stub for M2 bone transform optimization
Wires transform44 into build system and main.zig module table.
Module skeleton with mutex/logger, no hooks yet — analysis in progress.
2026-03-12 10:36:36 -07:00