Files
DucielandClaude Opus 5.5 8d74f1274f Loader thread: Sleep(0) instead of Sleep(1) after each file (0.9.3)
The client's single file-loader thread slept 1 ms after every file
(0x4434D2), so zone-change loading screens spent 55-75% of their time
asleep at ~2 ms per texture. loadsleep (default on, live) patches the
immediate of `mov ecx,1` to 0, only when the 10 bytes are the client's
(logged and left alone otherwise); switching it off restores the byte.
test_loadsleep checks the byte logic on a fake buffer. Not yet tested in
game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CmDrvE1QTJT4No3PkwJwv4
2026-11-18 01:23:53 +01:00

381 lines
21 KiB
Markdown

# ducielperf
A client DLL for the WoW 1.12.1 (build 5875) client used by OctoWoW, written in
C++. It is a focused rewrite of parts of
[WeirdUtils](https://codeberg.org/MarcelineVQ/WeirdUtils) (public domain),
which is written in Zig and needs a privately patched Zig compiler to build
correctly (stock Zig 0.16.0 miscompiles the x86 `__fastcall` convention the
client relies on).
## Features
| Module | What it does |
|---|---|
| **World Markers** | Up to 5 animated markers placed in the world (`/wm 1-5`, `/cwm`), synced to the group. Lua: `WorldMarker`, `ClearWorldMarker`, `GetWorldMarker`, `CanSetWorldMarker`. Ships the `WeirdUtils_WorldMarkers` addon (hidden) with key bindings, models and textures, all embedded in the DLL. |
| **Interact** | `InteractNearest(autoloot)` right-clicks the closest NPC/object within 5 yards; `LootAllCorpses([max])` loots nearby corpses one by one. Ships the `WeirdUtils_Interact` addon with key bindings. |
| **Clickthrough** | Smarter mouse targeting: lootable corpses first, then interactable objects, then interactable NPCs, then normal targeting. Ignores portals/rituals summoned by players outside your group. Off in battlegrounds. Keep SuperWoW's own corpse clickthrough (SuperAPI option) off. |
| **Transmog Fix** | Removes the frame drops caused by the server's transmog durability updates (e.g. on death). |
| **Performance** | Faster MPQ decompression (libdeflate instead of the client's zlib), an MPQ file-lookup cache, and Lua string speedups: a table-driven `%a %d %s ...` class test, a literal prefilter that answers "no match" for `string.find` / `string.gsub` / `string.gfind` without running the matcher when a required literal is absent from the text, and a faster string interning check. String building (`table.concat`, `string.rep`, `gsub`, `format`, `lower`/`upper`, `char`) no longer creates garbage: see below. It also claims WeirdUtils' `weirdperformance` mutex, so an old weirdutils.dll never runs its custom Lua GC alongside. |
For compatibility with existing addons (e.g. AutoMarker), the DLL keeps
WeirdUtils' Lua names: the `WeirdUtils` version table, `GetWeirdUtilsVersion()`,
and the `WeirdUtils_WorldMarkers` / `WeirdUtils_Interact` addon names.
### In-game settings (minimap button, `/ducielperf`)
The DLL ships a `DucielPerf` addon (embedded, listed in the addon list) with a
minimap button (left-click: settings, right-click: status, drag to move) and
`/ducielperf` or `/dperf` (`/dperf status`, `/dperf minimap`). Its tabs:
- **Performance / Diagnostics:** every `ducielperf.cfg` setting as a checkbox
or number field, with its explanation as a tooltip.
- **Features:** switch the feature modules; place or clear each world marker
on yourself or your target, clear all; current Interact/loot key bindings,
"Loot all corpses", and a shortcut to the key bindings window.
- **Status:** Lua memory, garbage collections and combat postponing, frame
raising, garbage-free strings, watchdog, frame counters, allocation
tracking (updated every second).
Changes are written to `<WoW>\ducielperf.cfg` (comments and other lines are
kept), so they are shared by every character and client. Most apply at once:
| Setting (`ducielperf.cfg` key) | Default | Applies |
|---|---|---|
| Faster MPQ decompression (`inflate`) | on | at once* |
| MPQ file-lookup cache (`filecache`) | on | at once* |
| Lua string speedups (`luastrings`) | on | at once* |
| Garbage-free string building (`strbuf`) | on | next start |
| Buffered SavedVariables saving (`svwrite`) | on | at once* |
| Fast frame raising (`framelevel`) | on | at once* |
| Faster loading screens (`loadsleep`) | on | at once |
| Postpone GC in combat (`gcdefer`), limit in MB (`gcdefer_mb`) | on, 128 | at once |
| Postpone GC while loading, login and `/reload` (`gcload`) | on | at once |
| Frame counters (`framestats`) | off | at once |
| Freeze watchdog (`watchdog`), threshold (`watchdog_ms`) | off, 1000 | at once |
| Allocation tracking (`alloctrack`) | off | at once |
| World markers (`worldmarkers`), Interact (`interact`), Transmog fix (`transmogfix`), exit error fix (`exitfix`) | on | next start |
| Smart mouse targeting (`clickthrough`) | on | at once* |
| Minimap button (`minimap`, `minimap_angle`) | on | at once |
\* A part switched off in `ducielperf.cfg` is not installed at all at
startup; switching it back on then needs a restart (the window says
"restart needed"). Once installed, it can be switched off and on at any
time: its hooks stay in place and call the client's own code while off.
Lua (used by the addon): `DucielPerf_SettingInfo(i)`, `DucielPerf_GetSetting(key)`,
`DucielPerf_SetSetting(key, value)`; see `src/core/settings.h`.
### Garbage-free string building
Lua 5.0 builds strings in a 512-byte `luaL_Buffer` on the C stack. When it
fills, the original turns each chunk into a Lua string and keeps merging the
pieces into longer strings, all of which become garbage: `table.concat`
allocated about 4-6x its result size. ducielperf replaces the six
`luaL_Buffer` functions (0x6F5300, 0x6F5120, 0x6F51F0, 0x6F5230, 0x6F5250,
0x6F5270): full chunks go to a C-heap buffer, and only the final string is
created. Argument checks, `table.getn`/`setn`, number conversion and error
messages are still the client's own code; only where the bytes wait changes,
so results are identical. It installs only if all six functions are the
client's unmodified code, and all six or none.
Measured on real Lua 5.0.3 with a 512-byte buffer (`tests/test_strbuf.cpp`,
Lua heap allocated per call after a full GC):
| Call | Result | Before | After | Time before / after |
|---|---|---|---|---|
| `table.concat` 500 lines (~50 B) | 25 KB | 110 KB | 25 KB | 28 / 20 µs |
| `string.rep` 100 KB | 100 KB | 276 KB | 100 KB | 255 / 178 µs |
| `string.gsub` 20 KB, 2500 matches | 22 KB | 60 KB | 22 KB | 135 / 127 µs |
| `string.format` with three 5 KB `%s` | 15 KB | 38 KB | 15 KB | 3.5 / 1.3 µs |
| `string.upper` 5 KB | 5 KB | 8.4 KB | 5 KB | 17 / 14.5 µs |
Results under 512 bytes were already garbage-free and are unchanged.
`DucielPerf_GetStringBufferStats()` returns results built, results over 512
bytes, their total bytes, and C-heap buffers currently live. Switch it off
with `strbuf=0` in `ducielperf.cfg`.
### Faster SavedVariables saving (logout, /reload)
The client writes each saved value with its own `WriteFile` call (OsWriteFile
0x42A5D0, called from the per-variable writer 0x7043F0), so an 8 MB
SavedVariables file (aux-addon, pfExtend) takes hundreds of thousands of
system calls: in a raid-night log, a logout spent about 6 s of an 8.5 s frame
in `NtWriteFile`. ducielperf collects the writes made while one variable is
written into a 256 KB buffer and writes it in large chunks, flushing before
the variable is finished and before the file is closed: same bytes, same
order (on this machine: 8 MB in 16-byte writes ~1 s, in 256 KB chunks ~3 ms).
Only the SavedVariables writer on the main thread is affected; other files
(combat log, chat log, config) are written as before. Switch it off with
`svwrite=0` in `ducielperf.cfg`.
### Faster frame raising (opening panels, popups, timers)
Showing a "toplevel" frame (character/spellbook panels via `ShowUIPanel`,
`StaticPopup` dialogs such as a group invite, the breath/fatigue
`MirrorTimer`, many addon windows) raises it above the other frames of its
strata. Before that, the client closes the gaps between the frame levels in
use (0x764EB0), and it does so by walking its list of every frame ever
created once per empty level and once per level it moves. With a large addon
set (tens of thousands of frames) one raise took 150 to 975 ms: 19 of the
watchdog's freeze reports in one day of play were in this loop.
ducielperf replaces it with one walk of the frame list: it marks the levels in
use, replays the client's algorithm on the level numbers alone, then moves
each frame once with the client's own level setter. Levels, the strata's
level count and the order of frames inside each level are the same as the
client's (checked by `test_framelevel` against a port of the client's code);
on 30,000 frames and 150 levels, 0.5 ms instead of 235 ms. If a shown frame is
not in the frame list, or a strata has more than 65,536 levels, the client's
own code runs. `DucielPerf_GetFrameLevelStats()` returns calls, fallbacks,
frames moved, total ms, worst ms and the length of the frame list. Switch it
off with `framelevel=0` in `ducielperf.cfg`.
### Faster loading screens (zone changes, instances)
The client loads textures and models on a single loader thread that calls
`Sleep(1)` after every file it reads (0x4434D2), even when more files are
waiting, while the main thread polls for them. A zone-change loading screen
loads 300-1900 files, so it spent 55-75% of its 2-5 s asleep at about 2 ms
per texture, with disk reads and decompression under 0.4 s (analysis in
`docs/loading-screen-analysis.md`). ducielperf patches that one byte so the
loader calls `Sleep(0)`: it yields to a ready thread and otherwise goes on to
the next file. The patch is written only if the code is the client's
(`mov ecx,1; call 0x429D50`); otherwise it is left alone and the bytes found
are logged. Switching it off restores the original byte. Switch: `loadsleep`
(settings window or `ducielperf.cfg`, applies at once).
### Allocation tracking (diagnostics, off by default)
Charges every Lua allocation to the addon whose code is running (game API
calls are charged to the addon that called them), to find addons that create
a lot of garbage. The `arg` table of a vararg function (`function f(...)`) is
charged to that function, not to its caller (the client builds it before the
callee's call record exists, so a wrapper would otherwise pay for it). While off, nothing is hooked; while on, it costs about
1 ns per allocation (benchmark) up to a few tens of ns with cold caches.
| Lua | Purpose |
|---|---|
| `DucielPerf_AllocTracking([on])` | Query or switch tracking (returns 1 when on). Switching on resets the counters. |
| `GetAddOnAllocations("name")` | Cumulative bytes, count for one addon folder. |
| `DucielPerf_GetAllocations([bytesT [, countT]])` | Fills `bytesT[name]` / `countT[name]` for every bucket; reuse the same tables to avoid garbage. |
| `DucielPerf_ResetAllocations()` | Zero all counters. |
Buckets are addon folder names plus `FrameXML`, `GlueXML`, `SV:<Addon>`
(loading that addon's SavedVariables files, account and character),
`(engine)` (no Lua code running, e.g. event arguments) and `(unknown)` (Lua code whose source is
not a file path, e.g. inline XML handlers; the first such names are written
to the log). These names don't overlap ClassicAPI's `GetAddOnMemoryUsage`,
which reports memory held, not allocated.
### Frame counters and freeze watchdog (diagnostics, off by default)
**Frame counters** measure how much of each frame the main thread spends in
MPQ file reads, decompression, file lookups, model and texture loading, and
network packet handling (other threads are ignored: only main-thread work
freezes a frame). Times are inclusive (a read that decompresses counts in
both). A counted call costs about 55 ns while on; the extra hooks
(processLoadedModelData, LoadTextureFromPath, TextureLoadCallback,
NetClient::ProcessMessage) only exist while counting is on. File-read and
model counters need World Markers or Interact active, decompression and file
lookups need the performance module.
**Freeze watchdog**: a background thread notices when a frame runs long. From
half the threshold on, it samples the main thread every 20 ms (suspend, read
registers, stack and the running Lua function into preallocated buffers,
resume: no locks or allocation while suspended). When the frame ends above
the threshold it appends a report to `Logs\ducielperf_freezes.log`: frame
time, that frame's counters, the most sampled code addresses (WoW.exe
addresses as virtual addresses, others as module+offset) and the Lua
functions that were running (source:line, and the C function they called).
The line is read from the VM's live position (the copy in the call record is
only refreshed now and then, which gave wrong lines or `:-1` before 0.8.0).
System DLL addresses are named from their export tables (e.g.
`ntdll.dll+0x7858C (NtDelayExecution+0xC)`), and a summary line groups the
main thread's samples: waiting/sleeping, file I/O, virtual memory, Lua VM
(WoW.exe), Lua string/table library, rest of WoW.exe, DXVK (`d3d9.dll`),
other DLLs (per module). During a long frame the threads the game and DXVK
started (loader, shader compiler) are also sampled every 100 ms (instruction
pointer only); the report lists the busy ones with the same summary and
their top addresses, which shows the work behind a loading-screen wait.
Minimized clients, and clients that are not the active window (they throttle
themselves to ~4 FPS), are skipped.
| Lua | Purpose |
|---|---|
| `DucielPerf_FrameStats([on])` | Query or switch counting (switching on resets). |
| `DucielPerf_GetFrameStats([lastT [, worstT [, totalT]]])` | Fills `frameMs` and `<name>Ms`, `<name>Calls`, `<name>Bytes` for `fileRead`, `decompress`, `fileFind`, `modelOpen`, `modelProcess`, `textureLoad`, `textureDone`, `packet`, `gc` (bytes = freed): last frame, slowest frame and totals (plus `frames`) since reset. Returns the frame count. Reuse the tables. |
| `DucielPerf_ResetFrameStats()` | Zero the counters. |
| `DucielPerf_Watchdog([on [, thresholdMs]])` | Query or switch the watchdog; returns on, threshold, reports written. Default threshold 1000 ms (minimum 100). |
To have them on from startup, create `<WoW>\ducielperf.cfg`:
```
framestats=1
watchdog=1
watchdog_ms=1000
```
### Garbage collector statistics (always on)
The client runs the stock Lua 5.0 collector: full, stop-the-world
collections whenever Lua memory passes its threshold. The collector entry
(0x6F7340) is hooked permanently; it runs a few times per hour, so this costs
nothing in between. Calls made while the client has GC disabled (flag at
`L+0x60`) are not counted. Times use QueryPerformanceCounter.
| Lua | Purpose |
|---|---|
| `DucielPerf_GetGCStats([t])` | Returns cycles, totalMs. Fills `t` with `cycles`, `totalMs`, `lastMs`, `maxMs`, `lastFreed`, `totalFreed` (bytes), `allocatedSinceLastCall` (memory growth + bytes freed since the previous call; 0 on the first call and after `/reload`), `memory` and `threshold` (bytes, current). |
GC time also appears per frame as the `gc` frame counter and in watchdog
reports. `allocatedSinceLastCall` keeps one baseline: use it from one caller.
#### GC deferral in combat (on by default)
A full collection pauses the game for about 1-2 ms per MB of Lua memory
(95-145 ms with ~60 MB in a raid; 23 of 56 boss-fight hitches in one raid
night). While the player is in combat (UNIT_FLAG_IN_COMBAT, read each frame),
collections are skipped and the threshold is raised by 2 MB each time, until
Lua memory has grown by `gcdefer_mb` (default 128 MB) since the first skipped
collection; then one collection runs and deferral continues from there. Combat
counts as ongoing until the flag has been off for 5 s (Feign Death, Vanish or a
release/rez drop it for a moment); then the next allocation collects. Costs: more Lua memory during long
fights (about 180 KB/s of garbage in a raid: ~55 MB for a 5-minute fight) and
one pause right after combat. Explicit `collectgarbage()` calls in combat are
deferred too.
| Setting / Lua | Purpose |
|---|---|
| `gcdefer=0` in `ducielperf.cfg` | Off at startup (default on). |
| `gcdefer_mb=N` | Most memory growth allowed per fight, 8-512 MB (default 128). |
| `DucielPerf_GCDefer([on])` | Query or switch at runtime; switching off ends a deferral in progress. |
`DucielPerf_GetGCStats(t)` adds `deferOn`, `deferring`, `deferred` (skipped
collections), `deferredFights`, `deferCapHits` and `deferPeakExtra` (largest
growth while deferring, bytes).
#### GC deferral while loading (on by default)
A login or `/reload` is one 9-11 s frame in which the client runs about 40
full collections (640-800 ms, ~160 MB freed) while FrameXML, addons and
SavedVariables load. From the moment the new in-game Lua state gets its
functions (Player_LoadScriptFunctions) until the first frame drawn in the
world, collections are skipped and the threshold is raised by 8 MB each time,
until Lua memory has grown by 256 MB since the first skip; then one collection
runs and deferral continues from there. When loading ends, the next allocation
collects once. A 60 s safety timeout ends the deferral if the world is never
entered. Explicit `collectgarbage()` calls while loading are deferred too.
`ducielperf.log` gets one line per load: `gcload: N collections skipped, +X
MB, final collection Y ms`.
| Setting | Purpose |
|---|---|
| `gcload=0` in `ducielperf.cfg` | Off (default on); also in the settings window, live. |
`DucielPerf_GetGCStats(t)` adds `loadOn`, `loading`, `loadDeferred` (skipped
collections, all loads), `loadCapHits`, `loadTimeouts` and `loadPeakExtra`
(largest growth while loading, bytes).
### Fixes
- Exit error #124 "Memory Invalid Block" (SGroupPtr): the client's exit-time
cleanup at 0x6BD0C0 is skipped.
## Install
1. Copy `build/ducielperf.dll` to `<WoW>\dll\`.
2. In `<WoW>\dlls.txt`, add `dll\ducielperf.dll` and remove the DLLs it replaces:
`dll\weirdutils.dll`, `dll\worldmarkers.dll`, `transmogfix.dll` and `dll\Interact.dll`.
3. Start the game. `Logs\ducielperf.log` lists the active modules.
If an old WeirdUtils DLL is still loaded, it skips any module ducielperf
already provides (WeirdUtils' per-module mutexes are honoured).
## Build
Requires Git Bash (or any bash) and Zig, used only as a C/C++ compiler (clang).
Put Zig at `tools/zig/zig.exe` (git-ignored) or on `PATH`.
```
./build.sh # release: build/ducielperf.dll
./build.sh debug # also logs per-event details to Logs\ducielperf.log
```
## Tests
```
tests/run_tests.sh 2000000 # fuzz the Lua pattern prefilter
tests/run_tests.sh 0 patterns.txt subjects.txt # real-world corpus
```
`test_watchdog` (32-bit) renders fake frames, one of them long, and checks the
watchdog report (duration, counters, sampled function, Lua source:line on a
fake Lua state) and that a frame under the threshold is not reported.
`test_svwrite` checks the SavedVariables write buffer: 40 random saves (about
1.4 million writes of 0 bytes to 160 KB) must produce byte-identical files
with far fewer OS writes, a close in the middle of a variable must flush, and
writes to other files are never buffered.
`test_config` checks `ducielperf.cfg` writing: comments and other keys kept,
duplicate lines collapsed, new keys appended, `watchdog_ms` never mistaken
for `watchdog`. `test_addon` runs the embedded settings addon in real Lua
5.0 on a mock of the WoW frame API (`tests/addon`) and drives it: tabs,
checkboxes, number fields, minimap button drag and hide, slash commands.
`test_framelevel` compacts 3,000 random fake frame worlds (frame list, level
counts, per-level lists of shown frames, frames at or above the level count)
both with a literal port of the client's 0x764EB0 and with the one-pass
version: every frame's level, every level count and the order inside every
level list must match. It also checks the fallback and times both.
`test_strbuf` (32-bit) proves the garbage-free `luaL_Buffer` on the real Lua
5.0.3 library: its six buffer functions are detoured (MinHook) to the same core
the DLL uses, and about 26,000 generated cases per run (`tests/strbuf_cases.lua`:
`table.concat` with separators, empty tables, `i`/`j` ranges incl. `j=0`,
`setn`/`n`, numbers incl. inf/nan, embedded `\0`, MB-sized results and invalid
entries; `string.rep`; `gsub` with string/table/function replacements,
callbacks that error mid-build and callbacks that build strings themselves;
`format` incl. `%q`; `lower`/`upper`; `char`) must give byte-identical results
or error messages with the original and with the replacement, each in its own
Lua state. It also checks that abandoned builds are released and reports
allocation and time for both.
`bench_alloctrack` (32-bit) checks allocation attribution on fake Lua 5.0
structures with the client's exact layout, and times allocations through a
MinHook detour with tracking on and off.
`test_prefilter` checks the prefilter against the real Lua 5.0.3 string
library (`tests/lua50`, unmodified upstream sources): whenever Lua finds a
match with `string.find`, `string.gsub` or `string.gfind`, the prefilter must
not have rejected it. Corpus files are one entry per line (e.g. patterns
extracted from addons, chat or combat-log lines).
## Layout
| Path | What |
|---|---|
| `src/dllmain.cpp` | Entry point: installs modules and core hooks |
| `src/core/` | Game memory and Lua API (`game.h`, `lua.h`), hooks/log/mutexes (`core.cpp`), embedded file serving (`files.cpp`, `embedded.cpp`), game lifecycle hooks (`scripts.cpp`) |
| `src/modules/` | One file per feature |
| `assets/` | Embedded addon files, models and textures |
| `third_party/minhook/` | [MinHook](https://github.com/TsudaKageyu/minhook) 1.3.4 (BSD 2-clause) |
| `third_party/libdeflate/` | [libdeflate](https://github.com/ebiggers/libdeflate) 1.26, decompression sources only (MIT) |
| `tests/` | Host-side tests; `tests/lua50` is Lua 5.0.3 (MIT), used only by the tests |
Calling conventions: Lua C functions are `int __fastcall f(lua_State L)` (the
VM calls them with `L` in ECX only). Game `__thiscall` functions are called and
detoured as `__fastcall` with an unused EDX argument.
## Roadmap
- Phase 3 (maybe): Lua allocator and incremental GC, including the `/reload`
freeze fix.
- World Markers extensions.
## License
Public domain ([Unlicense](LICENSE)), like WeirdUtils. Third-party code keeps
its own license: MinHook (BSD 2-clause), libdeflate (MIT), Lua 5.0.3 (MIT,
tests only).