grom/benilla-octo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Comments in the small crates: one line of fact each, no history, no record numbers (#93)

Browse Source 
Part of #41: the comments of the thirteen small crates (`benilla-adt`, `-blp`, `-buildstamp`, `-bytes`, `-dbc`, `-m2`, `-mpq`, `-srp`, `-visual`, `-wdt`, `-wmo`, `-worldview` and `benilla`) rewritten to the rule in `docs/METHOD.md`: what the code does and the reference fact behind it, in a line. 2,176 comment lines become 1,029.

- The citations that were only a private record number go first, mechanically, each edit shown to be a pure removal.
- The rewrite corrects the comments that stated wrong facts: the M2 header's selection-ring claim, the texture-transform and MCNK impassable-flag notes, a DBC reader note that said short records zero-pad (they do not, #17), and about twenty more found on the way. The SRP handshake's redraw and redial are marked as a deviation, with the measured reason.
- Verified: every file strips to identical code, and `cargo fmt --check` is clean. A second reader compared every file old against new and restored the facts the rewrite had dropped.

String literals are untouched; they are #92.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sam Sung
2026-09-24 15:51:18 +07:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 1ee056fc53
commit bf8a15de02
30 changed files with 842 additions and 1993 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/benilla-adt/src/combined_alpha.rs
+11 -14
View File
@@ -1,13 +1,11 @@
//! MCAL → one 64×64 RGBA alpha map (R/G/B = the opacity of texture layers 1/2/3; A = 255). Ported
//! from wow-adt's `CombinedAlphaMap`: layers are ingested in MCLY order (skipping the opaque base),
//! each decoded into the next channel; encodings are 4-bit packed (×17 — `n / 15`, what the
//! reference's RGBA4444 texel reads as; wow-adt's ×16 topped out at 240), 8-bit raw, or Blizzard RLE (token bit 7 = fill, else copy; low 7 bits = count). `fix_alpha`
//! reconstructs a 64×64 plane from 63×63 source by duplicating the previous pixel at the last row/col.
//! MCAL to one 64×64 RGBA alpha map, ported from wow-adt's `CombinedAlphaMap`: R, G, B are the
//! opacity of texture layers 1, 2, 3 and A is 255. Layers go in MCLY order after the opaque base. A
//! layer is 4-bit packed (×17: the reference packs it into an RGBA4444 texel, read as `n / 15`),
//! 8-bit raw, or RLE (token bit 7 fill, else copy; the low 7 bits are the count).
use crate::McnkChunk;
/// Assembles the per-layer MCAL alpha maps into one 64×64 RGBA buffer (`y`-major, then `x`, then
/// channel R,G,B,A).
/// The combined 64×64 RGBA alpha map, `y`-major.
pub struct CombinedAlphaMap {
map: Vec<u8>, // 64*64*4, [y][x][rgba]
x: usize,
@@ -20,12 +18,13 @@ pub struct CombinedAlphaMap {
const W: usize = 64;
impl CombinedAlphaMap {
/// Construct and ingest `chunk`'s alpha layers. `has_big_alpha`/`fix_alpha` select the uncompressed
/// encoding width and the 63→64 edge fix (vanilla: `false`, `true`).
/// Decode `chunk`'s alpha layers. `has_big_alpha` makes uncompressed layers 8-bit, and
/// `fix_alpha` fills the last row and column from their neighbours (a 63×63 source); vanilla
/// is `false`, `true`.
pub fn new(chunk: &McnkChunk, has_big_alpha: bool, fix_alpha: bool) -> Self {
let mut map = vec![0u8; W * W * 4];
for px in map.as_chunks_mut::<4>().0 {
px[3] = 255; // A unused, set opaque for tool visibility (matches wow-adt)
px[3] = 255; // A is unused; opaque, so a dump shows the channels
}
let mut s = Self {
map,
@@ -187,10 +186,8 @@ mod tests {
}
}
/// The reference packs a 4-bit layer weight into an RGBA4444 texel (the 64×64 and 32×32
/// packers `0x6b03d0`/`0x6b08d0`), and a 4-bit unorm channel reads as `n / 15` — so a fully
/// painted nibble is full coverage. `n × 16` stopped at 240: every "fully painted" road or rock
/// let 6% of the layer beneath show through.
/// The reference packs a 4-bit weight into an RGBA4444 texel (64×64 `0x6b03d0`, 32×32
/// `0x6b08d0`), read as `n / 15`, so nibble 15 is full coverage.
#[test]
fn four_bit_alpha_reads_as_n_over_15() {
let mut m = blank();
crates/benilla-adt/src/lib.rs
+64 -134
View File
@@ -1,27 +1,11 @@
//! An ADT terrain-tile reader for **WoW 1.12.1 (build 5875)** — in-repo, replacing `wow-adt`
//! (decision 0021).
//!
//! Vanilla ADTs are monolithic, chunked (`IFF`-style: a 4-char magic — stored **reversed** on disk —
//! plus a `u32` size, then payload). The root file holds `MTEX` (textures), `MMDX`/`MMID` (M2 paths),
//! `MWMO`/`MWID` (WMO paths), `MDDF`/`MODF` (placements), and 256 `MCNK` terrain chunks. Each `MCNK` is
//! a 128-byte header followed by sub-chunks (`MCVT` heights, `MCNR` normals, `MCLY` layers, `MCAL`
//! alpha, `MCSH` shadow, `MCLQ` liquid), which we locate by **scanning** the chunk payload for their
//! magics (robust, and identical to the header-offset data for vanilla).
//!
//! The shape mirrors what the renderer's terrain/liquid mesher consumes; quirks the consumer
//! compensates for (the MCNR `[x,z,y]` byte order, the split predominant-texture/noEffectDoodad
//! grids) are preserved deliberately. Proven against `wow-adt` over real ADTs during the decision-0021
//! migration (oracle test in git history); the `benilla-formats` terrain golden tests pin the meshing
//! end-to-end on every run.
//!
//! Byte access goes through `benilla-bytes` (decision 0064): every read is bounds-checked and a
//! truncated chunk/record is a typed [`Error::Truncated`], never a panic.
//! A reader for 1.12.1's monolithic root ADTs: IFF chunks (a reversed magic, a `u32` size, the
//! payload) holding the asset names, the placements and 256 `MCNK` terrain chunks.
use std::io::Cursor;
use benilla_bytes::{chunks, ByteExt};
/// A terrain vertex normal — stored on disk as `x, z, y` signed bytes.
/// A terrain vertex normal: three signed bytes, named `x, z, y` in disk order.
#[derive(Debug, Clone, Copy)]
pub struct VertexNormal {
pub x: i8,
@@ -30,8 +14,7 @@ pub struct VertexNormal {
}
impl VertexNormal {
/// `[x/127, y/127, z/127]` — i.e. on-disk `[b0, b2, b1]` / 127 (kept bit-identical to wow-adt; the
/// consumer cancels the z/y swap).
/// `[x, y, z] / 127`, which is disk bytes `[b0, b2, b1]`; the terrain mesher swaps them back.
pub fn to_normalized(&self) -> [f32; 3] {
[
f32::from(self.x) / 127.0,
@@ -41,19 +24,19 @@ impl VertexNormal {
}
}
/// MCVT — 145 heights (9×9 outer + 8×8 inner, stride-17 interleave).
/// MCVT: 145 heights (9×9 outer + 8×8 inner, stride-17 interleave).
#[derive(Debug, Clone)]
pub struct McvtChunk {
pub heights: Vec<f32>,
}
/// MCNR — one normal per MCVT vertex.
/// MCNR: one normal per MCVT vertex.
#[derive(Debug, Clone)]
pub struct McnrChunk {
pub normals: Vec<VertexNormal>,
}
/// MCLY per-layer flags (we only need the alpha-compression bit).
/// MCLY per-layer flags.
#[derive(Debug, Clone, Copy)]
pub struct MclyFlags {
pub value: u32,
@@ -66,7 +49,7 @@ impl MclyFlags {
}
}
/// MCLY — one texture layer (16 bytes on disk).
/// MCLY: one texture layer (16 bytes on disk).
#[derive(Debug, Clone)]
pub struct MclyLayer {
pub texture_id: u32,
@@ -75,19 +58,19 @@ pub struct MclyLayer {
pub effect_id: u32,
}
/// MCLY — the chunk's texture layers (0–4; layer 0 is the opaque base).
/// MCLY: the chunk's texture layers (0–4; layer 0 is the opaque base).
#[derive(Debug, Clone)]
pub struct MclyChunk {
pub layers: Vec<MclyLayer>,
}
/// MCAL — raw alpha-map bytes (indexed per layer via [`MclyLayer::offset_in_mcal`]).
/// MCAL: raw alpha-map bytes (indexed per layer via [`MclyLayer::offset_in_mcal`]).
#[derive(Debug, Clone)]
pub struct McalChunk {
pub data: Vec<u8>,
}
/// MCSH — a 64×64 1-bit baked shadow map (512 bytes).
/// MCSH: a 64×64 1-bit baked shadow map (512 bytes).
#[derive(Debug, Clone)]
pub struct McshChunk {
pub shadow_map: Vec<u8>,
@@ -104,11 +87,8 @@ impl McshChunk {
}
}
/// One MCLQ liquid vertex (8 bytes: 4 union bytes + an absolute height).
///
/// The leading 4 bytes are a **union whose meaning is the block's liquid type** — water/ocean spend
/// them on a depth byte + flow/foam, magma/slime on a `u16` texture-coordinate pair. Both readers are
/// below; which one applies is the caller's decision, from the cell nibble.
/// One MCLQ liquid vertex, 8 bytes: 4 union bytes and an absolute height. The union follows the
/// cell's liquid type: a depth byte and flow for water and ocean, a UV pair for magma and slime.
#[derive(Debug, Clone, Copy)]
pub struct LiquidVertex {
pub union_data: [u8; 4],
@@ -116,18 +96,14 @@ pub struct LiquidVertex {
}
impl LiquidVertex {
/// The depth byte (union byte 0) — drives the water depth-swatch on the render side. Water and
/// ocean blocks only; on a magma block these bytes are [`Self::texcoords`] instead.
/// The water or ocean depth byte (union byte 0).
pub fn depth_byte(&self) -> u8 {
self.union_data[0]
}
/// The authored `(s, t)` texture-coordinate pair — two little-endian `u16`s over the same 4
/// union bytes. **Magma blocks only**: the reference's lava vert-fill is single-stage and reads
/// its `tc0` straight from here, where water/ocean instead write a UV into the depth-ramp
/// texture (bit-exact `WoW.exe 0x68d890`: `u = (s as i32 as f64 · 3/256) as f32`, same for
/// `t`). The field is authored world-continuous — a chunk's east edge repeats its neighbour's
/// west edge exactly — so lava tiles seamlessly across MCNK borders.
/// A magma block's authored `(s, t)`, two little-endian `u16`s the reference's lava fill takes
/// as its UV (`0x68d890`: `u = (s as i32 as f64 · 3/256) as f32`, same for `t`). The field is
/// world-continuous, so lava tiles seamlessly across MCNK borders.
pub fn texcoords(&self) -> [u16; 2] {
[
u16::from_le_bytes([self.union_data[0], self.union_data[1]]),
@@ -136,50 +112,33 @@ impl LiquidVertex {
}
}
/// One MCLQ liquid **block** — a 9×9 absolute-height grid + an 8×8 cell-flag grid, `0x324` bytes.
///
/// An MCNK carries **one block per set liquid header bit** (bits 2–5), packed back to back and with
/// absent bits consuming nothing (the cursor walk at `0x6af7a3`–`0x6af7cb`). Most liquid chunks set
/// exactly one bit; **28 shipped Azeroth chunks set two** — a river *and* the sea, at a river mouth
/// — and carry two blocks at different heights (measured: `sizeMCLQ` 1616 = 8 + 2·`0x324`, block 0
/// the stream at z≈5.0, block 1 the ocean at z=0). Reading only the first is how the sea goes
/// missing there.
///
/// **No liquid type here on purpose.** The header-bit→type ordering (bit2 river / bit3 ocean /
/// bit4 magma / bit5 slime) is *inferred* — never byte-proven — while the per-cell flag low nibble
/// IS the type, as both consumers read it (`0x6ba970` `and al,0xf`, `0x68d9b0`). So this reader
/// takes only the block *count* from the flags and leaves the type to whoever reads
/// [`Self::tile_flags`]. (The old `LiquidType::from_mcnk_flags` priority guess labelled all 28
/// two-bit chunks "ocean" while handing back the river block's bytes.)
/// One MCLQ liquid block: a 9×9 absolute-height grid and an 8×8 cell-flag grid. An MCNK carries
/// one per set liquid header bit, back to back (`0x6af7a3`–`0x6af7cb`): two at the 28 shipped river
/// mouths. The liquid type is each cell's flag nibble (`0x6ba970`, `0x68d9b0`), not a header bit.
#[derive(Debug, Clone)]
pub struct MclqChunk {
pub min_height: f32,
pub max_height: f32,
pub vertices: Vec<LiquidVertex>,
/// Per-cell flags, row-major 8×8. Low nibble = liquid type (`0xf` = dry/hole); `0x80` shared.
/// `0x40` ("fishable") is carried but deliberately unread: its only compiled reader, the
/// reference's fishable query `0x69b5d0`, is a zero-caller dead island — fishability is the
/// server's verdict (`SPELL_FAILED_NOT_FISHABLE` passthru). Decision 1825.
/// Per-cell flags, row-major 8×8: the low nibble is the liquid type (`0xf` dry), `0x80`
/// shared. `0x40`, fishable, is unread: the reference's only reader (`0x69b5d0`) has no
/// caller, and the server decides fishability.
pub tile_flags: [u8; 64],
}
/// MCNK header flag **bit 1** — the chunk a mover may not enter (the "impassable" band that walls
/// Searing Gorge and the other mountain rims off from their neighbours).
///
/// The *producer* is the MCNK header walk (`0x6af5f0`: `hdr+0x00` bit1 → `chunk+0xc |= 0x40`);
/// the *name* is an inferred label for that bit. Consuming it is decision 1266 / report B129.
/// MCNK header flag bit 1: a chunk a mover may not enter, the bands along mountain rims such as
/// Searing Gorge's. The reference's header walk copies it to its chunk flag `0x40` (`0x6af5f0`).
pub const MCNK_IMPASSABLE: u32 = 0x2;
/// The fields of the 128-byte MCNK header the renderer reads. `pred_tex`/`no_effect_doodad`/
/// `unknown_8bytes` mirror wow-adt's (mis)split of the predominant-texture + noEffectDoodad region,
/// which the consumer reconstructs — kept identical on purpose.
/// The MCNK header fields the renderer reads. The names mislead: `pred_tex` and `no_effect_doodad`
/// are the two halves of the predominant-texture map at `+0x40`, and `unknown_8bytes` is the real
/// noEffectDoodad at `+0x50`; the consumer reassembles them.
#[derive(Debug, Clone)]
pub struct McnkHeader {
pub flags: u32,
pub index_x: u32,
pub index_y: u32,
/// `AreaTable.dbc` id of this chunk (header +0x34) — the zone/subzone the player is standing
/// in; drives zone music/ambience/reverb selection (decision 0070).
/// `AreaTable.dbc` id of the chunk's zone or subzone (`+0x34`).
pub area_id: u32,
pub holes_low_res: u16,
pub pred_tex: [u8; 8],
@@ -189,9 +148,7 @@ pub struct McnkHeader {
}
impl McnkHeader {
/// Is this chunk flagged [`MCNK_IMPASSABLE`]? The flag is a header bit, so its granularity is
/// the whole 33.333 yd chunk; in the shipped data the set chunks form contiguous ribbons along
/// zone rims (Searing Gorge's is 48 chunks of `Azeroth_33_44` alone).
/// Whether the raw header sets [`MCNK_IMPASSABLE`], which covers the whole 33.333 yd chunk.
pub fn impassable(&self) -> bool {
self.flags & MCNK_IMPASSABLE != 0
}
@@ -206,12 +163,11 @@ pub struct McnkChunk {
pub layers: Option<MclyChunk>,
pub alpha: Option<McalChunk>,
pub shadow: Option<McshChunk>,
/// The chunk's MCLQ liquid blocks — **one per set liquid header bit**, in on-disk order (see
/// [`MclqChunk`]). Empty on a dry chunk; two at a river mouth.
/// The MCLQ liquid blocks in disk order: none on a dry chunk, two at a river mouth.
pub liquids: Vec<MclqChunk>,
}
/// MDDF — a placed M2 doodad (36 bytes).
/// MDDF: a placed M2 doodad (36 bytes).
#[derive(Debug, Clone)]
pub struct DoodadPlacement {
pub name_id: u32,
@@ -222,7 +178,7 @@ pub struct DoodadPlacement {
pub flags: u16,
}
/// MODF — a placed WMO (64 bytes).
/// MODF: a placed WMO (64 bytes).
#[derive(Debug, Clone)]
pub struct WmoPlacement {
pub name_id: u32,
@@ -231,8 +187,8 @@ pub struct WmoPlacement {
pub rotation: [f32; 3],
pub flags: u16,
pub doodad_set: u16,
/// `WMOAreaTable.NameSetID` selector — which naming/audio variant of the building this
/// placement is (Northshire Abbey vs Tyr's Hand Abbey are one WMO, different name sets).
/// `WMOAreaTable.NameSetID`: which name and audio variant of the WMO this placement is
/// (Northshire Abbey and Tyr's Hand Abbey are one WMO).
pub name_set: u16,
}
@@ -247,8 +203,7 @@ pub struct RootAdt {
pub mcnk_chunks: Vec<McnkChunk>,
}
/// Either a vanilla monolithic root ADT (all we produce) — kept as an enum to mirror the consumer's
/// `ParsedAdt::Root(..)` match.
/// A parsed ADT; vanilla has only the monolithic root.
#[derive(Debug, Clone)]
pub enum ParsedAdt {
Root(Box<RootAdt>),
@@ -272,8 +227,7 @@ impl std::error::Error for Error {}
type Result<T> = std::result::Result<T, Error>;
/// Split a NUL-separated string blob into a `Vec<u32>` of start offsets paired with strings (used to
/// resolve MMDX/MWMO via MMID/MWID offsets).
/// The NUL-terminated string at `offset`: an MMDX or MWMO name by its MMID or MWID offset.
fn cstring_at(blob: &[u8], offset: usize) -> String {
let end = blob[offset.min(blob.len())..]
.iter()
@@ -283,7 +237,7 @@ fn cstring_at(blob: &[u8], offset: usize) -> String {
String::from_utf8_lossy(&blob[offset.min(blob.len())..end]).into_owned()
}
/// Parse a vanilla monolithic ADT. (`cursor` position is ignored; the file starts at the first chunk.)
/// Parse a vanilla monolithic ADT; the cursor's position is ignored.
pub fn parse_adt(cursor: &mut Cursor<&[u8]>) -> Result<ParsedAdt> {
let b: &[u8] = cursor.get_ref();
@@ -310,7 +264,7 @@ pub fn parse_adt(cursor: &mut Cursor<&[u8]>) -> Result<ParsedAdt> {
}
}
// Resolve model/WMO paths via the offset tables (the order MDDF/MODF `name_id` indexes).
// MDDF/MODF `name_id` indexes the MMID/MWID offset tables.
let models = mmid
.iter()
.map(|&o| cstring_at(&mmdx, o as usize))
@@ -338,7 +292,6 @@ fn split_strings(blob: &[u8]) -> Vec<String> {
.collect()
}
/// `name` tags the truncation error with the chunk this record list came from (MMID/MWID).
fn read_u32_list(data: &[u8], name: &'static str) -> Result<Vec<u32>> {
data.as_chunks::<4>()
.0
@@ -386,18 +339,17 @@ fn read_modf(data: &[u8]) -> Result<Vec<WmoPlacement>> {
r.f32_at(24).ok_or(Error::Truncated("MODF"))?,
r.f32_at(28).ok_or(Error::Truncated("MODF"))?,
],
// 0x20..0x38 = bounding box (6 f32); then:
// 0x20..0x38: the bounding box (6 f32), unread.
flags: r.u16_at(56).ok_or(Error::Truncated("MODF"))?,
doodad_set: r.u16_at(58).ok_or(Error::Truncated("MODF"))?,
name_set: r.u16_at(60).ok_or(Error::Truncated("MODF"))?,
// 0x3E scale u16 — unused.
// 0x3E: a u16, unread.
});
}
Ok(out)
}
/// Parse one MCNK: a 128-byte header, then sub-chunks located by scanning the payload for their
/// (reversed) magics.
/// Parse one MCNK: the 128-byte header, then the sub-chunks at the header's offsets.
fn read_mcnk(data: &[u8]) -> Result<McnkChunk> {
if data.len() < 128 {
return Err(Error::Truncated("MCNK header"));
@@ -441,14 +393,9 @@ fn read_mcnk(data: &[u8]) -> Result<McnkChunk> {
liquids: Vec::new(),
};
// Locate sub-chunks by the MCNK header offsets (the authoritative method — sequential scanning
// breaks when sub-chunks aren't contiguous, e.g. AhnQiraj). MCVT/MCNR offsets live in the
// `multipurpose_field` (0x14/0x18); the rest are explicit header fields. The offset base differs
// across sources (relative to the MCNK magic vs the MCNK data), so we auto-detect per sub-chunk by
// validating the (reversed) magic at both candidate positions. Returns the payload `(start, size)`.
// `ofs` comes straight off a (possibly corrupt) file's header, so both the "minus 8" candidate
// and the 8-byte header read use checked/bounds-checked arithmetic (`checked_sub`/`bytes_at`/
// `u32_at`) — a wrapped or out-of-range offset yields `None`, never a panic.
// Sub-chunks sit at the header's offsets, not in sequence (AhnQiraj's are not contiguous).
// An offset may count from the MCNK magic or from its data, so the reversed magic is checked
// at both; yields the payload's `(start, size)`.
let locate = |ofs: u32, magic: &[u8; 4]| -> Option<(usize, usize)> {
if ofs == 0 {
return None;
@@ -462,8 +409,7 @@ fn read_mcnk(data: &[u8]) -> Result<McnkChunk> {
}
None
};
// `start` always comes from a `locate` hit (bounds-proven above); `len` is attacker-controlled,
// so the end is saturating before clamping to the buffer.
// `locate` proves `start` in bounds; `len` is unvalidated, so the end saturates, then clamps.
let slice = |start: usize, len: usize| &data[start..start.saturating_add(len).min(data.len())];
if let Some((p, n)) = locate(
@@ -515,8 +461,8 @@ fn read_mcnk(data: &[u8]) -> Result<McnkChunk> {
}
chunk.layers = Some(MclyChunk { layers });
}
// MCAL's data length is the header's `size_alpha`, NOT the sub-chunk's declared size (they differ
// on some files, e.g. AhnQiraj — wow-adt reads `size_alpha`).
// MCAL's length is the header's `size_alpha`, not the sub-chunk's own size: they differ on some
// files (AhnQiraj).
if let Some((p, _)) = locate(
h.u32_at(0x24).ok_or(Error::Truncated("MCNK header"))?,
b"LACM",
@@ -538,8 +484,7 @@ fn read_mcnk(data: &[u8]) -> Result<McnkChunk> {
shadow_map[..k].copy_from_slice(&sub[..k]);
chunk.shadow = Some(McshChunk { shadow_map });
}
// MCLQ, like MCAL, takes its length from the MCNK header (`size_liquid`): the MCLQ sub-chunk's
// own declared size is always 0.
// MCLQ's length is the header's `size_liquid` too: its own declared size is always 0.
if let Some((p, _)) = locate(
h.u32_at(0x60).ok_or(Error::Truncated("MCNK header"))?,
b"QLCM",
@@ -553,14 +498,12 @@ fn read_mcnk(data: &[u8]) -> Result<McnkChunk> {
/// Bytes per MCLQ block: `{f32 min, f32 max, 81×8B verts, 64B cell flags, u32 nFlow, 2×40B flow}`.
const MCLQ_BLOCK: usize = 0x324;
/// Liquid header bits (2–5) — their **count** is the number of packed MCLQ blocks. Only the count is
/// read: the bit→type ordering is INFERRED, the cell nibble is VERIFIED (see [`MclqChunk`]).
/// Liquid header bits 2–5: how many are set is how many MCLQ blocks there are.
const MCNK_LIQUID_BITS: u32 = 0x3c;
/// Read the packed MCLQ blocks — one per set liquid header bit, back to back (the cursor walk
/// `0x6af7a3`–`0x6af7cb`). Stops early on a block the payload can't cover, so a truncated tail
/// degrades to the blocks that *are* whole rather than to nothing. A chunk with an MCLQ sub-chunk
/// but no liquid bit set still yields one block, matching the pre-multi-block reader.
/// Read the packed MCLQ blocks, stopping at the first one the payload cannot hold. A chunk with an
/// MCLQ sub-chunk but no liquid bit set still yields one block; the reference creates one per set
/// bit only (`0x6af760`).
fn read_mclq_blocks(sub: &[u8], mcnk_flags: u32) -> Vec<MclqChunk> {
let want = (mcnk_flags & MCNK_LIQUID_BITS).count_ones().max(1) as usize;
let mut out = Vec::with_capacity(want);
@@ -576,9 +519,8 @@ fn read_mclq_blocks(sub: &[u8], mcnk_flags: u32) -> Vec<MclqChunk> {
out
}
/// `None` (not an error) below the minimum size: too-short MCLQ data is treated as "no liquid",
/// matching the pre-0064 reader — a real-data tolerance, not a hardening carve-out. (The minimum is
/// the *read* span, short of the full [`MCLQ_BLOCK`]: the trailing flow vectors are unread.)
/// Too-short MCLQ data is no liquid, not an error. The minimum stops short of [`MCLQ_BLOCK`]: the
/// trailing flow data is unread.
fn read_mclq_block(sub: &[u8]) -> Option<MclqChunk> {
const NEED: usize = 8 + 81 * 8 + 64;
if sub.len() < NEED {
@@ -619,9 +561,7 @@ mod tests {
v
}
/// A 128-byte MCNK header with a single MCVT (145 heights) sub-chunk placed right after it,
/// referenced via the "offset relative to the MCNK magic" convention (multipurpose field @0x14
/// pointing at the MCVT chunk's own magic, 128 bytes in).
/// An MCNK header and one MCVT after it, the `0x14` offset pointing at MCVT's magic, 128 in.
fn mcnk_with_mcvt() -> Vec<u8> {
let mut data = vec![0u8; 128];
data[0x14..0x18].copy_from_slice(&128u32.to_le_bytes());
@@ -677,9 +617,7 @@ mod tests {
#[test]
fn partial_trailing_mddf_record_is_dropped_not_a_panic() {
// A record shorter than the 36-byte MDDF stride: `chunks_exact` drops the short remainder
// (a pre-existing, unchanged leniency — every *complete* 36-byte record is still
// bounds-checked via `ByteExt`, so this can never panic or misread).
// Shorter than the 36-byte stride: `as_chunks` drops the remainder.
let b = chunk(b"FDDM", &[0u8; 10]);
let ParsedAdt::Root(root) = parse(&b).expect("parses");
assert!(root.doodad_placements.is_empty());
@@ -693,9 +631,7 @@ mod tests {
#[test]
fn corrupt_mcnk_subchunk_offset_skips_cleanly_never_panics() {
// ofs = 3: `checked_sub(8)` on the second candidate is `None`, and the first candidate (3)
// doesn't land on a valid magic — must yield `None`, not wrap/panic (the verified
// `ofs.wrapping_sub(8)` trap this replaces).
// ofs = 3: the minus-8 candidate underflows and 3 holds no magic.
let mut data = vec![0u8; 128];
data[0x14..0x18].copy_from_slice(&3u32.to_le_bytes());
let b = chunk(b"KNCM", &data);
@@ -714,20 +650,16 @@ mod tests {
fn short_mclq_is_treated_as_no_liquid_not_an_error() {
let mut data = vec![0u8; 128];
data[0x60..0x64].copy_from_slice(&128u32.to_le_bytes()); // MCLQ offset
data[0x64..0x68].copy_from_slice(&4u32.to_le_bytes()); // size_liquid — far short of NEED
data[0x64..0x68].copy_from_slice(&4u32.to_le_bytes()); // size_liquid, far short of NEED
data.extend(chunk(b"QLCM", &[0u8; 4]));
let b = chunk(b"KNCM", &data);
let ParsedAdt::Root(root) = parse(&b).expect("parses");
assert!(root.mcnk_chunks[0].liquids.is_empty());
}
/// **One MCLQ block per set liquid header bit**, packed back to back — the shape that made the
/// sea vanish at 28 shipped river mouths when only the first was read (see [`MclqChunk`]).
/// Two bits set ⇒ two blocks, each keeping its OWN cell flags, so the consumer can tell the
/// river block from the ocean block by nibble.
/// Each block keeps its own cell flags, so the river and the ocean tell apart by nibble.
#[test]
fn a_two_bit_mcnk_yields_two_mclq_blocks_in_disk_order() {
// One block: min/max, 81×8B verts, 64B cell flags, then the flow tail — 0x324 total.
let block = |height: f32, nibble: u8| {
let mut b = vec![0u8; MCLQ_BLOCK];
b[0..4].copy_from_slice(&height.to_le_bytes());
@@ -738,11 +670,11 @@ mod tests {
b[8 + 81 * 8..8 + 81 * 8 + 64].fill(nibble);
b
};
let mut payload = block(5.0, 4); // bit 2 (river) — first on disk
payload.extend(block(0.0, 1)); // bit 3 (ocean) — second
let mut payload = block(5.0, 4); // the river, first on disk
payload.extend(block(0.0, 1)); // the ocean, second
let mut data = vec![0u8; 128];
data[0x00..0x04].copy_from_slice(&0x0cu32.to_le_bytes()); // liquid bits 2 AND 3
data[0x00..0x04].copy_from_slice(&0x0cu32.to_le_bytes()); // liquid bits 2 and 3
data[0x60..0x64].copy_from_slice(&128u32.to_le_bytes());
data[0x64..0x68].copy_from_slice(&((payload.len() + 8) as u32).to_le_bytes());
data.extend(chunk(b"QLCM", &payload));
@@ -757,9 +689,7 @@ mod tests {
assert_eq!(liquids[1].min_height, 0.0);
}
/// The magma union bytes are an authored `(s, t)` u16 pair, little-endian over the same 4 bytes
/// the water blocks spend on a depth byte — VERIFIED `0x68d890` reads them as `u16 → i32`, so
/// `0xffff` is 65535 and never −1.
/// `0x68d890` reads the magma pair as `u16 → i32`, so `0xffff` is 65535, never −1.
#[test]
fn magma_union_bytes_read_as_two_unsigned_u16_texcoords() {
let v = LiquidVertex {
crates/benilla-blp/src/bin/blpx.rs
+2 -4
View File
@@ -1,7 +1,5 @@
//! One-off inspection helper (session tool, not shipped): `blpx <in.blp> <out.ppm>` writes mip 0
//! as P6 PPM, alpha composited over magenta so cutout regions read at a glance (pipe through
//! `sips -s format png` for a viewable PNG). An `<out>` ending in `.rgba` instead dumps mip 0 as
//! raw RGBA8 bytes (width×height×4, row-major) for scripted analysis of the real alpha channel.
//! `blpx <in.blp> <out.ppm>`: mip 0 as a P6 PPM, alpha composited over magenta so cutouts show; an
//! `<out>` ending in `.rgba` gets raw RGBA8 instead (width×height×4, row-major).
fn main() {
let mut args = std::env::args().skip(1);
crates/benilla-blp/src/lib.rs
+64 -176
View File
@@ -1,35 +1,14 @@
//! A BLP2 texture decoder for **WoW 1.12.1 (build 5875)** — in-repo, replacing `wow-blp` (decision
//! 0021). Decodes to `Rgba8Unorm` (the gamma-byte form the renderer uploads verbatim).
//! A BLP2 texture decoder for WoW 1.12.1, which ships Direct content in three shapes:
//! - Raw1: a 256-entry BGRA palette and 1-byte indices, then a 0, 1, 4 or 8-bit alpha block.
//! - Raw3: uncompressed BGRA8.
//! - DXTC: S3TC blocks, `alpha_type` selecting DXT1 (0), DXT3 (1) or DXT5 (7).
//!
//! 1.12 ships **BLP2 / Direct** content in three shapes (no JPEG in practice):
//! - **Raw1** — a 256-entry **BGRA** palette + 1-byte indices, with a separate 0/1/4/8-bit alpha block.
//! - **Raw3** — uncompressed BGRA8.
//! - **DXTC** — S3TC blocks; `alpha_type` selects DXT1 (0) / DXT3 (1) / DXT5 (7), decoded via the same
//! `texpresso` codec `wow-blp` uses, so the output is byte-identical.
//!
//! **Two entry points, because the GPU wants the blocks and the CPU wants the pixels.**
//! [`decode`] always yields `Rgba8Unorm`; [`decode_native`] hands back the DXTC blocks *verbatim*
//! and only decodes the shapes that have no GPU-native form (Raw1/Raw3). The reference client
//! uploads the stored blocks untouched (`glCompressedTexImage2DARB` in `0x59f270`; a raw
//! passthrough — the device eats DXT), so the native path is both the faithful one and 4x-8x
//! cheaper in VRAM and texture bandwidth. Decoding is for consumers that read texels on the CPU.
//!
//! Two fidelity points that cost the old `wow-blp` two forks, folded in here natively:
//! 1. the palette is **BGRA**, not RGBA (reading it as RGBA swaps R↔B on every palettized Blizzard
//! atlas — the "Westfall clutter is blue" bug);
//! 2. a stale/unknown `alpha_type` byte (e.g. `2` on alpha-less particle atlases) is **not fatal** —
//! alpha presence is governed by `alpha_bits`; `alpha_type` only picks the DXT variant.
//!
//! Proven byte-for-byte against `wow-blp` over the real corpus during the decision-0021 migration
//! (oracle test in git history); the `benilla-formats` texture loaders exercise it on every run.
//!
//! Byte access goes through `benilla-bytes` (decision 0064): every header read is bounds-checked,
//! and `width`/`height` are capped at [`MAX_DIM`] so a corrupt header can't turn into a
//! multi-gigabyte allocation in [`decode_dxt`].
//! [`decode`] yields gamma-encoded `Rgba8Unorm`, never linearized; [`decode_native`] keeps the
//! DXTC blocks, which the reference uploads untouched (`glCompressedTexImage2DARB` in `0x59f270`).
use benilla_bytes::ByteExt;
/// Decode error. Anything outside the 1.12 BLP2 envelope surfaces here rather than guessing.
/// A BLP decode error: anything outside the 1.12 BLP2 envelope.
#[derive(Debug)]
pub enum Error {
NotBlp2,
@@ -39,8 +18,7 @@ pub enum Error {
OutOfBounds {
level: usize,
},
/// `width`/`height` from the header exceed [`MAX_DIM`] — most likely a corrupt or hostile
/// header, since no real 1.12 BLP is anywhere near this large.
/// The header's dimensions exceed [`MAX_DIM`], which no real 1.12 BLP comes near.
DimensionsTooLarge {
width: u32,
height: u32,
@@ -67,13 +45,10 @@ impl std::error::Error for Error {}
type Result<T> = std::result::Result<T, Error>;
/// A decoded BLP: mip-major `Rgba8Unorm`. `mips[0]` is the full image and is always present; deeper
/// levels follow while the file carries them. [`mip_chain_count`](DecodedBlp::mip_chain_count) is the
/// count `wow-blp` reports for the dimensions (what the renderer's authored-mip upload iterates).
/// A decoded BLP, `Rgba8Unorm` per mip; `mips[0]` is always present.
pub struct DecodedBlp {
pub width: u32,
pub height: u32,
/// One RGBA8 buffer per decoded mip level, level 0 first.
pub mips: Vec<MipLevel>,
mip_chain_count: usize,
}
@@ -86,9 +61,8 @@ pub struct MipLevel {
}
impl DecodedBlp {
/// The authored mip-chain length for these dimensions, matching `wow-blp`'s `mipmaps_count()`
/// (`max(log2 w, log2 h)` when the file has mipmaps, else 0). The renderer's mip-chain upload
/// iterates exactly this many levels; `mips` may hold one more (level 0) when this is 0.
/// The authored chain length: `max(log2 w, log2 h)` with mipmaps, else 0 (`mips` still holds
/// level 0).
pub fn mip_chain_count(&self) -> usize {
self.mip_chain_count
}
@@ -97,15 +71,9 @@ impl DecodedBlp {
const HEADER_SIZE: usize = 148; // magic..=mip_sizes[16]
const PALETTE_SIZE: usize = 256 * 4;
/// Sanity cap on header `width`/`height`. Real 1.12 art tops out at 1024×1024; this is pure
/// headroom above any real asset, and its only job is bounding allocation: [`decode_dxt`] sizes
/// both its block-padding buffer and its `w*h*4` RGBA output straight from these fields, so a
/// corrupt header claiming e.g. 65535×65535 would otherwise reserve gigabytes before a single
/// byte of pixel data is checked.
/// A cap on the header's dimensions, which size the buffers (1.12 art stops at 1024×1024).
const MAX_DIM: u32 = 8192;
/// `wow-blp`'s mip count formula (kept bit-identical, float `log2` and all, so the chain length we
/// report matches the oracle exactly).
fn mip_chain_count(width: u32, height: u32, has_mipmaps: bool) -> usize {
if has_mipmaps {
let w = (width as f32).log2() as usize;
@@ -125,13 +93,9 @@ fn level_size(width: u32, height: u32, level: usize) -> (u32, u32) {
}
/// The texel form a [`NativeBlp`] level carries.
///
/// A BLP is one of three shapes on disk (module header); only DXTC has a GPU-native form, so the
/// other two report [`Self::Rgba8Unorm`] and carry decoded pixels. Callers switch on this to pick a
/// `TextureFormat` and to size a level.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum BlpTexels {
/// Decoded pixels — the Raw1 (palettized) and Raw3 (BGRA8) shapes, which have no block form.
/// Decoded pixels, from Raw1 or Raw3, which have no block form.
Rgba8Unorm,
/// DXT1 blocks (8 bytes per 4x4), `alpha_type` 0. 1-bit alpha at most.
Bc1,
@@ -142,15 +106,12 @@ pub enum BlpTexels {
}
impl BlpTexels {
/// Is this a block-compressed form (i.e. can it be uploaded to the GPU verbatim)?
/// Whether this is a block form, uploadable verbatim.
pub fn is_block_compressed(self) -> bool {
!matches!(self, Self::Rgba8Unorm)
}
/// How many bytes a `width x height` level occupies in this form.
///
/// Block forms round **up** to whole 4x4 blocks, which is why a 2x2 or 1x1 tail mip still costs
/// one full block — exactly what the BLP stores for it, and what wgpu expects to receive.
/// The bytes a `width x height` level takes; block forms round up to whole 4x4 blocks.
pub fn level_bytes(self, width: u32, height: u32) -> usize {
match self {
Self::Rgba8Unorm => (width as usize) * (height as usize) * 4,
@@ -161,7 +122,6 @@ impl BlpTexels {
}
}
/// The `texpresso` codec for a block form, or `None` for [`Self::Rgba8Unorm`].
fn codec(self) -> Option<texpresso::Format> {
match self {
Self::Rgba8Unorm => None,
@@ -172,13 +132,13 @@ impl BlpTexels {
}
}
/// A BLP with its levels in whatever form the file stores them — see [`decode_native`].
/// A BLP with its levels in the form the file stores them.
pub struct NativeBlp {
pub width: u32,
pub height: u32,
/// What every entry of `mips` holds.
pub texels: BlpTexels,
/// One buffer per level, level 0 first. Never empty (same guarantee as [`DecodedBlp::mips`]).
/// One buffer per level, level 0 first; never empty.
pub mips: Vec<NativeMip>,
mip_chain_count: usize,
}
@@ -187,11 +147,8 @@ pub struct NativeBlp {
pub struct NativeMip {
pub width: u32,
pub height: u32,
/// Block bytes (DXTC) or RGBA8 pixels — per [`NativeBlp::texels`]. Always exactly
/// [`BlpTexels::level_bytes`] long: a sub-block tail level the header under-stores is
/// completed from the bytes that follow it in the file, as the reference copies it
/// (`Header::dxt_level_span`), and only a level the file ends inside is zero-padded —
/// matching what [`decode`] does before handing blocks to the codec.
/// Block bytes or RGBA8 pixels, per [`NativeBlp::texels`], exactly [`BlpTexels::level_bytes`]
/// long: a short level is completed from the bytes after it, as the reference reads it.
pub bytes: Vec<u8>,
}
@@ -212,20 +169,15 @@ struct Header<'a> {
offsets: Vec<u32>,
sizes: Vec<u32>,
palette: &'a [u8],
/// The authored chain length, `mip_chain_count(width, height, has_mipmaps)`.
chain: usize,
}
/// Parse and bounds-check the BLP2 header, palette and level table. Shared by [`decode`] and
/// [`decode_native`] so the two can never disagree about what a file says.
/// Parse and bounds-check the BLP2 header, palette and level table.
fn parse_header(bytes: &[u8]) -> Result<Header<'_>> {
if bytes.len() < HEADER_SIZE || &bytes[0..4] != b"BLP2" {
return Err(Error::NotBlp2);
}
// content @4 (u32; 1 = Direct — we only handle direct, the only kind 1.12 ships).
// These reads are already covered by the `HEADER_SIZE` length check above; routed through
// `ByteExt` anyway for uniformity (decision 0064) — no future reader should have to ask
// "is this offset guarded?".
// `content` @4 is unread: 1.12 ships only Direct (1).
let compression = bytes.u8_at(8).ok_or(Error::Truncated("header"))?;
let alpha_bits = bytes.u8_at(9).ok_or(Error::Truncated("header"))? as u32;
let alpha_type = bytes.u8_at(10).ok_or(Error::Truncated("header"))?;
@@ -239,15 +191,12 @@ fn parse_header(bytes: &[u8]) -> Result<Header<'_>> {
.map(|i| bytes.u32_at(84 + i * 4).ok_or(Error::Truncated("header")))
.collect::<Result<_>>()?;
// The corrupt-header guard (see `MAX_DIM`): bound before anything downstream sizes an
// allocation from these fields. Zero is left alone — see the decode_dxt/decode_raw1/
// decode_raw3 zero-pixel behavior, unchanged from before this migration.
// Checked before anything sizes a buffer from them; zero passes.
if width > MAX_DIM || height > MAX_DIM {
return Err(Error::DimensionsTooLarge { width, height });
}
// BLP2 direct content always carries a 256-entry palette right after the header (used only by
// Raw1; present-but-ignored for Raw3/DXT). Read as BGRA bytes.
// A 256-entry BGRA palette always follows the header; only Raw1 reads it.
let palette = bytes
.get(HEADER_SIZE..HEADER_SIZE + PALETTE_SIZE)
.ok_or(Error::BadColorMap)?;
@@ -267,8 +216,7 @@ fn parse_header(bytes: &[u8]) -> Result<Header<'_>> {
}
impl Header<'_> {
/// The raw stored bytes of level `level`, or `None` when the chain ends early (some textures
/// stop before the formula's count).
/// Level `level`'s stored bytes, or `None` where the chain ends before the formula's count.
fn level_data<'b>(&self, bytes: &'b [u8], level: usize) -> Result<Option<&'b [u8]>> {
let off = self.offsets[level] as usize;
let sz = self.sizes[level] as usize;
@@ -281,25 +229,12 @@ impl Header<'_> {
.ok_or(Error::OutOfBounds { level })
}
/// The bytes of DXTC level `level` **as the reference copies them** — `need` bytes from the
/// level's file offset, not the `sizes[level]` the header records — or `None` when the chain
/// ends early (same rule as [`Self::level_data`]).
///
/// The two disagree exactly on the **sub-block tail**. The BLP encoder stores a level as
/// `max(1, (w/4)·(h/4))` blocks — ONE block for a 2×16 or a 1×8 level, where the block grid the
/// GPU decodes needs four and two. The reference never reads the header's size for a DXT level.
/// On its shipped Direct3D default the fill copies each level through the block copier
/// `0x5a5780`, `max(4,w)·max(4,h)` bytes flat from `fileImage + mipOffsets[level]`; on the
/// OpenGL arm it hands the upload that same pointer (`0x5a8555`, no copy) and `0x59f270` sizes
/// the read from the format alone, `bpp · max(4, wL) · max(4, hL) / 8` (`0x59f4e6`–`0x59f515`).
/// Either way the GPU gets the full block grid from the level's offset, so a short level is
/// completed with the bytes that FOLLOW it in the file: the next levels' own authored blocks
/// (decisions 2020/2024). Padding the difference with
/// zeros instead (an all-zero BC block = colour black, alpha 0) is what turned the far rain
/// black: `RainDrop01.blp`'s levels 3 and 4 came out three-quarters and half black, and its
/// Mod2x lane reads no alpha (decision 2020, B358/B225). Only a level the FILE itself ends
/// inside is left short here, for [`pad_to`] to zero-fill — the reference reads past the file
/// into whatever its shared read buffer held before, which is nothing to be faithful to.
/// DXTC level `level` as the reference reads it: `bpp·max(4,w)·max(4,h)/8` bytes from the
/// level's offset, never the header's size (`0x5a8555` points each level into the file;
/// OpenGL sizes the read at `0x59f4e6`–`0x59f515`, Direct3D copies `max(4,w)·max(4,h)` bytes
/// via `0x5a5780`). The encoder stores `max(1, (w/4)·(h/4))` blocks, one for a 2×16 level
/// whose grid needs four, so a short level takes the next levels' bytes. Deviation: past the
/// file's end [`pad_to`] zero-fills, because the reference reads stale buffer bytes there.
fn dxt_level_span<'b>(
&self,
bytes: &'b [u8],
@@ -317,9 +252,8 @@ impl Header<'_> {
Ok(Some(&bytes[off..end]))
}
/// Which block form this file's DXTC levels are in. `alpha_type` 0→DXT1, 1→DXT3, 7→DXT5;
/// anything else (a stale byte — e.g. `2` on alpha-less particle atlases) → DXT1, since
/// `alpha_bits` already governs whether alpha is meaningful.
/// The DXTC form by `alpha_type`. Any other value (a stale `2` on alpha-less particle atlases)
/// is DXT1, not an error: `alpha_bits` governs alpha.
fn dxt_texels(&self) -> BlpTexels {
match self.alpha_type {
1 => BlpTexels::Bc2,
@@ -332,15 +266,12 @@ impl Header<'_> {
/// Decode a BLP2 texture (raw archive bytes) to RGBA8 mip levels.
pub fn decode(bytes: &[u8]) -> Result<DecodedBlp> {
let h = parse_header(bytes)?;
// Always decode level 0 (blp_to_rgba needs it even when the chain count is 0); then deeper levels
// up to the chain count, stopping if the file doesn't carry one.
// Level 0 always, then deeper levels up to the chain count while the file carries them.
let n = h.chain.clamp(1, 16);
let mut mips = Vec::with_capacity(n);
for level in 0..n {
let (lw, lh) = level_size(h.width, h.height, level);
// DXTC levels span what the reference copies (`dxt_level_span`), not what the header
// records — the two differ on the sub-block tail, and the difference was black.
let data = if h.compression == 2 {
let texels = h.dxt_texels();
h.dxt_level_span(bytes, level, texels.level_bytes(lw, lh))?
@@ -371,13 +302,8 @@ pub fn decode(bytes: &[u8]) -> Result<DecodedBlp> {
})
}
/// Decode a BLP2 texture **keeping its DXTC blocks verbatim** — the form the reference uploads.
///
/// DXTC levels come back as block bytes with [`NativeBlp::texels`] naming the variant; Raw1/Raw3
/// files, which have no block form, are decoded exactly as [`decode`] would and report
/// [`BlpTexels::Rgba8Unorm`]. Either way every level is padded to its full
/// [`BlpTexels::level_bytes`], so a caller can concatenate the chain and hand it to the GPU without
/// re-deriving sizes.
/// Decode a BLP2 texture keeping its DXTC blocks verbatim, the form the reference uploads; Raw1
/// and Raw3 decode as in [`decode`].
pub fn decode_native(bytes: &[u8]) -> Result<NativeBlp> {
let h = parse_header(bytes)?;
let texels = if h.compression == 2 {
@@ -421,12 +347,8 @@ pub fn decode_native(bytes: &[u8]) -> Result<NativeBlp> {
})
}
/// Decode one block-compressed level to RGBA8 — the inverse of keeping the blocks.
///
/// For a caller that took [`decode_native`]'s passthrough and then found it could not upload it
/// after all (no BC on the device): it can get the pixels without re-reading and re-parsing the
/// file. `texels` must be a block form; [`BlpTexels::Rgba8Unorm`] returns `bytes` unchanged, since
/// that is already the answer.
/// Decode one block-compressed level to RGBA8, for a device without BC support;
/// [`BlpTexels::Rgba8Unorm`] returns `bytes` unchanged.
pub fn decode_level(texels: BlpTexels, width: u32, height: u32, bytes: &[u8]) -> Vec<u8> {
let Some(fmt) = texels.codec() else {
return bytes.to_vec();
@@ -437,12 +359,8 @@ pub fn decode_level(texels: BlpTexels, width: u32, height: u32, bytes: &[u8]) ->
out
}
/// `data` grown to exactly `need` bytes with zeros (or truncated if the file over-stores).
///
/// After `Header::dxt_level_span` this only ever pads a level the FILE ends inside (a truncated
/// archive) — the sub-block tail the encoder under-stores is completed from the following levels
/// first, the way the reference does. Zero here is a BC block that decodes to black at alpha 0,
/// which is why the span comes first (decision 2020).
/// `data` zero-padded or truncated to exactly `need` bytes. A zero BC block is black at alpha 0,
/// so a DXTC level is completed from the file first (`Header::dxt_level_span`).
fn pad_to(data: &[u8], need: usize) -> Vec<u8> {
let mut out = vec![0u8; need];
let n = data.len().min(need);
@@ -450,7 +368,7 @@ fn pad_to(data: &[u8], need: usize) -> Vec<u8> {
out
}
/// Palettized: 1-byte indices into a 256-entry BGRA palette, then a packed alpha block (`alpha_bits`).
/// Palettized: 1-byte indices into the BGRA palette, then a packed `alpha_bits` alpha block.
fn decode_raw1(palette: &[u8], data: &[u8], w: u32, h: u32, alpha_bits: u32) -> Result<Vec<u8>> {
let px = (w as usize) * (h as usize);
let indices = data.get(..px).ok_or(Error::Truncated("raw1 indices"))?;
@@ -459,7 +377,7 @@ fn decode_raw1(palette: &[u8], data: &[u8], w: u32, h: u32, alpha_bits: u32) ->
for i in 0..px {
let ci = indices[i] as usize;
let p = ci * 4;
// Palette is BGRA: byte0=B, byte1=G, byte2=R. (The fork fix — RGBA reading swaps R↔B.)
// The palette is BGRA.
out[i * 4] = palette[p + 2]; // R
out[i * 4 + 1] = palette[p + 1]; // G
out[i * 4 + 2] = palette[p]; // B
@@ -468,7 +386,7 @@ fn decode_raw1(palette: &[u8], data: &[u8], w: u32, h: u32, alpha_bits: u32) ->
Ok(out)
}
/// Per-pixel alpha from the packed block. `0` ⇒ opaque (no alpha channel); `1/4/8` ⇒ packed bits.
/// Pixel `i`'s alpha from a block of 1, 4 or 8 bits per pixel; 0 bits is opaque.
fn alpha_at(alpha: &[u8], i: usize, alpha_bits: u32) -> u8 {
match alpha_bits {
1 => {
@@ -507,9 +425,7 @@ fn decode_raw3(data: &[u8], w: u32, h: u32) -> Result<Vec<u8>> {
Ok(out)
}
/// DXTC → RGBA8. The variant comes from [`Header::dxt_texels`]. `data` is the level's span as
/// the reference copies it (`Header::dxt_level_span`); anything still short of the block size —
/// a file that ends inside the level — is zero-padded so the level decodes.
/// DXTC to RGBA8; a level the file ends inside is zero-padded first.
fn decode_dxt(data: &[u8], w: u32, h: u32, texels: BlpTexels) -> Result<Vec<u8>> {
let fmt = texels.codec().expect("dxt_texels never returns Rgba8Unorm");
let blocks = pad_to(data, fmt.compressed_size(w as usize, h as usize));
@@ -522,7 +438,7 @@ fn decode_dxt(data: &[u8], w: u32, h: u32, texels: BlpTexels) -> Result<Vec<u8>>
mod tests {
use super::*;
/// Build a minimal-but-complete BLP2 header (magic..=mip_sizes[16], `HEADER_SIZE` bytes).
/// A BLP2 header, magic to `mip_sizes[16]`.
fn header(
compression: u8,
alpha_bits: u8,
@@ -553,9 +469,7 @@ mod tests {
#[test]
fn corrupt_header_huge_dims_errors_cleanly() {
// The fix under test: a header claiming an absurd width/height must fail fast, not size a
// multi-gigabyte allocation in decode_dxt. No palette/pixel bytes needed — the dimension
// check runs before either is touched.
// No palette or pixels: the dimension check runs first.
let b = header(3, 0, 0, 0, 65535, 65535, [0; 16], [0; 16]);
assert!(matches!(
decode(&b),
@@ -564,8 +478,6 @@ mod tests {
height: 65535
})
));
// Right at the cap is fine (still fails later for lack of palette/pixel bytes, but not on
// the dimension check); just over the cap on either axis alone is rejected.
let b = header(3, 0, 0, 0, MAX_DIM + 1, 4, [0; 16], [0; 16]);
assert!(matches!(decode(&b), Err(Error::DimensionsTooLarge { .. })));
let b = header(3, 0, 0, 0, 4, MAX_DIM + 1, [0; 16], [0; 16]);
@@ -574,7 +486,6 @@ mod tests {
#[test]
fn truncated_header_errors_cleanly() {
// Shorter than HEADER_SIZE, even with a valid magic, is rejected before any field read.
let mut b = vec![0u8; 10];
b[0..4].copy_from_slice(b"BLP2");
assert!(matches!(decode(&b), Err(Error::NotBlp2)));
@@ -582,15 +493,13 @@ mod tests {
assert!(matches!(decode(b"not a blp at all"), Err(Error::NotBlp2)));
}
/// Build a DXT-compressed (compression 2) BLP with a real mip chain: 8x8 down to 1x1, so the
/// tail levels are the sub-block ones that make block handling interesting.
/// A DXTC BLP (compression 2) with an 8x8 to 1x1 chain, so the tail levels are sub-block.
fn dxt_blp(alpha_type: u8, block_bytes: usize) -> Vec<u8> {
let mut offsets = [0u32; 16];
let mut sizes = [0u32; 16];
let mut payload = Vec::new();
let base = (HEADER_SIZE + PALETTE_SIZE) as u32;
// 8x8, 4x4, 2x2, 1x1 — mip_chain_count(8, 8, true) == 3, so both entry points read the
// first THREE levels (8x8, 4x4, 2x2); the 1x1 is stored but past the authored count.
// The chain count for 8x8 is 3, so the stored 1x1 is past it and unread.
for (i, (w, h)) in [(8u32, 8u32), (4, 4), (2, 2), (1, 1)]
.into_iter()
.enumerate()
@@ -599,7 +508,7 @@ mod tests {
let n = blocks * block_bytes;
offsets[i] = base + payload.len() as u32;
sizes[i] = n as u32;
// Deterministic non-trivial block payload so a byte swap would show.
// A non-trivial payload, so a byte swap would show.
payload.extend((0..n).map(|k| (k as u32 * 37 + i as u32 * 11) as u8));
}
let mut b = header(2, 8, alpha_type, 1, 8, 8, offsets, sizes);
@@ -608,10 +517,6 @@ mod tests {
b
}
/// The load-bearing equivalence for the native upload path: `decode_native` must hand back the
/// file's blocks *verbatim*, and running those blocks through the codec must reproduce exactly
/// what `decode` produces. If these ever diverge, a GPU-uploaded texture stops matching what
/// every CPU-side consumer sees.
#[test]
fn native_blocks_are_verbatim_and_decode_to_the_same_pixels() {
for (alpha_type, texels, block_bytes) in [
@@ -631,21 +536,17 @@ mod tests {
for (level, (n, d)) in native.mips.iter().zip(&decoded.mips).enumerate() {
assert_eq!((n.width, n.height), (d.width, d.height));
// Every level is exactly its full block size — including the 2x2 and 1x1 tails,
// which occupy one whole block each.
assert_eq!(
n.bytes.len(),
texels.level_bytes(n.width, n.height),
"level {level} block size"
);
// Verbatim: the bytes are the file's, not re-encoded.
let off = native_level_offset(&b, level);
assert_eq!(
&n.bytes[..],
&b[off..off + n.bytes.len()],
"level {level} must be the file's own blocks"
);
// And they decode to exactly what the RGBA path produced.
let fmt = texels.codec().unwrap();
let mut out = vec![0u8; (n.width as usize) * (n.height as usize) * 4];
fmt.decompress(&n.bytes, n.width as usize, n.height as usize, &mut out);
@@ -654,13 +555,11 @@ mod tests {
}
}
/// Where level `i`'s payload starts, read back out of the header we built.
/// Where `level`'s payload starts, read back from the header.
fn native_level_offset(blp: &[u8], level: usize) -> usize {
u32::from_le_bytes(blp[20 + level * 4..24 + level * 4].try_into().unwrap()) as usize
}
/// Raw1/Raw3 have no block form, so the native path decodes them and says so — a caller that
/// switches on `texels` gets a correct `Rgba8Unorm` upload rather than garbage.
#[test]
fn native_reports_rgba_for_the_shapes_with_no_block_form() {
let pixel_offset = (HEADER_SIZE + PALETTE_SIZE) as u32;
@@ -684,16 +583,12 @@ mod tests {
);
}
/// A level the header under-stores is completed from the bytes that FOLLOW it in the file —
/// the reference's upload (`0x59f4e6`–`0x59f515`: `bpp·max(4,w)·max(4,h)/8` bytes read from
/// the level's file offset) — so the concatenated chain a caller uploads is always exactly
/// the size wgpu expects, and it is the file's bytes, not zeros. Zeros were the bug: an
/// all-zero BC block is black.
/// The reference reads `bpp·max(4,w)·max(4,h)/8` bytes from the level's offset
/// (`0x59f4e6`–`0x59f515`), whatever the header records.
#[test]
fn a_short_level_is_completed_from_the_bytes_that_follow_it() {
let mut b = dxt_blp(0, 8);
// Level 2 is the 2x2 tail — one whole BC1 block. Shrink its recorded size to 3 bytes; the
// 1x1 level's 8 bytes follow it in the file.
// Level 2, the 2x2 tail, is one BC1 block: record it as 3 bytes, the 1x1's 8 following.
let short = 3u32;
b[84 + 2 * 4..88 + 2 * 4].copy_from_slice(&short.to_le_bytes());
let off = native_level_offset(&b, 2);
@@ -710,7 +605,7 @@ mod tests {
tail.bytes[short as usize..].iter().any(|&x| x != 0),
"nothing was zero-filled — that was B358's black"
);
// And decode() reads the same span — the two paths complete identically.
// `decode` reads the same span.
let decoded = decode(&b).unwrap();
assert_eq!(decoded.mips.len(), native.mips.len());
let mut out = vec![0u8; 2 * 2 * 4];
@@ -718,15 +613,12 @@ mod tests {
assert_eq!(out, decoded.mips.last().unwrap().rgba);
}
/// **The shipped shape of the bug**, in miniature: a DXT3 texture taller than it is wide,
/// whose 2×8 level the encoder stores as ONE block (`max(1, (2/4)·(8/4))`) where the block
/// grid needs two. The reference uploads `8·max(4,2)·max(4,8)/8 = 32` bytes from that level's
/// offset — the stored block plus the 1×4 level's block after it. `RainDrop01.blp` is this at
/// 16×128: its 2×16 and 1×8 levels store 16 bytes each where the grid needs 64 and 32.
/// A 2×8 DXT3 level is stored as one block (`max(1, (2/4)·(8/4))`) where the grid needs two;
/// the reference reads `8·max(4,2)·max(4,8)/8 = 32` bytes, taking the 1×4 level's block too.
/// `RainDrop01.blp` ships this shape at 16×128.
#[test]
fn a_sub_block_dxt3_level_spans_into_its_successors_as_the_reference_copies_it() {
// 4x16 DXT3: levels 4x16 (4 blocks), 2x8 (grid: 2 blocks; STORED: 1), 1x4 (1 block),
// 1x2 (1 block). mip_chain_count(4, 16, true) = 4, so all four are read.
// 4x16 DXT3: 4x16 (4 blocks), 2x8 (grid 2, stored 1), 1x4 (1), 1x2 (1); a chain of 4.
let mut offsets = [0u32; 16];
let mut sizes = [0u32; 16];
let mut payload = Vec::new();
@@ -757,12 +649,11 @@ mod tests {
l1.bytes[16..].iter().all(|&x| x == 0x30),
"second block: the 1x4 level's, which follows it in the file — never zeros"
);
// The levels that ARE a whole block are untouched by the rule.
// Whole-block levels are untouched.
assert!(native.mips[2].bytes.iter().all(|&x| x == 0x30));
assert!(native.mips[3].bytes.iter().all(|&x| x == 0x40));
// A short level the FILE ends inside is the one case left for the zero pad: the same
// texture authored with only its first two levels, the file ending right after the 2x8
// level's single stored block — the completion has nothing to read.
// A level the file ends inside is zero-padded: the same texture, two levels, ending after
// the 2x8's one stored block.
let mut offsets = [0u32; 16];
let mut sizes = [0u32; 16];
offsets[0] = base;
@@ -786,8 +677,7 @@ mod tests {
#[test]
fn tiny_raw3_decodes_expected_pixels() {
// A 2x2 uncompressed BGRA8 (Raw3) image, no mipmaps: the smallest concrete shape decode()
// handles. Pixel data sits right after the header + 256-entry BGRA palette.
// A 2x2 Raw3 image, no mipmaps; the pixels follow the header and palette.
let pixel_offset = (HEADER_SIZE + PALETTE_SIZE) as u32;
let pixel_size = 2 * 2 * 4;
let mut offsets = [0u32; 16];
@@ -795,8 +685,7 @@ mod tests {
offsets[0] = pixel_offset;
sizes[0] = pixel_size;
let mut b = header(3, 8, 0, 0, 2, 2, offsets, sizes);
b.resize(HEADER_SIZE + PALETTE_SIZE, 0); // palette bytes: unused by Raw3, present anyway
// Four BGRA8 pixels.
b.resize(HEADER_SIZE + PALETTE_SIZE, 0); // the palette, unused by Raw3
let bgra: &[u8] = &[
10, 20, 30, 40, // B G R A
50, 60, 70, 80, //
@@ -808,13 +697,12 @@ mod tests {
let decoded = decode(&b).expect("valid tiny BLP decodes");
assert_eq!(decoded.width, 2);
assert_eq!(decoded.height, 2);
assert_eq!(decoded.mip_chain_count(), 0); // no mipmaps: chain formula reports 0
assert_eq!(decoded.mips.len(), 1); // level 0 is still decoded regardless
assert_eq!(decoded.mip_chain_count(), 0); // no mipmaps
assert_eq!(decoded.mips.len(), 1); // level 0 is still decoded
let mip = &decoded.mips[0];
assert_eq!(mip.width, 2);
assert_eq!(mip.height, 2);
// BGRA -> RGBA per pixel.
assert_eq!(
mip.rgba,
vec![
crates/benilla-buildstamp/src/lib.rs
+22 -59
View File
@@ -1,61 +1,26 @@
//! The **build-id stamp emitter** — the `build.rs` body both launcher shims share.
//!
//! Two binaries carry the stamp now (`benilla` and `benilla-worldview`, decision 1160), and a
//! copy-pasted build script is exactly the thing that rots: one shim gets a fix, the other keeps
//! reporting a sha by a rule that changed. So the script is a function and each shim's `build.rs`
//! is one line that calls it.
//!
//! It stays a **build-dependency of the shims**, never of `benilla-app`: cargo dirties the compile
//! units of the package that owns a `rerun-if-changed` path on the file's mtime alone, and those
//! paths move on every commit, rebase and checkout. In a ~30-line shim that costs the shim's
//! recompile plus one relink; in the app crate it cost ~2 minutes of pure stamp tax per commit
//! across the gates, which is what decision 0993 measured and moved out.
//! The build stamp: the `build.rs` body the `benilla` and `benilla-worldview` launcher shims
//! share. It stays a build-dependency of the shims, never of `benilla-app`: cargo rebuilds the
//! package that owns a `rerun-if-changed` path whenever that path's mtime moves, and these paths
//! move on every commit, rebase and checkout.
use std::path::Path;
use std::process::Command;
/// Stamp the **commit this binary was built from** into the calling package: its `main.rs`
/// reads the four vars back with `env!` and hands them to `benilla_app`'s entry point as the
/// `BuildId` resource (`benilla-app/src/build_id.rs` — the runtime side and its two surfaces).
/// Stamp the commit this binary was built from into the calling package as four `rustc-env`
/// vars, which its `main.rs` reads back with `env!` into a `BuildId`:
///
/// Why it exists: people run benilla from a clone of the public snapshot repo, and a report ("this
/// looks wrong", "it crashed here") is only actionable if we know *which* code they ran. A git sha
/// is the only id that can't drift — the crate version is a permanent `0.1.0` and there are no
/// releases to number. The stamp is the commit the binary was built from, on any checkout.
/// - `BENILLA_GIT_SHA`: the full sha.
/// - `BENILLA_GIT_SHORT`: git's own abbreviation of it.
/// - `BENILLA_GIT_DATE`: the commit date (`%cs`, `YYYY-MM-DD`), not the build date.
/// - `BENILLA_PROFILE`: the profile directory's name, which tells `ship` from `release` where
/// cargo's `PROFILE` does not.
///
/// It emits four `rustc-env` vars. The three git ones come back **empty** when git can't answer (a
/// source zip with no `.git`, or no `git` on PATH), which the runtime side reports as an unknown
/// build rather than failing to compile:
///
/// - `BENILLA_GIT_SHA` — the full 40-char sha (what the panel copies to the clipboard).
/// - `BENILLA_GIT_SHORT` — git's own abbreviation, so the string matches the `pub/<sha>` tag names
/// (`pubsync` abbreviates the same way, in the same repo).
/// - `BENILLA_GIT_DATE` — the commit date (`%cs`, `YYYY-MM-DD`). Committer date, not build date:
/// it is a property of the sha, so it can never disagree with it.
///
/// …plus `BENILLA_PROFILE`, the profile directory name — `debug` / `release` / `ship` (0736's
/// shipping profile). Cargo's own `PROFILE` var collapses every release-like profile to `release`,
/// so `ship` and `release` would be indistinguishable; the profile *directory* inside `OUT_DIR`
/// carries the real name, which is what a "why is this slow" report needs to say.
///
/// ## Staleness — the only thing that could make this lie
///
/// A stamp is worthless if it can report a commit the binary isn't. The rerun triggers below are
/// therefore exactly the files that change when HEAD moves: `HEAD` itself (checkout, rebase,
/// detach) and the ref it names (a commit rewrites `refs/heads/<branch>`, not `HEAD`). Both are
/// resolved with `git rev-parse --git-path`, which applies the worktree rules — in a linked
/// worktree `.git` is a *file*, `HEAD` lives in `<main>/.git/worktrees/<name>/`, and `refs/heads/*`
/// stay in the common dir. Only paths that exist are emitted: cargo reruns a build script whose
/// watched path is *missing* on every build, and every one of those reruns relinks the app crate.
///
/// What the stamp deliberately does **not** claim is that the working tree was clean: it names the
/// commit the build came *from*. A dirty flag can only be as fresh as the last build-script run, so
/// in the normal dev loop (edit, run, don't stage) it would read "clean" while the binary carried
/// uncommitted work — a flag that is wrong exactly when it matters. "Which snapshot" is the
/// question a remote report needs answered, and a bare sha answers it without lying.
/// The git vars are empty when git cannot answer (no `.git`, no `git` on `PATH`), which the
/// runtime reports as an unknown build. The rerun triggers are `HEAD` and the ref it names,
/// resolved with `git rev-parse --git-path` so a linked worktree resolves too; only paths that
/// exist are emitted, since cargo reruns a build script whose watched path is missing on every
/// build.
pub fn emit() {
// Build scripts run with the package root as cwd, but be explicit: `-C` makes the git calls
// independent of that guarantee.
let dir = std::env::var("CARGO_MANIFEST_DIR").expect("cargo sets CARGO_MANIFEST_DIR");
let git = |args: &[&str]| -> Option<String> {
let out = Command::new("git")
@@ -79,14 +44,12 @@ pub fn emit() {
};
watch(git(&["rev-parse", "--git-path", "HEAD"]));
if let Some(git_ref) = git(&["symbolic-ref", "-q", "HEAD"]) {
// The loose ref file, plus `packed-refs` for the freshly-cloned/gc'd repo where the branch
// has no loose file at all (a public clone is exactly that).
// The loose ref, plus `packed-refs`: a fresh clone's branch has no loose file.
watch(git(&["rev-parse", "--git-path", &git_ref]));
watch(git(&["rev-parse", "--git-path", "packed-refs"]));
}
// A stamp that never changes when the rule is edited would be its own kind of stale. The
// shim's `build.rs` is watched by cargo automatically; this file is watched because cargo
// rebuilds a build script when its build-dependency changes.
// An edit to the rule restamps: cargo reruns a build script when `build.rs` or one of its
// build-dependencies changes.
println!("cargo::rerun-if-changed=build.rs");
for (var, args) in [
@@ -96,12 +59,12 @@ pub fn emit() {
] {
println!(
"cargo::rustc-env={var}={}",
git(args).unwrap_or_default() // empty ⇒ "unknown build" at runtime
git(args).unwrap_or_default() // empty: an unknown build at runtime
);
}
// `OUT_DIR` is `<target>/[<triple>/]<profile-dir>/build/<pkg>-<hash>/out` — the component
// before `build` is the profile directory. Falls back to cargo's coarse `PROFILE`.
// `OUT_DIR` is `<target>/[<triple>/]<profile-dir>/build/<pkg>-<hash>/out`; the fallback is
// cargo's coarse `PROFILE`.
let out_dir = std::env::var("OUT_DIR").unwrap_or_default();
let parts: Vec<_> = Path::new(&out_dir).components().collect();
let profile = parts
crates/benilla-bytes/src/lib.rs
+10 -26
View File
@@ -1,21 +1,7 @@
//! Bounds-checked byte readers + IFF chunk iteration shared by the format parsers
//! (decision 0064). Three primitives, nothing else:
//!
//! - [`ByteExt`] — fallible little-endian accessors on `[u8]`. There is deliberately **no**
//! panicking variant: parsers map `None` to their own truncation error with `?`, so an unguarded
//! offset is a compile-shape impossibility, not a convention.
//! - [`chunks`] — the IFF `(magic, payload)` iterator every chunked WoW format walks. **Lenient by
//! documented property**: iteration stops cleanly at a truncated 8-byte header, and a payload
//! whose declared size overruns the buffer is clamped to the buffer end (real 1.12 art needs the
//! tolerance — e.g. the trailing-junk chunks some Classic files carry).
//! - [`capped`] — the allocation guard for count-driven `Vec::with_capacity`: reserve at most what
//! the remaining input could possibly hold, so a corrupt header count can never reserve more
//! than the file's own size.
//!
//! Magic bytes are yielded exactly as stored (WoW writes IFF magics reversed on disk, so callers
//! match `b"DHOM"` for MOHD — same as the pre-0064 per-crate iterators).
//! Bounds-checked byte readers and IFF chunk iteration for the format parsers.
/// Fallible little-endian reads at an offset. `None` iff the read would run past the end.
/// Little-endian reads at an offset, `None` past the end. There is no panicking variant on
/// purpose: a parser maps `None` to its own truncation error.
pub trait ByteExt {
fn u8_at(&self, o: usize) -> Option<u8>;
fn u16_at(&self, o: usize) -> Option<u16>;
@@ -57,8 +43,9 @@ impl ByteExt for [u8] {
}
}
/// Iterate IFF chunks as `(magic-as-stored, payload)`. See the module doc for the (deliberate)
/// leniency: truncated header → clean stop; over-declared payload → clamped to the buffer end.
/// Iterate IFF chunks as `(magic, payload)`, the magic as stored: reversed on disk, so MOHD is
/// `b"DHOM"`. A truncated header ends iteration and an overrunning payload is clamped to the
/// buffer end, a tolerance real 1.12 art needs.
pub fn chunks(b: &[u8]) -> impl Iterator<Item = ([u8; 4], &[u8])> {
let mut pos = 0usize;
std::iter::from_fn(move || {
@@ -68,19 +55,16 @@ pub fn chunks(b: &[u8]) -> impl Iterator<Item = ([u8; 4], &[u8])> {
let magic = [b[pos], b[pos + 1], b[pos + 2], b[pos + 3]];
let size = b.u32_at(pos + 4)? as usize;
let start = pos + 8;
// `start <= b.len()` (checked above) and `size` came from a u32, so on 64-bit targets the
// sum cannot wrap; saturate anyway so the invariant doesn't depend on pointer width.
// Saturating, so a 32-bit target cannot wrap either.
let end = start.saturating_add(size).min(b.len());
pos = end;
Some((magic, &b[start..end]))
})
}
/// Cap a count-driven reservation by what `avail` remaining bytes could possibly hold, so a corrupt
/// header can at worst reserve the input's own size. `elem_size` of 0 is a caller bug; treat it as 1
/// rather than divide by zero. The *count* used for iteration stays the caller's — only the
/// up-front reservation is capped (a short file then fails at the bounds-checked read, not in the
/// allocator).
/// Cap a count-driven reservation at what `avail` bytes could hold, so a corrupt count reserves at
/// most the input's size; the caller still iterates its own count and fails at the checked read.
/// An `elem_size` of 0 counts as 1.
pub fn capped(count: usize, elem_size: usize, avail: usize) -> usize {
count.min(avail / elem_size.max(1))
}
crates/benilla-dbc/src/lib.rs
+21 -74
View File
@@ -1,20 +1,5 @@
//! A WDBC (client DBC) reader for **WoW 1.12.1 (build 5875)** — in-repo, replacing `wow-cdbc`
//! (decision 0021).
//!
//! A DBC is dead simple: a 20-byte `WDBC` header (record count, field count, record size, string-block
//! size), then `record_count × record_size` bytes of fixed-width **4-byte** fields, then a string
//! block. The file carries no column *types*, so the caller supplies a [`Schema`] (its expanded field
//! count must equal the header's `field_count`); each field is read as `u32`/`i32`/`f32`, or as a
//! [`StringRef`] offset into the string block. Layouts are validated clean-room against build 5875.
//!
//! Proven against `wow-cdbc` over real DBCs during the decision-0021 migration (oracle test in git
//! history); the catalog loaders in `benilla-formats` (Map/Light/Creature/GameObject/…) pin exact
//! decoded values end-to-end on every run.
//!
//! Byte access goes through `benilla-bytes` (decision 0064): header reads are bounds-checked, the
//! `record_count × record_size + string_block_size` size arithmetic is overflow-checked (a corrupt
//! header used to wrap past the parse-time guard and panic on the later re-slice), and record
//! reservations are capped by what the body could actually hold.
//! A WDBC reader for WoW 1.12.1: a 20-byte header, `record_count × record_size` bytes of 4-byte
//! fields, then a string block. The file has no column types; the caller supplies a [`Schema`].
use std::io::{Cursor, Write};
@@ -30,8 +15,7 @@ pub enum FieldType {
String,
}
/// One schema field: a name, a type, and a repeat `count` (1 for a scalar; N for an inline array, which
/// occupies N consecutive 4-byte slots).
/// One schema field; a `count` above 1 is an inline array of that many 4-byte slots.
#[derive(Debug, Clone)]
pub struct SchemaField {
pub name: String,
@@ -82,7 +66,7 @@ impl Schema {
self.key_field = Some(name.into());
}
/// Total 4-byte slots this schema covers (arrays expanded) — must equal the header `field_count`.
/// The 4-byte slots covered, arrays expanded; the header's `field_count` must match.
fn expanded_len(&self) -> usize {
self.fields.iter().map(|f| f.count).sum()
}
@@ -134,8 +118,7 @@ pub enum Error {
file: u32,
},
BadStringRef(u32),
/// `record_count × record_size + string_block_size` overflows `usize` — a corrupt header, not a
/// real file (see decision 0064: this used to wrap silently and panic on the later re-slice).
/// `record_count × record_size + string_block_size` overflows `usize`: a corrupt header.
SizeOverflow,
}
@@ -162,12 +145,7 @@ impl std::error::Error for Error {}
type Result<T> = std::result::Result<T, Error>;
/// Read a little-endian u32, zero-extending any bytes past the end of `b` (a record whose
/// `record_size` is shorter than `field_count × 4` — see the parse loop). This deliberately differs
/// from [`ByteExt::u32_at`], which is all-or-nothing (`None` on any out-of-range byte): a short tail
/// here is a legitimate, previously-observed record shape, not truncation, so it zero-pads instead of
/// failing. Built on the same bounds-checked primitive (`u8_at`) byte-by-byte so there is no raw
/// indexing left to panic.
/// A little-endian u32 whose bytes past the end of `b` read as zero, unlike [`ByteExt::u32_at`].
fn rd_u32_at(b: &[u8], o: usize) -> u32 {
let mut bytes = [0u8; 4];
for (k, slot) in bytes.iter_mut().enumerate() {
@@ -178,12 +156,7 @@ fn rd_u32_at(b: &[u8], o: usize) -> u32 {
u32::from_le_bytes(bytes)
}
/// The shared `record_count × record_size (+ string_block_size)` layout arithmetic, checked —
/// [`Header::record_count`]/`record_size`/`string_block_size` are attacker-controlled `u32`s, and a
/// corrupt combination must error, never silently wrap past a size guard onto a later panicking
/// re-slice (decision 0064: this is the verified bug the migration fixes). Both `parse` and
/// `parse_records` call this one function so the two can never disagree even if the code drifts
/// later. Returns `(record_bytes_len, record_bytes_len + string_block_size)`.
/// `(record bytes, record bytes + string block)`, checked: the header's counts are unvalidated.
fn checked_body_layout(
record_count: usize,
record_size: usize,
@@ -194,17 +167,16 @@ fn checked_body_layout(
Some((record_bytes_len, total))
}
/// A DBC opened for reading: header parsed, body+strings located, schema optional until applied.
/// A DBC opened for reading; a [`Schema`] is attached before decoding records.
pub struct DbcParser<'a> {
header: Header,
/// Everything after the 20-byte header: `record_count × record_size` record bytes + string block.
/// Everything after the 20-byte header: the record bytes, then the string block.
body: &'a [u8],
schema: Option<Schema>,
}
impl<'a> DbcParser<'a> {
/// Parse the header from a cursor over the whole DBC bytes (position is ignored; the file starts
/// with the header). Kept cursor-shaped to match the call sites.
/// Parse the header of a whole DBC; the cursor's position is ignored.
pub fn parse(cursor: &mut Cursor<&'a [u8]>) -> Result<Self> {
let all: &'a [u8] = cursor.get_ref();
if all.len() < 20 || &all[0..4] != b"WDBC" {
@@ -256,8 +228,6 @@ impl<'a> DbcParser<'a> {
let rc = self.header.record_count as usize;
let rs = self.header.record_size as usize;
let fc = self.header.field_count as usize;
// Same checked arithmetic `parse` used to size-check the header — recomputed here (not
// assumed) so the two can never disagree even if the code drifts later.
let (record_bytes_len, strings_end) =
checked_body_layout(rc, rs, self.header.string_block_size as usize)
.ok_or(Error::SizeOverflow)?;
@@ -271,7 +241,7 @@ impl<'a> DbcParser<'a> {
.ok_or(Error::Truncated("string block"))?
.to_vec();
// Expand the schema into a per-slot type list (arrays → repeated type) + per-slot column names.
// One type and one column name per 4-byte slot, arrays expanded.
let mut types = Vec::with_capacity(fc);
let mut names = Vec::with_capacity(fc);
for field in &schema.fields {
@@ -285,11 +255,8 @@ impl<'a> DbcParser<'a> {
}
}
// `rc` comes straight from the header, and `record_size == 0` lets any `rc` pass the size
// guard (0 × anything fits) — while `rd_u32_at` zero-extends rather than fails, so nothing
// below would stop a loop over the header's count from pushing every one of those records
// as zeros. A zero-byte record holds no field: a header that claims records at that size
// is refused, and the loop runs over what the record bytes hold, never the header's count.
// A `record_size` of 0 passes the size guard for any count and `rd_u32_at` never fails, so
// records claimed at size 0 are refused and the loop runs over what the bytes hold.
if rs == 0 && rc > 0 {
return Err(Error::Truncated("records claimed at record_size 0"));
}
@@ -345,8 +312,7 @@ impl RecordSet {
}
}
/// Write a record set as CSV (header row of column names, then one row per record; string refs
/// resolved). Used by the `benilla-extract` CLI. Minimal RFC-4180 quoting.
/// Write a record set as RFC-4180 CSV: column names, then one row per record, strings resolved.
pub fn export_to_csv<W: Write>(rs: &RecordSet, mut w: W) -> std::io::Result<()> {
writeln!(w, "{}", rs.field_names.join(","))?;
for record in &rs.records {
@@ -377,7 +343,7 @@ fn csv_quote(s: &str) -> String {
mod tests {
use super::*;
/// A minimal WDBC: 20-byte header + record bytes + string block.
/// A WDBC: the 20-byte header, the record bytes, the string block.
fn build_wdbc(
record_count: u32,
field_count: u32,
@@ -396,7 +362,6 @@ mod tests {
b
}
/// Two fields per record: `id` (UInt32), `name` (a string-block offset).
fn id_name_schema() -> Schema {
let mut s = Schema::new("Test");
s.add_field(SchemaField::new("id", FieldType::UInt32));
@@ -442,8 +407,6 @@ mod tests {
#[test]
fn rd_u32_at_zero_extends_short_tail() {
// Preserves the pre-0064 semantics exactly: a read that runs past the end zero-pads the
// missing bytes rather than failing, unlike `ByteExt::u32_at` (all-or-nothing).
let b = [0xAA, 0xBB];
assert_eq!(rd_u32_at(&b, 0), 0x0000_BBAA);
assert_eq!(rd_u32_at(&b, 1), 0x0000_00BB);
@@ -453,26 +416,20 @@ mod tests {
#[test]
fn header_size_arithmetic_overflow_errors_cleanly_not_panic() {
// Header fields are u32, so on a 64-bit host record_count*record_size+string_block_size can
// never actually overflow a 64-bit usize (u32::MAX^2 + u32::MAX < usize::MAX) — the real
// trigger is a 32-bit target wrapping in release, which is exactly what decision 0064 flags.
// Exercise the shared checked arithmetic directly with usize inputs a real u32 header could
// never produce on this host, to pin that it errors instead of silently wrapping.
// u32 header fields overflow only a 32-bit usize, so the arithmetic is driven directly.
assert_eq!(checked_body_layout(usize::MAX, 2, 0), None);
assert_eq!(checked_body_layout(4, 4, usize::MAX), None);
assert_eq!(checked_body_layout(usize::MAX, 1, 1), None);
assert_eq!(checked_body_layout(10, 4, 6), Some((40, 46)));
// And through the public API: a header whose declared size can't be satisfied by the (valid,
// in-bounds) body still errors cleanly rather than panicking on a later re-slice.
// Through the public API, an all-zero header parses.
let bytes = build_wdbc(0, 0, 0, &[], &[]);
assert!(DbcParser::parse(&mut Cursor::new(bytes.as_slice())).is_ok());
}
#[test]
fn truncated_body_errors_cleanly_not_panic() {
// Header claims 2 records of 8 bytes + a 10-byte string block, but the body holds only one
// record's worth of bytes and no string block at all.
// The header claims 2 records of 8 bytes; the body holds one.
let bytes = build_wdbc(2, 2, 8, &[0u8; 8], &[]);
let err = DbcParser::parse(&mut Cursor::new(bytes.as_slice()))
.err()
@@ -500,7 +457,6 @@ mod tests {
#[test]
fn schema_field_count_must_match_header() {
// Header declares 2 fields; a 3-slot schema is rejected before any record is touched.
let bytes = build_wdbc(1, 2, 8, &[0u8; 8], &[]);
let parser = DbcParser::parse(&mut Cursor::new(bytes.as_slice())).unwrap();
let mut wide = Schema::new("Wide");
@@ -515,7 +471,6 @@ mod tests {
#[test]
fn array_field_expands_to_consecutive_slots() {
// One record: a scalar `id` then a 3-wide `coords` array = 4 field slots.
let mut records = Vec::new();
records.extend_from_slice(&7u32.to_le_bytes()); // id
records.extend_from_slice(&10u32.to_le_bytes()); // coords[0]
@@ -533,7 +488,6 @@ mod tests {
.unwrap()
.parse_records()
.unwrap();
// Array occupies slots 1..=3, and the expanded column names carry the `[k]` index.
assert_eq!(
rs.field_names,
["id", "coords[0]", "coords[1]", "coords[2]"]
@@ -569,7 +523,6 @@ mod tests {
#[test]
fn get_string_handles_bad_ref_and_unterminated_tail() {
// A one-field record whose string ref we vary by hand.
let strings = b"hi\0tail"; // "tail" has no trailing NUL
let bytes = build_wdbc(1, 1, 4, &0u32.to_le_bytes(), strings);
let rs = {
@@ -583,9 +536,8 @@ mod tests {
.unwrap()
};
assert_eq!(rs.get_string(StringRef(0)).unwrap(), "hi");
// An offset into an un-terminated run reads to the end of the block.
assert_eq!(rs.get_string(StringRef(3)).unwrap(), "tail");
// Offset == len is the empty string at the very end (in bounds); past len errors.
// An offset equal to the length is the empty string; past it errors.
assert_eq!(rs.get_string(StringRef(strings.len() as u32)).unwrap(), "");
assert!(matches!(
rs.get_string(StringRef(strings.len() as u32 + 1)),
@@ -595,8 +547,6 @@ mod tests {
#[test]
fn csv_export_resolves_strings_and_quotes_per_rfc4180() {
// A row with a comma, a quote, and a newline must be wrapped and its quotes doubled; a plain
// row stays bare. Two fields: an id and a name that references the string block.
let strings = b"plain\0a,b\0he said \"hi\"\0line1\nline2\0";
// offsets: plain@0, "a,b"@6, 'he said "hi"'@10, "line1\nline2"@23
let mut records = Vec::new();
@@ -623,10 +573,7 @@ mod tests {
assert_eq!(csv, expected);
}
/// `record_size = 0` with a record count passes the size guard (0 × anything fits) and used
/// to run the record loop `record_count` times: `rd_u32_at` zero-extends rather than fails,
/// so a 20-byte file claiming `0xFFFFFFFF` records was 4 G pushes and an OOM kill — never
/// the error the old comment said the bounds-checked read would raise.
/// A 20-byte file claiming `u32::MAX` records of size 0 must not run the loop that many times.
#[test]
fn zero_record_size_with_records_is_refused_not_iterated() {
let bytes = build_wdbc(u32::MAX, 2, 0, &[], &[]);
@@ -635,7 +582,7 @@ mod tests {
.with_schema(id_name_schema())
.expect("schema matches");
assert!(matches!(parser.parse_records(), Err(Error::Truncated(_))));
// The honest empty file — no records, nothing to hold them — still decodes to nothing.
// No records at size 0 still decodes, to nothing.
let bytes = build_wdbc(0, 2, 0, &[], &[]);
let rs = DbcParser::parse(&mut Cursor::new(bytes.as_slice()))
.unwrap()
crates/benilla-m2/examples/dump_attach.rs
+3 -4
View File
@@ -1,7 +1,6 @@
//! Session probe: dump an M2's attachment table (+ each attach bone's parent/pivot) —
//! `cargo run -p benilla-m2 --example dump_attach <file.m2>`. Extract the file first via
//! `mpqx` (benilla-mpq). Built for the nocked-ammo investigation (decision 2273 follow-up), kept
//! because "which attach ids does this model actually have" keeps coming up.
//! Dump an M2's attachments and event markers with their bone's parent:
//! `cargo run -p benilla-m2 --example dump_attach <file.m2>`, extracted first with `mpqx`
//! (benilla-formats).
use std::io::Cursor;
crates/benilla-m2/examples/dump_skin.rs
+3 -4
View File
@@ -1,7 +1,6 @@
//! Session probe: dump an M2's skin-0 sections/batches with material + texture resolution —
//! `cargo run -p benilla-m2 --example dump_skin <file.m2>`. Extract the file first via `mpqx`
//! (benilla-mpq). Built for the bowstring round (decision 2273 follow-up), kept because "why is
//! this submesh (not) rendering" keeps coming up.
//! Dump an M2's textures, materials and skin-0 sections and batches:
//! `cargo run -p benilla-m2 --example dump_skin <file.m2>`, extracted first with `mpqx`
//! (benilla-formats).
use std::io::Cursor;
crates/benilla-m2/examples/dump_verts.rs
+2 -3
View File
@@ -1,6 +1,5 @@
//! Session probe: dump an M2's vertices (position / UV / bone binding) plus bone table —
//! `cargo run -p benilla-m2 --example dump_verts <file.m2>`. Built for the bowstring round:
//! find which triangles form the string and how they're textured/bound.
//! Dump an M2's bones and vertices (position, UV, bone binding):
//! `cargo run -p benilla-m2 --example dump_verts <file.m2>`.
use std::io::Cursor;
crates/benilla-m2/src/lib.rs
+46 -133
View File
@@ -1,27 +1,6 @@
//! An M2 (MD20) model reader for **WoW 1.12.1 (build 5875)** — in-repo, replacing the `wow-m2` fork
//! (decision 0021), scoped to the **render path** the renderer consumes.
//!
//! Vanilla `.m2` files are MD20 version **256/257**: a fixed header of `M2Array`s (each a `count` +
//! file `offset`), then the referenced arrays, with the **skin profile(s) embedded** in the same file.
//! We read only what the mesh builder needs — vertices, textures, materials (render flags), bones
//! (pivot + flags), the texture-lookup table, and skin profile 0 (indices / triangles / submeshes /
//! batches) — and deliberately **skip the cosmetic chunks** (particle emitters, lights, texture
//! animations). That sidesteps exactly what the `wow-m2` fork patched: those chunks can be malformed
//! in Classic art and abort a full parse; benilla-formats parses lights/particles itself from raw
//! bytes, so we never touch them here.
//!
//! Proven against `wow-m2` over real models during the decision-0021 migration (oracle test in git
//! history); the creature/GameObject mesh + cosmetic-chunk-model golden tests in benilla-formats pin
//! it end-to-end on every run.
//!
//! Byte access goes through `benilla-bytes` (decision 0064): every read is bounds-checked and a
//! truncated array/record is a typed [`Error::Truncated`], never a panic; header-driven `Vec`
//! reservations are capped by what the input could actually hold, so a corrupt count can no longer
//! OOM-abort the process before the first bounds-checked read ever runs.
//!
//! Split by concern: this file keeps the public re-export surface and the top-level [`parse_m2`]
//! entry + MD20 header walk; `model` holds the plain parsed-record types, `track` the M2Track
//! scalar/vector readers, `skin` the embedded skin-profile decoder, and `error` the parse error type.
//! An M2 (MD20) model reader for WoW 1.12.1. Vanilla models are MD20 version 256 or 257: a header
//! of `M2Array`s (`count`, file `offset`), the arrays they name, and the embedded skin profiles.
//! Particle emitters and lights are not read here; benilla-formats reads them from the raw bytes.
mod error;
mod model;
@@ -48,8 +27,7 @@ use benilla_bytes::{capped, ByteExt};
use error::Result;
use track::{track_fix16, track_quat, track_spline_f32, track_spline_vec3, track_vec3_timed};
/// Read a `C3Vector` (3×f32) at `o`, bounds-checked. `None` iff any of the three reads would run
/// past the end — callers map that to their own [`Error::Truncated`].
/// A `C3Vector` (3×f32) at `o`.
fn rd_c3(b: &[u8], o: usize) -> Option<C3> {
Some(C3 {
x: b.f32_at(o)?,
@@ -58,7 +36,7 @@ fn rd_c3(b: &[u8], o: usize) -> Option<C3> {
})
}
/// Parse an MD20 model (v256/257). `cursor` position is ignored; the file starts at the header.
/// Parse an MD20 model; the cursor's position is ignored.
pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
let b: &[u8] = cursor.get_ref();
if b.len() < 8 || &b[0..4] != b"MD20" {
@@ -69,8 +47,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
return Err(Error::UnsupportedVersion(version));
}
// Walk the header reading the M2Array `(count, offset)` pairs we need (8 bytes each), applying the
// pre-Wrath version conditionals. Layout order per wowdev ADT/M2 + the reference.
// Walk the header's `(count, offset)` M2Arrays, 8 bytes each, pre-Wrath layout.
let arr = |p: usize| -> Result<(u32, u32)> {
Ok((
b.u32_at(p).ok_or(Error::Truncated)?,
@@ -85,15 +62,10 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
p += 8;
let _anim = arr(p)?;
p += 8;
// AnimationLookup (header `+0x24`/`+0x28`): `AnimationData.dbc` id → this model's first sequence
// slot for it, `0xffff` where the model doesn't author it. The reference's "does this model own
// animation id X" predicate (`0x711960`) is exactly a bounds-checked read of this table, so it is
// parsed rather than inferred from the sequence list — see [`M2Model::owns_animation`].
// AnimationLookup (`+0x24`), which the reference's ownership test `0x711960` reads.
let animation_lookup_arr = arr(p)?;
p += 8;
// PlayableAnimationLookup (decision 0082, header `+0x2c`/`+0x30`, pre-Wrath only — dropped past
// version 263, which `parse_m2`'s own top guard already excludes, so this is unconditional on any
// version that reaches here; the `if` stays for documentation).
// PlayableAnimationLookup (`+0x2c`): pre-Wrath only, as is every version admitted above.
let playable_animation_lookup_arr = if (256..=263).contains(&version) {
let v = arr(p)?;
p += 8;
@@ -134,23 +106,16 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
p += 8;
let texture_lookup_table = arr(p)?;
p += 8;
// header 0x9c: texture_unit_lookup — the **env-vs-UV mapping table**. `texUnit.texture_coord_combo_index`
// (+0x12) indexes it; a value `< 3` names a UV channel, `>= 3` (real art: `0xffff`) is a
// *generated* environment coordinate (`0x70b8bd`).
//
// The three lookup slots here follow the reference's header walk `0x71cdf0` (renderFlags pinned
// at 0x84 ⇒ texLookup 0x94 · texUnitLookup 0x9c · transLookup 0xa4 · texAnimLookup 0xac), NOT
// one slot early. Byte-proof on real art:
// ElwynnTallWaterfall01 (2 transforms, batch combos 0/1) has [0,1] at 0xac, and
// StormwindMagePortal01 (4 weight tracks) has the identity [0,1,2,3] at 0xa4 with [0] at 0x9c
// — reading 0x9c as the transparency lookup silently dropped the portal's combo-1..3 tracks.
// header 0x9c: texture_unit_lookup, a stage's UV channel or environment coordinate. These
// three lookups sit where the reference's header walk `0x71cdf0` puts them, not a slot
// earlier: StormwindMagePortal01 has [0] at 0x9c and its weight lookup [0,1,2,3] at 0xa4.
let texture_unit_lookup_arr = arr(p)?;
p += 8;
// header 0xa4: transparency_lookup (u16) — `weight_combo_index` indexes it to reach a weight track.
// header 0xa4: transparency_lookup (u16), `weight_combo_index` to a weight track.
let transparency_lookup_arr = arr(p)?;
p += 8;
// header 0xac: texture_animation_lookup (u16) — `texture_transform_combo_index` (texUnit
// +0x16) indexes it to reach a texture transform (`0x70b897`); 0xffff = none.
// header 0xac: texture_animation_lookup (u16), `texture_transform_combo_index` to a texture
// transform (`0x70b897`); 0xffff is none.
let tex_anim_lookup_arr = arr(p)?;
p += 8;
// Authored bounds: AABB (min C3, max C3) then sphere radius.
@@ -167,7 +132,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
p += 12;
let bounding_sphere_radius = b.f32_at(p).ok_or(Error::Truncated)?;
p += 4;
// The collision box (min C3 + max C3) + collision sphere radius — the tight collision hull (28 bytes).
// The collision box and sphere radius: the tight hull.
let collision_box_min = read_box(p)?;
p += 12;
let collision_box_max = read_box(p)?;
@@ -178,18 +143,15 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
p += 8;
let bounding_vertices = arr(p)?;
p += 8;
// Continue the header to the attachments block: boundingNormals, then attachments + attachLookup.
let _bounding_normals = arr(p)?;
p += 8;
let attachments = arr(p)?;
p += 8;
let attach_lookup = arr(p)?;
p += 8;
// header 0x114: the animation-event table (the `$CSL`/`$BWR`/`$SND`… records) — the positional
// markers are kept below; the per-sequence fire keys are `benilla-formats`' concern.
// header 0x114: the animation events (`$CSL`, `$BWR`, `$SND`…); only their positions are read.
let events = arr(p)?;
// The follow-camera pivot height source (`0x50cbc0`): attachment **id 17**'s
// model-space Z. `None` if the model has no slot-17 attachment (caller falls back to the vertex box).
// The follow camera's pivot height (`0x50cbc0`): attachment id 17's model-space Z.
let pivot_attach_z = attachment_z(b, 17, attachments, attach_lookup);
// --- read the arrays we keep ---
@@ -197,11 +159,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
b.bytes_at(start, len).ok_or(Error::Truncated)
};
// Vertices: 48 bytes each. The reservation is capped by what the file could actually hold
// (decision 0064, `benilla_bytes::capped`): a corrupt/hostile `vertices.0` count can then at
// worst reserve the input's own size, never OOM-abort the process before the loop reaches its
// first bounds-checked read. The loop below still walks the *real* declared count — a short
// file fails cleanly at `get(..)?` instead.
// Vertices: 48 bytes each. The reservation is capped by the file; a short file fails at `get`.
let vert_avail = b.len().saturating_sub(vertices.1 as usize);
let mut verts = Vec::with_capacity(capped(vertices.0 as usize, 48, vert_avail));
for i in 0..vertices.0 as usize {
@@ -224,13 +182,8 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
for i in 0..textures.0 as usize {
let t = get(textures.1 as usize + i * 16, 16)?;
let ttype = M2TextureType::from_u32(t.u32_at(0).ok_or(Error::Truncated)?);
// `+0x04` — the ADDRESS MODE, and it decides silhouettes, not just tiling. Bit 0 = repeat U,
// bit 1 = repeat V; clear = clamp to edge. Content authors a cutout card's UVs *outside*
// `0..1` deliberately, so the border clamps to the texture's transparent edge and the card
// fades out to nothing — sampled with repeat instead, those margins wrap around into the
// opaque middle of the sheet and draw as solid geometry, with a hard seam where u crosses
// the wrap. This field was documented in this very comment and read by nobody until B52
// (the Dun Morogh snow-firs) was traced to it; decision 0763.
// `+0x04`, the address mode: bit 0 repeats U, bit 1 V, clear clamps to edge. Cutout cards
// put UVs outside `0..1` to clamp to their transparent edge; repeated, they draw solid.
let tflags = t.u32_at(4).ok_or(Error::Truncated)?;
let (fcount, fofs) = (
t.u32_at(8).ok_or(Error::Truncated)? as usize,
@@ -258,9 +211,8 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
});
}
// Bones: 108 bytes each in v256 — header 12 (id i32, flags u32, parent i16, submesh u16; vanilla
// has NO boneNameCRC field, unlike TBC+), then 3×28 pre-Wrath tracks, then pivot C3. flags @4,
// pivot @96. The reference zeroes NaN pivot components (a corruption guard); mirror it.
// Bones: 108 bytes in v256: key bone i32, flags u32, parent i16, submesh u16 (no boneNameCRC in
// vanilla), three 28-byte tracks, pivot C3 @96. The reference zeroes a NaN pivot component.
let bones_avail = b.len().saturating_sub(bones.1 as usize);
let mut bone_list = Vec::with_capacity(capped(bones.0 as usize, 108, bones_avail));
for i in 0..bones.0 as usize {
@@ -283,22 +235,16 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
});
}
// Attachments: 48 bytes each (id u32 @0, bone u32 @4, position C3 @8; a 28-byte visibility
// M2Track follows and is skipped — see [`M2Attachment`]). `id`/`bone` are narrowed to `u16`; a
// record whose id/bone doesn't fit, or whose bone indexes past `bone_list` (now fully parsed
// above), is dropped rather than kept malformed — real attachment ids run ≤ 36, so this never
// drops legitimate data.
// Attachments: 48 bytes (id, bone, position C3 @8, a skipped track). A record whose id or bone
// overflows `u16`, or whose bone is out of range, is dropped; real ids stop at 36.
let att_count = attachments.0 as usize;
let att_avail = b.len().saturating_sub(attachments.1 as usize);
// The whole table must fit the file before it is walked — the loop's `get(..)?` reaches that
// verdict anyway, but the per-record index vector below is sized from the count, and a raw
// u32::MAX there was an 8 GiB non-zero fill before the first bounds-checked read.
// The table must fit the file first: `emitted_at` below is sized from the raw count.
if capped(att_count, 48, att_avail) < att_count {
return Err(Error::Truncated);
}
let mut attachment_list = Vec::with_capacity(att_count);
// A dropped record shifts the emitted indices, and the AttachLookup below indexes the FILE's
// records — so the translation is carried here rather than reconstructed later.
// AttachLookup indexes the file's records; a dropped one shifts ours, so map file to emitted.
let mut emitted_at = vec![0xffffu16; att_count]; // proven to fit above
for (i, emitted) in emitted_at.iter_mut().enumerate() {
let a = get(attachments.1 as usize + i * 48, 48)?;
@@ -319,12 +265,9 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
});
}
// AttachLookup (header `+0x10c`): attachment **id** → the one record it resolves to. This is
// the reference's only id→record map (`0x710310`: `id >= count` ⇒ the `0xffff` sentinel, else
// `lookup[id]`) and it is emphatically NOT a scan of the table — shipped weapons author
// several records under one id (`Stave_2H_Long_D_05`: eight records, ids 0-3 twice, one at
// each end of the staff) and the lookup names exactly one, leaving the other unreachable.
// Stored translated into emitted-list indices, `0xffff` where the id resolves to nothing.
// AttachLookup (`+0x10c`): attachment id to its one record, the reference's only id map
// (`0x710310`). Shipped weapons repeat ids (`Stave_2H_Long_D_05`: 0-3 twice) and the lookup
// names one. Stored as emitted indices, `0xffff` for none.
let lk_avail = b.len().saturating_sub(attach_lookup.1 as usize);
let mut attach_lookup_list = Vec::with_capacity(capped(attach_lookup.0 as usize, 2, lk_avail));
for i in 0..attach_lookup.0 as usize {
@@ -334,9 +277,8 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
attach_lookup_list.push(emitted_at.get(raw as usize).copied().unwrap_or(0xffff));
}
// Event markers: 44 bytes each (identifier 4CC @0, data u32 @4, bone u32 @8, position C3 @12;
// a 20-byte enabled M2TrackBase follows and is skipped). Same bone-range discipline as the
// attachments above; kept in file order (queries take the first ident match).
// Event markers: 44 bytes (4CC, data u32, bone u32 @8, position C3 @12, a skipped 20-byte
// track), dropped like attachments, kept in file order.
let ev_avail = b.len().saturating_sub(events.1 as usize);
let mut event_markers = Vec::with_capacity(capped(events.0 as usize, 44, ev_avail));
for i in 0..events.0 as usize {
@@ -357,7 +299,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
});
}
// AnimationLookup: one u16 each — the id → sequence-slot table `owns_animation` reads.
// AnimationLookup: one u16 each.
let al_avail = b.len().saturating_sub(animation_lookup_arr.1 as usize);
let mut animation_lookup =
Vec::with_capacity(capped(animation_lookup_arr.0 as usize, 2, al_avail));
@@ -369,7 +311,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
);
}
// PlayableAnimationLookup (decision 0082): one dword each, low16 resolved id / high16 dir flags.
// PlayableAnimationLookup: one dword each, low16 resolved id / high16 dir flags.
let pal_avail = b
.len()
.saturating_sub(playable_animation_lookup_arr.1 as usize);
@@ -399,15 +341,9 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
);
}
// Each M2Color (stride 0x38) is an RGB **colour** track @ +0x00 (C3Vector keys) + an **alpha**
// track @ +0x1c (fix16); each M2TextureWeight (stride 0x1c) is a single **weight** track. The
// alpha/weight values gate batch visibility (a constant 0 hides the batch — the zero-alpha
// cull `0x707b3a`–`0x707b5c`); the RGB is the per-batch tint multiplied into the vertex colour.
// `track_fix16`/`track_vec3_timed` bounds-check internally and return an empty key list for
// an out-of-range track (kept exactly — real art relies on that "no keys" tolerance). That
// same tolerance is why the *outer* loops run over the whole records the file holds past the
// block's offset, never the raw count: a reader that cannot fail would otherwise push one
// empty track per claimed record, and a corrupt count claims billions.
// M2Color (stride 0x38): an RGB track @ +0x00 and a fix16 alpha track @ +0x1c. M2TextureWeight
// (stride 0x1c): one weight track. An out-of-range track reads as no keys, so these loops run
// over the records the file holds, never the raw count.
let colors_avail = b.len().saturating_sub(colors_arr.1 as usize);
let colors_cap = capped(colors_arr.0 as usize, 0x38, colors_avail);
let mut color_alpha_tracks = Vec::with_capacity(colors_cap);
@@ -446,10 +382,8 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
);
}
// Each M2TextureTransform (header 0x74, stride 0x54) is 3 back-to-back M2Tracks — translation
// (C3Vector) @+0x00, rotation (quaternion) @+0x1c, scaling (C3Vector) @+0x38 (the M2 load
// `0x70ebd0`). Same "no keys" tolerance as the colour/weight tracks above, and the same
// file-bounded loop for the same reason.
// M2TextureTransform (stride 0x54): translation @+0x00, rotation @+0x1c, scaling @+0x38
// (`0x70ebd0`), bounded by the file like the colour tracks.
let ttf_avail = b.len().saturating_sub(tex_anim_arr.1 as usize);
let ttf_cap = capped(tex_anim_arr.0 as usize, 0x54, ttf_avail);
let mut texture_transforms = Vec::with_capacity(ttf_cap);
@@ -472,9 +406,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
);
}
// Global-sequence durations (header `0x14` count / `0x18` offset, u32 ms each): the free-running
// loop clocks a `gseq`-tagged track wraps on (`t mod global_sequences[gseq]`). Bounds-tolerant
// like the tracks: a truncated table yields what fits.
// Global-sequence durations (`0x14`, u32 ms): the clocks `gseq` tracks wrap on.
let gseq_arr = (
b.u32_at(0x14).ok_or(Error::Truncated)?,
b.u32_at(0x18).ok_or(Error::Truncated)?,
@@ -488,9 +420,7 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
global_sequences.push(ms);
}
// Collision hull: raw bytes (triangles = count×2 u16, vertices = count×12 C3Vector). Many props
// carry none (count 0) → empty. `get(..)` is already bounds-checked before the `to_vec()`, so a
// corrupt count fails at the checked slice, never at an unbounded allocation.
// The collision hull, raw: triangle indices (u16) and vertices (C3Vector).
let bt_bytes = get(
bounding_triangles.1 as usize,
bounding_triangles.0 as usize * 2,
@@ -543,17 +473,8 @@ pub fn parse_m2(cursor: &mut Cursor<&[u8]>) -> Result<M2Format> {
})
}
/// Parse the MD20 **camera** array — header `count@0x124` / `offset@0x128`, stride `0x7c` (see
/// [`M2Camera`] for the byte-verified field map).
///
/// **The one reader for those offsets.** [`parse_m2`] fills [`M2Model::cameras`] with it, and
/// callers that want a camera *without* paying for a whole model parse (the portrait/pane framing
/// on the asset-load path, the cinematic fly-bys) call it directly — so the record layout is
/// written down once. The header offsets are absolute rather than walked because every version
/// [`parse_m2`] accepts (256–263) carries the identical array order up to this point.
///
/// Bounds-tolerant throughout: a truncated table yields the records that fit, and a model with no
/// camera array yields an empty `Vec` (the overwhelmingly common case).
/// Parse the MD20 camera array (header `0x124`, stride `0x7c`) alone; the offset is absolute, as
/// versions 256–263 agree up to it. A truncated table yields the records that fit.
pub fn parse_cameras(b: &[u8]) -> Vec<M2Camera> {
let (Some(count), Some(ofs)) = (b.u32_at(0x124), b.u32_at(0x128)) else {
return Vec::new();
@@ -562,8 +483,7 @@ pub fn parse_cameras(b: &[u8]) -> Vec<M2Camera> {
let mut out = Vec::with_capacity(capped(count as usize, 0x7c, avail));
for i in 0..count as usize {
let rec = ofs as usize + i * 0x7c;
// A record is read whole or not at all — a file whose camera array is cut short yields the
// records that fit, never a half-read one with default tracks standing in for the tail.
// A record is read whole or not at all.
if rec.checked_add(0x7c).is_none_or(|end| end > b.len()) {
break;
}
@@ -575,8 +495,6 @@ pub fn parse_cameras(b: &[u8]) -> Vec<M2Camera> {
) else {
break;
};
// The bases are plain `[f32; 3]` (not `C3`) so they add to their track's sampled value
// componentwise, the way the reference's publish pass composes them.
let (Some(position_base), Some(target_base)) = (
rd_c3(b, rec + 0x2c).map(|c| [c.x, c.y, c.z]),
rd_c3(b, rec + 0x54).map(|c| [c.x, c.y, c.z]),
@@ -598,8 +516,7 @@ pub fn parse_cameras(b: &[u8]) -> Vec<M2Camera> {
out
}
/// Parse the MD20 **CameraLookup** table — header `count@0x12c` / `offset@0x130`, `u16` entries.
/// See [`M2Model::camera_lookup`] for what it selects.
/// Parse the MD20 CameraLookup (header `0x12c`, `u16` each); see [`M2Model::camera_lookup`].
pub fn parse_camera_lookup(b: &[u8]) -> Vec<u16> {
let (Some(count), Some(ofs)) = (b.u32_at(0x12c), b.u32_at(0x130)) else {
return Vec::new();
@@ -615,11 +532,7 @@ pub fn parse_camera_lookup(b: &[u8]) -> Vec<u16> {
out
}
/// The model-space **Z** of an M2 attachment selected by **attachment id** (not array index) — the
/// follow-camera pivot reads id 17 this way. `attachments`/`lookup` are the header `(count, offset)`
/// pairs; the id → array-index map (`lookup`, `u16` each) picks the record, whose `position` C3 sits at
/// `+8` (vanilla stride 48: `id u32 · bone u32 · position C3 · animate M2Track`). `None` when the id has
/// no lookup slot or the record/bytes are out of range.
/// Attachment `id`'s model-space Z through the lookup, from the file's `(count, offset)` pairs.
fn attachment_z(b: &[u8], id: usize, attachments: (u32, u32), lookup: (u32, u32)) -> Option<f32> {
if id >= lookup.0 as usize {
return None;
@@ -630,7 +543,7 @@ fn attachment_z(b: &[u8], id: usize, attachments: (u32, u32), lookup: (u32, u32)
return None;
}
let rec = attachments.1 as usize + idx * 48;
// position C3 at rec+8 → its Z is rec+16.
// `position` is at +8 of the 48-byte record, so its Z is at +16.
b.f32_at(rec + 16)
}
crates/benilla-m2/src/model.rs
+74 -189
View File
@@ -1,6 +1,4 @@
//! The plain M2 data types: the parsed record shapes (vertices, textures, materials, bones,
//! attachments, the authored bounds) and the top-level [`M2Model`]/[`M2Format`] the parser
//! (`crate::parse_m2`, in `lib.rs`) builds.
//! The parsed M2 record types and the [`M2Model`] that `crate::parse_m2` builds.
use std::ffi::CString;
@@ -22,8 +20,7 @@ pub struct C2 {
pub y: f32,
}
/// M2 texture purpose. Vanilla uses `Hardcoded` (the `filename` is the real path) or the `Monster*`
/// skin slots (filename usually empty; the slot is filled from the creature's display variation).
/// M2 texture type: `Hardcoded` names its file; `Monster*` slots take the creature display's skins.
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum M2TextureType {
Hardcoded,
@@ -45,8 +42,7 @@ impl M2TextureType {
}
}
/// The texture's filename, kept as a `CString` so callers can `.string.to_string_lossy()` (matching
/// the reference's shape).
/// A texture's filename.
pub struct M2ArrayString {
pub string: CString,
}
@@ -54,7 +50,7 @@ pub struct M2ArrayString {
/// One M2 texture definition.
pub struct M2Texture {
pub texture_type: M2TextureType,
/// Address mode on U (`flags & 0x1`): set = the sampler REPEATS, clear = it CLAMPS to edge.
/// Address mode on U (`flags & 0x1`): set repeats, clear clamps to edge.
pub wrap_x: bool,
/// Address mode on V (`flags & 0x2`).
pub wrap_y: bool,
@@ -93,17 +89,12 @@ impl M2BoneFlags {
}
}
/// One M2 bone. The render path reads billboard flags + pivot; the skeletal-animation path also needs
/// the **parent index** (`-1` = root) to compose bone matrices up the tree and to build the joint
/// hierarchy + inverse bind poses (decision 0019). Vanilla bone record is stride 108: keyBoneId i32
/// @+0x00, flags @+0x04, parent i16 @+0x08, pivot C3 @+0x60 (no boneNameCRC, unlike TBC+).
/// One M2 bone: keyBoneId i32 @+0x00, flags @+0x04, parent i16 @+0x08, pivot C3 @+0x60.
pub struct M2Bone {
/// The bone's `KeyBoneID` (`-1` = none): the semantic anchor table — 0/1 arms L/R, 2/3
/// shoulders L/R, 4 spine-low, 5 waist, 6 head… The per-arm animation masks resolve their
/// subtree roots through it.
/// `KeyBoneID`, `-1` for none: 0/1 arms L/R, 2/3 shoulders L/R, 4 spine low, 5 waist, 6 head…
pub key_bone: i16,
pub flags: M2BoneFlags,
/// Parent bone index, or `-1` for a root bone (vanilla bone record `+0x08`).
/// Parent bone index, `-1` for a root.
pub parent: i16,
pub pivot: C3,
}
@@ -114,10 +105,8 @@ impl M2Bone {
}
}
/// One M2 vertex (48 bytes on disk). Layout: position C3 @0, bone_weights [u8;4] @0x0c,
/// bone_indices [u8;4] @0x10, normal C3 @0x14, tex_coords C2 @0x20 (a second UV @0x28 we skip). The
/// weights + indices are the 4-bone skin binding the skeletal-animation path uploads (decision 0019);
/// the static render path uses only `bone_indices[0]` (billboard glow-card pivot).
/// One M2 vertex, 48 bytes: position @0, weights @0x0c, indices @0x10, normal @0x14, UV @0x20,
/// and an unread second UV @0x28.
pub struct M2Vertex {
pub position: C3,
/// Per-vertex bone influence weights (`/255`), paired with [`Self::bone_indices`].
@@ -127,10 +116,7 @@ pub struct M2Vertex {
pub tex_coords: C2,
}
/// Lookup tables + collision geometry read from the header. The renderer uses the texture-combo →
/// texture mapping; benilla-formats reads the collision hull from the two `bounding_*` byte blocks
/// (the MD20 `boundingVertices` are `C3Vector`s = 12 bytes; `boundingTriangles` are `u16` = 2 bytes —
/// kept as raw bytes so callers slice them as they need).
/// The texture lookup table and the collision hull's raw bytes.
pub struct M2RawData {
pub texture_lookup_table: Vec<u16>,
/// Collision-hull triangle indices, raw bytes (`count × 2`, `u16` each).
@@ -139,12 +125,8 @@ pub struct M2RawData {
pub bounding_vertices: Vec<u8>,
}
/// The authored bounds from the MD20 header. Two boxes: the **bounding** box (+ sphere radius) is the
/// render/cull volume (the all-animation vertex extent — large); the **collision** box (+ sphere radius)
/// is the tight collision hull (~the body). The **render sphere** sizes the *unit* selection ring
/// (`0.5 × radius × scale`, the client's cached `[unit+0x2b0]`) and drives render culling; the
/// **collision box** is the *GameObject* ring's source (`0x711a20`/`0x713700`) — parsed here for
/// that path.
/// The MD20 header's authored bounds: the bounding box and sphere are the render and cull volume
/// over every animation, the collision box and sphere the tight hull.
pub struct M2Header {
pub bounding_box_min: [f32; 3],
pub bounding_box_max: [f32; 3],
@@ -154,103 +136,68 @@ pub struct M2Header {
pub collision_sphere_radius: f32,
}
/// One row of the M2's baked **PlayableAnimationLookup** table (decision 0082, `0x711bf0`): the
/// model's own precomputed answer to "if the game requests `AnimationData.dbc` id X, which id do I
/// actually play, and in which direction/variant". Header `count@+0x2c`/`offset@+0x30` (pre-Wrath
/// M2 only — the array is dropped past version 263), stride 4 (one dword per row): low16 = the
/// resolved id, high16 = a direction/variant playback code. `nPlayableAnimationLookup` is a fixed
/// **203** across the entire retail 1.12.1 corpus (measured) — the per-model table is a baked cache
/// of the `AnimationData.dbc` Fallback-column walk (PATH 2, below), frozen against that model's
/// actual sequence set at export (a chicken lacking Attack2H bakes `[18] = [17] = 16`).
/// A PlayableAnimationLookup row (`0x711bf0`, header `+0x2c`): the id this model plays for a
/// requested `AnimationData.dbc` id, and a direction code. Every 1.12.1 model has 203 rows.
#[derive(Clone, Copy)]
pub struct M2PlayableAnim {
/// The `AnimationData.dbc` id this model actually plays for the row's requested id.
/// The `AnimationData.dbc` id this model plays for the row's requested id.
pub resolved_id: u16,
/// Direction/variant playback code — **not yet consumed** by benilla (decision 0082 flags its
/// exact playback semantics, `~0x7126d2`, as untraced; carried here for a future slice).
/// The direction or variant playback code (near `0x7126d2`); parsed, not applied: the
/// reference's use of it is untraced.
pub dir_flags: u16,
}
/// One M2 attachment-point record (vanilla stride 48: `id u32 @+0`, `bone u32 @+4`, `position C3 @+8`,
/// then a 28-byte visibility `M2Track` we skip). `id`/`bone` are narrowed to `u16` — safe on real data
/// (attachment ids run ≤ 36; `bone` is checked against the bone array during parse) — and any record
/// that doesn't fit, or whose bone is out of the model's bone-array range, is dropped rather than kept
/// malformed.
///
/// Empirical ground truth (direct dump of `HumanMale.m2`, `decisions/0072`): on character models the
/// attachment `position` equals its bone's pivot — the attach bones are leaves sitting exactly at the
/// attach point, so `position` and the bone pivot agree. Known ids: `0` shield (left forearm), `1`
/// right hand, `2` left hand, `26`/`27` right/left back sheath, `28` shield back, `30`/`31` lower-back
/// pair, `32`/`33` hip pair.
/// One attachment point: a 48-byte vanilla record, `id` and `bone` narrowed to `u16`. On character
/// models `position` equals the attach bone's pivot. Ids: 0 shield (left forearm), 1 right hand,
/// 2 left hand, 26/27 right/left back sheath, 28 shield back, 30/31 lower back, 32/33 hips.
#[derive(Clone, Copy)]
pub struct M2Attachment {
pub id: u16,
pub bone: u16,
/// Model-space position, **raw WoW space** — matches how [`M2Bone::pivot`] is stored; the WoW→Bevy
/// bake happens at the render boundary (`benilla-assets`).
/// Model-space position, raw WoW axes like [`M2Bone::pivot`].
pub position: [f32; 3],
}
/// One M2 **animation-event marker** (vanilla stride 44: `identifier 4CC @+0`, `data u32 @+4`,
/// `bone u32 @+8`, `position C3 @+12`, then a 20-byte enabled `M2TrackBase` we skip — its
/// per-sequence fire keys surface through `benilla-formats`' per-sequence event stream instead).
/// This is the record's **positional half**: the client's `0x7130e0`/`0x7131b0` scan exactly this
/// table by 4CC and transform `position` by `bone`'s live matrix — the cast-release launch points
/// `$CSL`/`$CSR`/`$CST` (casting hand left/right/two-hand), the bow anchors `$WTT`/`$WTB`, the
/// ranged release `$BWR`. Idents are NOT unique per model (character models carry six `$CSD`
/// records, one per voice payload); the client's queries take the first match. A record whose
/// `bone` doesn't fit `u16` or indexes past the bone array is dropped rather than kept malformed.
/// The positional half of an animation-event record: the reference (`0x7130e0`, `0x7131b0`) finds
/// one by 4CC, first match, and moves `position` by `bone`'s matrix. Cast launch points
/// `$CSL`/`$CSR`/`$CST`, bow anchors `$WTT`/`$WTB`, ranged release `$BWR`.
#[derive(Clone, Copy)]
pub struct M2EventMarker {
/// The identifier 4CC, stored forward (`*b"$CSL"`).
pub ident: [u8; 4],
pub bone: u16,
/// Model-space position, **raw WoW space** — same convention as [`M2Attachment::position`].
/// Model-space position, raw WoW axes.
pub position: [f32; 3],
}
/// One M2 **texture transform** (header `0x74`, file stride `0x54`): 3 back-to-back M2Tracks —
/// `translation` (C3Vector) @`+0x00`, `rotation` (quaternion) @`+0x1c`, `scaling` (C3Vector)
/// @`+0x38` (the M2 load `0x70ebd0`, runtime `[model+0x98]`). Sampled per frame these
/// drive UV animation — flowing waterfalls, scrolling energy fields. How the sampled TRS is
/// *applied* at draw (texture matrix vs CPU UVs, pivot/composition convention) is open; parsed
/// and carried here, not yet consumed.
/// One M2 texture transform (header `0x74`, stride `0x54`, loaded at `0x70ebd0`), its UV-animation
/// tracks at `+0x00`, `+0x1c`, `+0x38`; this crate only carries the parsed tracks.
pub struct M2TextureTransform {
pub translation: M2Vec3Track,
pub rotation: M2QuatTrack,
pub scaling: M2Vec3Track,
}
/// One M2 **camera** record (header `cameras` @`0x124`, file stride `0x7c`) — the authored
/// eye/target path a `Cameras\*.m2` cinematic fly-by *is*, and the same record the portrait bake
/// and the `<Model>` panes frame themselves with.
///
/// Field map from the reference's M2 load `0x70ebd0` + the publish pass `0x718960`:
/// `type@+0x00 · fov@+0x04 · farClip@+0x08 · nearClip@+0x0c · positions@+0x10 ·
/// position_base@+0x2c · target@+0x38 · target_position_base@+0x54 · roll@+0x60`. The load copies
/// only fov/far/near into the runtime `CCamera`; the tracks are evaluated by the animate kernel and
/// **published against the bases** — `eye = position_base + positions(t)`,
/// `target = target_position_base + target(t)`, `roll = roll(t)` — so a keyless track leaves the
/// base as the value. Character and creature models author a single `(0,0,0)` key on all three
/// (their pane camera is a frozen constant); the fly-bys are the case the tracks exist for.
/// One M2 camera record (header `0x124`, stride `0x7c`, loaded at `0x70ebd0`): a `Cameras\*.m2`
/// fly-by's path, and the framing of the portrait bake and `<Model>` panes. The publish pass
/// `0x718960` adds each track to its base (`eye = position_base + positions(t)`), so a keyless
/// track leaves the base.
#[derive(Clone)]
pub struct M2Camera {
/// The record's `type` word. `0` = portrait, `1` = "characterinfo" (the `<PlayerModel>` pane's
/// raw index 1); every shipped `Cameras\*.m2` fly-by authors `-1`.
/// The `type` word (`+0x00`): 0 portrait, 1 the `<PlayerModel>` pane, `-1` on every fly-by.
pub camera_type: i32,
/// **Diagonal** field of view, radians — the reference's camera build `0x7ac640` takes the
/// half-angle as `θ = (fov/2)/√(aspect²+1)`, *not* `fov/2`. All ten fly-bys author `0.7854`
/// (45°).
/// Diagonal field of view, radians (`+0x04`): `0x7ac640` takes the half-angle as
/// `(fov/2)/√(aspect²+1)`, not `fov/2`.
pub fov: f32,
/// Far clip as authored (`+0x08`). Every fly-by carries `27.777779` (= 1000/36).
/// Far clip (`+0x08`); every fly-by carries `27.777779` (1000/36).
pub far_clip: f32,
/// Near clip as authored (`+0x0c`). Every fly-by carries `0.22222222` (= 8/36).
/// Near clip (`+0x0c`); every fly-by carries `0.22222222` (8/36).
pub near_clip: f32,
/// The eye path, relative to [`Self::position_base`].
/// The eye path (`+0x10`), relative to [`Self::position_base`].
pub positions: M2Vec3SplineTrack,
/// The eye's base position, model space (`+0x2c`).
pub position_base: [f32; 3],
/// The look-at path, relative to [`Self::target_base`].
/// The look-at path (`+0x38`), relative to [`Self::target_base`].
pub target: M2Vec3SplineTrack,
/// The look-at base position, model space (`+0x54`).
pub target_base: [f32; 3],
@@ -258,142 +205,80 @@ pub struct M2Camera {
pub roll: M2ScalarSplineTrack,
}
/// The parsed model (render subset).
/// The parsed model.
pub struct M2Model {
pub vertices: Vec<M2Vertex>,
pub textures: Vec<M2Texture>,
pub materials: Vec<M2Material>,
/// Per-`M2Color` **alpha** track (full [`M2ScalarTrack`]: interp + gseq tag + timed keys), one
/// per color record (header `0x54`). A `texUnit.colorIndex` (direct) selects one; no keys ⇒ the
/// factor doesn't apply. The alpha governs batch visibility (a constant 0 hides the batch) and,
/// time-varying, the animated material combine (`0x707680`).
/// Each `M2Color`'s alpha track (header `0x54`), picked directly by a batch's `colorIndex`; no
/// keys means no factor. It gates visibility and feeds the material combine (`0x707680`).
pub color_alpha_tracks: Vec<M2ScalarTrack>,
/// Per-`M2Color` **RGB colour** track (full [`M2Vec3Track`]: interp + gseq tag + timed
/// `C3Vector` keys, the colour record's `+0x00` track), one per record. Selected by the same
/// `texUnit.colorIndex`. This is the per-batch tint the real client multiplies into the vertex
/// colour — load-bearing for additive glow cards drawn with a *neutral* glow texture (e.g.
/// `GenericGlow_Alpha_128`), whose warmth lives entirely here; dropping it renders the glow
/// grey/white. Time-varying tracks (e.g. a spell effect's white-hot flash cooling to red) carry
/// their timestamps so the render side can animate the tint. No keys ⇒ no tint.
/// Each `M2Color`'s RGB track: the batch tint multiplied into the vertex colour, none without
/// keys. Neutral glow cards (`GenericGlow_Alpha_128`) take all their colour from it.
pub color_rgb_tracks: Vec<M2Vec3Track>,
/// Per-`M2TextureWeight` **weight** track (full [`M2ScalarTrack`]), one per record (header
/// `0x64`). Selected via `texUnit.weight_combo_index → transparency_lookup → here` (two-hop).
/// Each `M2TextureWeight`'s weight track (header `0x64`), via [`Self::transparency_lookup`].
pub transparency_tracks: Vec<M2ScalarTrack>,
/// The transparency lookup table (header `0xa4`, u16): `weight_combo_index` indexes this to reach
/// [`Self::transparency_tracks`] (two-hop, `0x707b19`–`0x707b26`).
/// The transparency lookup (header `0xa4`, u16): `weight_combo_index` to
/// [`Self::transparency_tracks`] (`0x707b19`–`0x707b26`).
pub transparency_lookup: Vec<u16>,
/// The **texture-unit lookup** table (header `0x9c`, u16): `texture_coord_combo_index` (texUnit
/// `+0x12`) indexes this to decide where a stage's texture coordinates come from. A value
/// `< 3` names one of the vertex UV channels; **`>= 3` (real art: `0xffff`) is a *generated*
/// environment coordinate** — the model authors no usable UVs for that stage and the runtime
/// derives them from the view-space reflection vector. See
/// [`M2Model::stage_is_env_mapped`].
/// The texture-unit lookup (header `0x9c`), by `texture_coord_combo_index`: a UV channel or a
/// generated environment coordinate (see [`Self::stage_is_env_mapped`]).
pub texture_unit_lookup: Vec<u16>,
/// The texture-transform records (see [`M2TextureTransform`]), selected via
/// `texture_transform_combo_index → texture_transform_lookup → here`.
/// The texture transforms, reached through [`Self::texture_transform_lookup`].
pub texture_transforms: Vec<M2TextureTransform>,
/// The texture-animation lookup table (header `0xac`, u16): `texture_transform_combo_index`
/// (texUnit `+0x16`, `0x70b897`) indexes this to reach
/// [`Self::texture_transforms`]; `0xffff` = no animation.
/// The texture-animation lookup (header `0xac`, u16): `texture_transform_combo_index`
/// (`0x70b897`) to [`Self::texture_transforms`], `0xffff` for none.
pub texture_transform_lookup: Vec<u16>,
/// Global-sequence durations (header `0x14/0x18`, ms): the free-running loop clocks
/// `gseq`-tagged tracks wrap on.
/// Global-sequence durations (header `0x14`, ms): the clocks `gseq` tracks wrap on.
pub global_sequences: Vec<u32>,
pub bones: Vec<M2Bone>,
/// The authored camera records (header `0x124`) — see [`M2Camera`]. Empty for the overwhelming
/// majority of models; a `Cameras\*.m2` fly-by carries exactly one.
/// The camera records (header `0x124`): one in a `Cameras\*.m2` fly-by, none in most models.
pub cameras: Vec<M2Camera>,
/// The **CameraLookup** table (header `0x12c`, u16): a camera *purpose* index → the slot in
/// [`Self::cameras`] that serves it. The portrait bake selects through it (`cameraLookup[0]`,
/// `0x525266`); the `<Model>` pane indexes [`Self::cameras`] raw instead (`0x76cec0`). Carried
/// so a selection can be written the way the reference writes it rather than assumed 1:1.
/// CameraLookup (header `0x12c`): a camera purpose to its [`Self::cameras`] slot. The portrait
/// bake reads it (`0x525266`); the `<Model>` pane indexes cameras directly (`0x76cec0`).
pub camera_lookup: Vec<u16>,
pub raw_data: M2RawData,
pub header: M2Header,
/// Model-space **Z** of attachment id 17 — the reference's follow-camera pivot height
/// (`0x50cbc0`: the camera targets `feet + (attach17.z + 0.0972)·scale`). `None` when the model
/// has no slot-17 attachment (a non-character model → the camera falls back to the box).
/// Model-space Z of attachment id 17, the follow camera's pivot height (`0x50cbc0` targets
/// `feet + (z + 0.0972)·scale`).
pub pivot_attach_z: Option<f32>,
/// The full attachment-point table (see [`M2Attachment`]) — every record whose id/bone fit `u16`
/// and whose bone indexes into [`Self::bones`]. `pivot_attach_z` above is the id-17 special case
/// kept as its own field (the camera pivot's one historical consumer). **Records are addressed
/// by ATTACH ID through [`Self::attachment`], never by scanning this table** (see
/// [`Self::attach_lookup`]).
/// The surviving attachment records; address them by id via [`Self::attachment`], never scan.
pub attachments: Vec<M2Attachment>,
/// The model's **AttachLookup** (header `+0x10c`), translated into [`Self::attachments`]
/// indices: row `i` is the record attachment **id** `i` resolves to, `0xffff` for none. Read
/// it through [`Self::attachment`], which is the reference's `0x710310` — bounds check and
/// sentinel included.
/// AttachLookup (header `+0x10c`) as [`Self::attachments`] indices, `0xffff` for none.
pub attach_lookup: Vec<u16>,
/// The animation-event table's **positional markers** (see [`M2EventMarker`]) — every record
/// whose bone fits `u16` and indexes into [`Self::bones`], in file order (queries take the
/// first ident match, the client's `0x7130e0` scan order). The cast-release launch points
/// (`$CSL`/`$CSR`/`$CST`), the ranged release (`$BWR`), the bow anchors, the sound/impact tags'
/// anchor points.
/// The animation-event markers in file order; a query takes the first ident match (`0x7130e0`).
pub event_markers: Vec<M2EventMarker>,
/// The model's **AnimationLookup** (header `+0x24`/`+0x28`): row `i` is the file slot of this
/// model's first sequence for `AnimationData.dbc` id `i`, or `0xffff` where it authors none.
/// This is the table behind the reference's ownership predicate — see [`M2Model::owns_animation`].
/// Note the table is *short*: a model's `nAnimationLookup` stops just past the highest id it
/// authors, so an id beyond the end reads as the out-of-bounds sentinel, i.e. **not owned**.
/// AnimationLookup (header `+0x24`): row `i` is the first sequence slot for
/// `AnimationData.dbc` id `i`, `0xffff` for none; it ends past the highest authored id.
pub animation_lookup: Vec<u16>,
/// The model's baked **PlayableAnimationLookup** (decision 0082 — see [`M2PlayableAnim`]): row `i`
/// is the model's own precomputed substitute for requested `AnimationData.dbc` id `i`. Empty for a
/// version past 263 (the array is dropped) or a model whose header couldn't be read that far — the
/// resolver degrades to identity in that case.
/// The PlayableAnimationLookup: row `i` is the model's substitute for requested id `i`.
pub playable_animation_lookup: Vec<M2PlayableAnim>,
/// Embedded skin-profile array `(count, file_offset)` — `parse_embedded_skin` reads from here.
/// The embedded skin profiles' `(count, file_offset)`.
pub(crate) views: (u32, u32),
pub(crate) version: u32,
}
impl M2Model {
/// **Are this batch's stage-`stage` texture coordinates GENERATED (environment-mapped) rather
/// than read from the vertex UVs?**
///
/// The reference's gate, at `0x70b8bd`
/// (`cmp word[ [MD20+0xa0][texCoordSet + stage] ], 0x2; jbe`): a [`Self::texture_unit_lookup`]
/// entry `<= 2` names a vertex UV channel, anything higher (real art authors `0xffff`) is a
/// **generated environment coordinate**. Such a stage can never be `textureTransform`-driven,
/// and the shipped vertex program writes it as the sphere-map remap of the view-space
/// reflection vector — `uv = normalize(P − 2(P·N)N).xy · 0.5 + 0.5`, the `(0.5,0,0,0.5 /
/// 0,0.5,0,0.5)` matrix at `0x70b8d0`.
///
/// **An out-of-range index reads as env-mapped**, matching the reference: `0x70b8bd` indexes
/// the table unguarded and falls through to the same `else` branch that writes the env remap.
/// That is also the only safe direction — a model whose art relies on generated coordinates
/// authors *no usable UVs at all* (`GnomeSubwayGlass.m2`: all 330 vertices at exactly
/// `(0,0)`), so mistaking env for UV paints the whole mesh in a single corner texel.
/// Whether `batch`'s stage `stage` takes generated environment coordinates: the reference's
/// gate `0x70b8bd` treats a [`Self::texture_unit_lookup`] entry above 2 (art: `0xffff`), and an
/// out-of-range index, as the sphere map of the view-space reflection (`0x70b8d0`), which no
/// texture transform drives.
pub fn stage_is_env_mapped(&self, batch: &crate::SkinBatch, stage: u16) -> bool {
let idx = batch.texture_coord_combo_index as usize + stage as usize;
self.texture_unit_lookup.get(idx).is_none_or(|&v| v > 2)
}
/// **Does this model author `AnimationData.dbc` id `anim_id`?** — the reference's `0x711960`,
/// byte-for-byte: a bounds-checked read of [`Self::animation_lookup`] (`0x710310` returns the
/// `0xffff` sentinel for an out-of-range index) compared against `0xffff`.
///
/// This is the predicate the GameObject animation arm branches on when the substate's animation
/// id is missing from the model (the four-way remap in `0x5f3930`), so it
/// must answer exactly as the reference does — including for an id past the table's end, which
/// is **not owned** even if some later mechanism could reach a sequence carrying it.
/// Whether the model authors `AnimationData.dbc` id `anim_id`: the reference's `0x711960`, a
/// bounds-checked [`Self::animation_lookup`] read, so an id past the end is not owned.
pub fn owns_animation(&self, anim_id: u16) -> bool {
self.animation_lookup
.get(anim_id as usize)
.is_some_and(|&slot| slot != 0xffff)
}
/// **The attachment record an attach id resolves to** — the reference's `0x710310` +
/// `0x712f70` pair, byte-for-byte: a bounds-checked [`Self::attach_lookup`] read
/// (`id >= count` ⇒ the `0xffff` sentinel) and then the record it names. `None` where the id
/// has no record, which is how the reference behaves too: `0x712f70` stores the sentinel and
/// every consumer of that field (`0x7140aa`, `0x718668`, `0x719266`) tests `cmp 0xffff; je`
/// and skips the child, so an unresolvable id hangs **nothing** rather than falling back to
/// the model origin.
///
/// Never scan [`Self::attachments`] for a matching id instead: on the ~5% of weapon models
/// that author duplicate ids the two disagree, and the lookup is the authority (decision
/// 0805).
/// The record attach id `id` resolves to through [`Self::attach_lookup`] (`0x710310`,
/// `0x712f70`). An unresolved id hangs nothing: its users (`0x7140aa`, `0x718668`, `0x719266`)
/// skip the sentinel. A scan of [`Self::attachments`] disagrees where weapons repeat ids.
pub fn attachment(&self, id: u16) -> Option<&M2Attachment> {
let idx = *self.attach_lookup.get(id as usize)?;
(idx != 0xffff)
@@ -402,7 +287,7 @@ impl M2Model {
}
}
/// Holds the parsed model (mirrors the reference's `parse_m2(..).model()` shape).
/// Holds the parsed model.
pub struct M2Format {
pub(crate) model: M2Model,
}
crates/benilla-m2/src/skin.rs
+18 -34
View File
@@ -1,18 +1,14 @@
//! Embedded skin-profile parsing: [`Skin`]/[`SkinSection`]/[`SkinBatch`] and
//! [`M2Model::parse_embedded_skin`], which decodes one of the model's embedded skin profiles
//! (indices / triangles / submeshes / batches) by file offset.
//! Embedded skin profiles: indices, triangles, submeshes and batches.
use benilla_bytes::{capped, ByteExt};
use crate::error::{Error, Result};
use crate::model::M2Model;
/// One skin submesh section (we keep the triangle range the renderer iterates, plus the section's
/// `skinSectionId` — the geoset/mesh-part ID the character compositor selects visibility by).
/// One skin submesh: its triangle range and geoset id.
pub struct SkinSection {
/// `skinSectionId` (mesh-part / geoset ID): `group*100 + variant`. For a character model this is
/// what geoset selection toggles (hide all but the chosen hair/facial-hair/body variants); `0` for a
/// model with a single unnamed geoset (most creatures/doodads).
/// `skinSectionId`, the geoset id `group*100 + variant` that character geoset selection
/// toggles; 0 on a single-geoset model.
pub id: u16,
pub triangle_start: u16,
pub triangle_count: u16,
@@ -20,26 +16,23 @@ pub struct SkinSection {
/// One skin draw batch (section + material + texture-combo references).
pub struct SkinBatch {
/// Batch flags (texUnit `+0x00`) — kept raw, unclassified. Read by the diagnostic dumps while
/// we carve which bits the reference's batch path actually reads; the renderer ignores them.
/// Batch flags (texUnit `+0x00`), raw; the renderer ignores them.
pub flags: u16,
/// Shader / render-order selector (texUnit `+0x02`), same status as [`Self::flags`].
/// Shader or render-order selector (texUnit `+0x02`), raw; the renderer ignores it.
pub shader_id: u16,
pub skin_section_index: u16,
pub texture_combo_index: u16,
/// Index into [`crate::M2Model::texture_unit_lookup`] → where this stage's texture coordinates
/// come from (a UV channel, or a generated environment coordinate). texUnit `+0x12`
/// (`0x70b8a4`).
/// Index into [`crate::M2Model::texture_unit_lookup`], the stage's coordinate source (texUnit
/// `+0x12`, `0x70b8a4`).
pub texture_coord_combo_index: u16,
pub material_index: u16,
/// Direct index into [`crate::M2Model::color_alpha_tracks`] (`0xffff` = none). texUnit `+0x08`.
/// Index into [`crate::M2Model::color_alpha_tracks`], `0xffff` for none (texUnit `+0x08`).
pub color_index: u16,
/// Index into [`crate::M2Model::transparency_lookup`] → a transparency track. texUnit `+0x14`.
/// Index into [`crate::M2Model::transparency_lookup`] (texUnit `+0x14`).
pub weight_combo_index: u16,
/// Texture count (texUnit `+0x0e`); `0` ⇒ the transparency factor is not applied (verified gate).
/// Texture count (texUnit `+0x0e`); at 0 the transparency factor is not applied.
pub texture_count: u16,
/// Index into [`crate::M2Model::texture_transform_lookup`] → a texture transform (UV
/// animation). texUnit `+0x16` (`0x70b897`).
/// Index into [`crate::M2Model::texture_transform_lookup`] (texUnit `+0x16`, `0x70b897`).
pub texture_transform_combo_index: u16,
}
@@ -66,9 +59,8 @@ impl Skin {
}
impl M2Model {
/// Decode embedded skin profile `index` from the original file `bytes`. The skin profiles are an
/// array of 44-byte `M2View` headers at `views.offset`; each references its indices / triangles /
/// submeshes / batches by file offset.
/// Decode embedded skin profile `index` from the file `bytes`: one of the 44-byte `M2View`
/// headers, each naming its arrays by file offset.
pub fn parse_embedded_skin(&self, bytes: &[u8], index: usize) -> Result<Skin> {
let (count, ofs) = self.views;
if index >= count as usize {
@@ -87,9 +79,7 @@ impl M2Model {
let (n_sub, o_sub) = rd_arr(vp + 24)?;
let (n_bat, o_bat) = rd_arr(vp + 32)?;
// `bytes_at(o, n * 2)` bounds-checks the whole run before any element is read, so — unlike
// the header-driven reservations above — `n` is already provably within the file's own size
// once this succeeds; no separate `capped()` needed for the `Vec::with_capacity(n)` below.
// `bytes_at` checks the whole run first, so `n` needs no `capped`.
let read_u16s = |n: usize, o: usize| -> Result<Vec<u16>> {
let slice = bytes.bytes_at(o, n * 2).ok_or(Error::Truncated)?;
let mut v = Vec::with_capacity(n);
@@ -101,12 +91,9 @@ impl M2Model {
let indices = read_u16s(n_idx, o_idx)?;
let triangles = read_u16s(n_tri, o_tri)?;
// M2SkinSection: 32 bytes pre-Wrath (v<260), 48 with the later centre-bounds; indexStart @8,
// indexCount @10 are the triangle range either way.
// M2SkinSection: 32 bytes before v260, 48 from it; the triangle range is @8 and @10.
let sec = if self.version < 260 { 32 } else { 48 };
// Reservations capped (decision 0064): `n_sub`/`n_bat` come straight from the M2View header,
// unvalidated, and the loops below check bounds only per-element — same shape as the header
// arrays above, so the same guard applies.
// `n_sub` and `n_bat` are unvalidated, so the reservations are capped.
let sub_avail = bytes.len().saturating_sub(o_sub);
let mut submeshes = Vec::with_capacity(capped(n_sub, sec, sub_avail));
for i in 0..n_sub {
@@ -119,10 +106,7 @@ impl M2Model {
triangle_count: s.u16_at(10).ok_or(Error::Truncated)?,
});
}
// M2TextureUnit (batch): 24 bytes — skinSectionIndex@4, colorIndex@8, materialIndex@0xa,
// textureCount@0xe, textureComboIndex@0x10, textureCoordComboIndex@0x12 (the env-vs-UV
// source), weightComboIndex@0x14 (the visibility-factor refs), textureTransformComboIndex@0x16
// (UV animation).
// M2TextureUnit, a batch: 24 bytes.
let bat_avail = bytes.len().saturating_sub(o_bat);
let mut batches = Vec::with_capacity(capped(n_bat, 24, bat_avail));
for i in 0..n_bat {
crates/benilla-m2/src/tests.rs
+27 -55
View File
@@ -1,11 +1,9 @@
use super::*;
// The field walk in `parse_m2` reads M2Arrays up through the event table (the last read, at
// header offset 0x114) and stops there — the fixture header covers exactly that walk.
// The header through the event table at 0x114, as far as `parse_m2` walks.
const HEADER_LEN: usize = 284;
// Byte offsets of the M2Arrays this parser reads, matching the field walk in `parse_m2` for a
// v256 (pre-Wrath) header.
// v256 header offsets of the M2Arrays under test.
const OFS_PLAYABLE_ANIM_LOOKUP: usize = 44;
const OFS_TEX_ANIM: usize = 0x74;
const OFS_TEX_UNIT_LOOKUP: usize = 0x9c;
@@ -17,8 +15,7 @@ const OFS_VIEWS: usize = 76;
const OFS_ATTACHMENTS: usize = 260;
const OFS_EVENTS: usize = 276;
/// A minimal v256 MD20 header: the magic + version, every M2Array zeroed (count 0, offset 0).
/// Callers patch specific arrays with [`set_arr`] and append payload bytes after `HEADER_LEN`.
/// A v256 MD20 header with every M2Array empty; tests patch arrays with [`set_arr`].
fn header() -> Vec<u8> {
let mut b = vec![0u8; HEADER_LEN];
b[0..4].copy_from_slice(b"MD20");
@@ -69,12 +66,8 @@ fn m2track(interp: u16, gseq: u16, ts: (u32, u32), vals: (u32, u32)) -> Vec<u8>
t
}
/// **A fix16 key is SIGNED.** Real art authors "hide me" as `0x8001` — `−32767`, i.e. `−1.0` —
/// which read unsigned decodes to `+1.00006` and sails through the reference's `A ≤ 0` batch cull.
/// `TanarisTrollGate.m2` switches between its intact gate and its burnt twin with exactly these
/// ±1 keys, so reading them unsigned drew both copies at once (B138, decision 1460). Both records
/// that carry a fix16 track — the M2Color **alpha** (header `0x54`, stride `0x38`, track @ `+0x1c`)
/// and the M2TextureWeight (header `0x64`, stride `0x1c`) — share the decode, so both are checked.
/// `TanarisTrollGate.m2` swaps its intact and burnt gates with these ±1 keys. Both fix16 records
/// are checked: the M2Color alpha (header `0x54`, track @ `+0x1c`) and the M2TextureWeight.
#[test]
fn a_negative_fix16_key_decodes_signed() {
let mut b = header();
@@ -82,14 +75,14 @@ fn a_negative_fix16_key_decodes_signed() {
b.extend(0u32.to_le_bytes());
b.extend(333u32.to_le_bytes());
let val_ofs = b.len() as u32;
b.extend(0x7fffu16.to_le_bytes()); // +1.0 — "draw me"
b.extend(0x8001u16.to_le_bytes()); // −1.0 — "hide me"
// One M2Color: a keyless RGB track @ +0x00, then the 2-key alpha track @ +0x1c.
b.extend(0x7fffu16.to_le_bytes()); // +1.0, drawn
b.extend(0x8001u16.to_le_bytes()); // −1.0, hidden
// One M2Color: a keyless RGB track, then the alpha track.
let color_ofs = b.len() as u32;
b.extend(m2track(0, 0xffff, (0, 0), (0, 0)));
b.extend(m2track(1, 0xffff, (2, ts_ofs), (2, val_ofs)));
set_arr(&mut b, 0x54, 1, color_ofs);
// One M2TextureWeight: the same keys again, so the weight side is covered too.
// One M2TextureWeight with the same keys.
let weight_ofs = b.len() as u32;
b.extend(m2track(1, 0xffff, (2, ts_ofs), (2, val_ofs)));
set_arr(&mut b, 0x64, 1, weight_ofs);
@@ -128,8 +121,7 @@ fn texture_transform_translation_track_decodes() {
b.extend(c.to_le_bytes());
}
}
// One M2TextureTransform record (stride 0x54): a 2-key gseq-0 translation track, then keyless
// rotation + scaling tracks.
// One M2TextureTransform: a 2-key gseq-0 translation, then keyless rotation and scaling.
let rec_ofs = b.len() as u32;
b.extend(m2track(1, 0, (2, ts_ofs), (2, val_ofs)));
b.extend(m2track(0, 0xffff, (0, 0), (0, 0)));
@@ -156,11 +148,7 @@ fn texture_transform_translation_track_decodes() {
assert!(t.scaling.keys.is_empty());
}
/// The three adjacent u16 lookup slots must not be confused: texUnitLookup@0x9c ·
/// transLookup@0xa4 · texAnimLookup@0xac (the reference's header walk `0x71cdf0`, not one slot
/// early). Shaped like the real StormwindMagePortal01 header — [0] at
/// 0x9c, the identity [0,1,2,3] at 0xa4, [0xffff] at 0xac — where reading 0x9c as the transparency
/// lookup silently dropped the combo-1..3 weight tracks.
/// Shaped like StormwindMagePortal01's header: [0] at 0x9c, [0,1,2,3] at 0xa4, [0xffff] at 0xac.
#[test]
fn transparency_lookup_reads_0xa4_not_the_texture_unit_lookup_at_0x9c() {
let mut b = header();
@@ -186,8 +174,7 @@ fn transparency_lookup_reads_0xa4_not_the_texture_unit_lookup_at_0x9c() {
fn playable_animation_lookup_decodes_low16_high16() {
let mut b = header();
let pal_ofs = b.len() as u32;
// Row 0: identity, no dir flags. Row 1: HumanMale's real example, read off the shipped asset
// (`playableAnimationLookup[6] = 0x00030001`) — resolved id 1, dir-flags code 3.
// Row 1 is HumanMale's `playableAnimationLookup[6] = 0x00030001`: id 1, code 3.
b.extend(0u32.to_le_bytes());
b.extend(0x0003_0001u32.to_le_bytes());
set_arr(&mut b, OFS_PLAYABLE_ANIM_LOOKUP, 2, pal_ofs);
@@ -210,7 +197,7 @@ fn bone_record(pivot: [f32; 3]) -> Vec<u8> {
}
fn attachment_record(id: u32, bone: u32, position: [f32; 3]) -> Vec<u8> {
let mut b = vec![0u8; 48]; // id+bone+position (20 bytes) + the skipped 28-byte visibility track
let mut b = vec![0u8; 48]; // id, bone, position (20 bytes), the skipped 28-byte track
b[0..4].copy_from_slice(&id.to_le_bytes());
b[4..8].copy_from_slice(&bone.to_le_bytes());
b[8..12].copy_from_slice(&position[0].to_le_bytes());
@@ -222,7 +209,7 @@ fn attachment_record(id: u32, bone: u32, position: [f32; 3]) -> Vec<u8> {
#[test]
fn attachments_parse_and_skip_out_of_range_records() {
let mut b = header();
// Two bones, so a valid attachment bone index is 0 or 1.
// Two bones, so a valid bone index is 0 or 1.
let bones_ofs = b.len() as u32;
b.extend(bone_record([0.0, 0.0, 0.0]));
b.extend(bone_record([1.0, 1.0, 1.0]));
@@ -243,7 +230,7 @@ fn attachments_parse_and_skip_out_of_range_records() {
}
fn event_record(ident: &[u8; 4], bone: u32, position: [f32; 3]) -> Vec<u8> {
let mut b = vec![0u8; 44]; // ident+data+bone+position (24 bytes) + the skipped 20-byte track
let mut b = vec![0u8; 44]; // ident, data, bone, position (24 bytes), the skipped 20-byte track
b[0..4].copy_from_slice(ident);
b[8..12].copy_from_slice(&bone.to_le_bytes());
b[12..16].copy_from_slice(&position[0].to_le_bytes());
@@ -300,11 +287,7 @@ fn one_vertex_decodes_all_fields() {
#[test]
fn hostile_vertex_count_errs_cleanly_not_oom() {
// The bug this migration fixes: `vertices.0` is read straight off the file with no relation
// to the buffer's actual size. Pre-0064, `Vec::with_capacity(vertices.0 as usize)` tried to
// reserve `u32::MAX * 48` bytes up front and aborted the process. Post-0064 the reservation
// is capped by the remaining input (here: zero bytes after the header), so this returns an
// ordinary `Err` from the first bounds-checked read instead.
// `u32::MAX` vertices must fail at the first read, not reserve `u32::MAX * 48` bytes.
let mut b = header();
set_arr(&mut b, OFS_VERTICES, u32::MAX, HEADER_LEN as u32);
assert!(matches!(parse(&b), Err(Error::Truncated)));
@@ -312,10 +295,7 @@ fn hostile_vertex_count_errs_cleanly_not_oom() {
#[test]
fn hostile_attachment_count_errs_cleanly_not_oom() {
// The vertex case's sibling one block later: the per-record index vector `emitted_at` was
// sized from the raw attachment count (u32::MAX → an 8 GiB non-zero fill that touches every
// page) while the reservation beside it was capped. The table is now refused up front when
// the file cannot hold it — the verdict the loop's bounds-checked read reached one line later.
// `emitted_at` is sized from the count, so a table the file cannot hold is refused first.
let mut b = header();
set_arr(&mut b, OFS_ATTACHMENTS, u32::MAX, HEADER_LEN as u32);
assert!(matches!(parse(&b), Err(Error::Truncated)));
@@ -323,10 +303,8 @@ fn hostile_attachment_count_errs_cleanly_not_oom() {
#[test]
fn hostile_track_record_counts_walk_only_what_the_file_holds() {
// The colour (0x54), transparency (0x64) and texture-transform (0x74) readers are lenient by
// design — an out-of-range track is "no keys" — so a raw count of u32::MAX used to mean four
// billion empty tracks pushed one per iteration until the OS killed the process. The loop
// bound is now the number of whole records the file holds past the block's offset.
// The colour (0x54), weight (0x64) and texture-transform (0x74) readers never fail, so the
// file bounds their loops, not the count.
let mut b = header();
set_arr(&mut b, 0x54, u32::MAX, HEADER_LEN as u32);
set_arr(&mut b, 0x64, u32::MAX, HEADER_LEN as u32);
@@ -358,20 +336,18 @@ fn truncated_vertex_record_errs_cleanly() {
fn hostile_shapes_do_not_panic() {
assert!(matches!(parse(&[]), Err(Error::NotMd20)));
assert!(matches!(parse(&[0u8; 7]), Err(Error::NotMd20)));
// A recognizable magic with an unsupported version.
let mut b = vec![0u8; HEADER_LEN];
b[0..4].copy_from_slice(b"MD20");
b[4..8].copy_from_slice(&999u32.to_le_bytes());
assert!(matches!(parse(&b), Err(Error::UnsupportedVersion(999))));
// A header truncated mid-array.
assert!(matches!(
parse(&header()[..HEADER_LEN - 4]),
Err(Error::Truncated)
));
}
/// Build a full synthetic MD20 file with one embedded skin profile (one index, one triangle, one
/// submesh, one batch) so `parse_embedded_skin` has something real to decode.
/// An MD20 file with one embedded skin profile: three indices, three triangle indices, one
/// submesh, one batch.
fn model_with_one_skin() -> Vec<u8> {
let mut b = header();
let vp = HEADER_LEN as u32; // M2View header right after the fixed header
@@ -436,7 +412,7 @@ fn embedded_skin_out_of_range_index_errs() {
#[test]
fn hostile_submesh_count_errs_cleanly_not_oom() {
// Same OOM-abort shape as the vertex case, but inside `parse_embedded_skin`'s M2View arrays.
// The vertex case, in `parse_embedded_skin`'s M2View arrays.
let mut b = model_with_one_skin();
let vp = HEADER_LEN; // M2View header offset (see `model_with_one_skin`)
let end = b.len() as u32;
@@ -450,8 +426,7 @@ fn hostile_submesh_count_errs_cleanly_not_oom() {
// --- cameras (header 0x124/0x128) + the cubic samplers ---------------------------------------
/// The camera array sits past [`HEADER_LEN`], so camera fixtures carry a header long enough to
/// hold `cameras`@`0x124` and `cameraLookup`@`0x12c`.
/// A header long enough for `cameras`@`0x124` and `cameraLookup`@`0x12c`, past [`HEADER_LEN`].
const CAM_HEADER_LEN: usize = 0x134;
const OFS_CAMERAS: usize = 0x124;
const OFS_CAMERA_LOOKUP: usize = 0x12c;
@@ -465,8 +440,7 @@ fn cam_header() -> Vec<u8> {
/// One fixture key: `(ms, value, in_tan, out_tan)`.
type SplineKeyFixture = (u32, [f32; 3], [f32; 3], [f32; 3]);
/// Append a cubic `C3Vector` track's payload and return the 28 track bytes pointing at it, laid
/// out `{value, inTan, outTan}` per the 0x24 key.
/// Append a cubic `C3Vector` track's `{value, inTan, outTan}` keys and return the track's 28 bytes.
fn spline_vec3_track(b: &mut Vec<u8>, interp: u16, keys: &[SplineKeyFixture]) -> Vec<u8> {
let times_ofs = b.len() as u32;
for (ms, ..) in keys {
@@ -492,9 +466,8 @@ fn spline_vec3_track(b: &mut Vec<u8>, interp: u16, keys: &[SplineKeyFixture]) ->
t
}
/// A one-camera model whose position track carries the two keys `interp` will be sampled between:
/// `value 0 → 3` on X with tangents `out[k0] = 1`, `in[k1] = 2` — four numbers chosen so the four
/// interpolation legs land on four *different* answers.
/// A one-camera model whose position track runs X from 0 to 3 over 1000 ms, with
/// `outTan[k0] = 1` and `inTan[k1] = 2`.
fn model_with_one_camera(interp: u16) -> Vec<u8> {
let mut b = cam_header();
let rec = b.len() as u32;
@@ -549,8 +522,7 @@ fn camera_record_fields_and_lookup_parse() {
#[test]
fn cubic_sampler_takes_all_four_interp_legs() {
// Between `value 0` and `value 3`, with `outTan[k0] = 1` and `inTan[k1] = 2`, at the exact
// midpoint. Hand-computed from the bases in `M2Track::sample_ms`:
// At the midpoint, hand-computed from the bases in `M2Track::sample_ms`:
// step → value[k0] = 0
// linear → 0 + (3−0)·0.5 = 1.5
// Bézier → 0.125·0 + 0.375·1 + 0.375·2 + 0.125·3 = 1.5
crates/benilla-m2/src/track.rs
+43 -84
View File
@@ -1,34 +1,21 @@
//! M2Track readers: the typed [`M2Track`] (with the [`M2ScalarTrack`] / [`M2Vec3Track`] /
//! [`M2QuatTrack`] aliases) plus the track-parsing helpers the header walk (`crate::parse_m2`)
//! calls for the M2Color, M2TextureWeight, and M2TextureTransform tracks.
//! M2Track readers and the cubic track sampler.
use benilla_bytes::ByteExt;
/// One typed `M2Track`, fully read: the interpolation type, the global-sequence tag, the
/// per-sequence key **ranges**, and the keys as **absolute global-timeline milliseconds** paired
/// with values. The v256 track (stride `0x1c`): `interp`@0, `global_seq`@2, interpolation_ranges
/// `M2Array`@`0x04/0x08`, timestamps `M2Array`@`0x0c/0x10` (u32 ms), values `M2Array`@`0x14/0x18`.
/// A sequence-timeline track (`gseq == 0xffff`) keys inside each sequence's absolute time band; a
/// global-sequence track loops on `global_sequences[gseq]`'s own clock (`0x713d50`). Key count is
/// `min(timestamps, values)` — vanilla art occasionally pads one array.
/// One `M2Track` (v256, `0x1c` bytes): `interp`@0, `gseq`@2, then ranges, timestamps (u32 ms) and
/// values as `(count, offset)` pairs @`0x04`, `0x0c`, `0x14`. Keys are absolute timeline ms: a
/// sequence track keys inside each sequence's band, a `gseq` track loops on its global sequence
/// (`0x713d50`). The key count is `min(timestamps, values)`, as vanilla art pads one array.
#[derive(Clone, Debug)]
pub struct M2Track<V> {
/// Interpolation type: `0` = step (nearest-previous key), nonzero = linear (the scalar sampler
/// `0x71af20`'s `cmp word[track],0` two-way dispatch).
/// `0` step (the previous key), nonzero linear: the scalar sampler's two-way test (`0x71af20`).
pub interp: u16,
/// Global-sequence index, `0xffff` = an ordinary sequence-timeline track.
pub gseq: u16,
/// The per-sequence **key-index window** `(lo, hi)`, one entry per sequence in file order —
/// the array the reference's key search indexes by the playing sequence's slot before it ever
/// looks at a timestamp (`0x713d50`: `[track+8][idx*8+{0,4}]`; an empty array is the
/// `[track+4]==0` fallback, "search the whole key list").
///
/// These are **brackets**, not the playable key set: `hi` routinely points at a key in a LATER
/// sequence's band, which is why selecting in-clip keys through this window instead of by
/// timestamp froze creatures at a garbage pose (benilla decision 0133). Its load-bearing use is
/// the other one — a band that keys nothing resolves to `keys[lo]` (`lo >= hi` ⇒ the degenerate
/// `{lo, lo, 0}` result), so this array says *exactly* what the reference holds there instead
/// of leaving it to a nearest-key approximation.
/// The per-sequence key-index window `(lo, hi)`, one per sequence in file order, which the
/// reference's key search reads before any timestamp (`0x713d50`; empty means search all
/// keys). Brackets, not the clip's keys: `hi` often lands in a later sequence's band. A band
/// with no keys resolves to `keys[lo]`.
pub ranges: Vec<(u32, u32)>,
/// `(absolute ms, value)` keys, file order (time-ascending within a band).
pub keys: Vec<(u32, V)>,
@@ -45,25 +32,23 @@ impl<V> Default for M2Track<V> {
}
}
/// A scalar (`fix16`, `0..=1`) track — the M2Color **alpha** and M2TextureWeight **weight** tracks.
/// A scalar track: the M2Color alpha and the M2TextureWeight weight.
pub type M2ScalarTrack = M2Track<f32>;
/// A `C3Vector` track — the M2TextureTransform **translation** / **scaling** tracks.
/// A `C3Vector` track: the M2Color RGB and the texture-transform translation and scaling.
pub type M2Vec3Track = M2Track<[f32; 3]>;
/// A quaternion (4×f32, raw v256 floats) track — the M2TextureTransform **rotation** track.
/// A quaternion track (4×f32): the texture-transform rotation.
pub type M2QuatTrack = M2Track<[f32; 4]>;
impl<V: Copy + PartialEq> M2Track<V> {
/// All key values equal (or a single key): the track is a constant — `Some(value)`; `None` for a
/// keyless or genuinely time-varying track.
/// The track's value when every key holds the same one.
pub fn constant(&self) -> Option<V> {
let (_, first) = *self.keys.first()?;
self.keys.iter().all(|&(_, v)| v == first).then_some(first)
}
}
/// Read one `M2Track<V>` (see [`M2Track`]): the shared `interp`/`gseq`/timestamp walk with a
/// per-value reader. Out-of-range reads yield an empty key list (= "no keys", the factor/transform
/// does not apply) — real art relies on that tolerance.
/// Read one `M2Track<V>` with a per-value reader. An out-of-range track reads as no keys, a
/// tolerance real art needs.
fn track_read<V>(
b: &[u8],
track_ofs: usize,
@@ -85,9 +70,8 @@ fn track_read<V>(
let keys = (0..n)
.map_while(|i| b.u32_at(to + i * 4).zip(read_val(b, vo + i * val_size)))
.collect();
// The per-sequence `(lo, hi)` key-index windows (`M2Array`@0x04/0x08, 8-byte entries) — see
// [`M2Track::ranges`]. A short/absent array reads as empty, which is the reference's own
// `[track+4] == 0` "no ranges" fallback.
// The `(lo, hi)` windows, 8 bytes each; an unreadable array is empty, the reference's
// no-ranges case.
let ranges = match b.u32_at(track_ofs + 0x04).zip(b.u32_at(track_ofs + 0x08)) {
Some((rn, ro)) => (0..rn as usize)
.map_while(|i| {
@@ -109,32 +93,22 @@ fn rd_vec3(b: &[u8], o: usize) -> Option<[f32; 3]> {
Some([b.f32_at(o)?, b.f32_at(o + 4)?, b.f32_at(o + 8)?])
}
/// Read a scalar `fix16` track (**`int16`**/32767 values). Used for the colour-**alpha** and
/// transparency-**weight** tracks that gate batch visibility and drive the animated material combine.
///
/// **The key is SIGNED** — `movsx`, not `movzx` (`dest = (f32)(int16 P[k0]) * (1/0x7fff)`, bytes
/// `movsx edx,word[P+k0*2]; fild; fmul [0x811610]; fstp dest` at `0x715b2f`–`0x715b46`, dispatched
/// at the M2Color-alpha site `0x715b21` (`colors[]` stride 0x38, track @ +0x1c) and the
/// transparency-weight site `0x715ce2`). Read unsigned, the authored "hide me" key `0x8001` decodes
/// as `+1.00006` instead of `−1.0`, so a batch the reference culls (`A ≤ 0`, `0x707b3a`–`0x707b5c`)
/// draws at full alpha instead: that is how Zul'Farrak's troll gate drew its BURNT twin on top
/// of its intact self and z-fought (B138, decision 1460). Values outside `[0, 1]` are the
/// artist's own encoding, not a data quirk — the combine consumes them as signed floats and the
/// cull tests `≤ 0`.
/// Read a `fix16` scalar track, `int16 / 32767`. The key is signed (`movsx`, then `fmul` by the
/// `1/0x7fff` at `0x811610`, `0x715b2f`–`0x715b46`; from the colour alpha `0x715b21` and the weight
/// `0x715ce2`): art authors `0x8001` (−1.0) to hide a batch, which the cull (`A ≤ 0`,
/// `0x707b3a`–`0x707b5c`) drops. Values outside `[0, 1]` are authored, not noise.
pub(crate) fn track_fix16(b: &[u8], track_ofs: usize) -> M2ScalarTrack {
track_read(b, track_ofs, 2, |b, o| {
b.u16_at(o).map(|v| f32::from(v as i16) / 32767.0)
})
}
/// Read a timed `C3Vector` (3×f32) track — the texture-transform **translation**/**scaling**
/// tracks and the M2Color **RGB** track.
/// Read a `C3Vector` (3×f32) track.
pub(crate) fn track_vec3_timed(b: &[u8], track_ofs: usize) -> M2Vec3Track {
track_read(b, track_ofs, 12, rd_vec3)
}
/// Read a timed quaternion (4×f32, raw v256 floats — vanilla does not compress quat keys) track —
/// the texture-transform **rotation** track.
/// Read a quaternion track: 4×f32, as vanilla does not compress quaternion keys.
pub(crate) fn track_quat(b: &[u8], track_ofs: usize) -> M2QuatTrack {
track_read(b, track_ofs, 16, |b, o| {
Some([
@@ -146,16 +120,10 @@ pub(crate) fn track_quat(b: &[u8], track_ofs: usize) -> M2QuatTrack {
})
}
/// One key of a **cubic** M2 track — the reference's `M2SplineKey<T>`: the value plus its in/out
/// tangents, `{value@+0, in_tan@+1·sizeof(T), out_tan@+2·sizeof(T)}` (the vec3 key is stride
/// `0x24` `{value@+0, inTan@+0xc, outTan@+0x18}`, the scalar-float key stride `0xc`
/// `{value@+0, inTan@+4, outTan@+8}`).
///
/// **The wide key is the stride whatever `interp` says.** The reference's cubic element loops
/// address every key as `payload + k*0x24` (`0x716b51`) / `payload + k*0xc` (`0x7173cc`) *before*
/// the four-way interp dispatch, and the STEP/LINEAR legs then read the `value` sub-field of that
/// same wide key. Real art relies on it: `Cameras\FlyByDwarf.m2`'s roll track is authored
/// `interp = 0` and still carries a 12-byte key.
/// One key of a cubic track: the value, then the in and out tangents, each `sizeof(T)` apart. The
/// wide key is the stride whatever `interp` says: the reference addresses keys as `k*0x24`
/// (`0x716b51`) or `k*0xc` (`0x7173cc`) before dispatching, and step and linear read its `value`
/// (`Cameras\FlyByDwarf.m2`'s roll track is `interp = 0` with 12-byte keys).
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct M2SplineKey<V> {
pub value: V,
@@ -163,18 +131,17 @@ pub struct M2SplineKey<V> {
pub out_tan: V,
}
/// A cubic `C3Vector` track — the M2Camera **position**/**target** tracks.
/// A cubic `C3Vector` track: the M2Camera position and target.
pub type M2Vec3SplineTrack = M2Track<M2SplineKey<[f32; 3]>>;
/// A cubic scalar-float track — the M2Camera **roll** track.
/// A cubic scalar track: the M2Camera roll.
pub type M2ScalarSplineTrack = M2Track<M2SplineKey<f32>>;
/// A value a cubic track can hold: the four basis weights are scalars, so the leaf only has to
/// know how to build `Σ wᵢ·Pᵢ` componentwise.
/// A value a cubic track can hold.
pub trait CubicValue: Copy {
/// `w0·p0 + w1·p1 + w2·p2 + w3·p3`.
fn combine(w: [f32; 4], p: [Self; 4]) -> Self;
/// The LINEAR leg, in the reference's own form `a + (b − a)·t` (`0x716cf1`) — not the
/// algebraically equal `(1−t)·a + t·b`, which rounds differently in f32.
/// The linear leg in the reference's form `a + (b − a)·t` (`0x716cf1`), which rounds
/// differently in f32 from `(1−t)·a + t·b`.
fn lerp(a: Self, b: Self, t: f32) -> Self;
}
@@ -197,24 +164,16 @@ impl CubicValue for [f32; 3] {
}
impl<V: CubicValue> M2Track<M2SplineKey<V>> {
/// Sample this cubic track at absolute global-timeline `ms`, **end-clamped** at both ends.
/// Sample at absolute timeline `ms`, clamped at both ends, by the cubic loops' four-way
/// `interp` dispatch (the bone loops have only two):
///
/// The four-way `interp` dispatch is the reference's own, and it is the *cubic element loops'*
/// dispatch — not the two-way `cmp word[track],0; jne <linear>` the bone/TRS loops collapse to
/// (the switch is per-loop, by track type). The bases, with `t` the key-interval fraction:
/// - `0` step: `value[k0]` (`0x716b5f`).
/// - `1` linear (`0x716cf1`).
/// - `2` Bézier over `{value[k0], outTan[k0], inTan[k1], value[k1]}` (`0x716c41`), what every
/// `Cameras\*.m2` fly-by authors for position and target.
/// - `3` Hermite over `value[k0]`, `outTan[k0]`, `value[k1]`, `inTan[k1]` (`0x716b9e`).
///
/// - `0` **STEP** — `value[k0]`, no tangent read (`0x716b5f`).
/// - `1` **LINEAR** — `value[k0] + (value[k1] − value[k0])·t` (`0x716cf1`).
/// - `2` **BÉZIER** — the cubic Bernstein basis over control points
/// `{value[k0], outTan[k0], inTan[k1], value[k1]}`: `B0 = (1−t)³`, `B1 = 3t(1−t)²`,
/// `B2 = 3t²(1−t)`, `B3 = t³` (`0x716c41`). This is what every shipped `Cameras\*.m2`
/// fly-by authors on its position and target tracks.
/// - `3` **HERMITE** — `h00·value[k0] + h10·outTan[k0] + h01·value[k1] + h11·inTan[k1]` with
/// `h00 = 2t³−3t²+1`, `h10 = t³−2t²+t`, `h01 = 3t²−2t³`, `h11 = t³−t²` (`0x716b9e`).
///
/// Note which tangent each control point comes from: the **outgoing** tangent of the key
/// being left and the **incoming** tangent of the key being entered. Swapping them looks
/// almost right and drifts wrong exactly where the path curves hardest.
/// A span takes the out tangent of the key it leaves and the in tangent of the key it enters.
pub fn sample_ms(&self, ms: u32) -> Option<V> {
let first = self.keys.first()?;
let last = self.keys.last()?;
@@ -272,12 +231,12 @@ fn rd_spline<V>(
})
}
/// Read a cubic `C3Vector` track (key stride `0x24`) — the M2Camera position/target tracks.
/// Read a cubic `C3Vector` track (key stride `0x24`).
pub(crate) fn track_spline_vec3(b: &[u8], track_ofs: usize) -> M2Vec3SplineTrack {
track_read(b, track_ofs, 0x24, |b, o| rd_spline(b, o, 12, rd_vec3))
}
/// Read a cubic scalar-float track (key stride `0xc`) — the M2Camera roll track.
/// Read a cubic scalar track (key stride `0xc`).
pub(crate) fn track_spline_f32(b: &[u8], track_ofs: usize) -> M2ScalarSplineTrack {
track_read(b, track_ofs, 0xc, |b, o| {
rd_spline(b, o, 4, |b, o| b.f32_at(o))
crates/benilla-mpq/examples/mpqcat.rs
+6 -8
View File
@@ -1,15 +1,13 @@
//! `mpqcat` — write one file out of an MPQ to stdout. The instrument for reading the real client's
//! own data (FrameXML, GlobalStrings, a BLP header) when a fidelity question needs the source
//! rather than a memory of it. Several decision records cite an external `mpqx` for this; nothing
//! in the workspace could do it until now, so every lookup was a hunt.
//! `mpqcat`: write one file out of an MPQ to stdout, to read the client's own data (FrameXML,
//! GlobalStrings, a BLP header).
//!
//! ```text
//! cargo run -q -p benilla-mpq --example mpqcat -- <archive.MPQ> 'Interface\FrameXML\MacroFrame.xml'
//! cargo run -q -p benilla-mpq --example mpqcat -- <archive.MPQ> 'Interface\FrameXML\UIParent.lua'
//! ```
//!
//! Archive names are the client's own backslash paths. A miss is an error, not empty output — the
//! same file often lives in several archives (patch beats interface beats base), and "not here"
//! and "here but empty" are answers you must be able to tell apart.
//! Names are the client's backslash paths. A miss is an error, not empty output: the same file
//! often lives in several archives (patch over interface over base), and "not here" must read
//! apart from "here but empty".
use std::io::Write;
fn main() -> Result<(), Box<dyn std::error::Error>> {
crates/benilla-mpq/src/crypto.rs
+12 -23
View File
@@ -1,15 +1,8 @@
//! The Storm (MPQ) crypto primitives — byte-exact with the real client.
//!
//! Three pieces, all driven by one statically-generated 0x500-entry table:
//! - [`hash_string`] — Blizzard's string hash, used for hash-table indexing (`TABLE_OFFSET`), the two
//! filename verifiers (`NAME_A`/`NAME_B`), and table/file keys (`FILE_KEY`).
//! - [`decrypt_block`] — the in-place table/sector decryptor.
//!
//! 1.12.1 archives carry **no encrypted files** (verified across the real `Data/`), so only the hash
//! and block *tables* are ever decrypted. The known StormLib vectors are pinned as tests below.
//! The Storm (MPQ) crypto, byte-exact with the client: the string hash and the block decryptor,
//! both over one 0x500-entry table. 1.12.1 archives carry no encrypted files, so only the hash and
//! block tables are ever decrypted.
/// The MPQ encryption table (0x500 u32s), generated by the StormLib seed walk. `const fn` so it's
/// baked into the binary, identical to the table the client builds at runtime.
/// The 0x500-entry encryption table, built at compile time by the seed walk the client runs.
const fn encryption_table() -> [u32; 0x500] {
let mut table = [0u32; 0x500];
let mut seed: u32 = 0x0010_0001;
@@ -32,20 +25,20 @@ const fn encryption_table() -> [u32; 0x500] {
static TABLE: [u32; 0x500] = encryption_table();
/// Hash-type selectors (the table-quadrant offsets the client uses).
/// Hash-type selectors: the table-quadrant offsets the client uses.
pub mod hash_type {
/// Hash-table slot index.
pub const TABLE_OFFSET: u32 = 0x000;
/// Filename verifier A (stored in the hash entry).
/// Filename verifier A, stored in the hash entry.
pub const NAME_A: u32 = 0x100;
/// Filename verifier B (stored in the hash entry).
/// Filename verifier B, stored in the hash entry.
pub const NAME_B: u32 = 0x200;
/// File / table decryption key.
/// File and table decryption key.
pub const FILE_KEY: u32 = 0x300;
}
/// Blizzard's string hash. Normalizes `/`→`\` and ASCII-uppercases (matching the client's
/// `AsciiToUpper` table, which only folds `a`–`z`), then runs the seed mix.
/// Blizzard's string hash: `/` reads as `\`, and only `a`-`z` are uppercased, as the client's
/// `AsciiToUpper` table does.
pub fn hash_string(name: &str, kind: u32) -> u32 {
let mut seed1: u32 = 0x7FED_7FED;
let mut seed2: u32 = 0xEEEE_EEEE;
@@ -64,8 +57,7 @@ pub fn hash_string(name: &str, kind: u32) -> u32 {
seed1
}
/// Decrypt a block of `u32`s in place with `key` (the StormLib block cipher). A zero key is a no-op,
/// as in the client. Used for the hash/block tables; file data is never encrypted in 1.12.1.
/// Decrypt `u32`s in place with `key`; a zero key is a no-op, as in the client.
pub fn decrypt_block(data: &mut [u32], mut key: u32) {
if key == 0 {
return;
@@ -83,8 +75,7 @@ pub fn decrypt_block(data: &mut [u32], mut key: u32) {
}
}
/// Test-only inverse of [`decrypt_block`] — lets the reader's round-trip tests (here and in `lib`)
/// build the encrypted hash/block tables a real archive stores. Same walk, XOR on the plaintext side.
/// The inverse of [`decrypt_block`], for tests that build the encrypted tables an archive stores.
#[cfg(test)]
pub(crate) fn encrypt_block(data: &mut [u32], mut key: u32) {
let mut seed: u32 = 0xEEEE_EEEE;
@@ -104,7 +95,6 @@ pub(crate) fn encrypt_block(data: &mut [u32], mut key: u32) {
mod tests {
use super::*;
/// StormLib's canonical vectors — these pin our table + hash to the real client byte-for-byte.
#[test]
fn hash_vectors_match_stormlib() {
assert_eq!(
@@ -128,7 +118,6 @@ mod tests {
assert_eq!(a, b);
}
/// decrypt∘encrypt round-trips (encrypt is the same walk with the xor on the other side).
#[test]
fn decrypt_inverts_encrypt() {
let key = hash_string("(hash table)", hash_type::FILE_KEY);
crates/benilla-mpq/src/lib.rs
+69 -162
View File
@@ -1,38 +1,14 @@
//! A read-only MPQ archive reader for **WoW 1.12.1 (build 5875)** — in-repo, replacing `wow-mpq`
//! (decision 0021).
//! A read-only MPQ reader for the 1.12.1 `Data/` chain: format V1/V2, every file `COMPRESS`-flagged
//! and sectored, no encrypted or single-unit files, no PTCH patches. Anything else is a hard error.
//!
//! Deliberately narrow to what the real `Data/` chain actually is (verified by probing every archive):
//! format **V1/V2**, every file `COMPRESS`-flagged and **sectored**, **no encrypted files**, **no
//! single-unit**, **no PTCH** incremental patches. So this does header parse → hash/block **table**
//! decrypt → name lookup → sectored read (per-sector inflate). It does *not* write archives, decrypt
//! file data, verify signatures, or load the `(attributes)` table — the wasteful per-open parse that
//! motivated 0021. Anything outside the verified envelope is a hard error, never a silent guess.
//! The patch archives carry delete markers (flag `0x02000000`, size 0): the path is deleted from
//! the composite chain. [`Archive::contains`] reports one present and [`Archive::read_file`]
//! refuses it with [`Error::NotFound`], so a chain walker asks [`Archive::is_delete_marker`] and
//! stops there.
//!
//! **Delete-markers.** The patch archives *do* carry file tombstones — a block entry flagged
//! `MPQ_FILE_DELETE_MARKER` (`0x02000000`), `size == 0`, meaning "this path is deleted from the
//! composite; do not use any lower-priority archive's copy." (`patch.MPQ` uses these to remove stock
//! FrameXML it replaces with backported addons — decision 0246.) These are **not** readable files:
//! [`Archive::read_file`] refuses one with [`Error::NotFound`] rather than returning the empty buffer
//! a naive `size == 0` stored-read would (the silent mislead that cost 0246 a day), and
//! [`Archive::is_delete_marker`] lets the chain walker see a tombstone and stop instead of falling
//! through. A delete-marked entry still *exists* in the hash table, so [`Archive::contains`] reports
//! it present — the chain must consult [`Archive::is_delete_marker`] to tell "deleted" from "readable".
//!
//! ## Concurrency
//! The parsed [`Index`] (header offsets + decrypted hash/block tables) is immutable, so [`Archive`] is
//! a cheap `Arc` handle that is `Send + Sync` and `Clone`. [`Archive::read_file`] is `&self`: it opens
//! a **fresh** OS file handle for the read, so concurrent reads share no seek state and need no lock.
//! Opening the handle is a cheap syscall; the expensive table parse happens once, in [`Archive::open`].
//!
//! Correctness was proven by a differential pass against `wow-mpq` over real archives — 37,971 reads
//! byte-identical across all 13 (decision 0021; the throwaway oracle test is in git history). The
//! `benilla-formats` loaders now exercise this reader end-to-end against real data on every run.
//!
//! Byte access goes through `benilla-bytes` (decision 0064), migrated minimally: the local rd_u16/
//! rd_u32 are gone in favor of `ByteExt`, and every allocation sized from a header/block-table count
//! or a sector length — all attacker-controllable in a hostile archive — is capped by what the
//! archive file could actually hold (`capped`), so a lying header fails with [`Error::Corrupt`]
//! instead of aborting the allocator. Nothing else here changes.
//! [`Archive`] is a cheap `Arc` handle; each read opens its own file handle, so reads need no lock.
//! Every allocation sized from a header count or a sector length is capped by the file's size, so
//! a lying header fails with [`Error::Corrupt`].
use std::fs::File;
use std::io::{self, Read, Seek, SeekFrom};
@@ -44,24 +20,22 @@ use benilla_bytes::{capped, ByteExt};
mod crypto;
use crypto::{decrypt_block, hash_string, hash_type};
/// `MPQ\x1A` — the archive header signature (little-endian u32).
/// `MPQ\x1A` as a little-endian `u32`: the archive header signature.
const SIGNATURE: u32 = 0x1A51_504D;
/// `MPQ\x1B` — the user-data header signature (some archives offset the real header past it).
/// `MPQ\x1B`: a user-data header, which points past itself to the real one.
const USERDATA_SIGNATURE: u32 = 0x1B51_504D;
// Block-table entry flags (only the ones the 1.12.1 envelope can present; others are asserted absent).
const FLAG_IMPLODE: u32 = 0x0000_0100;
const FLAG_COMPRESS: u32 = 0x0000_0200;
const FLAG_ENCRYPTED: u32 = 0x0001_0000;
const FLAG_SINGLE_UNIT: u32 = 0x0100_0000;
/// A patch-archive tombstone: the path is deleted from the composite chain (size 0). Recognised so a
/// tombstone reads as [`Error::NotFound`], not an empty buffer (decision 0246).
/// A patch archive's delete marker (size 0): the path is deleted from the composite chain.
const FLAG_DELETE_MARKER: u32 = 0x0200_0000;
const FLAG_EXISTS: u32 = 0x8000_0000;
/// Hash-table sentinels.
const HASH_EMPTY: u32 = 0xFFFF_FFFF; // slot never used — stop probing
const HASH_DELETED: u32 = 0xFFFF_FFFE; // slot deleted — skip, keep probing
const HASH_EMPTY: u32 = 0xFFFF_FFFF; // never used: ends the probe
const HASH_DELETED: u32 = 0xFFFF_FFFE; // deleted: skip, keep probing
/// One decrypted hash-table entry (16 bytes on disk).
#[derive(Clone, Copy)]
@@ -71,8 +45,7 @@ struct HashEntry {
block_index: u32,
}
/// One decrypted block-table entry (16 bytes on disk). The on-disk `compressed_size` (word 1) is
/// skipped — sector sizes come from the offset table, so it's never needed for reading.
/// One decrypted block-table entry (16 bytes on disk).
#[derive(Clone, Copy)]
struct BlockEntry {
file_pos: u32,
@@ -80,39 +53,33 @@ struct BlockEntry {
flags: u32,
}
/// The immutable, parsed-once index of an archive: where the data starts, the sector size, and the
/// decrypted hash/block tables. Shared behind an `Arc` so [`Archive`] clones are cheap and reads borrow
/// it without locking.
/// An archive's index, parsed once and immutable, shared behind an `Arc`.
struct Index {
path: PathBuf,
/// Byte offset of the MPQ header within the file (0 for vanilla, but found, not assumed).
/// Byte offset of the MPQ header in the file: 0 in 1.12 data, but found, not assumed.
archive_offset: u64,
/// `512 << block_size` — the uncompressed sector size.
/// The uncompressed sector size, `512 << block_size`.
sector_size: usize,
hash_table: Vec<HashEntry>,
block_table: Vec<BlockEntry>,
}
/// An open MPQ archive: a cheap, `Clone`, `Send + Sync` handle over a shared [`Index`]. Reads open a
/// fresh OS handle, so they are `&self` and lock-free.
/// An open MPQ archive, a cheap `Clone + Send + Sync` handle.
#[derive(Clone)]
pub struct Archive {
index: Arc<Index>,
}
/// Errors a read can produce. Anything outside the verified 1.12.1 envelope (encrypted file, unknown
/// codec, …) surfaces here rather than being papered over.
/// A read error; anything outside the 1.12.1 envelope surfaces here.
#[derive(Debug)]
pub enum Error {
Io(io::Error),
NotMpq,
NotFound(String),
/// A file or codec shape the 1.12.1 envelope shouldn't contain — investigate, don't guess.
/// A file or codec shape 1.12.1 data does not contain.
Unsupported(String),
Decompress(String),
/// A header/block-table count or sector length claims more than the archive file could possibly
/// hold. Caught by the capped reservations (decision 0064) before the allocator would; a corrupt
/// or truncated archive, not a coding bug.
/// A count or length claims more than the archive file holds: a corrupt or truncated archive.
Corrupt(String),
}
@@ -138,7 +105,6 @@ impl From<io::Error> for Error {
type Result<T> = std::result::Result<T, Error>;
/// Reinterpret a byte buffer (length a multiple of 4) as a `Vec<u32>` (little-endian) for table decrypt.
fn to_u32s(bytes: &[u8]) -> Vec<u32> {
bytes
.as_chunks::<4>()
@@ -148,40 +114,30 @@ fn to_u32s(bytes: &[u8]) -> Vec<u32> {
.collect()
}
/// Bytes actually available in the archive file from `pos` to EOF, clamped into `usize`. Table counts
/// and sector lengths are attacker-controllable header/block-table values (decision 0064); every
/// reservation sized from one of them is capped against this, so a lying value reserves at most the
/// archive's own size and the mismatch surfaces as a clean [`Error::Corrupt`], never an allocator abort.
/// Bytes from `pos` to EOF, saturating: the cap for every reservation sized from a header value.
fn avail_from(file_len: u64, pos: u64) -> usize {
usize::try_from(file_len.saturating_sub(pos)).unwrap_or(usize::MAX)
}
impl Archive {
/// Open and index an archive: find the header, then read+decrypt the hash and block tables. The
/// file handle used here is dropped on return; reads reopen their own.
/// Open an archive: find the header, then read and decrypt the hash and block tables.
pub fn open(path: impl AsRef<Path>) -> Result<Self> {
let path = path.as_ref().to_path_buf();
let mut file = File::open(&path)?;
let file_len = file.metadata()?.len();
// Find the header. Vanilla archives put it at 0, but the format allows a user-data header
// first, so scan 512-aligned boundaries for the `MPQ\x1A` signature (resolving `MPQ\x1B`).
let archive_offset = find_header(&mut file, file_len)?;
// Read the 32-byte V1 header prefix (V2/V3/V4 only extend it; the V1 fields are all we need —
// every real table offset fits in u32, so the V2 hi-block table is irrelevant here).
// The 32-byte V1 header: later versions only extend it, and every 1.12 table offset fits
// a `u32`, so the V2 hi-block table is never needed.
file.seek(SeekFrom::Start(archive_offset))?;
let mut hdr = [0u8; 32];
file.read_exact(&mut hdr)?;
// Every field below is a fixed offset within a buffer that was just `read_exact`'d to exactly
// 32 bytes, so `u16_at`/`u32_at` can only return `None` if that invariant is somehow broken —
// map it to `Error::NotMpq` (a header we can't decode isn't a valid one) rather than trust it.
let signature = hdr.u32_at(0).ok_or(Error::NotMpq)?;
debug_assert_eq!(signature, SIGNATURE);
let block_size_shift = hdr.u16_at(14).ok_or(Error::NotMpq)?;
// The shift is a raw header u16: unchecked, a corrupt value ≥ 55 overflows the `usize`
// shift (a panic in debug, a masked shift in release — both wrong). 23 (512 B << 23 = 4 GiB
// sectors) is already far beyond any real archive (1.12 data uses 3 → 4 KiB sectors).
// A raw header `u16`, capped before `512 << shift` can overflow; 1.12 data uses 3 (4 KiB
// sectors), and 23 is already 4 GiB.
if block_size_shift > 23 {
return Err(Error::Corrupt(format!(
"block size shift {block_size_shift} exceeds the sanity cap"
@@ -217,20 +173,17 @@ impl Archive {
})
}
/// Whether the archive holds `name` (case-insensitive; `/` and `\` equivalent). No I/O.
/// Whether the archive holds `name`, case-insensitive with `/` and `\` alike. No I/O.
pub fn contains(&self, name: &str) -> bool {
self.index.find(name).is_some()
}
/// The uncompressed size of `name`, if present. No I/O (read from the block table).
/// The uncompressed size of `name`, from the block table.
pub fn file_size(&self, name: &str) -> Option<u32> {
self.index.find(name).map(|b| b.file_size)
}
/// Whether `name`'s entry is a **delete-marker** (a patch tombstone) rather than a readable file.
/// [`Archive::contains`] is still `true` for one (the hash entry exists); the chain walker calls
/// this to tell "the client deleted this path" (stop, don't fall through) from "readable file"
/// (decision 0246). No I/O.
/// Whether `name` is a delete marker: a chain walker stops there instead of falling through.
pub fn is_delete_marker(&self, name: &str) -> bool {
self.index
.find(name)
@@ -242,8 +195,7 @@ impl Archive {
&self.index.path
}
/// Read and decompress a file by its internal path. `/` or `\`; case-insensitive. `&self`: opens a
/// fresh OS handle, so concurrent reads need no lock.
/// Read and decompress a file by its internal path (`/` or `\`, case-insensitive).
pub fn read_file(&self, name: &str) -> Result<Vec<u8>> {
let idx = &self.index;
let block = idx.find(name).ok_or_else(|| Error::NotFound(name.into()))?;
@@ -251,13 +203,11 @@ impl Archive {
if block.flags & FLAG_EXISTS == 0 {
return Err(Error::NotFound(name.into()));
}
// A delete-marker is a tombstone, not a file: `size == 0` with no data. Reading it as a
// stored 0-byte file would hand back an empty buffer that looks like a real (empty) file —
// the silent mislead of 0246. It is "not found" (the composite deleted this path).
// A delete marker is not an empty file: the composite deleted this path.
if block.flags & FLAG_DELETE_MARKER != 0 {
return Err(Error::NotFound(name.into()));
}
// The verified 1.12.1 envelope: no encrypted files, no single-unit. Refuse rather than guess.
// Outside the 1.12.1 envelope: refuse rather than guess.
if block.flags & FLAG_ENCRYPTED != 0 {
return Err(Error::Unsupported(format!("encrypted file {name}")));
}
@@ -271,18 +221,11 @@ impl Archive {
let compressed = block.flags & (FLAG_COMPRESS | FLAG_IMPLODE) != 0;
let implode_only = block.flags & FLAG_IMPLODE != 0 && block.flags & FLAG_COMPRESS == 0;
// `file_size` (hence `sector_count`) and each sector's `comp_len` all come from the
// block-table entry / on-disk offset table — the same attacker-controllable-header shape as
// the hash/block tables (decision 0064). Cap every reservation below by what the archive file
// could actually hold from `file_pos` on, so a corrupt entry fails cleanly instead of the
// allocator aborting.
let file_len = file.metadata()?.len();
let avail = avail_from(file_len, file_pos);
if !compressed {
// Stored: raw bytes, exactly file_size — which the archive must hold from `file_pos`
// on. A stored entry claiming more is a corrupt block table, refused before the
// buffer is sized rather than after a 4 GiB one comes back short.
// Stored: exactly `file_size` raw bytes, refused up front if the file cannot hold them.
if capped(file_size, 1, avail) < file_size {
return Err(Error::Corrupt(format!(
"{name}: stored size ({file_size}) larger than the archive"
@@ -294,9 +237,8 @@ impl Archive {
return Ok(out);
}
// Sectored read. The offset table is `sector_count + 1` u32s at file_pos; sector offsets are
// absolute from file_pos, so a SECTOR_CRC table (present only on the patch archives) sits
// between the offset table and the first sector and is skipped for free.
// Sectored: `sector_count + 1` offsets at `file_pos`, measured from it, so the patch
// archives' SECTOR_CRC table between them and the first sector is skipped for free.
let sector_count = file_size.div_ceil(idx.sector_size);
let otab_len = sector_count.checked_add(1).ok_or_else(|| {
Error::Corrupt(format!("{name}: sector count overflow ({sector_count})"))
@@ -334,11 +276,9 @@ impl Archive {
"{name}: sector {i} length ({comp_len}) larger than the archive"
)));
}
// Every earlier sector yielded at most its own `want` (`decompress` is bounded to
// `expected`), so `out.len() <= file_size` and this cannot wrap; `want > 0` in-loop.
// Each earlier sector yielded at most its own `want` (`decompress` is bounded), so
// `out.len() <= file_size` and this cannot wrap.
let want = (file_size - out.len()).min(idx.sector_size); // last sector may be short
// Two equal consecutive offsets are a mis-built table, not a sector: zero bytes can
// carry no codec byte and no payload, so refuse before anything indexes into them.
if comp_len == 0 {
return Err(Error::Corrupt(format!("{name}: sector {i} is empty")));
}
@@ -348,14 +288,13 @@ impl Archive {
file.read_exact(&mut raw)?;
if comp_len >= want {
// Not actually compressed (a sector that wouldn't shrink is stored verbatim).
// A sector that would not shrink is stored verbatim.
out.extend_from_slice(&raw[..want]);
} else if implode_only {
// IMPLODE files carry no per-sector method byte. (Verified absent in 1.12.1; kept for
// completeness so the shape is explicit rather than a silent fall-through.)
// IMPLODE sectors carry no method byte; 1.12.1 data has none.
out.extend_from_slice(&decompress(0x08, &raw, want, name)?);
} else {
// COMPRESS: leading byte is the codec mask, rest is the payload.
// COMPRESS: the leading byte is the codec mask, the rest the payload.
let method = raw[0]; // non-empty: refused above
out.extend_from_slice(&decompress(method, &raw[1..], want, name)?);
}
@@ -391,8 +330,8 @@ impl Index {
}
}
/// Scan 512-aligned boundaries for the MPQ header, resolving a `MPQ\x1B` user-data header to the real
/// one it points at. Returns the byte offset of the `MPQ\x1A` header.
/// The `MPQ\x1A` header's offset: the first 512-aligned match, or where a `MPQ\x1B` user-data
/// header points.
fn find_header(file: &mut File, file_len: u64) -> Result<u64> {
let mut off = 0u64;
while off + 4 <= file_len {
@@ -404,7 +343,7 @@ fn find_header(file: &mut File, file_len: u64) -> Result<u64> {
match u32::from_le_bytes(sig) {
SIGNATURE => return Ok(off),
USERDATA_SIGNATURE => {
// user-data header: u32 sig, u32 user_data_size, u32 header_offset (relative to here).
// User-data header: u32 sig, u32 user_data_size, u32 header_offset from here.
let mut rest = [0u8; 8];
file.read_exact(&mut rest)?;
let header_offset = u32::from_le_bytes([rest[4], rest[5], rest[6], rest[7]]) as u64;
@@ -423,10 +362,7 @@ fn read_hash_table(
count: usize,
file_len: u64,
) -> Result<Vec<HashEntry>> {
// `count` is a header u32 (decision 0064): cap the reservation by what the archive could actually
// hold from `pos` on, so a lying header can at worst reserve the file's own size. If the capped
// reservation had to shrink below `count`, the header is lying (or the file is truncated) — fail
// cleanly here rather than let the entry-building loop below index past a short `words`.
// A `count` the file cannot hold is refused before the loop indexes past a short `words`.
let avail = avail_from(file_len, pos);
let cap = capped(count, 16, avail);
if cap < count {
@@ -434,7 +370,7 @@ fn read_hash_table(
"hash table: header claims {count} entries, only room for {cap} in the archive"
)));
}
let mut bytes = vec![0u8; cap * 16]; // == count * 16, proven <= avail above — never a wild reserve
let mut bytes = vec![0u8; cap * 16]; // == count * 16, proven <= avail above
file.seek(SeekFrom::Start(pos))?;
file.read_exact(&mut bytes)?;
let mut words = to_u32s(&bytes);
@@ -445,7 +381,7 @@ fn read_hash_table(
HashEntry {
name_a: words[o],
name_b: words[o + 1],
// words[o+2] packs locale(u16)+platform(u16); 1.12.1 data is locale-neutral, ignore.
// words[o + 2] is locale and platform, unread: 1.12.1 data is locale-neutral.
block_index: words[o + 3],
}
})
@@ -458,7 +394,7 @@ fn read_block_table(
count: usize,
file_len: u64,
) -> Result<Vec<BlockEntry>> {
// Same treatment as `read_hash_table` above: `count` is an attacker-controllable header u32.
// A `count` the file cannot hold is refused, as in `read_hash_table`.
let avail = avail_from(file_len, pos);
let cap = capped(count, 16, avail);
if cap < count {
@@ -479,7 +415,7 @@ fn read_block_table(
let o = i * 4;
BlockEntry {
file_pos: words[o],
// words[o + 1] = compressed_size on disk; unused (offset table sizes sectors).
// words[o + 1] is compressed_size, unread: the offset table sizes the sectors.
file_size: words[o + 2],
flags: words[o + 3],
}
@@ -487,16 +423,13 @@ fn read_block_table(
.collect())
}
/// Decompress one sector payload by its MPQ codec mask. 1.12.1 uses ZLIB (and store, handled by the
/// caller); any other mask is an error so the differential pass flags it for us to add deliberately.
/// Decompress one sector by its codec mask; 1.12.1 data uses zlib alone.
fn decompress(method: u8, data: &[u8], expected: usize, name: &str) -> Result<Vec<u8>> {
match method {
0x02 => {
use flate2::read::ZlibDecoder;
let mut out = Vec::with_capacity(expected);
// Bounded to `expected`: a stream that inflates past its sector's slot would push the
// caller's output past `file_size` and wrap the next sector's `want` — a malformed
// sector, not a shape 1.12 data has, and never the caller's arithmetic to absorb.
// Bounded: a sector inflating past its slot would wrap the caller's next `want`.
ZlibDecoder::new(data)
.take(expected as u64)
.read_to_end(&mut out)
@@ -514,9 +447,7 @@ mod tests {
use super::*;
use std::sync::atomic::{AtomicU32, Ordering};
/// A unique path under the OS temp dir — no `tempfile` dependency (decision 0064 keeps the leaf
/// crates dependency-light), so uniqueness is hand-rolled: pid + a per-process counter, enough to
/// never collide across parallel `cargo test` threads in this crate.
/// A unique temp path (pid and a counter), so parallel tests never collide.
fn temp_path(tag: &str) -> PathBuf {
static COUNTER: AtomicU32 = AtomicU32::new(0);
let n = COUNTER.fetch_add(1, Ordering::Relaxed);
@@ -528,7 +459,7 @@ mod tests {
))
}
/// Build a minimal 32-byte V1 MPQ header (unused fields zeroed) at the start of a buffer.
/// A minimal 32-byte V1 MPQ header, unused fields zeroed.
fn header(
block_size_shift: u16,
hash_table_pos: u32,
@@ -546,8 +477,7 @@ mod tests {
h
}
/// Write `bytes` to a fresh temp file, run `Archive::open` on it, and remove the file afterwards
/// regardless of the outcome.
/// `Archive::open` over `bytes` in a temp file, removed whatever the outcome.
fn open_temp(tag: &str, bytes: &[u8]) -> Result<Archive> {
let path = temp_path(tag);
std::fs::write(&path, bytes).expect("write temp archive");
@@ -558,9 +488,7 @@ mod tests {
#[test]
fn open_rejects_hash_table_count_bigger_than_the_file() {
// A header claiming u32::MAX hash-table entries in a 40-byte file: pre-0064 this would have
// tried `vec![0u8; u32::MAX as usize * 16]` (~68 GiB) and aborted the allocator. It must now
// return a clean `Error::Corrupt` instead.
// u32::MAX hash-table entries in a 40-byte file, ~68 GiB if the reservation were uncapped.
let hdr = header(0, 32, 32, u32::MAX, 0);
let mut bytes = hdr.to_vec();
bytes.extend_from_slice(&[0u8; 8]); // pad past the header so `pos` (32) is in-bounds
@@ -585,8 +513,7 @@ mod tests {
#[test]
fn open_accepts_a_valid_archive_with_empty_tables() {
// The boundary case the cap must not reject: zero-entry tables pointing exactly at EOF. Proves
// the new cap logic doesn't disturb a legitimately-shaped (if degenerate) archive.
// Zero-entry tables pointing exactly at EOF: the boundary the cap must not reject.
let hdr = header(0, 32, 32, 0, 0);
let archive =
open_temp("empty_tables", &hdr).expect("a well-formed empty-table archive must open");
@@ -594,8 +521,7 @@ mod tests {
assert_eq!(archive.file_size("anything"), None);
}
/// Like [`open_temp`] but leaves the backing file on disk so [`Archive::read_file`] (which reopens
/// the archive by path) works; returns the path for the caller to remove.
/// Like [`open_temp`], but the file stays for `read_file`, which reopens it by path.
fn open_temp_kept(tag: &str, bytes: &[u8]) -> (Archive, PathBuf) {
let path = temp_path(tag);
std::fs::write(&path, bytes).expect("write temp archive");
@@ -603,18 +529,15 @@ mod tests {
(archive, path)
}
/// Build a minimal V1 archive holding exactly one entry (`name`) with the given block `flags` and
/// stored `data`. Tables are encrypted with the real keys so the reader's decrypt round-trips
/// them — enough to exercise flag handling (delete-marker vs. a plain stored file).
/// A minimal V1 archive holding one stored entry, its tables encrypted with the real keys.
fn archive_with_one_entry(name: &str, flags: u32, data: &[u8]) -> Vec<u8> {
archive_with_one_block(name, flags, data, data.len() as u32)
}
/// [`archive_with_one_entry`] with the block entry's `file_size` set independently of the
/// bytes actually appended — the shape of a lying block table.
/// [`archive_with_one_entry`] with `file_size` set apart from the data: a lying block table.
fn archive_with_one_block(name: &str, flags: u32, data: &[u8], file_size: u32) -> Vec<u8> {
use crypto::{encrypt_block, hash_type};
const HASH_SLOTS: u32 = 4; // power of two — the reader masks with len-1
const HASH_SLOTS: u32 = 4; // a power of two: the reader masks with len - 1
let hash_pos = 32u32;
let block_pos = hash_pos + HASH_SLOTS * 16;
let data_pos = block_pos + 16; // one 16-byte block entry
@@ -625,7 +548,7 @@ mod tests {
hash[slot * 4] = hash_string(name, hash_type::NAME_A);
hash[slot * 4 + 1] = hash_string(name, hash_type::NAME_B);
hash[slot * 4 + 2] = 0; // locale | platform
hash[slot * 4 + 3] = 0; // → block index 0
hash[slot * 4 + 3] = 0; // block index 0
encrypt_block(&mut hash, hash_string("(hash table)", hash_type::FILE_KEY));
// Block table (plaintext): one entry pointing at the trailing data.
@@ -643,8 +566,7 @@ mod tests {
bytes
}
/// The builder is honest: a plain stored (EXISTS-only) entry reads its bytes back and is not a
/// delete-marker — so the negative result in the next test can't be a broken-archive artifact.
/// The control for the delete-marker test: the same builder's plain entry reads back.
#[test]
fn stored_entry_reads_its_bytes() {
let (arc, path) = open_temp_kept(
@@ -657,10 +579,6 @@ mod tests {
let _ = std::fs::remove_file(&path);
}
/// A delete-marker (`EXISTS | DELETE_MARKER`, size 0) is a tombstone, not an empty file: it is
/// still `contains`ed (the hash entry exists) and flagged `is_delete_marker`, but `read_file`
/// refuses it with `NotFound` rather than handing back an empty buffer (decision 0246 — the silent
/// mislead that made the trainer arc quote a stale, deleted stock file).
#[test]
fn delete_marker_is_not_a_readable_empty_file() {
let name = "Interface\\FrameXML\\ClassTrainerFrame.xml";
@@ -670,8 +588,7 @@ mod tests {
);
assert!(arc.contains(name), "the hash entry exists");
assert!(arc.is_delete_marker(name), "flagged as a tombstone");
// The file is present on disk, so a NotFound here is the tombstone guard firing — not a
// missing backing file.
// The backing file exists, so NotFound here is the delete-marker guard.
match arc.read_file(name) {
Err(Error::NotFound(_)) => {}
other => panic!("expected NotFound for a tombstone, got {other:?}"),
@@ -681,19 +598,15 @@ mod tests {
#[test]
fn avail_from_never_underflows_when_pos_exceeds_file_len() {
// `pos` derived from a corrupt header can point past EOF; the helper must saturate to 0
// rather than wrap `file_len - pos` around to a huge usize.
assert_eq!(avail_from(10, 100), 0);
assert_eq!(avail_from(100, 10), 90);
assert_eq!(avail_from(0, 0), 0);
}
/// A sector offset table with two equal consecutive offsets names a zero-length sector; the
/// codec-byte read (`raw[0]`) indexed that empty buffer and panicked on the first read of the
/// file. A private server's mis-built `patch-*.MPQ` is the realistic source.
/// Two equal consecutive offsets name a zero-length sector, with no codec byte to read.
#[test]
fn an_empty_sector_is_refused_not_indexed() {
// One 8-byte file, one sector: offsets [8, 8] — zero bytes for a sector that must yield 8.
// One 8-byte file, one sector: offsets [8, 8].
let mut data = Vec::new();
data.extend_from_slice(&8u32.to_le_bytes());
data.extend_from_slice(&8u32.to_le_bytes());
@@ -708,9 +621,6 @@ mod tests {
let _ = std::fs::remove_file(&path);
}
/// A sector that inflates past its own slot used to run the output past `file_size`, and the
/// next sector's `want = file_size - out.len()` wrapped (a panic in dev, a `usize::MAX`
/// reservation in release). The inflate is bounded to the sector's expected size instead.
#[test]
fn an_over_inflating_sector_cannot_run_the_output_past_file_size() {
use flate2::{write::ZlibEncoder, Compression};
@@ -721,8 +631,8 @@ mod tests {
e.write_all(raw).unwrap();
e.finish().unwrap()
};
// 612 bytes of file = one full 512-byte sector (shift 0) + a 100-byte tail. Sector 0's
// stream inflates to 1000 bytes, twice its slot.
// 612 bytes: a full 512-byte sector (shift 0) and a 100-byte tail; sector 0 inflates to
// 1000 bytes, twice its slot.
let s0 = zlib(&[0u8; 1000]);
let s1 = zlib(&[7u8; 100]);
let otab_len = 12u32;
@@ -745,10 +655,7 @@ mod tests {
let _ = std::fs::remove_file(&path);
}
/// A stored (uncompressed) entry's buffer was sized straight from the block table's
/// `file_size` — `vec![0u8; 4 GiB]` before the read that would have come back short. It is
/// now refused against what the archive holds from the entry's position, like the sectored
/// path's reservations; `Corrupt`, not the read's `Io`, pins that the guard fired first.
/// `Corrupt`, not the read's `Io`, pins that the guard fires before the buffer is sized.
#[test]
fn a_stored_size_larger_than_the_archive_is_refused_before_allocating() {
let (arc, path) = open_temp_kept(
crates/benilla-srp/examples/provision_probe.rs
+6 -11
View File
@@ -1,18 +1,13 @@
//! Provision a probe account for the local vmangos `realmd` DB (decision 0450: a scripted run
//! logs in as `probeN`/`pprobeN`/`Probe<N-spelled>`, one account per checkout that runs probes).
//! A new checkout means provisioning its account first; this is that recipe as one command
//! (decision 0530 — a checkout once went without it, and had no working identity).
//! Provision a probe account (`probeN`, password `pprobeN`, character `Probe<N spelled>`) in the
//! local vmangos `realmd` DB, one per checkout that runs probes. It touches no DB: it prints the
//! SQL and the character-create line to run.
//!
//! No DB access here — it prints the SQL and the char-create probe line; run those yourself.
//!
//! Validate the stored-hex convention against an existing row first (0450's discipline):
//! Check the hex convention against an existing row first, then emit the new slot's row:
//! cargo run -p benilla-srp --example provision_probe -- check <USER> <PASS> <s-hex> <v-hex>
//! then emit the new slot's row:
//! cargo run -p benilla-srp --example provision_probe -- emit <N>
//!
//! Hex convention (validated against PROBE0's stored row, matching vmangos `BigNumber`): the DB
//! stores `v`/`s` as big-endian hex of the number; the salt bytes fed to SHA1 are that number's
//! little-endian bytes.
//! The DB stores `v` and `s` as the big-endian hex of the number, as vmangos `BigNumber` writes it;
//! SRP hashes the salt as that number's little-endian bytes.
use benilla_srp::{generate_account, password_verifier, NormalizedString};
crates/benilla-srp/src/lib.rs
+66 -152
View File
@@ -1,40 +1,19 @@
//! SRP6 client + vanilla world-header crypto for **WoW 1.12.1 (build 5875)** — in-repo, replacing
//! `wow_srp` (decision 0021).
//!
//! WoW uses a lightly customised SRP6 for logon: a fixed 32-byte safe prime `N`, generator `g = 7`,
//! multiplier `k = 3`, and a bespoke "interleave" that folds the shared secret `S` into the 40-byte
//! session key. We implement only the **client** side (the realmd handshake) plus the world server's
//! header obfuscation — the only pieces benilla needs.
//!
//! All byte arrays are **little endian** on the wire (as the client sends them).
//!
//! Proven by a full SRP6 round-trip against `wow_srp`'s server + a byte-exact header-cipher diff
//! during the decision-0021 migration (oracle test in git history); ongoing regression coverage is
//! the oracle-free known-answer + golden tests in this crate.
//! SRP6 client and world-header crypto for 1.12.1 (build 5875): the client side of the realmd
//! handshake and the world server's header obfuscation. Every byte array is little endian, as on
//! the wire.
//!
//! # Encoding-unambiguous handshakes
//!
//! The values the handshake feeds to SHA-1 (`A`, `B`, `K`, the salt, `M1`) are *numbers*, and the two
//! implementations in the wild serialize a number differently:
//! The 1.12.1 client hashes `A`, `B`, `K`, the salt and `M1` at their declared widths, zero-padded
//! (`0x5d3650`); vmangos (`SHA1::Generator::UpdateData(BigNumber const&)`) and cmangos
//! (`Sha1Hash::UpdateBigNumbers`) drop high-order zero bytes, so a value with one hashes
//! differently and realmd answers `WOW_FAIL_UNKNOWN_ACCOUNT` (0x04) to a correct password. The low
//! end of `S` has the same split ([`calculate_interleaved`]).
//!
//! - The **1.12.1 client** writes each at its declared width, zero-padded in the high bytes — `A`/`B`
//! 32, `K` 40, `M1` 20 (byte-exact from `WoW.exe` `0x5d3650`).
//! - The **mangos family** (vmangos `SHA1::Generator::UpdateData(BigNumber const&)`, cmangos
//! `Sha1Hash::UpdateBigNumbers`) writes `BigNumber::AsByteArray()` with no minimum — **high-order
//! zero bytes dropped**.
//!
//! They agree only while no value happens to have a high-order zero byte, and disagree silently when
//! one does: realmd answers `WOW_FAIL_UNKNOWN_ACCOUNT` (0x04) to a perfectly correct password. There
//! is a matching disagreement at the other end of `S` (see [`calculate_interleaved`]). Measured
//! against a live vmangos, a client that picks a side loses ~1 handshake in 45 — see
//! `benilla-protocol`'s `srp_encoding_probe` example, which forces each case and prints the verdict.
//!
//! benilla picks **neither side**. `a` is ours to draw, and every ambiguous value is downstream of
//! it, so [`SrpClientChallenge::new`] simply redraws until the handshake it is about to send is one
//! both conventions serialize identically — see [`is_width_stable`]. What we send is then bit-for-bit
//! the real client's arithmetic *and* accepted by a mangos-family server, with no branch on which
//! kind of server we are talking to. The one value we cannot draw is the server's `B`; `benilla-
//! protocol`'s `logon` redials for a fresh challenge when that one lands ambiguous.
//! Deviation: [`SrpClientChallenge::new`] redraws `a` until every value it sends serializes the
//! same both ways ([`is_width_stable`]), and `benilla-protocol`'s `logon` redials when the server's
//! `B` lands ambiguous, because a client that picks either serialization fails about 1 handshake
//! in 45 against vmangos. The arithmetic stays the client's; only an ambiguous draw is declined.
use num_bigint::BigInt;
use rand::{thread_rng, RngCore};
@@ -44,25 +23,24 @@ pub mod vanilla_header;
pub use vanilla_header::{DecrypterHalf, EncrypterHalf, HeaderCrypto, ProofSeed};
/// Session-key length in bytes — always 40 (two concatenated SHA-1 hashes).
/// Session-key length in bytes: two interleaved SHA-1 digests.
pub const SESSION_KEY_LENGTH: usize = 40;
/// Proof (`M1`/`M2`) length in bytes — a SHA-1 hash.
/// Proof (`M1`, `M2`) length in bytes: one SHA-1 digest.
pub const PROOF_LENGTH: usize = 20;
/// Public-key (`A`/`B`) length in bytes.
pub const PUBLIC_KEY_LENGTH: usize = 32;
/// Salt length in bytes.
pub const SALT_LENGTH: usize = 32;
/// Generator `g` — statically 7 for WoW.
/// The generator `g`.
pub const GENERATOR: u8 = 7;
/// The WoW safe prime `N`, little endian (as sent in `CMD_AUTH_LOGON_CHALLENGE_Server`).
/// The safe prime `N`, little endian, as `CMD_AUTH_LOGON_CHALLENGE_Server` sends it.
pub const LARGE_SAFE_PRIME_LITTLE_ENDIAN: [u8; 32] = [
0xb7, 0x9b, 0x3e, 0x2a, 0x87, 0x82, 0x3c, 0xab, 0x8f, 0x5e, 0xbf, 0xbf, 0x8e, 0xb1, 0x1, 0x8,
0x53, 0x50, 0x6, 0x29, 0x8b, 0x5b, 0xad, 0xbd, 0x5b, 0x53, 0xe1, 0x89, 0x5e, 0x64, 0x4b, 0x89,
];
/// The SRP multiplier `k` — statically 3 in this (pre-SRP6a) flavour.
/// The SRP multiplier `k`, fixed at 3 (SRP6, not 6a).
const K_VALUE: u8 = 3;
// --- bigint helpers (little-endian, unsigned magnitude) -------------------------------------------
// --- bigint helpers (little-endian, unsigned magnitude) ------------------------------------------
fn from_le(bytes: &[u8]) -> BigInt {
BigInt::from_bytes_le(num_bigint::Sign::Plus, bytes)
@@ -84,19 +62,16 @@ fn sha1(parts: &[&[u8]]) -> [u8; 20] {
h.finalize().into()
}
/// Does this little-endian value hash the same whichever serialization the peer uses — its declared
/// width (the 1.12.1 client) or its minimal length (the mangos family)? True exactly when the
/// high-order byte is non-zero, since minimal encoding differs from padded only by dropping those.
/// See the crate docs, "Encoding-unambiguous handshakes".
/// Whether a little-endian value hashes the same at its declared width and at minimal length:
/// exactly when its high-order byte is non-zero.
fn is_width_stable(little_endian: &[u8]) -> bool {
matches!(little_endian.last(), Some(&b) if b != 0)
}
// --- normalized string ----------------------------------------------------------------------------
// --- normalized string ---------------------------------------------------------------------------
/// A username/password normalised the way the 1.12 client does it: ASCII only (no control chars),
/// uppercased, 1..=16 bytes. The SRP6 hashes are computed over this form, and the uppercased account
/// name is what's sent on the wire.
/// A username or password as the 1.12 client normalises it: ASCII without control characters,
/// uppercased, 1..=16 bytes. SRP6 hashes this form, and the account name goes on the wire in it.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct NormalizedString {
s: String,
@@ -122,7 +97,6 @@ impl std::fmt::Display for NormalizedStringError {
impl std::error::Error for NormalizedStringError {}
impl NormalizedString {
/// Validate + uppercase `s`. See [`NormalizedString`].
pub fn new(s: impl AsRef<str>) -> Result<Self, NormalizedStringError> {
let s = s.as_ref();
if s.is_empty() || s.len() > 16 {
@@ -145,7 +119,7 @@ impl AsRef<str> for NormalizedString {
}
}
// --- public key -----------------------------------------------------------------------------------
// --- public key ----------------------------------------------------------------------------------
/// A validated SRP public key (`A` or `B`), stored little endian. Rejected if it is exactly zero or
/// exactly the safe prime `N` (the only 32-byte values that are `0 mod N`).
@@ -157,7 +131,6 @@ pub struct PublicKey {
/// Error from [`PublicKey::from_le_bytes`].
#[derive(Debug)]
pub enum InvalidPublicKeyError {
/// The key is all zeros.
IsZero,
/// The key is `0 mod N` (equal to the safe prime).
ModLargeSafePrimeIsZero,
@@ -174,10 +147,8 @@ impl std::fmt::Display for InvalidPublicKeyError {
impl std::error::Error for InvalidPublicKeyError {}
impl PublicKey {
/// Build from little-endian bytes, validating the key (see [`PublicKey`]).
pub fn from_le_bytes(key: [u8; 32]) -> Result<Self, InvalidPublicKeyError> {
// Valid unless every byte is either 0 or matches N[i] — i.e. the key is exactly 0 or exactly
// N. (A multiple of N ≥ 2N needs 33 bytes, so those are the only two zero residues.)
// A byte-wise test for 0 and `N`, the only 32-byte multiples of `N` (`2N` needs 33 bytes).
let only_zero_or_prime = key
.iter()
.zip(LARGE_SAFE_PRIME_LITTLE_ENDIAN.iter())
@@ -192,14 +163,12 @@ impl PublicKey {
Ok(Self { key })
}
/// The key as little-endian bytes.
pub const fn as_le_bytes(&self) -> &[u8; 32] {
&self.key
}
/// Whether this key hashes identically under both serialization conventions (crate docs,
/// "Encoding-unambiguous handshakes"). [`SrpClientChallenge::new`] guarantees it for the `A` it
/// draws; for the server's `B` it is luck, and the caller's cue to redial for a fresh challenge.
/// Whether this key hashes the same under both conventions; for the server's `B`, false is the
/// cue to redial for a fresh challenge.
pub const fn is_width_stable(&self) -> bool {
self.key[31] != 0
}
@@ -209,10 +178,9 @@ impl PublicKey {
}
}
// --- SRP6 client ----------------------------------------------------------------------------------
// --- SRP6 client ---------------------------------------------------------------------------------
/// `H( SHA1(N) XOR SHA1(g) )` — folded into the client proof `M1`. Computed from the server-supplied
/// `g`/`N` (they're fixed for WoW, but we don't assume it).
/// `SHA1(N) XOR SHA1(g)`, the head of `M1`, from the `g` and `N` the server sent.
fn xor_hash(generator: u8, large_safe_prime: &[u8; 32]) -> [u8; 20] {
let n_hash = sha1(&[large_safe_prime]);
let g_hash = sha1(&[&[generator]]);
@@ -245,19 +213,12 @@ fn calculate_u(client_public_key: &PublicKey, server_public_key: &PublicKey) ->
])
}
/// Fold the shared secret `S` (32 LE bytes) into the 40-byte session key: split the even / odd bytes,
/// SHA-1 each half, then interleave the two digests. This is WoW's specific `SHA1_Interleave`.
/// Fold `S` (32 LE bytes) into the 40-byte session key: SHA-1 the even and the odd bytes apart and
/// interleave the two digests (WoW's `SHA1_Interleave`).
///
/// This is the crate-doc encoding split again, at the *low* end of `S`. The SRP-6 RFC strips leading
/// zero bytes before the split and the real client does too (from `WoW.exe` `0x5d3360`, bit-exact)
/// — leading in its little-endian `S` meaning the **low** bytes. vmangos does not: it hashes all 32
/// unconditionally (`SRP6::HashSessionKey`, `S.AsByteArray(32)`). A trim would derive a different
/// `K`, hence a different `M1`, for the ~1-in-256 `S` ending in a zero low byte — an intermittent
/// `WOW_FAIL_UNKNOWN_ACCOUNT` (0x04) on a correct password.
///
/// We hash all 32 bytes, and [`SrpClientChallenge::new`] only keeps an `S` whose low byte is
/// non-zero — which makes the strip a no-op, so this `K` is simultaneously the real client's and
/// vmangos'. Neither convention is chosen; the ambiguous inputs are simply never presented.
/// The client strips leading zero bytes first, the low bytes of a little-endian `S` (`0x5d3360`);
/// vmangos hashes all 32 (`SRP6::HashSessionKey`, `S.AsByteArray(32)`). This hashes all 32, and
/// [`SrpClientChallenge::new`] never keeps an `S` with a zero low byte, so this `K` is both.
fn calculate_interleaved(s: &[u8; 32]) -> [u8; 40] {
let mut e = [0u8; 16];
for (i, b) in s.iter().step_by(2).enumerate() {
@@ -279,7 +240,7 @@ fn calculate_interleaved(s: &[u8; 32]) -> [u8; 40] {
out
}
/// `M2 = SHA1( A | M1 | K )` — what the server proves back and we verify.
/// `M2 = SHA1( A | M1 | K )`, the server's proof back.
fn calculate_server_proof(
client_public_key: &PublicKey,
client_proof: &[u8; 20],
@@ -288,9 +249,8 @@ fn calculate_server_proof(
sha1(&[client_public_key.as_le_bytes(), client_proof, session_key])
}
/// First step of the client logon: given the server's challenge values, computes our public key `A`,
/// the proof `M1`, and the session key. Send `A` + `M1` in `CMD_AUTH_LOGON_PROOF_Client`, then call
/// [`SrpClientChallenge::verify_server_proof`] with the server's `M2`.
/// The client's side of the logon proof: `A` and `M1` go in `CMD_AUTH_LOGON_PROOF_Client`, and the
/// server's `M2` is checked with [`SrpClientChallenge::verify_server_proof`].
#[derive(Debug, Clone)]
pub struct SrpClientChallenge {
username: NormalizedString,
@@ -299,22 +259,17 @@ pub struct SrpClientChallenge {
session_key: [u8; 40],
}
/// How many ephemerals [`SrpClientChallenge::new`] will draw looking for an encoding-unambiguous
/// handshake. Each draw succeeds ~97.4% of the time, so the loop all but always ends on the first;
/// the bound only exists so a degenerate `N`/`g` from a hostile server cannot spin forever. On
/// exhaustion we send the last draw anyway — the same handshake we would have sent before this
/// guarantee existed, i.e. it can fail the logon but cannot corrupt one.
/// The cap on ephemerals [`SrpClientChallenge::new`] draws, so a hostile `N`/`g` cannot spin it
/// forever. Each passes about 97.4% of the time; the last is sent regardless, which can fail the
/// logon but never corrupt it.
const MAX_EPHEMERAL_DRAWS: u32 = 512;
impl SrpClientChallenge {
/// Compute `A`, `M1`, and the session key from the server challenge. Mirrors the real client:
/// random 32-byte private key `a`, `A = g^a mod N`, `S = (B - k·g^x)^(a + u·x) mod N`, session
/// key = interleave(S), `M1 = SHA1( H(N)^H(g) | SHA1(user) | salt | A | B | K )`.
///
/// `a` is redrawn until every value the handshake serializes is one both conventions in the wild
/// encode identically (crate docs, "Encoding-unambiguous handshakes"): `A`, `K` and `M1` with no
/// high-order zero byte, and `S` with no low-order one. The arithmetic is untouched — this only
/// declines to *use* an ephemeral whose handshake would be read two ways.
/// Compute `A`, `M1` and the session key as the client does: a random 32-byte `a`,
/// `A = g^a mod N`, `S = (B - k·g^x)^(a + u·x) mod N`, `K = interleave(S)` and
/// `M1 = SHA1( H(N)^H(g) | SHA1(user) | salt | A | B | K )`. Deviation: `a` is redrawn until
/// `A`, `K` and `M1` have no high-order zero byte and `S` no low-order one, because the client
/// and vmangos hash those values differently.
pub fn new(
username: NormalizedString,
password: NormalizedString,
@@ -334,10 +289,7 @@ impl SrpClientChallenge {
)
}
/// [`Self::new`] with the ephemeral's draw source injected. `new` hands it `thread_rng`; the
/// tests hand it a scripted one, so each guard in the loop below is exercised on purpose —
/// a draw known to trip it, then a clean one — rather than by the ~1-in-137 chance a random
/// sweep gives each (decision 2331).
/// [`Self::new`] with the draw source injected, so a test can script a draw that trips a guard.
fn new_with_rng<R: RngCore>(
rng: &mut R,
username: NormalizedString,
@@ -351,7 +303,6 @@ impl SrpClientChallenge {
let g = BigInt::from(generator);
let k = BigInt::from(K_VALUE);
// Everything the draw does not move, hoisted out of the loop.
let x = from_le(&calculate_x(&username, &password, &salt));
let s_base = server_public_key.as_bigint() - &k * g.modpow(&x, &n);
let xor = xor_hash(generator, &large_safe_prime);
@@ -364,7 +315,6 @@ impl SrpClientChallenge {
rng.fill_bytes(&mut private_key);
let a = from_le(&private_key);
// A = g^a mod N
let client_public_key = to_padded_32_le(&g.modpow(&a, &n));
if !is_width_stable(&client_public_key) && !last {
continue;
@@ -376,8 +326,7 @@ impl SrpClientChallenge {
// S = (B - k·(g^x mod N))^(a + u·x) mod N
let s = to_padded_32_le(&s_base.modpow(&(&a + &u * &x), &n));
// A zero low byte is the one the real client's interleave would strip (see
// `calculate_interleaved`) — decline it so our K is both implementations' K.
// The client's interleave strips a zero low byte: decline it, so `K` is both sides'.
if s[0] == 0 && !last {
continue;
}
@@ -387,7 +336,6 @@ impl SrpClientChallenge {
continue;
}
// M1 = SHA1( xor_hash | SHA1(username) | salt | A | B | K )
let client_proof = sha1(&[
&xor,
&username_hash,
@@ -396,8 +344,7 @@ impl SrpClientChallenge {
server_public_key.as_le_bytes(),
&session_key,
]);
// M1 is itself hashed back into the server's M2, so it needs the guarantee too — this is
// the case realmd *accepts* while the M2 it replies with fails our check.
// M1 feeds the server's M2: an ambiguous one passes realmd but fails our M2 check.
if !is_width_stable(&client_proof) && !last {
continue;
}
@@ -412,18 +359,17 @@ impl SrpClientChallenge {
unreachable!("the final draw is accepted unconditionally")
}
/// Our proof `M1` (little endian) — send in `CMD_AUTH_LOGON_PROOF_Client`.
/// Our proof `M1`, little endian.
pub const fn client_proof(&self) -> &[u8; 20] {
&self.client_proof
}
/// Our public key `A` (little endian) — send in `CMD_AUTH_LOGON_PROOF_Client`.
/// Our public key `A`, little endian.
pub const fn client_public_key(&self) -> &[u8; 32] {
&self.client_public_key
}
/// Verify the server's proof `M2`. On success the SRP handshake is complete and the session key
/// is held in the returned [`SrpClient`].
/// Verify the server's proof `M2`; the returned [`SrpClient`] holds the session key.
pub fn verify_server_proof(
self,
server_proof: [u8; 20],
@@ -450,9 +396,7 @@ impl SrpClientChallenge {
/// The server's proof `M2` did not match what we computed (usually a wrong password).
#[derive(Debug)]
pub struct MatchProofsError {
/// The proof the server sent.
pub server_proof: [u8; 20],
/// The proof we expected.
pub expected: [u8; 20],
}
impl std::fmt::Display for MatchProofsError {
@@ -477,11 +421,9 @@ impl SrpClient {
}
}
// --- account creation (server-side verifier) ------------------------------------------------------
// --- account creation (server-side verifier) -----------------------------------------------------
/// The SRP6 **password verifier** `v = g^x mod N` (little endian) for a given salt — the value a
/// server stores alongside the salt so it never holds the raw password. Deterministic in
/// `(username, password, salt)`. See [`generate_account`] for the new-account path.
/// The SRP6 password verifier `v = g^x mod N`, little endian: what a server stores beside the salt.
pub fn password_verifier(
username: &NormalizedString,
password: &NormalizedString,
@@ -493,14 +435,9 @@ pub fn password_verifier(
to_padded_32_le(&g.modpow(&x, &n))
}
/// Generate a fresh random salt and the matching [`password_verifier`] for a new account. Returns
/// `(salt, verifier)`, both little endian.
///
/// The salt's top bit is forced, exactly as vmangos' own account creation does it
/// (`BigNumber::SetRand(256)` → `BN_rand(_bn, 256, 0, 1)`, whose `top = 0` sets the MSB). The salt is
/// the one hashed value fixed for the life of the account rather than redrawn per handshake, so a
/// high-order zero byte there is not a 1-in-256 *login* failure but a permanently unloggable account
/// (crate docs, "Encoding-unambiguous handshakes").
/// A fresh random salt and its [`password_verifier`] for a new account, both little endian. The
/// salt's top bit is set, as vmangos's `BigNumber::SetRand(256)` does (`BN_rand(_bn, 256, 0, 1)`):
/// a salt lives as long as its account, so an ambiguous one would never log in.
pub fn generate_account(
username: &NormalizedString,
password: &NormalizedString,
@@ -514,10 +451,8 @@ pub fn generate_account(
#[cfg(test)]
mod tests {
//! Oracle-free regression tests. The SRP6 *interleave* and `x` derivation are pinned to published
//! WoW SRP6 test vectors (the gtker/wow_srp corpus, MIT — the same values used to byte-validate
//! this crate against `wow_srp` during the decision-0021 migration). The header cipher + password
//! verifier are pinned to golden outputs captured from that validated implementation.
//! The interleave and `x` are pinned to the published WoW SRP6 vectors (the `wow_srp` crate's
//! MIT corpus); the header cipher and password verifier to goldens from that implementation.
use super::*;
fn hx(s: &str) -> Vec<u8> {
@@ -550,11 +485,6 @@ mod tests {
}
}
/// A low-order zero byte in `S` must *not* shorten the split: we hash all 32 bytes, so `S` and
/// `S` with its low byte zeroed differ only in that byte, never in length. (Trimming is the
/// SRP-6 RFC's — and the real client's — behaviour, and against vmangos it is what made ~1 login
/// in 256 come back as "wrong password"; `SrpClientChallenge::new` keeps the two identical by
/// never drawing an `S` with a zero low byte at all.)
#[test]
fn interleave_does_not_trim_low_order_zero_bytes() {
let mut s = [0u8; 32];
@@ -565,8 +495,7 @@ mod tests {
zeroed[0] = 0;
zeroed[1] = 0;
// Even halves: byte 0 is the only difference; odd halves: byte 1. Both must change, and the
// trimming implementation would instead have re-split the remaining 30 bytes wholesale.
// The untrimmed split, built by hand; a trim would re-split the remaining 30 bytes.
let mut expect_even = [0u8; 16];
let mut expect_odd = [0u8; 16];
for i in 0..16 {
@@ -587,8 +516,7 @@ mod tests {
#[test]
fn calculate_x_known_answer() {
// Fixed salt (big-endian in the corpus); user/pass → x (big-endian in the corpus). We store
// both little-endian, so reverse the corpus hex.
// The corpus writes the salt and `x` big-endian; we store both little-endian.
let salt: [u8; 32] = rev(hx(
"CAC94AF32D817BA64B13F18FDEDEF92AD4ED7EF7AB0E19E9F2AE13C828AEAF57",
))
@@ -623,12 +551,6 @@ mod tests {
assert!(!is_width_stable(&[]));
}
/// The crate's guarantee as a property: every handshake we hand out is one both serialization
/// conventions read identically, so no value carries a high-order zero byte. Seeded (2331), so
/// the 128 server keys and every ephemeral drawn against them are the same on every machine —
/// a red run reproduces. The per-guard proof is `each_encoding_guard_rejects_the_draw_it_exists_for`;
/// the live gate is `benilla-protocol`'s `srp_encoding_probe`, which forces each case against
/// a real realmd.
#[test]
fn every_drawn_handshake_is_encoding_unambiguous() {
use rand::{rngs::StdRng, SeedableRng};
@@ -685,9 +607,8 @@ mod tests {
}
}
/// The server side of the fixture the guard vectors were found against: a fixed `B` (with the
/// high bit `logon` keeps) and a fixed salt. Change either and the vectors below stop meaning
/// anything — the test says so.
/// The `B` (high bit set, as `logon` keeps) and salt the guard vectors were found against;
/// change either and the vectors mean nothing.
const FIXTURE_B: [u8; 32] = [
5, 42, 79, 116, 153, 190, 227, 8, 45, 82, 119, 156, 193, 230, 11, 48, 85, 122, 159, 196,
233, 14, 51, 88, 125, 162, 199, 236, 17, 54, 91, 128,
@@ -696,11 +617,9 @@ mod tests {
3, 14, 25, 36, 47, 58, 69, 80, 91, 102, 113, 124, 135, 146, 157, 168, 179, 190, 201, 212,
223, 234, 245, 0, 11, 22, 33, 44, 55, 66, 77, 88,
];
/// Private keys that, against [`FIXTURE_B`]/[`FIXTURE_SALT`], trip exactly one guard each —
/// found once by a seeded search (`StdRng::seed_from_u64(2331)`, first hit per guard) and
/// pinned. `TRIPS_A`: `A` gets a high-order zero byte. `TRIPS_S`: `S` gets a low-order zero
/// byte. `TRIPS_K`: the interleaved key gets a high-order zero. `TRIPS_M1`: the proof does.
/// `CLEAN` passes all four.
/// Private keys that each trip one guard against the fixture: `TRIPS_A` gives `A` a high-order
/// zero byte, `TRIPS_S` gives `S` a low-order one, `TRIPS_K` and `TRIPS_M1` give the key and
/// the proof a high-order one; `CLEAN` trips none.
const TRIPS_A: [u8; 32] = [
237, 250, 2, 239, 106, 197, 124, 117, 132, 5, 226, 189, 212, 217, 169, 146, 39, 212, 214,
11, 198, 57, 225, 84, 17, 219, 168, 107, 118, 105, 11, 95,
@@ -728,10 +647,7 @@ mod tests {
to_padded_32_le(&BigInt::from(GENERATOR).modpow(&from_le(private_key), &n))
}
/// The crate's guarantee, exercised on purpose (decision 2331): each guard in the draw loop is
/// handed a private key KNOWN to trip it, then a clean one, and the handshake that comes back
/// must be the clean draw's. A dropped guard fails its case outright — the old random sweep
/// caught a dropped guard with "~97 %" probability, which is a 3 % escape by design.
/// Each guard gets a key known to trip it, then a clean one, and must return the clean draw.
#[test]
fn each_encoding_guard_rejects_the_draw_it_exists_for() {
let user = NormalizedString::new("alice").unwrap();
@@ -791,8 +707,6 @@ mod tests {
assert_eq!(rng.0.len(), 1, "a clean first draw is the handshake");
}
/// A salt is fixed for the life of an account rather than redrawn per handshake, so an ambiguous
/// one is not a flaky login but an account that can never log in.
#[test]
fn generated_salts_are_encoding_unambiguous() {
let user = NormalizedString::new("alice").unwrap();
crates/benilla-srp/src/vanilla_header.rs
+18 -38
View File
@@ -1,24 +1,16 @@
//! Vanilla (1.12) world-packet **header obfuscation**.
//!
//! After `CMSG_AUTH_SESSION`, every world-packet *header* (not the body) is run through a stateful
//! byte cipher keyed by the SRP session key. It is not a real cipher — each byte is XORed with a
//! rotating session-key byte and chained against the previous ciphertext byte:
//! The 1.12 world-packet header obfuscation. After `CMSG_AUTH_SESSION` every header, not the body,
//! runs through a byte cipher keyed by the 40-byte session key, `i` cycling over the key and
//! `last_c` the previous ciphertext byte, both starting at 0:
//!
//! ```text
//! encrypt: c = (p ^ key[i]) + last_c ; decrypt: p = (c - last_c) ^ key[i]
//! ```
//!
//! with `i` cycling over the 40-byte key and `last_c` the previous ciphertext byte (both start 0).
//! Client headers are 6 bytes (a `u16` BE size and `u32` LE opcode); server headers are **always 4**
//! (a `u16` BE size and `u16` LE opcode). There is no wider-size variant in 1.12: vmangos writes
//! every outgoing header as `ServerPktHeader { uint16 size; uint16 cmd; }` (WorldSocket.cpp — a
//! larger body is *truncated* into the u16, never given a wider field) and routes big object
//! updates through `SMSG_COMPRESSED_UPDATE_OBJECT` instead. The 3-byte large-packet size is a
//! later-expansion convention; nothing in this stack handles it, by design.
//!
//! The handshake: the server's `SMSG_AUTH_CHALLENGE` carries a random seed; the client picks its own
//! seed (a [`ProofSeed`]), proves it knows the session key via
//! [`ProofSeed::into_client_header_crypto`], and from then on encrypts/decrypts headers.
//! Client headers are 6 bytes (`u16` BE size, `u32` LE opcode); server headers are always 4
//! (`u16` BE size, `u16` LE opcode). vmangos writes every header as
//! `ServerPktHeader { uint16 size; uint16 cmd; }`, truncating a larger size (`WorldSocket.cpp`),
//! and sends big object updates as `SMSG_COMPRESSED_UPDATE_OBJECT`; the 3-byte large-packet size
//! is a later expansion's.
use std::io::{Read, Write};
@@ -48,7 +40,7 @@ fn world_server_proof(
h.finalize().into()
}
/// Encryption half of the header cipher — kept with the write half of a split connection.
/// The header cipher's encrypting half, for the write side of a split connection.
#[derive(Debug, Clone)]
pub struct EncrypterHalf {
session_key: [u8; SESSION_KEY_LENGTH],
@@ -77,7 +69,6 @@ impl EncrypterHalf {
header
}
/// Write an encrypted client header to `w`.
pub fn write_encrypted_client_header<W: Write>(
&mut self,
mut w: W,
@@ -89,7 +80,7 @@ impl EncrypterHalf {
}
}
/// Decryption half of the header cipher — kept with the read half of a split connection.
/// The header cipher's decrypting half, for the read side of a split connection.
#[derive(Debug, Clone)]
pub struct DecrypterHalf {
session_key: [u8; SESSION_KEY_LENGTH],
@@ -110,8 +101,7 @@ impl DecrypterHalf {
}
}
/// Decrypt a 4-byte server header into `(size, opcode)`. (1.12 server headers are always 4
/// bytes — see the module doc; there is no wider-size variant to handle.)
/// Decrypt a 4-byte server header; 1.12 has no wider-size variant.
pub fn decrypt_server_header(&mut self, mut data: [u8; SERVER_HEADER_LENGTH]) -> ServerHeader {
self.decrypt(&mut data);
ServerHeader {
@@ -120,7 +110,6 @@ impl DecrypterHalf {
}
}
/// Read + decrypt a 4-byte server header from `r`.
pub fn read_and_decrypt_server_header<R: Read>(
&mut self,
mut r: R,
@@ -134,13 +123,12 @@ impl DecrypterHalf {
/// A decrypted server-packet header.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ServerHeader {
/// Body size in bytes, *including* the opcode field but not the size field.
/// Body size in bytes, counting the opcode field but not the size field.
pub size: u16,
/// Server opcode.
pub opcode: u16,
}
/// Both halves of the header cipher. Split with [`Self::split`] for separate read/write paths.
/// Both halves of the header cipher.
#[derive(Debug, Clone)]
pub struct HeaderCrypto {
encrypt: EncrypterHalf,
@@ -148,26 +136,21 @@ pub struct HeaderCrypto {
}
impl HeaderCrypto {
/// Mutable access to the decryption half.
pub fn decrypter(&mut self) -> &mut DecrypterHalf {
&mut self.decrypt
}
/// Mutable access to the encryption half.
pub fn encrypter(&mut self) -> &mut EncrypterHalf {
&mut self.encrypt
}
/// Split into independent encrypt / decrypt halves (one per direction of a split socket).
/// Split into halves, one per direction of a split socket.
pub fn split(self) -> (EncrypterHalf, DecrypterHalf) {
(self.encrypt, self.decrypt)
}
/// The cipher built straight from a session key, skipping the proof step — the *peer* side of
/// [`ProofSeed::into_client_header_crypto`]. Both ends run the same state machine with inverse
/// operations from (index 0, previous 0), so a server's [`EncrypterHalf`] is exactly what a
/// client's [`DecrypterHalf`] inverts. Lets a test harness stand up a fake world server that
/// speaks real encrypted headers.
/// The cipher from a session key alone, skipping the proof: a server's [`EncrypterHalf`] is
/// what a client's [`DecrypterHalf`] inverts, so a fake world server speaks real headers.
pub fn from_session_key(session_key: [u8; SESSION_KEY_LENGTH]) -> Self {
Self::new(session_key)
}
@@ -188,15 +171,13 @@ impl HeaderCrypto {
}
}
/// The client's random seed for the world handshake. Created before `CMSG_AUTH_SESSION`; turned into
/// the proof + [`HeaderCrypto`] once we know the server's seed.
/// The client's random seed for the world handshake.
#[derive(Debug, Clone, Copy)]
pub struct ProofSeed {
seed: u32,
}
impl ProofSeed {
/// A new random client seed.
#[allow(clippy::new_without_default)]
pub fn new() -> Self {
Self {
@@ -209,8 +190,7 @@ impl ProofSeed {
self.seed
}
/// Compute the client proof (binds the session key to both seeds) and the [`HeaderCrypto`] used
/// for all subsequent headers. Valid once the server replies with a successful `SMSG_AUTH_RESPONSE`.
/// The client proof, binding the session key to both seeds, and the header cipher.
pub fn into_client_header_crypto(
self,
username: &NormalizedString,
crates/benilla-visual/src/lib.rs
+63 -152
View File
@@ -1,45 +1,31 @@
//! Perceptual image-diff metrics for the Phase-5 visual A/B render harness (decision 0008).
//!
//! The harness captures deterministic screenshots of benilla (`$WOW_CAPTURE`, see the `capture` module
//! in the `benilla` crate) and diffs them. This crate is the pure-math half: given two equally-sized
//! images it reports how far they diverge ([`Metrics`]) and renders a heatmap of *where* ([`diff_image`]).
//!
//! Three uses, one tool: (1) **self-regression** — capture baselines on the current pipeline, then diff
//! every linear-HDR rework step against them so a machine catches a regression before the director's
//! eye; (2) **determinism check** — diffing two captures of the same scenario must come out ≈0, which is
//! what makes (1) trustworthy; (3) **flicker** ([`envelope`]) — the same arithmetic over *time* instead
//! of over versions: a burst of adjacent frames from a parked camera (`WOW_LIVE_SHOT_COUNT`), collapsed
//! to where the picture would not hold still.
//! Image-diff metrics for the capture harness (`WOW_CAPTURE`, the `capture` module in
//! `benilla-app`): how far two equal-size images diverge ([`Metrics`]) and where ([`diff_image`]),
//! and over a burst of frames, which pixels would not hold still ([`envelope`], [`toggles`]).
pub mod relight;
use image::RgbImage;
/// A pixel counts as "changed" for [`Metrics::pct_over`] if any channel differs by more than this many
/// byte units. 8/255 ≈ 3% — above dithering/rounding noise, below a real visual shift.
/// A pixel counts toward [`Metrics::pct_over`] when a channel differs by more than this (8/255,
/// about 3%): above dithering noise, below a real visual shift.
pub const OVER_THRESHOLD: u8 = 8;
/// Per-image difference metrics. Channel deltas are in 0..255 byte units; `pct_over` is a fraction 0..1.
/// Per-image difference metrics: channel deltas in 0..255 byte units, fractions in 0..1.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Metrics {
/// Mean absolute per-channel difference (0..255). The headline "how different" number.
/// Mean absolute per-channel difference (0..255).
pub mae: f64,
/// Root-mean-square per-channel difference (0..255) — weights large local deltas more than `mae`.
/// Root-mean-square per-channel difference (0..255), weighting large local deltas.
pub rmse: f64,
/// Largest single-channel difference anywhere (0..255).
pub max_delta: u8,
/// Fraction of pixels (0..1) whose largest channel delta exceeds [`OVER_THRESHOLD`].
pub pct_over: f64,
/// How many pixels differ **at all** (any channel, by any amount).
///
/// The number that separates "a render changed" from "a silhouette pixel landed on the other
/// side of an MSAA tie": both read as a scary `max_delta` and an `mae` of 0.000, and only the
/// count tells them apart. A handful of pixels in a 5.76 M-pixel frame is a tie flipping with
/// the binary; a region is a regression. Answering that used to take a hand-rolled PNG
/// decoder in a scratchpad, every time.
/// Pixels that differ at all: a handful is an MSAA tie flipping, a region a regression, though
/// both show a large `max_delta` and an `mae` of 0.000.
pub changed: u64,
/// Where the largest delta is, `(x, y)` — `(0, 0)` when nothing differs. A tie flip recurs at
/// the *same* coordinate build after build, which is most of the evidence that it is one.
/// Where the largest delta is, `(0, 0)` when nothing differs; a tie flip recurs at the same
/// spot build after build.
pub worst_at: (u32, u32),
}
@@ -90,9 +76,7 @@ pub fn compare(a: &RgbImage, b: &RgbImage) -> anyhow::Result<Metrics> {
})
}
/// Render an amplified per-channel abs-difference image: each output channel is `|a-b| * amplify`,
/// clamped to 255. This colourises both *where* and *in which channel* the images diverge (a red shift
/// shows red), so the diff is readable at a glance. Errors on a size mismatch.
/// The per-channel difference `|a - b| * amplify`, clamped to 255, so a red shift shows red.
pub fn diff_image(a: &RgbImage, b: &RgbImage, amplify: u32) -> anyhow::Result<RgbImage> {
if a.dimensions() != b.dimensions() {
anyhow::bail!(
@@ -114,12 +98,12 @@ pub fn diff_image(a: &RgbImage, b: &RgbImage, amplify: u32) -> anyhow::Result<Rg
Ok(out)
}
/// How far a burst of frames moved, and where — the output of [`envelope`].
/// How far a burst of frames moved, and where: [`envelope`]'s output.
#[derive(Debug, Clone)]
pub struct Envelope {
/// Per-channel `(max − min)` across the stack, amplified — the picture of *where* it flickers.
/// Per-channel `max - min` across the stack, amplified.
pub image: RgbImage,
/// Largest single-channel swing anywhere (0..255). The headline "does it flicker at all".
/// Largest single-channel swing anywhere (0..255).
pub max_swing: u8,
/// Mean per-channel swing (0..255) over the whole frame.
pub mean_swing: f64,
@@ -127,14 +111,9 @@ pub struct Envelope {
pub pct_unstable: f64,
}
/// Collapse a burst of equally-sized frames to their per-pixel **envelope** — `max − min` per channel
/// across the whole stack. This is the flicker instrument: a still image cannot show a temporal
/// artefact, but the envelope of adjacent frames from a *parked* camera can, and it localises the
/// unstable pixels in one picture instead of `n−1` pairwise diffs.
///
/// A pixel that holds still contributes 0 whatever its colour, so a scene that is merely *wrong* comes
/// out black — only what refuses to settle lights up. Errors on fewer than two frames or a size
/// mismatch.
/// A burst's per-pixel envelope, `max - min` per channel across the whole stack: from a parked
/// camera only what refuses to settle lights up, however wrong a still pixel is. Errors on fewer
/// than two frames or a size mismatch.
pub fn envelope(frames: &[RgbImage], amplify: u32) -> anyhow::Result<Envelope> {
let (w, h) = burst_dimensions(frames, 2)?;
let mut image = RgbImage::new(w, h);
@@ -169,38 +148,26 @@ pub fn envelope(frames: &[RgbImage], amplify: u32) -> anyhow::Result<Envelope> {
})
}
/// How often a burst *reversed direction*, and where — the output of [`toggles`].
/// How often a burst reversed direction, and where: [`toggles`]'s output.
#[derive(Debug, Clone)]
pub struct Toggles {
/// Per-channel reversal count across the stack, amplified — the picture of *where* it toggles.
/// Per-channel reversal count across the stack, amplified.
pub image: RgbImage,
/// Most reversals any one channel made (0..frames−2).
/// Most reversals any one channel made (0..frames-2).
pub max_reversals: u32,
/// Fraction of pixels (0..1) that reversed at least [`TOGGLE_MIN_REVERSALS`] times.
pub pct_toggling: f64,
/// Row-major `width × height` flags: did this pixel toggle? Carried out of the scan because
/// *where* the toggling pixels sit relative to each other is a separate reading ([`shape`]) and
/// re-thresholding the amplified image would only recover it approximately.
/// Row-major per-pixel toggle flags for [`shape`], exact where re-thresholding `image` is not.
pub mask: Vec<bool>,
}
/// A pixel counts as "toggling" for [`Toggles::pct_toggling`] at this many direction reversals. Two
/// is the smallest count that cannot be a single overshoot, and it already needs a four-frame
/// up-down-up run — which smooth motion across an edge does not produce.
/// Direction reversals that make a pixel toggle: two cannot be a single overshoot, and smooth
/// motion across an edge never makes the up-down-up run they need.
pub const TOGGLE_MIN_REVERSALS: u32 = 2;
/// Count, per pixel and channel, how many times the value **reversed direction** across the burst,
/// ignoring steps smaller than `min_delta`.
///
/// This is [`envelope`]'s companion for the case the envelope cannot serve: **a moving camera**.
/// When the view is panning, every edge in the frame sweeps across pixels and `max − min` lights up
/// the whole picture, so it says nothing. But smooth motion is *monotone* — a pixel being crossed by
/// an edge goes one way and stays going that way — while z-fighting and an unstable draw order
/// **alternate**, A/B/A/B, however the camera moves. Counting reversals separates the two.
///
/// The director's report is what forced this: the Far Watch Post tower "mostly only flickers while
/// moving the cam, not while still" — the one condition under which a parked burst is blind by
/// construction (decision 0653's blind spot, named in 0656).
/// Count, per pixel and channel, the direction reversals across the burst, ignoring steps under
/// `min_delta`: the flicker reading for a moving camera. Smooth motion is monotone; z-fighting and
/// an unstable draw order alternate however the camera moves.
pub fn toggles(frames: &[RgbImage], min_delta: u8, amplify: u32) -> anyhow::Result<Toggles> {
let (w, h) = burst_dimensions(frames, 3)?;
let mut image = RgbImage::new(w, h);
@@ -210,8 +177,7 @@ pub fn toggles(frames: &[RgbImage], min_delta: u8, amplify: u32) -> anyhow::Resu
for (x, y, out) in image.enumerate_pixels_mut() {
let mut pixel_max = 0u32;
for c in 0..3 {
// `dir` is the sign of the last step big enough to count; a run of sub-threshold steps
// neither reverses nor resets it, so dithering noise cannot manufacture a toggle.
// `dir` is the last counted step's sign; smaller steps neither reverse nor reset it.
let (mut dir, mut reversals) = (0i8, 0u32);
for pair in frames.windows(2) {
let (a, b) = (
@@ -246,7 +212,7 @@ pub fn toggles(frames: &[RgbImage], min_delta: u8, amplify: u32) -> anyhow::Resu
})
}
/// A rectangle of the frame, in pixels — `x1`/`y1` exclusive.
/// A rectangle of the frame in pixels, `x1` and `y1` exclusive.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Rect {
pub x0: u32,
@@ -279,10 +245,7 @@ pub struct Region {
pub bounds: Rect,
/// Toggling pixels in the run (≤ the bounds' area).
pub pixels: u64,
/// Up to [`REGION_SAMPLES`] pixels spread evenly through the run, in scan order — coordinates
/// Every pixel in the run, in scan order. Kept whole because the run's *own* time series
/// ([`Region::steps`]) is the reading that follows, and it has to be over the run, not over a
/// bounding box that also contains whatever else was passing through.
/// Every pixel in the run, in scan order; [`Region::steps`] reads these, not the bounding box.
pub members: Vec<(u32, u32)>,
}
@@ -292,33 +255,25 @@ pub const REGION_SAMPLES: usize = 8;
/// One frame-to-frame step of a run, from [`Region::steps`].
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Step {
/// The run's mean luma (0..255) in the earlier frame. The *level*, not just the change: a
/// two-state flip's ratio is what identifies the multiplier behind it, and a delta alone hides it.
/// The run's mean luma (0..255) in the earlier frame.
pub mean_from: f64,
/// The run's mean R/G/B (0..255) in the earlier frame. Whether a two-state flip preserves the
/// channel ratios says what kind of flip it is: a pure **intensity** scale (one light term
/// switching on or off) keeps the hue, while swapping to a differently-coloured lighting lane
/// does not — and luma alone cannot tell those apart.
/// The run's mean R, G and B (0..255) in the earlier frame.
pub mean_rgb_from: [f64; 3],
/// Change in the run's mean luma (0..255 units) from this frame to the next.
pub mean_delta: f64,
/// Fraction of the run's pixels (0..1) that moved the **same way** as the mean did. Near 1 = the
/// whole surface changed together; near 0.5 = as many pixels went up as down.
/// Fraction of the run's pixels (0..1) that moved the same way as the mean: near 1 the surface
/// changed together, near 0.5 as many went up as down.
pub agreement: f64,
}
impl Region {
/// How much of its own bounding box the run actually fills, 0..1. A solid surface approaches 1;
/// a scattering of speckles inside a wide box is near 0.
/// How much of its bounding box the run fills, 0..1: a solid surface nears 1, speckle 0.
pub fn fill(&self) -> f64 {
let area = u64::from(self.bounds.width()) * u64::from(self.bounds.height());
self.pixels as f64 / area.max(1) as f64
}
/// Up to [`REGION_SAMPLES`] pixels spread evenly through the run — coordinates that are
/// *provably* in it, to hand to the in-game ray pick (`WOW_PICK`). Reading them off a heatmap by
/// eye lands next to the run as often as in it, and then the pick names the wrong surface with
/// total confidence.
/// Up to [`REGION_SAMPLES`] pixels spread through the run, for the ray pick (`WOW_PICK`).
pub fn samples(&self) -> Vec<(u32, u32)> {
let step = (self.members.len() / REGION_SAMPLES).max(1);
self.members
@@ -329,14 +284,9 @@ impl Region {
.collect()
}
/// The run's own frame-to-frame time series — the reading that separates a surface being
/// **re-shaded** from one whose **edges are moving**.
///
/// [`shape`] says the toggling pixels form one coherent surface; it cannot say what the surface
/// is doing. If the whole run brightens and dims together (`agreement` near 1, `mean_delta`
/// swinging), it is being lit differently frame to frame — a shading defect. If half its pixels
/// go up while the other half go down (`agreement` near 0.5, `mean_delta` near 0), nothing is
/// being re-lit: a boundary is sweeping across it, and the run is an edge, not a surface.
/// The run's frame-to-frame series: the whole run brightening and dimming together (`agreement`
/// near 1) is a surface re-shaded; half up and half down (`agreement` near 0.5, `mean_delta`
/// near 0) is an edge sweeping across it.
pub fn steps(&self, frames: &[RgbImage]) -> Vec<Step> {
frames
.windows(2)
@@ -358,8 +308,7 @@ impl Region {
.collect();
let n = deltas.len().max(1) as f64;
let mean_delta = deltas.iter().sum::<f64>() / n;
// A pixel that did not move is counted as disagreeing: it is evidence against "the
// whole surface moved together", which is exactly the claim under test.
// A pixel that did not move counts against the surface moving together.
let same = deltas
.iter()
.filter(|d| **d != 0.0 && d.is_sign_positive() == mean_delta.is_sign_positive())
@@ -375,33 +324,26 @@ impl Region {
}
}
/// Rec. 601 luma — one number per pixel for "how bright", so a colour shift and a brightness shift
/// are not confused with each other.
/// Rec. 601 luma.
fn luma(p: &image::Rgb<u8>) -> f64 {
0.299 * f64::from(p[0]) + 0.587 * f64::from(p[1]) + 0.114 * f64::from(p[2])
}
/// The **spatial** structure of a toggle map — the reading that separates two defects the toggle
/// percentage alone cannot tell apart.
///
/// Both z-fighting and a visibility flip make pixels alternate, so both score the same on
/// [`Toggles::pct_toggling`]. They differ in *shape*: z-fighting resolves per fragment, so it lands
/// as moiré banding or speckle — many small runs, low neighbour agreement. A whole surface blinking
/// in and out (a culling, portal-visibility or draw-submission flip) lands as **one large solid run**
/// with the surface's own silhouette — few runs, high fill, high coherence. That distinction decides
/// which subsystem is at fault, so it is measured rather than eyeballed off a heatmap.
/// The spatial structure of a toggle map. Z-fighting resolves per fragment, landing as many small
/// runs of low coherence; a whole surface blinking (a culling, portal-visibility or draw-submission
/// flip) lands as one large solid run. Both score the same on [`Toggles::pct_toggling`].
#[derive(Debug, Clone, PartialEq)]
pub struct Shape {
/// Connected runs of at least [`MIN_REGION_PIXELS`], largest first.
pub regions: Vec<Region>,
/// Toggling pixels that fell in runs too small to list.
pub scattered: u64,
/// Mean fraction of the 4-neighbours of a toggling pixel that also toggle, 0..1. Near 1 = solid
/// areas; near 0 = isolated pixels. A one-pixel-wide moiré band sits near 0.5.
/// Mean fraction of a toggling pixel's 4-neighbours that also toggle, 0..1: solid areas near 1,
/// isolated pixels near 0, a one-pixel moiré band near 0.5.
pub coherence: f64,
}
/// A connected run smaller than this is noise, not a surface — counted in [`Shape::scattered`].
/// A run smaller than this is noise, counted in [`Shape::scattered`].
pub const MIN_REGION_PIXELS: u64 = 16;
/// Group a toggle mask into connected runs (4-connectivity) and measure their coherence.
@@ -418,8 +360,7 @@ pub fn shape(t: &Toggles) -> Shape {
continue;
}
toggling += 1;
// Off-frame neighbours count as non-toggling: an edge pixel is genuinely less enclosed,
// and pretending otherwise would inflate the coherence of anything touching the border.
// Off-frame neighbours count as non-toggling, so a run on the border is not inflated.
let n = u64::from(x > 0 && t.mask[idx(x - 1, y)])
+ u64::from(x + 1 < wu && t.mask[idx(x + 1, y)])
+ u64::from(y > 0 && t.mask[idx(x, y - 1)])
@@ -428,7 +369,7 @@ pub fn shape(t: &Toggles) -> Shape {
}
}
// Iterative flood fill — a recursive one blows the stack on a full-screen region.
// Iterative: a recursive flood fill overflows the stack on a full-screen region.
let mut seen = vec![false; wu * hu];
let mut regions: Vec<Region> = Vec::new();
let mut scattered = 0u64;
@@ -477,8 +418,7 @@ pub fn shape(t: &Toggles) -> Shape {
}
}
if pixels >= MIN_REGION_PIXELS {
// The flood fill visits in stack order, so sort to scan order — otherwise
// `Region::samples` spreads its picks through the traversal, not through the run.
// Scan order, so `Region::samples` spreads through the run, not the traversal.
members.sort_unstable_by_key(|&(x, y)| (y, x));
regions.push(Region {
bounds: b,
@@ -510,11 +450,8 @@ pub fn crop(img: &RgbImage, rect: Rect) -> RgbImage {
})
}
/// Magnify by an integer factor, nearest-neighbour — every output pixel is a source pixel verbatim.
/// This is the only scaling the `crop` subcommand offers on purpose: a resampler blends neighbours,
/// and a blended edge has already minted a false finding once (a `sips` downscale's "magenta
/// fringe" read as a render defect). Zoom may enlarge, never shrink: measurements happen at source
/// resolution or not at all.
/// Magnify by an integer factor, nearest-neighbour, so every output pixel is a source pixel: a
/// resampler's blended edge reads as a render defect. It never shrinks.
pub fn zoom(img: &RgbImage, factor: u32) -> RgbImage {
let s = factor.max(1);
RgbImage::from_fn(img.width() * s, img.height() * s, |x, y| {
@@ -522,8 +459,7 @@ pub fn zoom(img: &RgbImage, factor: u32) -> RgbImage {
})
}
/// Lay tiles out left-to-right, top-to-bottom in a `cols`-wide grid on a mid-grey mat, so a burst's
/// crops read as one contact sheet. Tiles of different sizes are placed at their own size.
/// Lay tiles out in a `cols`-wide grid on a dark grey mat, one contact sheet; each keeps its size.
pub fn contact_strip(tiles: &[RgbImage], cols: u32, gap: u32) -> RgbImage {
let cols = cols.max(1);
let (tw, th) = tiles
@@ -545,7 +481,6 @@ pub fn contact_strip(tiles: &[RgbImage], cols: u32, gap: u32) -> RgbImage {
out
}
/// The common size of a burst, erroring unless it has at least `min` equally-sized frames.
fn burst_dimensions(frames: &[RgbImage], min: usize) -> anyhow::Result<(u32, u32)> {
if frames.len() < min {
anyhow::bail!("needs at least {min} frames, got {}", frames.len());
@@ -563,9 +498,7 @@ fn burst_dimensions(frames: &[RgbImage], min: usize) -> anyhow::Result<(u32, u32
Ok((w, h))
}
/// Stitch two images side by side (`left | right`) with a `gap`-px dark separator, for at-a-glance A/B
/// comparison (e.g. the faithful vs modern render). Heights may differ; the output is the max height
/// with each image top-aligned.
/// Stitch two images side by side with a `gap`-px dark separator, top-aligned at the taller height.
pub fn compose_side_by_side(left: &RgbImage, right: &RgbImage, gap: u32) -> RgbImage {
let h = left.height().max(right.height());
let w = left.width() + gap + right.width();
@@ -596,9 +529,6 @@ mod tests {
assert_eq!(m.worst_at, (0, 0));
}
/// The single-pixel case this exists for: one silhouette pixel on the wrong side of an MSAA
/// tie. `mae` rounds to 0.000 and `pct_over` to 0.00% in a frame this size, so the *only*
/// signals that separate it from a real render change are the count and the coordinate.
#[test]
fn one_flipped_pixel_is_counted_and_located() {
let a = solid(100, 80, [40, 40, 40]);
@@ -613,8 +543,7 @@ mod tests {
#[test]
fn constant_offset_matches_offset() {
// Every channel of `b` is 10 below `a`, so mae == rmse == max_delta == 10, and (10 > 8) so
// every pixel is "over" → pct_over == 1.
// `b` is 10 below `a`: mae == rmse == max_delta == 10, and 10 > 8 puts every pixel over.
let a = solid(8, 5, [100, 100, 100]);
let b = solid(8, 5, [90, 90, 90]);
let m = compare(&a, &b).unwrap();
@@ -626,7 +555,6 @@ mod tests {
#[test]
fn small_offset_is_under_threshold() {
// A 5-byte shift is below OVER_THRESHOLD (8), so no pixel counts as changed even though mae>0.
let a = solid(8, 5, [100, 100, 100]);
let b = solid(8, 5, [95, 95, 95]);
let m = compare(&a, &b).unwrap();
@@ -650,8 +578,6 @@ mod tests {
#[test]
fn zoom_is_pixel_verbatim() {
// A 2x1 two-colour image zoomed ×3: every output pixel must be one of the two source
// values, block-aligned — any third colour would mean a resampler blended an edge.
let mut img = solid(2, 1, [10, 20, 30]);
img.put_pixel(1, 0, Rgb([200, 100, 50]));
let z = zoom(&img, 3);
@@ -687,8 +613,6 @@ mod tests {
#[test]
fn a_still_burst_has_no_envelope() {
// The point of max−min: a scene that never moves comes out black however bright it is, so a
// merely *wrong* render cannot masquerade as a flickering one.
let img = solid(4, 4, [200, 30, 90]);
let e = envelope(&[img.clone(), img.clone(), img], 8).unwrap();
assert_eq!(e.max_swing, 0);
@@ -699,9 +623,7 @@ mod tests {
#[test]
fn envelope_spans_the_whole_stack_not_just_neighbours() {
// 10 → 60 → 10: consecutive pairs each swing 50, and so does the envelope. But a value that
// drifts monotonically (10 → 35 → 60) swings only 25 per pair while the envelope still says
// 50 — which is the reason this is a stack operation and not a chain of pairwise diffs.
// 10, 60, 10 swings 50 per pair; 10, 35, 60 only 25 per pair, yet both envelopes are 50.
let base = solid(2, 1, [10, 10, 10]);
let alternating = {
let mut m = base.clone();
@@ -760,9 +682,6 @@ mod tests {
#[test]
fn a_monotone_sweep_never_toggles_however_far_it_moves() {
// The whole point: this is what a moving camera does to a pixel an edge sweeps across —
// a huge envelope (0 → 240) and zero reversals. `envelope` calls it maximally unstable;
// `toggles` correctly says it is not flickering.
let sweep = ramp(&[0, 60, 120, 180, 240]);
assert_eq!(envelope(&sweep, 1).unwrap().max_swing, 240);
let t = toggles(&sweep, 4, 60).unwrap();
@@ -772,7 +691,7 @@ mod tests {
#[test]
fn an_alternating_pixel_toggles_every_step() {
// A/B/A/B/A — z-fighting's signature. Four steps, three of them reversals.
// A/B/A/B/A, z-fighting's signature: four steps, three of them reversals.
let t = toggles(&ramp(&[10, 90, 10, 90, 10]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 3);
assert_eq!(t.pct_toggling, 1.0);
@@ -781,19 +700,16 @@ mod tests {
#[test]
fn sub_threshold_noise_neither_toggles_nor_forgets_the_direction() {
// ±1 dither must not manufacture reversals …
// ±1 dither makes no reversals,
let t = toggles(&ramp(&[50, 51, 50, 51, 50, 51]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 0);
// … and a run of it in the middle of a real climb must not reset the direction either,
// or every noisy edge would read as one reversal per crossing.
// nor does it reset the direction in the middle of a real climb.
let t = toggles(&ramp(&[0, 40, 41, 40, 41, 80]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 0);
}
#[test]
fn a_single_overshoot_is_one_reversal_and_below_the_toggle_bar() {
// Down-then-up once: real, but not a flicker — `TOGGLE_MIN_REVERSALS` is what draws that
// line, so a lone direction change must not be counted as a toggling pixel.
let t = toggles(&ramp(&[10, 90, 40]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 1);
assert_eq!(t.pct_toggling, 0.0);
@@ -818,8 +734,7 @@ mod tests {
assert_eq!(out.get_pixel(4, 0), &Rgb([40, 50, 60])); // right image
}
/// A burst in which exactly the pixels selected by `flips` alternate 0/200 and everything else
/// holds still — the shape tests differ only in *which* pixels those are.
/// A burst where the pixels `flips` selects alternate 0/200 and the rest hold still.
fn alternating(
w: u32,
h: u32,
@@ -838,7 +753,6 @@ mod tests {
#[test]
fn one_solid_block_reads_as_a_single_filled_run() {
// A whole surface blinking: one run, filling its own box, every interior pixel enclosed.
let frames = alternating(20, 20, 6, |x, y| {
(4..12).contains(&x) && (4..12).contains(&y)
});
@@ -866,8 +780,6 @@ mod tests {
#[test]
fn a_checkerboard_covers_the_same_area_with_none_of_the_shape() {
// Z-fighting resolving per fragment. Same region of the frame, ~half the pixels toggling,
// but no pixel touches another — the reading that separates it from the block above.
let frames = alternating(20, 20, 6, |x, y| {
(4..12).contains(&x) && (4..12).contains(&y) && (x + y) % 2 == 0
});
@@ -879,11 +791,10 @@ mod tests {
#[test]
fn runs_come_back_largest_first_and_short_ones_are_scattered() {
// Two blocks plus a speck: the big one leads, the speck falls below MIN_REGION_PIXELS.
let frames = alternating(40, 20, 6, |x, y| {
let big = (2..12).contains(&x) && (2..12).contains(&y); // 100 px
let small = (20..25).contains(&x) && (2..7).contains(&y); // 25 px
let speck = (30..33).contains(&x) && y == 10; // 3 px — noise
let speck = (30..33).contains(&x) && y == 10; // 3 px, noise
big || small || speck
});
let s = shape(&toggles(&frames, 4, 60).unwrap());
@@ -900,7 +811,7 @@ mod tests {
let s = shape(&toggles(&frames, 4, 60).unwrap());
assert!(s.regions.is_empty());
assert_eq!(s.scattered, 0);
assert_eq!(s.coherence, 0.0); // no toggling pixels — not a divide-by-zero
assert_eq!(s.coherence, 0.0); // no toggling pixels, and no divide-by-zero
}
#[test]
crates/benilla-visual/src/main.rs
+52 -145
View File
@@ -1,19 +1,6 @@
//! `benilla-visual` — diff captures from the Phase-5 visual A/B harness (decision 0008).
//!
//! Usage:
//! benilla-visual diff <a.png> <b.png> [--out <diff.png>] [--fail <mae>] [--amplify <n>]
//! benilla-visual diff-dir <dir_a> <dir_b> [--out <diff_dir>] [--fail <mae>] [--amplify <n>]
//! benilla-visual flicker <burst_dir> [--out <envelope.png>] [--fail <mae>] [--amplify <n>]
//! benilla-visual stat <img.png> [--rect <x>,<y>,<w>,<h>]
//!
//! `diff` compares two images; `diff-dir` compares every `*.png` present in *both* directories by name.
//! Prints the metrics; writes amplified heatmap(s) when `--out` is given; exits non-zero if any image's
//! MAE exceeds `--fail` (when given). Typical loop: capture baselines, change the renderer, re-capture,
//! `diff-dir baseline candidate --out diff --fail 1.5`.
//!
//! `flicker` is the same arithmetic over *time*: point it at a `WOW_LIVE_SHOT_COUNT` burst (adjacent
//! frames, parked camera) and it prints the frame-to-frame table and the whole-stack envelope, and
//! writes the envelope heatmap — the picture of which pixels would not hold still.
//! `benilla-visual`: diff captures, and read flicker and motion out of a burst of frames; `--help`
//! lists the subcommands. The regression loop: capture baselines, change the renderer, re-capture,
//! then `diff-dir baseline candidate --out diff --fail 1.5`.
use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
@@ -24,32 +11,24 @@ use benilla_visual::{
Metrics, Rect, OVER_THRESHOLD,
};
/// A frame-to-frame step must move a channel by this much to count as a direction change (override
/// with `--toggle-delta`). Above PNG/dither noise, below any real surface swap.
///
/// **The pan rate matters more than this number.** A moving camera sweeps texture detail across
/// every pixel, and detail is not monotone — at 8°/s (≈15 px/frame at 3200 px) the map saturates at
/// 93% and says nothing. Keep the sweep **sub-pixel per frame** (≈0.2°/s) so ordinary surfaces
/// barely move while a depth-comparison flip still swaps whole surfaces at full amplitude.
/// The smallest step that counts as a direction change (`--toggle-delta`): above PNG and dither
/// noise, below a surface swap. The pan rate matters more: keep a sweep sub-pixel per frame (about
/// 0.2°/s), since at 8°/s texture detail saturates the map.
const TOGGLE_MIN_DELTA: u8 = 4;
/// Reversal counts are small integers, so the toggle map needs its own (much larger) gain than the
/// 0..255 swings the envelope amplifies: at 60, three reversals saturate.
/// The toggle map's gain: reversal counts are small, and at 60 three of them saturate.
const TOGGLE_AMPLIFY: u32 = 60;
/// Gap (px) between the two halves of a side-by-side compose.
const COMPOSE_GAP: u32 = 6;
/// Tiles per row on a `hotspot` contact sheet — wide enough that a 24-frame burst reads as a few
/// rows of adjacent frames rather than one unscannable ribbon.
/// Tiles per row on a `hotspot` contact sheet.
const STRIP_COLS: u32 = 6;
/// Pixels of context kept around a hotspot crop (override with `--pad`). Enough to see what the
/// toggling region borders — which is usually the whole point.
/// Pixels of context around a hotspot crop (`--pad`), enough to see what the run borders.
const DEFAULT_PAD: u32 = 24;
/// How many runs `hotspot` reports a time series for. Enough to see whether the top runs flip on the
/// same frames (one cause) or independently (several), narrow enough to stay one screen.
/// Runs `hotspot` prints a series for: enough to see whether the top runs flip together.
const SERIES_RUNS: usize = 4;
/// Default amplification for the heatmap output (per-channel abs-diff ×N, clamped).
@@ -147,8 +126,7 @@ fn main() -> Result<()> {
opts.out.as_deref(),
opts.amplify,
)?;
// Under `--fail` this is a gate, and a shot that never landed must sink it — see the
// note in `diff_dir` (decision 0743).
// Under `--fail` this is a gate, and a missing shot fails it.
if opts.fail.is_some() && !unpaired.is_empty() {
bail!(
"{} unpaired image(s) — a shot is missing from one side, so this comparison is \
@@ -240,17 +218,8 @@ fn main() -> Result<()> {
Ok(())
}
/// What one named pixel *did*, frame by frame — the colour counterpart to the client's `WOW_DEPTH`.
///
/// The aggregate readings (`flicker`, `hotspot`) answer "which pixels would not hold still" over a
/// whole region, which is the right question when you are looking for a defect and the wrong one once
/// you have found it. Correlating a renderer-side probe against the pixels needs the opposite: *this*
/// pixel, *these* frames, no averaging. Run `WOW_DEPTH` and `--at` on the same coordinates and the two
/// logs line up frame for frame — "the colour changed here but the depth did not" is a fact you cannot
/// get from a region mean, because a run mean over 19 000 pixels hides which of them moved.
///
/// It also guards the trap this exists inside: a burst that happens to contain **no** flip reads as a
/// clean negative. The per-frame Δ makes an absent phenomenon obvious instead of invisible.
/// One named pixel's colour frame by frame, to line up against the client's `WOW_DEPTH` log at the
/// same coordinates; the per-frame delta shows a burst that caught no flip at all.
fn series(dir: &Path, at: &str) -> Result<()> {
let pixels = parse_at(at)?;
let names = pngs(dir)?;
@@ -284,17 +253,8 @@ fn series(dir: &Path, at: &str) -> Result<()> {
Ok(())
}
/// Cut a window out of a capture, magnify it, and report what was actually cut — the vetted
/// replacement for the ad-hoc `sips`/`ffmpeg` crop pipelines that kept minting false findings
/// (`sips --cropOffset` takes (y, x) and silently ignores what it can't parse; a guessed ffmpeg
/// window read "no eyes" off a frame the model had left). Three honesty rules: a rect that misses
/// the frame entirely is an ERROR, never an edge-clamped guess; a rect that only partially fits is
/// clamped *out loud*; and every pixel sample prints in SOURCE coordinates at source resolution, so
/// nothing downstream does coordinate math on a zoomed image.
/// Per-channel min / mean / max over a rect (or the whole frame) — the "is this region flat?"
/// instrument. A frame edge that shows the render target's clear colour instead of art reads as
/// `min == max` on every channel (decision 1619, the glue framing's void check); a region of art
/// never does. Printed, not judged: the caller compares two rects or two captures.
/// Per-channel min, mean and max over a rect or the whole frame: an edge showing the clear colour
/// instead of art reads `min == max` on every channel, which a region of art never does.
fn stat_cmd(path: &Path, rect: Option<&str>) -> Result<()> {
let img = load(path)?;
let (w, h) = img.dimensions();
@@ -389,7 +349,7 @@ fn crop_cmd(path: &Path, rect: &str, at: Option<&str>, scale: u32, out: &Path) -
zoomed.width(),
zoomed.height(),
);
// Samples: the rect's centre always, plus any --at points — all in source coordinates.
// Samples: the rect's centre and any `--at` points, in source coordinates.
let mut samples = vec![((r.x0 + r.x1) / 2, (r.y0 + r.y1) / 2)];
if let Some(at) = at {
samples.extend(parse_at(at)?);
@@ -414,8 +374,7 @@ fn crop_cmd(path: &Path, rect: &str, at: Option<&str>, scale: u32, out: &Path) -
Ok(())
}
/// `"x,y,w,h"` → the rect it names. Strict for the same reason as [`parse_at`]: a half-parsed
/// window silently crops the wrong place, and a wrong crop reads as a finding.
/// `"x,y,w,h"` → the rect it names; strict, since a half-parsed window crops the wrong place.
fn parse_rect(spec: &str) -> Result<Rect> {
let parts: Vec<&str> = spec.split(',').map(str::trim).collect();
let [x, y, w, h] = parts.as_slice() else {
@@ -437,7 +396,7 @@ fn parse_rect(spec: &str) -> Result<Rect> {
})
}
/// `"x,y;x,y"` → pixels. Strict: a typo here would silently report the wrong pixel's history.
/// `"x,y;x,y"` → pixels; strict, since a typo would report the wrong pixel's history.
fn parse_at(spec: &str) -> Result<Vec<(u32, u32)>> {
spec.split(';')
.filter(|s| !s.trim().is_empty())
@@ -455,27 +414,11 @@ fn parse_at(spec: &str) -> Result<Vec<(u32, u32)>> {
.collect()
}
/// Stitch every `*.png` present in both dirs into `left | right` side-by-side images under `out`.
/// **The sub-pixel motion ruler** (`flow <burst_dir> [--rect "<x>,<y>,<w>,<h>"]`).
///
/// `flicker` and `series` answer *how much* a pixel changed. Neither can answer the question a
/// "the animation ticks" report actually asks — is the thing in front of the pixel advancing
/// **evenly**? — and the obvious way to ask it is a trap: recover one silhouette edge's sub-pixel
/// position frame by frame, and the ruler is built out of 8-bit pixels, so it carries its own
/// 1/255 quantisation and a 0.2 px step cannot be separated from the tool's noise floor. That
/// ambiguity is exactly what left the first staircase reading unattributed.
///
/// So estimate the whole window's displacement at once, by least squares over every pixel in it
/// (Lucas–Kanade, the standard first-order flow solve): thousands of gradients vote on one
/// `(dx, dy)`, the 8-bit floor averages down by `√N`, and the estimate lands two orders of
/// magnitude under a pixel. A breathing body is not a rigid translation and does not need to be —
/// what is read off the series is whether the aggregate advances smoothly, and a staircase in the
/// RENDER shows up as a staircase here whatever the body underneath is doing.
///
/// Columns are the per-frame displacement, its magnitude, and the **second difference** — the
/// curvature-per-frame discriminator the world-space jitter meter uses, on this side of the glass.
/// The summary counts **stalled** frames (under a fifth of the median step): a smooth pan has
/// none, a staircase is mostly them.
/// The sub-pixel motion ruler: one least-squares (Lucas-Kanade) displacement per frame pair over
/// the whole window, so thousands of gradients average the 8-bit floor away and a render that
/// advances in steps reads as steps. Columns are the displacement, its magnitude and the second
/// difference; a stalled frame moves under a fifth of the median step. It assumes constant
/// brightness: for a shaded, deforming body use `edge`.
fn flow(dir: &Path, rect: Option<&str>) -> Result<()> {
let names = pngs(dir)?;
if names.len() < 3 {
@@ -487,8 +430,7 @@ fn flow(dir: &Path, rect: Option<&str>) -> Result<()> {
}
let first = load(&dir.join(names.iter().next().expect("non-empty")))?;
let (w, h) = first.dimensions();
// The default window is the whole frame minus the one-pixel border the central differences
// need. A named rect is clamped out loud, never silently (the `crop` honesty rule).
// A named rect is clamped out loud, never silently; the solve skips the one-pixel border.
let win = match rect {
Some(spec) => {
let want = parse_rect(spec)?;
@@ -581,8 +523,7 @@ fn flow(dir: &Path, rect: Option<&str>) -> Result<()> {
Ok(())
}
/// One frame's luma over `win`, row-major — the flow solve's input (f32 is exact for 8-bit sums
/// and halves the working set against a 3200x1800 f64 buffer).
/// One frame's luma over `win`, row-major; `f32` halves the working set of a 3200x1800 frame.
fn luma_window(img: &image::RgbImage, win: &Rect) -> Vec<f32> {
let mut out = Vec::with_capacity(((win.x1 - win.x0) * (win.y1 - win.y0)) as usize);
for y in win.y0..win.y1 {
@@ -596,10 +537,8 @@ fn luma_window(img: &image::RgbImage, win: &Rect) -> Vec<f32> {
out
}
/// Least-squares first-order optical flow between two luma windows: the `(dx, dy)` that best
/// explains `b` as `a` shifted, solved once over every interior pixel. `None` when the window
/// carries no usable gradient structure (a flat sky — the normal matrix is singular and any
/// answer would be invented).
/// The `(dx, dy)` that best explains `b` as `a` shifted, over every interior pixel; `None` when
/// the normal matrix is singular, as over a flat sky.
fn lucas_kanade(a: &[f32], b: &[f32], w: usize, h: usize) -> Option<(f64, f64)> {
let (mut sxx, mut sxy, mut syy, mut sxt, mut syt) = (0.0f64, 0.0, 0.0, 0.0, 0.0);
for y in 1..h - 1 {
@@ -616,8 +555,7 @@ fn lucas_kanade(a: &[f32], b: &[f32], w: usize, h: usize) -> Option<(f64, f64)>
}
}
let det = sxx * syy - sxy * sxy;
// Scale-relative: the normal matrix grows with the window, so an absolute epsilon would call
// a big flat window solvable and a small textured one singular.
// Relative, since the normal matrix grows with the window.
if det <= 1e-12 * (sxx * syy).max(f64::MIN_POSITIVE) {
return None;
}
@@ -646,11 +584,10 @@ fn compose_dir(da: &Path, db: &Path, out: &Path) -> Result<()> {
Ok(())
}
/// Report how far a burst of adjacent frames moved: the frame-to-frame table, then the whole-stack
/// envelope. Returns the worst consecutive-pair metrics (the `--fail` subject) — the envelope is a
/// *stack* number and can exceed any single pair, so failing on the pair is the conservative gate.
/// Print a burst's frame-to-frame table, envelope and toggles; the worst pair's metrics are the
/// `--fail` subject, since the envelope can exceed any single pair.
fn flicker(dir: &Path, out: Option<&Path>, amplify: u32, toggle_delta: u8) -> Result<Metrics> {
let names: Vec<String> = pngs(dir)?.into_iter().collect(); // BTreeSet — already shot order
let names: Vec<String> = pngs(dir)?.into_iter().collect(); // a BTreeSet: already shot order
if names.len() < 2 {
bail!(
"{} holds {} *.png — a flicker burst needs at least 2 (WOW_LIVE_SHOT_COUNT=<n>)",
@@ -691,8 +628,7 @@ fn flicker(dir: &Path, out: Option<&Path>, amplify: u32, toggle_delta: u8) -> Re
e.mean_swing,
e.pct_unstable * 100.0
);
// The moving-camera reading. Printed always, because which of the two numbers is meaningful
// depends on whether the camera was parked — and a burst does not carry that fact.
// Printed always: a burst does not record whether the camera was parked.
let t = toggles(&frames, toggle_delta, TOGGLE_AMPLIFY)?;
println!(
"toggles of {} frames (delta {toggle_delta}): max reversals {:>3} toggling pixels {:>6.2}% \
@@ -718,12 +654,8 @@ fn flicker(dir: &Path, out: Option<&Path>, amplify: u32, toggle_delta: u8) -> Re
Ok(worst)
}
/// Follow the toggle map to the thing it found: measure the toggling pixels' **shape**, then crop
/// the largest run out of every frame into one contact sheet.
///
/// The map says *where*; this says *what kind*, and shows the frames that prove it. A run that fills
/// its box and holds together is one surface blinking (visibility/culling); many thin runs are
/// z-fighting resolving per fragment. Both read identically on the toggle percentage.
/// Measure the toggling pixels' shape, print the top runs' series and relight fits, and crop the
/// largest run out of every frame into one contact sheet.
fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
let names: Vec<String> = pngs(dir)?.into_iter().collect();
let frames: Vec<image::RgbImage> = names
@@ -751,8 +683,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
r.bounds.x0,
r.bounds.y0,
);
// Pasteable straight into the in-game ray pick, which is the next question every time:
// WOW_PICK="<this>" names the surfaces at these exact pixels, front to back.
// For the ray pick: `WOW_PICK` names the surfaces at these pixels, front to back.
let pick = r
.samples()
.iter()
@@ -767,9 +698,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
benilla_visual::MIN_REGION_PIXELS
);
};
// What the top runs are actually *doing*, frame by frame: re-shaded together, or edges moving.
// Side by side, because whether they flip on the SAME steps is the difference between one global
// cause and several local ones — and that is invisible in any single run's series.
// The top runs side by side: flipping on the same steps means one cause, not several.
let series: Vec<Vec<benilla_visual::Step>> = s
.regions
.iter()
@@ -789,11 +718,8 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
.collect();
println!(" {:>2} -> {:<2} {}", i, i + 1, cols.join(" "));
}
// The two extremes of run #0, per channel — the *levels*, which is all this line is. Read the
// ratios as description, never as a diagnosis: equal ratios rule out a scalar multiply and
// nothing else, since an ADDED light moves the three channels by different factors too. The
// reading that actually separates "one surface, re-lit" from "two surfaces" is the per-pixel
// affine fit printed below it (`benilla_visual::relight`).
// Run #0's extremes per channel, a description only: equal ratios rule out a scalar multiply
// and nothing else. The relight fit below is the diagnosis.
let s0 = &series[0];
let lo = s0
.iter()
@@ -812,10 +738,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
hi.mean_rgb_from[0], hi.mean_rgb_from[1], hi.mean_rgb_from[2],
ratio.join(" / "),
);
// Same surface re-lit, or a different surface? Fit each run's pixels across its own biggest
// frame-to-frame step. High R² = the pattern survived the flip and only its gain/offset moved,
// so it is one surface being shaded differently; low R² = the two frames show different
// surfaces, whatever the means do.
// Each run fitted across its biggest step: high R² is one surface re-lit, low R² two surfaces.
println!(
" across each run's biggest step — is it one surface re-lit? (R² near 1 = yes), \
against its quietest step as the control:"
@@ -837,8 +760,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
)
})
.collect();
// The control is what licenses the verdict: a low R² on the flip only means "different
// surfaces" if the same pixels under the same pan fit tightly when they did NOT flip.
// A low R² on the flip means two surfaces only if the quiet step fits tightly.
let verdict = if ctl.worst_r2() < 0.9 {
"no control — the pan alone decorrelates these pixels; fit says nothing"
} else if fit.worst_r2() > 0.9 {
@@ -860,8 +782,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
);
}
let rect = biggest.bounds.padded(pad, w, h);
// The toggle map goes in as the first tile, so the sheet carries its own legend: this is the
// region, and here is what the frames did inside it.
// The toggle map is the first tile, the sheet's own legend.
let mut tiles = vec![crop(&t.image, rect)];
tiles.extend(frames.iter().map(|f| crop(f, rect)));
if let Some(parent) = out.parent().filter(|p| !p.as_os_str().is_empty()) {
@@ -882,9 +803,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
Ok(())
}
/// `foo.png` + `-toggle` → `foo-toggle.png`. The two maps answer the same question under opposite
/// camera conditions, so they are written as a pair rather than behind a second flag nobody would
/// remember to pass.
/// `foo.png` + `-toggle` → `foo-toggle.png`.
fn sibling(out: &Path, suffix: &str) -> PathBuf {
let ext = out.extension().and_then(|e| e.to_str()).unwrap_or("png");
let stem = out.file_stem().and_then(|s| s.to_str()).unwrap_or("out");
@@ -915,15 +834,9 @@ fn over_fail(m: &Metrics, fail: Option<f64>) -> bool {
fail.is_some_and(|t| m.mae > t)
}
/// Load an image as RGB (dropping any alpha — capture windows are opaque).
/// One silhouette's sub-pixel position along a scanline, per frame — the shading-independent
/// answer to "did the rendered geometry MOVE smoothly?".
///
/// Per frame and per scanline: take the row's own darkest and brightest samples, put the
/// threshold halfway between them, and linearly interpolate the crossing. Renormalising to the
/// row's own ends every frame is the whole point — a uniform brightness change (the body turning
/// into the light) moves `lo` and `hi` together and leaves the crossing where it was, while a
/// geometric shift moves it. That is the property [`flow`] lacks.
/// One silhouette's sub-pixel position along a scanline, per frame: where the row crosses the
/// midpoint of its own darkest and brightest samples, so a uniform brightness change leaves it in
/// place and only a geometric shift moves it.
fn edge(dir: &Path, at: &str) -> Result<()> {
let names = pngs(dir)?;
if names.len() < 3 {
@@ -972,7 +885,7 @@ fn edge(dir: &Path, at: &str) -> Result<()> {
.collect();
let lo = row.iter().copied().fold(f64::INFINITY, f64::min);
let hi = row.iter().copied().fold(f64::NEG_INFINITY, f64::max);
// Too little contrast to call an edge — report it rather than invent a position.
// Too little contrast to call an edge: report it, never invent a position.
if hi - lo < 12.0 {
pos.push(f64::NAN);
continue;
@@ -1051,11 +964,10 @@ fn diff_one(a: &Path, b: &Path, out: Option<&Path>, amplify: u32) -> Result<Metr
Ok(m)
}
/// Compare every `*.png` present in both dirs (by file name). Returns the worst (highest-MAE) result.
/// What one `diff-dir` run found: the worst pair by MAE, and every image present on only ONE side.
/// What one `diff-dir` run found: the worst pair by MAE, and every image on only one side.
struct DirDiff {
worst: Option<(String, Metrics)>,
/// Files with no counterpart, so not diffed at all — see the note in [`diff_dir`].
/// Files with no counterpart, so not diffed.
unpaired: Vec<String>,
}
@@ -1069,11 +981,8 @@ fn diff_dir(da: &Path, db: &Path, out: Option<&Path>, amplify: u32) -> Result<Di
db.display()
);
}
// UNPAIRED shots are ANNOUNCED, never skipped in silence. Pairing on the intersection alone is
// how a sweep reports all-green with a shot missing: on 2026-07-28 one `water-night` capture
// exited 0 without writing its PNG, and `selfcheck` diffed the other eight and passed clean. A
// gate that quietly narrows its own scope is worse than no gate. Reported always; fatal to the
// CALLER when `--fail` is in force, i.e. whenever this is being used as a gate (decision 0743).
// Unpaired shots are announced, never skipped in silence, so a capture that wrote no PNG
// cannot let the rest pass clean; the caller fails on them under `--fail`.
let unpaired: Vec<String> = a
.difference(&b)
.map(|n| format!("{}/{n}", da.display()))
@@ -1117,9 +1026,8 @@ fn pngs(dir: &Path) -> Result<BTreeSet<String>> {
}
fn fmt_metrics(m: &Metrics) -> String {
// `px` and the worst pixel's coordinate are what tell a render change from an MSAA tie: both
// show `MAE 0.000` with an alarming `max`, and only the count and the location separate them
// (a handful of pixels, at the same coordinate every build, is a tie).
// `px` and the worst pixel's coordinate tell an MSAA tie from a render change: a handful of
// pixels at the same spot every build is a tie.
let where_ = if m.changed == 0 {
String::new()
} else {
@@ -1187,7 +1095,6 @@ mod tests {
#[test]
fn rect_spec_is_strict() {
// "x,y,w,h" and nothing else — a half-parsed window crops the wrong place silently.
let r = parse_rect("10, 20, 30, 40").unwrap();
assert_eq!((r.x0, r.y0, r.x1, r.y1), (10, 20, 40, 60));
assert!(parse_rect("10,20,30").is_err());
crates/benilla-visual/src/relight.rs
+20 -56
View File
@@ -1,26 +1,11 @@
//! **Is it the same surface lit differently, or a different surface?** — the reading that separates
//! the two explanations a two-state flip always admits.
//! Tells one surface lit differently from two different surfaces. Equal channel ratios rule out
//! only a scalar multiply, since an added coloured light changes the ratios too; the spatial
//! pattern tells them apart. Re-lighting is an affine map per pixel, `bright ≈ gain·dim + offset`,
//! so the fit is tight, while two surfaces fit loosely however well their means line up. This fits
//! one frame's pixels onto the other's per channel and reports R².
//!
//! [`Region::steps`] measures that a run alternates between two levels and that its pixels move
//! together. That is where the evidence used to stop, and the gap mattered: B38's tent was read off
//! the per-channel *ratios* as "two materials trading places", on the argument that one light term
//! switching would scale all three channels equally. **That argument is wrong.** Equal ratios rule
//! out a scalar multiply and nothing else — an *additive* term (a coloured light arriving or
//! leaving) changes the ratios freely. Measured there: dim `(77, 42, 32)` → bright `(192, 79, 35)`
//! is "×2.50 / ×1.89 / ×1.12", which reads as two materials, and is *also* exactly the dim state
//! plus a warm `(115, 37, 3)`. The ratios cannot tell those apart, so they were never evidence.
//!
//! What does tell them apart is the **spatial pattern**, which the means throw away. Re-lighting a
//! surface is an affine map on its pixels — every pixel keeps its place in the pattern, so
//! `bright ≈ gain·dim + offset` holds *per pixel* and the fit is tight. Two different surfaces have
//! unrelated patterns (canvas weave vs plank grain), and no affine map relates them: the fit is
//! loose however well the means happen to line up. So: least-squares fit one frame's pixels onto the
//! other's, per channel, and report R². That is a number, not an inference from a number.
//!
//! Fit across the run's **largest single frame-to-frame step**, never across the whole burst: the
//! camera pans during a capture, and a pixel only names the same bit of world for as long as the
//! image holds still under it. Adjacent frames at the sub-pixel-per-frame pan the toggle map needs
//! (decision 0656) are the same view; frames twenty apart are not.
//! The fit spans the run's largest single frame-to-frame step, never the whole burst: the camera
//! pans, and a pixel names the same bit of world only while the image holds still under it.
use image::RgbImage;
@@ -29,26 +14,20 @@ use crate::Region;
/// A per-channel affine fit of one frame's pixels onto another's, over a [`Region`]'s own pixels.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Relight {
/// Per channel, the `a` in `to ≈ a·from + b`. Near 1 with a non-zero [`Relight::offset`] is a
/// light being *added*; well above 1 with an offset near 0 is one being *scaled*.
/// Per channel, the `a` in `to ≈ a·from + b`: near 1 with an offset is a light added, well
/// above 1 with none a light scaled.
pub gain: [f64; 3],
/// Per channel, the `b` in `to ≈ a·from + b`, in 0..255 units.
pub offset: [f64; 3],
/// Per channel coefficient of determination, 0..1 — how much of the target frame's variation
/// across the run the fit explains. **This is the reading.** Near 1: one surface, re-lit (the
/// pattern survived, only its scale and offset moved). Low: the two frames show *different*
/// surfaces, and no amount of re-lighting maps one to the other.
/// Per channel R², 0..1: near 1 is one surface re-lit, low is two different surfaces.
pub r2: [f64; 3],
/// Did this channel vary across the run at all? A flat channel (uniform, or clipped to 0/255)
/// has nothing for a fit to explain, so its R² is 0 for a reason that is **not** evidence of
/// different surfaces — and without this flag that 0 would outvote the channels that do carry
/// evidence, turning a plain re-light into a false "different surfaces".
/// Whether the channel varied across the run: a flat or clipped channel's R² of 0 is no
/// evidence, and must not outvote the others.
pub determinate: [bool; 3],
}
impl Relight {
/// The weakest **determinate** channel's R² — the honest summary, since "one surface, re-lit"
/// has to hold in every channel that carries evidence. 0 when no channel does.
/// The weakest determinate channel's R², or 0 when no channel is determinate.
pub fn worst_r2(&self) -> f64 {
(0..3)
.filter(|&c| self.determinate[c])
@@ -58,11 +37,8 @@ impl Relight {
}
}
/// Least-squares fit of `to`'s pixels onto `from`'s, per channel, over `region`'s own pixels.
///
/// A channel with no variation across the run (a flat, saturated, or clipped channel) has nothing
/// for a fit to explain; its R² is reported as 0 rather than dividing by zero, which reads as
/// "this channel is not evidence" — the conservative direction.
/// Least-squares fit of `to`'s pixels onto `from`'s per channel, over `region`'s own pixels; a flat
/// channel reports R² 0 and is not determinate.
pub fn relight(region: &Region, from: &RgbImage, to: &RgbImage) -> Relight {
let mut out = Relight {
gain: [0.0; 3],
@@ -106,20 +82,13 @@ pub fn relight(region: &Region, from: &RgbImage, to: &RgbImage) -> Relight {
out
}
/// The index of the frame-to-frame step where the run's mean luma moved most — the flip itself,
/// and the pair [`relight`] should be fitted across. `None` for a burst with fewer than two frames.
/// The step where the run's mean luma moved most: the flip, and the pair [`relight`] fits across.
pub fn biggest_step(region: &Region, frames: &[RgbImage]) -> Option<usize> {
extreme_step(region, frames, true)
}
/// The step where the run's mean luma moved **least** — the **control**, and the reason a low R² on
/// the biggest step can be believed at all.
///
/// The capture pans, so consecutive frames are never quite the same image, and a textured surface
/// sliding a fraction of a pixel decorrelates on its own. That means a low R² is only evidence of
/// *different surfaces* if a step where the run did **not** flip scores high across the same pixels
/// under the same motion. Quiet step near 1 and flip step near 0 is a real reading; both low means
/// the fit is measuring the pan and nothing else.
/// The step where the run's mean luma moved least: the control. The pan alone decorrelates a
/// textured surface, so a low R² on the flip means two surfaces only if this step fits tightly.
pub fn quietest_step(region: &Region, frames: &[RgbImage]) -> Option<usize> {
extreme_step(region, frames, false)
}
@@ -161,7 +130,7 @@ mod tests {
}
}
/// A textured surface: a pattern with real variation for a fit to have to explain.
/// A patterned surface, with variation for a fit to explain.
fn textured(w: u32, h: u32, f: impl Fn(u32, u32) -> [u8; 3]) -> RgbImage {
RgbImage::from_fn(w, h, |x, y| Rgb(f(x, y)))
}
@@ -169,7 +138,7 @@ mod tests {
#[test]
fn the_same_surface_plus_a_warm_light_fits_as_gain_one_and_an_offset() {
let dim = textured(16, 16, |x, y| [(x * 8) as u8, (y * 6) as u8, (x + y) as u8]);
// Exactly the B38 shape: add a warm constant, leave the pattern alone.
// A warm constant added, the pattern untouched.
let bright = textured(16, 16, |x, y| {
[(x * 8 + 115) as u8, (y * 6 + 37) as u8, (x + y + 3) as u8]
});
@@ -200,8 +169,6 @@ mod tests {
assert!(r.offset[0].abs() < 1e-6, "{r:?}");
}
/// The discrimination the module exists for: two surfaces whose *means* differ exactly as a
/// re-light would, but whose patterns are unrelated. The means cannot tell them apart; R² can.
#[test]
fn two_different_surfaces_do_not_fit_however_well_their_means_line_up() {
// Vertical stripes vs horizontal stripes: same mean, same spread, no affine relation.
@@ -221,9 +188,6 @@ mod tests {
assert!((r.offset[0] - 90.0).abs() < 1e-9, "{r:?}");
}
/// The flat-channel rule has to be a *skip*, not a zero vote: a surface whose blue is clipped
/// flat but whose red and green track perfectly is one surface being re-lit, and reporting the
/// dead channel's 0 as the verdict would call it two.
#[test]
fn a_flat_channel_does_not_outvote_the_ones_carrying_evidence() {
let dim = textured(16, 16, |x, y| [(x * 8) as u8, (y * 6) as u8, 255]);
crates/benilla-wdt/src/lib.rs
+49 -110
View File
@@ -1,44 +1,29 @@
//! WDT (map tile table) reader + tile↔world coords for **WoW 1.12.1 (build 5875)** — in-repo,
//! replacing `wow-wdt` (decision 0021).
//! The 1.12.1 WDT (map tile table) reader, and the tile and world coordinate helpers.
//!
//! A vanilla `.wdt` is a tiny chunked file (`MVER`, `MPHD`, `MAIN`, and — on a **WMO-only map** —
//! `MWMO` + one `MODF`). Two things in it matter to the streamer:
//!
//! - **`MAIN`** — a 64×64 table of 8-byte entries whose flag bit 0 (`0x1`) marks "this tile has an
//! `.adt`".
//! - **The global WMO** — `MPHD` dword0 bit 0 marks a map with **no terrain at all**, whose whole
//! world is one building placed by the single `MODF` entry after `MWMO`. 20 of the 43 shipped
//! maps are built this way (every WMO dungeon: the Blackrock pair, Gnomeregan, Dire Maul, the
//! Stockade…); on those maps `MAIN` is entirely empty and this placement *is* the map
//! (decision 0688).
//!
//! The coord helpers map a world `(x, y)` to its `(tile_x, tile_y)` and back, on the 64-tile,
//! 533⅓-yd grid.
//!
//! Proven against `wow-wdt` over a real map's WDT during the decision-0021 migration (oracle test in
//! git history); the `benilla-formats` terrain tests pin tile selection + world coords end-to-end.
//! A `.wdt` holds `MVER`, `MPHD`, `MAIN` and, on a WMO-only map, `MWMO` and one `MODF`. `MAIN` is a
//! 64×64 table of 8-byte entries whose flag bit 0 marks a tile with an `.adt`. `MPHD` dword 0 bit 0
//! marks a map with no terrain, whose whole world is the one building the `MODF` places: 20 of the
//! 43 shipped maps, every WMO dungeon, each with an empty `MAIN`.
use std::io::{Read, Seek, SeekFrom};
/// WoW client generation. We only target Classic (1.12.1); kept as an enum so the reader signature
/// matches the call sites.
/// The client generation; 1.12.1 is the only one.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum WowVersion {
Classic,
}
/// Re-export so `benilla_wdt::version::WowVersion` resolves (mirrors the original crate's module path).
/// Re-export, so `benilla_wdt::version::WowVersion` resolves.
pub mod version {
pub use super::WowVersion;
}
/// Tiles per map edge (64×64).
/// Tiles per map edge.
const MAP_SIZE: usize = 64;
/// Yards per ADT tile (1/64 of the map edge).
const TILE_YARDS: f32 = 533.333_3;
/// World `(x, y)` → ADT tile `(tile_x, tile_y)`. WoW's axes are swapped vs the tile grid: `tile_x`
/// comes from world **y**, `tile_y` from world **x**. Clamped to `0..=63`.
/// World `(x, y)` → ADT tile `(tile_x, tile_y)`, clamped to `0..=63`. The axes swap: `tile_x`
/// comes from world y, `tile_y` from world x.
pub fn world_to_tile(world_x: f32, world_y: f32) -> (u32, u32) {
let offset = 32.0 * TILE_YARDS;
let tile_x = ((offset - world_y) / TILE_YARDS) as u32;
@@ -46,17 +31,13 @@ pub fn world_to_tile(world_x: f32, world_y: f32) -> (u32, u32) {
(tile_x.min(63), tile_y.min(63))
}
/// Chunks per tile edge (16×16 per ADT) and per map edge (1024).
const CHUNKS_PER_TILE: u32 = 16;
/// Yards per terrain chunk (1/16 of a tile, 33⅓ yd).
const CHUNK_YARDS: f32 = TILE_YARDS / CHUNKS_PER_TILE as f32;
/// World `(x, y)` → the **chunk** index `(chunk_x, chunk_y)` on the 1024×1024 chunk grid — the same
/// axis swap as [`world_to_tile`], and `chunk >> 4` is exactly that tile. Clamped to `0..=1023`.
///
/// This is the reference's own streaming coordinate: `0x672730` computes
/// `fistp((17066.666 − pos) × 0.03 − 0.5)` per axis — the containing chunk — and sizes its
/// residency windows in these units.
/// World `(x, y)` → chunk `(chunk_x, chunk_y)` on the 1024×1024 grid, clamped to `0..=1023`, with
/// [`world_to_tile`]'s axis swap and `chunk >> 4` its tile. It is the reference's streaming
/// coordinate: `0x672730` computes `fistp((17066.666 - pos) * 0.03 - 0.5)` per axis and sizes its
/// residency windows in chunks.
pub fn world_to_chunk(world_x: f32, world_y: f32) -> (u32, u32) {
let offset = 32.0 * TILE_YARDS;
let max = MAP_SIZE as u32 * CHUNKS_PER_TILE - 1;
@@ -65,8 +46,7 @@ pub fn world_to_chunk(world_x: f32, world_y: f32) -> (u32, u32) {
(chunk_x.min(max), chunk_y.min(max))
}
/// ADT tile `(tile_x, tile_y)` → the world `(x, y)` of its max-corner origin (inverse of
/// [`world_to_tile`]).
/// ADT tile → the world `(x, y)` of its max corner, the inverse of [`world_to_tile`].
pub fn tile_to_world(tile_x: u32, tile_y: u32) -> (f32, f32) {
let offset = 32.0 * TILE_YARDS;
(
@@ -75,7 +55,7 @@ pub fn tile_to_world(tile_x: u32, tile_y: u32) -> (f32, f32) {
)
}
/// Info about one map tile (the subset the streamer reads).
/// One map tile, as the streamer reads it.
#[derive(Debug, Clone, Copy)]
pub struct TileInfo {
pub x: usize,
@@ -84,27 +64,21 @@ pub struct TileInfo {
pub has_adt: bool,
}
/// The single building that **is** a WMO-only map — the `MWMO` path plus the one `MODF` entry that
/// follows it (decision 0688). Position/rotation are already in the same convention as an ADT's own
/// MODF placements (`benilla_formats::WmoInstance`), so the streamer places it through the exact
/// path it places any other building with.
/// The one building that is a WMO-only map: the `MWMO` path and its `MODF` entry, already in
/// `benilla_formats::WmoInstance`'s convention.
#[derive(Debug, Clone)]
pub struct GlobalWmo {
/// `MWMO` root path (e.g. `world\wmo\dungeon\az_stormwindprisons\stormwindjail.wmo`).
pub model: String,
/// Position in WoW world coords (X north, Y west, Z up).
///
/// Taken **raw** from the MODF, without the `32·533⅓ − v` corner-origin remap an ADT's
/// placements get: this entry is authored in world coords already. Every shipped WMO-only map
/// stores `(0, 0, 0)` here, and the remap would put the whole dungeon 24 km from its own
/// entrance — falsified against all 26 of the server's real entry points (decision 0688).
/// Position in world coords (X north, Y west, Z up), taken raw: the `32·533⅓ - v` remap an
/// ADT's placements get would put every shipped WMO-only map, which stores `(0, 0, 0)`, 24 km
/// from its entrance.
pub position: [f32; 3],
/// Euler rotation in **degrees** (X, Y, Z); Y is the heading about the up axis. Live data, not
/// always identity: Dire Maul is authored at `ry = 180°` while the other 19 maps sit at 0.
/// Euler rotation in degrees (X, Y, Z), Y the heading; Dire Maul's `ry` is 180°, the rest 0.
pub rotation: [f32; 3],
/// MODF doodad-set index — which *additional* prop set to show beyond the always-on set 0.
/// MODF doodad-set index: the prop set shown beside the always-on set 0.
pub doodad_set: u16,
/// MODF name-set index — the placement's `WMOAreaTable.NameSetID` naming/audio variant.
/// MODF name-set index, the placement's `WMOAreaTable.NameSetID` variant.
pub name_set: u16,
}
@@ -113,12 +87,10 @@ pub struct GlobalWmo {
pub struct WdtFile {
/// `has_adt` per tile, indexed `y * 64 + x` (the on-disk MAIN order).
has_adt: Vec<bool>,
/// The whole map, on a map with no terrain; `None` on an ADT map.
global_wmo: Option<GlobalWmo>,
}
impl WdtFile {
/// Tile info at `(x, y)`, or `None` out of range.
pub fn get_tile(&self, x: usize, y: usize) -> Option<TileInfo> {
if x >= MAP_SIZE || y >= MAP_SIZE {
return None;
@@ -130,8 +102,7 @@ impl WdtFile {
})
}
/// The map's single global building, if this is a WMO-only map (`MPHD` bit 0) — the entire
/// world on such a map. `None` on an ADT map, where the ground comes from the tiles instead.
/// The map's one building, on a WMO-only map (`MPHD` bit 0).
pub fn global_wmo(&self) -> Option<&GlobalWmo> {
self.global_wmo.as_ref()
}
@@ -151,9 +122,7 @@ impl<R: Read + Seek> WdtReader<R> {
}
}
/// One chunk's `size`-byte payload — refused before the buffer is sized when the stream
/// (ending at `stream_end`) cannot hold it, so a raw size never reaches the allocator
/// unchecked.
/// One chunk's payload, refused before the buffer is sized if the stream cannot hold it.
fn read_payload(&mut self, size: u64, stream_end: u64) -> std::io::Result<Vec<u8>> {
let remaining = stream_end.saturating_sub(self.reader.stream_position()?);
if size > remaining {
@@ -167,12 +136,8 @@ impl<R: Read + Seek> WdtReader<R> {
Ok(buf)
}
/// Parse the WDT, returning the tile grid and (on a WMO-only map) the global WMO placement.
/// Walks chunks until EOF; everything but `MPHD`/`MAIN`/`MWMO`/`MODF` is skipped.
/// Parse the WDT, walking chunks to EOF and skipping all but `MPHD`, `MAIN`, `MWMO` and `MODF`.
pub fn read(&mut self) -> std::io::Result<WdtFile> {
// Chunk sizes are raw on-disk u32s: each payload buffer below is bounded by what the
// stream still holds (`read_payload`), so a corrupt size fails as the short read it is
// instead of asking the allocator for up to 4 GiB first.
let start = self.reader.stream_position()?;
let stream_end = self.reader.seek(SeekFrom::End(0))?;
self.reader.seek(SeekFrom::Start(start))?;
@@ -185,11 +150,10 @@ impl<R: Read + Seek> WdtReader<R> {
if !read_full(&mut self.reader, &mut hdr)? {
break; // clean EOF
}
// Chunk magics are stored reversed on disk; MAIN ⇒ "NIAM".
// Chunk magics are stored reversed on disk: MAIN is "NIAM".
let size = u32::from_le_bytes([hdr[4], hdr[5], hdr[6], hdr[7]]) as u64;
match &hdr[0..4] {
// MPHD — only dword0 bit 0 is ever read: "this map has no terrain, its world is
// the global WMO below" (the reference reads exactly this one bit, @0x694810).
// MPHD: the reference reads only dword 0 bit 0, the no-terrain flag (`0x694810`).
b"DHPM" if size >= 4 => {
let buf = self.read_payload(size, stream_end)?;
wmo_only = u32::from_le_bytes([buf[0], buf[1], buf[2], buf[3]]) & 0x1 != 0;
@@ -212,8 +176,8 @@ impl<R: Read + Seek> WdtReader<R> {
.collect();
has_adt = Some(grid);
}
// MWMO — one NUL-terminated path on a WMO-only map, and a zero-length stub on an
// ADT map (which is why the `wmo_only` flag, not this chunk, is the gate).
// MWMO: a NUL-terminated path on a WMO-only map, a zero-length stub on an ADT map,
// so the `wmo_only` flag, not this chunk, is the gate.
b"OMWM" if size > 0 => {
let buf = self.read_payload(size, stream_end)?;
let end = buf.iter().position(|&b| b == 0).unwrap_or(buf.len());
@@ -222,7 +186,7 @@ impl<R: Read + Seek> WdtReader<R> {
wmo_path = Some(path);
}
}
// MODF — exactly one 64-byte placement here (the ADT variant carries many).
// MODF: exactly one 64-byte placement here.
b"FDOM" if size >= 64 => {
modf = Some(self.read_payload(size, stream_end)?);
}
@@ -234,15 +198,14 @@ impl<R: Read + Seek> WdtReader<R> {
let has_adt = has_adt.ok_or_else(|| {
std::io::Error::new(std::io::ErrorKind::InvalidData, "WDT missing MAIN chunk")
})?;
// A WMO-only map with no placement to show would be an empty void, so demand both parts
// before claiming one — a malformed header bit alone must not suppress the tile grid.
// The flag and a placement, or no global WMO: a stray header bit must not hide the grid.
let global_wmo = match (wmo_only, wmo_path, modf) {
(true, Some(model), Some(e)) => Some(GlobalWmo {
model,
position: [f32_at(&e, 8), f32_at(&e, 12), f32_at(&e, 16)],
rotation: [f32_at(&e, 20), f32_at(&e, 24), f32_at(&e, 28)],
// 0x20..0x38 is the bounding box; the placement words follow it. (`uniqueId` at
// +4 is dead here — the reference overwrites it from its own counter @0xc9a320.)
// 0x20..0x38 is the bounding box. `uniqueId` at +4 is unread: the reference
// overwrites it from its own counter (`0xc9a320`).
doodad_set: u16_at(&e, 58),
name_set: u16_at(&e, 60),
}),
@@ -255,21 +218,19 @@ impl<R: Read + Seek> WdtReader<R> {
}
}
/// Little-endian `f32` at `off` (0.0 past the end — callers have already length-checked the entry).
fn f32_at(b: &[u8], off: usize) -> f32 {
b.get(off..off + 4)
.and_then(|s| s.try_into().ok())
.map_or(0.0, f32::from_le_bytes)
}
/// Little-endian `u16` at `off` (0 past the end).
fn u16_at(b: &[u8], off: usize) -> u16 {
b.get(off..off + 2)
.and_then(|s| s.try_into().ok())
.map_or(0, u16::from_le_bytes)
}
/// Read exactly `buf.len()` bytes; `Ok(false)` on a clean EOF before any byte, `Ok(true)` on success.
/// Read exactly `buf.len()` bytes; `Ok(false)` on a clean EOF before any byte.
fn read_full<R: Read>(reader: &mut R, buf: &mut [u8]) -> std::io::Result<bool> {
let mut filled = 0;
while filled < buf.len() {
@@ -297,14 +258,12 @@ mod tests {
use super::*;
use std::io::Cursor;
// Mirror of the crate-private grid constants, so the tests read the geometry independently of
// the values the code under test happens to use.
// The grid constants restated, so the tests do not read the code's own.
const T: f32 = 533.333_3;
const OFFSET: f32 = 32.0 * T;
/// Build a minimal on-disk WDT: an ignored `MVER`, an ignored `MPHD`, then a `MAIN` whose flag
/// bit 0 is set exactly for the `set_tiles` `(x, y)` list. A trailing unknown chunk exercises
/// the seek-past-unknown path *after* `MAIN` too.
/// A minimal WDT: `MVER`, a zero `MPHD`, a `MAIN` flagging exactly `set_tiles`, and an unknown
/// chunk after it if `trailing`.
fn synth_wdt(set_tiles: &[(usize, usize)], trailing: bool) -> Vec<u8> {
let mut out = Vec::new();
let chunk = |magic: &[u8; 4], payload: &[u8], out: &mut Vec<u8>| {
@@ -312,11 +271,8 @@ mod tests {
out.extend_from_slice(&(payload.len() as u32).to_le_bytes());
out.extend_from_slice(payload);
};
// MVER (magic reversed on disk) — version 18, ignored by the reader.
chunk(b"REVM", &18u32.to_le_bytes(), &mut out);
// MPHD — 32 bytes of header we don't read.
chunk(b"DHPM", &[0u8; 32], &mut out);
// MAIN — 64×64 × 8 bytes; set bit 0 of `flags` for the requested tiles.
let count = MAP_SIZE * MAP_SIZE;
let mut main = vec![0u8; count * 8];
for &(x, y) in set_tiles {
@@ -334,8 +290,8 @@ mod tests {
WdtReader::new(Cursor::new(bytes), WowVersion::Classic).read()
}
/// A WMO-only map's on-disk shape: `MPHD` bit 0, an all-empty `MAIN`, then `MWMO` + one 64-byte
/// `MODF`. Field values are distinct primes-ish so a column slip can't pass.
/// A WMO-only map: `MPHD` bit 0, an empty `MAIN`, `MWMO` and one 64-byte `MODF` of distinct
/// values, so a column slip fails.
fn synth_wmo_only_wdt(mphd_bit0: bool, with_modf: bool) -> Vec<u8> {
let mut out = Vec::new();
let chunk = |magic: &[u8; 4], payload: &[u8], out: &mut Vec<u8>| {
@@ -352,15 +308,15 @@ mod tests {
if with_modf {
let mut e = [0u8; 64];
e[0..4].copy_from_slice(&0u32.to_le_bytes()); // nameId
e[4..8].copy_from_slice(&0xFFFF_FFFFu32.to_le_bytes()); // uniqueId (dead)
e[4..8].copy_from_slice(&0xFFFF_FFFFu32.to_le_bytes()); // uniqueId, unread
for (i, v) in [11.0f32, 22.0, 33.0].iter().enumerate() {
e[8 + i * 4..12 + i * 4].copy_from_slice(&v.to_le_bytes()); // position
}
for (i, v) in [1.0f32, 180.0, 2.0].iter().enumerate() {
e[20 + i * 4..24 + i * 4].copy_from_slice(&v.to_le_bytes()); // rotation
}
// 0x20..0x38 bounding box left zero — we don't read it.
e[56..58].copy_from_slice(&7u16.to_le_bytes()); // flags (dead)
// 0x20..0x38, the bounding box, left zero: unread.
e[56..58].copy_from_slice(&7u16.to_le_bytes()); // flags, unread
e[58..60].copy_from_slice(&3u16.to_le_bytes()); // doodadSet
e[60..62].copy_from_slice(&5u16.to_le_bytes()); // nameSet
chunk(b"FDOM", &e, &mut out);
@@ -368,8 +324,6 @@ mod tests {
out
}
/// The WMO-only branch end to end: the `MPHD` gate, the `MWMO` path, and every `MODF` column
/// the streamer places the building with. Position is read **raw** — no corner-origin remap.
#[test]
fn reads_the_global_wmo_of_a_wmo_only_map() {
let wdt = parse(synth_wmo_only_wdt(true, true)).expect("parses");
@@ -378,27 +332,21 @@ mod tests {
assert_eq!(g.position, [11.0, 22.0, 33.0]);
assert_eq!(g.rotation, [1.0, 180.0, 2.0]);
assert_eq!((g.doodad_set, g.name_set), (3, 5));
// Such a map authors no tiles — the grid still parses, it's just empty.
assert!(!wdt.get_tile(32, 32).expect("in range").has_adt);
}
/// An ADT map carries a zero-length `MWMO` stub and no `MODF`; it must not claim a global WMO
/// (which would spawn a phantom building over real terrain).
#[test]
fn an_adt_map_has_no_global_wmo() {
let wdt = parse(synth_wdt(&[(30, 30)], false)).expect("parses");
assert!(wdt.global_wmo().is_none());
// …and neither does a malformed file that sets the bit but ships no placement: a stray
// header bit must degrade to "no global WMO", never to a half-built one.
// Nor a file with the bit but no placement,
let half = parse(synth_wmo_only_wdt(true, false)).expect("parses");
assert!(half.global_wmo().is_none());
// …nor one that ships the placement without the flag.
// nor one with the placement but no bit.
let unflagged = parse(synth_wmo_only_wdt(false, true)).expect("parses");
assert!(unflagged.global_wmo().is_none());
}
/// The chunk grid nests in the tile grid: `chunk >> 4` is the tile `world_to_tile` names, at
/// every chunk centre of a whole tile and at the map's far clamp.
#[test]
fn the_chunk_index_nests_in_the_tile_index() {
let (ox, oy) = tile_to_world(32, 44);
@@ -418,14 +366,12 @@ mod tests {
#[test]
fn world_origin_is_map_center_tile() {
// World (0, 0) sits at the 32,32 tile boundary — the documented map center.
assert_eq!(world_to_tile(0.0, 0.0), (32, 32));
}
#[test]
fn tile_center_round_trips_exactly() {
// Sample the *center* of each tile (corner minus half a tile on both axes); a half-tile
// margin absorbs f32 rounding so truncation can't spill into a neighbour.
// Tile centres: the half-tile margin absorbs f32 rounding.
for &(tx, ty) in &[(0, 0), (1, 10), (20, 31), (33, 33), (50, 62), (63, 63)] {
let (cx, cy) = tile_to_world(tx, ty);
let center = (cx - 0.5 * T, cy - 0.5 * T);
@@ -448,7 +394,6 @@ mod tests {
#[test]
fn world_to_tile_clamps_both_extremes() {
// Far past the max corner saturates low; far past the min corner clamps to the last tile.
assert_eq!(world_to_tile(1.0e9, 1.0e9), (0, 0));
assert_eq!(world_to_tile(-1.0e9, -1.0e9), (63, 63));
}
@@ -459,7 +404,6 @@ mod tests {
assert!(wdt.get_tile(3, 5).unwrap().has_adt);
assert!(wdt.get_tile(63, 0).unwrap().has_adt);
assert!(wdt.get_tile(0, 63).unwrap().has_adt);
// An untouched tile reads empty, and the returned coords echo the query.
let empty = wdt.get_tile(10, 10).unwrap();
assert!(!empty.has_adt);
assert_eq!((empty.x, empty.y), (10, 10));
@@ -467,7 +411,6 @@ mod tests {
#[test]
fn skips_unknown_chunks_before_and_after_main() {
// MVER/MPHD precede MAIN and a bogus chunk follows it; all must be seek-skipped cleanly.
let wdt = parse(synth_wdt(&[(1, 1)], true)).unwrap();
assert!(wdt.get_tile(1, 1).unwrap().has_adt);
}
@@ -482,7 +425,6 @@ mod tests {
#[test]
fn missing_main_chunk_is_an_error() {
// Only MVER present — no MAIN ⇒ InvalidData.
let mut only_mver = Vec::new();
only_mver.extend_from_slice(b"REVM");
only_mver.extend_from_slice(&4u32.to_le_bytes());
@@ -493,7 +435,7 @@ mod tests {
#[test]
fn empty_stream_has_no_main() {
// A clean EOF at the very first header is not a truncation error, but still no MAIN.
// A clean EOF, not a truncation, but still no MAIN.
let err = parse(Vec::new()).unwrap_err();
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
}
@@ -505,10 +447,7 @@ mod tests {
assert_eq!(err.kind(), std::io::ErrorKind::UnexpectedEof);
}
/// A chunk claiming more bytes than the stream holds is refused before its buffer is sized:
/// the `MPHD`/`MWMO`/`MODF` reads used to `vec![0; size]` off the raw u32 first (a 4 GiB
/// commit a lazy allocator hides and Windows aborts on) and only then hit EOF. The message
/// pins that it is the guard, not the read after it, that refuses.
/// The message pins that the guard refuses, not the read after it.
#[test]
fn a_chunk_size_past_the_end_of_stream_is_refused_before_allocating() {
for magic in [b"DHPM", b"OMWM", b"FDOM"] {
crates/benilla-wmo/src/lib.rs
+64 -119
View File
@@ -1,24 +1,13 @@
//! A WMO (World Map Object) reader for **WoW 1.12.1 (build 5875)** — in-repo, replacing `wow-wmo`
//! (decision 0021), scoped to what the renderer consumes.
//!
//! A WMO is split across a **root** file (`MOHD` header, `MOTX` texture blob, `MOMT` materials, group
//! info, doodads, lights) and one **group** file per group (`MOGP` super-chunk wrapping `MOVT`/`MONR`/
//! `MOTV`/`MOVI`/`MOBA`/`MOCV`). [`parse_wmo`] returns whichever the bytes are. Chunks are IFF-style:
//! a 4-char magic stored **reversed** on disk, a `u32` size, then payload.
//!
//! Byte access goes through `benilla-bytes` (decision 0064): every read is bounds-checked and a
//! truncated chunk is a typed [`Error::Truncated`], never a panic or a silently-empty struct.
//!
//! Only the render path lives here; benilla-formats hand-parses MOLT/MOGI/MODS/MOPY itself. Proven
//! against `wow-wmo` over real WMOs during the decision-0021 migration (oracle test in git history);
//! the WMO mesh/collision golden tests in benilla-formats pin it end-to-end on every run.
//! A 1.12.1 WMO (World Map Object) reader, scoped to what the renderer consumes. A root file holds
//! `MOHD`, `MOTX` and `MOMT` among its tables; each group file holds a `MOGP` super-chunk wrapping
//! the geometry sub-chunks. A chunk is a 4-char magic stored reversed, a `u32` size, the payload.
use std::collections::HashMap;
use std::io::Cursor;
use benilla_bytes::{chunks, ByteExt};
/// A parsed WMO file — either the root or one group.
/// A parsed WMO file: the root or one group.
pub enum ParsedWmo {
Root(WmoRoot),
Group(WmoGroup),
@@ -27,8 +16,8 @@ pub enum ParsedWmo {
/// The WMO root: textures, materials, and the group count.
pub struct WmoRoot {
pub textures: Vec<String>,
/// MOTX byte-offset → index into [`textures`](Self::textures) (materials reference textures by the
/// byte offset of their name in the MOTX blob).
/// MOTX byte offset → index into [`textures`](Self::textures): a material names its texture by
/// the offset of the name in the blob.
pub texture_offset_index_map: HashMap<u32, u32>,
pub materials: Vec<Material>,
/// Group count (`MOHD.nGroups`).
@@ -41,33 +30,20 @@ pub struct Material {
pub blend_mode: u32,
/// Byte offset of texture 1's name in the MOTX blob.
pub texture_1: u32,
/// The authored **SIDN** self-illum colour (MOMT+0x10, stored BGRA on disk — here RGB). The real
/// client's per-frame updater scales it by the night fraction and adds it as the material
/// EMISSION on lit batches (`flags & 0x10` — the windows-glow-at-night mechanism). Meaningless
/// (usually zero) when the SIDN flag is clear.
/// The SIDN self-illumination colour (MOMT+0x10, BGRA on disk, RGB here): the client scales it
/// by the night fraction and adds it as emission on batches with `flags & 0x10`, the windows'
/// glow at night.
pub sidn_rgb: [u8; 3],
/// **MOMT+0x1C — `diffColor`**, the material's authored diffuse colour (BGRA on disk, RGB here).
///
/// Read for exactly one consumer: it is the **body colour of an interior WMO liquid pool**. The
/// reference's interior water kernel `0x6b6420` runs with lighting forced off and no pixel
/// shader, and its whole RGB is this dword taken raw — `Cf = MOMT[MLIQ.materialId].diffColor`,
/// combined with the animated sheet as `clamp(Cf + Ct)`. The pool's material's own alpha at
/// `+0x1f` is discarded; opacity comes from the per-vertex LUT instead.
///
/// The offset is anchored on both sides by fields this repo verified independently of water:
/// `+0x18` texture 2 and `+0x20` [`Self::ground_type`]. Stride 0x40, base `[CMapObj+0x1d8]`
/// (`0x6c3ace` + `0x6c3ad7 shr edx,6`).
/// MOMT+0x1C `diffColor` (BGRA on disk, RGB here), read only as an interior liquid's body
/// colour: the reference's interior water kernel `0x6b6420` runs unlit with no pixel shader,
/// its RGB this dword raw, `Cf = MOMT[MLIQ.materialId].diffColor`, combined with the animated
/// sheet as `clamp(Cf + Ct)`. Its alpha at `+0x1f` is discarded; opacity comes per vertex.
/// Stride 0x40, base `[CMapObj+0x1d8]` (`0x6c3ace`, `0x6c3ad7 shr edx,6`).
pub diff_color: [u8; 3],
/// **MOMT+0x20 — the surface's `TerrainType.dbc` id**, i.e. what walking on this material
/// sounds like. The footstep chain's WMO leg: the client's down-ray arbitrates a terrain probe
/// against a WMO probe and the nearer surface supplies the terrain type, so a building's own
/// floor — not the ADT beneath it — decides the step indoors.
///
/// Verified from the shipped 5875 data, not from a format doc: across all **815 root WMOs /
/// 10 299 materials** this dword only ever holds `{0,1,2,3,4,5,7,8,10}` — exactly the
/// `TerrainType.dbc` id domain (ids 0–10) — and **10 = "None" on 10 075 of them**, the
/// unauthored default, which resolves through `SoundID 0` to the generic `*Dirt` kit rather
/// than to silence. Only ~224 materials name a surface explicitly.
/// MOMT+0x20: the surface's `TerrainType.dbc` id, what a footstep on it sounds like. The
/// client's down-ray takes the nearer of the terrain and WMO probes, so a building's floor
/// decides the step indoors. The 5875 data holds only `{0,1,2,3,4,5,7,8,10}`; 10 ("None", on
/// 10 075 of 10 299 materials) resolves through `SoundID 0` to the generic `*Dirt` kit.
pub ground_type: u32,
}
@@ -110,62 +86,40 @@ pub struct Batch {
pub material_id: u8,
}
/// A MOPY per-face material/flags entry (2 bytes), one per triangle — drives collision filtering.
/// A MOPY entry, one triangle's flags and material (2 bytes); it drives collision filtering.
pub struct MopyEntry {
pub flags: u8,
pub material_id: u8,
}
/// One WMO group's MLIQ liquid grid (the water/lava/slime surface embedded in the group — canals,
/// fountains, dungeon pools). A flat `xverts × yverts` height grid over `xtiles × ytiles` cells, in
/// **WMO model space** (WoW axes, yards); the placement transform lifts it into the world. Cell
/// membership + type come from the per-tile flag low nibble (`& 0xf`): `0xf` = hole (no liquid), else
/// the nibble is the liquid type indexing the reference's texture table (render dispatch
/// `0x6b62e0`, water kernels `0x6b6420`/`0x6b6630`, magma/slime `0x6b68f0`).
/// A WMO group's MLIQ liquid grid: `xverts × yverts` heights over `xtiles × ytiles` cells in WMO
/// model space (WoW axes, yards). A tile flag's low nibble is `0xf` for a hole, else the liquid
/// type indexing the reference's texture table (dispatch `0x6b62e0`, water `0x6b6420` and
/// `0x6b6630`, magma and slime `0x6b68f0`).
pub struct WmoLiquid {
/// Vertex-grid dimensions (`xtiles = xverts − 1`, `ytiles = yverts − 1`).
/// Vertex-grid dimensions (`xtiles = xverts - 1`, `ytiles = yverts - 1`).
pub xverts: u32,
pub yverts: u32,
pub xtiles: u32,
pub ytiles: u32,
/// The `(0,0)` grid corner in WMO model space (WoW `[x, y, z]`, yards). Grid vertex `(i, j)` sits
/// at `(base.x + i·MLIQ_CELL_STEP, base.y + j·MLIQ_CELL_STEP, heights[j·xverts + i])`.
/// The `(0,0)` grid corner in WMO model space (WoW `[x, y, z]`, yards). Grid vertex `(i, j)`
/// sits at `(base.x + i·MLIQ_CELL_STEP, base.y + j·MLIQ_CELL_STEP, heights[j·xverts + i])`.
pub base: [f32; 3],
/// MLIQ `materialId` (index into the root MOMT — unused by the animated-water render path, which
/// selects its texture from the tile-flag type; kept for completeness).
/// MLIQ `materialId`, an index into the root MOMT, whose `diffColor` colours an interior pool.
pub material_id: u16,
/// Per-vertex absolute liquid height (WMO-local Z), row-major `j·xverts + i`, `xverts·yverts` long.
/// The MLIQ vertex is 8 bytes and the height is its trailing `f32`; what the leading 4 bytes mean
/// is per liquid type — see [`Self::opacity`] for the water reading.
/// Per-vertex liquid height (WMO-local Z), row-major `j·xverts + i`: the trailing `f32` of each
/// 8-byte MLIQ vertex, whose leading 4 bytes depend on the liquid type.
pub heights: Vec<f32>,
/// Per-vertex **opacity index** — the water vertex's byte 0, row-major beside [`Self::heights`].
///
/// WMO liquid carries no bathymetry, so nothing here can be ramped by depth the way ADT MCLQ's
/// water is; this byte **is** the authored opacity. The reference indexes a 256-entry alpha ramp
/// (`[0xca7f10]`, rebuilt per frame from the zone's river shallow/deep alpha pair) with it and
/// uses the result as the vertex alpha — the water arms bind no depth-ramp texture at all
/// (zero references to the ADT ramp globals `0xc7fbc0`/`0xc81768`/`0xc7fcd8` anywhere in
/// `[0x6b0000, 0x6c4000)`).
///
/// It was parsed as nothing for a long time — the field it replaced was documented as "flow
/// data" — which is why every WMO pool in benilla rendered at one pinned opacity.
///
/// On the **magma/slime** arm the same four bytes are an authored `int16` `(s, t)` UV pair
/// instead, so byte 0 there is half a texture coordinate and means nothing as an opacity. The
/// consumer picks by liquid type; this parser just carries the byte.
///
/// That split is also what corroborates the reading against the shipped data, independently of
/// the disassembly. Censused over every drawn liquid vertex in the corpus: on the **water** arms
/// the byte is structured — Stormwind's canals are **91.2% a single value (86)** across 3492
/// vertices with only 41 distinct values, exactly the shape of an authored per-surface opacity —
/// while on **magma/slime** it is near-uniform noise over all 256 values (each ≈0.7–1.4%), which
/// is what half of a texture coordinate should look like. A byte that is a constant on one arm
/// and white noise on the other is not being read at the wrong offset.
/// Per-vertex opacity index, a water vertex's byte 0, row-major beside [`Self::heights`]. WMO
/// liquid has no depth, so this byte is the authored opacity: the reference indexes a 256-entry
/// alpha ramp with it (`[0xca7f10]`, rebuilt per frame from the zone's river shallow and deep
/// alphas) and binds no depth-ramp texture (no reference to the ADT ramp globals `0xc7fbc0`,
/// `0xc81768`, `0xc7fcd8` in `[0x6b0000, 0x6c4000)`). On magma and slime the four bytes are an
/// `int16` `(s, t)` UV pair instead; the consumer picks by liquid type.
pub opacity: Vec<u8>,
/// Per-tile flag bytes, row-major `j·xtiles + i`, `xtiles·ytiles` long. Low nibble = liquid type;
/// `0xf` = hole (skip the tile); `0x80` shared (the strip-builder gate). `0x40` ("fishable") is
/// carried but deliberately unread: the reference's WMO-side reader `0x6b9e50` is only reached
/// from the zero-caller `0x69b5d0` island — fishability is the server's verdict. Decision 1825.
/// Per-tile flags, row-major `j·xtiles + i`: the low nibble the liquid type (`0xf` a hole),
/// `0x80` shared (the strip-builder gate). `0x40` (fishable) is unread: the reference's reader
/// `0x6b9e50` is reachable only from the uncalled `0x69b5d0`, and the server decides fishing.
pub tile_flags: Vec<u8>,
}
@@ -173,9 +127,8 @@ pub struct WmoLiquid {
pub struct WmoGroup {
/// MOGP group flags (bit test `& 0x48` selects exterior vs interior).
pub flags: u32,
/// MOGP `groupLiquid` @ header `0x34` — the liquid **type override**: when `!= 0xf` it names the
/// whole group's liquid type directly (`0x6b9f10`); `0xf` (the common
/// case) defers to the per-tile MLIQ nibble. `0xf` when the header is too short to carry it.
/// MOGP `groupLiquid` at header `0x34`: other than `0xf` it overrides every tile's liquid type
/// (`0x6b9f10`); `0xf`, also the value for a header too short to carry it, defers to the tiles.
pub group_liquid: u32,
pub vertex_positions: Vec<Vec3>,
pub vertex_normals: Vec<Vec3>,
@@ -183,9 +136,7 @@ pub struct WmoGroup {
pub vertex_colors: Vec<Color>,
pub vertex_indices: Vec<u16>,
pub render_batches: Vec<Batch>,
/// MOPY per-face material/flags (one per triangle).
pub material_info: Vec<MopyEntry>,
/// The group's MLIQ liquid grid, if it carries one (`QILM` sub-chunk present).
pub liquid: Option<WmoLiquid>,
}
@@ -240,10 +191,10 @@ fn parse_root(b: &[u8]) -> Result<WmoRoot> {
root.textures = t;
root.texture_offset_index_map = m;
}
// MOMT entry is 64 bytes: flags@0, shader@4, blend_mode@8, texture_1@12, sidnColor@16, …
// A 64-byte MOMT entry: flags@0, shader@4, blend_mode@8, texture_1@12, sidnColor@16.
b"TMOM" => {
for m in p.as_chunks::<64>().0 {
// CImVector — BGRA bytes on disk, so the LE u32 reads B | G<<8 | R<<16 | A<<24.
// CImVector: BGRA on disk, so the LE u32 reads B | G<<8 | R<<16 | A<<24.
let sidn = m.u32_at(0x10).ok_or(Error::Truncated("MOMT"))?;
let diff = m.u32_at(0x1c).ok_or(Error::Truncated("MOMT"))?;
root.materials.push(Material {
@@ -262,8 +213,7 @@ fn parse_root(b: &[u8]) -> Result<WmoRoot> {
Ok(root)
}
/// MOTX is a NUL-separated string blob; materials reference each name by its byte offset. Build the
/// texture list + an offset→index map (matching wow-wmo: record the offset at each string's first byte).
/// MOTX, a NUL-separated name blob: the texture list, and each name's first-byte offset → index.
fn parse_motx(blob: &[u8]) -> (Vec<String>, HashMap<u32, u32>) {
let mut textures = Vec::new();
let mut map = HashMap::new();
@@ -283,12 +233,10 @@ fn parse_motx(blob: &[u8]) -> (Vec<String>, HashMap<u32, u32>) {
(textures, map)
}
/// Parse a MOGP super-chunk payload: a 68-byte group header (flags @8) then the geometry sub-chunks.
/// A payload shorter than the header parses to an empty group (a real-data tolerance, kept from the
/// pre-0064 reader) — but a *sub-chunk* shorter than its record size is a hard [`Error::Truncated`].
/// Parse a MOGP payload: a 68-byte header (flags at 8), then the geometry sub-chunks. A payload
/// shorter than the header is an empty group.
fn parse_group(mogp: &[u8]) -> Result<WmoGroup> {
let flags = mogp.u32_at(8).unwrap_or(0);
// groupLiquid @ 0x34 (the type override; 0xf = "use the per-tile MLIQ nibble"). Absent header ⇒ 0xf.
let group_liquid = mogp.u32_at(0x34).unwrap_or(0xf);
let mut g = WmoGroup {
flags,
@@ -305,8 +253,8 @@ fn parse_group(mogp: &[u8]) -> Result<WmoGroup> {
if mogp.len() < 68 {
return Ok(g);
}
// `extend` (not assign): a sub-chunk that appears more than once concatenates, matching wow-adt's
// push-loop (some WMOs carry a second MOTV, e.g. KL_PvPBarracks).
// Push, never assign: a repeated sub-chunk concatenates (some WMOs carry a second MOTV, e.g.
// KL_PvPBarracks).
for (magic, s) in chunks(&mogp[68..]) {
match &magic {
b"TVOM" => {
@@ -341,7 +289,8 @@ fn parse_group(mogp: &[u8]) -> Result<WmoGroup> {
.push(c.u16_at(0).ok_or(Error::Truncated("MOVI"))?);
}
}
// MOBA: bbox_min[i16;3] bbox_max[i16;3] start_index(u32@12) count(u16@16) min/max(u16) flags(u8@22) material_id(u8@23)
// MOBA: bbox_min[i16;3] bbox_max[i16;3] start_index(u32@12) count(u16@16)
// min/max(u16) flags(u8@22) material_id(u8@23)
b"ABOM" => {
for c in s.as_chunks::<24>().0 {
g.render_batches.push(Batch {
@@ -361,7 +310,6 @@ fn parse_group(mogp: &[u8]) -> Result<WmoGroup> {
});
}
}
// MOPY: 2 bytes/face — flags + material id, one per triangle (collision filtering).
b"YPOM" => {
for c in s.as_chunks::<2>().0 {
g.material_info.push(MopyEntry {
@@ -370,8 +318,6 @@ fn parse_group(mogp: &[u8]) -> Result<WmoGroup> {
});
}
}
// MLIQ: the group's liquid grid — a 30-byte SMOLiquidHeader, then `xverts·yverts`
// 8-byte vertices (height at +4), then `xtiles·ytiles` 1-byte tile flags.
b"QILM" => g.liquid = parse_mliq(s)?,
_ => {}
}
@@ -379,11 +325,11 @@ fn parse_group(mogp: &[u8]) -> Result<WmoGroup> {
Ok(g)
}
/// Parse an `MLIQ` sub-chunk payload into a [`WmoLiquid`]. Returns `Ok(None)` when the payload is
/// too short for the header or its declared grid (a truncated/garbage chunk yields no liquid rather
/// than a hard error — the group still renders its geometry).
/// Parse an `MLIQ` payload into a [`WmoLiquid`]; no liquid when the grid is empty, oversized or
/// longer than the payload, so the group still renders.
fn parse_mliq(s: &[u8]) -> Result<Option<WmoLiquid>> {
// SMOLiquidHeader: xverts@0 yverts@4 xtiles@8 ytiles@12 base[f32;3]@16 materialId(u16)@28 (30 B).
// SMOLiquidHeader, 30 B: xverts@0 yverts@4 xtiles@8 ytiles@12 base[f32;3]@16
// materialId(u16)@28; then 8-byte vertices (height at +4), then 1-byte tile flags.
let (Some(xverts), Some(yverts), Some(xtiles), Some(ytiles)) =
(s.u32_at(0), s.u32_at(4), s.u32_at(8), s.u32_at(12))
else {
@@ -391,7 +337,7 @@ fn parse_mliq(s: &[u8]) -> Result<Option<WmoLiquid>> {
};
let nverts = (xverts as usize).saturating_mul(yverts as usize);
let ntiles = (xtiles as usize).saturating_mul(ytiles as usize);
// Sanity-cap the grid so a corrupt header can't ask us to allocate gigabytes (real grids are ≤~32²).
// A corrupt header must not size gigabytes; real grids are about 32² at most.
if nverts == 0 || ntiles == 0 || nverts > 1 << 20 || ntiles > 1 << 20 {
return Ok(None);
}
@@ -404,13 +350,12 @@ fn parse_mliq(s: &[u8]) -> Result<Option<WmoLiquid>> {
let vbase = 30usize;
let tbase = vbase + nverts * 8;
if s.len() < tbase + ntiles {
return Ok(None); // declared grid runs past the payload — treat as no liquid
return Ok(None); // the declared grid runs past the payload: no liquid
}
let heights: Vec<f32> = (0..nverts)
.map(|k| s.f32_at(vbase + k * 8 + 4).unwrap_or(0.0))
.collect();
// Byte 0 of the same 8-byte vertex — the water opacity index (see `WmoLiquid::opacity`). The
// bounds are already guaranteed by the `tbase` check above.
// Byte 0 of each vertex, in bounds by the `tbase` check above.
let opacity: Vec<u8> = (0..nverts).map(|k| s[vbase + k * 8]).collect();
let tile_flags = s[tbase..tbase + ntiles].to_vec();
Ok(Some(WmoLiquid {
@@ -488,7 +433,7 @@ mod tests {
#[test]
fn group_parses_mliq_liquid_grid() {
// A 2×2-vertex (1×1-tile) MLIQ: header + 4 verts (8 B each, height at +4) + 1 tile byte.
// A 2×2-vertex (1×1-tile) MLIQ: header, 4 verts (8 B each, height at +4), 1 tile byte.
let mut mliq = Vec::new();
mliq.extend(2u32.to_le_bytes()); // xverts
mliq.extend(2u32.to_le_bytes()); // yverts
@@ -497,10 +442,10 @@ mod tests {
mliq.extend([10.0f32, 20.0, -5.0].map(f32::to_le_bytes).concat()); // base xyz
mliq.extend(115u16.to_le_bytes()); // materialId
for h in [-5.0f32, -5.0, -5.0, -5.0] {
mliq.extend([0u8, 0, 0, 0]); // flow bytes
mliq.extend([0u8, 0, 0, 0]); // leading bytes, opacity first
mliq.extend(h.to_le_bytes()); // height
}
mliq.push(0x44); // tile flag: nibble 4 (lake_a), fishable (0x40) — a wet tile
mliq.push(0x44); // tile flag: type 4 (lake_a), fishable (0x40)
let mut mogp = vec![0u8; 68];
mogp[8..12].copy_from_slice(&0x2000u32.to_le_bytes()); // flags
@@ -522,7 +467,7 @@ mod tests {
#[test]
fn truncated_mliq_yields_no_liquid_not_a_panic() {
// A header that declares a grid larger than the payload → None (the group still parses).
// A header declaring a grid larger than the payload: no liquid, and the group still parses.
let mut mliq = Vec::new();
mliq.extend(9u32.to_le_bytes()); // xverts
mliq.extend(9u32.to_le_bytes()); // yverts
@@ -530,7 +475,7 @@ mod tests {
mliq.extend(8u32.to_le_bytes()); // ytiles
mliq.extend([0.0f32, 0.0, 0.0].map(f32::to_le_bytes).concat());
mliq.extend(0u16.to_le_bytes());
// …but no vertex/tile bytes follow.
// and no vertex or tile bytes follow.
let mut mogp = vec![0u8; 68];
mogp.extend(chunk(b"QILM", &mliq));
let b = chunk(b"PGOM", &mogp);
@@ -542,7 +487,7 @@ mod tests {
#[test]
fn truncated_mohd_is_an_error_not_a_panic() {
// MOHD payload of 3 bytes: the pre-0064 reader panicked on rd_u32(p, 4).
// A 3-byte MOHD payload cannot hold nGroups at 4.
let b = chunk(b"DHOM", &[0u8; 3]);
assert!(matches!(parse(&b), Err(Error::Truncated("MOHD"))));
}
@@ -551,11 +496,11 @@ mod tests {
fn hostile_shapes_do_not_panic() {
assert!(matches!(parse(&[]), Err(Error::NotWmo)));
assert!(matches!(parse(&[0u8; 7]), Err(Error::NotWmo)));
// Over-declared chunk size clamps (lenient) — parses to an empty-ish root.
// An over-declared chunk size clamps, leniently.
let mut b = chunk(b"DHOM", &[0u8; 64]);
b[4..8].copy_from_slice(&u32::MAX.to_le_bytes());
assert!(matches!(parse(&b), Err(Error::Truncated("MOHD")) | Ok(_)));
// A short MOGP (below the 68-byte header) is an empty group, kept from the old reader.
// A MOGP shorter than its 68-byte header is an empty group.
let b = chunk(b"PGOM", &[0u8; 10]);
let Ok(ParsedWmo::Group(g)) = parse(&b) else {
panic!("short MOGP should parse to an empty group");
crates/benilla-worldview/build.rs
+1 -1
View File
@@ -1,4 +1,4 @@
//! Stamps the commit this binary was built from — see `benilla-buildstamp`, which owns the rule.
//! Stamps the commit this binary was built from.
fn main() {
benilla_buildstamp::emit();
crates/benilla-worldview/src/main.rs
+3 -15
View File
@@ -1,18 +1,6 @@
//! The `benilla-worldview` launcher — the world viewer's shim, the `benilla` launcher's twin.
//!
//! **What this binary is for** (decision 1160): `benilla-app` is being split into `benilla-world`
//! (the engine: renders a WoW world, streams it, flies over it, clicks on it) and the game that
//! stands on it. A crate boundary alone cannot hold that line — in Bevy, code couples through
//! *resources at runtime* with no symbol crossing between the crates — so the enforcer is a second
//! program that boots the engine with **no game attached**. It is built by
//! `cargo clippy --workspace --all-targets`, which is already a gate, so wiring a game concept back
//! into the engine breaks it the same day.
//!
//! It is also where `benilla-editor` starts: a window that loads a map and lets you fly around is
//! the editor's first milestone.
//!
//! Like `benilla`, this is a shim so the every-commit git stamp dirties ~20 lines instead of the
//! app crate (decision 0993).
//! The `benilla-worldview` launcher: boots the `benilla-world` engine with no game attached, so an
//! engine system that needs a gameplay resource fails here, a coupling no compiler sees. Like
//! `benilla`, a shim that carries the build stamp, so a new commit recompiles only these lines.
use benilla_world::build_id::BuildId;
crates/benilla/build.rs
+1 -2
View File
@@ -1,5 +1,4 @@
//! Stamps the commit this binary was built from — see `benilla-buildstamp`, which owns the rule
//! (and the reason it lives in this ~30-line shim rather than in `benilla-app`: decision 0993).
//! Stamps the commit this binary was built from.
fn main() {
benilla_buildstamp::emit();
crates/benilla/src/main.rs
+2 -6
View File
@@ -1,9 +1,5 @@
//! The `benilla` launcher — a shim whose whole job is to carry `build.rs`'s git stamp.
//!
//! Everything real lives in `benilla-app`; this package exists so the build-id stamp — whose
//! watched git paths make cargo re-dirty the package on **every** commit, rebase, and checkout
//! (see `build.rs`'s header) — invalidates these few lines and a relink instead of the whole app
//! crate, its clippy pass, and its integration-test links. Decision 0993 has the measurements.
//! The `benilla` launcher: a shim that carries the build stamp, so a new commit recompiles these
//! few lines and relinks instead of rebuilding `benilla-app`.
use benilla_app::BuildId;