grom/benilla-octo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Comments in the small crates: one line of fact each, no history, no record numbers (#93)

Browse Source 
Part of #41: the comments of the thirteen small crates (`benilla-adt`, `-blp`, `-buildstamp`, `-bytes`, `-dbc`, `-m2`, `-mpq`, `-srp`, `-visual`, `-wdt`, `-wmo`, `-worldview` and `benilla`) rewritten to the rule in `docs/METHOD.md`: what the code does and the reference fact behind it, in a line. 2,176 comment lines become 1,029.

- The citations that were only a private record number go first, mechanically, each edit shown to be a pure removal.
- The rewrite corrects the comments that stated wrong facts: the M2 header's selection-ring claim, the texture-transform and MCNK impassable-flag notes, a DBC reader note that said short records zero-pad (they do not, #17), and about twenty more found on the way. The SRP handshake's redraw and redial are marked as a deviation, with the measured reason.
- Verified: every file strips to identical code, and `cargo fmt --check` is clean. A second reader compared every file old against new and restored the facts the rewrite had dropped.

String literals are untouched; they are #92.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Sam Sung
2026-09-24 15:51:18 +07:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 1ee056fc53
commit bf8a15de02
30 changed files with 842 additions and 1993 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/benilla-visual/src/lib.rs
+63 -152
View File
@@ -1,45 +1,31 @@
//! Perceptual image-diff metrics for the Phase-5 visual A/B render harness (decision 0008).
//!
//! The harness captures deterministic screenshots of benilla (`$WOW_CAPTURE`, see the `capture` module
//! in the `benilla` crate) and diffs them. This crate is the pure-math half: given two equally-sized
//! images it reports how far they diverge ([`Metrics`]) and renders a heatmap of *where* ([`diff_image`]).
//!
//! Three uses, one tool: (1) **self-regression** — capture baselines on the current pipeline, then diff
//! every linear-HDR rework step against them so a machine catches a regression before the director's
//! eye; (2) **determinism check** — diffing two captures of the same scenario must come out ≈0, which is
//! what makes (1) trustworthy; (3) **flicker** ([`envelope`]) — the same arithmetic over *time* instead
//! of over versions: a burst of adjacent frames from a parked camera (`WOW_LIVE_SHOT_COUNT`), collapsed
//! to where the picture would not hold still.
//! Image-diff metrics for the capture harness (`WOW_CAPTURE`, the `capture` module in
//! `benilla-app`): how far two equal-size images diverge ([`Metrics`]) and where ([`diff_image`]),
//! and over a burst of frames, which pixels would not hold still ([`envelope`], [`toggles`]).
pub mod relight;
use image::RgbImage;
/// A pixel counts as "changed" for [`Metrics::pct_over`] if any channel differs by more than this many
/// byte units. 8/255 ≈ 3% — above dithering/rounding noise, below a real visual shift.
/// A pixel counts toward [`Metrics::pct_over`] when a channel differs by more than this (8/255,
/// about 3%): above dithering noise, below a real visual shift.
pub const OVER_THRESHOLD: u8 = 8;
/// Per-image difference metrics. Channel deltas are in 0..255 byte units; `pct_over` is a fraction 0..1.
/// Per-image difference metrics: channel deltas in 0..255 byte units, fractions in 0..1.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Metrics {
/// Mean absolute per-channel difference (0..255). The headline "how different" number.
/// Mean absolute per-channel difference (0..255).
pub mae: f64,
/// Root-mean-square per-channel difference (0..255) — weights large local deltas more than `mae`.
/// Root-mean-square per-channel difference (0..255), weighting large local deltas.
pub rmse: f64,
/// Largest single-channel difference anywhere (0..255).
pub max_delta: u8,
/// Fraction of pixels (0..1) whose largest channel delta exceeds [`OVER_THRESHOLD`].
pub pct_over: f64,
/// How many pixels differ **at all** (any channel, by any amount).
///
/// The number that separates "a render changed" from "a silhouette pixel landed on the other
/// side of an MSAA tie": both read as a scary `max_delta` and an `mae` of 0.000, and only the
/// count tells them apart. A handful of pixels in a 5.76 M-pixel frame is a tie flipping with
/// the binary; a region is a regression. Answering that used to take a hand-rolled PNG
/// decoder in a scratchpad, every time.
/// Pixels that differ at all: a handful is an MSAA tie flipping, a region a regression, though
/// both show a large `max_delta` and an `mae` of 0.000.
pub changed: u64,
/// Where the largest delta is, `(x, y)` — `(0, 0)` when nothing differs. A tie flip recurs at
/// the *same* coordinate build after build, which is most of the evidence that it is one.
/// Where the largest delta is, `(0, 0)` when nothing differs; a tie flip recurs at the same
/// spot build after build.
pub worst_at: (u32, u32),
}
@@ -90,9 +76,7 @@ pub fn compare(a: &RgbImage, b: &RgbImage) -> anyhow::Result<Metrics> {
})
}
/// Render an amplified per-channel abs-difference image: each output channel is `|a-b| * amplify`,
/// clamped to 255. This colourises both *where* and *in which channel* the images diverge (a red shift
/// shows red), so the diff is readable at a glance. Errors on a size mismatch.
/// The per-channel difference `|a - b| * amplify`, clamped to 255, so a red shift shows red.
pub fn diff_image(a: &RgbImage, b: &RgbImage, amplify: u32) -> anyhow::Result<RgbImage> {
if a.dimensions() != b.dimensions() {
anyhow::bail!(
@@ -114,12 +98,12 @@ pub fn diff_image(a: &RgbImage, b: &RgbImage, amplify: u32) -> anyhow::Result<Rg
Ok(out)
}
/// How far a burst of frames moved, and where — the output of [`envelope`].
/// How far a burst of frames moved, and where: [`envelope`]'s output.
#[derive(Debug, Clone)]
pub struct Envelope {
/// Per-channel `(max − min)` across the stack, amplified — the picture of *where* it flickers.
/// Per-channel `max - min` across the stack, amplified.
pub image: RgbImage,
/// Largest single-channel swing anywhere (0..255). The headline "does it flicker at all".
/// Largest single-channel swing anywhere (0..255).
pub max_swing: u8,
/// Mean per-channel swing (0..255) over the whole frame.
pub mean_swing: f64,
@@ -127,14 +111,9 @@ pub struct Envelope {
pub pct_unstable: f64,
}
/// Collapse a burst of equally-sized frames to their per-pixel **envelope** — `max − min` per channel
/// across the whole stack. This is the flicker instrument: a still image cannot show a temporal
/// artefact, but the envelope of adjacent frames from a *parked* camera can, and it localises the
/// unstable pixels in one picture instead of `n−1` pairwise diffs.
///
/// A pixel that holds still contributes 0 whatever its colour, so a scene that is merely *wrong* comes
/// out black — only what refuses to settle lights up. Errors on fewer than two frames or a size
/// mismatch.
/// A burst's per-pixel envelope, `max - min` per channel across the whole stack: from a parked
/// camera only what refuses to settle lights up, however wrong a still pixel is. Errors on fewer
/// than two frames or a size mismatch.
pub fn envelope(frames: &[RgbImage], amplify: u32) -> anyhow::Result<Envelope> {
let (w, h) = burst_dimensions(frames, 2)?;
let mut image = RgbImage::new(w, h);
@@ -169,38 +148,26 @@ pub fn envelope(frames: &[RgbImage], amplify: u32) -> anyhow::Result<Envelope> {
})
}
/// How often a burst *reversed direction*, and where — the output of [`toggles`].
/// How often a burst reversed direction, and where: [`toggles`]'s output.
#[derive(Debug, Clone)]
pub struct Toggles {
/// Per-channel reversal count across the stack, amplified — the picture of *where* it toggles.
/// Per-channel reversal count across the stack, amplified.
pub image: RgbImage,
/// Most reversals any one channel made (0..frames−2).
/// Most reversals any one channel made (0..frames-2).
pub max_reversals: u32,
/// Fraction of pixels (0..1) that reversed at least [`TOGGLE_MIN_REVERSALS`] times.
pub pct_toggling: f64,
/// Row-major `width × height` flags: did this pixel toggle? Carried out of the scan because
/// *where* the toggling pixels sit relative to each other is a separate reading ([`shape`]) and
/// re-thresholding the amplified image would only recover it approximately.
/// Row-major per-pixel toggle flags for [`shape`], exact where re-thresholding `image` is not.
pub mask: Vec<bool>,
}
/// A pixel counts as "toggling" for [`Toggles::pct_toggling`] at this many direction reversals. Two
/// is the smallest count that cannot be a single overshoot, and it already needs a four-frame
/// up-down-up run — which smooth motion across an edge does not produce.
/// Direction reversals that make a pixel toggle: two cannot be a single overshoot, and smooth
/// motion across an edge never makes the up-down-up run they need.
pub const TOGGLE_MIN_REVERSALS: u32 = 2;
/// Count, per pixel and channel, how many times the value **reversed direction** across the burst,
/// ignoring steps smaller than `min_delta`.
///
/// This is [`envelope`]'s companion for the case the envelope cannot serve: **a moving camera**.
/// When the view is panning, every edge in the frame sweeps across pixels and `max − min` lights up
/// the whole picture, so it says nothing. But smooth motion is *monotone* — a pixel being crossed by
/// an edge goes one way and stays going that way — while z-fighting and an unstable draw order
/// **alternate**, A/B/A/B, however the camera moves. Counting reversals separates the two.
///
/// The director's report is what forced this: the Far Watch Post tower "mostly only flickers while
/// moving the cam, not while still" — the one condition under which a parked burst is blind by
/// construction (decision 0653's blind spot, named in 0656).
/// Count, per pixel and channel, the direction reversals across the burst, ignoring steps under
/// `min_delta`: the flicker reading for a moving camera. Smooth motion is monotone; z-fighting and
/// an unstable draw order alternate however the camera moves.
pub fn toggles(frames: &[RgbImage], min_delta: u8, amplify: u32) -> anyhow::Result<Toggles> {
let (w, h) = burst_dimensions(frames, 3)?;
let mut image = RgbImage::new(w, h);
@@ -210,8 +177,7 @@ pub fn toggles(frames: &[RgbImage], min_delta: u8, amplify: u32) -> anyhow::Resu
for (x, y, out) in image.enumerate_pixels_mut() {
let mut pixel_max = 0u32;
for c in 0..3 {
// `dir` is the sign of the last step big enough to count; a run of sub-threshold steps
// neither reverses nor resets it, so dithering noise cannot manufacture a toggle.
// `dir` is the last counted step's sign; smaller steps neither reverse nor reset it.
let (mut dir, mut reversals) = (0i8, 0u32);
for pair in frames.windows(2) {
let (a, b) = (
@@ -246,7 +212,7 @@ pub fn toggles(frames: &[RgbImage], min_delta: u8, amplify: u32) -> anyhow::Resu
})
}
/// A rectangle of the frame, in pixels — `x1`/`y1` exclusive.
/// A rectangle of the frame in pixels, `x1` and `y1` exclusive.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Rect {
pub x0: u32,
@@ -279,10 +245,7 @@ pub struct Region {
pub bounds: Rect,
/// Toggling pixels in the run (≤ the bounds' area).
pub pixels: u64,
/// Up to [`REGION_SAMPLES`] pixels spread evenly through the run, in scan order — coordinates
/// Every pixel in the run, in scan order. Kept whole because the run's *own* time series
/// ([`Region::steps`]) is the reading that follows, and it has to be over the run, not over a
/// bounding box that also contains whatever else was passing through.
/// Every pixel in the run, in scan order; [`Region::steps`] reads these, not the bounding box.
pub members: Vec<(u32, u32)>,
}
@@ -292,33 +255,25 @@ pub const REGION_SAMPLES: usize = 8;
/// One frame-to-frame step of a run, from [`Region::steps`].
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Step {
/// The run's mean luma (0..255) in the earlier frame. The *level*, not just the change: a
/// two-state flip's ratio is what identifies the multiplier behind it, and a delta alone hides it.
/// The run's mean luma (0..255) in the earlier frame.
pub mean_from: f64,
/// The run's mean R/G/B (0..255) in the earlier frame. Whether a two-state flip preserves the
/// channel ratios says what kind of flip it is: a pure **intensity** scale (one light term
/// switching on or off) keeps the hue, while swapping to a differently-coloured lighting lane
/// does not — and luma alone cannot tell those apart.
/// The run's mean R, G and B (0..255) in the earlier frame.
pub mean_rgb_from: [f64; 3],
/// Change in the run's mean luma (0..255 units) from this frame to the next.
pub mean_delta: f64,
/// Fraction of the run's pixels (0..1) that moved the **same way** as the mean did. Near 1 = the
/// whole surface changed together; near 0.5 = as many pixels went up as down.
/// Fraction of the run's pixels (0..1) that moved the same way as the mean: near 1 the surface
/// changed together, near 0.5 as many went up as down.
pub agreement: f64,
}
impl Region {
/// How much of its own bounding box the run actually fills, 0..1. A solid surface approaches 1;
/// a scattering of speckles inside a wide box is near 0.
/// How much of its bounding box the run fills, 0..1: a solid surface nears 1, speckle 0.
pub fn fill(&self) -> f64 {
let area = u64::from(self.bounds.width()) * u64::from(self.bounds.height());
self.pixels as f64 / area.max(1) as f64
}
/// Up to [`REGION_SAMPLES`] pixels spread evenly through the run — coordinates that are
/// *provably* in it, to hand to the in-game ray pick (`WOW_PICK`). Reading them off a heatmap by
/// eye lands next to the run as often as in it, and then the pick names the wrong surface with
/// total confidence.
/// Up to [`REGION_SAMPLES`] pixels spread through the run, for the ray pick (`WOW_PICK`).
pub fn samples(&self) -> Vec<(u32, u32)> {
let step = (self.members.len() / REGION_SAMPLES).max(1);
self.members
@@ -329,14 +284,9 @@ impl Region {
.collect()
}
/// The run's own frame-to-frame time series — the reading that separates a surface being
/// **re-shaded** from one whose **edges are moving**.
///
/// [`shape`] says the toggling pixels form one coherent surface; it cannot say what the surface
/// is doing. If the whole run brightens and dims together (`agreement` near 1, `mean_delta`
/// swinging), it is being lit differently frame to frame — a shading defect. If half its pixels
/// go up while the other half go down (`agreement` near 0.5, `mean_delta` near 0), nothing is
/// being re-lit: a boundary is sweeping across it, and the run is an edge, not a surface.
/// The run's frame-to-frame series: the whole run brightening and dimming together (`agreement`
/// near 1) is a surface re-shaded; half up and half down (`agreement` near 0.5, `mean_delta`
/// near 0) is an edge sweeping across it.
pub fn steps(&self, frames: &[RgbImage]) -> Vec<Step> {
frames
.windows(2)
@@ -358,8 +308,7 @@ impl Region {
.collect();
let n = deltas.len().max(1) as f64;
let mean_delta = deltas.iter().sum::<f64>() / n;
// A pixel that did not move is counted as disagreeing: it is evidence against "the
// whole surface moved together", which is exactly the claim under test.
// A pixel that did not move counts against the surface moving together.
let same = deltas
.iter()
.filter(|d| **d != 0.0 && d.is_sign_positive() == mean_delta.is_sign_positive())
@@ -375,33 +324,26 @@ impl Region {
}
}
/// Rec. 601 luma — one number per pixel for "how bright", so a colour shift and a brightness shift
/// are not confused with each other.
/// Rec. 601 luma.
fn luma(p: &image::Rgb<u8>) -> f64 {
0.299 * f64::from(p[0]) + 0.587 * f64::from(p[1]) + 0.114 * f64::from(p[2])
}
/// The **spatial** structure of a toggle map — the reading that separates two defects the toggle
/// percentage alone cannot tell apart.
///
/// Both z-fighting and a visibility flip make pixels alternate, so both score the same on
/// [`Toggles::pct_toggling`]. They differ in *shape*: z-fighting resolves per fragment, so it lands
/// as moiré banding or speckle — many small runs, low neighbour agreement. A whole surface blinking
/// in and out (a culling, portal-visibility or draw-submission flip) lands as **one large solid run**
/// with the surface's own silhouette — few runs, high fill, high coherence. That distinction decides
/// which subsystem is at fault, so it is measured rather than eyeballed off a heatmap.
/// The spatial structure of a toggle map. Z-fighting resolves per fragment, landing as many small
/// runs of low coherence; a whole surface blinking (a culling, portal-visibility or draw-submission
/// flip) lands as one large solid run. Both score the same on [`Toggles::pct_toggling`].
#[derive(Debug, Clone, PartialEq)]
pub struct Shape {
/// Connected runs of at least [`MIN_REGION_PIXELS`], largest first.
pub regions: Vec<Region>,
/// Toggling pixels that fell in runs too small to list.
pub scattered: u64,
/// Mean fraction of the 4-neighbours of a toggling pixel that also toggle, 0..1. Near 1 = solid
/// areas; near 0 = isolated pixels. A one-pixel-wide moiré band sits near 0.5.
/// Mean fraction of a toggling pixel's 4-neighbours that also toggle, 0..1: solid areas near 1,
/// isolated pixels near 0, a one-pixel moiré band near 0.5.
pub coherence: f64,
}
/// A connected run smaller than this is noise, not a surface — counted in [`Shape::scattered`].
/// A run smaller than this is noise, counted in [`Shape::scattered`].
pub const MIN_REGION_PIXELS: u64 = 16;
/// Group a toggle mask into connected runs (4-connectivity) and measure their coherence.
@@ -418,8 +360,7 @@ pub fn shape(t: &Toggles) -> Shape {
continue;
}
toggling += 1;
// Off-frame neighbours count as non-toggling: an edge pixel is genuinely less enclosed,
// and pretending otherwise would inflate the coherence of anything touching the border.
// Off-frame neighbours count as non-toggling, so a run on the border is not inflated.
let n = u64::from(x > 0 && t.mask[idx(x - 1, y)])
+ u64::from(x + 1 < wu && t.mask[idx(x + 1, y)])
+ u64::from(y > 0 && t.mask[idx(x, y - 1)])
@@ -428,7 +369,7 @@ pub fn shape(t: &Toggles) -> Shape {
}
}
// Iterative flood fill — a recursive one blows the stack on a full-screen region.
// Iterative: a recursive flood fill overflows the stack on a full-screen region.
let mut seen = vec![false; wu * hu];
let mut regions: Vec<Region> = Vec::new();
let mut scattered = 0u64;
@@ -477,8 +418,7 @@ pub fn shape(t: &Toggles) -> Shape {
}
}
if pixels >= MIN_REGION_PIXELS {
// The flood fill visits in stack order, so sort to scan order — otherwise
// `Region::samples` spreads its picks through the traversal, not through the run.
// Scan order, so `Region::samples` spreads through the run, not the traversal.
members.sort_unstable_by_key(|&(x, y)| (y, x));
regions.push(Region {
bounds: b,
@@ -510,11 +450,8 @@ pub fn crop(img: &RgbImage, rect: Rect) -> RgbImage {
})
}
/// Magnify by an integer factor, nearest-neighbour — every output pixel is a source pixel verbatim.
/// This is the only scaling the `crop` subcommand offers on purpose: a resampler blends neighbours,
/// and a blended edge has already minted a false finding once (a `sips` downscale's "magenta
/// fringe" read as a render defect). Zoom may enlarge, never shrink: measurements happen at source
/// resolution or not at all.
/// Magnify by an integer factor, nearest-neighbour, so every output pixel is a source pixel: a
/// resampler's blended edge reads as a render defect. It never shrinks.
pub fn zoom(img: &RgbImage, factor: u32) -> RgbImage {
let s = factor.max(1);
RgbImage::from_fn(img.width() * s, img.height() * s, |x, y| {
@@ -522,8 +459,7 @@ pub fn zoom(img: &RgbImage, factor: u32) -> RgbImage {
})
}
/// Lay tiles out left-to-right, top-to-bottom in a `cols`-wide grid on a mid-grey mat, so a burst's
/// crops read as one contact sheet. Tiles of different sizes are placed at their own size.
/// Lay tiles out in a `cols`-wide grid on a dark grey mat, one contact sheet; each keeps its size.
pub fn contact_strip(tiles: &[RgbImage], cols: u32, gap: u32) -> RgbImage {
let cols = cols.max(1);
let (tw, th) = tiles
@@ -545,7 +481,6 @@ pub fn contact_strip(tiles: &[RgbImage], cols: u32, gap: u32) -> RgbImage {
out
}
/// The common size of a burst, erroring unless it has at least `min` equally-sized frames.
fn burst_dimensions(frames: &[RgbImage], min: usize) -> anyhow::Result<(u32, u32)> {
if frames.len() < min {
anyhow::bail!("needs at least {min} frames, got {}", frames.len());
@@ -563,9 +498,7 @@ fn burst_dimensions(frames: &[RgbImage], min: usize) -> anyhow::Result<(u32, u32
Ok((w, h))
}
/// Stitch two images side by side (`left | right`) with a `gap`-px dark separator, for at-a-glance A/B
/// comparison (e.g. the faithful vs modern render). Heights may differ; the output is the max height
/// with each image top-aligned.
/// Stitch two images side by side with a `gap`-px dark separator, top-aligned at the taller height.
pub fn compose_side_by_side(left: &RgbImage, right: &RgbImage, gap: u32) -> RgbImage {
let h = left.height().max(right.height());
let w = left.width() + gap + right.width();
@@ -596,9 +529,6 @@ mod tests {
assert_eq!(m.worst_at, (0, 0));
}
/// The single-pixel case this exists for: one silhouette pixel on the wrong side of an MSAA
/// tie. `mae` rounds to 0.000 and `pct_over` to 0.00% in a frame this size, so the *only*
/// signals that separate it from a real render change are the count and the coordinate.
#[test]
fn one_flipped_pixel_is_counted_and_located() {
let a = solid(100, 80, [40, 40, 40]);
@@ -613,8 +543,7 @@ mod tests {
#[test]
fn constant_offset_matches_offset() {
// Every channel of `b` is 10 below `a`, so mae == rmse == max_delta == 10, and (10 > 8) so
// every pixel is "over" → pct_over == 1.
// `b` is 10 below `a`: mae == rmse == max_delta == 10, and 10 > 8 puts every pixel over.
let a = solid(8, 5, [100, 100, 100]);
let b = solid(8, 5, [90, 90, 90]);
let m = compare(&a, &b).unwrap();
@@ -626,7 +555,6 @@ mod tests {
#[test]
fn small_offset_is_under_threshold() {
// A 5-byte shift is below OVER_THRESHOLD (8), so no pixel counts as changed even though mae>0.
let a = solid(8, 5, [100, 100, 100]);
let b = solid(8, 5, [95, 95, 95]);
let m = compare(&a, &b).unwrap();
@@ -650,8 +578,6 @@ mod tests {
#[test]
fn zoom_is_pixel_verbatim() {
// A 2x1 two-colour image zoomed ×3: every output pixel must be one of the two source
// values, block-aligned — any third colour would mean a resampler blended an edge.
let mut img = solid(2, 1, [10, 20, 30]);
img.put_pixel(1, 0, Rgb([200, 100, 50]));
let z = zoom(&img, 3);
@@ -687,8 +613,6 @@ mod tests {
#[test]
fn a_still_burst_has_no_envelope() {
// The point of max−min: a scene that never moves comes out black however bright it is, so a
// merely *wrong* render cannot masquerade as a flickering one.
let img = solid(4, 4, [200, 30, 90]);
let e = envelope(&[img.clone(), img.clone(), img], 8).unwrap();
assert_eq!(e.max_swing, 0);
@@ -699,9 +623,7 @@ mod tests {
#[test]
fn envelope_spans_the_whole_stack_not_just_neighbours() {
// 10 → 60 → 10: consecutive pairs each swing 50, and so does the envelope. But a value that
// drifts monotonically (10 → 35 → 60) swings only 25 per pair while the envelope still says
// 50 — which is the reason this is a stack operation and not a chain of pairwise diffs.
// 10, 60, 10 swings 50 per pair; 10, 35, 60 only 25 per pair, yet both envelopes are 50.
let base = solid(2, 1, [10, 10, 10]);
let alternating = {
let mut m = base.clone();
@@ -760,9 +682,6 @@ mod tests {
#[test]
fn a_monotone_sweep_never_toggles_however_far_it_moves() {
// The whole point: this is what a moving camera does to a pixel an edge sweeps across —
// a huge envelope (0 → 240) and zero reversals. `envelope` calls it maximally unstable;
// `toggles` correctly says it is not flickering.
let sweep = ramp(&[0, 60, 120, 180, 240]);
assert_eq!(envelope(&sweep, 1).unwrap().max_swing, 240);
let t = toggles(&sweep, 4, 60).unwrap();
@@ -772,7 +691,7 @@ mod tests {
#[test]
fn an_alternating_pixel_toggles_every_step() {
// A/B/A/B/A — z-fighting's signature. Four steps, three of them reversals.
// A/B/A/B/A, z-fighting's signature: four steps, three of them reversals.
let t = toggles(&ramp(&[10, 90, 10, 90, 10]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 3);
assert_eq!(t.pct_toggling, 1.0);
@@ -781,19 +700,16 @@ mod tests {
#[test]
fn sub_threshold_noise_neither_toggles_nor_forgets_the_direction() {
// ±1 dither must not manufacture reversals …
// ±1 dither makes no reversals,
let t = toggles(&ramp(&[50, 51, 50, 51, 50, 51]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 0);
// … and a run of it in the middle of a real climb must not reset the direction either,
// or every noisy edge would read as one reversal per crossing.
// nor does it reset the direction in the middle of a real climb.
let t = toggles(&ramp(&[0, 40, 41, 40, 41, 80]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 0);
}
#[test]
fn a_single_overshoot_is_one_reversal_and_below_the_toggle_bar() {
// Down-then-up once: real, but not a flicker — `TOGGLE_MIN_REVERSALS` is what draws that
// line, so a lone direction change must not be counted as a toggling pixel.
let t = toggles(&ramp(&[10, 90, 40]), 4, 60).unwrap();
assert_eq!(t.max_reversals, 1);
assert_eq!(t.pct_toggling, 0.0);
@@ -818,8 +734,7 @@ mod tests {
assert_eq!(out.get_pixel(4, 0), &Rgb([40, 50, 60])); // right image
}
/// A burst in which exactly the pixels selected by `flips` alternate 0/200 and everything else
/// holds still — the shape tests differ only in *which* pixels those are.
/// A burst where the pixels `flips` selects alternate 0/200 and the rest hold still.
fn alternating(
w: u32,
h: u32,
@@ -838,7 +753,6 @@ mod tests {
#[test]
fn one_solid_block_reads_as_a_single_filled_run() {
// A whole surface blinking: one run, filling its own box, every interior pixel enclosed.
let frames = alternating(20, 20, 6, |x, y| {
(4..12).contains(&x) && (4..12).contains(&y)
});
@@ -866,8 +780,6 @@ mod tests {
#[test]
fn a_checkerboard_covers_the_same_area_with_none_of_the_shape() {
// Z-fighting resolving per fragment. Same region of the frame, ~half the pixels toggling,
// but no pixel touches another — the reading that separates it from the block above.
let frames = alternating(20, 20, 6, |x, y| {
(4..12).contains(&x) && (4..12).contains(&y) && (x + y) % 2 == 0
});
@@ -879,11 +791,10 @@ mod tests {
#[test]
fn runs_come_back_largest_first_and_short_ones_are_scattered() {
// Two blocks plus a speck: the big one leads, the speck falls below MIN_REGION_PIXELS.
let frames = alternating(40, 20, 6, |x, y| {
let big = (2..12).contains(&x) && (2..12).contains(&y); // 100 px
let small = (20..25).contains(&x) && (2..7).contains(&y); // 25 px
let speck = (30..33).contains(&x) && y == 10; // 3 px — noise
let speck = (30..33).contains(&x) && y == 10; // 3 px, noise
big || small || speck
});
let s = shape(&toggles(&frames, 4, 60).unwrap());
@@ -900,7 +811,7 @@ mod tests {
let s = shape(&toggles(&frames, 4, 60).unwrap());
assert!(s.regions.is_empty());
assert_eq!(s.scattered, 0);
assert_eq!(s.coherence, 0.0); // no toggling pixels — not a divide-by-zero
assert_eq!(s.coherence, 0.0); // no toggling pixels, and no divide-by-zero
}
#[test]
crates/benilla-visual/src/main.rs
+52 -145
View File
@@ -1,19 +1,6 @@
//! `benilla-visual` — diff captures from the Phase-5 visual A/B harness (decision 0008).
//!
//! Usage:
//! benilla-visual diff <a.png> <b.png> [--out <diff.png>] [--fail <mae>] [--amplify <n>]
//! benilla-visual diff-dir <dir_a> <dir_b> [--out <diff_dir>] [--fail <mae>] [--amplify <n>]
//! benilla-visual flicker <burst_dir> [--out <envelope.png>] [--fail <mae>] [--amplify <n>]
//! benilla-visual stat <img.png> [--rect <x>,<y>,<w>,<h>]
//!
//! `diff` compares two images; `diff-dir` compares every `*.png` present in *both* directories by name.
//! Prints the metrics; writes amplified heatmap(s) when `--out` is given; exits non-zero if any image's
//! MAE exceeds `--fail` (when given). Typical loop: capture baselines, change the renderer, re-capture,
//! `diff-dir baseline candidate --out diff --fail 1.5`.
//!
//! `flicker` is the same arithmetic over *time*: point it at a `WOW_LIVE_SHOT_COUNT` burst (adjacent
//! frames, parked camera) and it prints the frame-to-frame table and the whole-stack envelope, and
//! writes the envelope heatmap — the picture of which pixels would not hold still.
//! `benilla-visual`: diff captures, and read flicker and motion out of a burst of frames; `--help`
//! lists the subcommands. The regression loop: capture baselines, change the renderer, re-capture,
//! then `diff-dir baseline candidate --out diff --fail 1.5`.
use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
@@ -24,32 +11,24 @@ use benilla_visual::{
Metrics, Rect, OVER_THRESHOLD,
};
/// A frame-to-frame step must move a channel by this much to count as a direction change (override
/// with `--toggle-delta`). Above PNG/dither noise, below any real surface swap.
///
/// **The pan rate matters more than this number.** A moving camera sweeps texture detail across
/// every pixel, and detail is not monotone — at 8°/s (≈15 px/frame at 3200 px) the map saturates at
/// 93% and says nothing. Keep the sweep **sub-pixel per frame** (≈0.2°/s) so ordinary surfaces
/// barely move while a depth-comparison flip still swaps whole surfaces at full amplitude.
/// The smallest step that counts as a direction change (`--toggle-delta`): above PNG and dither
/// noise, below a surface swap. The pan rate matters more: keep a sweep sub-pixel per frame (about
/// 0.2°/s), since at 8°/s texture detail saturates the map.
const TOGGLE_MIN_DELTA: u8 = 4;
/// Reversal counts are small integers, so the toggle map needs its own (much larger) gain than the
/// 0..255 swings the envelope amplifies: at 60, three reversals saturate.
/// The toggle map's gain: reversal counts are small, and at 60 three of them saturate.
const TOGGLE_AMPLIFY: u32 = 60;
/// Gap (px) between the two halves of a side-by-side compose.
const COMPOSE_GAP: u32 = 6;
/// Tiles per row on a `hotspot` contact sheet — wide enough that a 24-frame burst reads as a few
/// rows of adjacent frames rather than one unscannable ribbon.
/// Tiles per row on a `hotspot` contact sheet.
const STRIP_COLS: u32 = 6;
/// Pixels of context kept around a hotspot crop (override with `--pad`). Enough to see what the
/// toggling region borders — which is usually the whole point.
/// Pixels of context around a hotspot crop (`--pad`), enough to see what the run borders.
const DEFAULT_PAD: u32 = 24;
/// How many runs `hotspot` reports a time series for. Enough to see whether the top runs flip on the
/// same frames (one cause) or independently (several), narrow enough to stay one screen.
/// Runs `hotspot` prints a series for: enough to see whether the top runs flip together.
const SERIES_RUNS: usize = 4;
/// Default amplification for the heatmap output (per-channel abs-diff ×N, clamped).
@@ -147,8 +126,7 @@ fn main() -> Result<()> {
opts.out.as_deref(),
opts.amplify,
)?;
// Under `--fail` this is a gate, and a shot that never landed must sink it — see the
// note in `diff_dir` (decision 0743).
// Under `--fail` this is a gate, and a missing shot fails it.
if opts.fail.is_some() && !unpaired.is_empty() {
bail!(
"{} unpaired image(s) — a shot is missing from one side, so this comparison is \
@@ -240,17 +218,8 @@ fn main() -> Result<()> {
Ok(())
}
/// What one named pixel *did*, frame by frame — the colour counterpart to the client's `WOW_DEPTH`.
///
/// The aggregate readings (`flicker`, `hotspot`) answer "which pixels would not hold still" over a
/// whole region, which is the right question when you are looking for a defect and the wrong one once
/// you have found it. Correlating a renderer-side probe against the pixels needs the opposite: *this*
/// pixel, *these* frames, no averaging. Run `WOW_DEPTH` and `--at` on the same coordinates and the two
/// logs line up frame for frame — "the colour changed here but the depth did not" is a fact you cannot
/// get from a region mean, because a run mean over 19 000 pixels hides which of them moved.
///
/// It also guards the trap this exists inside: a burst that happens to contain **no** flip reads as a
/// clean negative. The per-frame Δ makes an absent phenomenon obvious instead of invisible.
/// One named pixel's colour frame by frame, to line up against the client's `WOW_DEPTH` log at the
/// same coordinates; the per-frame delta shows a burst that caught no flip at all.
fn series(dir: &Path, at: &str) -> Result<()> {
let pixels = parse_at(at)?;
let names = pngs(dir)?;
@@ -284,17 +253,8 @@ fn series(dir: &Path, at: &str) -> Result<()> {
Ok(())
}
/// Cut a window out of a capture, magnify it, and report what was actually cut — the vetted
/// replacement for the ad-hoc `sips`/`ffmpeg` crop pipelines that kept minting false findings
/// (`sips --cropOffset` takes (y, x) and silently ignores what it can't parse; a guessed ffmpeg
/// window read "no eyes" off a frame the model had left). Three honesty rules: a rect that misses
/// the frame entirely is an ERROR, never an edge-clamped guess; a rect that only partially fits is
/// clamped *out loud*; and every pixel sample prints in SOURCE coordinates at source resolution, so
/// nothing downstream does coordinate math on a zoomed image.
/// Per-channel min / mean / max over a rect (or the whole frame) — the "is this region flat?"
/// instrument. A frame edge that shows the render target's clear colour instead of art reads as
/// `min == max` on every channel (decision 1619, the glue framing's void check); a region of art
/// never does. Printed, not judged: the caller compares two rects or two captures.
/// Per-channel min, mean and max over a rect or the whole frame: an edge showing the clear colour
/// instead of art reads `min == max` on every channel, which a region of art never does.
fn stat_cmd(path: &Path, rect: Option<&str>) -> Result<()> {
let img = load(path)?;
let (w, h) = img.dimensions();
@@ -389,7 +349,7 @@ fn crop_cmd(path: &Path, rect: &str, at: Option<&str>, scale: u32, out: &Path) -
zoomed.width(),
zoomed.height(),
);
// Samples: the rect's centre always, plus any --at points — all in source coordinates.
// Samples: the rect's centre and any `--at` points, in source coordinates.
let mut samples = vec![((r.x0 + r.x1) / 2, (r.y0 + r.y1) / 2)];
if let Some(at) = at {
samples.extend(parse_at(at)?);
@@ -414,8 +374,7 @@ fn crop_cmd(path: &Path, rect: &str, at: Option<&str>, scale: u32, out: &Path) -
Ok(())
}
/// `"x,y,w,h"` → the rect it names. Strict for the same reason as [`parse_at`]: a half-parsed
/// window silently crops the wrong place, and a wrong crop reads as a finding.
/// `"x,y,w,h"` → the rect it names; strict, since a half-parsed window crops the wrong place.
fn parse_rect(spec: &str) -> Result<Rect> {
let parts: Vec<&str> = spec.split(',').map(str::trim).collect();
let [x, y, w, h] = parts.as_slice() else {
@@ -437,7 +396,7 @@ fn parse_rect(spec: &str) -> Result<Rect> {
})
}
/// `"x,y;x,y"` → pixels. Strict: a typo here would silently report the wrong pixel's history.
/// `"x,y;x,y"` → pixels; strict, since a typo would report the wrong pixel's history.
fn parse_at(spec: &str) -> Result<Vec<(u32, u32)>> {
spec.split(';')
.filter(|s| !s.trim().is_empty())
@@ -455,27 +414,11 @@ fn parse_at(spec: &str) -> Result<Vec<(u32, u32)>> {
.collect()
}
/// Stitch every `*.png` present in both dirs into `left | right` side-by-side images under `out`.
/// **The sub-pixel motion ruler** (`flow <burst_dir> [--rect "<x>,<y>,<w>,<h>"]`).
///
/// `flicker` and `series` answer *how much* a pixel changed. Neither can answer the question a
/// "the animation ticks" report actually asks — is the thing in front of the pixel advancing
/// **evenly**? — and the obvious way to ask it is a trap: recover one silhouette edge's sub-pixel
/// position frame by frame, and the ruler is built out of 8-bit pixels, so it carries its own
/// 1/255 quantisation and a 0.2 px step cannot be separated from the tool's noise floor. That
/// ambiguity is exactly what left the first staircase reading unattributed.
///
/// So estimate the whole window's displacement at once, by least squares over every pixel in it
/// (Lucas–Kanade, the standard first-order flow solve): thousands of gradients vote on one
/// `(dx, dy)`, the 8-bit floor averages down by `√N`, and the estimate lands two orders of
/// magnitude under a pixel. A breathing body is not a rigid translation and does not need to be —
/// what is read off the series is whether the aggregate advances smoothly, and a staircase in the
/// RENDER shows up as a staircase here whatever the body underneath is doing.
///
/// Columns are the per-frame displacement, its magnitude, and the **second difference** — the
/// curvature-per-frame discriminator the world-space jitter meter uses, on this side of the glass.
/// The summary counts **stalled** frames (under a fifth of the median step): a smooth pan has
/// none, a staircase is mostly them.
/// The sub-pixel motion ruler: one least-squares (Lucas-Kanade) displacement per frame pair over
/// the whole window, so thousands of gradients average the 8-bit floor away and a render that
/// advances in steps reads as steps. Columns are the displacement, its magnitude and the second
/// difference; a stalled frame moves under a fifth of the median step. It assumes constant
/// brightness: for a shaded, deforming body use `edge`.
fn flow(dir: &Path, rect: Option<&str>) -> Result<()> {
let names = pngs(dir)?;
if names.len() < 3 {
@@ -487,8 +430,7 @@ fn flow(dir: &Path, rect: Option<&str>) -> Result<()> {
}
let first = load(&dir.join(names.iter().next().expect("non-empty")))?;
let (w, h) = first.dimensions();
// The default window is the whole frame minus the one-pixel border the central differences
// need. A named rect is clamped out loud, never silently (the `crop` honesty rule).
// A named rect is clamped out loud, never silently; the solve skips the one-pixel border.
let win = match rect {
Some(spec) => {
let want = parse_rect(spec)?;
@@ -581,8 +523,7 @@ fn flow(dir: &Path, rect: Option<&str>) -> Result<()> {
Ok(())
}
/// One frame's luma over `win`, row-major — the flow solve's input (f32 is exact for 8-bit sums
/// and halves the working set against a 3200x1800 f64 buffer).
/// One frame's luma over `win`, row-major; `f32` halves the working set of a 3200x1800 frame.
fn luma_window(img: &image::RgbImage, win: &Rect) -> Vec<f32> {
let mut out = Vec::with_capacity(((win.x1 - win.x0) * (win.y1 - win.y0)) as usize);
for y in win.y0..win.y1 {
@@ -596,10 +537,8 @@ fn luma_window(img: &image::RgbImage, win: &Rect) -> Vec<f32> {
out
}
/// Least-squares first-order optical flow between two luma windows: the `(dx, dy)` that best
/// explains `b` as `a` shifted, solved once over every interior pixel. `None` when the window
/// carries no usable gradient structure (a flat sky — the normal matrix is singular and any
/// answer would be invented).
/// The `(dx, dy)` that best explains `b` as `a` shifted, over every interior pixel; `None` when
/// the normal matrix is singular, as over a flat sky.
fn lucas_kanade(a: &[f32], b: &[f32], w: usize, h: usize) -> Option<(f64, f64)> {
let (mut sxx, mut sxy, mut syy, mut sxt, mut syt) = (0.0f64, 0.0, 0.0, 0.0, 0.0);
for y in 1..h - 1 {
@@ -616,8 +555,7 @@ fn lucas_kanade(a: &[f32], b: &[f32], w: usize, h: usize) -> Option<(f64, f64)>
}
}
let det = sxx * syy - sxy * sxy;
// Scale-relative: the normal matrix grows with the window, so an absolute epsilon would call
// a big flat window solvable and a small textured one singular.
// Relative, since the normal matrix grows with the window.
if det <= 1e-12 * (sxx * syy).max(f64::MIN_POSITIVE) {
return None;
}
@@ -646,11 +584,10 @@ fn compose_dir(da: &Path, db: &Path, out: &Path) -> Result<()> {
Ok(())
}
/// Report how far a burst of adjacent frames moved: the frame-to-frame table, then the whole-stack
/// envelope. Returns the worst consecutive-pair metrics (the `--fail` subject) — the envelope is a
/// *stack* number and can exceed any single pair, so failing on the pair is the conservative gate.
/// Print a burst's frame-to-frame table, envelope and toggles; the worst pair's metrics are the
/// `--fail` subject, since the envelope can exceed any single pair.
fn flicker(dir: &Path, out: Option<&Path>, amplify: u32, toggle_delta: u8) -> Result<Metrics> {
let names: Vec<String> = pngs(dir)?.into_iter().collect(); // BTreeSet — already shot order
let names: Vec<String> = pngs(dir)?.into_iter().collect(); // a BTreeSet: already shot order
if names.len() < 2 {
bail!(
"{} holds {} *.png — a flicker burst needs at least 2 (WOW_LIVE_SHOT_COUNT=<n>)",
@@ -691,8 +628,7 @@ fn flicker(dir: &Path, out: Option<&Path>, amplify: u32, toggle_delta: u8) -> Re
e.mean_swing,
e.pct_unstable * 100.0
);
// The moving-camera reading. Printed always, because which of the two numbers is meaningful
// depends on whether the camera was parked — and a burst does not carry that fact.
// Printed always: a burst does not record whether the camera was parked.
let t = toggles(&frames, toggle_delta, TOGGLE_AMPLIFY)?;
println!(
"toggles of {} frames (delta {toggle_delta}): max reversals {:>3} toggling pixels {:>6.2}% \
@@ -718,12 +654,8 @@ fn flicker(dir: &Path, out: Option<&Path>, amplify: u32, toggle_delta: u8) -> Re
Ok(worst)
}
/// Follow the toggle map to the thing it found: measure the toggling pixels' **shape**, then crop
/// the largest run out of every frame into one contact sheet.
///
/// The map says *where*; this says *what kind*, and shows the frames that prove it. A run that fills
/// its box and holds together is one surface blinking (visibility/culling); many thin runs are
/// z-fighting resolving per fragment. Both read identically on the toggle percentage.
/// Measure the toggling pixels' shape, print the top runs' series and relight fits, and crop the
/// largest run out of every frame into one contact sheet.
fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
let names: Vec<String> = pngs(dir)?.into_iter().collect();
let frames: Vec<image::RgbImage> = names
@@ -751,8 +683,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
r.bounds.x0,
r.bounds.y0,
);
// Pasteable straight into the in-game ray pick, which is the next question every time:
// WOW_PICK="<this>" names the surfaces at these exact pixels, front to back.
// For the ray pick: `WOW_PICK` names the surfaces at these pixels, front to back.
let pick = r
.samples()
.iter()
@@ -767,9 +698,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
benilla_visual::MIN_REGION_PIXELS
);
};
// What the top runs are actually *doing*, frame by frame: re-shaded together, or edges moving.
// Side by side, because whether they flip on the SAME steps is the difference between one global
// cause and several local ones — and that is invisible in any single run's series.
// The top runs side by side: flipping on the same steps means one cause, not several.
let series: Vec<Vec<benilla_visual::Step>> = s
.regions
.iter()
@@ -789,11 +718,8 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
.collect();
println!(" {:>2} -> {:<2} {}", i, i + 1, cols.join(" "));
}
// The two extremes of run #0, per channel — the *levels*, which is all this line is. Read the
// ratios as description, never as a diagnosis: equal ratios rule out a scalar multiply and
// nothing else, since an ADDED light moves the three channels by different factors too. The
// reading that actually separates "one surface, re-lit" from "two surfaces" is the per-pixel
// affine fit printed below it (`benilla_visual::relight`).
// Run #0's extremes per channel, a description only: equal ratios rule out a scalar multiply
// and nothing else. The relight fit below is the diagnosis.
let s0 = &series[0];
let lo = s0
.iter()
@@ -812,10 +738,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
hi.mean_rgb_from[0], hi.mean_rgb_from[1], hi.mean_rgb_from[2],
ratio.join(" / "),
);
// Same surface re-lit, or a different surface? Fit each run's pixels across its own biggest
// frame-to-frame step. High R² = the pattern survived the flip and only its gain/offset moved,
// so it is one surface being shaded differently; low R² = the two frames show different
// surfaces, whatever the means do.
// Each run fitted across its biggest step: high R² is one surface re-lit, low R² two surfaces.
println!(
" across each run's biggest step — is it one surface re-lit? (R² near 1 = yes), \
against its quietest step as the control:"
@@ -837,8 +760,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
)
})
.collect();
// The control is what licenses the verdict: a low R² on the flip only means "different
// surfaces" if the same pixels under the same pan fit tightly when they did NOT flip.
// A low R² on the flip means two surfaces only if the quiet step fits tightly.
let verdict = if ctl.worst_r2() < 0.9 {
"no control — the pan alone decorrelates these pixels; fit says nothing"
} else if fit.worst_r2() > 0.9 {
@@ -860,8 +782,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
);
}
let rect = biggest.bounds.padded(pad, w, h);
// The toggle map goes in as the first tile, so the sheet carries its own legend: this is the
// region, and here is what the frames did inside it.
// The toggle map is the first tile, the sheet's own legend.
let mut tiles = vec![crop(&t.image, rect)];
tiles.extend(frames.iter().map(|f| crop(f, rect)));
if let Some(parent) = out.parent().filter(|p| !p.as_os_str().is_empty()) {
@@ -882,9 +803,7 @@ fn hotspot(dir: &Path, out: &Path, toggle_delta: u8, pad: u32) -> Result<()> {
Ok(())
}
/// `foo.png` + `-toggle` → `foo-toggle.png`. The two maps answer the same question under opposite
/// camera conditions, so they are written as a pair rather than behind a second flag nobody would
/// remember to pass.
/// `foo.png` + `-toggle` → `foo-toggle.png`.
fn sibling(out: &Path, suffix: &str) -> PathBuf {
let ext = out.extension().and_then(|e| e.to_str()).unwrap_or("png");
let stem = out.file_stem().and_then(|s| s.to_str()).unwrap_or("out");
@@ -915,15 +834,9 @@ fn over_fail(m: &Metrics, fail: Option<f64>) -> bool {
fail.is_some_and(|t| m.mae > t)
}
/// Load an image as RGB (dropping any alpha — capture windows are opaque).
/// One silhouette's sub-pixel position along a scanline, per frame — the shading-independent
/// answer to "did the rendered geometry MOVE smoothly?".
///
/// Per frame and per scanline: take the row's own darkest and brightest samples, put the
/// threshold halfway between them, and linearly interpolate the crossing. Renormalising to the
/// row's own ends every frame is the whole point — a uniform brightness change (the body turning
/// into the light) moves `lo` and `hi` together and leaves the crossing where it was, while a
/// geometric shift moves it. That is the property [`flow`] lacks.
/// One silhouette's sub-pixel position along a scanline, per frame: where the row crosses the
/// midpoint of its own darkest and brightest samples, so a uniform brightness change leaves it in
/// place and only a geometric shift moves it.
fn edge(dir: &Path, at: &str) -> Result<()> {
let names = pngs(dir)?;
if names.len() < 3 {
@@ -972,7 +885,7 @@ fn edge(dir: &Path, at: &str) -> Result<()> {
.collect();
let lo = row.iter().copied().fold(f64::INFINITY, f64::min);
let hi = row.iter().copied().fold(f64::NEG_INFINITY, f64::max);
// Too little contrast to call an edge — report it rather than invent a position.
// Too little contrast to call an edge: report it, never invent a position.
if hi - lo < 12.0 {
pos.push(f64::NAN);
continue;
@@ -1051,11 +964,10 @@ fn diff_one(a: &Path, b: &Path, out: Option<&Path>, amplify: u32) -> Result<Metr
Ok(m)
}
/// Compare every `*.png` present in both dirs (by file name). Returns the worst (highest-MAE) result.
/// What one `diff-dir` run found: the worst pair by MAE, and every image present on only ONE side.
/// What one `diff-dir` run found: the worst pair by MAE, and every image on only one side.
struct DirDiff {
worst: Option<(String, Metrics)>,
/// Files with no counterpart, so not diffed at all — see the note in [`diff_dir`].
/// Files with no counterpart, so not diffed.
unpaired: Vec<String>,
}
@@ -1069,11 +981,8 @@ fn diff_dir(da: &Path, db: &Path, out: Option<&Path>, amplify: u32) -> Result<Di
db.display()
);
}
// UNPAIRED shots are ANNOUNCED, never skipped in silence. Pairing on the intersection alone is
// how a sweep reports all-green with a shot missing: on 2026-07-28 one `water-night` capture
// exited 0 without writing its PNG, and `selfcheck` diffed the other eight and passed clean. A
// gate that quietly narrows its own scope is worse than no gate. Reported always; fatal to the
// CALLER when `--fail` is in force, i.e. whenever this is being used as a gate (decision 0743).
// Unpaired shots are announced, never skipped in silence, so a capture that wrote no PNG
// cannot let the rest pass clean; the caller fails on them under `--fail`.
let unpaired: Vec<String> = a
.difference(&b)
.map(|n| format!("{}/{n}", da.display()))
@@ -1117,9 +1026,8 @@ fn pngs(dir: &Path) -> Result<BTreeSet<String>> {
}
fn fmt_metrics(m: &Metrics) -> String {
// `px` and the worst pixel's coordinate are what tell a render change from an MSAA tie: both
// show `MAE 0.000` with an alarming `max`, and only the count and the location separate them
// (a handful of pixels, at the same coordinate every build, is a tie).
// `px` and the worst pixel's coordinate tell an MSAA tie from a render change: a handful of
// pixels at the same spot every build is a tie.
let where_ = if m.changed == 0 {
String::new()
} else {
@@ -1187,7 +1095,6 @@ mod tests {
#[test]
fn rect_spec_is_strict() {
// "x,y,w,h" and nothing else — a half-parsed window crops the wrong place silently.
let r = parse_rect("10, 20, 30, 40").unwrap();
assert_eq!((r.x0, r.y0, r.x1, r.y1), (10, 20, 40, 60));
assert!(parse_rect("10,20,30").is_err());
crates/benilla-visual/src/relight.rs
+20 -56
View File
@@ -1,26 +1,11 @@
//! **Is it the same surface lit differently, or a different surface?** — the reading that separates
//! the two explanations a two-state flip always admits.
//! Tells one surface lit differently from two different surfaces. Equal channel ratios rule out
//! only a scalar multiply, since an added coloured light changes the ratios too; the spatial
//! pattern tells them apart. Re-lighting is an affine map per pixel, `bright ≈ gain·dim + offset`,
//! so the fit is tight, while two surfaces fit loosely however well their means line up. This fits
//! one frame's pixels onto the other's per channel and reports R².
//!
//! [`Region::steps`] measures that a run alternates between two levels and that its pixels move
//! together. That is where the evidence used to stop, and the gap mattered: B38's tent was read off
//! the per-channel *ratios* as "two materials trading places", on the argument that one light term
//! switching would scale all three channels equally. **That argument is wrong.** Equal ratios rule
//! out a scalar multiply and nothing else — an *additive* term (a coloured light arriving or
//! leaving) changes the ratios freely. Measured there: dim `(77, 42, 32)` → bright `(192, 79, 35)`
//! is "×2.50 / ×1.89 / ×1.12", which reads as two materials, and is *also* exactly the dim state
//! plus a warm `(115, 37, 3)`. The ratios cannot tell those apart, so they were never evidence.
//!
//! What does tell them apart is the **spatial pattern**, which the means throw away. Re-lighting a
//! surface is an affine map on its pixels — every pixel keeps its place in the pattern, so
//! `bright ≈ gain·dim + offset` holds *per pixel* and the fit is tight. Two different surfaces have
//! unrelated patterns (canvas weave vs plank grain), and no affine map relates them: the fit is
//! loose however well the means happen to line up. So: least-squares fit one frame's pixels onto the
//! other's, per channel, and report R². That is a number, not an inference from a number.
//!
//! Fit across the run's **largest single frame-to-frame step**, never across the whole burst: the
//! camera pans during a capture, and a pixel only names the same bit of world for as long as the
//! image holds still under it. Adjacent frames at the sub-pixel-per-frame pan the toggle map needs
//! (decision 0656) are the same view; frames twenty apart are not.
//! The fit spans the run's largest single frame-to-frame step, never the whole burst: the camera
//! pans, and a pixel names the same bit of world only while the image holds still under it.
use image::RgbImage;
@@ -29,26 +14,20 @@ use crate::Region;
/// A per-channel affine fit of one frame's pixels onto another's, over a [`Region`]'s own pixels.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Relight {
/// Per channel, the `a` in `to ≈ a·from + b`. Near 1 with a non-zero [`Relight::offset`] is a
/// light being *added*; well above 1 with an offset near 0 is one being *scaled*.
/// Per channel, the `a` in `to ≈ a·from + b`: near 1 with an offset is a light added, well
/// above 1 with none a light scaled.
pub gain: [f64; 3],
/// Per channel, the `b` in `to ≈ a·from + b`, in 0..255 units.
pub offset: [f64; 3],
/// Per channel coefficient of determination, 0..1 — how much of the target frame's variation
/// across the run the fit explains. **This is the reading.** Near 1: one surface, re-lit (the
/// pattern survived, only its scale and offset moved). Low: the two frames show *different*
/// surfaces, and no amount of re-lighting maps one to the other.
/// Per channel R², 0..1: near 1 is one surface re-lit, low is two different surfaces.
pub r2: [f64; 3],
/// Did this channel vary across the run at all? A flat channel (uniform, or clipped to 0/255)
/// has nothing for a fit to explain, so its R² is 0 for a reason that is **not** evidence of
/// different surfaces — and without this flag that 0 would outvote the channels that do carry
/// evidence, turning a plain re-light into a false "different surfaces".
/// Whether the channel varied across the run: a flat or clipped channel's R² of 0 is no
/// evidence, and must not outvote the others.
pub determinate: [bool; 3],
}
impl Relight {
/// The weakest **determinate** channel's R² — the honest summary, since "one surface, re-lit"
/// has to hold in every channel that carries evidence. 0 when no channel does.
/// The weakest determinate channel's R², or 0 when no channel is determinate.
pub fn worst_r2(&self) -> f64 {
(0..3)
.filter(|&c| self.determinate[c])
@@ -58,11 +37,8 @@ impl Relight {
}
}
/// Least-squares fit of `to`'s pixels onto `from`'s, per channel, over `region`'s own pixels.
///
/// A channel with no variation across the run (a flat, saturated, or clipped channel) has nothing
/// for a fit to explain; its R² is reported as 0 rather than dividing by zero, which reads as
/// "this channel is not evidence" — the conservative direction.
/// Least-squares fit of `to`'s pixels onto `from`'s per channel, over `region`'s own pixels; a flat
/// channel reports R² 0 and is not determinate.
pub fn relight(region: &Region, from: &RgbImage, to: &RgbImage) -> Relight {
let mut out = Relight {
gain: [0.0; 3],
@@ -106,20 +82,13 @@ pub fn relight(region: &Region, from: &RgbImage, to: &RgbImage) -> Relight {
out
}
/// The index of the frame-to-frame step where the run's mean luma moved most — the flip itself,
/// and the pair [`relight`] should be fitted across. `None` for a burst with fewer than two frames.
/// The step where the run's mean luma moved most: the flip, and the pair [`relight`] fits across.
pub fn biggest_step(region: &Region, frames: &[RgbImage]) -> Option<usize> {
extreme_step(region, frames, true)
}
/// The step where the run's mean luma moved **least** — the **control**, and the reason a low R² on
/// the biggest step can be believed at all.
///
/// The capture pans, so consecutive frames are never quite the same image, and a textured surface
/// sliding a fraction of a pixel decorrelates on its own. That means a low R² is only evidence of
/// *different surfaces* if a step where the run did **not** flip scores high across the same pixels
/// under the same motion. Quiet step near 1 and flip step near 0 is a real reading; both low means
/// the fit is measuring the pan and nothing else.
/// The step where the run's mean luma moved least: the control. The pan alone decorrelates a
/// textured surface, so a low R² on the flip means two surfaces only if this step fits tightly.
pub fn quietest_step(region: &Region, frames: &[RgbImage]) -> Option<usize> {
extreme_step(region, frames, false)
}
@@ -161,7 +130,7 @@ mod tests {
}
}
/// A textured surface: a pattern with real variation for a fit to have to explain.
/// A patterned surface, with variation for a fit to explain.
fn textured(w: u32, h: u32, f: impl Fn(u32, u32) -> [u8; 3]) -> RgbImage {
RgbImage::from_fn(w, h, |x, y| Rgb(f(x, y)))
}
@@ -169,7 +138,7 @@ mod tests {
#[test]
fn the_same_surface_plus_a_warm_light_fits_as_gain_one_and_an_offset() {
let dim = textured(16, 16, |x, y| [(x * 8) as u8, (y * 6) as u8, (x + y) as u8]);
// Exactly the B38 shape: add a warm constant, leave the pattern alone.
// A warm constant added, the pattern untouched.
let bright = textured(16, 16, |x, y| {
[(x * 8 + 115) as u8, (y * 6 + 37) as u8, (x + y + 3) as u8]
});
@@ -200,8 +169,6 @@ mod tests {
assert!(r.offset[0].abs() < 1e-6, "{r:?}");
}
/// The discrimination the module exists for: two surfaces whose *means* differ exactly as a
/// re-light would, but whose patterns are unrelated. The means cannot tell them apart; R² can.
#[test]
fn two_different_surfaces_do_not_fit_however_well_their_means_line_up() {
// Vertical stripes vs horizontal stripes: same mean, same spread, no affine relation.
@@ -221,9 +188,6 @@ mod tests {
assert!((r.offset[0] - 90.0).abs() < 1e-9, "{r:?}");
}
/// The flat-channel rule has to be a *skip*, not a zero vote: a surface whose blue is clipped
/// flat but whose red and green track perfectly is one surface being re-lit, and reporting the
/// dead channel's 0 as the verdict would call it two.
#[test]
fn a_flat_channel_does_not_outvote_the_ones_carrying_evidence() {
let dim = textured(16, 16, |x, y| [(x * 8) as u8, (y * 6) as u8, 255]);