app+formats: take the realm's own FrameXML list, not a curated copy of it

`benilla.toc` enumerated every interface file, so a row the realm adds
in a client update was *silently* absent: no "not found", no warning,
just a global nobody defined. That is what `/xpboost` was, and the only
way to find it was to diff two MPQ archives by hand.

The manifest now decides order and substitution, and the realm's own
`FrameXML.toc` decides membership: every row it carries that the
manifest does not name loads after it, in the chain's order, and the
boot log names them. `DECLINED` is the short list of rows this client
must not run (`StatsFrame.xml`, whose OnUpdate is the only reader of a
`GetDebugStats` we do not have), each with its reason; a row we merely
have no verb for still loads and raises where it is used, which beats
silence. `WOW_FRAMEXML_TAIL=0` opts out and says what it is leaving out.

Finding that out turned up the reason the row was missing in the first
place, which is worse and is now fixed too. `Chain::open` mounts
`patch-?.MPQ` in the reference's order — case-folded, last wins — so
`patch-A.mpq` mounts **above** `patch-5.mpq`. The realm ships its
interface in `patch-5`; this update added six rows to that toc; and the
140-row copy inside an older `patch-A.mpq` kept winning. The interface
was loading August's list of files.

So `Chain::read_all` returns every archive's copy of a name in
ascending priority, and `realm_toc_rows` reads a shadowed copy when it
holds every row the winner does *plus* more. That case is unambiguous —
nothing was removed, so nothing can be lost — and it is a packaging
accident with no intent behind it either way. A copy that disagrees by
removal is a real change of mind we cannot arbitrate: the winner stands
and the log says what was passed over. Both paths say which archive, so
the clean fix on disk is one line of log away.

Verified against grom's own install and his addon corpus: with the
`OctoXpBoost` row deleted from the manifest — tomorrow's update,
simulated — the tail finds it, resolves it to a readable chain path, and
`/xpboost` is a live slash command with a function behind it. The tail
is empty with the manifest as it stands, which is the point: the
mechanism is armed and changes nothing today.

Tests: the merge takes both lists as arguments so it runs where there is
no client data, since that is the machine where a regression in it would
otherwise skip unseen; the letter-above-digit mount order is pinned on
its own, because that fact is what cost the six rows; `read_all`'s
ordering contract is checked against a real chain; and a `DECLINED` row
that the realm stops shipping now fails as stale. The 269 data-gated
failures are the same 269, name for name, as at HEAD.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SgstFWtn1RxEJ5kVrpgYwn
This commit is contained in:
2026-10-23 16:27:03 -06:00
co-authored by Claude Opus 5
parent cf0e56a4cf
commit eaadc9f73b
3 changed files with 360 additions and 9 deletions
+20 -9
View File
@@ -1,6 +1,10 @@
//! The in-game interface: its manifest, [`MANIFEST`], and the boot split that loads it in two
//! phases. An entry with a path is the reference's own file off the player's patch chain, a bare
//! filename is one we ship, and the manifest's order is the load order across both.
//!
//! [`interface_files`], not the manifest alone, is that list: the realm's own `FrameXML.toc` has
//! the last word on *membership*, so a row it gains in a client update loads without anyone
//! editing this crate (`reference_ui::tail`).
use bevy::prelude::*;
@@ -18,6 +22,19 @@ pub(super) fn manifest_files() -> Vec<String> {
Addon::builtin().toc.files
}
/// The whole interface in load order: [`MANIFEST`], then every row the realm's own `FrameXML.toc`
/// carries that the manifest does not name (`reference_ui::tail`).
///
/// The tail goes last for two reasons: the manifest's order deviates from the chain's deliberately
/// in places, so there is no unambiguous position to interleave an unknown row into; and the
/// chain's toc puts the realm's own additions late itself. Their order *relative to each other* is
/// the chain's, which is what a pair like `Turtle_GroupUI.xml` and the fix that patches it needs.
fn interface_files() -> Vec<String> {
let mut files = Addon::builtin().toc.files;
files.extend(reference_ui::tail().iter().cloned());
files
}
/// The manifest's entries that name a file we ship rather than one off the player's install.
#[cfg(test)]
pub(super) fn shipped_manifest_files() -> Vec<String> {
@@ -255,7 +272,7 @@ pub(crate) fn load_default_ui(script: &UiScript) -> Vec<String> {
sex: 2,
});
let mut failures = silenced_ui_load(&mut &*script, |script| {
let mut failures = load_manifest(script, &Addon::builtin().toc.files);
let mut failures = load_manifest(script, &interface_files());
failures.extend(bootstrap_positions(script));
failures
});
@@ -298,10 +315,7 @@ fn load_manifest(script: &UiScript, files: &[String]) -> Vec<String> {
/// atlas; the reference's glue has its own `GlueFonts.xml`. The font objects declared outside it
/// (FRIZQT 13, 14 and 20, un-outlined) repeat combinations it has, so the atlas plan loses nothing.
pub(crate) fn load_font_registry(script: &UiScript) -> Vec<String> {
load_manifest(
script,
Addon::builtin().toc.files.get(..1).unwrap_or_default(),
)
load_manifest(script, interface_files().get(..1).unwrap_or_default())
}
/// The in-game UI, every entry after `Fonts.xml`, then every third-party addon, on entering the
@@ -321,10 +335,7 @@ pub(crate) fn load_ingame_ui(
// Bounded, addons included: a chunk that never returns fails as a load error instead of
// freezing the loading screen. The caller disarms the budget once the edge is done.
script.set_instruction_budget(super::addons::LOAD_INSTRUCTION_BUDGET);
let mut failures = load_manifest(
script,
Addon::builtin().toc.files.get(1..).unwrap_or_default(),
);
let mut failures = load_manifest(script, interface_files().get(1..).unwrap_or_default());
failures.extend(bootstrap_positions(script));
// Each addon's `ADDON_LOADED` fires as that addon finishes, as the reference does (`0x51f5ad`).
failures.extend(super::addons::load_third_party(
@@ -5,6 +5,14 @@
//! `assets/ui/benilla.toc` is the one load order. An entry with a path separator comes off the
//! chain, a bare filename is a file we ship ([`is_chain_entry`]), and a name defined by both goes
//! to the later line. Without client data the chain files are absent, and the log says so once.
//!
//! That manifest is not the whole list, though, because it cannot be: the realm adds rows to its
//! own `FrameXML.toc` whenever it ships a feature, and a curated enumeration drops each one
//! *silently* — no "not found", no warning, just a global nobody defined, surfacing days later as
//! an addon error or a slash command that does nothing ([`unnamed_chain_rows`] says what that cost
//! once). So the manifest decides order and substitution, and the chain's own toc decides
//! membership: every row the realm carries that the manifest does not name loads after it
//! ([`tail`]), in the chain's order, except the few [`DECLINED`] names this client must not run.
use std::sync::OnceLock;
@@ -25,6 +33,169 @@ pub(super) fn is_chain_entry(entry: &str) -> bool {
entry.contains('\\') || entry.contains('/')
}
/// The realm's own interface manifest, and the prefix its rows are relative to.
const CHAIN_TOC: &str = "Interface\\FrameXML\\FrameXML.toc";
const FRAMEXML_PREFIX: &str = "Interface\\FrameXML\\";
/// Chain rows this client declines to load, each with the reason, because [`tail`] would
/// otherwise pick them up. A row belongs here only where loading it is *worse* than leaving the
/// feature out; a row we merely have no verb for still loads, raises where it is used, and says so
/// in the log, which is the outcome we want over silence.
const DECLINED: &[(&str, &str)] = &[(
"StatsFrame.xml",
"the debug-build frame-stat overlay, whose OnUpdate is the only reader of `GetDebugStats()` — \
a binding 5875 has and this client does not. Loading it publishes a `ToggleStats` that raises \
twice a second once pressed, where leaving it out leaves `ToggleStats` nil, which is what an \
unbound key already does. Build `GetDebugStats` and the row costs nothing",
)];
/// The rows of `chain_toc` that `manifest` does not name and [`DECLINED`] does not refuse, in the
/// chain's own order: what a client update added to the realm's interface.
///
/// Split out from [`tail`] and taking both lists so the merge is testable without client data,
/// which matters because the machine this is usually run on has none and would skip the test that
/// covers it. Comparison is case-insensitive and separator-insensitive: the realm writes
/// `OctoXpBoost\OctoXpBoost.lua` where the manifest writes a full `Interface\FrameXML\…` path,
/// and neither spelling is stable across rows.
fn unnamed_chain_rows(chain_toc: &[String], manifest: &[String]) -> Vec<String> {
let key = |s: &str| s.replace('/', "\\").to_ascii_lowercase();
let named: std::collections::HashSet<String> = manifest.iter().map(|m| key(m)).collect();
let declined: std::collections::HashSet<String> =
DECLINED.iter().map(|(n, _)| key(n)).collect();
chain_toc
.iter()
.filter(|row| {
let row = key(row);
let full = key(&format!("{FRAMEXML_PREFIX}{row}"));
!declined.contains(&row) && !named.contains(&row) && !named.contains(&full)
})
.map(|row| format!("{FRAMEXML_PREFIX}{}", row.replace('/', "\\")))
.collect()
}
/// The realm's interface manifest as its rows, or `None` without client data.
///
/// Deviation, and the only interesting thing in this file: where a **shadowed** copy of
/// `FrameXML.toc` strictly extends the winning one, the shadowed copy is read instead.
///
/// The chain's mount order is the reference's (`Chain::open`), and it sorts `patch-?.MPQ` by
/// case-folded name, so `patch-A.mpq` mounts *above* `patch-5.mpq` and its copy of any file wins.
/// On grom's install that is exactly what happened: the realm ships its interface in `patch-5`, a
/// client update on 2026-10-23 added six rows to that toc, and the 140-row copy inside an older
/// `patch-A.mpq` kept winning — so the interface loaded August's list of files and `/xpboost`
/// simply did not exist, with nothing in any log to say why.
///
/// Honouring the winner there is faithful to a packaging accident and to nothing else: nobody
/// chose for an old copy of the realm's own manifest to mount high, and the cost is a feature
/// missing in silence. So a copy that contains *every* row the winner has, plus more, is taken as
/// the later edition of the same list, which it is — nothing was removed, so nothing can be lost
/// by reading it. A copy that disagrees by **removal** is a real change of mind we cannot
/// arbitrate, and then the winner stands and the log says what was passed over.
fn realm_toc_rows() -> Option<Vec<String>> {
let copies = chain()?.read_all(CHAIN_TOC);
let rows = |bytes: &[u8]| Toc::parse(&benilla_ui::source::decode(bytes)).files;
// Ascending priority, so the last copy is the one a read answers with.
let (winner_at, winner_bytes) = copies.last()?;
let winner = rows(winner_bytes);
let key = |s: &str| s.replace('/', "\\").to_ascii_lowercase();
let held: std::collections::HashSet<String> = winner.iter().map(|r| key(r)).collect();
let mut best: Option<(&std::path::Path, Vec<String>)> = None;
for (at, bytes) in copies.iter().rev().skip(1) {
let theirs = rows(bytes);
let keys: std::collections::HashSet<String> = theirs.iter().map(|r| key(r)).collect();
if !held.is_subset(&keys) {
// Not an extension of the winner: it drops rows the winner has.
if theirs.len() > winner.len() {
warn!(
"framexml: {} carries a {}-row FrameXML.toc against the {} rows of {}, which \
mounts above it \u{2014} passed over, because it also drops rows the winning \
copy has, so it is a different list and not a later edition of it",
at.display(),
theirs.len(),
winner.len(),
winner_at.display()
);
}
continue;
}
if theirs.len() > best.as_ref().map_or(winner.len(), |(_, b)| b.len()) {
best = Some((at, theirs));
}
}
match best {
Some((at, rows)) => {
warn!(
"framexml: reading the {}-row FrameXML.toc in {} rather than the {} rows of {}, \
which mounts above it: the shadowed copy holds every row the winner does and {} \
more, so it is the realm's later manifest with an older copy on top of it. The \
clean fix is on disk \u{2014} that archive should not carry the realm's interface",
rows.len(),
at.display(),
winner.len(),
winner_at.display(),
rows.len() - winner.len()
);
Some(rows)
}
None => Some(winner),
}
}
/// Every row the realm's `FrameXML.toc` carries that the manifest does not name, prefixed into
/// chain paths and in the chain's order — loaded after the manifest, which is where the chain's own
/// toc puts the realm's additions. Computed once per process, and the log line is the whole point
/// of it: after a client update it names what arrived without anyone having to diff two archives.
///
/// `WOW_FRAMEXML_TAIL=0` turns it off, for the one case this cannot be right — a realm row that
/// breaks the UI outright. Then the log says what is being left out, rather than nothing.
pub(super) fn tail() -> &'static [String] {
static TAIL: OnceLock<Vec<String>> = OnceLock::new();
TAIL.get_or_init(|| {
let Some(rows) = realm_toc_rows() else {
// No client data, or a chain without the row: `chain()` has already said so.
return Vec::new();
};
let manifest = Addon::builtin().toc.files;
let unnamed = unnamed_chain_rows(&rows, &manifest);
let off = std::env::var("WOW_FRAMEXML_TAIL").is_ok_and(|v| v == "0");
let declined: Vec<&str> = DECLINED
.iter()
.map(|(n, _)| *n)
.filter(|n| {
rows.iter().any(|r| {
r.eq_ignore_ascii_case(n) || r.replace('/', "\\").eq_ignore_ascii_case(n)
})
})
.collect();
match (unnamed.is_empty(), off) {
(true, _) => info!(
"framexml: the realm's toc has {} rows, all named by benilla.toc ({} declined: {})",
rows.len(),
declined.len(),
declined.join(", ")
),
(false, false) => info!(
"framexml: the realm's toc has {} rows and benilla.toc names all but {} \u{2014} \
loading them after it, in the chain's order (WOW_FRAMEXML_TAIL=0 opts out): {}",
rows.len(),
unnamed.len(),
unnamed.join(", ")
),
(false, true) => warn!(
"framexml: WOW_FRAMEXML_TAIL=0 \u{2014} NOT loading the {} row(s) the realm's toc \
carries that benilla.toc does not name, so whatever they build does not exist: {}",
unnamed.len(),
unnamed.join(", ")
),
}
if off {
Vec::new()
} else {
unnamed
}
})
}
/// The stock interface as an [`Addon`] over the chain; `files` are full chain paths.
pub(super) fn addon(files: Vec<String>) -> Addon {
Addon::new(
@@ -85,6 +256,103 @@ fn is_word(c: char) -> bool {
mod tests {
use benilla_ui::script::UiScript;
fn rows(list: &[&str]) -> Vec<String> {
list.iter().map(|s| (*s).to_string()).collect()
}
/// The merge that keeps a client update from going unnoticed: a row the realm's toc carries and
/// `benilla.toc` does not name is picked up, in the chain's order, whichever way either side
/// spells it.
///
/// This one deliberately takes both lists as arguments rather than reading the chain, so it
/// runs on a machine with no client data — where the ~1600 data-gated tests skip, and where a
/// regression in exactly this logic would therefore pass unseen.
#[test]
fn a_row_the_realms_toc_adds_is_picked_up_and_a_declined_one_is_not() {
let chain = rows(&[
"Fonts.xml",
"ChatFrame.xml",
"StatsFrame.xml",
"Turtle_GroupUI\\Turtle_GroupUI.xml",
"OctoGroupUIFix\\OctoGroupUIFix.lua",
"OctoBrandNew\\OctoBrandNew.lua",
]);
let manifest = rows(&[
"Fonts.xml",
"Interface\\FrameXML\\ChatFrame.xml",
// The realm writes a backslash, the manifest a forward slash: the same row.
"Interface/FrameXML/Turtle_GroupUI/Turtle_GroupUI.xml",
// Case is not stable across the realm's own rows either.
"Interface\\FrameXML\\octogroupuifix\\OctoGroupUIFix.lua",
"GameMenuFrame.xml",
]);
assert_eq!(
super::unnamed_chain_rows(&chain, &manifest),
rows(&["Interface\\FrameXML\\OctoBrandNew\\OctoBrandNew.lua"]),
"only the row nobody named, and never the declined one"
);
}
/// Order is the chain's, not the manifest's and not alphabetical, because a pair like
/// `Turtle_GroupUI.xml` and the fix that patches it only works one way round.
#[test]
fn the_new_rows_keep_the_realms_own_order() {
let chain = rows(&["Zeta\\Zeta.lua", "Alpha\\Alpha.lua", "Mid\\Mid.lua"]);
assert_eq!(
super::unnamed_chain_rows(&chain, &[]),
rows(&[
"Interface\\FrameXML\\Zeta\\Zeta.lua",
"Interface\\FrameXML\\Alpha\\Alpha.lua",
"Interface\\FrameXML\\Mid\\Mid.lua",
])
);
}
/// Every row [`super::tail`] hands the loader is one the chain can actually read back. The
/// tail builds those paths by prefixing the realm's toc rows, and a prefixing or separator
/// slip would not fail anything — the row would simply not load, which is the silence this
/// whole mechanism exists to end. Prints the tail, so `--nocapture` after a client update
/// says what arrived.
#[test]
fn every_row_the_tail_adds_is_readable_from_the_chain() {
let _data = benilla_formats::wow_data_or_skip!();
let tail = super::tail();
eprintln!(
"the realm's toc carries {} row(s) benilla.toc does not name: {}",
tail.len(),
if tail.is_empty() {
"none".to_string()
} else {
tail.join(", ")
}
);
let unreadable: Vec<&String> = tail.iter().filter(|r| super::read(r).is_none()).collect();
assert!(
unreadable.is_empty(),
"the tail named rows the chain cannot read back, so they would not load and nothing \
would say so: {unreadable:?}"
);
}
/// A [`super::DECLINED`] row must still be a row the realm ships, or it is a note about a file
/// nobody has any more — and the next reader would take it for a live decision.
#[test]
fn every_declined_row_is_one_the_realm_still_carries() {
let _data = benilla_formats::wow_data_or_skip!();
let bytes = super::read(super::CHAIN_TOC).expect("the realm's own FrameXML.toc");
let carried = benilla_ui::toc::Toc::parse(&benilla_ui::source::decode(&bytes)).files;
let stale: Vec<&str> = super::DECLINED
.iter()
.map(|(n, _)| *n)
.filter(|n| !carried.iter().any(|r| r.eq_ignore_ascii_case(n)))
.collect();
assert!(
stale.is_empty(),
"these are declined but the realm's toc no longer lists them \u{2014} drop the rows: {}",
stale.join(", ")
);
}
/// Every `RegisterForSave`'d UI global defaults to the value the reference's own
/// `UIOptionsFrame_Init` assigns, the FrameXML half of the rule `cvars::REGISTERED` holds for
/// CVars. A name the reference declares elsewhere (`SHOW_OFFLINE_GUILD_MEMBERS`,
+72
View File
@@ -135,6 +135,32 @@ impl Chain {
self.read(name)
}
/// Every archive's copy of `name` with the archive it came from, **ascending priority**, so the
/// last is the one [`Self::read`] answers with and the others are shadowed.
///
/// A read never wants this: one name is one file, and the winner is the answer. A *diagnosis*
/// does, because a shadowed copy is invisible by construction and some of them matter — an
/// archive mounting above the realm's own can carry a stale `FrameXML.toc`, and then the
/// interface silently loads an older list of files than the realm ships, which is a missing
/// feature with nothing anywhere to say why. Stops at the first tombstone from the top, as a
/// read does: a copy under a delete marker is not shadowed, it is deleted.
pub fn read_all(&self, name: &str) -> Vec<(&Path, Vec<u8>)> {
let mut out = Vec::new();
for archive in self.archives.iter().rev() {
if !archive.contains(name) {
continue;
}
if archive.is_delete_marker(name) {
break;
}
if let Ok(bytes) = archive.read_file(name) {
out.push((archive.path(), bytes));
}
}
out.reverse();
out
}
/// The chain's named files with sizes, for development and extraction; a file in no listfile
/// (most of `texture.MPQ`) is readable by name but not listed. Unions every archive's
/// `(listfile)`, as each names only its own files; sizes come from the winning archive.
@@ -233,6 +259,52 @@ mod tests {
assert!(!is_patch_glob_match("mypatch-2.MPQ"));
}
/// A letter patch mounts **above** a digit patch, because the reference sorts the glob's hits
/// by case-folded name and walks them so the last wins. Pinned on its own because it is not a
/// detail: a realm that ships its interface in `patch-5.mpq` has it shadowed by any
/// `patch-A.mpq` on the same disk, which on grom's install silently cost six FrameXML rows and
/// the `/xpboost` command (`ui_script::reference_ui::realm_toc_rows`).
#[test]
fn a_letter_patch_mounts_above_a_digit_patch() {
let dir = owned(&["patch-A.mpq", "patch-5.mpq", "patch-1.mpq", "Patch-Q.mpq"]);
assert_eq!(
mount_order(&dir),
owned(&["patch-1.mpq", "patch-5.mpq", "patch-A.mpq", "Patch-Q.mpq"]),
"ascending priority: the later entry's copy of a file wins"
);
}
/// [`Chain::read_all`]'s contract, on a real chain: the copies come back in ascending priority
/// and the last of them is the one [`Chain::read`] answers with.
#[test]
fn read_all_ends_with_the_copy_a_read_answers_with() {
let Some(data) = crate::wow_data() else {
eprintln!("skipped: no client data");
return;
};
let chain = Chain::open(&data).expect("open the patch chain");
// A name every vanilla chain carries in more than one archive.
let name = "Interface\\FrameXML\\FrameXML.toc";
let copies = chain.read_all(name);
assert!(
copies.len() > 1,
"this chain holds {} copy of {name}, so the ordering is untested here",
copies.len()
);
let (at, bytes) = copies.last().expect("a copy");
assert_eq!(
*bytes,
chain.read(name).expect("the winning copy"),
"the last copy must be the one a read returns; it came from {}",
at.display()
);
assert_eq!(
Some(*at),
chain.find_file_archive(name),
"and the archive it names must be the one the resolver picks"
);
}
#[test]
fn mount_order_is_the_carved_law() {
// base.MPQ is telemetry-only in the reference and never mounts.