`WOW_UI_EVENTS=<secs>` (`script/event_prof.rs`): per event, what one fire
costs across every handler it reaches, and which frame took the most of the
worst fire.
The gap it fills is a dispatcher. `WOW_UI_HANDLERS` keys on `(frame, script)`,
which is the right key for an `OnUpdate` and the wrong one for an `OnEvent`,
because every addon library registers *one* frame for *every* event it wants and
fans out in Lua. So the largest identifiable cost in grom's 2026-10-21 session,
eleven separate times, read as
[ui-handlers] 312.5 312.5 0.00 72522.3 AceEvent30Frame:OnEvent
— a true row that names a library's shared frame and no addon, no event and no
cause. `[sys-timing]` saw the same cost as `ui_quest::feed_quest` at 85.7 ms for
one run in 76, because every handler runs synchronously inside whichever system
fired, so the system holding the stack gets the bill.
Two conventions the report sticks to:
- `self` is nest-adjusted, in both columns. A handler's own time has any event it
fired taken back out (`event::fire` differences `event_prof::nested_ns` across
the call), or every event in a cascade reports the cascade and the row naming
the real cost is the one that looks innocent. `total` keeps it, which is what a
`[sys-timing]` span sees.
- A fire is one `fire_event`, however wide it fans out, because that is the unit
a dropped frame is measured in. `calls` is beside it for the width.
Measured on the player's own 92-addon interface (`tick_bench`'s new `[events]`
row, three fires per event from a settled start):
us/fire worst event
5068 14759 QUEST_DETAIL <- HCQuest.lua:139 took 14,097 us
764 1324 QUEST_LOG_UPDATE
662 1564 QUEST_FINISHED
Read `us/fire` against `worst` and that 14.8 ms is **one** fire of three: the
other two cost ~0.1 ms, so it is HCQuest building something once and keeping it,
not a cost paid per quest. With that read correctly, no quest event costs more
than ~1.5 ms recurring offline — so the bench does not reproduce the live 57-73
ms and says so, rather than offering a number that looks like an answer. Hence
the instrument, armed in `./measure.sh`.
Also adds `./measure.sh --frames`, which arms two meters that already existed and
have never been armed: `WOW_FRAME_PHASES=50` and `WOW_RENDER_MS=1`. A frame's
wall time is `max(main app, render app)` since bevy pipelines the render app onto
its own thread, so when the render app is the longer, `wall` climbs while `cpu`,
`main`, `gpu` and every per-system `worst` all read healthy — exactly what that
session's **161 ms frame in a second whose costliest system worst was 2.9 ms over
105 runs** looks like, and what 43 of its 64 spike-seconds look like. Its own
mode because both perturb what they measure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SgstFWtn1RxEJ5kVrpgYwn
361 lines
14 KiB
Rust
361 lines
14 KiB
Rust
//! Per-event cost attribution (`WOW_UI_EVENTS=<secs>`): what **one occurrence** of each event
|
|
//! costs, summed across every handler it fans out to, plus which frame took the most of it.
|
|
//!
|
|
//! The gap this fills is a dispatcher. [`super::handler_prof`] keys on `(frame, script)`, which is
|
|
//! the right key for an `OnUpdate` and the wrong one for an `OnEvent`, because the addon libraries
|
|
//! every interface is built on register **one** frame for **every** event they care about and fan
|
|
//! out in Lua. So a 72 ms hitch reads as `AceEvent30Frame:OnEvent · calls/f 0.00 · worst 72522 us`
|
|
//! — a true row that names a library's shared frame and no addon, no event and no cause. In grom's
|
|
//! 2026-10-21 session that one row was the largest identifiable cost in the game, eleven times.
|
|
//!
|
|
//! An event's cost is not a frame's cost and not a handler's cost: it is what the engine pays for
|
|
//! deciding to tell the interface something, which is the thing engine code chooses and can stop
|
|
//! choosing. `[sys-timing]` can see that total — `ui_quest::feed_quest` read 85.7 ms, one run in
|
|
//! 76 — but only as the system that happened to be holding the stack, since every handler runs
|
|
//! synchronously inside whichever system fired. This report is that number broken out by event.
|
|
//!
|
|
//! Two timers per handler fire while armed, and a relaxed atomic load per fire while not.
|
|
|
|
use std::collections::HashMap;
|
|
use std::sync::atomic::{AtomicBool, Ordering};
|
|
use std::time::Instant;
|
|
|
|
use mlua::Lua;
|
|
|
|
use super::UiScript;
|
|
|
|
/// How many events the periodic report prints before rolling the rest into a `… N more` line.
|
|
const REPORT_ROWS: usize = 14;
|
|
|
|
/// A single fan-out this slow is a dropped frame at any framerate a player plays at.
|
|
const STALL_US: f64 = 2000.0;
|
|
|
|
/// Whether any VM in the process profiles events: process-wide, so the per-fire check needs no
|
|
/// `app_data` lookup.
|
|
static ARMED: AtomicBool = AtomicBool::new(false);
|
|
|
|
#[inline]
|
|
pub(super) fn armed() -> bool {
|
|
ARMED.load(Ordering::Relaxed)
|
|
}
|
|
|
|
/// The report period from `WOW_UI_EVENTS=<secs>`, read once; unparseable or non-positive is off.
|
|
fn env_period() -> Option<f32> {
|
|
static PERIOD: std::sync::OnceLock<Option<f32>> = std::sync::OnceLock::new();
|
|
*PERIOD.get_or_init(|| {
|
|
std::env::var("WOW_UI_EVENTS")
|
|
.ok()
|
|
.and_then(|v| v.parse::<f32>().ok())
|
|
.filter(|secs| *secs > 0.0)
|
|
})
|
|
}
|
|
|
|
/// One event name's accumulation over the current window.
|
|
#[derive(Default, Clone)]
|
|
struct Slot {
|
|
/// Occurrences of the event, not handler calls: one `fire_event` is one fire however many
|
|
/// frames listen.
|
|
fires: u32,
|
|
/// Handler calls those fires made, so a cost can be read against a fan-out width.
|
|
calls: u32,
|
|
/// Every handler of every fire, with the cost of events those handlers fired in turn taken
|
|
/// out — so a cascade's total is not charged to every event in it. Work a handler provokes
|
|
/// that is *not* another event (a layout resolve, an `OnShow` it triggered) stays here, where
|
|
/// it belongs: the event is why it happened.
|
|
self_ns: u64,
|
|
/// The same including events its handlers fired in turn.
|
|
total_ns: u64,
|
|
/// The costliest single fan-out, and the frame that took the most of *that* fan-out. The mean
|
|
/// is not what a player feels and the fan-out's widest handler is where to look first.
|
|
worst_ns: u64,
|
|
worst_frame: Option<u32>,
|
|
worst_frame_ns: u64,
|
|
}
|
|
|
|
/// The profiler's state, one per VM, keyed by event name.
|
|
#[derive(Default)]
|
|
pub(crate) struct EventProf {
|
|
/// Whether this VM records; [`armed`] is the process-wide gate.
|
|
on: bool,
|
|
/// Seconds between reports; `0.0` records and never prints.
|
|
period: f32,
|
|
/// Wall seconds and frames since the last report, the divisor of every printed rate.
|
|
window: f32,
|
|
frames: u32,
|
|
/// Nanos already charged to nested fires, one entry per open fan-out.
|
|
stack: Vec<u64>,
|
|
/// Every nested fan-out's cost since this VM started, monotonic. [`note_handler`]'s caller
|
|
/// reads it either side of a handler to subtract the events that handler fired, which is the
|
|
/// only way to tell "this handler is slow" from "this handler fired a slow event".
|
|
nested_ns: u64,
|
|
/// The open fan-out's running handler tally: `(calls, self ns, worst frame, worst frame ns)`.
|
|
open: Vec<(u32, u64, Option<u32>, u64)>,
|
|
rows: HashMap<String, Slot>,
|
|
}
|
|
|
|
/// An open fan-out, closed by its `Drop`, so the nesting stack stays balanced through an unwind.
|
|
pub(super) struct Dispatch<'a> {
|
|
lua: &'a Lua,
|
|
event: &'a str,
|
|
/// `None` while this VM is not recording, which makes the guard a no-op.
|
|
started: Option<Instant>,
|
|
}
|
|
|
|
impl<'a> Dispatch<'a> {
|
|
/// Open a fan-out of `event`. Cheap and inert unless this VM is recording.
|
|
pub(super) fn open(lua: &'a Lua, event: &'a str) -> Dispatch<'a> {
|
|
let started = match lua.app_data_mut::<EventProf>() {
|
|
Some(mut prof) if prof.on => {
|
|
prof.stack.push(0);
|
|
prof.open.push((0, 0, None, 0));
|
|
Some(Instant::now())
|
|
}
|
|
_ => None,
|
|
};
|
|
Dispatch {
|
|
lua,
|
|
event,
|
|
started,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Drop for Dispatch<'_> {
|
|
fn drop(&mut self) {
|
|
let Some(started) = self.started else {
|
|
return;
|
|
};
|
|
let total = started.elapsed().as_nanos() as u64;
|
|
let Some(mut prof) = self.lua.app_data_mut::<EventProf>() else {
|
|
return;
|
|
};
|
|
let children = prof.stack.pop().unwrap_or(0);
|
|
if let Some(parent) = prof.stack.last_mut() {
|
|
*parent += total;
|
|
}
|
|
// Unconditional: a top-level fan-out cannot be inside a handler, since a handler only runs
|
|
// inside a fan-out, so nothing reads this across one.
|
|
prof.nested_ns = prof.nested_ns.wrapping_add(total);
|
|
let (calls, self_ns, worst_frame, worst_frame_ns) = prof.open.pop().unwrap_or_default();
|
|
let slot = prof.rows.entry(self.event.to_string()).or_default();
|
|
slot.fires += 1;
|
|
slot.calls += calls;
|
|
slot.self_ns += self_ns;
|
|
slot.total_ns += total;
|
|
// The fan-out's own handlers, not the nested events they triggered: a hitch is charged to
|
|
// the event whose handlers ran, and a nested event gets its own row for its own.
|
|
let fanout = total.saturating_sub(children);
|
|
if fanout > slot.worst_ns {
|
|
slot.worst_ns = fanout;
|
|
slot.worst_frame = worst_frame;
|
|
slot.worst_frame_ns = worst_frame_ns;
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Nanos spent in nested fan-outs so far, for [`note_handler`]'s caller to difference across a
|
|
/// handler. Zero when this VM is not recording.
|
|
pub(super) fn nested_ns(lua: &Lua) -> u64 {
|
|
lua.app_data_ref::<EventProf>()
|
|
.filter(|prof| prof.on)
|
|
.map_or(0, |prof| prof.nested_ns)
|
|
}
|
|
|
|
/// Charge one handler call of the open fan-out, `ns` being its own time with any event it fired
|
|
/// taken out. Called from [`super::event::fire`], which is the only place that knows both the
|
|
/// event name and the frame.
|
|
pub(super) fn note_handler(lua: &Lua, id: u32, ns: u64) {
|
|
let Some(mut prof) = lua.app_data_mut::<EventProf>() else {
|
|
return;
|
|
};
|
|
if !prof.on {
|
|
return;
|
|
}
|
|
let Some(open) = prof.open.last_mut() else {
|
|
return;
|
|
};
|
|
open.0 += 1;
|
|
open.1 += ns;
|
|
if ns > open.3 {
|
|
open.2 = Some(id);
|
|
open.3 = ns;
|
|
}
|
|
}
|
|
|
|
/// One event's cost over the profiler's current window.
|
|
#[derive(Clone, Debug, PartialEq)]
|
|
pub struct EventRow {
|
|
pub event: String,
|
|
/// Occurrences, not handler calls.
|
|
pub fires: u32,
|
|
pub calls: u32,
|
|
pub self_us: f64,
|
|
pub total_us: f64,
|
|
/// The costliest single fan-out.
|
|
pub worst_us: f64,
|
|
/// The frame that took the most of that fan-out, already labelled.
|
|
pub worst_frame: Option<String>,
|
|
pub worst_frame_us: f64,
|
|
}
|
|
|
|
impl UiScript {
|
|
/// Arm or disarm event profiling for this VM, as [`UiScript::profile_handlers`] does for
|
|
/// handlers. `period_secs` of `0.0` records without printing, for a test or a bench.
|
|
pub fn profile_events(&self, on: bool, period_secs: f32) {
|
|
if let Some(mut prof) = self.lua.app_data_mut::<EventProf>() {
|
|
prof.on = on;
|
|
prof.period = period_secs;
|
|
}
|
|
if on {
|
|
ARMED.store(true, Ordering::Relaxed);
|
|
}
|
|
}
|
|
|
|
/// Every event's cost this window, costliest first by the self time of its handlers.
|
|
pub fn event_profile(&self) -> Vec<EventRow> {
|
|
// Snapshot first and label after: `handler_label` borrows the model, and the profiler's
|
|
// own borrow must be released before it does.
|
|
let raw: Vec<(String, Slot)> = {
|
|
let Some(prof) = self.lua.app_data_ref::<EventProf>() else {
|
|
return Vec::new();
|
|
};
|
|
prof.rows
|
|
.iter()
|
|
.map(|(event, slot)| (event.clone(), slot.clone()))
|
|
.collect()
|
|
};
|
|
let mut rows: Vec<EventRow> = raw
|
|
.into_iter()
|
|
.map(|(event, s)| EventRow {
|
|
event,
|
|
fires: s.fires,
|
|
calls: s.calls,
|
|
self_us: ns_to_us(s.self_ns),
|
|
total_us: ns_to_us(s.total_ns),
|
|
worst_us: ns_to_us(s.worst_ns),
|
|
worst_frame: s.worst_frame.map(|id| self.handler_label(id, "OnEvent")),
|
|
worst_frame_us: ns_to_us(s.worst_frame_ns),
|
|
})
|
|
.collect();
|
|
rows.sort_by(|a, b| {
|
|
b.self_us
|
|
.total_cmp(&a.self_us)
|
|
.then_with(|| a.event.cmp(&b.event))
|
|
});
|
|
rows
|
|
}
|
|
|
|
pub(super) fn report_event_profile(&mut self, elapsed: f32) {
|
|
if !armed() {
|
|
return;
|
|
}
|
|
let due = match self.lua.app_data_mut::<EventProf>() {
|
|
Some(mut prof) if prof.on => {
|
|
prof.frames += 1;
|
|
prof.window += elapsed;
|
|
prof.period > 0.0 && prof.window >= prof.period
|
|
}
|
|
_ => false,
|
|
};
|
|
if !due {
|
|
return;
|
|
}
|
|
// The rows first: the label walk's model borrow must end before the profiler's mutable one.
|
|
let rows = self.event_profile();
|
|
let mut prof = self
|
|
.lua
|
|
.app_data_mut::<EventProf>()
|
|
.expect("profiler app_data — checked above");
|
|
print_report(&rows, prof.window, prof.frames);
|
|
prof.window = 0.0;
|
|
prof.frames = 0;
|
|
prof.rows.clear();
|
|
}
|
|
}
|
|
|
|
#[allow(clippy::cast_precision_loss)]
|
|
fn ns_to_us(ns: u64) -> f64 {
|
|
ns as f64 / 1000.0
|
|
}
|
|
|
|
/// The `[ui-events]` block: the window, then the costliest [`REPORT_ROWS`] events by the self time
|
|
/// of their handlers, then the stalling ones the table left out.
|
|
fn print_report(rows: &[EventRow], window: f32, frames: u32) {
|
|
let per_frame = f64::from(frames.max(1));
|
|
let fires: u32 = rows.iter().map(|r| r.fires).sum();
|
|
let total: f64 = rows.iter().map(|r| r.self_us).sum();
|
|
eprintln!(
|
|
"[ui-events] {window:.2}s · {frames} frames · {} event(s) · {fires} fire(s) · \
|
|
{:.1} us/f in handlers",
|
|
rows.len(),
|
|
total / per_frame,
|
|
);
|
|
eprintln!("[ui-events] us/f fires/f us/fire worst worst handler / event");
|
|
for row in rows.iter().take(REPORT_ROWS) {
|
|
let per_fire = row.self_us / f64::from(row.fires.max(1));
|
|
eprintln!(
|
|
"[ui-events] {:>6.1} {:>8.2} {:>8.1} {:>8.1} {} · {}",
|
|
row.self_us / per_frame,
|
|
f64::from(row.fires) / per_frame,
|
|
per_fire,
|
|
row.worst_us,
|
|
row.event,
|
|
describe_worst(row),
|
|
);
|
|
}
|
|
if rows.len() > REPORT_ROWS {
|
|
let rest: f64 = rows.iter().skip(REPORT_ROWS).map(|r| r.self_us).sum();
|
|
eprintln!(
|
|
"[ui-events] {:>6.1} … {} more",
|
|
rest / per_frame,
|
|
rows.len() - REPORT_ROWS
|
|
);
|
|
}
|
|
// The events the table's own ranking hides: one that fires once a minute and costs 70 ms when
|
|
// it does has no mean worth printing and is the only thing the player noticed.
|
|
let mut stalls: Vec<&EventRow> = rows
|
|
.iter()
|
|
.skip(REPORT_ROWS)
|
|
.filter(|r| r.worst_us >= STALL_US)
|
|
.collect();
|
|
stalls.sort_by(|a, b| b.worst_us.total_cmp(&a.worst_us));
|
|
if !stalls.is_empty() {
|
|
eprintln!("[ui-events] worst single fan-out, among the events not listed above:");
|
|
for row in stalls.iter().take(REPORT_ROWS) {
|
|
eprintln!(
|
|
"[ui-events] {:>8.1} us over {} fire(s) {} · {}",
|
|
row.worst_us,
|
|
row.fires,
|
|
row.event,
|
|
describe_worst(row),
|
|
);
|
|
}
|
|
}
|
|
}
|
|
|
|
/// `most of it in <frame> (<us>)`, or a note that no handler was timed — an event with listeners
|
|
/// that all unregistered mid-walk, or one fired with nothing registered at all.
|
|
fn describe_worst(row: &EventRow) -> String {
|
|
match &row.worst_frame {
|
|
Some(frame) => format!(
|
|
"{} call(s), worst in {frame} ({:.1} us)",
|
|
row.calls, row.worst_frame_us
|
|
),
|
|
None => format!("{} call(s), no handler timed", row.calls),
|
|
}
|
|
}
|
|
|
|
/// Install the profiler's `app_data` slot, armed from the environment. Unconditional: mlua refuses
|
|
/// to insert app data while any is borrowed, so the slot must exist before the VM runs.
|
|
pub(super) fn install(lua: &Lua) {
|
|
let period = env_period();
|
|
lua.set_app_data(EventProf {
|
|
on: period.is_some(),
|
|
period: period.unwrap_or_default(),
|
|
..EventProf::default()
|
|
});
|
|
if period.is_some() {
|
|
ARMED.store(true, Ordering::Relaxed);
|
|
}
|
|
}
|