collective
Browse / github.com/storytold/soundcraft / patch
Playback: audio callback load and dropout meter (System Usage, ui.audio_load)
Playback: audio callback load and dropout meter (System Usage, ui.audio_load) ## What A load and dropout meter for the audio callback. - Each device callback is timed against how long its buffer lasts. **Window ▸ System Usage** shows: - the average and peak load over the last half second, with a bar; - the heaviest callback; - the number of *late* callbacks: callbacks whose render took longer than the buffer lasts, so the device ran dry (a dropout). - A Reset button starts the worst load and the count again. - `ui.audio_load {reset?}` returns the same for agents: `measured`, `average`, `peak`, `worst`, `late`, `callbacks`, `buffer_ms`. The MCP reaches it throu
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- sanderdewijs (imported from pull request #1067, open)
- Base
7dc56a4dbeebf57eacece09536726c57844fc59d- Size
- +267 −19 in 8 files
- Review
- pending
- Shared
- 2026-10-28
- Id
0b6d909ed87edbf176bfb314
How it went
No other operator has reported using it yet.
Diff
raw · crates/playback/src/lib.rs crates/playback/src/load.rs crates/ui-egui/src/menus.rs crates/ui-egui/src/windows.rs crates/ui-egui/tests/it/audio_load.rs crates/ui-egui/tests/it/main.rs docs/gaps.md docs/hardware-parity.md
@@ -23,9 +23,11 @@ #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] pub mod devices;+mod load; pub mod record; pub use devices::AudioConfig;+pub use load::AudioLoad; use soundcraft_dsp::{Plugin, PluginEditor, PluginStateReader}; use soundcraft_mix::{MAX_CHANNELS, MixEngine, PreparedInstance, StripMeter, itu_stereo_fold, main_channels};@@ -557,6 +559,7 @@ impl Side { pub struct Player { shared: Arc<Shared>, fault: Arc<Fault>,+ load: Arc<load::LoadMeter>, /// The block count last seen while playing, and since when. #[cfg(not(target_arch = "wasm32"))] watch: Mutex<Option<(u64, std::time::Instant)>>,@@ -592,6 +595,17 @@ impl Player { p } + /// How much of each device callback's time the render takes, and how many were late+ /// (dropouts). Not measured without a device.+ pub fn audio_load(&self) -> AudioLoad {+ self.load.snapshot()+ }++ /// Starts the worst load and the late count again.+ pub fn reset_audio_load(&self) {+ self.load.reset();+ }+ /// What stopped the audio output, once: a bug in the engine, or the device went away or /// stopped asking for audio while playing. pub fn trouble(&self) -> Option<OutputTrouble> {@@ -643,17 +657,19 @@ impl Player { let (gtx, grx) = channel(); let state = AudioState::new(rx, Arc::clone(&shared), Arc::clone(&session), f64::from(rate), gtx); let fault = Arc::new(Fault::default());+ let load = Arc::new(load::LoadMeter::default());+ let out = Output { channels, rate: f64::from(rate), fault: &fault, shared: &shared, load: &load }; let stream = match format {- cpal::SampleFormat::I32 => build_output::<i32>(&device, *config, state, channels, &fault, &shared),- cpal::SampleFormat::I16 => build_output::<i16>(&device, *config, state, channels, &fault, &shared),- _ => build_output::<f32>(&device, *config, state, channels, &fault, &shared),+ cpal::SampleFormat::I32 => build_output::<i32>(&device, *config, state, &out),+ cpal::SampleFormat::I16 => build_output::<i16>(&device, *config, state, &out),+ _ => build_output::<f32>(&device, *config, state, &out), }?;- Ok(Opened { stream, tx, grx, fault })+ Ok(Opened { stream, tx, grx, fault, load }) }, |opened| start_stream(&opened.stream), ); match opened {- Some((config, Opened { stream, tx, grx, fault })) => {+ Some((config, Opened { stream, tx, grx, fault, load })) => { let rate = config.sample_rate; let channels = usize::from(config.channels); let buffer = match config.buffer_size {@@ -667,6 +683,7 @@ impl Player { Player { shared, fault,+ load, #[cfg(not(target_arch = "wasm32"))] watch: Mutex::new(None), tx,@@ -745,6 +762,7 @@ impl Player { Player { shared, fault,+ load: Arc::default(), watch: Mutex::new(None), tx, side,@@ -769,6 +787,7 @@ impl Player { Player { shared, fault: Arc::default(),+ load: Arc::default(), tx, side, _stream: None,@@ -1057,6 +1076,7 @@ struct Opened { tx: Sender<Cmd>, grx: Receiver<Vec<Box<dyn Plugin>>>, fault: Arc<Fault>,+ load: Arc<load::LoadMeter>, } /// Build and start the attempts in order; the first that builds and starts wins. A build error or@@ -1077,28 +1097,37 @@ fn first_started<A: std::fmt::Debug, S, E: std::fmt::Display>( None } -/// The output stream: the mix renders f32 into a scratch buffer allocated here (so the callback-/// never allocates), converted to the device's sample type.-fn build_output<T>(- device: &cpal::Device,- config: cpal::StreamConfig,- mut state: AudioState,+/// What an output stream's callback shares with the player.+struct Output<'a> { channels: usize,- fault: &Arc<Fault>,- shared: &Arc<Shared>,-) -> Result<cpal::Stream, cpal::Error>+ /// Device frames per second.+ rate: f64,+ fault: &'a Arc<Fault>,+ shared: &'a Arc<Shared>,+ load: &'a Arc<load::LoadMeter>,+}++/// The output stream: the mix renders f32 into a scratch buffer allocated here (so the callback+/// never allocates), converted to the device's sample type. Each callback is timed for the load+/// meter.+fn build_output<T>(device: &cpal::Device, config: cpal::StreamConfig, mut state: AudioState, out: &Output) -> Result<cpal::Stream, cpal::Error> where T: cpal::SizedSample + cpal::FromSample<f32>, { use cpal::traits::DeviceTrait;+ let (channels, rate) = (out.channels, out.rate); let frame = channels.max(1); let mut scratch = vec![0.0f32; 8192 * frame]; let chunk = scratch.len();- let (failed, guard, gone) = (Arc::clone(fault), Arc::clone(shared), Arc::clone(fault));+ let (failed, guard, gone) = (Arc::clone(out.fault), Arc::clone(out.shared), Arc::clone(out.fault));+ let meter = Arc::clone(out.load);+ let mut window = load::LoadWindow::default(); device.build_output_stream( config, move |data: &mut [T], _: &cpal::OutputCallbackInfo| { mark_audio_thread();+ #[cfg(not(target_arch = "wasm32"))]+ let started = std::time::Instant::now(); for out in data.chunks_mut(chunk) { let Some(s) = scratch.get_mut(..out.len()) else { return }; guarded_block(&failed, &guard, s, |d| state.fill(d, channels));@@ -1106,6 +1135,10 @@ where *o = T::from_sample(*x); } }+ #[cfg(not(target_arch = "wasm32"))]+ meter.record(&mut window, started.elapsed().as_secs_f64(), (data.len() / frame) as f64 / rate);+ #[cfg(target_arch = "wasm32")]+ let _ = (&meter, &mut window, rate); }, // cpal may call this on the audio thread (on ALSA in a loop while the device stays // broken): marked, so the logger hands the record off instead of writing it.new file mode 100644@@ -0,0 +1,156 @@+//! The audio callback's load: how long each device callback takes to render, against how long its+//! buffer lasts. The audio thread is the only writer and the UI reads; both lock-free and+//! allocation-free.+//!+//! A callback that takes longer than its buffer lasts is *late*: the device ran out of audio+//! before the callback returned, which is heard as a dropout. cpal doesn't report the device's+//! own overload notices, so late callbacks are counted here instead.++use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, Ordering::Relaxed};++/// Audio the average and peak are taken over before they are published, in seconds.+const WINDOW_S: f64 = 0.5;++/// The callback load, for the UI and agents. Loads are render time over buffer duration: 1.0 is+/// all of it.+#[derive(Debug, Clone, Copy, PartialEq, Default, serde::Serialize)]+pub struct AudioLoad {+ /// False until a device callback has been timed (no device, the silent clock or the web).+ pub measured: bool,+ /// Averaged over the last half second of audio.+ pub average: f32,+ /// The heaviest callback of the last half second.+ pub peak: f32,+ /// The heaviest callback since the stream opened, or since the last reset.+ pub worst: f32,+ /// Callbacks that took longer than their buffer lasts (dropouts), since the stream opened or+ /// the last reset.+ pub late: u64,+ /// Callbacks timed since the stream opened, or since the last reset.+ pub callbacks: u64,+ /// How long the last callback's buffer lasts, in milliseconds.+ pub buffer_ms: f32,+}++/// The published load (f32 values as bits).+#[derive(Default)]+pub(crate) struct LoadMeter {+ measured: AtomicBool,+ average: AtomicU32,+ peak: AtomicU32,+ worst: AtomicU32,+ late: AtomicU64,+ callbacks: AtomicU64,+ buffer_ms: AtomicU32,+ /// Asked by the UI; the audio thread clears the counters at its next callback.+ reset: AtomicBool,+}++/// The audio thread's own running window.+#[derive(Debug, Default)]+pub(crate) struct LoadWindow {+ busy: f64,+ audio: f64,+ peak: f64,+}++impl LoadMeter {+ /// One callback: `busy` seconds rendering `audio` seconds of output. Audio thread only.+ pub(crate) fn record(&self, w: &mut LoadWindow, busy: f64, audio: f64) {+ if !(audio > 0.0 && busy.is_finite() && busy >= 0.0) {+ return;+ }+ if self.reset.swap(false, Relaxed) {+ self.worst.store(0, Relaxed);+ self.late.store(0, Relaxed);+ self.callbacks.store(0, Relaxed);+ }+ let load = busy / audio;+ self.callbacks.store(self.callbacks.load(Relaxed).saturating_add(1), Relaxed);+ if load > 1.0 {+ self.late.store(self.late.load(Relaxed).saturating_add(1), Relaxed);+ }+ if load as f32 > f32::from_bits(self.worst.load(Relaxed)) {+ self.worst.store((load as f32).to_bits(), Relaxed);+ }+ self.buffer_ms.store(((audio * 1000.0) as f32).to_bits(), Relaxed);+ self.measured.store(true, Relaxed);+ w.busy += busy;+ w.audio += audio;+ w.peak = w.peak.max(load);+ if w.audio >= WINDOW_S {+ self.average.store(((w.busy / w.audio) as f32).to_bits(), Relaxed);+ self.peak.store((w.peak as f32).to_bits(), Relaxed);+ *w = LoadWindow::default();+ }+ }++ pub(crate) fn snapshot(&self) -> AudioLoad {+ let f = |a: &AtomicU32| f32::from_bits(a.load(Relaxed));+ AudioLoad {+ measured: self.measured.load(Relaxed),+ average: f(&self.average),+ peak: f(&self.peak),+ worst: f(&self.worst),+ late: self.late.load(Relaxed),+ callbacks: self.callbacks.load(Relaxed),+ buffer_ms: f(&self.buffer_ms),+ }+ }++ /// Starts the worst load and the counts again (at the next callback).+ pub(crate) fn reset(&self) {+ self.reset.store(true, Relaxed);+ }+}++#[cfg(test)]+mod tests {+ use super::*;++ #[test]+ fn averages_and_peaks_are_published_per_window_and_late_callbacks_counted() {+ let m = LoadMeter::default();+ let mut w = LoadWindow::default();+ assert!(!m.snapshot().measured);+ // 125 ms buffers: three at 20% and one late at 120% close a half-second window.+ for busy in [0.025, 0.025, 0.15, 0.025] {+ m.record(&mut w, busy, 0.125);+ }+ let s = m.snapshot();+ assert!(s.measured);+ assert!((s.average - 0.45).abs() < 1e-6, "{s:?}");+ assert!((s.peak - 1.2).abs() < 1e-6, "{s:?}");+ assert!((s.worst - 1.2).abs() < 1e-6, "{s:?}");+ assert_eq!((s.late, s.callbacks), (1, 4));+ assert!((s.buffer_ms - 125.0).abs() < 1e-3);++ // Until the next window closes, the last one stays; then it is replaced. The worst load+ // and the counts carry on.+ for _ in 0..3 {+ m.record(&mut w, 0.0125, 0.125);+ }+ assert!((m.snapshot().average - 0.45).abs() < 1e-6);+ m.record(&mut w, 0.0125, 0.125);+ let s = m.snapshot();+ assert!((s.average - 0.1).abs() < 1e-6 && (s.peak - 0.1).abs() < 1e-6, "{s:?}");+ assert!((s.worst - 1.2).abs() < 1e-6);+ assert_eq!((s.late, s.callbacks), (1, 8));++ m.reset();+ m.record(&mut w, 0.0375, 0.125);+ let s = m.snapshot();+ assert_eq!((s.late, s.callbacks), (0, 1));+ assert!((s.worst - 0.3).abs() < 1e-6);+ }++ #[test]+ fn hostile_timings_are_ignored() {+ let m = LoadMeter::default();+ let mut w = LoadWindow::default();+ for (busy, audio) in [(0.01, 0.0), (0.01, -1.0), (f64::NAN, 0.1), (f64::INFINITY, 0.1), (-0.01, 0.1), (0.01, f64::NAN)] {+ m.record(&mut w, busy, audio);+ }+ assert_eq!(m.snapshot(), AudioLoad::default());+ }+}@@ -67,6 +67,7 @@ pub const UI_COMMANDS: &[(&str, &str, &str, Option<&str>)] = &[ ("window.rtp", "MIDI Real-Time Properties", "", None), ("window.playback_engine", "Playback Engine", "", None), ("ui.audio_device", "Audio Device", "", None),+ ("ui.audio_load", "Audio Load", "", None), ("window.io_setup", "I/O Setup", "", None), ("window.shortcuts", "Keyboard Shortcuts", "", None), ("view.mix_section", "Mix Window View", "", None),@@ -518,6 +519,9 @@ pub fn run_ui_command(app: &mut SoundApp, id: &str, p: &Value) -> Option<Result< if id == "ui.audio_device" { return Some(crate::windows::audio_device_command(app, p)); }+ if id == "ui.audio_load" {+ return Some(crate::windows::audio_load_command(app, p));+ } let toggle = |v: &mut bool| { *v = p.get("value").and_then(Value::as_bool).unwrap_or(!*v); json!({"value": *v})@@ -169,10 +169,27 @@ fn disk_usage(app: &mut SoundApp, ctx: &egui::Context) { fn system_usage(app: &mut SoundApp, ctx: &egui::Context) { let mut open = app.ui.show_system_usage;- win(ctx, &mut open, "System Usage", vec2(320.0, 140.0), |ui| {+ win(ctx, &mut open, "System Usage", vec2(320.0, 200.0), |ui| { ui.label(format!("UI frame time: {:.1} ms", app.frame_ms)); ui.label(format!("Audio: {}", app.player.as_ref().map_or("no engine".to_string(), |p| format!("{} @ {} Hz", p.device_name, p.device_rate))));- let plugins: usize = app.engine.session().tracks.iter().map(|t| t.mixer.inserts.iter().flatten().count()).sum();+ let load = app.player.as_ref().map(|p| p.audio_load()).unwrap_or_default();+ if load.measured {+ ui.label(format!("Audio load: {:.0}% (peak {:.0}%) of {:.1} ms", load.average * 100.0, load.peak * 100.0, load.buffer_ms));+ ui.add(egui::ProgressBar::new(load.peak.clamp(0.0, 1.0)).desired_height(6.0));+ ui.horizontal(|ui| {+ ui.label(format!("Late callbacks (dropouts): {} · worst {:.0}%", load.late, load.worst * 100.0));+ if ui.small_button("Reset").clicked()+ && let Some(p) = &app.player+ {+ p.reset_audio_load();+ }+ });+ ctx.request_repaint_after(std::time::Duration::from_millis(250));+ } else {+ ui.label("Audio load: not measured");+ }+ let session = app.engine.session();+ let plugins: usize = session.tracks.iter().map(|t| t.mixer.inserts.iter().flatten().count() + usize::from(t.instrument.is_some())).sum(); ui.label(format!("Active plugin instances: {plugins}")); ui.label(format!("Tracks: {}", app.engine.session().tracks.len())); ui.label(format!("Undo steps: {}", app.engine.undo_history().len()));@@ -587,6 +604,22 @@ pub fn playback_engine(app: &mut SoundApp, ctx: &egui::Context) { app.ui.show_playback_engine = open; } +/// `ui.audio_load`: `{reset?: bool}` → the audio callback's load (`soundcraft_playback::AudioLoad`):+/// render time over buffer duration, averaged (`average`) and at its heaviest (`peak`) over the+/// last half second, the heaviest (`worst`) and the callbacks that ran past their buffer (`late`,+/// dropouts) since the stream opened. `reset: true` starts `worst` and the counts again after+/// returning them. Without an audio device `measured` is false.+pub fn audio_load_command(app: &mut SoundApp, p: &serde_json::Value) -> Result<serde_json::Value, String> {+ let Some(player) = &app.player else {+ return serde_json::to_value(soundcraft_playback::AudioLoad::default()).map_err(|e| e.to_string());+ };+ let load = player.audio_load();+ if p.get("reset").and_then(serde_json::Value::as_bool) == Some(true) {+ player.reset_audio_load();+ }+ serde_json::to_value(load).map_err(|e| e.to_string())+}+ /// `ui.audio_device`: `{list?: bool}` returns the audio systems and their devices; /// `{system?, output?, input?, sample_rate?, buffer_size?}` (a `null` value means "default") sets /// the choice and reopens the devices. Always returns the choice and the open device.new file mode 100644@@ -0,0 +1,16 @@+//! `ui.audio_load`: the audio callback's load, for agents.+use serde_json::json;+use soundcraft_ui_egui::{Services, SoundApp};++#[test]+fn the_audio_load_is_unmeasured_without_an_audio_engine() {+ let mut app = SoundApp::new(soundcraft_engine::Engine::default(), None, Services::default());+ for p in [json!({}), json!({"reset": true})] {+ let r = app.run("ui.audio_load", p).unwrap();+ assert_eq!(r["measured"], false, "{r}");+ for key in ["average", "peak", "worst", "late", "callbacks", "buffer_ms"] {+ assert!(r[key].is_number(), "{key}: {r}");+ }+ }+ assert!(soundcraft_ui_egui::menus::UI_COMMANDS.iter().any(|c| c.0 == "ui.audio_load"), "listed for agents");+}@@ -5,6 +5,7 @@ mod appearance; mod audio_device;+mod audio_load; mod auto_rename; mod bounce; mod bounce_dialog;@@ -1,6 +1,6 @@ # Where SoundCraft falls short of Pro Tools -> **Last reviewed:** 2026-10-19 · **Last updated:** 2026-10-25 · **Change:** trivial (G-04: Clip Looping crossfade and loop count; previously trivial: G-08 and G-20: built-in plugin count, the sampler shipped) · **Target:** Avid Pro Tools Ultimate 2026.4.1+> **Last reviewed:** 2026-10-19 · **Last updated:** 2026-10-25 · **Change:** trivial (G-01: audio load and dropout meter; G-04: Clip Looping crossfade and loop count) · **Target:** Avid Pro Tools Ultimate 2026.4.1 Every known shortfall, one entry each, ranked by how much it stops a Pro Tools user from doing real work in SoundCraft. This is the work list: pick from the top. Rows marked **alpha blocker**@@ -46,6 +46,8 @@ When you close a gap, delete its entry (or shrink it) in the same PR, and bump t device, sample rate and device buffer size (kept with the preferences, per machine; `ui.audio_device` for agents); f32 and 16/32-bit integer devices; ASIO behind the opt-in `asio` build feature (verified on Windows with an SSL 2+ at 128 samples).+- **Done (2026-10-25):** Window › System Usage shows the audio callback's load (render time over+ buffer duration, average and peak) and counts late callbacks (dropouts); `ui.audio_load` for agents. - **Missing:** routing to physical channels (I/O Setup), one duplex stream on one clock (input and output are still separate cpal streams), latency reporting and record-offset compensation, ASIO in official builds (owner decision: Steinberg's SDK is proprietary or GPLv3), JACK, and@@ -226,6 +228,7 @@ missing. Out of scope unless the owner decides otherwise. | Date | Change | Summary | |---|---|---| | 2026-10-25 | trivial | G-04: Clip Looping (#1030) lacks the loop crossfade option, and Number of Loops stops at 512 while length and fill make up to 1000 loops |+| 2026-10-25 | trivial | G-01: Window › System Usage shows the audio callback load and late callbacks (dropouts); `ui.audio_load` | | 2026-10-24 | trivial | G-08: 27 processors + 3 instruments, as the plugin registry holds; G-20: the sampler shipped (#108), so it is no longer listed as missing | | 2026-10-19 | trivial | G-04: Float32 encoding retains finite headroom (#791); integer clipping and non-finite handling retained | | 2026-10-18 | trivial | G-04: mono device stereo/surround fold-down fixed and regression-tested; estimates unchanged |@@ -1,6 +1,6 @@ # Hardware parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-13 · **Change:** trivial (bounces of a bus fed by an aux compensate its latency (#246); previously minor: audio device, rate and buffer choice; ASIO feature) · **Target:** Avid Pro Tools Ultimate 2026.4.1+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-25 · **Change:** trivial (audio callback load and dropout meter; previously trivial: bounces of a bus fed by an aux compensate its latency (#246); previously minor: audio device, rate and buffer choice; ASIO feature) · **Target:** Avid Pro Tools Ultimate 2026.4.1 A DAW is judged by how it drives audio interfaces, MIDI gear, control surfaces and sync. This is SoundCraft's weakest dimension. Ours is measured from `crates/playback` (cpal) and the absence of@@ -21,6 +21,7 @@ Legend: ✅ works · 🟡 partial · ❌ missing. | Buffer size (32–2048) | ✅ | 🟡 device buffer when the driver allows it (not yet verified); mix in 512-sample blocks | 🟡 device buffer 16–8192 (verified 128 on ASIO, 256 on WASAPI); mix in 512-sample blocks | 🟡 (not yet verified) | 🟡 | ❌ | | Duplex stream on one clock | ✅ | ❌ separate input/output streams | ❌ | ❌ | ❌ | ❌ | | Hardware I/O mapping (I/O Setup to physical channels) | ✅ | 🟡 multichannel output in order; no per-channel map | 🟡 | 🟡 | 🟡 | ❌ |+| Audio load and dropout meter (System Usage) | ✅ | 🟡 (not yet verified) | 🟡 (not yet verified) | 🟡 (not yet verified) | 🟡 (not yet verified) | ❌ not measured | | Low-latency monitoring | ✅ | ❌ (`options.low_latency_monitoring` is a stored flag) | ❌ | ❌ | ❌ | ❌ | | Latency reporting and record offset compensation | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | | Sample rate follows the session; resample when it can't | ✅ (switches device) | 🟡 resamples (input pitch bug fixed, #28) | 🟡 | 🟡 | 🟡 | 🟡 |@@ -72,6 +73,7 @@ Legend: ✅ works · 🟡 partial · ❌ missing. | Date | Change | Summary | |---|---|---|+| 2026-10-25 | trivial | Audio callback load (average, peak, worst) and late callbacks (dropouts) in System Usage; `ui.audio_load` | | 2026-10-13 | trivial | bounces of a bus fed by an aux compensate its latency (#246) | | 2026-10-12 | minor | Audio device settings: system, output, input, sample rate and buffer size; opt-in ASIO | | 2026-10-10 | major | Created: audit of audio interface, MIDI, control-surface, sync and performance support against Pro Tools 2026.4.1 |