collective
Browse / github.com/storytold/filmcraft / patch
gpu, color: decode log curves on the GPU from the CPU's own table
gpu, color: decode log curves on the GPU from the CPU's own table Depends on #3213: this branch contains its commit. Only the last commit (1a1884fb) belongs to this PR; merge #3213 first. ## What With HDR sequences on the GPU, a log source (S-Log3, V-Log, C-Log, LogC, Apple Log) still decoded on the CPU: InputTransform::gpu_params refused Curve::Log and the camera-gamut compression stage, so camera footage - what HDR grading is mostly done on - kept a scalar f32 decode per frame. - color: GpuCurve::Lut, and InputTransform::gpu_lut() hands the shader the decode table the CPU interpolates (a log curve is a table on the CPU too: 8192 entries over -0.1..1.1).
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- zsoli69 (imported from pull request #3219, open)
- Base
10d626ccc4888e0f020f9ca314c2bc9148ded4f4- Size
- +553 −42 in 12 files
- Review
- pending
- Shared
- 2026-10-28
- Id
4dca431d9569b22f59908df6
How it went
No other operator has reported using it yet.
Diff
raw · crates/color/src/lib.rs crates/color/src/transform.rs crates/golden/tests/golden.rs crates/gpu/src/composite.wgsl crates/gpu/src/fx_tests.rs crates/gpu/src/lib.rs crates/gpu/src/tests.rs crates/render/src/colorman.rs crates/render/src/plan.rs crates/ui-egui/src/app/gpu.rs docs/architecture.md docs/testing.md
@@ -20,7 +20,7 @@ pub use grade::GradeSpace; pub use log::LogCurve; pub use lut::{Lut, Lut1d, Lut3d, LutFormat}; pub use spaces::{ColorPipeline, ColorSpace, Curve, Gamut, WorkingSpace};-pub use transform::{DecodeTable, GpuCurve, GpuInputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap};+pub use transform::{DecodeTable, GpuCurve, GpuInputParams, GpuOutputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap}; /// YUV ↔ RGB matrix coefficients. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]@@ -260,38 +260,49 @@ impl InputTransform { self.apply(rgb.map(|v| self.table.lookup(v))) } - /// The transform as plain numbers for a shader, which evaluates the curve and the BT.2390- /// EETF directly instead of sampling the tables. `None` for what a shader does not do: log- /// curves and the camera-gamut compression.+ /// The transform as plain numbers for a shader, which evaluates the curve, the HLG OOTF, the+ /// gamut matrix, the BT.2390 EETF and the gamut compression stage by stage. The one curve it+ /// does not evaluate is a log one: for that it samples the CPU's own decode table+ /// ([`gpu_lut`](Self::gpu_lut)) through [`lut`](GpuInputParams::lut). pub fn gpu_params(&self) -> Option<GpuInputParams> { let curve = match self.source.curve() { Curve::Sdr | Curve::Srgb => GpuCurve::Sdr, Curve::Linear => GpuCurve::Linear, Curve::Pq => GpuCurve::Pq, Curve::Hlg => GpuCurve::Hlg,- Curve::Log(_) => return None,+ Curve::Log(_) => GpuCurve::Lut, };- if self.compress {- return None;- }+ let lut = (curve == GpuCurve::Lut).then_some((self.table.lo, self.table.hi, self.table.values.len() as u32)); Some(GpuInputParams { curve, hlg: self.hlg, matrix: (!self.identity_matrix).then_some(self.matrix), tone: self.tone.as_ref().map(|t| (t.src_peak_nits as f32, t.dst_peak_nits as f32)), desaturate: self.desaturate,+ lut,+ compress: self.compress, }) }++ /// The per-channel decode table a shader samples for a log curve ([`GpuCurve::Lut`]): the same+ /// numbers [`convert`](Self::convert) interpolates, so the two agree up to the shader's own+ /// interpolation. None for a curve the shader evaluates itself.+ pub fn gpu_lut(&self) -> Option<&[f32]> {+ matches!(self.source.curve(), Curve::Log(_)).then_some(self.table.values.as_slice())+ } } /// How a shader decodes a channel ([`decode_channel`] without the table): SDR with the sRGB curve,-/// linear, PQ in working units (cd/m² over [`REFERENCE_WHITE_NITS`]), HLG to scene light.+/// linear, PQ in working units (cd/m² over [`REFERENCE_WHITE_NITS`]), HLG to scene light, or a log+/// curve, which the CPU also decodes through a table: the shader samples the same table from a+/// lookup texture ([`InputTransform::gpu_lut`]). #[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum GpuCurve { Sdr, Linear, Pq, Hlg,+ Lut, } /// [`InputTransform`] for a shader ([`InputTransform::gpu_params`]), stage by stage as@@ -307,6 +318,11 @@ pub struct GpuInputParams { pub tone: Option<(f32, f32)>, /// Luma weights to desaturate into the working gamut with ([`desaturate_into_gamut`]). pub desaturate: Option<[f32; 3]>,+ /// A log curve: the decode table's domain and length (`lo`, `hi`, entries) for the lookup+ /// texture [`InputTransform::gpu_lut`] hands over. None for a curve the shader evaluates itself.+ pub lut: Option<(f32, f32, u32)>,+ /// Camera-gamut compression ([`gamut_compress`]) after the matrix.+ pub compress: bool, } /// Whether gamut `a` lies inside gamut `b` (so no compression is needed going a → b).@@ -389,6 +405,37 @@ impl OutputTransform { curve => l.map(|v| encode_channel(v as f64, curve, Range::Full).clamp(0.0, 1.0) as f32), } }++ /// The transform as plain numbers for a shader, which runs [`apply`](Self::apply)'s stages+ /// (BT.2390 tone mapping, the working → target matrix, desaturation into the target gamut) and+ /// encodes the target's SDR curve itself. `None` for a target the shader does not encode (an+ /// HDR export target), which stays on the CPU.+ pub fn gpu_params(&self) -> Option<GpuOutputParams> {+ if self.target.curve() != Curve::Sdr {+ return None;+ }+ Some(GpuOutputParams {+ target: self.target,+ matrix: (!self.identity_matrix).then_some(self.matrix),+ tone: self.tone.as_ref().map(|t| (t.src_peak_nits as f32, t.dst_peak_nits as f32)),+ compress: self.compress,+ })+ }+}++/// [`OutputTransform`] as plain numbers for a shader ([`OutputTransform::gpu_params`]): the stages+/// after the compositor — tone mapping, working → target matrix, gamut desaturation, in the order+/// [`OutputTransform::apply`] runs them — with the target's SDR curve left to the shader's resolve.+#[derive(Clone, Copy, Debug, PartialEq)]+pub struct GpuOutputParams {+ /// The target colour space (SDR: the shader's resolve is the sRGB curve).+ pub target: ColorSpace,+ /// Working → target primaries (row-major), when they differ.+ pub matrix: Option<[[f32; 3]; 3]>,+ /// BT.2390 tone mapping (source peak, target peak) in cd/m², on max(R,G,B) as a ratio.+ pub tone: Option<(f32, f32)>,+ /// Target luma weights to desaturate into the target gamut with ([`desaturate_into_gamut`]).+ pub compress: Option<[f32; 3]>, } /// Convenience: a log colour space → display Rec. 709 (tone mapped), used for the built-in@@ -511,22 +558,40 @@ mod tests { } #[test]- fn gpu_params_cover_display_referred_media_only() {- // HLG / PQ / BT.2020 into Rec. 709: every stage the shader has (#965)- let p = |cs| InputTransform::new(cs, Range::Limited, &ColorPipeline::REC709, None).gpu_params();- let hlg = p(ColorSpace::Rec2100Hlg).unwrap();+ fn gpu_params_cover_every_curve_and_stage() {+ // HLG / PQ / BT.2020 into Rec. 709: every stage the shader evaluates itself (#965)+ let p = |cs| InputTransform::new(cs, Range::Limited, &ColorPipeline::REC709, None);+ let hlg = p(ColorSpace::Rec2100Hlg).gpu_params().unwrap(); assert_eq!(hlg.curve, GpuCurve::Hlg); assert!(hlg.hlg.is_some() && hlg.matrix.is_some() && hlg.desaturate.is_some()); assert_eq!(hlg.tone, Some((HDR_PEAK_NITS as f32, REFERENCE_WHITE_NITS as f32)));- let pq = InputTransform::new(ColorSpace::Rec2100Pq, Range::Limited, &ColorPipeline::REC709, Some(4000.0)).gpu_params().unwrap();- assert_eq!((pq.curve, pq.hlg, pq.tone), (GpuCurve::Pq, None, Some((4000.0, REFERENCE_WHITE_NITS as f32))));- let sdr2020 = p(ColorSpace::Rec2020).unwrap();+ assert!((hlg.lut, hlg.compress) == (None, false));+ let pq = p(ColorSpace::Rec2100Pq).gpu_params().unwrap();+ assert_eq!((pq.curve, pq.hlg, pq.tone), (GpuCurve::Pq, None, Some((HDR_PEAK_NITS as f32, REFERENCE_WHITE_NITS as f32))));+ let sdr2020 = p(ColorSpace::Rec2020).gpu_params().unwrap(); assert_eq!((sdr2020.curve, sdr2020.tone), (GpuCurve::Sdr, None)); assert!(sdr2020.matrix.is_some() && sdr2020.desaturate.is_some());- // log curves and camera-gamut compression stay on the CPU+ // log curves: the shader samples the CPU's own table, and a camera gamut's compression+ // runs there too (the stage at the end of `InputTransform::apply`) for cs in ColorSpace::ALL.into_iter().filter(|c| c.is_log()) {- assert_eq!(p(cs), None, "{cs:?}");+ let t = p(cs);+ let g = t.gpu_params().unwrap_or_else(|| panic!("{cs:?} has no shader form"));+ assert_eq!(g.curve, GpuCurve::Lut, "{cs:?}");+ let (lo, hi, n) = g.lut.expect("a log curve has a table");+ let lut = t.gpu_lut().expect("a log curve has a table");+ assert_eq!((lo, hi, n as usize), (-0.1, 1.1, lut.len()), "{cs:?}");+ assert!(g.compress, "{cs:?}: a camera gamut wider than 709 gets compressed");+ // the sampled table is the transform's own: the shader and `convert` agree+ let signal = 0.5f32;+ let i = ((signal - lo) / (hi - lo)).clamp(0.0, 1.0) * (lut.len() - 1) as f32;+ let k = (i as usize).min(lut.len() - 2);+ let f = i - k as f32;+ let sampled = lut[k] + (lut[k + 1] - lut[k]) * f;+ assert!((t.table.lookup(signal) - sampled).abs() < 1e-6, "{cs:?}"); }+ // the shader form of a log source is what `gpu_input` needs to plan it on the GPU+ assert!(p(ColorSpace::SLog3SGamut3Cine).gpu_lut().is_some());+ assert!(p(ColorSpace::Rec709).gpu_lut().is_none()); } #[test]@@ -550,4 +615,20 @@ mod tests { // plain pipeline is an identity assert!(OutputTransform::display(&ColorPipeline::REC709).is_identity()); }++ #[test]+ fn output_gpu_params_cover_the_display_transform() {+ // a plain pipeline: the shader's sRGB resolve is the whole transform+ let plain = OutputTransform::display(&ColorPipeline::REC709).gpu_params().unwrap();+ assert!(plain.matrix.is_none() && plain.tone.is_none() && plain.compress.is_none());+ // an HDR working space on a SDR monitor: tone mapping, the working → 709 matrix and the+ // gamut compression, the stages the resolve pass runs+ let hdr = OutputTransform::display(&self::hdr(WorkingSpace::Rec2100Hlg)).gpu_params().unwrap();+ assert_eq!(hdr.target, ColorSpace::Rec709);+ assert!(hdr.matrix.is_some() && hdr.compress.is_some());+ assert_eq!(hdr.tone, Some((HDR_PEAK_NITS as f32, REFERENCE_WHITE_NITS as f32)));+ // an HDR export target has no shader encode yet: it stays on the CPU+ let export = OutputTransform::new(&self::hdr(WorkingSpace::Rec2100Hlg), ColorSpace::Rec2100Pq);+ assert!(export.gpu_params().is_none());+ } }@@ -775,6 +775,9 @@ fn gpu_matches_cpu_on_golden_scenes() { let cpu = render_cpu(&s); let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources); let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" };+ // the resolve pass runs the sequence's display transform, as the app sets it (an HDR+ // working space composites in working linear and is tone mapped there)+ c.set_display_transform(display_for(&s, &plan)); c.composite(&plan); let (gw, gh, mut px) = c.read_output().expect("GPU readback"); px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);@@ -793,6 +796,44 @@ fn gpu_matches_cpu_on_golden_scenes() { assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n")); } +/// The resolve pass's display transform for `plan` of scene `s`, chosen as the app chooses it+/// (`gpu_present`): none for a CPU-rendered frame, which is already in monitor space.+fn display_for(s: &Scene, plan: &filmcraft_render::plan::FramePlan) -> Option<filmcraft_color::GpuOutputParams> {+ let pipe = s.project.sequence(s.seq).map(|q| q.settings.color);+ filmcraft_gpu::sequence_display(plan, pipe.and_then(|p| filmcraft_render::colorman::display_transform(&p).gpu_params()))+}++/// A frame of an HDR (Rec. 2100 PQ) sequence that the compositor cannot finish (a Wipe) comes+/// back as one CPU image, already converted for the monitor: the GPU shows it as it is, without+/// tone mapping it a second time.+#[test]+fn gpu_shows_an_hdr_cpu_frame_as_rendered() {+ let Some((dev, q)) = device("gpu hdr cpu frame", false) else {+ return;+ };+ let mut b = Builder::new(1);+ b.p.sequence_mut(b.seq).unwrap().settings.color =+ filmcraft_color::ColorPipeline { working: filmcraft_color::WorkingSpace::Rec2100Pq, ..filmcraft_color::ColorPipeline::REC709 };+ let pq = ColorInfo { transfer: Transfer::Pq, primaries: Primaries::Bt2020, ..ColorInfo::SRGB_FULL };+ let a = b.encoded(DemoScene::CityNight, ColorSpace::Rec2100Pq, 6.0, Some(pq), None);+ let z = b.demo(DemoScene::Aurora);+ let ca = b.place(0, a, 0, 24);+ let cz = b.place(0, z, 24, 24);+ b.transition(0, "wipe", ca, cz, 24, 12);+ let s = b.at(24);+ let t = RATE.tick_of(s.frame);+ let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources);+ assert!(matches!(plan, filmcraft_render::plan::FramePlan::Image(_)), "a Wipe is rendered on the CPU");+ let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);+ c.set_display_transform(display_for(&s, &plan));+ c.composite(&plan);+ let (gw, gh, mut px) = c.read_output().expect("GPU readback");+ px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);+ let d = diff(&render_cpu(&s), &Rgba8::new(gw, gh, px)).unwrap();+ let mean_rgb = d.mean_abs * 4.0 / 3.0;+ assert!(d.p99 <= 6 && mean_rgb < 1.5, "HDR CPU frame on the GPU differs from the CPU: {d}");+}+ /// Per-pixel max RGB difference (99th percentile, mean) over the pixels at least 1.5 px from every /// plan layer's quad outline: the CPU resampler fades a layer's edge over one more pixel than the /// GPU rasterises, and with blend modes those edge pixels are a visible share of a 320×180 frame.@@ -22,6 +22,7 @@ struct U { q3: vec4<f32>, // gamut matrix row 1, tone-mapping source peak cd/m² q4: vec4<f32>, // gamut matrix row 2, tone-mapping target peak cd/m² q5: vec4<f32>, // desaturation luma weights, desaturation on+ q6: vec4<f32>, // log decode table: lo, hi, entries, camera-gamut compression on }; @group(0) @binding(0) var<uniform> u: U;@@ -32,6 +33,10 @@ struct U { @group(0) @binding(4) var backdrop: texture_2d<f32>; // The alpha plane of a Y'CbCr source (ProRes 4444, ...); a dummy texture when `u.p3.y` is 0. @group(0) @binding(5) var tex3: texture_2d<f32>;+// A log source's decode table (`filmcraft_color::GpuCurve::Lut`), `u.q6.z` entries in rows of+// `LUT_ROW`: the same numbers `DecodeTable::lookup` interpolates, so the shader needs no log curve+// formula of its own.+@group(0) @binding(6) var lut_tex: texture_2d<f32>; struct VOut { @builtin(position) pos: vec4<f32>,@@ -122,16 +127,56 @@ fn eetf(nits: f32, s: f32, d: f32) -> f32 { return pq1(e2 * smax) * 10000.0; } +// `DecodeTable::lookup`: the table's domain (`q6.xy`) mapped to 0..entries-1 and interpolated.+fn lut_lookup(v: f32) -> f32 {+ let entries = u32(u.q6.z);+ if entries < 2u { return v; }+ let n = f32(entries - 1u);+ let p = clamp((v - u.q6.x) / max(u.q6.y - u.q6.x, 1e-9), 0.0, 1.0) * n;+ let i = min(u32(p), entries - 2u);+ let f = p - f32(i);+ return mix(lut_entry(i), lut_entry(i + 1u), f);+}++// Entry `i` of the decode table (`GpuCompositor::lut_texture` lays it out in rows of `LUT_ROW`).+const LUT_ROW: u32 = 256u;+fn lut_entry(i: u32) -> f32 {+ return textureLoad(lut_tex, vec2<i32>(i32(i % LUT_ROW), i32(i / LUT_ROW)), 0).r;+}++// `gamut_compress` (crates/color): a camera gamut's out-of-gamut colour softened towards the+// achromatic axis, in-gamut colour untouched.+fn compress_channel(v: f32, ach: f32, s: f32) -> f32 {+ let th = 0.8;+ let d = (ach - v) / ach;+ if d < th { return v; }+ let x = (d - th) / s;+ let dc = th + s * x / pow(1.0 + pow(x, 1.2), 1.0 / 1.2);+ return ach - dc * ach;+}++fn gamut_compress(c: vec3<f32>) -> vec3<f32> {+ let ach = max(max(c.r, c.g), c.b);+ if ach <= 1e-9 { return max(c, vec3(0.0)); }+ let s = (1.25 - 0.8) / pow(pow((1.25 - 0.8) / (1.0 - 0.8), 1.2) - 1.0, 1.0 / 1.2);+ return vec3(compress_channel(c.r, ach, s), compress_channel(c.g, ach, s), compress_channel(c.b, ach, s));+}+ // Signal → working linear: `InputTransform::convert` (curve, HLG OOTF, gamut matrix, tone-// mapping, desaturation into the working gamut). The CPU's decode tables clamp the signal to 0..1.+// mapping, gamut compression, desaturation into the working gamut). The CPU's decode tables clamp+// the signal to 0..1; a log curve is one of those tables (`lut_lookup`). fn managed(rgb: vec3<f32>) -> vec3<f32> {- let v = clamp(rgb, vec3(0.0), vec3(1.0)); let curve = u32(u.q0.y); let white = u.q0.w;+ // The CPU clamps inside each decode table: 0..1 for the curves it evaluates with a table of+ // its own, and the log table's own domain (`q6.xy`, -0.1..1.1) for the one sampled here.+ var v = clamp(rgb, vec3(0.0), vec3(1.0));+ if curve == 4u { v = clamp(rgb, vec3(u.q6.x), vec3(u.q6.y)); } var c = v; if curve == 0u { c = srgb_to_linear(v); } if curve == 2u { c = vec3(pq1(v.r), pq1(v.g), pq1(v.b)) * (10000.0 / white); } if curve == 3u { c = hlg_inv(v); }+ if curve == 4u { c = vec3(lut_lookup(v.r), lut_lookup(v.g), lut_lookup(v.b)); } if u.q0.z > 0.0 { let ys = max(dot(u.q1.xyz, c), 0.0); var g = 0.0;@@ -154,6 +199,7 @@ fn managed(rgb: vec3<f32>) -> vec3<f32> { c = c * k; } }+ if u.q6.w > 0.5 { c = gamut_compress(c); } if u.q5.w > 0.5 { let mn = min(min(c.r, c.g), c.b); if mn < 0.0 {@@ -394,6 +440,21 @@ fn fs_blend(in: VOut) -> @location(0) vec4<f32> { @group(0) @binding(0) var accum: texture_2d<f32>; +// The resolve pass's display transform (`filmcraft_color::OutputTransform`, #965): the sequence's+// working linear → monitor. Same stages as `OutputTransform::apply`, in the same order: BT.2390+// tone mapping (from an HDR working space), the working → target gamut matrix, then desaturation+// into the target gamut. `a.x` is 0 for a plain sequence (the accumulator is already monitor+// space) and the sRGB encode below is the whole transform.+struct O {+ a: vec4<f32>, // on, target curve (0 sdr), tone mapping on, matrix on+ b: vec4<f32>, // desaturation on, tone source peak cd/m², tone target peak cd/m², reference white cd/m²+ m0: vec4<f32>, // matrix row 0, desaturation luma weight 0+ m1: vec4<f32>, // matrix row 1, desaturation luma weight 1+ m2: vec4<f32>, // matrix row 2, desaturation luma weight 2+};++@group(0) @binding(1) var<uniform> disp: O;+ @vertex fn vs_full(@builtin(vertex_index) vi: u32) -> @builtin(position) vec4<f32> { let x = f32((vi << 1u) & 2u) * 2.0 - 1.0;@@ -401,9 +462,44 @@ fn vs_full(@builtin(vertex_index) vi: u32) -> @builtin(position) vec4<f32> { return vec4(x, y, 0.0, 1.0); } +// Working linear → monitor linear.+fn display(rgb: vec3<f32>) -> vec3<f32> {+ if disp.a.x < 0.5 { return rgb; }+ var c = rgb;+ if disp.a.z > 0.5 {+ // `ToneMap::apply`: one scale from max(R,G,B), the EETF sampled in the PQ domain+ let m = max(max(c.r, c.g), c.b);+ if m > 0.0 {+ let nits = m * disp.b.w;+ var k = 1.0;+ if nits >= disp.b.y {+ // above the source peak: clip to the target peak+ k = disp.b.z / disp.b.w / m;+ } else if nits > 1e-6 {+ k = eetf(nits, disp.b.y, disp.b.z) / nits;+ }+ c = c * k;+ }+ }+ if disp.a.w > 0.5 { c = vec3(dot(disp.m0.xyz, c), dot(disp.m1.xyz, c), dot(disp.m2.xyz, c)); }+ if disp.b.x > 0.5 {+ // `desaturate_into_gamut`: out-of-gamut colours towards their luminance, in-gamut exactly+ let mn = min(min(c.r, c.g), c.b);+ if mn < 0.0 {+ let y = dot(vec3(disp.m0.w, disp.m1.w, disp.m2.w), c);+ if y <= 0.0 {+ c = vec3(0.0);+ } else {+ c = max(y + (c - y) * (y / (y - mn)), vec3(0.0));+ }+ }+ }+ return c;+}+ @fragment fn fs_full(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {- let c = clamp(textureLoad(accum, vec2<i32>(pos.xy), 0).rgb, vec3(0.0), vec3(1.0));+ let c = clamp(display(textureLoad(accum, vec2<i32>(pos.xy), 0).rgb), vec3(0.0), vec3(1.0)); let lo = c * 12.92; let hi = 1.055 * pow(c, vec3(1.0 / 2.4)) - 0.055; return vec4(select(hi, lo, c <= vec3(0.0031308)), 1.0);@@ -574,6 +574,50 @@ fn lumetri_unsupported_sections_fall_back() { assert!(op.as_ref().is_some_and(|o| !o.gpu_ok())); } +/// A log source with effects: the effect stage draws it onto the working image through the CPU's+/// decode table (binding 6) as a plain layer does; with the dummy bound there an S-Log3 clip with+/// any GPU effect came out black.+#[test]+fn log_source_with_effects_matches_cpu() {+ use filmcraft_color::{ColorPipeline, ColorSpace, InputTransform, Range};+ let Some((dev, q)) = device() else {+ eprintln!("no GPU adapter; skipping");+ return;+ };+ let pipe = ColorPipeline { working: filmcraft_color::WorkingSpace::Rec2100Hlg, wide_gamut: true, auto_tone_map: true };+ let t = Arc::new(InputTransform::new(ColorSpace::SLog3SGamut3Cine, Range::Limited, &pipe, None));+ let (w, h) = (256usize, 4usize);+ // a luma ramp over the whole limited range, one chroma value throughout (see+ // `gpu_matches_cpu_for_a_log_source`)+ let y: Vec<u8> = (0..w * h).map(|i| (16 + (i % w) * 219 / w) as u8).collect();+ let frame = Arc::new(VideoFrame {+ width: w as u32,+ height: h as u32,+ data: PixelData::Yuv8 { planes: [Arc::new(y), Arc::new(vec![100; w * h]), Arc::new(vec![150; w * h])], chroma: Chroma::C444, alpha: None },+ color: filmcraft_color::ColorInfo { range: Range::Limited, ..filmcraft_color::ColorInfo::REC709 },+ par: (1, 1),+ pts: Default::default(),+ });+ let mut cxh = cx(Tick::ZERO, 1.0);+ cxh.working = pipe.working;+ let e = effect("brightness_contrast", &[("brightness", fl(-10.0)), ("contrast", fl(20.0))]);+ let op = FxOp::eval(&e, &cxh, w, h).expect("brightness / contrast op");+ assert!(op.gpu_ok());+ let fx = Some(Arc::new(LayerFx { size: (w as u32, h as u32), decimation: 1, ops: vec![op] }));+ let plan = FramePlan::Layers {+ width: w,+ height: h,+ layers: vec![PlanLayer { frame, matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx, input: Some(t) }],+ };+ let cpu = execute_cpu(&plan).over_black_rgba8();+ let mut c = GpuCompositor::new(&dev, &q);+ c.composite(&plan);+ let (_, _, gpu) = c.read_output().expect("readback");+ let (p99, mean, max) = stats8(&cpu, &gpu, &vec![true; w * h]);+ eprintln!("log source with effects (S-Log3 -> HLG): p99 {p99}, mean {mean:.2}, max {max}");+ assert!(p99 <= 6 && mean < 1.5, "p99 {p99}, mean {mean}, max {max}");+}+ /// Layers with effect chains in a composited plan — YUV and RGBA sources, a decimated working /// image, rotated / scaled placement, partial opacity and alpha, blend modes — match the CPU plan /// executor.@@ -710,6 +754,44 @@ fn layers_with_effects_fall_back_to_the_cpu_without_a_stage() { } } +/// A colour-managed layer (PQ media) with effects comes out the same with and without the GPU+/// effect stage: the CPU fallback decodes it through the layer's input transform too.+#[test]+fn cpu_effect_fallback_keeps_the_input_transform() {+ use filmcraft_color::{ColorInfo, ColorPipeline, ColorSpace, InputTransform, Primaries};+ let Some((dev, q)) = device() else {+ eprintln!("no GPU adapter; skipping");+ return;+ };+ let (w, h) = (256u32, 4u32);+ let y: Vec<u16> = (0..h).flat_map(|_| (0..w).map(|x| (64 + x * 876 / 255) as u16)).collect();+ let n = (w * h) as usize;+ let data = PixelData::Yuv16 { planes: [Arc::new(y), Arc::new(vec![480; n]), Arc::new(vec![560; n])], chroma: Chroma::C444, bits: 10, alpha: None };+ let color = ColorInfo { matrix: Matrix::Bt2020Ncl, transfer: Transfer::Pq, primaries: Primaries::Bt2020, range: Range::Limited };+ let frame = Arc::new(VideoFrame { width: w, height: h, data, color, par: (1, 1), pts: Default::default() });+ let t = Arc::new(InputTransform::new(ColorSpace::from_info(&color), color.range, &ColorPipeline::REC709, None));+ let e = effect("brightness_contrast", &[("brightness", fl(-10.0)), ("contrast", fl(20.0))]);+ let op = FxOp::eval(&e, &cx(Tick::ZERO, 1.0), w as usize, h as usize).expect("brightness / contrast op");+ let layer = PlanLayer {+ fx: Some(Arc::new(LayerFx { size: (w, h), decimation: 1, ops: vec![op] })),+ input: Some(t),+ ..PlanLayer::new(frame, Affine::IDENTITY, 1.0, Blend::Normal)+ };+ let plan = FramePlan::Layers { width: w as usize, height: h as usize, layers: vec![layer] };+ let cpu = execute_cpu(&plan).over_black_rgba8();+ for stage in [true, false] {+ let mut c = GpuCompositor::new(&dev, &q);+ if !stage {+ c.fx = None;+ }+ c.composite(&plan);+ let (_, _, gpu) = c.read_output().expect("readback");+ let (p99, mean, max) = stats8(&cpu, &gpu, &vec![true; n]);+ eprintln!("PQ layer with effects, effect stage {stage}: p99 {p99}, mean {mean:.2}, max {max}");+ assert!(p99 <= 6 && mean < 1.5, "stage {stage}: p99 {p99}, mean {mean}, max {max}");+ }+}+ /// The half-float upload is lossless for this input, so a 1:1 effect source draw must be too. /// Odd dimensions expose interpolation drift that can invent opacity before any effect runs. #[test]@@ -84,6 +84,8 @@ pub struct GpuCompositor { accum_readback_buf: Option<(wgpu::Buffer, u64)>, uploads: HashMap<(usize, u32, u32), Uploaded>, clock: u64,+ /// The resolve pass's display transform (`set_display_transform`), as plain numbers.+ display: [f32; DISPLAY_FLOATS], dummy: wgpu::TextureView, /// The effect stage (None on devices without compute shaders or storage textures: layers /// with effects are then rendered on the CPU here).@@ -253,6 +255,16 @@ pub fn prepare_frame(f: &VideoFrame) -> Option<Prepared> { } } +/// The display transform to resolve a *sequence* plan with: `display` (the sequence's,+/// `colorman::display_transform(..).gpu_params()`) for a plan composited in its working space,+/// none for a [`FramePlan::Image`]. That image is `render_sequence`'s output, already converted+/// for the monitor, and converting it again would tone map and gamut map it twice. (The CPU+/// images the compositor makes itself, for a transition it cannot mix, are in working space and+/// keep the transform: they are drawn inside a working-space plan.)+pub fn sequence_display(plan: &FramePlan, display: Option<filmcraft_color::GpuOutputParams>) -> Option<filmcraft_color::GpuOutputParams> {+ if matches!(plan, FramePlan::Image(_)) { None } else { display }+}+ /// Convert every layer of a plan for upload. Thread-safe and GPU-free: run it on a worker. pub fn prepare(plan: &FramePlan) -> PreparedPlan { match plan {@@ -294,6 +306,7 @@ impl GpuCompositor { tex_entry(2), tex_entry(3), tex_entry(5),+ tex_entry(6), ], }); let blend_bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {@@ -310,11 +323,21 @@ impl GpuCompositor { tex_entry(3), tex_entry(4), tex_entry(5),+ tex_entry(6), ], }); let final_bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor { label: Some("final"),- entries: &[wgpu::BindGroupLayoutEntry { binding: 0, ..tex_entry(0) }],+ entries: &[+ wgpu::BindGroupLayoutEntry { binding: 0, ..tex_entry(0) },+ // the resolve pass's display transform (`set_display_transform`)+ wgpu::BindGroupLayoutEntry {+ binding: 1,+ visibility: wgpu::ShaderStages::FRAGMENT,+ ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },+ count: None,+ },+ ], }); let pl = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor { label: Some("layer"), bind_group_layouts: &[Some(&layer_bgl)], immediate_size: 0 });@@ -420,6 +443,7 @@ impl GpuCompositor { accum_readback_buf: None, uploads: HashMap::new(), clock: 0,+ display: [0.0; DISPLAY_FLOATS], dummy, fx, transition: transition::PushStage::new(device),@@ -477,6 +501,36 @@ impl GpuCompositor { t.create_view(&Default::default()) } + /// Upload the input transform's log decode table as a 1D texture (`GpuCurve::Lut`). A log curve+ /// is a table on the CPU too, so the shader samples the same numbers instead of a second+ /// formula: 8192 entries, 32 KB, re-made per layer per frame (the picture's own upload is three+ /// orders of magnitude larger). The entries are laid out in rows of [`LUT_ROW`] (entry `i` at+ /// `(i % LUT_ROW, i / LUT_ROW)`), so the table fits the smallest 2D texture limit (WebGL2's+ /// 2048) instead of being one 8192-wide row.+ fn lut_texture(&mut self, values: &[f32]) -> wgpu::TextureView {+ let rows = u32::try_from(values.len().div_ceil(LUT_ROW as usize)).unwrap_or(1).max(1);+ let mut bytes: Vec<u8> = values.iter().flat_map(|v| v.to_le_bytes()).collect();+ bytes.resize(LUT_ROW as usize * rows as usize * 4, 0);+ let t = self.device.create_texture(&wgpu::TextureDescriptor {+ label: Some("input-lut"),+ size: wgpu::Extent3d { width: LUT_ROW, height: rows, depth_or_array_layers: 1 },+ mip_level_count: 1,+ sample_count: 1,+ dimension: wgpu::TextureDimension::D2,+ format: wgpu::TextureFormat::R32Float,+ usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,+ view_formats: &[],+ });+ self.queue.write_texture(+ wgpu::TexelCopyTextureInfo { texture: &t, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },+ &bytes,+ wgpu::TexelCopyBufferLayout { offset: 0, bytes_per_row: Some(4 * LUT_ROW), rows_per_image: Some(rows) },+ wgpu::Extent3d { width: LUT_ROW, height: rows, depth_or_array_layers: 1 },+ );+ self.uploaded_bytes += bytes.len() as u64;+ t.create_view(&Default::default())+ }+ /// Upload (or reuse) the textures of a frame, using `prep` when it was converted beforehand. fn upload(&mut self, f: &VideoFrame, prep: Option<&Prepared>) -> (usize, u32, u32) { let id = match &f.data {@@ -628,17 +682,20 @@ impl GpuCompositor { let taps = if footprint > 1.25 { footprint.ceil().min(4.0) } else { 1.0 }; let _ = Matrix::Bt709; // colour management (q0..q5), stage by stage as `InputTransform::apply`- let q = src.input.map_or([0.0; 24], |p| {+ let q = src.input.map_or([0.0; 28], |p| { let curve = match p.curve { filmcraft_color::GpuCurve::Sdr => 0.0, filmcraft_color::GpuCurve::Linear => 1.0, filmcraft_color::GpuCurve::Pq => 2.0, filmcraft_color::GpuCurve::Hlg => 3.0,+ // a log curve: the shader samples the CPU's decode table (binding 6)+ filmcraft_color::GpuCurve::Lut => 4.0, }; let (hl, hk) = p.hlg.unwrap_or(([0.0; 3], 0.0)); let mx = p.matrix.unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]); let (sp, dp) = p.tone.unwrap_or((0.0, 0.0)); let dl = p.desaturate.unwrap_or([0.0; 3]);+ let (lut_lo, lut_hi, lut_n) = p.lut.unwrap_or((0.0, 1.0, 0)); let on = |b: bool| if b { 1.0 } else { 0.0 }; #[rustfmt::skip] let q = [@@ -648,6 +705,7 @@ impl GpuCompositor { mx[1][0], mx[1][1], mx[1][2], sp, mx[2][0], mx[2][1], mx[2][2], dp, dl[0], dl[1], dl[2], on(p.desaturate.is_some()),+ lut_lo, lut_hi, lut_n as f32, on(p.compress), ]; q });@@ -699,11 +757,58 @@ impl GpuCompositor { buf } + /// The resolve pass's display transform uniform ([`set_display_transform`](Self::set_display_transform)).+ fn display_buffer(&self) -> wgpu::Buffer {+ let bytes: Vec<u8> = self.display.iter().flat_map(|v| v.to_le_bytes()).collect();+ let buf = self.device.create_buffer(&wgpu::BufferDescriptor {+ label: Some("display-u"),+ size: bytes.len() as u64,+ usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,+ mapped_at_creation: false,+ });+ self.queue.write_buffer(&buf, 0, &bytes);+ buf+ }+ /// Composite a plan; returns the output view (sRGB, over black) and its size. pub fn composite(&mut self, plan: &FramePlan) -> (wgpu::TextureView, (u32, u32)) { self.composite_prepared(plan, None) } + /// Set the resolve pass's display transform (the sequence's working space → monitor,+ /// `filmcraft_color::OutputTransform::gpu_params`). Layers are composited in the sequence's+ /// working linear space, so an HDR / wide-gamut sequence needs it to come out right; `None`+ /// (the default) leaves the accumulator as the monitor's own space, as for a plain sequence.+ /// Only the resolve pass uses it: a caller that reads the working-linear accumulator back+ /// (the export path) is unaffected.+ pub fn set_display_transform(&mut self, p: Option<filmcraft_color::GpuOutputParams>) {+ let Some(p) = p else {+ self.display = [0.0; DISPLAY_FLOATS];+ return;+ };+ let mx = p.matrix.unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]);+ let (sp, dp) = p.tone.unwrap_or((0.0, 0.0));+ let dl = p.compress.unwrap_or([0.0; 3]);+ let on = |b: bool| if b { 1.0 } else { 0.0 };+ let curve = match p.target.curve() {+ filmcraft_color::Curve::Sdr | filmcraft_color::Curve::Srgb => 0.0,+ filmcraft_color::Curve::Linear => 1.0,+ filmcraft_color::Curve::Pq => 2.0,+ filmcraft_color::Curve::Hlg => 3.0,+ // not reachable: `gpu_params` refuses a target the shader cannot encode+ filmcraft_color::Curve::Log(_) => 0.0,+ };+ #[rustfmt::skip]+ let d = [+ 1.0, curve, on(p.tone.is_some()), on(p.matrix.is_some()),+ on(p.compress.is_some()), sp, dp, filmcraft_color::REFERENCE_WHITE_NITS as f32,+ mx[0][0], mx[0][1], mx[0][2], dl[0],+ mx[1][0], mx[1][1], mx[1][2], dl[1],+ mx[2][0], mx[2][1], mx[2][2], dl[2],+ ];+ self.display = d;+ }+ /// [`composite`](Self::composite) with texel conversions already done by [`prepare`] (the /// result is identical; only the upload work on this thread differs). pub fn composite_prepared(&mut self, plan: &FramePlan, prep: Option<&PreparedPlan>) -> (wgpu::TextureView, (u32, u32)) {@@ -749,7 +854,9 @@ impl GpuCompositor { if self.fx.is_some() && lfx.size.0.max(lfx.size.1) <= max_side { return None; }- let img = filmcraft_render::plan::effect_image(&l.frame, lfx);+ // decoded through the layer's input transform, as `execute_cpu` does; the image is+ // then working-space linear, so the new layer has none+ let img = filmcraft_render::plan::effect_image_with(&l.frame, lfx, l.input.as_deref()); Some(PlanLayer::new(Arc::new(VideoFrame::rgba_f32(img.w as u32, img.h as u32, img.px)), l.matrix, l.opacity, l.blend)) }) .collect();@@ -788,6 +895,14 @@ impl GpuCompositor { // HDR / wide-gamut Y'CbCr: the planner's input transform, in the shader (#965) let src = SrcInfo { input: l.input.as_ref().filter(|_| src.kind == 2).and_then(|t| t.gpu_params()), ..src }; let views = self.uploads.get(k).map_or_else(|| std::array::from_fn(|_| self.dummy.clone()), |up| up.views.clone());+ // a log source's decode table, which the shader samples (`GpuCurve::Lut`) wherever the+ // source is drawn (onto the working image too); the compositor's own dummy for every+ // other curve+ let lut = l.input.as_deref().and_then(filmcraft_color::InputTransform::gpu_lut);+ let lut_view = match lut {+ Some(values) => self.lut_texture(values),+ None => self.dummy.clone(),+ }; let job = match &l.fx { Some(lfx) => { // the source drawn onto the working image (box-decimated by n, as the CPU decodes)@@ -806,6 +921,7 @@ impl GpuCompositor { wgpu::BindGroupEntry { binding: 2, resource: wgpu::BindingResource::TextureView(&views[1]) }, wgpu::BindGroupEntry { binding: 3, resource: wgpu::BindingResource::TextureView(&views[2]) }, wgpu::BindGroupEntry { binding: 5, resource: wgpu::BindingResource::TextureView(&views[3]) },+ wgpu::BindGroupEntry { binding: 6, resource: wgpu::BindingResource::TextureView(&lut_view) }, ], }); match self.fx.as_mut() {@@ -844,6 +960,7 @@ impl GpuCompositor { wgpu::BindGroupEntry { binding: 2, resource: wgpu::BindingResource::TextureView(&tex[1]) }, wgpu::BindGroupEntry { binding: 3, resource: wgpu::BindingResource::TextureView(&tex[2]) }, wgpu::BindGroupEntry { binding: 5, resource: wgpu::BindingResource::TextureView(&tex[3]) },+ wgpu::BindGroupEntry { binding: 6, resource: wgpu::BindingResource::TextureView(&lut_view) }, ]; let (layout, region) = match (&backdrop_view, l.blend.reads_destination()) { (Some(b), true) => {@@ -858,10 +975,14 @@ impl GpuCompositor { if let Some(f) = self.fx.as_mut() { f.end_frame(); }+ let display_buf = self.display_buffer(); let final_bg = self.device.create_bind_group(&wgpu::BindGroupDescriptor { label: Some("final"), layout: &self.final_bgl,- entries: &[wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(&accum_view) }],+ entries: &[+ wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(&accum_view) },+ wgpu::BindGroupEntry { binding: 1, resource: display_buf.as_entire_binding() },+ ], }); let mut enc = self.device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("filmcraft-composite") }); // Runs of fixed-function layers draw in one pass. A layer that reads the destination@@ -1002,6 +1123,7 @@ impl GpuCompositor { wgpu::BindGroupEntry { binding: 2, resource: wgpu::BindingResource::TextureView(&views[1]) }, wgpu::BindGroupEntry { binding: 3, resource: wgpu::BindingResource::TextureView(&views[2]) }, wgpu::BindGroupEntry { binding: 5, resource: wgpu::BindingResource::TextureView(&views[3]) },+ wgpu::BindGroupEntry { binding: 6, resource: wgpu::BindingResource::TextureView(&self.dummy) }, ], }); let stage = self.fx.as_mut()?;@@ -1172,8 +1294,13 @@ struct SrcInfo { input: Option<filmcraft_color::GpuInputParams>, } +/// Entries per row of a log decode table's texture (`lut_texture`; `LUT_ROW` in `composite.wgsl`).+const LUT_ROW: u32 = 256;+ /// Floats in a layer's uniform block (`U` in `composite.wgsl`: 13 vec4).-const UNIFORM_FLOATS: usize = 52;+const UNIFORM_FLOATS: usize = 56;+/// Floats in the display transform uniform (`struct O` in `composite.wgsl`): 5 vec4.+const DISPLAY_FLOATS: usize = 20; /// Poll a future once: wgpu's native error scopes resolve immediately. `None` if it is still /// pending (then the caller assumes the worst).@@ -152,6 +152,57 @@ fn gpu_matches_cpu_plan() { assert_eq!(c.uploaded_bytes, before); } +#[test]+fn gpu_matches_cpu_for_a_log_source() {+ use filmcraft_color::{ColorPipeline, ColorSpace, GpuCurve, InputTransform, Range};+ let Some((dev, q)) = device() else {+ eprintln!("no GPU adapter; skipping");+ return;+ };+ // S-Log3/S-Gamut3.Cine into the Rec. 2100 HLG working space: the shader cannot evaluate the log+ // curve, so it samples the CPU's decode table (binding 6), then runs the camera-gamut+ // compression stage. A constant chroma per run isolates the transform from the chroma+ // upsampling difference (bilinear in the shader, nearest on the CPU) that `gpu_matches_cpu_plan`+ // tolerates near edges.+ let pipe = ColorPipeline { working: filmcraft_color::WorkingSpace::Rec2100Hlg, wide_gamut: true, auto_tone_map: true };+ let t = Arc::new(InputTransform::new(ColorSpace::SLog3SGamut3Cine, Range::Limited, &pipe, None));+ let params = t.gpu_params().expect("a log source has a shader form");+ assert_eq!(params.curve, GpuCurve::Lut);+ assert!(params.lut.is_some() && params.compress && params.matrix.is_some(), "{params:?}");+ assert!(t.gpu_lut().is_some_and(|v| v.len() > 1024));+ let (w, h) = (256usize, 1usize);+ let mut c = GpuCompositor::new(&dev, &q);+ for (u, v) in [(128u8, 128u8), (64, 192), (192, 64)] {+ // a luma ramp over the frame's whole limited range, one chroma value throughout+ let y: Vec<u8> = (0..w).map(|i| (16 + i * 219 / w) as u8).collect();+ let frame = Arc::new(VideoFrame {+ width: w as u32,+ height: h as u32,+ data: PixelData::Yuv8 { planes: [Arc::new(y), Arc::new(vec![u; w]), Arc::new(vec![v; w])], chroma: Chroma::C444, alpha: None },+ color: filmcraft_color::ColorInfo { range: Range::Limited, ..filmcraft_color::ColorInfo::REC709 },+ par: (1, 1),+ pts: Default::default(),+ });+ let plan = FramePlan::Layers {+ width: w,+ height: h,+ layers: vec![PlanLayer { frame, matrix: Affine::IDENTITY, opacity: 1.0, blend: Blend::Normal, fx: None, input: Some(t.clone()) }],+ };+ let cpu = execute_cpu(&plan).over_black_rgba8();+ c.composite(&plan);+ let (gw, gh, gpu) = c.read_output().expect("readback");+ assert_eq!((gw as usize, gh as usize), (w, h));+ let mut diffs: Vec<u32> =+ cpu.chunks(4).zip(gpu.chunks(4)).map(|(a, b)| (0..3).map(|k| (a[k] as i32 - b[k] as i32).unsigned_abs()).max().unwrap_or(0)).collect();+ diffs.sort_unstable();+ let p99 = diffs[diffs.len() * 99 / 100];+ let mean = diffs.iter().sum::<u32>() as f64 / diffs.len() as f64;+ // f16 accumulator quantization plus the sampled table's own interpolation, nothing more+ assert!(p99 <= 6 && mean < 1.5, "log source U={u} V={v}: p99 {p99}, mean {mean}");+ eprintln!("log source (S-Log3 -> HLG working space) U={u} V={v}: p99 {p99}, mean {mean:.2}");+ }+}+ #[test] fn half_float_conversion() { for v in [0.0f32, 1.0, 0.5, 0.123, 65504.0, -2.0, 1e-5] {@@ -55,13 +55,19 @@ pub fn input_transform(cs: ColorSpace, range: Range, pipe: &ColorPipeline, peak_ } /// The input transform of a frame the GPU compositor can colour manage itself (#965): a Y'CbCr-/// frame of display-referred HDR or wide-gamut media (PQ, HLG, BT.2020, P3) on a plain pipeline,-/// whose working → monitor transform is the identity. None for log media (camera-gamut-/// compression), RGB frames and other pipelines, which [`decode`] converts on the CPU.+/// frame whose signal → working space stages the shader runs (the curves it evaluates itself, a log+/// curve through the CPU's own decode table, the gamut matrix, the tone map, the camera-gamut+/// compression). The sequence's working → monitor transform runs in the compositor's resolve pass+/// ([`OutputTransform::gpu_params`]), so an HDR working space is fine; when the resolve pass cannot+/// finish it, the whole frame stays on the CPU. None for RGB frames and for a transform with no+/// shader form, which [`decode`] converts on the CPU. pub fn gpu_input(project: &Project, item: ItemId, pipe: &ColorPipeline, cs: ColorSpace, frame: &VideoFrame) -> Option<Arc<InputTransform>> {- if !pipe.is_plain() || !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) {+ if !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) { return None; }+ // The compositor's resolve pass finishes the frame with the sequence's display transform:+ // when the shader cannot encode that, the whole frame stays on the CPU.+ display_transform(pipe).gpu_params()?; let t = input_transform(cs, frame.color.range, pipe, source_peak_nits(project, item)); t.gpu_params().map(|_| t) }@@ -229,7 +229,12 @@ fn gpu_chain<'a>(project: &Project, item: &'a TrackItem, opts: RenderOptions) -> pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOptions, sources: &dyn SourceProvider) -> FramePlan { let Some(seq) = project.sequence(seq_id) else { return FramePlan::Image(crate::Image::new(1, 1)) }; let (w, h) = output_size(seq, opts.scale);- if whole_frame_on_cpu(project, seq, t) {+ // An HDR / wide-gamut sequence plans as layers too: its sources are decoded into the working+ // space on the GPU (`colorman::gpu_input`, which a log curve enters through the CPU's decode+ // table) and the compositor's resolve pass runs the sequence's display transform (working+ // space → monitor). Frames the compositor cannot finish stay on the CPU: an adjustment layer,+ // a transition it cannot mix, or a display transform the shader cannot encode (an HDR target).+ if !layered_at(project, seq, t) || crate::colorman::display_transform(&seq.settings.color).gpu_params().is_none() { return FramePlan::Image(crate::render_sequence(project, seq_id, t, opts, sources)); } let mut layers = PlanStack::default();@@ -256,13 +261,17 @@ pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOption /// Whether [`plan_frame`] hands the frame at `t` back as one CPU image ([`FramePlan::Image`]). A /// GPU renderer gains nothing on such a frame, so a caller with only a few of them (the export's /// pool) can render it on the CPU without taking one.+/// Whether the frame at `t` needs the CPU compositor: an HDR / wide-gamut sequence, an adjustment+/// layer, or a transition the compositor cannot mix. Callers that render on the CPU only (the+/// export's pool) use it to skip taking a GPU plan; [`plan_frame`] itself plans an HDR / wide-gamut+/// frame as layers whenever the compositor can finish it. pub fn is_cpu_frame(project: &Project, seq_id: ItemId, t: Tick) -> bool { project.sequence(seq_id).is_none_or(|seq| whole_frame_on_cpu(project, seq, t)) } -/// The frames [`plan_frame`] does not plan: an HDR / wide-gamut sequence composites and converts-/// on the CPU, and so does a frame with an adjustment layer or a complex transition anywhere at-/// `t` (whole-frame fallback).+/// The frames the CPU-only path ([`is_cpu_frame`]) covers: an HDR / wide-gamut sequence (its+/// display transform and any log-source decode), an adjustment layer, or a complex transition+/// anywhere at `t` (whole-frame fallback). fn whole_frame_on_cpu(project: &Project, seq: &Sequence, t: Tick) -> bool { !seq.settings.color.is_plain() || !layered_at(project, seq, t) }@@ -469,9 +478,10 @@ fn push_item( let time = crate::video_source_time(item, t, src.info().frame_rate()); let Ok(frame) = src.video_frame(FrameRequest { time, scale: want }) else { return }; let cs = crate::colorman::source_space(project, item.item, &frame);- // HDR and wide-gamut media on a Rec. 709 sequence is converted on the GPU too (#965); log- // media and other pipelines are converted on the CPU (below), and so are blended- // in-between frames (Frame Blending / Optical Flow on speed-changed clips)+ // HDR, wide-gamut and log media is converted on the GPU (#965): the shader's own curves, a+ // log curve through the CPU's decode table. What has no shader form is converted below on+ // the CPU, and so are blended in-between frames (Frame Blending / Optical Flow on+ // speed-changed clips) let pipe = &seq.settings.color; let input = if crate::colorman::needs_management(pipe, cs, &frame) { crate::colorman::gpu_input(project, item.item, pipe, cs, &frame) } else { None }; let plain =@@ -46,6 +46,15 @@ impl FilmcraftApp { self.disable_gpu("device error"); return None; }+ // The resolve pass finishes a *sequence* frame: the sequence's working space → monitor+ // (`colorman::display_transform`). A plain sequence's transform is the identity, and item+ // frames (Source monitor, thumbnails) are already in their item's space.+ let pipe = match key.target {+ Target::Sequence(id) | Target::SequencePlan(id) => self.session.project.sequence(id).map(|q| q.settings.color),+ _ => None,+ };+ // A CPU-rendered frame (`FramePlan::Image`) is already in monitor space.+ let display = filmcraft_gpu::sequence_display(&plan.plan, pipe.and_then(|c| filmcraft_render::colorman::display_transform(&c).gpu_params())); let g = self.gpu.as_mut()?; if g.last_key == Some(key) && let Some(t) = g.texture@@ -57,6 +66,7 @@ impl FilmcraftApp { return None; } let t0 = web_time::Instant::now();+ g.compositor.set_display_transform(display); let ran = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { let (view, size) = g.compositor.composite_prepared(&plan.plan, Some(&plan.prepared)); (view.clone(), size)@@ -561,9 +561,12 @@ frame (Y'CbCr/RGB + metadata) ─► source colour space: Interpret Footage over - **Maths** in `filmcraft-color` (`transform`, `log`, `spaces`; formulas and sources in [crates/color/README.md](../crates/color/README.md)); the renderer side is `render::colorman`. - **Fast path.** A Rec. 709 sequence without wide gamut and media whose metadata says Rec. 709 /- sRGB decodes exactly as before, and its layers stay on the GPU. Log, HDR or wide-gamut media,- and every layer of an HDR/wide-gamut sequence, are converted on the CPU (the GPU path draws them- as pre-rendered images, like layers with effects).+ sRGB decodes exactly as before, and its layers stay on the GPU. Display-referred HDR or+ wide-gamut media (PQ, HLG, BT.2020, P3) is converted in the compositor's shader, and an HDR /+ wide-gamut sequence composites on the GPU too: the resolve pass runs its display transform+ (working space → monitor, `OutputTransform::gpu_params`); a display transform the shader cannot+ encode keeps the whole frame on the CPU. Log media is converted in the shader as well, through+ the CPU's own decode table (`InputTransform::gpu_lut`), camera-gamut compression included. - **Outputs.** `RenderOptions::working_output` returns working-space pixels (HDR exports, scopes); otherwise the top-level render is converted for an SDR monitor. The other colour effects work on display-encoded values clamped to 0..1 (they clip HDR highlights above@@ -146,8 +146,9 @@ Each codec README has the full fixture matrix and the measured results. to Black at 25 %, Wipe at 50 %, Timecode / Clip Name burn-in text, graphic clips (also a built-in graphics template with overridden properties and per-character styles with a pinned box), and colour management: S-Log3/S-Gamut3.Cine footage interpreted into Rec. 709 (`log_to_rec709`) and- Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`; the footage is demo frames- re-encoded by the test's `Encoded` source). Each frame is compared+ Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`), and Lumetri grading in a+ Rec. 2100 PQ sequence, tone mapped for the monitor (`lumetri_hdr_pq`); the footage of both is+ demo frames re-encoded by the test's `Encoded` source. Each frame is compared with `crates/golden/goldens/<scene>.png` by `filmcraft_testkit::golden`: **PSNR ≥ 45 dB, max abs ≤ 12, 99th-percentile per-pixel max channel difference ≤ 2** (8-bit sRGB levels; `Tolerance::RENDER`). On failure the actual frame and a ×8 difference image go to@@ -176,6 +177,9 @@ Each codec README has the full fixture matrix and the measured results. This found a real bug: the GPU upload cache was keyed by pixel-buffer address without keeping the buffer alive, so a new frame allocated at a freed frame's address was drawn with the stale texture (fixed; regression test `upload_cache_keeps_buffers_alive`).+- `gpu_matches_cpu_on_golden_scenes` sets each scene's display transform as the app does, so the+ colour-managed scenes (`hdr_tone_map`, and `lumetri_hdr_pq` in a Rec. 2100 PQ sequence) are+ converted and tone mapped on the GPU too. ## 4. UI: control channel and screenshots