collective
Browse / github.com/storytold/filmcraft / patch
render, gpu: Lumetri on the GPU in HDR (PQ / HLG) sequences
render, gpu: Lumetri on the GPU in HDR (PQ / HLG) sequences Depends on #3213: this branch contains its commit. Only the last commit (b905c7fc) belongs to this PR; merge #3213 first. Sibling PRs of this series add rows to the same revision-history tables in `docs/`; whichever merges second has a one-line conflict there (keep both rows). ## What The GPU Lumetri op refused an HDR working space, so every Lumetri clip in a Rec. 2100 sequence rendered on the CPU. - render: FxOp::Lumetri carries the grading space (GradeSpace) and HDR Specular, and its CPU apply() grades the same signal as fn lumetri: the PQ / HLG signal normalised to HDR White, contrast t
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- zsoli69 (imported from pull request #3220, open)
- Base
10d626ccc4888e0f020f9ca314c2bc9148ded4f4- Size
- +547 −46 in 18 files
- Review
- pending
- Shared
- 2026-10-28
- Id
2d12d28f8e0b0733a6f40872
How it went
No other operator has reported using it yet.
Diff
raw · crates/color/src/lib.rs crates/color/src/transform.rs crates/golden/tests/golden.rs crates/gpu/src/composite.wgsl crates/gpu/src/fx.rs crates/gpu/src/fx.wgsl crates/gpu/src/fx_tests.rs crates/gpu/src/lib.rs crates/render/src/colorman.rs crates/render/src/effects.rs crates/render/src/gpufx.rs crates/render/src/plan.rs crates/ui-egui/src/app/gpu.rs docs/architecture.md docs/gaps.md docs/hardware-parity.md docs/target-app-parity.md docs/testing.md
@@ -20,7 +20,7 @@ pub use grade::GradeSpace; pub use log::LogCurve; pub use lut::{Lut, Lut1d, Lut3d, LutFormat}; pub use spaces::{ColorPipeline, ColorSpace, Curve, Gamut, WorkingSpace};-pub use transform::{DecodeTable, GpuCurve, GpuInputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap};+pub use transform::{DecodeTable, GpuCurve, GpuInputParams, GpuOutputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap}; /// YUV ↔ RGB matrix coefficients. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]@@ -389,6 +389,37 @@ impl OutputTransform { curve => l.map(|v| encode_channel(v as f64, curve, Range::Full).clamp(0.0, 1.0) as f32), } }++ /// The transform as plain numbers for a shader, which runs [`apply`](Self::apply)'s stages+ /// (BT.2390 tone mapping, the working → target matrix, desaturation into the target gamut) and+ /// encodes the target's SDR curve itself. `None` for a target the shader does not encode (an+ /// HDR export target), which stays on the CPU.+ pub fn gpu_params(&self) -> Option<GpuOutputParams> {+ if self.target.curve() != Curve::Sdr {+ return None;+ }+ Some(GpuOutputParams {+ target: self.target,+ matrix: (!self.identity_matrix).then_some(self.matrix),+ tone: self.tone.as_ref().map(|t| (t.src_peak_nits as f32, t.dst_peak_nits as f32)),+ compress: self.compress,+ })+ }+}++/// [`OutputTransform`] as plain numbers for a shader ([`OutputTransform::gpu_params`]): the stages+/// after the compositor — tone mapping, working → target matrix, gamut desaturation, in the order+/// [`OutputTransform::apply`] runs them — with the target's SDR curve left to the shader's resolve.+#[derive(Clone, Copy, Debug, PartialEq)]+pub struct GpuOutputParams {+ /// The target colour space (SDR: the shader's resolve is the sRGB curve).+ pub target: ColorSpace,+ /// Working → target primaries (row-major), when they differ.+ pub matrix: Option<[[f32; 3]; 3]>,+ /// BT.2390 tone mapping (source peak, target peak) in cd/m², on max(R,G,B) as a ratio.+ pub tone: Option<(f32, f32)>,+ /// Target luma weights to desaturate into the target gamut with ([`desaturate_into_gamut`]).+ pub compress: Option<[f32; 3]>, } /// Convenience: a log colour space → display Rec. 709 (tone mapped), used for the built-in@@ -550,4 +581,20 @@ mod tests { // plain pipeline is an identity assert!(OutputTransform::display(&ColorPipeline::REC709).is_identity()); }++ #[test]+ fn output_gpu_params_cover_the_display_transform() {+ // a plain pipeline: the shader's sRGB resolve is the whole transform+ let plain = OutputTransform::display(&ColorPipeline::REC709).gpu_params().unwrap();+ assert!(plain.matrix.is_none() && plain.tone.is_none() && plain.compress.is_none());+ // an HDR working space on a SDR monitor: tone mapping, the working → 709 matrix and the+ // gamut compression, the stages the resolve pass runs+ let hdr = OutputTransform::display(&self::hdr(WorkingSpace::Rec2100Hlg)).gpu_params().unwrap();+ assert_eq!(hdr.target, ColorSpace::Rec709);+ assert!(hdr.matrix.is_some() && hdr.compress.is_some());+ assert_eq!(hdr.tone, Some((HDR_PEAK_NITS as f32, REFERENCE_WHITE_NITS as f32)));+ // an HDR export target has no shader encode yet: it stays on the CPU+ let export = OutputTransform::new(&self::hdr(WorkingSpace::Rec2100Hlg), ColorSpace::Rec2100Pq);+ assert!(export.gpu_params().is_none());+ } }@@ -775,6 +775,9 @@ fn gpu_matches_cpu_on_golden_scenes() { let cpu = render_cpu(&s); let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources); let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" };+ // the resolve pass runs the sequence's display transform, as the app sets it (an HDR+ // working space composites in working linear and is tone mapped there)+ c.set_display_transform(display_for(&s, &plan)); c.composite(&plan); let (gw, gh, mut px) = c.read_output().expect("GPU readback"); px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);@@ -793,6 +796,44 @@ fn gpu_matches_cpu_on_golden_scenes() { assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n")); } +/// The resolve pass's display transform for `plan` of scene `s`, chosen as the app chooses it+/// (`gpu_present`): none for a CPU-rendered frame, which is already in monitor space.+fn display_for(s: &Scene, plan: &filmcraft_render::plan::FramePlan) -> Option<filmcraft_color::GpuOutputParams> {+ let pipe = s.project.sequence(s.seq).map(|q| q.settings.color);+ filmcraft_gpu::sequence_display(plan, pipe.and_then(|p| filmcraft_render::colorman::display_transform(&p).gpu_params()))+}++/// A frame of an HDR (Rec. 2100 PQ) sequence that the compositor cannot finish (a Wipe) comes+/// back as one CPU image, already converted for the monitor: the GPU shows it as it is, without+/// tone mapping it a second time.+#[test]+fn gpu_shows_an_hdr_cpu_frame_as_rendered() {+ let Some((dev, q)) = device("gpu hdr cpu frame", false) else {+ return;+ };+ let mut b = Builder::new(1);+ b.p.sequence_mut(b.seq).unwrap().settings.color =+ filmcraft_color::ColorPipeline { working: filmcraft_color::WorkingSpace::Rec2100Pq, ..filmcraft_color::ColorPipeline::REC709 };+ let pq = ColorInfo { transfer: Transfer::Pq, primaries: Primaries::Bt2020, ..ColorInfo::SRGB_FULL };+ let a = b.encoded(DemoScene::CityNight, ColorSpace::Rec2100Pq, 6.0, Some(pq), None);+ let z = b.demo(DemoScene::Aurora);+ let ca = b.place(0, a, 0, 24);+ let cz = b.place(0, z, 24, 24);+ b.transition(0, "wipe", ca, cz, 24, 12);+ let s = b.at(24);+ let t = RATE.tick_of(s.frame);+ let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources);+ assert!(matches!(plan, filmcraft_render::plan::FramePlan::Image(_)), "a Wipe is rendered on the CPU");+ let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);+ c.set_display_transform(display_for(&s, &plan));+ c.composite(&plan);+ let (gw, gh, mut px) = c.read_output().expect("GPU readback");+ px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);+ let d = diff(&render_cpu(&s), &Rgba8::new(gw, gh, px)).unwrap();+ let mean_rgb = d.mean_abs * 4.0 / 3.0;+ assert!(d.p99 <= 6 && mean_rgb < 1.5, "HDR CPU frame on the GPU differs from the CPU: {d}");+}+ /// Per-pixel max RGB difference (99th percentile, mean) over the pixels at least 1.5 px from every /// plan layer's quad outline: the CPU resampler fades a layer's edge over one more pixel than the /// GPU rasterises, and with blend modes those edge pixels are a visible share of a 320×180 frame.@@ -394,6 +394,21 @@ fn fs_blend(in: VOut) -> @location(0) vec4<f32> { @group(0) @binding(0) var accum: texture_2d<f32>; +// The resolve pass's display transform (`filmcraft_color::OutputTransform`, #965): the sequence's+// working linear → monitor. Same stages as `OutputTransform::apply`, in the same order: BT.2390+// tone mapping (from an HDR working space), the working → target gamut matrix, then desaturation+// into the target gamut. `a.x` is 0 for a plain sequence (the accumulator is already monitor+// space) and the sRGB encode below is the whole transform.+struct O {+ a: vec4<f32>, // on, target curve (0 sdr), tone mapping on, matrix on+ b: vec4<f32>, // desaturation on, tone source peak cd/m², tone target peak cd/m², reference white cd/m²+ m0: vec4<f32>, // matrix row 0, desaturation luma weight 0+ m1: vec4<f32>, // matrix row 1, desaturation luma weight 1+ m2: vec4<f32>, // matrix row 2, desaturation luma weight 2+};++@group(0) @binding(1) var<uniform> disp: O;+ @vertex fn vs_full(@builtin(vertex_index) vi: u32) -> @builtin(position) vec4<f32> { let x = f32((vi << 1u) & 2u) * 2.0 - 1.0;@@ -401,9 +416,44 @@ fn vs_full(@builtin(vertex_index) vi: u32) -> @builtin(position) vec4<f32> { return vec4(x, y, 0.0, 1.0); } +// Working linear → monitor linear.+fn display(rgb: vec3<f32>) -> vec3<f32> {+ if disp.a.x < 0.5 { return rgb; }+ var c = rgb;+ if disp.a.z > 0.5 {+ // `ToneMap::apply`: one scale from max(R,G,B), the EETF sampled in the PQ domain+ let m = max(max(c.r, c.g), c.b);+ if m > 0.0 {+ let nits = m * disp.b.w;+ var k = 1.0;+ if nits >= disp.b.y {+ // above the source peak: clip to the target peak+ k = disp.b.z / disp.b.w / m;+ } else if nits > 1e-6 {+ k = eetf(nits, disp.b.y, disp.b.z) / nits;+ }+ c = c * k;+ }+ }+ if disp.a.w > 0.5 { c = vec3(dot(disp.m0.xyz, c), dot(disp.m1.xyz, c), dot(disp.m2.xyz, c)); }+ if disp.b.x > 0.5 {+ // `desaturate_into_gamut`: out-of-gamut colours towards their luminance, in-gamut exactly+ let mn = min(min(c.r, c.g), c.b);+ if mn < 0.0 {+ let y = dot(vec3(disp.m0.w, disp.m1.w, disp.m2.w), c);+ if y <= 0.0 {+ c = vec3(0.0);+ } else {+ c = max(y + (c - y) * (y / (y - mn)), vec3(0.0));+ }+ }+ }+ return c;+}+ @fragment fn fs_full(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {- let c = clamp(textureLoad(accum, vec2<i32>(pos.xy), 0).rgb, vec3(0.0), vec3(1.0));+ let c = clamp(display(textureLoad(accum, vec2<i32>(pos.xy), 0).rgb), vec3(0.0), vec3(1.0)); let lo = c * 12.92; let hi = 1.055 * pow(c, vec3(1.0 / 2.4)) - 0.055; return vec4(select(hi, lo, c <= vec3(0.0031308)), 1.0);@@ -206,8 +206,16 @@ fn steps(op: &FxOp) -> Vec<Step> { vround, vfeather, aspect,+ grade,+ specular, .. } => {+ // the grading signal: mode 0 sRGB, 1 PQ (p3.w = norm), 2 HLG (p3.w = white, p4.w = gamma)+ let (mode, g0, g1) = match *grade {+ filmcraft_color::GradeSpace::Sdr => (0u32, 0.0, 0.0),+ filmcraft_color::GradeSpace::Pq { norm, .. } => (1, norm, 0.0),+ filmcraft_color::GradeSpace::Hlg { white, gamma } => (2, white, gamma),+ }; let ge = [gains[0] * exposure, gains[1] * exposure, gains[2] * exposure]; let b0 = -bl * 0.15; let w0 = 1.0 - wh * 0.15;@@ -233,17 +241,17 @@ fn steps(op: &FxOp) -> Vec<Step> { *vmid, *vfeather, vround_aspect,- 0.0,+ g0, st_k[0], st_k[1], st_k[2],- 0.0,+ g1, ht_k[0], ht_k[1], ht_k[2],- 0.0,+ *specular, ];- out.push(step(OP_LUMETRI, [*creative_on as u32, *vignette_on as u32, 0, 0], &p));+ out.push(step(OP_LUMETRI, [*creative_on as u32, *vignette_on as u32, mode, 0], &p)); } } out@@ -441,6 +441,83 @@ fn color_op(op: u32, c: vec3<f32>) -> vec3<f32> { } } +// `GradeSpace::encode` / `decode` (filmcraft-color): the signal Lumetri grades, by u.i1.z+// (0 sRGB, 1 PQ normalised by u.p3.w, 2 HLG of a u.p3.w cd/m² display with system gamma u.p4.w).+const REFERENCE_WHITE: f32 = 203.0;+const LUMA2020: vec3<f32> = vec3(0.26270021, 0.67799807, 0.05930172);++fn pq_eotf1(e: f32) -> f32 {+ let m1 = 0.15930176; let m2 = 78.84375; let c1 = 0.8359375; let c2 = 18.851563; let c3 = 18.6875;+ let p = powf(max(e, 0.0), 1.0 / m2);+ return powf(max(p - c1, 0.0) / (c2 - c3 * p), 1.0 / m1);+}++fn pq_inverse_eotf1(y: f32) -> f32 {+ let m1 = 0.15930176; let m2 = 78.84375; let c1 = 0.8359375; let c2 = 18.851563; let c3 = 18.6875;+ let p = powf(max(y, 0.0), m1);+ return powf((c1 + c2 * p) / (1.0 + c3 * p), m2);+}++fn hlg_oetf1(l: f32) -> f32 {+ let a = 0.17883277; let b = 0.28466892; let c = 0.5599107;+ if l <= 1.0 / 12.0 {+ return sqrt(3.0 * max(l, 0.0));+ }+ return a * log(12.0 * l - b) + c;+}++fn hlg_inverse_oetf1(e: f32) -> f32 {+ let a = 0.17883277; let b = 0.28466892; let c = 0.5599107;+ if e <= 0.5 {+ return e * e / 3.0;+ }+ return (exp((e - c) / a) + b) / 12.0;+}++fn grade_enc(c: vec3<f32>) -> vec3<f32> {+ let mode = u.i1.z;+ if mode == 1u {+ let norm = u.p3.w;+ let k = REFERENCE_WHITE / 10000.0;+ let m = max(c, vec3(0.0)) * k;+ return vec3(pq_inverse_eotf1(min(m.x, 1.0)), pq_inverse_eotf1(min(m.y, 1.0)), pq_inverse_eotf1(min(m.z, 1.0))) / norm;+ }+ if mode == 2u {+ let white = u.p3.w;+ let gamma = u.p4.w;+ let fd = max(c, vec3(0.0)) * (REFERENCE_WHITE / white);+ let yd = dot(LUMA2020, fd);+ if yd <= 0.0 {+ return vec3(0.0);+ }+ let k = powf(yd, (1.0 - gamma) / gamma);+ return vec3(hlg_oetf1(fd.x * k), hlg_oetf1(fd.y * k), hlg_oetf1(fd.z * k));+ }+ return enc(c);+}++fn grade_dec(v: vec3<f32>) -> vec3<f32> {+ let mode = u.i1.z;+ if mode == 1u {+ let norm = u.p3.w;+ let q = clamp(v * norm, vec3(0.0), vec3(1.0));+ return vec3(pq_eotf1(q.x), pq_eotf1(q.y), pq_eotf1(q.z)) * (10000.0 / REFERENCE_WHITE);+ }+ if mode == 2u {+ let white = u.p3.w;+ let gamma = u.p4.w;+ let q = clamp(v, vec3(0.0), vec3(1.5));+ let es = vec3(hlg_inverse_oetf1(q.x), hlg_inverse_oetf1(q.y), hlg_inverse_oetf1(q.z));+ let ys = dot(LUMA2020, es);+ if ys <= 0.0 {+ return vec3(0.0);+ }+ let k = powf(ys, gamma - 1.0);+ return es * (k * white / REFERENCE_WHITE);+ }+ return dec(v);+}+ fn lumetri_op(c: vec3<f32>, p: vec2<i32>) -> vec3<f32> { let s = size(); let creative_on = u.i1.x != 0u;@@ -449,7 +526,7 @@ fn lumetri_op(c: vec3<f32>, p: vec2<i32>) -> vec3<f32> { // white balance + exposure in linear light let ge = u.p0.xyz; let lin = c * ge;- var v = enc(lin);+ var v = grade_enc(lin); // whites / blacks: endpoints let b0 = u.p0.w;@@ -471,17 +548,27 @@ fn lumetri_op(c: vec3<f32>, p: vec2<i32>) -> vec3<f32> { v = v * k; } - // contrast: smooth S-curve around mid grey+ // contrast: smooth S-curve around mid grey (HDR: over 0 … HDR White; speculars above it+ // keep their distance from white) let contrast = u.p1.w;+ let hdr = u.i1.z != 0u; if abs(contrast) > 1e-4 { let k = 1.0 + contrast; let q = clamp(v, vec3(0.0), vec3(1.0)); let sc = q * q * (vec3(3.0) - 2.0 * q);+ var o: vec3<f32>; if k >= 1.0 {- v = q + (sc - q) * (k - 1.0);+ o = q + (sc - q) * (k - 1.0); } else {- v = vec3(0.5) + (q - vec3(0.5)) * k;+ o = vec3(0.5) + (q - vec3(0.5)) * k; }+ v = select(o, v, vec3(hdr) & (v > vec3(1.0)));+ }++ // HDR Specular: brightness of the highlights above HDR White+ let specular = u.p5.w;+ if abs(specular) > 1e-4 {+ v = select(v, vec3(1.0) + (v - vec3(1.0)) * max(1.0 + specular, 0.0), v > vec3(1.0)); } // faded film: lift blacks and compress@@ -527,7 +614,7 @@ fn lumetri_op(c: vec3<f32>, p: vec2<i32>) -> vec3<f32> { } } - return dec(v);+ return grade_dec(v); } // One output pixel of every op but the running-sum box blur.@@ -566,12 +566,65 @@ fn lumetri_unsupported_sections_fall_back() { let op = FxOp::eval(&e, &cx(Tick::ZERO, 1.0), w, h); assert!(op.as_ref().is_some_and(|o| !o.gpu_ok())); - // HDR working space- let e = effect("lumetri", &[]);+ // HDR still falls back for the sections the shader does not have let mut cx_hdr = cx(Tick::ZERO, 1.0);- cx_hdr.working = filmcraft_color::WorkingSpace::Rec2100Pq;- let op = FxOp::eval(&e, &cx_hdr, w, h);- assert!(op.as_ref().is_some_and(|o| !o.gpu_ok()));+ cx_hdr.working = filmcraft_color::WorkingSpace::Rec2100Hlg;+ let e = effect("lumetri", &[("sharpen", fl(20.0))]);+ assert!(FxOp::eval(&e, &cx_hdr, w, h).is_some_and(|o| !o.gpu_ok()));+}++/// Lumetri's basic, creative and vignette sections in an HDR working space (the PQ / HLG grading+/// signal, contrast that leaves speculars alone, HDR Specular) match the CPU op on the GPU.+#[test]+fn hdr_lumetri_matches_cpu() {+ let Some((dev, q)) = device() else {+ eprintln!("no GPU adapter; skipping");+ return;+ };+ let mut c = GpuCompositor::new(&dev, &q);+ let (w, h) = (67u32, 41u32);+ let px = picture(w, h);+ let frame = VideoFrame::rgba_f32(w, h, px.clone());+ let grades: [&[(&str, ParamValue)]; 4] = [+ &[("exposure", fl(-0.5)), ("saturation", fl(150.0))],+ &[("contrast", fl(40.0)), ("highlights", fl(-20.0)), ("shadows", fl(25.0)), ("hdr_specular", fl(-30.0)), ("temperature", fl(15.0))],+ &[("contrast", fl(-50.0)), ("whites", fl(20.0)), ("blacks", fl(-10.0)), ("hdr_white", fl(200.0)), ("hdr_specular", fl(60.0))],+ &[+ ("creative_on", ParamValue::Bool(true)),+ ("vibrance", fl(30.0)),+ ("faded_film", fl(20.0)),+ ("shadow_tint", col(0.4, 0.45, 0.6)),+ ("vignette_amount", fl(-2.0)),+ ],+ ];+ let mut failures = Vec::new();+ for working in [filmcraft_color::WorkingSpace::Rec2100Hlg, filmcraft_color::WorkingSpace::Rec2100Pq] {+ let mut cxh = cx(Tick::ZERO, 1.0);+ cxh.working = working;+ for params in grades {+ let e = effect("lumetri", params);+ let op = FxOp::eval(&e, &cxh, w as usize, h as usize).expect("lumetri op");+ assert!(op.gpu_ok(), "{working:?} {params:?}: not GPU-capable");+ let mut cpu = filmcraft_render::Image { w: w as usize, h: h as usize, px: px.clone() };+ op.apply(&mut cpu);+ // HDR Specular only acts above HDR White: the picture must reach it, or the parity+ // check below would pass without running that branch+ if params.iter().any(|(k, _)| *k == "hdr_specular") {+ let off: Vec<(&str, ParamValue)> = params.iter().map(|(k, v)| (*k, if *k == "hdr_specular" { fl(0.0) } else { v.clone() })).collect();+ let flat = FxOp::eval(&effect("lumetri", &off), &cxh, w as usize, h as usize).expect("lumetri op");+ let mut plain = filmcraft_render::Image { w: w as usize, h: h as usize, px: px.clone() };+ flat.apply(&mut plain);+ assert_ne!(cpu.px, plain.px, "{working:?} {params:?}: HDR Specular changed nothing");+ }+ let (_, _, gpu) = c.effect_image(&frame, &LayerFx { size: (w, h), decimation: 1, ops: vec![op] }).expect("effect image");+ let (worst, flips) = compare(&cpu.px, &gpu);+ eprintln!("{working:?} {params:?}: max rel diff {worst:.2e}, {:.3}% flipped", flips * 100.0);+ if worst > EXACT * 10.0 || flips > 0.0 {+ failures.push(format!("{working:?} {params:?}: {worst:.2e}, flips {flips:.4}"));+ }+ }+ }+ assert!(failures.is_empty(), "HDR Lumetri differs from the CPU:\n{}", failures.join("\n")); } /// Layers with effect chains in a composited plan — YUV and RGBA sources, a decimated working@@ -710,6 +763,44 @@ fn layers_with_effects_fall_back_to_the_cpu_without_a_stage() { } } +/// A colour-managed layer (PQ media) with effects comes out the same with and without the GPU+/// effect stage: the CPU fallback decodes it through the layer's input transform too.+#[test]+fn cpu_effect_fallback_keeps_the_input_transform() {+ use filmcraft_color::{ColorInfo, ColorPipeline, ColorSpace, InputTransform, Primaries};+ let Some((dev, q)) = device() else {+ eprintln!("no GPU adapter; skipping");+ return;+ };+ let (w, h) = (256u32, 4u32);+ let y: Vec<u16> = (0..h).flat_map(|_| (0..w).map(|x| (64 + x * 876 / 255) as u16)).collect();+ let n = (w * h) as usize;+ let data = PixelData::Yuv16 { planes: [Arc::new(y), Arc::new(vec![480; n]), Arc::new(vec![560; n])], chroma: Chroma::C444, bits: 10, alpha: None };+ let color = ColorInfo { matrix: Matrix::Bt2020Ncl, transfer: Transfer::Pq, primaries: Primaries::Bt2020, range: Range::Limited };+ let frame = Arc::new(VideoFrame { width: w, height: h, data, color, par: (1, 1), pts: Default::default() });+ let t = Arc::new(InputTransform::new(ColorSpace::from_info(&color), color.range, &ColorPipeline::REC709, None));+ let e = effect("brightness_contrast", &[("brightness", fl(-10.0)), ("contrast", fl(20.0))]);+ let op = FxOp::eval(&e, &cx(Tick::ZERO, 1.0), w as usize, h as usize).expect("brightness / contrast op");+ let layer = PlanLayer {+ fx: Some(Arc::new(LayerFx { size: (w, h), decimation: 1, ops: vec![op] })),+ input: Some(t),+ ..PlanLayer::new(frame, Affine::IDENTITY, 1.0, Blend::Normal)+ };+ let plan = FramePlan::Layers { width: w as usize, height: h as usize, layers: vec![layer] };+ let cpu = execute_cpu(&plan).over_black_rgba8();+ for stage in [true, false] {+ let mut c = GpuCompositor::new(&dev, &q);+ if !stage {+ c.fx = None;+ }+ c.composite(&plan);+ let (_, _, gpu) = c.read_output().expect("readback");+ let (p99, mean, max) = stats8(&cpu, &gpu, &vec![true; n]);+ eprintln!("PQ layer with effects, effect stage {stage}: p99 {p99}, mean {mean:.2}, max {max}");+ assert!(p99 <= 6 && mean < 1.5, "stage {stage}: p99 {p99}, mean {mean}, max {max}");+ }+}+ /// The half-float upload is lossless for this input, so a 1:1 effect source draw must be too. /// Odd dimensions expose interpolation drift that can invent opacity before any effect runs. #[test]@@ -84,6 +84,8 @@ pub struct GpuCompositor { accum_readback_buf: Option<(wgpu::Buffer, u64)>, uploads: HashMap<(usize, u32, u32), Uploaded>, clock: u64,+ /// The resolve pass's display transform (`set_display_transform`), as plain numbers.+ display: [f32; DISPLAY_FLOATS], dummy: wgpu::TextureView, /// The effect stage (None on devices without compute shaders or storage textures: layers /// with effects are then rendered on the CPU here).@@ -253,6 +255,16 @@ pub fn prepare_frame(f: &VideoFrame) -> Option<Prepared> { } } +/// The display transform to resolve a *sequence* plan with: `display` (the sequence's,+/// `colorman::display_transform(..).gpu_params()`) for a plan composited in its working space,+/// none for a [`FramePlan::Image`]. That image is `render_sequence`'s output, already converted+/// for the monitor, and converting it again would tone map and gamut map it twice. (The CPU+/// images the compositor makes itself, for a transition it cannot mix, are in working space and+/// keep the transform: they are drawn inside a working-space plan.)+pub fn sequence_display(plan: &FramePlan, display: Option<filmcraft_color::GpuOutputParams>) -> Option<filmcraft_color::GpuOutputParams> {+ if matches!(plan, FramePlan::Image(_)) { None } else { display }+}+ /// Convert every layer of a plan for upload. Thread-safe and GPU-free: run it on a worker. pub fn prepare(plan: &FramePlan) -> PreparedPlan { match plan {@@ -314,7 +326,16 @@ impl GpuCompositor { }); let final_bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor { label: Some("final"),- entries: &[wgpu::BindGroupLayoutEntry { binding: 0, ..tex_entry(0) }],+ entries: &[+ wgpu::BindGroupLayoutEntry { binding: 0, ..tex_entry(0) },+ // the resolve pass's display transform (`set_display_transform`)+ wgpu::BindGroupLayoutEntry {+ binding: 1,+ visibility: wgpu::ShaderStages::FRAGMENT,+ ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },+ count: None,+ },+ ], }); let pl = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor { label: Some("layer"), bind_group_layouts: &[Some(&layer_bgl)], immediate_size: 0 });@@ -420,6 +441,7 @@ impl GpuCompositor { accum_readback_buf: None, uploads: HashMap::new(), clock: 0,+ display: [0.0; DISPLAY_FLOATS], dummy, fx, transition: transition::PushStage::new(device),@@ -699,11 +721,58 @@ impl GpuCompositor { buf } + /// The resolve pass's display transform uniform ([`set_display_transform`](Self::set_display_transform)).+ fn display_buffer(&self) -> wgpu::Buffer {+ let bytes: Vec<u8> = self.display.iter().flat_map(|v| v.to_le_bytes()).collect();+ let buf = self.device.create_buffer(&wgpu::BufferDescriptor {+ label: Some("display-u"),+ size: bytes.len() as u64,+ usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,+ mapped_at_creation: false,+ });+ self.queue.write_buffer(&buf, 0, &bytes);+ buf+ }+ /// Composite a plan; returns the output view (sRGB, over black) and its size. pub fn composite(&mut self, plan: &FramePlan) -> (wgpu::TextureView, (u32, u32)) { self.composite_prepared(plan, None) } + /// Set the resolve pass's display transform (the sequence's working space → monitor,+ /// `filmcraft_color::OutputTransform::gpu_params`). Layers are composited in the sequence's+ /// working linear space, so an HDR / wide-gamut sequence needs it to come out right; `None`+ /// (the default) leaves the accumulator as the monitor's own space, as for a plain sequence.+ /// Only the resolve pass uses it: a caller that reads the working-linear accumulator back+ /// (the export path) is unaffected.+ pub fn set_display_transform(&mut self, p: Option<filmcraft_color::GpuOutputParams>) {+ let Some(p) = p else {+ self.display = [0.0; DISPLAY_FLOATS];+ return;+ };+ let mx = p.matrix.unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]);+ let (sp, dp) = p.tone.unwrap_or((0.0, 0.0));+ let dl = p.compress.unwrap_or([0.0; 3]);+ let on = |b: bool| if b { 1.0 } else { 0.0 };+ let curve = match p.target.curve() {+ filmcraft_color::Curve::Sdr | filmcraft_color::Curve::Srgb => 0.0,+ filmcraft_color::Curve::Linear => 1.0,+ filmcraft_color::Curve::Pq => 2.0,+ filmcraft_color::Curve::Hlg => 3.0,+ // not reachable: `gpu_params` refuses a target the shader cannot encode+ filmcraft_color::Curve::Log(_) => 0.0,+ };+ #[rustfmt::skip]+ let d = [+ 1.0, curve, on(p.tone.is_some()), on(p.matrix.is_some()),+ on(p.compress.is_some()), sp, dp, filmcraft_color::REFERENCE_WHITE_NITS as f32,+ mx[0][0], mx[0][1], mx[0][2], dl[0],+ mx[1][0], mx[1][1], mx[1][2], dl[1],+ mx[2][0], mx[2][1], mx[2][2], dl[2],+ ];+ self.display = d;+ }+ /// [`composite`](Self::composite) with texel conversions already done by [`prepare`] (the /// result is identical; only the upload work on this thread differs). pub fn composite_prepared(&mut self, plan: &FramePlan, prep: Option<&PreparedPlan>) -> (wgpu::TextureView, (u32, u32)) {@@ -749,7 +818,9 @@ impl GpuCompositor { if self.fx.is_some() && lfx.size.0.max(lfx.size.1) <= max_side { return None; }- let img = filmcraft_render::plan::effect_image(&l.frame, lfx);+ // decoded through the layer's input transform, as `execute_cpu` does; the image is+ // then working-space linear, so the new layer has none+ let img = filmcraft_render::plan::effect_image_with(&l.frame, lfx, l.input.as_deref()); Some(PlanLayer::new(Arc::new(VideoFrame::rgba_f32(img.w as u32, img.h as u32, img.px)), l.matrix, l.opacity, l.blend)) }) .collect();@@ -858,10 +929,14 @@ impl GpuCompositor { if let Some(f) = self.fx.as_mut() { f.end_frame(); }+ let display_buf = self.display_buffer(); let final_bg = self.device.create_bind_group(&wgpu::BindGroupDescriptor { label: Some("final"), layout: &self.final_bgl,- entries: &[wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(&accum_view) }],+ entries: &[+ wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(&accum_view) },+ wgpu::BindGroupEntry { binding: 1, resource: display_buf.as_entire_binding() },+ ], }); let mut enc = self.device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("filmcraft-composite") }); // Runs of fixed-function layers draw in one pass. A layer that reads the destination@@ -1174,6 +1249,8 @@ struct SrcInfo { /// Floats in a layer's uniform block (`U` in `composite.wgsl`: 13 vec4). const UNIFORM_FLOATS: usize = 52;+/// Floats in the display transform uniform (`struct O` in `composite.wgsl`): 5 vec4.+const DISPLAY_FLOATS: usize = 20; /// Poll a future once: wgpu's native error scopes resolve immediately. `None` if it is still /// pending (then the caller assumes the worst).@@ -55,13 +55,18 @@ pub fn input_transform(cs: ColorSpace, range: Range, pipe: &ColorPipeline, peak_ } /// The input transform of a frame the GPU compositor can colour manage itself (#965): a Y'CbCr-/// frame of display-referred HDR or wide-gamut media (PQ, HLG, BT.2020, P3) on a plain pipeline,-/// whose working → monitor transform is the identity. None for log media (camera-gamut-/// compression), RGB frames and other pipelines, which [`decode`] converts on the CPU.+/// frame whose signal → working space stages the shader runs (PQ, HLG, BT.2020, P3; log curves and+/// camera-gamut compression are what it cannot). The sequence's working → monitor transform runs in+/// the compositor's resolve pass (`OutputTransform::gpu_params`), so an HDR working space is fine;+/// when the resolve pass cannot finish it, the whole frame stays on the CPU. None for log media,+/// RGB frames and pipelines the shader cannot finish, which [`decode`] converts on the CPU. pub fn gpu_input(project: &Project, item: ItemId, pipe: &ColorPipeline, cs: ColorSpace, frame: &VideoFrame) -> Option<Arc<InputTransform>> {- if !pipe.is_plain() || !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) {+ if !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) { return None; }+ // The compositor's resolve pass finishes the frame with the sequence's display transform:+ // when the shader cannot encode that, the whole frame stays on the CPU.+ display_transform(pipe).gpu_params()?; let t = input_transform(cs, frame.color.range, pipe, source_peak_nits(project, item)); t.gpu_params().map(|_| t) }@@ -1558,6 +1558,23 @@ mod lumetri_cpu_parity_tests { } eprintln!("{name}: max sample diff vs fn lumetri: {max_diff:.2e}"); }++ // HDR working spaces: the op grades the same PQ / HLG signal as fn lumetri+ for working in [filmcraft_color::WorkingSpace::Rec2100Hlg, filmcraft_color::WorkingSpace::Rec2100Pq] {+ let cx = FxCtx { working, ..cx };+ for (name, params) in cases.iter().chain(&[("specular", vec![("contrast", fl(30.0)), ("hdr_specular", fl(-40.0)), ("exposure", fl(1.0))])]) {+ let e = effect(params);+ let mut img_orig = test_picture(w, h);+ let mut img_copy = img_orig.clone();+ lumetri(&mut img_orig, &e, &cx);+ let op = FxOp::eval(&e, &cx, w, h).unwrap_or_else(|| panic!("{name}: eval returned None"));+ assert!(op.gpu_ok(), "{working:?} {name}: expected gpu_ok == true");+ op.apply(&mut img_copy);+ for (i, (&orig, ©)) in img_orig.px.iter().zip(&img_copy.px).enumerate() {+ assert!((orig - copy).abs() <= 1e-6, "{working:?} {name}: sample {i}: orig={orig}, copy={copy}");+ }+ }+ } } /// #719: positive Roundness rounds the Lumetri vignette (it used to be ignored), +100@@ -9,7 +9,7 @@ //! op to the GPU when [`FxOp::gpu_ok`] says the shader covers it (finite numbers, no minifying //! resample), otherwise the clip is rendered on the CPU as before. -use filmcraft_color::{hsl_to_rgb, linear_to_srgb, luma709, rgb_to_hsl};+use filmcraft_color::{GradeSpace, hsl_to_rgb, linear_to_srgb, luma709, rgb_to_hsl}; use filmcraft_geom::Affine; use filmcraft_project::EffectInstance; use rayon::prelude::*;@@ -195,6 +195,11 @@ pub enum FxOp { vround: f32, vfeather: f32, aspect: f32,+ /// The signal the grade works on: sRGB (SDR) or the PQ / HLG signal normalised to HDR+ /// White (an HDR working space).+ grade: GradeSpace,+ /// HDR Specular (0 for SDR).+ specular: f32, gpu_capable: bool, }, }@@ -529,7 +534,7 @@ impl FxOp { let hsl_on = b(e, "hsl_on"); - let gpu_capable = !is_hdr && !has_input_lut && !has_look_lut && look == 0 && sharpen.abs() <= 1e-3 && !has_curves && !has_wheels && !hsl_on;+ let gpu_capable = !has_input_lut && !has_look_lut && look == 0 && sharpen.abs() <= 1e-3 && !has_curves && !has_wheels && !hsl_on; let bf = |id: &str| if basic_on { f(e, id, cx) } else { 0.0 }; let temp = bf("temperature") / 100.0;@@ -572,6 +577,8 @@ impl FxOp { vround, vfeather, aspect,+ grade: gs,+ specular: if is_hdr && basic_on { f(e, "hdr_specular", cx) / 100.0 } else { 0.0 }, gpu_capable, } }@@ -620,8 +627,36 @@ impl FxOp { FxOp::AlphaAdjust { opacity, .. } => opacity.is_finite(), FxOp::Vignette { amount, midpoint, roundness, feather, target } => fin(&[*amount, *midpoint, *roundness, *feather]) && fin(target), FxOp::VideoLimiter { max, comp, axis, warning_color, .. } => *axis <= 3 && max.is_finite() && comp.is_finite() && fin(warning_color),- FxOp::Lumetri { gains, exposure, contrast, hl, sh, wh, bl, sat, faded, vib, st, ht, va, vmid, vround, vfeather, aspect, gpu_capable, .. } => {+ FxOp::Lumetri {+ gains,+ exposure,+ contrast,+ hl,+ sh,+ wh,+ bl,+ sat,+ faded,+ vib,+ st,+ ht,+ va,+ vmid,+ vround,+ vfeather,+ aspect,+ grade,+ specular,+ gpu_capable,+ ..+ } => { *gpu_capable+ && specular.is_finite()+ && match *grade {+ GradeSpace::Sdr => true,+ GradeSpace::Pq { white, norm } => white.is_finite() && norm.is_finite() && norm > 0.0,+ GradeSpace::Hlg { white, gamma } => white.is_finite() && gamma.is_finite() && gamma > 0.0,+ } && fin(gains) && fin(&[*exposure, *contrast, *hl, *sh, *wh, *bl, *sat, *faded, *vib]) && fin(st)@@ -942,8 +977,12 @@ impl FxOp { vround, vfeather, aspect,+ grade,+ specular, .. } => {+ let (gs, specular) = (*grade, *specular);+ let hdr = gs.is_hdr(); let gains = *gains; let (exposure, contrast, hl, sh, wh, bl, sat) = (*exposure, *contrast, *hl, *sh, *wh, *bl, *sat); let (creative_on, faded, vib, st, ht) = (*creative_on, *faded, *vib, *st, *ht);@@ -952,7 +991,7 @@ impl FxOp { img.map_rgb(|c, x, y| { // white balance + exposure in linear light let lin = [c[0] * gains[0] * exposure, c[1] * gains[1] * exposure, c[2] * gains[2] * exposure];- let mut v = enc(lin);+ let mut v = gs.encode(lin); // whites / blacks: endpoints let b0 = -bl * 0.15; let w0 = 1.0 - wh * 0.15;@@ -966,15 +1005,23 @@ impl FxOp { let k = nl / l; v = v.map(|q| q * k); }- // contrast: smooth S-curve around mid grey+ // contrast: smooth S-curve around mid grey (HDR: over 0 … HDR White;+ // speculars above it keep their distance from white) if contrast.abs() > 1e-4 { let k = 1.0 + contrast; v = v.map(|q| {+ if hdr && q > 1.0 {+ return q;+ } let q = q.clamp(0.0, 1.0); let s = q * q * (3.0 - 2.0 * q); if k >= 1.0 { q + (s - q) * (k - 1.0) } else { 0.5 + (q - 0.5) * k } }); }+ // HDR Specular: brightness of the highlights above HDR White+ if specular.abs() > 1e-4 {+ v = v.map(|q| if q > 1.0 { 1.0 + (q - 1.0) * (1.0 + specular).max(0.0) } else { q });+ } // faded film: lift blacks and compress if faded > 0.0 { v = v.map(|q| q * (1.0 - 0.25 * faded) + 0.12 * faded);@@ -1001,7 +1048,7 @@ impl FxOp { let k = 1.0 + va * 0.2 * e2; v = v.map(|q| if va < 0.0 { q * k.max(0.0) } else { q + (1.0 - q) * (k - 1.0) }); }- dec(v)+ gs.decode(v) }); } }@@ -229,7 +229,13 @@ fn gpu_chain<'a>(project: &Project, item: &'a TrackItem, opts: RenderOptions) -> pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOptions, sources: &dyn SourceProvider) -> FramePlan { let Some(seq) = project.sequence(seq_id) else { return FramePlan::Image(crate::Image::new(1, 1)) }; let (w, h) = output_size(seq, opts.scale);- if whole_frame_on_cpu(project, seq, t) {+ // An HDR / wide-gamut sequence plans as layers too: its sources are decoded into the working+ // space (the shader's input transform for display-referred media, `colorman::decode` on the+ // CPU for log media and camera gamuts) and the compositor's resolve pass runs the sequence's+ // display transform (working space → monitor). Frames the compositor cannot finish stay on+ // the CPU: an adjustment layer, a transition it cannot mix, or a display transform the shader+ // cannot encode (an HDR target).+ if !layered_at(project, seq, t) || crate::colorman::display_transform(&seq.settings.color).gpu_params().is_none() { return FramePlan::Image(crate::render_sequence(project, seq_id, t, opts, sources)); } let mut layers = PlanStack::default();@@ -256,13 +262,17 @@ pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOption /// Whether [`plan_frame`] hands the frame at `t` back as one CPU image ([`FramePlan::Image`]). A /// GPU renderer gains nothing on such a frame, so a caller with only a few of them (the export's /// pool) can render it on the CPU without taking one.+/// Whether the frame at `t` needs the CPU compositor: an HDR / wide-gamut sequence, an adjustment+/// layer, or a transition the compositor cannot mix. Callers that render on the CPU only (the+/// export's pool) use it to skip taking a GPU plan; [`plan_frame`] itself plans an HDR / wide-gamut+/// frame as layers whenever the compositor can finish it. pub fn is_cpu_frame(project: &Project, seq_id: ItemId, t: Tick) -> bool { project.sequence(seq_id).is_none_or(|seq| whole_frame_on_cpu(project, seq, t)) } -/// The frames [`plan_frame`] does not plan: an HDR / wide-gamut sequence composites and converts-/// on the CPU, and so does a frame with an adjustment layer or a complex transition anywhere at-/// `t` (whole-frame fallback).+/// The frames the CPU-only path ([`is_cpu_frame`]) covers: an HDR / wide-gamut sequence (its+/// display transform and any log-source decode), an adjustment layer, or a complex transition+/// anywhere at `t` (whole-frame fallback). fn whole_frame_on_cpu(project: &Project, seq: &Sequence, t: Tick) -> bool { !seq.settings.color.is_plain() || !layered_at(project, seq, t) }@@ -46,6 +46,15 @@ impl FilmcraftApp { self.disable_gpu("device error"); return None; }+ // The resolve pass finishes a *sequence* frame: the sequence's working space → monitor+ // (`colorman::display_transform`). A plain sequence's transform is the identity, and item+ // frames (Source monitor, thumbnails) are already in their item's space.+ let pipe = match key.target {+ Target::Sequence(id) | Target::SequencePlan(id) => self.session.project.sequence(id).map(|q| q.settings.color),+ _ => None,+ };+ // A CPU-rendered frame (`FramePlan::Image`) is already in monitor space.+ let display = filmcraft_gpu::sequence_display(&plan.plan, pipe.and_then(|c| filmcraft_render::colorman::display_transform(&c).gpu_params())); let g = self.gpu.as_mut()?; if g.last_key == Some(key) && let Some(t) = g.texture@@ -57,6 +66,7 @@ impl FilmcraftApp { return None; } let t0 = web_time::Instant::now();+ g.compositor.set_display_transform(display); let ran = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { let (view, size) = g.compositor.composite_prepared(&plan.plan, Some(&plan.prepared)); (view.clone(), size)@@ -561,9 +561,12 @@ frame (Y'CbCr/RGB + metadata) ─► source colour space: Interpret Footage over - **Maths** in `filmcraft-color` (`transform`, `log`, `spaces`; formulas and sources in [crates/color/README.md](../crates/color/README.md)); the renderer side is `render::colorman`. - **Fast path.** A Rec. 709 sequence without wide gamut and media whose metadata says Rec. 709 /- sRGB decodes exactly as before, and its layers stay on the GPU. Log, HDR or wide-gamut media,- and every layer of an HDR/wide-gamut sequence, are converted on the CPU (the GPU path draws them- as pre-rendered images, like layers with effects).+ sRGB decodes exactly as before, and its layers stay on the GPU. Display-referred HDR or+ wide-gamut media (PQ, HLG, BT.2020, P3) is converted in the compositor's shader, and an HDR /+ wide-gamut sequence composites on the GPU too: the resolve pass runs its display transform+ (working space → monitor, `OutputTransform::gpu_params`); a display transform the shader cannot+ encode keeps the whole frame on the CPU. Log media and camera gamuts are converted on the CPU+ (the GPU path draws them as pre-rendered images, like layers with effects). - **Outputs.** `RenderOptions::working_output` returns working-space pixels (HDR exports, scopes); otherwise the top-level render is converted for an SDR monitor. The other colour effects work on display-encoded values clamped to 0..1 (they clip HDR highlights above@@ -574,8 +577,9 @@ frame (Y'CbCr/RGB + metadata) ─► source colour space: Interpret Footage over Range**): `filmcraft_color::GradeSpace` (PQ: `PQ⁻¹(nits/10000) / PQ⁻¹(white/10000)`; HLG: the BT.2100 inverse OOTF of a display with that peak, then the OETF). So the sliders and curves span 0 … HDR White like black … white in SDR, exposure is stops of light, and highlights above HDR- White pass through (contrast leaves them alone; **HDR Specular** scales them). Rec. 709- sequences grade exactly as before. HSL Secondary ▸ Refine: **Denoise** (median of the key,+ White pass through (contrast leaves them alone; **HDR Specular** scales them). The GPU Lumetri+ op (basic, creative, vignette) grades the same signal. Rec. 709 sequences grade exactly as+ before. HSL Secondary ▸ Refine: **Denoise** (median of the key, radius 1–3 px) and **Blur** (Gaussian of the key, σ ≤ 20 px), both scaled with the playback resolution. - **HDR metadata.** `VideoStreamInfo::hdr` (`HdrMetadata`: ST 2086 mastering luminance, MaxCLL,@@ -1,6 +1,6 @@ # Where FilmCraft falls short of Premiere Pro -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: ROADMAP's "Where we are lacking" re-measured against Premiere Pro 26.5.2 and the open issue list) · **Target:** Adobe Premiere Pro 2026 (26.5.2)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-24 · **Change:** minor (G5: Lumetri basic / creative / vignette on the GPU in Rec. 2100 PQ / HLG sequences too) · **Target:** Adobe Premiere Pro 2026 (26.5.2) Every known shortfall, one entry each, ranked by what most stops an editor using FilmCraft for real work. This is the work list: agents should prefer it over adding checklist items. Numbers are in@@ -105,7 +105,7 @@ platforms** G4, G9, G17; **performance** G16; **ecosystem** G11; **measurement** - **Have:** hardware decode on macOS (VideoToolbox H.264 / HEVC), Windows (Media Foundation / D3D11: H.264, HEVC, VP9, AV1) and Linux (VA-API and NVDEC: H.264, HEVC); hardware encode VideoToolbox H.264 / HEVC 8-bit, NVENC H.264 (Windows, Linux) and HEVC incl. Main 10 HDR- (Windows); GPU compositing, blend modes, 34 effect ids incl. Lumetri basic / creative / vignette,+ (Windows); GPU compositing, blend modes, 34 effect ids incl. Lumetri basic / creative / vignette (also in Rec. 2100 PQ / HLG sequences; a Lumetri with an input LUT, look, sharpen, curves, wheels or HSL set renders on the CPU), GPU export rendering (#421). - **Missing:** zero-copy decoded frames into wgpu (every frame is read back to the CPU and re-uploaded); VideoToolbox Main 10 / HDR HEVC encode and B-frames; ProRes hardware decode /@@ -323,3 +323,4 @@ platforms** G4, G9, G17; **performance** G16; **ecosystem** G11; **measurement** | 2026-10-12 | minor | G10: MP1 (`A_MPEG/L1`) in Matroska imports (#1402) | | 2026-10-12 | minor | G10: OpenEXR PXR24 import | | 2026-10-24 | minor | G8: dragging the cut on a transition's timeline in Effect Controls moves the cut, and the transition moves with it (#577) |+| 2026-10-24 | minor | G5: Lumetri basic / creative / vignette on the GPU in Rec. 2100 PQ / HLG sequences too |@@ -1,6 +1,6 @@ # Hardware parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: hardware decode / encode per platform and codec, GPU effects, devices; re-measured after NVDEC #549 and GPU export #421) · **Target:** Adobe Premiere Pro 2026 (26.5.2)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-24 · **Change:** minor (GPU effects: Lumetri basic / creative / vignette in Rec. 2100 PQ / HLG sequences too) · **Target:** Adobe Premiere Pro 2026 (26.5.2) Each hardware feature Premiere uses, per platform, against ours. Implementation detail lives in [`crates/platform/README.md`](../crates/platform/README.md) (decode / encode backends, the one@@ -50,7 +50,7 @@ against 793 s with our software encoder (SSIM 0.990). NVENC HEVC round trip 48.9 | Feature | Premiere (Mercury Playback Engine: Metal, CUDA, OpenCL) | Ours (wgpu: Metal, D3D12, Vulkan) | % | |---|---|---|---| | Compositing, blend modes, transforms | GPU | **GPU** (#32), CPU compositor is the oracle | 90% |-| Accelerated video effects | ~82 of ~92 effects marked Accelerated (reference list `plan/premiere/10-effects-list.md`) | **34 effect ids** (`render::gpufx::GPU_EFFECTS`): colour, levels, blurs, sharpen, crop, transform, flips, vignette, video limiter, ASC CDL, channel mix, Lumetri basic / creative / vignette | ~40% |+| Accelerated video effects | ~82 of ~92 effects marked Accelerated (reference list `plan/premiere/10-effects-list.md`) | **34 effect ids** (`render::gpufx::GPU_EFFECTS`): colour, levels, blurs, sharpen, crop, transform, flips, vignette, video limiter, ASC CDL, channel mix, Lumetri basic / creative / vignette (SDR and Rec. 2100 PQ / HLG; with no input LUT, look, sharpen, curves, wheels or HSL set) | ~40% | | Lumetri curves, wheels, HSL secondary | GPU | CPU | 0% | | Keys (Ultra Key), masks, distortions, transitions | GPU | CPU (masks have WGSL parity tests, not in the live path) | 10% | | Export rendering on the GPU | yes | **yes** (#421; Export ▸ GPU Rendering, Auto): 2.4× (1080p) to 3.2× (4K) faster on effects-heavy timelines | 80% |@@ -85,3 +85,4 @@ one encoder ~4–8 h ([target-app-parity.md](target-app-parity.md#calibration)). | Date | Change | Summary | |---|---|---| | 2026-10-10 | major | Created from ROADMAP.md's performance-and-hardware row and `crates/platform/README.md`; added Linux NVDEC (#549), GPU export and Lumetri on the GPU (#421), Premiere's per-platform hardware list, devices and I/O |+| 2026-10-24 | minor | GPU effects: Lumetri basic / creative / vignette in Rec. 2100 PQ / HLG sequences too |@@ -1,6 +1,6 @@ # Parity with Adobe Premiere Pro -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (full re-measure against Premiere Pro 2026 26.5.2; replaces the scorecard and estimate that lived in ROADMAP.md) · **Target:** Adobe Premiere Pro 2026 (26.5.2, macOS)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-24 · **Change:** minor (Colour: Lumetri on the GPU in Rec. 2100 PQ / HLG sequences too) · **Target:** Adobe Premiere Pro 2026 (26.5.2, macOS) The authoritative assessment of how close FilmCraft is to Premiere Pro, and how much work is left. [ROADMAP.md](../ROADMAP.md) summarizes it; [gaps.md](gaps.md) lists every shortfall one at a time.@@ -112,7 +112,7 @@ agent-hours to close the area to Premiere, sequential. | Media import and codecs | 12% | ~75% | ~60% | 150–250 | Own bit-exact decoders for H.264, HEVC, VP9, AV1, ProRes, DNx, APV, MPEG-2, AAC, AC-3, Opus. But H.264 only 8-bit 4:2:0 and HEVC no 4:2:2 in software (most 10-bit cameras, #626); no camera RAW (R3D, ARRIRAW, BRAW, Sony RAW, Canon RAW, ProRes RAW, #345, #383); no E-AC-3 (#647), AVI (#598), WMV, DV, XAVC / P2 metadata; Sony start timecode (#460); WebM black (#432) | [codec-parity.md](codec-parity.md), [file-format-parity.md](file-format-parity.md) | | Audio | 10% | ~88% | ~65% | 50–90 | Mixer graph, automation, all 53 effects, Essential Sound, 5.1, voice-over, Remix, multi-stream container audio (#361). Device and playback reports: crackle / cut-outs (#336, #410, #602, #605, #617); Enhance Speech is DSP, not a model; ALSA fallback fixed (#254) | | | Panels and UI fidelity | 10% | ~88% | ~60% | 120–200 | All main panels. Missing: on-screen transform handles in the Program monitor (#639), Effect Controls keyframe zoom / divider / row lines (#640–#645), docking rearrangement (#493), multi-monitor windows (#466), filmstrip thumbnails (#614), marquee in Project panel (#578) | [ui-parity.md](ui-parity.md) |-| Colour | 8% | ~90% | ~70% | 40–70 | Lumetri complete incl. scopes, LUTs, colour management, HDR export. Lumetri on the GPU covers basic, creative and vignette only; HDR-aware Lumetri maths, macOS EDR display, D-Log M; a Lumetri report (#628) | |+| Colour | 8% | ~90% | ~70% | 40–70 | Lumetri complete incl. scopes, LUTs, colour management, HDR export. Lumetri on the GPU covers basic, creative and vignette only (in SDR and Rec. 2100 PQ / HLG sequences), and only while no input LUT, look, sharpen, curves, wheels or HSL is set (those render the effect on the CPU); HDR-aware Lumetri maths, macOS EDR display, D-Log M; a Lumetri report (#628) | | | Graphics, titles, captions | 8% | ~85% | ~65% | 50–90 | Text engine with shaping and bidi, shapes, gradients (#514), our `.fcgt` templates (`.mogrt` refused by design), rolls / crawls, all caption formats. Arabic titles reported broken (#395); no 608/708 embedding in the video stream; speech-to-text off in release builds | | | Export and delivery | 8% | ~80% | ~65% | 60–110 | H.264 (own encoder), HEVC through hardware only, ProRes, DNxHR, APV, MXF OP1a / OP-Atom, image sequences, GIF, WAV / AIFF, presets, queue, GPU export rendering (#421). Missing: AV1, MPEG-2 / DVD / Blu-ray, DCP, JPEG 2000 / XS MXF, AS-10 / AS-11, XDCAM HD, EXR / DPX / Targa sequences, MP3, AAC-only audio, ProRes 4444 in the UI (#342), interlaced, smart render; bitrate target missed (#371) | [file-format-parity.md](file-format-parity.md) | | Project files and interchange | 8% | ~70% | ~45% | 90–160 | `.fcproj` with migrations, auto-save, recovery; FCP7 XML, FCPXML, AAF, OMF, EDL, OTIO, ALE. **Premiere's own `.prproj` cannot be opened** (the main beta blocker); AAF / OMF never validated in Media Composer / Pro Tools; XML import of a Premiere sequence took 97 s (#461) | [file-format-parity.md](file-format-parity.md) |@@ -354,6 +354,7 @@ control surfaces), and validation in Avid Media Composer / Pro Tools. | Date | Change | Summary | |---|---|---|+| 2026-10-24 | minor | Colour: Lumetri on the GPU in Rec. 2100 PQ / HLG sequences too | | 2026-10-10 | minor | Readiness table: hours to ~95% for each audience (full ~1,450–2,600, mainstream ~500–850, essentials ~180–350); full number confirmed as an additive weighted sum (no change) | | 2026-10-10 | minor | Added the three readiness numbers: full ready-for-real-work formula written out (~55% -> ~56%, same inputs, now computed rather than rounded down), mainstream practitioner ~55% and essentials user ~60% with written weights and discounts, and counted user evidence (31 people praising, 0 switch reports, ~64 core-path vs ~29 niche open bugs) | | 2026-10-10 | minor | Stage checked against the new core-workflow gate (docs/roadmap.md#alpha-gate): passes, stays alpha |@@ -146,8 +146,9 @@ Each codec README has the full fixture matrix and the measured results. to Black at 25 %, Wipe at 50 %, Timecode / Clip Name burn-in text, graphic clips (also a built-in graphics template with overridden properties and per-character styles with a pinned box), and colour management: S-Log3/S-Gamut3.Cine footage interpreted into Rec. 709 (`log_to_rec709`) and- Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`; the footage is demo frames- re-encoded by the test's `Encoded` source). Each frame is compared+ Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`), and Lumetri grading in a+ Rec. 2100 PQ sequence, tone mapped for the monitor (`lumetri_hdr_pq`); the footage of both is+ demo frames re-encoded by the test's `Encoded` source. Each frame is compared with `crates/golden/goldens/<scene>.png` by `filmcraft_testkit::golden`: **PSNR ≥ 45 dB, max abs ≤ 12, 99th-percentile per-pixel max channel difference ≤ 2** (8-bit sRGB levels; `Tolerance::RENDER`). On failure the actual frame and a ×8 difference image go to@@ -176,6 +177,9 @@ Each codec README has the full fixture matrix and the measured results. This found a real bug: the GPU upload cache was keyed by pixel-buffer address without keeping the buffer alive, so a new frame allocated at a freed frame's address was drawn with the stale texture (fixed; regression test `upload_cache_keeps_buffers_alive`).+- `gpu_matches_cpu_on_golden_scenes` sets each scene's display transform as the app does, so the+ colour-managed scenes (`hdr_tone_map`, and `lumetri_hdr_pq` in a Rec. 2100 PQ sequence) are+ converted and tone mapped on the GPU too. ## 4. UI: control channel and screenshots