collective
Browse / github.com/storytold/filmcraft / patch
gpu, render: composite HDR / wide-gamut sequences on the GPU
gpu, render: composite HDR / wide-gamut sequences on the GPU Related to #965 (HEVC preview performance). ## What A sequence whose working space is not Rec. 709 (Rec. 2100 PQ / HLG, or wide gamut) planned every frame as one CPU image: the composite, the input transforms and the working -> monitor transform all ran on the CPU, the largest cost of 4K HDR playback. - color: OutputTransform::gpu_params() (GpuOutputParams) carries the stages after the compositor - BT.2390 tone mapping, the working -> target matrix, the gamut desaturation - in apply()'s order, leaving the SDR encode to the shader. None for a target the shader does not encode (an HDR export
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- zsoli69 (imported from pull request #3213, open)
- Base
10d626ccc4888e0f020f9ca314c2bc9148ded4f4- Size
- +303 −18 in 11 files
- Review
- pending
- Shared
- 2026-10-28
- Id
e9cb3d64e5b2b1a439bbcd5b
How it went
No other operator has reported using it yet.
Diff
raw · crates/color/src/lib.rs crates/color/src/transform.rs crates/golden/tests/golden.rs crates/gpu/src/composite.wgsl crates/gpu/src/fx_tests.rs crates/gpu/src/lib.rs crates/render/src/colorman.rs crates/render/src/plan.rs crates/ui-egui/src/app/gpu.rs docs/architecture.md docs/testing.md
@@ -20,7 +20,7 @@ pub use grade::GradeSpace; pub use log::LogCurve; pub use lut::{Lut, Lut1d, Lut3d, LutFormat}; pub use spaces::{ColorPipeline, ColorSpace, Curve, Gamut, WorkingSpace};-pub use transform::{DecodeTable, GpuCurve, GpuInputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap};+pub use transform::{DecodeTable, GpuCurve, GpuInputParams, GpuOutputParams, InputTransform, OutputTransform, REFERENCE_WHITE_NITS, ToneMap}; /// YUV ↔ RGB matrix coefficients. #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]@@ -389,6 +389,37 @@ impl OutputTransform { curve => l.map(|v| encode_channel(v as f64, curve, Range::Full).clamp(0.0, 1.0) as f32), } }++ /// The transform as plain numbers for a shader, which runs [`apply`](Self::apply)'s stages+ /// (BT.2390 tone mapping, the working → target matrix, desaturation into the target gamut) and+ /// encodes the target's SDR curve itself. `None` for a target the shader does not encode (an+ /// HDR export target), which stays on the CPU.+ pub fn gpu_params(&self) -> Option<GpuOutputParams> {+ if self.target.curve() != Curve::Sdr {+ return None;+ }+ Some(GpuOutputParams {+ target: self.target,+ matrix: (!self.identity_matrix).then_some(self.matrix),+ tone: self.tone.as_ref().map(|t| (t.src_peak_nits as f32, t.dst_peak_nits as f32)),+ compress: self.compress,+ })+ }+}++/// [`OutputTransform`] as plain numbers for a shader ([`OutputTransform::gpu_params`]): the stages+/// after the compositor — tone mapping, working → target matrix, gamut desaturation, in the order+/// [`OutputTransform::apply`] runs them — with the target's SDR curve left to the shader's resolve.+#[derive(Clone, Copy, Debug, PartialEq)]+pub struct GpuOutputParams {+ /// The target colour space (SDR: the shader's resolve is the sRGB curve).+ pub target: ColorSpace,+ /// Working → target primaries (row-major), when they differ.+ pub matrix: Option<[[f32; 3]; 3]>,+ /// BT.2390 tone mapping (source peak, target peak) in cd/m², on max(R,G,B) as a ratio.+ pub tone: Option<(f32, f32)>,+ /// Target luma weights to desaturate into the target gamut with ([`desaturate_into_gamut`]).+ pub compress: Option<[f32; 3]>, } /// Convenience: a log colour space → display Rec. 709 (tone mapped), used for the built-in@@ -550,4 +581,20 @@ mod tests { // plain pipeline is an identity assert!(OutputTransform::display(&ColorPipeline::REC709).is_identity()); }++ #[test]+ fn output_gpu_params_cover_the_display_transform() {+ // a plain pipeline: the shader's sRGB resolve is the whole transform+ let plain = OutputTransform::display(&ColorPipeline::REC709).gpu_params().unwrap();+ assert!(plain.matrix.is_none() && plain.tone.is_none() && plain.compress.is_none());+ // an HDR working space on a SDR monitor: tone mapping, the working → 709 matrix and the+ // gamut compression, the stages the resolve pass runs+ let hdr = OutputTransform::display(&self::hdr(WorkingSpace::Rec2100Hlg)).gpu_params().unwrap();+ assert_eq!(hdr.target, ColorSpace::Rec709);+ assert!(hdr.matrix.is_some() && hdr.compress.is_some());+ assert_eq!(hdr.tone, Some((HDR_PEAK_NITS as f32, REFERENCE_WHITE_NITS as f32)));+ // an HDR export target has no shader encode yet: it stays on the CPU+ let export = OutputTransform::new(&self::hdr(WorkingSpace::Rec2100Hlg), ColorSpace::Rec2100Pq);+ assert!(export.gpu_params().is_none());+ } }@@ -775,6 +775,9 @@ fn gpu_matches_cpu_on_golden_scenes() { let cpu = render_cpu(&s); let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources); let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" };+ // the resolve pass runs the sequence's display transform, as the app sets it (an HDR+ // working space composites in working linear and is tone mapped there)+ c.set_display_transform(display_for(&s, &plan)); c.composite(&plan); let (gw, gh, mut px) = c.read_output().expect("GPU readback"); px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);@@ -793,6 +796,44 @@ fn gpu_matches_cpu_on_golden_scenes() { assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n")); } +/// The resolve pass's display transform for `plan` of scene `s`, chosen as the app chooses it+/// (`gpu_present`): none for a CPU-rendered frame, which is already in monitor space.+fn display_for(s: &Scene, plan: &filmcraft_render::plan::FramePlan) -> Option<filmcraft_color::GpuOutputParams> {+ let pipe = s.project.sequence(s.seq).map(|q| q.settings.color);+ filmcraft_gpu::sequence_display(plan, pipe.and_then(|p| filmcraft_render::colorman::display_transform(&p).gpu_params()))+}++/// A frame of an HDR (Rec. 2100 PQ) sequence that the compositor cannot finish (a Wipe) comes+/// back as one CPU image, already converted for the monitor: the GPU shows it as it is, without+/// tone mapping it a second time.+#[test]+fn gpu_shows_an_hdr_cpu_frame_as_rendered() {+ let Some((dev, q)) = device("gpu hdr cpu frame", false) else {+ return;+ };+ let mut b = Builder::new(1);+ b.p.sequence_mut(b.seq).unwrap().settings.color =+ filmcraft_color::ColorPipeline { working: filmcraft_color::WorkingSpace::Rec2100Pq, ..filmcraft_color::ColorPipeline::REC709 };+ let pq = ColorInfo { transfer: Transfer::Pq, primaries: Primaries::Bt2020, ..ColorInfo::SRGB_FULL };+ let a = b.encoded(DemoScene::CityNight, ColorSpace::Rec2100Pq, 6.0, Some(pq), None);+ let z = b.demo(DemoScene::Aurora);+ let ca = b.place(0, a, 0, 24);+ let cz = b.place(0, z, 24, 24);+ b.transition(0, "wipe", ca, cz, 24, 12);+ let s = b.at(24);+ let t = RATE.tick_of(s.frame);+ let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources);+ assert!(matches!(plan, filmcraft_render::plan::FramePlan::Image(_)), "a Wipe is rendered on the CPU");+ let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q);+ c.set_display_transform(display_for(&s, &plan));+ c.composite(&plan);+ let (gw, gh, mut px) = c.read_output().expect("GPU readback");+ px.as_chunks_mut::<4>().0.iter_mut().for_each(|p| p[3] = 255);+ let d = diff(&render_cpu(&s), &Rgba8::new(gw, gh, px)).unwrap();+ let mean_rgb = d.mean_abs * 4.0 / 3.0;+ assert!(d.p99 <= 6 && mean_rgb < 1.5, "HDR CPU frame on the GPU differs from the CPU: {d}");+}+ /// Per-pixel max RGB difference (99th percentile, mean) over the pixels at least 1.5 px from every /// plan layer's quad outline: the CPU resampler fades a layer's edge over one more pixel than the /// GPU rasterises, and with blend modes those edge pixels are a visible share of a 320×180 frame.@@ -394,6 +394,21 @@ fn fs_blend(in: VOut) -> @location(0) vec4<f32> { @group(0) @binding(0) var accum: texture_2d<f32>; +// The resolve pass's display transform (`filmcraft_color::OutputTransform`, #965): the sequence's+// working linear → monitor. Same stages as `OutputTransform::apply`, in the same order: BT.2390+// tone mapping (from an HDR working space), the working → target gamut matrix, then desaturation+// into the target gamut. `a.x` is 0 for a plain sequence (the accumulator is already monitor+// space) and the sRGB encode below is the whole transform.+struct O {+ a: vec4<f32>, // on, target curve (0 sdr), tone mapping on, matrix on+ b: vec4<f32>, // desaturation on, tone source peak cd/m², tone target peak cd/m², reference white cd/m²+ m0: vec4<f32>, // matrix row 0, desaturation luma weight 0+ m1: vec4<f32>, // matrix row 1, desaturation luma weight 1+ m2: vec4<f32>, // matrix row 2, desaturation luma weight 2+};++@group(0) @binding(1) var<uniform> disp: O;+ @vertex fn vs_full(@builtin(vertex_index) vi: u32) -> @builtin(position) vec4<f32> { let x = f32((vi << 1u) & 2u) * 2.0 - 1.0;@@ -401,9 +416,44 @@ fn vs_full(@builtin(vertex_index) vi: u32) -> @builtin(position) vec4<f32> { return vec4(x, y, 0.0, 1.0); } +// Working linear → monitor linear.+fn display(rgb: vec3<f32>) -> vec3<f32> {+ if disp.a.x < 0.5 { return rgb; }+ var c = rgb;+ if disp.a.z > 0.5 {+ // `ToneMap::apply`: one scale from max(R,G,B), the EETF sampled in the PQ domain+ let m = max(max(c.r, c.g), c.b);+ if m > 0.0 {+ let nits = m * disp.b.w;+ var k = 1.0;+ if nits >= disp.b.y {+ // above the source peak: clip to the target peak+ k = disp.b.z / disp.b.w / m;+ } else if nits > 1e-6 {+ k = eetf(nits, disp.b.y, disp.b.z) / nits;+ }+ c = c * k;+ }+ }+ if disp.a.w > 0.5 { c = vec3(dot(disp.m0.xyz, c), dot(disp.m1.xyz, c), dot(disp.m2.xyz, c)); }+ if disp.b.x > 0.5 {+ // `desaturate_into_gamut`: out-of-gamut colours towards their luminance, in-gamut exactly+ let mn = min(min(c.r, c.g), c.b);+ if mn < 0.0 {+ let y = dot(vec3(disp.m0.w, disp.m1.w, disp.m2.w), c);+ if y <= 0.0 {+ c = vec3(0.0);+ } else {+ c = max(y + (c - y) * (y / (y - mn)), vec3(0.0));+ }+ }+ }+ return c;+}+ @fragment fn fs_full(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {- let c = clamp(textureLoad(accum, vec2<i32>(pos.xy), 0).rgb, vec3(0.0), vec3(1.0));+ let c = clamp(display(textureLoad(accum, vec2<i32>(pos.xy), 0).rgb), vec3(0.0), vec3(1.0)); let lo = c * 12.92; let hi = 1.055 * pow(c, vec3(1.0 / 2.4)) - 0.055; return vec4(select(hi, lo, c <= vec3(0.0031308)), 1.0);@@ -710,6 +710,44 @@ fn layers_with_effects_fall_back_to_the_cpu_without_a_stage() { } } +/// A colour-managed layer (PQ media) with effects comes out the same with and without the GPU+/// effect stage: the CPU fallback decodes it through the layer's input transform too.+#[test]+fn cpu_effect_fallback_keeps_the_input_transform() {+ use filmcraft_color::{ColorInfo, ColorPipeline, ColorSpace, InputTransform, Primaries};+ let Some((dev, q)) = device() else {+ eprintln!("no GPU adapter; skipping");+ return;+ };+ let (w, h) = (256u32, 4u32);+ let y: Vec<u16> = (0..h).flat_map(|_| (0..w).map(|x| (64 + x * 876 / 255) as u16)).collect();+ let n = (w * h) as usize;+ let data = PixelData::Yuv16 { planes: [Arc::new(y), Arc::new(vec![480; n]), Arc::new(vec![560; n])], chroma: Chroma::C444, bits: 10, alpha: None };+ let color = ColorInfo { matrix: Matrix::Bt2020Ncl, transfer: Transfer::Pq, primaries: Primaries::Bt2020, range: Range::Limited };+ let frame = Arc::new(VideoFrame { width: w, height: h, data, color, par: (1, 1), pts: Default::default() });+ let t = Arc::new(InputTransform::new(ColorSpace::from_info(&color), color.range, &ColorPipeline::REC709, None));+ let e = effect("brightness_contrast", &[("brightness", fl(-10.0)), ("contrast", fl(20.0))]);+ let op = FxOp::eval(&e, &cx(Tick::ZERO, 1.0), w as usize, h as usize).expect("brightness / contrast op");+ let layer = PlanLayer {+ fx: Some(Arc::new(LayerFx { size: (w, h), decimation: 1, ops: vec![op] })),+ input: Some(t),+ ..PlanLayer::new(frame, Affine::IDENTITY, 1.0, Blend::Normal)+ };+ let plan = FramePlan::Layers { width: w as usize, height: h as usize, layers: vec![layer] };+ let cpu = execute_cpu(&plan).over_black_rgba8();+ for stage in [true, false] {+ let mut c = GpuCompositor::new(&dev, &q);+ if !stage {+ c.fx = None;+ }+ c.composite(&plan);+ let (_, _, gpu) = c.read_output().expect("readback");+ let (p99, mean, max) = stats8(&cpu, &gpu, &vec![true; n]);+ eprintln!("PQ layer with effects, effect stage {stage}: p99 {p99}, mean {mean:.2}, max {max}");+ assert!(p99 <= 6 && mean < 1.5, "stage {stage}: p99 {p99}, mean {mean}, max {max}");+ }+}+ /// The half-float upload is lossless for this input, so a 1:1 effect source draw must be too. /// Odd dimensions expose interpolation drift that can invent opacity before any effect runs. #[test]@@ -84,6 +84,8 @@ pub struct GpuCompositor { accum_readback_buf: Option<(wgpu::Buffer, u64)>, uploads: HashMap<(usize, u32, u32), Uploaded>, clock: u64,+ /// The resolve pass's display transform (`set_display_transform`), as plain numbers.+ display: [f32; DISPLAY_FLOATS], dummy: wgpu::TextureView, /// The effect stage (None on devices without compute shaders or storage textures: layers /// with effects are then rendered on the CPU here).@@ -253,6 +255,16 @@ pub fn prepare_frame(f: &VideoFrame) -> Option<Prepared> { } } +/// The display transform to resolve a *sequence* plan with: `display` (the sequence's,+/// `colorman::display_transform(..).gpu_params()`) for a plan composited in its working space,+/// none for a [`FramePlan::Image`]. That image is `render_sequence`'s output, already converted+/// for the monitor, and converting it again would tone map and gamut map it twice. (The CPU+/// images the compositor makes itself, for a transition it cannot mix, are in working space and+/// keep the transform: they are drawn inside a working-space plan.)+pub fn sequence_display(plan: &FramePlan, display: Option<filmcraft_color::GpuOutputParams>) -> Option<filmcraft_color::GpuOutputParams> {+ if matches!(plan, FramePlan::Image(_)) { None } else { display }+}+ /// Convert every layer of a plan for upload. Thread-safe and GPU-free: run it on a worker. pub fn prepare(plan: &FramePlan) -> PreparedPlan { match plan {@@ -314,7 +326,16 @@ impl GpuCompositor { }); let final_bgl = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor { label: Some("final"),- entries: &[wgpu::BindGroupLayoutEntry { binding: 0, ..tex_entry(0) }],+ entries: &[+ wgpu::BindGroupLayoutEntry { binding: 0, ..tex_entry(0) },+ // the resolve pass's display transform (`set_display_transform`)+ wgpu::BindGroupLayoutEntry {+ binding: 1,+ visibility: wgpu::ShaderStages::FRAGMENT,+ ty: wgpu::BindingType::Buffer { ty: wgpu::BufferBindingType::Uniform, has_dynamic_offset: false, min_binding_size: None },+ count: None,+ },+ ], }); let pl = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor { label: Some("layer"), bind_group_layouts: &[Some(&layer_bgl)], immediate_size: 0 });@@ -420,6 +441,7 @@ impl GpuCompositor { accum_readback_buf: None, uploads: HashMap::new(), clock: 0,+ display: [0.0; DISPLAY_FLOATS], dummy, fx, transition: transition::PushStage::new(device),@@ -699,11 +721,58 @@ impl GpuCompositor { buf } + /// The resolve pass's display transform uniform ([`set_display_transform`](Self::set_display_transform)).+ fn display_buffer(&self) -> wgpu::Buffer {+ let bytes: Vec<u8> = self.display.iter().flat_map(|v| v.to_le_bytes()).collect();+ let buf = self.device.create_buffer(&wgpu::BufferDescriptor {+ label: Some("display-u"),+ size: bytes.len() as u64,+ usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,+ mapped_at_creation: false,+ });+ self.queue.write_buffer(&buf, 0, &bytes);+ buf+ }+ /// Composite a plan; returns the output view (sRGB, over black) and its size. pub fn composite(&mut self, plan: &FramePlan) -> (wgpu::TextureView, (u32, u32)) { self.composite_prepared(plan, None) } + /// Set the resolve pass's display transform (the sequence's working space → monitor,+ /// `filmcraft_color::OutputTransform::gpu_params`). Layers are composited in the sequence's+ /// working linear space, so an HDR / wide-gamut sequence needs it to come out right; `None`+ /// (the default) leaves the accumulator as the monitor's own space, as for a plain sequence.+ /// Only the resolve pass uses it: a caller that reads the working-linear accumulator back+ /// (the export path) is unaffected.+ pub fn set_display_transform(&mut self, p: Option<filmcraft_color::GpuOutputParams>) {+ let Some(p) = p else {+ self.display = [0.0; DISPLAY_FLOATS];+ return;+ };+ let mx = p.matrix.unwrap_or([[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]]);+ let (sp, dp) = p.tone.unwrap_or((0.0, 0.0));+ let dl = p.compress.unwrap_or([0.0; 3]);+ let on = |b: bool| if b { 1.0 } else { 0.0 };+ let curve = match p.target.curve() {+ filmcraft_color::Curve::Sdr | filmcraft_color::Curve::Srgb => 0.0,+ filmcraft_color::Curve::Linear => 1.0,+ filmcraft_color::Curve::Pq => 2.0,+ filmcraft_color::Curve::Hlg => 3.0,+ // not reachable: `gpu_params` refuses a target the shader cannot encode+ filmcraft_color::Curve::Log(_) => 0.0,+ };+ #[rustfmt::skip]+ let d = [+ 1.0, curve, on(p.tone.is_some()), on(p.matrix.is_some()),+ on(p.compress.is_some()), sp, dp, filmcraft_color::REFERENCE_WHITE_NITS as f32,+ mx[0][0], mx[0][1], mx[0][2], dl[0],+ mx[1][0], mx[1][1], mx[1][2], dl[1],+ mx[2][0], mx[2][1], mx[2][2], dl[2],+ ];+ self.display = d;+ }+ /// [`composite`](Self::composite) with texel conversions already done by [`prepare`] (the /// result is identical; only the upload work on this thread differs). pub fn composite_prepared(&mut self, plan: &FramePlan, prep: Option<&PreparedPlan>) -> (wgpu::TextureView, (u32, u32)) {@@ -749,7 +818,9 @@ impl GpuCompositor { if self.fx.is_some() && lfx.size.0.max(lfx.size.1) <= max_side { return None; }- let img = filmcraft_render::plan::effect_image(&l.frame, lfx);+ // decoded through the layer's input transform, as `execute_cpu` does; the image is+ // then working-space linear, so the new layer has none+ let img = filmcraft_render::plan::effect_image_with(&l.frame, lfx, l.input.as_deref()); Some(PlanLayer::new(Arc::new(VideoFrame::rgba_f32(img.w as u32, img.h as u32, img.px)), l.matrix, l.opacity, l.blend)) }) .collect();@@ -858,10 +929,14 @@ impl GpuCompositor { if let Some(f) = self.fx.as_mut() { f.end_frame(); }+ let display_buf = self.display_buffer(); let final_bg = self.device.create_bind_group(&wgpu::BindGroupDescriptor { label: Some("final"), layout: &self.final_bgl,- entries: &[wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(&accum_view) }],+ entries: &[+ wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(&accum_view) },+ wgpu::BindGroupEntry { binding: 1, resource: display_buf.as_entire_binding() },+ ], }); let mut enc = self.device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("filmcraft-composite") }); // Runs of fixed-function layers draw in one pass. A layer that reads the destination@@ -1174,6 +1249,8 @@ struct SrcInfo { /// Floats in a layer's uniform block (`U` in `composite.wgsl`: 13 vec4). const UNIFORM_FLOATS: usize = 52;+/// Floats in the display transform uniform (`struct O` in `composite.wgsl`): 5 vec4.+const DISPLAY_FLOATS: usize = 20; /// Poll a future once: wgpu's native error scopes resolve immediately. `None` if it is still /// pending (then the caller assumes the worst).@@ -55,13 +55,18 @@ pub fn input_transform(cs: ColorSpace, range: Range, pipe: &ColorPipeline, peak_ } /// The input transform of a frame the GPU compositor can colour manage itself (#965): a Y'CbCr-/// frame of display-referred HDR or wide-gamut media (PQ, HLG, BT.2020, P3) on a plain pipeline,-/// whose working → monitor transform is the identity. None for log media (camera-gamut-/// compression), RGB frames and other pipelines, which [`decode`] converts on the CPU.+/// frame whose signal → working space stages the shader runs (PQ, HLG, BT.2020, P3; log curves and+/// camera-gamut compression are what it cannot). The sequence's working → monitor transform runs in+/// the compositor's resolve pass (`OutputTransform::gpu_params`), so an HDR working space is fine;+/// when the resolve pass cannot finish it, the whole frame stays on the CPU. None for log media,+/// RGB frames and pipelines the shader cannot finish, which [`decode`] converts on the CPU. pub fn gpu_input(project: &Project, item: ItemId, pipe: &ColorPipeline, cs: ColorSpace, frame: &VideoFrame) -> Option<Arc<InputTransform>> {- if !pipe.is_plain() || !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) {+ if !matches!(frame.data, filmcraft_frame::PixelData::Yuv8 { .. } | filmcraft_frame::PixelData::Yuv16 { .. }) { return None; }+ // The compositor's resolve pass finishes the frame with the sequence's display transform:+ // when the shader cannot encode that, the whole frame stays on the CPU.+ display_transform(pipe).gpu_params()?; let t = input_transform(cs, frame.color.range, pipe, source_peak_nits(project, item)); t.gpu_params().map(|_| t) }@@ -229,7 +229,13 @@ fn gpu_chain<'a>(project: &Project, item: &'a TrackItem, opts: RenderOptions) -> pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOptions, sources: &dyn SourceProvider) -> FramePlan { let Some(seq) = project.sequence(seq_id) else { return FramePlan::Image(crate::Image::new(1, 1)) }; let (w, h) = output_size(seq, opts.scale);- if whole_frame_on_cpu(project, seq, t) {+ // An HDR / wide-gamut sequence plans as layers too: its sources are decoded into the working+ // space (the shader's input transform for display-referred media, `colorman::decode` on the+ // CPU for log media and camera gamuts) and the compositor's resolve pass runs the sequence's+ // display transform (working space → monitor). Frames the compositor cannot finish stay on+ // the CPU: an adjustment layer, a transition it cannot mix, or a display transform the shader+ // cannot encode (an HDR target).+ if !layered_at(project, seq, t) || crate::colorman::display_transform(&seq.settings.color).gpu_params().is_none() { return FramePlan::Image(crate::render_sequence(project, seq_id, t, opts, sources)); } let mut layers = PlanStack::default();@@ -256,13 +262,17 @@ pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOption /// Whether [`plan_frame`] hands the frame at `t` back as one CPU image ([`FramePlan::Image`]). A /// GPU renderer gains nothing on such a frame, so a caller with only a few of them (the export's /// pool) can render it on the CPU without taking one.+/// Whether the frame at `t` needs the CPU compositor: an HDR / wide-gamut sequence, an adjustment+/// layer, or a transition the compositor cannot mix. Callers that render on the CPU only (the+/// export's pool) use it to skip taking a GPU plan; [`plan_frame`] itself plans an HDR / wide-gamut+/// frame as layers whenever the compositor can finish it. pub fn is_cpu_frame(project: &Project, seq_id: ItemId, t: Tick) -> bool { project.sequence(seq_id).is_none_or(|seq| whole_frame_on_cpu(project, seq, t)) } -/// The frames [`plan_frame`] does not plan: an HDR / wide-gamut sequence composites and converts-/// on the CPU, and so does a frame with an adjustment layer or a complex transition anywhere at-/// `t` (whole-frame fallback).+/// The frames the CPU-only path ([`is_cpu_frame`]) covers: an HDR / wide-gamut sequence (its+/// display transform and any log-source decode), an adjustment layer, or a complex transition+/// anywhere at `t` (whole-frame fallback). fn whole_frame_on_cpu(project: &Project, seq: &Sequence, t: Tick) -> bool { !seq.settings.color.is_plain() || !layered_at(project, seq, t) }@@ -46,6 +46,15 @@ impl FilmcraftApp { self.disable_gpu("device error"); return None; }+ // The resolve pass finishes a *sequence* frame: the sequence's working space → monitor+ // (`colorman::display_transform`). A plain sequence's transform is the identity, and item+ // frames (Source monitor, thumbnails) are already in their item's space.+ let pipe = match key.target {+ Target::Sequence(id) | Target::SequencePlan(id) => self.session.project.sequence(id).map(|q| q.settings.color),+ _ => None,+ };+ // A CPU-rendered frame (`FramePlan::Image`) is already in monitor space.+ let display = filmcraft_gpu::sequence_display(&plan.plan, pipe.and_then(|c| filmcraft_render::colorman::display_transform(&c).gpu_params())); let g = self.gpu.as_mut()?; if g.last_key == Some(key) && let Some(t) = g.texture@@ -57,6 +66,7 @@ impl FilmcraftApp { return None; } let t0 = web_time::Instant::now();+ g.compositor.set_display_transform(display); let ran = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { let (view, size) = g.compositor.composite_prepared(&plan.plan, Some(&plan.prepared)); (view.clone(), size)@@ -561,9 +561,12 @@ frame (Y'CbCr/RGB + metadata) ─► source colour space: Interpret Footage over - **Maths** in `filmcraft-color` (`transform`, `log`, `spaces`; formulas and sources in [crates/color/README.md](../crates/color/README.md)); the renderer side is `render::colorman`. - **Fast path.** A Rec. 709 sequence without wide gamut and media whose metadata says Rec. 709 /- sRGB decodes exactly as before, and its layers stay on the GPU. Log, HDR or wide-gamut media,- and every layer of an HDR/wide-gamut sequence, are converted on the CPU (the GPU path draws them- as pre-rendered images, like layers with effects).+ sRGB decodes exactly as before, and its layers stay on the GPU. Display-referred HDR or+ wide-gamut media (PQ, HLG, BT.2020, P3) is converted in the compositor's shader, and an HDR /+ wide-gamut sequence composites on the GPU too: the resolve pass runs its display transform+ (working space → monitor, `OutputTransform::gpu_params`); a display transform the shader cannot+ encode keeps the whole frame on the CPU. Log media and camera gamuts are converted on the CPU+ (the GPU path draws them as pre-rendered images, like layers with effects). - **Outputs.** `RenderOptions::working_output` returns working-space pixels (HDR exports, scopes); otherwise the top-level render is converted for an SDR monitor. The other colour effects work on display-encoded values clamped to 0..1 (they clip HDR highlights above@@ -146,8 +146,9 @@ Each codec README has the full fixture matrix and the measured results. to Black at 25 %, Wipe at 50 %, Timecode / Clip Name burn-in text, graphic clips (also a built-in graphics template with overridden properties and per-character styles with a pinned box), and colour management: S-Log3/S-Gamut3.Cine footage interpreted into Rec. 709 (`log_to_rec709`) and- Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`; the footage is demo frames- re-encoded by the test's `Encoded` source). Each frame is compared+ Rec. 2100 PQ footage tone mapped into Rec. 709 (`hdr_tone_map`), and Lumetri grading in a+ Rec. 2100 PQ sequence, tone mapped for the monitor (`lumetri_hdr_pq`); the footage of both is+ demo frames re-encoded by the test's `Encoded` source. Each frame is compared with `crates/golden/goldens/<scene>.png` by `filmcraft_testkit::golden`: **PSNR ≥ 45 dB, max abs ≤ 12, 99th-percentile per-pixel max channel difference ≤ 2** (8-bit sRGB levels; `Tolerance::RENDER`). On failure the actual frame and a ×8 difference image go to@@ -176,6 +177,9 @@ Each codec README has the full fixture matrix and the measured results. This found a real bug: the GPU upload cache was keyed by pixel-buffer address without keeping the buffer alive, so a new frame allocated at a freed frame's address was drawn with the stale texture (fixed; regression test `upload_cache_keeps_buffers_alive`).+- `gpu_matches_cpu_on_golden_scenes` sets each scene's display transform as the app does, so the+ colour-managed scenes (`hdr_tone_map`, and `lumetri_hdr_pq` in a Rec. 2100 PQ sequence) are+ converted and tone mapped on the GPU too. ## 4. UI: control channel and screenshots