collective
Browse / github.com/storytold/filmcraft / patch
h264: High 10 and High 4:2:2 (Intra) decoding — 9/10-bit and 4:2:2 pictures
h264: High 10 and High 4:2:2 (Intra) decoding — 9/10-bit and 4:2:2 pictures ## What H.264 **High 10** and **High 4:2:2 Intra** decoding in our own decoder, so 10-bit and 4:2:2 camera originals recorded as intra-only H.264 open everywhere in software. This covers Sony XAVC Intra (FX6, FX3, FX9, Venice …) and AVC-Intra, which so far showed the unreadable slate. Part of [G3](docs/gaps.md#g3-10-bit-and-422-camera-media-needs-hardware). Related to #626, but it does **not** close it: that Lumix file is long-GOP 4:2:2, and 4:2:2 *inter* prediction is still reported as unsupported (see "Not in this PR"). ## How The decoder is now generic over the sample storage type (`u8`
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- bysh71 (imported from pull request #3375, open)
- Base
8fda2f232198c4c79ff9cfe5b6a347bb1375fa9e- Size
- +986 −367 in 18 files
- Review
- pending
- Shared
- 2026-10-28
- Id
1159d1e878939ca6e88401a3
How it went
No other operator has reported using it yet.
Diff
raw · crates/codecs/src/video.rs crates/h264/README.md crates/h264/examples/h264dec.rs crates/h264/src/cabac_mb.rs crates/h264/src/deblock.rs crates/h264/src/decoder.rs crates/h264/src/inter.rs crates/h264/src/intra.rs crates/h264/src/lib.rs crates/h264/src/params.rs crates/h264/src/picture.rs crates/h264/src/slice.rs crates/h264/src/slicedec.rs crates/h264/src/transform.rs crates/h264/tests/it/high_bit_depth.rs crates/h264/tests/it/main.rs docs/codec-parity.md docs/gaps.md
@@ -306,25 +306,37 @@ impl H264Decoder { let draft = p.draft; let (w, h) = (p.width as usize, p.height as usize); let (cw, ch) = (p.chroma_width as usize, p.chroma_height as usize);- let y = tight_plane(p.y, p.y_stride, w, h);- let u = tight_plane(p.u, p.uv_stride, cw, ch);- let v = tight_plane(p.v, p.uv_stride, cw, ch);+ let chroma = if p.chroma_format_idc == 2 { filmcraft_frame::Chroma::C422 } else { filmcraft_frame::Chroma::C420 };+ let data = match p.wide {+ Some(wp) => filmcraft_frame::PixelData::Yuv16 {+ planes: [+ Arc::new(tight_plane(wp.y, p.y_stride, w, h)),+ Arc::new(tight_plane(wp.u, p.uv_stride, cw, ch)),+ Arc::new(tight_plane(wp.v, p.uv_stride, cw, ch)),+ ],+ chroma,+ bits: p.bit_depth,+ alpha: None,+ },+ None => filmcraft_frame::PixelData::Yuv8 {+ planes: [+ Arc::new(tight_plane(p.y, p.y_stride, w, h)),+ Arc::new(tight_plane(p.u, p.uv_stride, cw, ch)),+ Arc::new(tight_plane(p.v, p.uv_stride, cw, ch)),+ ],+ chroma,+ alpha: None,+ },+ }; let color = vui_color(p.width, p.height, p.color.primaries, p.color.matrix, p.color.transfer, p.color.full_range); let par = sar_par(p.sar);- let frame = VideoFrame {- width: p.width,- height: p.height,- data: filmcraft_frame::PixelData::Yuv8 { planes: [Arc::new(y), Arc::new(u), Arc::new(v)], chroma: filmcraft_frame::Chroma::C420, alpha: None },- color,- par,- pts: filmcraft_time::Tick::ZERO,- };+ let frame = VideoFrame { width: p.width, height: p.height, data, color, par, pts: filmcraft_time::Tick::ZERO }; DecodedFrame { pts: p.pts, frame, draft } } } /// An H.264 decoder error as a [`CodecError`]: a feature the decoder does not implement (High-/// 4:2:2, field pictures...) is [`CodecError::Unsupported`], so the media pool shows the+/// 4:4:4, field pictures...) is [`CodecError::Unsupported`], so the media pool shows the /// unreadable slate instead of a black picture (#626); anything else is a decode error. fn h264_error(e: filmcraft_h264::Error) -> CodecError { match e {@@ -373,6 +385,7 @@ impl VideoDecoder for H264Decoder { /// (`filmcraft_frame::pool`) instead of the allocator. fn recycle_h264_planes() { filmcraft_h264::set_plane_allocator(filmcraft_frame::pool::take_u8);+ filmcraft_h264::set_plane_allocator_u16(filmcraft_frame::pool::take_u16); } /// Worker threads each H.264 decoder uses (frame threading; 1 on wasm).@@ -19,7 +19,10 @@ oracles and fixture generators. - Slice header: all fields incl. reference list modification, prediction weight table and decoded reference picture marking. POC types 0, 1, 2. `frame_num` gap handling ("non-existing" frames).-- Entropy coding: CAVLC and CABAC (all syntax elements needed for frame coding, 4:2:0).+- Entropy coding: CAVLC and CABAC (all syntax elements needed for frame coding, 4:2:0 and 4:2:2).+- Sample formats: 8-bit 4:2:0; 9- and 10-bit 4:2:0 (High 10, intra and inter); 4:2:2 at 8–10 bits+ for intra pictures (High 4:2:2 Intra: Sony XAVC Intra, AVC-Intra). The decoder is generic over the+ sample storage (`u8` for 8-bit, `u16` above); high bit depth pictures come out in `Picture::wide`. - Macroblocks: I (Intra 4x4 / 8x8 / 16x16, I_PCM), P (all partitions, P_8x8ref0, P_Skip), B (all partitions and sub-partitions, B_Skip, B_Direct_16x16, B_Direct_8x8), spatial and temporal direct prediction with/without `direct_8x8_inference`, constrained intra prediction.@@ -203,21 +206,20 @@ file; `H264_BENCH_ITERS=n` (and `H264_THREADS=t`, `H264_DRAFT=1`) turns it into - Interlaced coding (field pictures, PAFF, MBAFF): streams with `frame_mbs_only_flag = 0` return `Error::Unsupported`.-- Only 8-bit 4:2:0. High 10 / 4:2:2 / 4:4:4 / monochrome and lossless- (`qpprime_y_zero_transform_bypass`) return `Error::Unsupported`.+- Inter prediction in 4:2:2 streams (long-GOP High 4:2:2), 4:4:4, monochrome, bit depths above 10,+ unequal luma / chroma bit depths and lossless (`qpprime_y_zero_transform_bypass`) return+ `Error::Unsupported`. - FMO (slice groups), SP/SI slices, data partitioning (Extended profile), MVC/SVC NAL units. - Error concealment is minimal: undecodable slices leave the prediction/previous content; missing references are replaced by other references. - No SIMD intrinsics (the hot loops are written to auto-vectorise); CABAC residual decoding is inherently serial and now dominates. -### Extending to High 10 and 4:2:2+### Extending to 4:2:2 inter prediction -The chroma format and bit depth are isolated behind `Sps::check_supported`. The parts that are-format-specific today: `Planes`/`FrameRow` sample type (`u8`; a `u16` variant or a generic sample-type is needed for bit depths > 8, together with `Clip1` ranges, `QpBdOffset` in the QP-derivations and the scaled tC0/alpha/beta in deblocking), chroma block geometry (`MbWidthC` /-`MbHeightC` = 8x8 in `slicedec`, `intra::pred_chroma`, the chroma DC transform and the CABAC/CAVLC-chroma DC parsing — the 2x4 total_zeros tables and ctxBlockCat 3 `NumC8x8` handling are already in-place), chroma motion-vector vertical scaling and the chroma deblocking edges (4:2:2 has four-horizontal chroma edges).+High 10 and 4:2:2 intra are in place (generic `Sample`, `Format` with bit depth and chroma lines per+macroblock). For 4:2:2 P/B pictures the remaining pieces are the chroma motion compensation+(vertical chroma vectors in full-resolution units: `yIntC = yAL + (mvCLX[1] >> 2)`,+`yFracC = (mvCLX[1] & 3) << 1`, 8.4.2.2.2), 4:2:2 chroma blocks in `mc_rect` / the weighted+prediction scratch, and inter bS for the horizontal chroma edges 4 and 12 with the 8x8 transform;+then drop the check in `Decoder::handle_slice`.@@ -1,4 +1,5 @@-//! Decode an Annex-B H.264 file to raw yuv420p: `h264dec in.h264 [out.yuv]`.+//! Decode an Annex-B H.264 file to raw planar YUV (yuv420p / yuv422p, 16-bit little-endian+//! samples above 8 bits): `h264dec in.h264 [out.yuv]`. //! Prints timing information to stderr. `H264_BENCH_ITERS=n` (with `H264_THREADS=t`, and //! `H264_DRAFT=1` for draft mode) turns it into a benchmark. @@ -35,9 +36,17 @@ fn main() { for p in pics { n += 1; if let Some(o) = out.as_mut() {- o.write_all(&p.y).unwrap();- o.write_all(&p.u).unwrap();- o.write_all(&p.v).unwrap();+ if let Some(w) = &p.wide {+ // little-endian 16-bit samples (yuv420p10le / yuv422p10le)+ for plane in [&w.y, &w.u, &w.v] {+ let bytes: Vec<u8> = plane.iter().flat_map(|v| v.to_le_bytes()).collect();+ o.write_all(&bytes).unwrap();+ }+ } else {+ o.write_all(&p.y).unwrap();+ o.write_all(&p.u).unwrap();+ o.write_all(&p.v).unwrap();+ } } } };@@ -4,7 +4,7 @@ use crate::cabac::Cabac; use crate::error::{Result, ensure, invalid}; use crate::mbtypes::*;-use crate::picture::MbKind;+use crate::picture::{MbKind, Sample}; use crate::slice::SliceType; use crate::slicedec::{MbCur, SliceDecoder}; @@ -33,11 +33,13 @@ const IDENT_CTX: [u8; 64] = { }; /// ctxIdxInc = Min(levelListIdx / NumC8x8, 2) for 4:2:0 chroma DC (ctxBlockCat 3). const CHROMA_DC_CTX: [u8; 4] = [0, 1, 2, 2];+/// The same for 4:2:2 chroma DC (NumC8x8 = 2).+const CHROMA422_DC_CTX: [u8; 8] = [0, 0, 1, 1, 2, 2, 2, 2]; /// Table 9-40 ctxIdxBlockCatOffset for ctxBlockCat 0..=5: (coded_block_flag, significant/last, abs level). const CAT_OFFSET: [(usize, usize, usize); 6] = [(0, 0, 0), (4, 15, 10), (8, 29, 20), (12, 44, 30), (16, 47, 39), (0, 0, 0)]; -impl SliceDecoder<'_> {+impl<S: Sample> SliceDecoder<'_, S> { /// Decode slice_data() with CABAC. `rbsp` is the whole slice RBSP. pub fn decode_cabac(&mut self, rbsp: &[u8]) -> Result<()> { let start = self.sh.header_bits.div_ceil(8) * 8;@@ -371,7 +373,8 @@ impl SliceDecoder<'_> { let n = if bx < 0 { self.nb[0] } else { self.nb[1] }; let Some(m) = n else { return intra_cur as usize }; let st = &self.pic.mbs[m];- let (x, y) = if bx < 0 { (1, by as usize) } else { (bx as usize, 1) };+ let bottom = self.fmt.chroma_blocks() / 2 - 1;+ let (x, y) = if bx < 0 { (1, by as usize) } else { (bx as usize, bottom) }; (st.nnz_c[comp][y * 2 + x] != 0) as usize } @@ -385,6 +388,7 @@ impl SliceDecoder<'_> { let (sig_base, last_base, abs_base) = if cat == 5 { (402, 417, 426) } else { (105 + CAT_OFFSET[cat].1, 166 + CAT_OFFSET[cat].1, 227 + CAT_OFFSET[cat].2) }; let (sig_tab, last_tab): (&[u8], &[u8]) = match cat {+ 3 if max_num == 8 => (&CHROMA422_DC_CTX, &CHROMA422_DC_CTX), 3 => (&CHROMA_DC_CTX, &CHROMA_DC_CTX), 5 => (&SIG8_FRAME, &LAST8), _ => (&IDENT_CTX, &IDENT_CTX),@@ -490,12 +494,13 @@ impl SliceDecoder<'_> { } } let cbp_c = cbp >> 4;+ let nb = self.fmt.chroma_blocks(); if cbp_c != 0 { for comp in 0..2 { let bit = 1 + comp as u8; let inc = self.cbf_dc_cond(self.nb[0], bit, intra) + 2 * self.cbf_dc_cond(self.nb[1], bit, intra);- let mut lv = [0i32; 4];- if self.residual_block_cabac(c, &mut lv, 3, 4, inc) > 0 {+ let mut lv = [0i32; 8];+ if self.residual_block_cabac(c, &mut lv, 3, nb, inc) > 0 { self.mb_mut().cbf_dc |= 1 << bit; self.put_chroma_dc(comp, &lv); }@@ -503,7 +508,7 @@ impl SliceDecoder<'_> { } if cbp_c == 2 { for comp in 0..2 {- for b in 0..4 {+ for b in 0..nb { let (bx, by) = ((b & 1) as i32, (b >> 1) as i32); let inc = self.cbf_chroma_ac_cond(comp, bx - 1, by, intra) + 2 * self.cbf_chroma_ac_cond(comp, bx, by - 1, intra); let mut lv = [0i32; 16];@@ -525,15 +530,14 @@ impl SliceDecoder<'_> { if info.kind == MbKind::IPcm { // pcm alignment + samples, then re-initialise the arithmetic decoder let pos = c.bit_pos().div_ceil(8);- let data = c.data();- ensure!(data.len() >= pos + 384, "truncated I_PCM macroblock");- let Some(&samples) = data.get(pos..).and_then(|d| d.first_chunk::<384>()) else {+ let n = self.pcm_bytes();+ let Some(samples) = c.data().get(pos..pos + n) else { return invalid("truncated I_PCM macroblock"); };- c.set_bit_pos((pos + 384) * 8);- c.init_engine()?; self.finish_pcm();- self.write_pcm(&samples);+ self.write_pcm(samples)?;+ c.set_bit_pos((pos + n) * 8);+ c.init_engine()?; self.prev_qp_delta_nz = false; return Ok(()); }@@ -1,6 +1,6 @@-//! Deblocking filter (8.7) for progressive frames, 4:2:0, 8-bit.+//! Deblocking filter (8.7) for progressive frames, 4:2:0 and 4:2:2, any bit depth. -use crate::picture::{MbKind, MbState};+use crate::picture::{MbKind, MbState, Sample}; use crate::slicedec::{PicState, SliceInfo}; use crate::tables::{ALPHA, BETA, TC0}; @@ -142,6 +142,8 @@ struct EdgeParams { tc0: [i32; 4], alpha: i32, beta: i32,+ /// Largest sample value.+ max: i32, } /// Filter one edge for `N` lines at once (lane `i` = line `i` across the edge). `s[k][i]` holds@@ -150,7 +152,7 @@ struct EdgeParams { #[inline(always)] fn filter_lanes<const N: usize>(s: &mut [[i16; N]; 8], ep: &EdgeParams, chroma: bool) { let per = N / 4;- let (alpha, beta) = (ep.alpha as i16, ep.beta as i16);+ let (alpha, beta, max) = (ep.alpha as i16, ep.beta as i16, ep.max as i16); let mut bs_on = [false; N]; let mut tc0 = [0i16; N]; for k in 0..4 {@@ -182,7 +184,7 @@ fn filter_lanes<const N: usize>(s: &mut [[i16; N]; 8], ep: &EdgeParams, chroma: } else { let tc = tc0[i] + 1; let delta = ((((q0[i] - p0[i]) << 2) + (p1[i] - q1[i]) + 4) >> 3).max(-tc).min(tc);- ((p0[i] + delta).clamp(0, 255), (q0[i] - delta).clamp(0, 255))+ ((p0[i] + delta).max(0).min(max), (q0[i] - delta).max(0).min(max)) }; s[3][i] = sel(on[i], np0, p0[i]); s[4][i] = sel(on[i], nq0, q0[i]);@@ -217,8 +219,8 @@ fn filter_lanes<const N: usize>(s: &mut [[i16; N]; 8], ep: &EdgeParams, chroma: let avg = (p0[i] + q0[i] + 1) >> 1; let np1 = p1[i] + ((p2[i] + avg - (p1[i] << 1)) >> 1).max(-t0).min(t0); let nq1 = q1[i] + ((q2[i] + avg - (q1[i] << 1)) >> 1).max(-t0).min(t0);- s[3][i] = sel(on[i], (p0[i] + delta).clamp(0, 255), p0[i]);- s[4][i] = sel(on[i], (q0[i] - delta).clamp(0, 255), q0[i]);+ s[3][i] = sel(on[i], (p0[i] + delta).max(0).min(max), p0[i]);+ s[4][i] = sel(on[i], (q0[i] - delta).max(0).min(max), q0[i]); s[2][i] = sel(on[i] & ap, np1, p1[i]); s[5][i] = sel(on[i] & aq, nq1, q1[i]); }@@ -227,60 +229,58 @@ fn filter_lanes<const N: usize>(s: &mut [[i16; N]; 8], ep: &EdgeParams, chroma: /// Filter a vertical edge whose q0 column is at `x`, for `N` lines starting at row `y`. #[inline(always)]-fn filter_vertical<const N: usize>(pix: &mut [u8], stride: usize, x: usize, y: usize, ep: &EdgeParams, chroma: bool) {- // Each line's eight samples as one little-endian word: byte k = sample k (p3 .. q3).- let mut w = [0u64; N];- for (i, w) in w.iter_mut().enumerate() {+fn filter_vertical<S: Sample, const N: usize>(pix: &mut [S], stride: usize, x: usize, y: usize, ep: &EdgeParams, chroma: bool) {+ let mut s = [[0i16; N]; 8];+ for i in 0..N { let o = (y + i) * stride + x - 4;- let Some(&b) = pix.get(o..).and_then(<[u8]>::first_chunk::<8>) else {+ let Some(line) = pix.get(o..).and_then(<[S]>::first_chunk::<8>) else { return; };- *w = u64::from_le_bytes(b);- }- let mut s = [[0i16; N]; 8];- for (k, row) in s.iter_mut().enumerate() {- for i in 0..N {- row[i] = ((w[i] >> (8 * k)) & 0xff) as i16;+ for (k, row) in s.iter_mut().enumerate() {+ row[i] = line[k].to_i16(); } } filter_lanes(&mut s, ep, chroma);- for (i, w) in w.iter_mut().enumerate() {- *w = (0..8).fold(0u64, |acc, k| acc | ((s[k][i] as u8 as u64) << (8 * k)));- }- for (i, w) in w.iter().enumerate() {+ let (k0, k1) = if chroma { (3, 5) } else { (1, 7) };+ for i in 0..N { let o = (y + i) * stride + x - 4;- pix[o..o + 8].copy_from_slice(&w.to_le_bytes());+ let Some(line) = pix.get_mut(o..).and_then(<[S]>::first_chunk_mut::<8>) else {+ return;+ };+ for k in k0..k1 {+ line[k] = S::clip(s[k][i] as i32, ep.max);+ } } } /// Filter a horizontal edge whose q0 row is `y`, for `N` columns starting at `x`. #[inline(always)]-fn filter_horizontal<const N: usize>(pix: &mut [u8], stride: usize, x: usize, y: usize, ep: &EdgeParams, chroma: bool) {+fn filter_horizontal<S: Sample, const N: usize>(pix: &mut [S], stride: usize, x: usize, y: usize, ep: &EdgeParams, chroma: bool) { let base = (y - 4) * stride + x; let mut s = [[0i16; N]; 8]; for (k, row) in s.iter_mut().enumerate() {- let Some(src) = pix.get(base + k * stride..).and_then(<[u8]>::first_chunk::<N>) else {+ let Some(src) = pix.get(base + k * stride..).and_then(<[S]>::first_chunk::<N>) else { return; }; for i in 0..N {- row[i] = src[i] as i16;+ row[i] = src[i].to_i16(); } } filter_lanes(&mut s, ep, chroma); let (k0, k1) = if chroma { (3, 5) } else { (1, 7) }; for (k, row) in s.iter().enumerate().take(k1).skip(k0) {- let Some(dst) = pix.get_mut(base + k * stride..).and_then(<[u8]>::first_chunk_mut::<N>) else {+ let Some(dst) = pix.get_mut(base + k * stride..).and_then(<[S]>::first_chunk_mut::<N>) else { return; }; for i in 0..N {- dst[i] = row[i] as u8;+ dst[i] = S::clip(row[i] as i32, ep.max); } } } /// Deblock one macroblock (macroblocks must be processed in raster order). #[allow(clippy::needless_range_loop)]-pub fn deblock_mb(pic: &mut PicState, addr: usize, mb_w: usize) {+pub fn deblock_mb<S: Sample>(pic: &mut PicState<S>, addr: usize, mb_w: usize) { let q = &pic.mbs[addr]; if q.slice_num == u32::MAX { return;@@ -311,9 +311,7 @@ pub fn deblock_mb(pic: &mut PicState, addr: usize, mb_w: usize) { bs[dir][0] = [4; 4]; } for e in 1..4 {- if !(t8 && e % 2 == 1) {- bs[dir][e] = [3; 4];- }+ bs[dir][e] = [3; 4]; } } } else {@@ -343,9 +341,6 @@ pub fn deblock_mb(pic: &mut PicState, addr: usize, mb_w: usize) { continue; } for e in 1..4 {- if t8 && e % 2 == 1 {- continue;- } for k in 0..4 { let (rq, rp) = if dir == 0 { (k * 4 + e, k * 4 + e - 1) } else { (e * 4 + k, e * 4 + k - 4) }; bs[dir][e][k] = if (q.nz_mask >> rp) & 1 != 0 || (q.nz_mask >> rq) & 1 != 0 { 2 } else { motion_bs(&qm[rp], &qm[rq]) };@@ -354,6 +349,10 @@ pub fn deblock_mb(pic: &mut PicState, addr: usize, mb_w: usize) { } } let qp_of = |st: &MbState| if st.kind == MbKind::IPcm { 0 } else { st.qp as i32 };+ let fmt = pic.planes.fmt;+ // alpha, beta and tC0 scale with the bit depth (8-232, 8-233, 8-235)+ let scale = 1 << (fmt.bit_depth - 8);+ let max = fmt.max(); let q_qp = qp_of(q); let q_qpc = q.qpc; let p_qp = [left.map(|n| (qp_of(&pic.mbs[n]), pic.mbs[n].qpc)), top.map(|n| (qp_of(&pic.mbs[n]), pic.mbs[n].qpc))];@@ -362,14 +361,21 @@ pub fn deblock_mb(pic: &mut PicState, addr: usize, mb_w: usize) { let index_a = (qpav + alpha_off).clamp(0, 51) as usize; let index_b = (qpav + beta_off).clamp(0, 51) as usize; let tc = |b: u8| if (1..4).contains(&b) { TC0[index_a][b as usize - 1] as i32 } else { 0 };- EdgeParams { bs: b, tc0: [tc(b[0]), tc(b[1]), tc(b[2]), tc(b[3])], alpha: ALPHA[index_a] as i32, beta: BETA[index_b] as i32 }+ EdgeParams {+ bs: b,+ tc0: [tc(b[0]) * scale, tc(b[1]) * scale, tc(b[2]) * scale, tc(b[3]) * scale],+ alpha: ALPHA[index_a] as i32 * scale,+ beta: BETA[index_b] as i32 * scale,+ max,+ } }; let width = pic.planes.width; let cwidth = pic.planes.cwidth; // luma for dir in 0..2 { for e in 0..4 {- if bs[dir][e] == [0; 4] {+ // with the 8x8 transform only the edges of 8x8 blocks are luma transform edges+ if bs[dir][e] == [0; 4] || (t8 && e % 2 == 1) { continue; } let qpp = if e == 0 { p_qp[dir].map(|p| p.0).unwrap_or(q_qp) } else { q_qp };@@ -380,30 +386,39 @@ pub fn deblock_mb(pic: &mut PicState, addr: usize, mb_w: usize) { continue; } if dir == 0 {- filter_vertical::<16>(&mut pic.planes.y, width, mx * 16 + e * 4, my * 16, &ep, false);+ filter_vertical::<S, 16>(&mut pic.planes.y, width, mx * 16 + e * 4, my * 16, &ep, false); } else {- filter_horizontal::<16>(&mut pic.planes.y, width, mx * 16, my * 16 + e * 4, &ep, false);+ filter_horizontal::<S, 16>(&mut pic.planes.y, width, mx * 16, my * 16 + e * 4, &ep, false); } } }- // chroma (4:2:0): edges at chroma samples 0 and 4 use the bS of luma edges 0 and 2+ // Chroma edges use the bS of the luma edge at the corresponding luma position (8.7.2.1):+ // vertical chroma edges 0 / 4 are luma edges 0 / 2; horizontal chroma edges 0 / 4 are luma+ // edges 0 / 2 in 4:2:0, and edges 0 / 4 / 8 / 12 luma edges 0..=3 in 4:2:2 (chroma blocks+ // always use the 4x4 transform).+ let hc = fmt.mb_hc; for c in 0..2 { for dir in 0..2 {- for ce in 0..2 {- let e = ce * 2;+ let n_edges = if dir == 1 && hc == 16 { 4 } else { 2 };+ for ce in 0..n_edges {+ let e = if n_edges == 4 { ce } else { ce * 2 }; if bs[dir][e] == [0; 4] { continue; }- let qpp = if e == 0 { p_qp[dir].map(|p| p.1[c]).unwrap_or(q_qpc[c]) } else { q_qpc[c] } as i32;+ let qpp = if ce == 0 { p_qp[dir].map(|p| p.1[c]).unwrap_or(q_qpc[c]) } else { q_qpc[c] } as i32; let ep = params(qpp, q_qpc[c] as i32, bs[dir][e]); if ep.alpha == 0 || ep.beta == 0 { continue; } let plane = if c == 0 { &mut pic.planes.cb } else { &mut pic.planes.cr }; if dir == 0 {- filter_vertical::<8>(plane, cwidth, mx * 8 + ce * 4, my * 8, &ep, true);+ if hc == 16 {+ filter_vertical::<S, 16>(plane, cwidth, mx * 8 + ce * 4, my * 16, &ep, true);+ } else {+ filter_vertical::<S, 8>(plane, cwidth, mx * 8 + ce * 4, my * 8, &ep, true);+ } } else {- filter_horizontal::<8>(plane, cwidth, mx * 8, my * 8 + ce * 4, &ep, true);+ filter_horizontal::<S, 8>(plane, cwidth, mx * 8, my * hc + ce * 4, &ep, true); } } }@@ -432,7 +447,7 @@ mod tests { let index_a = (next() % 52) as usize; let index_b = (next() % 52) as usize; let tc = |b: u8| if (1..4).contains(&b) { TC0[index_a][b as usize - 1] as i32 } else { 0 };- let ep = EdgeParams { bs, tc0: bs.map(tc), alpha: ALPHA[index_a] as i32, beta: BETA[index_b] as i32 };+ let ep = EdgeParams { bs, tc0: bs.map(tc), alpha: ALPHA[index_a] as i32, beta: BETA[index_b] as i32, max: 255 }; let base = (next() % 256) as i32; let spread = 1 + (next() % 24) as i32; let mut lines = [[0u8; 8]; 16];@@ -484,9 +499,9 @@ mod speed { let (w, h) = (1024usize, 256usize); let mut pix: Vec<u8> = (0..w * h).map(|i| (128 + ((i * 7919) % 13) as i32 - 6) as u8).collect(); let eps = [- EdgeParams { bs: [4; 4], tc0: [0; 4], alpha: 40, beta: 9 },- EdgeParams { bs: [3; 4], tc0: [2; 4], alpha: 40, beta: 9 },- EdgeParams { bs: [2, 1, 0, 2], tc0: [1, 1, 0, 1], alpha: 40, beta: 9 },+ EdgeParams { bs: [4; 4], tc0: [0; 4], alpha: 40, beta: 9, max: 255 },+ EdgeParams { bs: [3; 4], tc0: [2; 4], alpha: 40, beta: 9, max: 255 },+ EdgeParams { bs: [2, 1, 0, 2], tc0: [1, 1, 0, 1], alpha: 40, beta: 9, max: 255 }, ]; let t0 = std::time::Instant::now(); let mut n = 0;@@ -494,7 +509,7 @@ mod speed { for my in 1..h / 16 { for mx in 1..w / 16 { for (j, ep) in eps.iter().enumerate() {- filter_horizontal::<16>(&mut pix, w, mx * 16, my * 16 + j * 4, ep, false);+ filter_horizontal::<u8, 16>(&mut pix, w, mx * 16, my * 16 + j * 4, ep, false); n += 1; } }@@ -506,7 +521,7 @@ mod speed { for my in 1..h / 16 { for mx in 1..w / 16 { for (j, ep) in eps.iter().enumerate() {- filter_vertical::<16>(&mut pix, w, mx * 16 + j * 4, my * 16, ep, false);+ filter_vertical::<u8, 16>(&mut pix, w, mx * 16 + j * 4, my * 16, ep, false); } } }@@ -9,11 +9,11 @@ use crate::dpb::{Dpb, Output, OutputMeta, RefMark}; use crate::error::{Error, Result, ensure, invalid, unsupported}; use crate::params::{Pps, Sps};-use crate::picture::{Frame, FrameRef, MbKind, MbState, Planes, RefPic};+use crate::picture::{Format, Frame, FrameRef, MbKind, MbState, Planes, RefPic, Sample}; use crate::slice::{NalHeader, Poc, PocState, SliceHeader, SliceType, is_new_picture, nal_type}; use crate::slicedec::{PicState, SliceDecoder}; use crate::transform::LevelScale;-use crate::{ColorInfo, Picture};+use crate::{ColorInfo, Picture, WidePlanes}; use filmcraft_bitstream::{BitReader, annexb_nals, length_prefixed_nals, unescape_rbsp}; use std::collections::VecDeque; use std::sync::{Arc, Mutex};@@ -76,8 +76,31 @@ pub struct DecodeStats { struct Shared { error: Mutex<Option<Error>>, stats: Mutex<DecodeStats>,- /// Recycled picture buffers.- pool: Mutex<Vec<PicState>>,+ /// Recycled picture buffers (8-bit and high bit depth pictures).+ pool: Mutex<Vec<PicState<u8>>>,+ pool16: Mutex<Vec<PicState<u16>>>,+}++/// The recycled picture buffers of one sample type.+trait PoolSample: Sample {+ fn pool(shared: &Shared) -> &Mutex<Vec<PicState<Self>>>;+}++impl PoolSample for u8 {+ fn pool(shared: &Shared) -> &Mutex<Vec<PicState<u8>>> {+ &shared.pool+ }+}++impl PoolSample for u16 {+ fn pool(shared: &Shared) -> &Mutex<Vec<PicState<u16>>> {+ &shared.pool16+ }+}++/// Sample format of the pictures of a (supported) SPS.+fn format_of(sps: &Sps) -> Format {+ Format { bit_depth: sps.bit_depth_luma, mb_hc: if sps.chroma_array_type() == 2 { 16 } else { 8 } } } /// H.264 decoder.@@ -349,6 +372,9 @@ impl Decoder { if matches!(sh.slice_type, SliceType::Sp | SliceType::Si) { return unsupported("SP/SI slices"); }+ if sps.chroma_array_type() == 2 && !sh.slice_type.is_intra() {+ return unsupported("inter prediction in 4:2:2 streams (only intra pictures are implemented)");+ } if sh.redundant_pic_cnt > 0 { return Ok(()); // redundant slices are ignored }@@ -408,7 +434,7 @@ impl Decoder { // Activate SPS; a resolution change flushes the DPB. let changed = match &self.active_sps { None => true,- Some(a) => a.width() != sps.width() || a.height() != sps.height() || a.max_dpb_frames() != sps.max_dpb_frames(),+ Some(a) => a.width() != sps.width() || a.height() != sps.height() || a.max_dpb_frames() != sps.max_dpb_frames() || format_of(a) != format_of(sps), }; if changed && self.active_sps.is_some() { let mut outs = Vec::new();@@ -421,6 +447,7 @@ impl Decoder { self.dpb.max_reorder = sps.max_num_reorder_frames().min(self.dpb.capacity); let mb_w = sps.pic_width_in_mbs as usize; let mb_h = sps.frame_height_in_mbs() as usize;+ let fmt = format_of(sps); if sh.idr { let mut outs = Vec::new(); self.dpb.idr(sh.no_output_of_prior_pics, &mut outs);@@ -442,18 +469,12 @@ impl Decoder { let mut make = |_fnum: u32| { let id = *next_id; *next_id += 1;- let planes = match &last {- Some(f) if f.mb_w == mb_w && f.mb_h() == mb_h => {- let mut p = Planes::new(mb_w * 16, mb_h * 16);- let (y, u, v) = f.copy_cropped((0, 0, mb_w * 16, mb_h * 16));- p.y = y;- p.cb = u;- p.cr = v;- p- }- _ => Planes::gray(mb_w * 16, mb_h * 16),+ let frame = if fmt.wide() {+ concealment_frame::<u16>(last.as_deref(), id, mb_w, mb_h, fmt)+ } else {+ concealment_frame::<u8>(last.as_deref(), id, mb_w, mb_h, fmt) };- (Arc::new(Frame::from_planes(id, 0, &planes)), 0, meta.clone())+ (Arc::new(frame), 0, meta.clone()) }; let mut upd = |fnum: u32| poc_state.update_gap_frame(fnum, sps); self.shared.stats.lock().map(|mut s| s.frame_num_gaps += 1).ok();@@ -466,7 +487,7 @@ impl Decoder { let frame_poc = if sh.has_mmco5() { 0 } else { poc.frame() }; let id = self.next_id; self.next_id += 1;- let frame = Arc::new(Frame::new(id, frame_poc, mb_w, mb_h));+ let frame = Arc::new(Frame::new(id, frame_poc, mb_w, mb_h, fmt)); self.pending = Some(PendingPic { frame, sps: sps.clone(), first: sh.clone(), poc, pts, key: sh.idr, slices: Vec::new() }); Ok(()) }@@ -511,11 +532,36 @@ impl Decoder { } } +/// A "non-existing" frame for a frame_num gap: a copy of `last` (when it has the same size and+/// format), else grey.+fn concealment_frame<S: Sample>(last: Option<&Frame>, id: u32, mb_w: usize, mb_h: usize, fmt: Format) -> Frame {+ let mut p = Planes::<S>::gray(mb_w * 16, mb_h * 16, fmt);+ if let Some(f) = last+ && f.mb_w == mb_w+ && f.mb_h() == mb_h+ && f.fmt == fmt+ && let Some((y, u, v)) = f.copy_cropped::<S>((0, 0, mb_w * 16, mb_h * 16))+ {+ p.y = y;+ p.cb = u;+ p.cr = v;+ }+ Frame::from_planes(id, 0, &p)+}+ /// Decode all slices of one picture and publish its rows. fn run_job(frame: FrameRef, slices: Vec<SliceJob>, shared: &Shared, draft: bool) {- let reuse = shared.pool.lock().ok().and_then(|mut p| p.pop());- let mut pic = match reuse {- Some(mut p) if p.mb_w == frame.mb_w && p.mb_h == frame.mb_h() => {+ if frame.fmt.wide() {+ run_job_typed::<u16>(frame, slices, shared, draft);+ } else {+ run_job_typed::<u8>(frame, slices, shared, draft);+ }+}++fn run_job_typed<S: PoolSample>(frame: FrameRef, slices: Vec<SliceJob>, shared: &Shared, draft: bool) {+ let reuse = S::pool(shared).lock().ok().and_then(|mut p| p.pop());+ let mut pic: PicState<S> = match reuse {+ Some(mut p) if p.mb_w == frame.mb_w && p.mb_h == frame.mb_h() && p.planes.fmt == frame.fmt => { p.reset(frame.clone()); p }@@ -536,7 +582,7 @@ fn run_job(frame: FrameRef, slices: Vec<SliceJob>, shared: &Shared, draft: bool) Ok(e) => e, Err(_) => { // Make sure readers never block on this frame.- let fallback = Planes::gray(frame.width, frame.height);+ let fallback = Planes::<S>::gray(frame.width, frame.height, frame.fmt); let blank = vec![MbState::default(); frame.mb_w * frame.mb_h()]; for r in 0..frame.mb_h() { if !frame.is_published(r) {@@ -570,14 +616,14 @@ fn run_job(frame: FrameRef, slices: Vec<SliceJob>, shared: &Shared, draft: bool) if let (Some(e), Ok(mut slot)) = (err, shared.error.lock()) { slot.get_or_insert(e); }- if let Ok(mut p) = shared.pool.lock()+ if let Ok(mut p) = S::pool(shared).lock() && p.len() < 4 { p.push(pic); } } -fn decode_slice(s: &SliceJob, pic: &mut PicState) -> Result<()> {+fn decode_slice<S: Sample>(s: &SliceJob, pic: &mut PicState<S>) -> Result<()> { let mut sd = SliceDecoder::new(&s.sh, &s.pps, &s.sps, pic, &s.refs, &s.ls)?; if s.pps.entropy_coding_mode { sd.decode_cabac(&s.rbsp)@@ -606,9 +652,19 @@ fn output_meta(sps: &Sps, pts: i64, key: bool) -> OutputMeta { fn make_picture(o: &Output<FrameRef>) -> Picture { let (cx, cy, cw, ch) = o.meta.crop; let (cx, cy, cw, ch) = (cx as usize, cy as usize, cw as usize, ch as usize);- let (y, u, v) = o.frame.copy_cropped((cx, cy, cw, ch));- let (ccw, cch) = (cw.div_ceil(2), ch.div_ceil(2));+ let fmt = o.frame.fmt;+ let crop = (cx, cy, cw, ch);+ let ((y, u, v), wide) = if fmt.wide() {+ let (y, u, v) = o.frame.copy_cropped::<u16>(crop).unwrap_or_default();+ ((Vec::new(), Vec::new(), Vec::new()), Some(WidePlanes { y, u, v }))+ } else {+ (o.frame.copy_cropped::<u8>(crop).unwrap_or_default(), None)+ };+ let (ccw, cch) = (cw.div_ceil(2), if fmt.mb_hc == 16 { ch } else { ch.div_ceil(2) }); Picture {+ wide,+ bit_depth: fmt.bit_depth,+ chroma_format_idc: fmt.chroma_format_idc(), width: cw as u32, height: ch as u32, chroma_width: ccw as u32,@@ -1,5 +1,7 @@ //! Inter prediction sample interpolation (8.4.2.2) and weighted sample prediction (8.4.2.3). +use crate::picture::Sample;+ /// A reference plane (used by the plane-based helpers in tests). #[cfg(test)] #[derive(Clone, Copy)]@@ -45,48 +47,49 @@ fn tap6(a: i32, b: i32, c: i32, d: i32, e: i32, f: i32) -> i32 { a - 5 * b + 20 * c + 20 * d - 5 * e + f } +#[cfg(test)] #[inline(always)] fn clip1(v: i32) -> u8 { v.clamp(0, 255) as u8 } -/// Copy `n` bytes with a fixed-size move for the common block widths.+/// Copy `n` samples with a fixed-size move for the common block widths. #[inline(always)]-fn copy_n(dst: &mut [u8], src: &[u8], n: usize) {- fn fixed<const N: usize>(dst: &mut [u8], src: &[u8]) {+fn copy_n<S: Sample>(dst: &mut [S], src: &[S], n: usize) {+ fn fixed<S: Sample, const N: usize>(dst: &mut [S], src: &[S]) { if let (Some(d), Some(s)) = (dst.first_chunk_mut::<N>(), src.first_chunk::<N>()) { *d = *s; } } match n {- 16 => fixed::<16>(dst, src),- 8 => fixed::<8>(dst, src),- 4 => fixed::<4>(dst, src),- 2 => fixed::<2>(dst, src),+ 16 => fixed::<S, 16>(dst, src),+ 8 => fixed::<S, 8>(dst, src),+ 4 => fixed::<S, 4>(dst, src),+ 2 => fixed::<S, 2>(dst, src), _ => dst[..n].copy_from_slice(&src[..n]), } } #[inline(always)]-fn avg(a: u8, b: u8) -> u8 {- ((a as u16 + b as u16 + 1) >> 1) as u8+fn avg<S: Sample>(a: S, b: S, max: i32) -> S {+ S::clip((a.to_i32() + b.to_i32() + 1) >> 1, max) } /// Horizontal 6-tap (unclipped) at every position of `row` for `n` outputs: out[c] uses row[c..c + 6]. #[inline(always)]-fn htap_row(row: &[u8], out: &mut [i32], n: usize) {+fn htap_row<S: Sample>(row: &[S], out: &mut [i32], n: usize) { let row = &row[..n + 5]; for (o, w) in out[..n].iter_mut().zip(row.windows(6)) {- *o = tap6(w[0] as i32, w[1] as i32, w[2] as i32, w[3] as i32, w[4] as i32, w[5] as i32);+ *o = tap6(w[0].to_i32(), w[1].to_i32(), w[2].to_i32(), w[3].to_i32(), w[4].to_i32(), w[5].to_i32()); } } /// Vertical 6-tap (unclipped) over six rows at columns 0..n. #[inline(always)]-fn vtap_row(rows: [&[u8]; 6], out: &mut [i32], n: usize) {+fn vtap_row<S: Sample>(rows: [&[S]; 6], out: &mut [i32], n: usize) { let [a, b, c, d, e, f] = rows.map(|r| &r[..n]); for i in 0..n {- out[i] = tap6(a[i] as i32, b[i] as i32, c[i] as i32, d[i] as i32, e[i] as i32, f[i] as i32);+ out[i] = tap6(a[i].to_i32(), b[i].to_i32(), c[i].to_i32(), d[i].to_i32(), e[i].to_i32(), f[i].to_i32()); } } @@ -104,13 +107,15 @@ pub fn mc_luma(p: PlaneRef, x: i32, y: i32, fx: u32, fy: u32, bw: usize, bh: usi fetch_window(p, x, y, bw, bh, &mut win); (&win[..], WIN) };- mc_luma_win(src, ss, fx, fy, bw, bh, out, os);+ mc_luma_win(src, ss, fx, fy, bw, bh, out, os, 255); } /// Luma interpolation from a prepared window: `src[0]` is the sample at (x - 2, y - 2) and `ss` the /// window stride; the window covers (bw + 5) x (bh + 5) samples. #[allow(clippy::too_many_arguments)]-pub fn mc_luma_win(src: &[u8], ss: usize, fx: u32, fy: u32, bw: usize, bh: usize, out: &mut [u8], os: usize) {+pub fn mc_luma_win<S: Sample>(src: &[S], ss: usize, fx: u32, fy: u32, bw: usize, bh: usize, out: &mut [S], os: usize, max: i32) {+ let clip1 = |v: i32| S::clip(v, max);+ let avg = |a: S, b: S| avg(a, b, max); // window row r (0..bh+5) starts at src[r * ss]; sample G of block (r, c) is at window (r + 2, c + 2) let wrow = |r: usize| &src[r * ss..r * ss + bw + 5]; let mut t = [0i32; 16];@@ -236,14 +241,14 @@ pub fn mc_chroma(p: PlaneRef, x: i32, y: i32, fx: u32, fy: u32, bw: usize, bh: u } (&win[..], 9) };- mc_chroma_win(src, ss, fx, fy, bw, bh, out, os);+ mc_chroma_win(src, ss, fx, fy, bw, bh, out, os, 255); } /// Chroma interpolation from a prepared (bw + 1) x (bh + 1) window whose first sample is the integer /// position of the block. #[allow(clippy::too_many_arguments)]-pub fn mc_chroma_win(src: &[u8], ss: usize, fx: u32, fy: u32, bw: usize, bh: usize, out: &mut [u8], os: usize) {- let (fx, fy) = (fx as u16, fy as u16);+pub fn mc_chroma_win<S: Sample>(src: &[S], ss: usize, fx: u32, fy: u32, bw: usize, bh: usize, out: &mut [S], os: usize, max: i32) {+ let (fx, fy) = (fx as i32, fy as i32); let (w00, w10, w01, w11) = ((8 - fx) * (8 - fy), fx * (8 - fy), (8 - fx) * fy, fx * fy); if fx == 0 && fy == 0 { for r in 0..bh {@@ -256,8 +261,8 @@ pub fn mc_chroma_win(src: &[u8], ss: usize, fx: u32, fy: u32, bw: usize, bh: usi let r1 = &src[(r + 1) * ss..(r + 1) * ss + bw + 1]; let o = &mut out[r * os..r * os + bw]; for (c, o) in o.iter_mut().enumerate() {- let v = w00 * r0[c] as u16 + w10 * r0[c + 1] as u16 + w01 * r1[c] as u16 + w11 * r1[c + 1] as u16;- *o = ((v + 32) >> 6) as u8;+ let v = w00 * r0[c].to_i32() + w10 * r0[c + 1].to_i32() + w01 * r1[c].to_i32() + w11 * r1[c + 1].to_i32();+ *o = S::clip((v + 32) >> 6, max); } } }@@ -273,7 +278,8 @@ pub enum Weight { /// Combine prediction(s) into `dst` (stride `ds`). `p0`/`p1` have stride `ps`. #[allow(clippy::too_many_arguments)]-pub fn weighted_store(dst: &mut [u8], ds: usize, p0: Option<&[u8]>, p1: Option<&[u8]>, ps: usize, bw: usize, bh: usize, w: Weight) {+pub fn weighted_store<S: Sample>(dst: &mut [S], ds: usize, p0: Option<&[S]>, p1: Option<&[S]>, ps: usize, bw: usize, bh: usize, w: Weight, max: i32) {+ let clip1 = |v: i32| S::clip(v, max); match (p0, p1, w) { (Some(a), Some(b), Weight::Default) => { for r in 0..bh {@@ -281,7 +287,7 @@ pub fn weighted_store(dst: &mut [u8], ds: usize, p0: Option<&[u8]>, p1: Option<& let ra = &a[r * ps..r * ps + bw]; let rb = &b[r * ps..r * ps + bw]; for ((d, &x), &y) in d.iter_mut().zip(ra).zip(rb) {- *d = ((x as u32 + y as u32 + 1) >> 1) as u8;+ *d = S::clip((x.to_i32() + y.to_i32() + 1) >> 1, max); } } }@@ -294,7 +300,7 @@ pub fn weighted_store(dst: &mut [u8], ds: usize, p0: Option<&[u8]>, p1: Option<& let ra = &a[r * ps..r * ps + bw]; let rb = &b[r * ps..r * ps + bw]; for ((d, &x), &y) in d.iter_mut().zip(ra).zip(rb) {- *d = clip1(((x as i32 * w0 + y as i32 * w1 + round) >> sh) + off);+ *d = clip1(((x.to_i32() * w0 + y.to_i32() * w1 + round) >> sh) + off); } } }@@ -314,7 +320,7 @@ pub fn weighted_store(dst: &mut [u8], ds: usize, p0: Option<&[u8]>, p1: Option<& let d = &mut dst[r * ds..r * ds + bw]; let ra = &a[r * ps..r * ps + bw]; for (d, &x) in d.iter_mut().zip(ra) {- *d = clip1(((x as i32 * wt + round) >> log_wd) + o);+ *d = clip1(((x.to_i32() * wt + round) >> log_wd) + o); } } }@@ -505,9 +511,9 @@ mod tests { let a = [100u8; 4]; let b = [50u8; 4]; let mut d = [0u8; 4];- weighted_store(&mut d, 4, Some(&a), Some(&b), 4, 4, 1, Weight::Default);+ weighted_store(&mut d, 4, Some(&a), Some(&b), 4, 4, 1, Weight::Default, 255); assert_eq!(d, [75; 4]);- weighted_store(&mut d, 4, Some(&a), None, 4, 4, 1, Weight::Weighted { log_wd: 5, w0: 16, w1: 0, o0: 3, o1: 0 });+ weighted_store(&mut d, 4, Some(&a), None, 4, 4, 1, Weight::Weighted { log_wd: 5, w0: 16, w1: 0, o0: 3, o1: 0 }, 255); assert_eq!(d, [53; 4]); } }@@ -1,4 +1,7 @@-//! Intra prediction (8.3): Intra_4x4, Intra_8x8 (with reference filtering), Intra_16x16 and chroma.+//! Intra prediction (8.3): Intra_4x4, Intra_8x8 (with reference filtering), Intra_16x16 and chroma+//! (4:2:0 and 4:2:2), for any sample type / bit depth.++use crate::picture::{Format, Sample}; /// Neighbouring sample availability for one block. #[derive(Clone, Copy, Debug, Default)]@@ -30,16 +33,16 @@ impl Edge { } } -fn load_edge(plane: &[u8], stride: usize, x: usize, y: usize, n: usize, a: Avail) -> Edge {- let mut e = Edge { top: [128; 16], left: [128; 8], tl: 128 };+fn load_edge<S: Sample>(plane: &[S], stride: usize, x: usize, y: usize, n: usize, a: Avail, mid: i32) -> Edge {+ let mut e = Edge { top: [mid; 16], left: [mid; 8], tl: mid }; if a.top { let row = &plane[(y - 1) * stride + x..]; for i in 0..n {- e.top[i] = row[i] as i32;+ e.top[i] = row[i].to_i32(); } if a.top_right { for i in n..2 * n {- e.top[i] = row[i] as i32;+ e.top[i] = row[i].to_i32(); } } else { let v = e.top[n - 1];@@ -50,18 +53,19 @@ fn load_edge(plane: &[u8], stride: usize, x: usize, y: usize, n: usize, a: Avail } if a.left { for i in 0..n {- e.left[i] = plane[(y + i) * stride + x - 1] as i32;+ e.left[i] = plane[(y + i) * stride + x - 1].to_i32(); } } if a.top_left {- e.tl = plane[(y - 1) * stride + x - 1] as i32;+ e.tl = plane[(y - 1) * stride + x - 1].to_i32(); } e } /// Directional / DC prediction shared by Intra_4x4 and Intra_8x8 (8.3.1.2.x, 8.3.2.2.x).-fn pred_nxn(mode: u8, e: &Edge, n: i32, a: Avail, out: &mut [u8], stride: usize) {- let mut put = |x: i32, y: i32, v: i32| out[y as usize * stride + x as usize] = v as u8;+fn pred_nxn<S: Sample>(mode: u8, e: &Edge, n: i32, a: Avail, out: &mut [S], stride: usize, fmt: Format) {+ let max = fmt.max();+ let mut put = |x: i32, y: i32, v: i32| out[y as usize * stride + x as usize] = S::clip(v, max); match mode { 0 => { for y in 0..n {@@ -85,7 +89,7 @@ fn pred_nxn(mode: u8, e: &Edge, n: i32, a: Avail, out: &mut [u8], stride: usize) (true, true) => (st + sl + n) >> (shift + 1), (false, true) => (sl + (n >> 1)) >> shift, (true, false) => (st + (n >> 1)) >> shift,- _ => 128,+ _ => fmt.mid(), }; for y in 0..n { for x in 0..n {@@ -185,14 +189,14 @@ fn pred_nxn(mode: u8, e: &Edge, n: i32, a: Avail, out: &mut [u8], stride: usize) } /// Intra_4x4 prediction written into `plane` at (x, y).-pub fn pred4x4(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, a: Avail) {- let e = load_edge(plane, stride, x, y, 4, a);- pred_nxn(mode, &e, 4, a, &mut plane[y * stride + x..], stride);+pub fn pred4x4<S: Sample>(plane: &mut [S], stride: usize, x: usize, y: usize, mode: u8, a: Avail, fmt: Format) {+ let e = load_edge(plane, stride, x, y, 4, a, fmt.mid());+ pred_nxn(mode, &e, 4, a, &mut plane[y * stride + x..], stride, fmt); } /// Intra_8x8 prediction (with reference sample filtering) written into `plane` at (x, y).-pub fn pred8x8(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, a: Avail) {- let p = load_edge(plane, stride, x, y, 8, a);+pub fn pred8x8<S: Sample>(plane: &mut [S], stride: usize, x: usize, y: usize, mode: u8, a: Avail, fmt: Format) {+ let p = load_edge(plane, stride, x, y, 8, a, fmt.mid()); let mut f = p; if a.top { f.top[0] = if a.top_left { (p.tl + 2 * p.top[0] + p.top[1] + 2) >> 2 } else { (3 * p.top[0] + p.top[1] + 2) >> 2 };@@ -216,26 +220,22 @@ pub fn pred8x8(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, a: } f.left[7] = (p.left[6] + 3 * p.left[7] + 2) >> 2; }- pred_nxn(mode, &f, 8, a, &mut plane[y * stride + x..], stride);-}--#[inline(always)]-fn clip(v: i32) -> u8 {- v.clamp(0, 255) as u8+ pred_nxn(mode, &f, 8, a, &mut plane[y * stride + x..], stride, fmt); } /// Intra_16x16 prediction (8.3.3) for the macroblock at luma (x, y).-pub fn pred16x16(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, a: Avail) {+pub fn pred16x16<S: Sample>(plane: &mut [S], stride: usize, x: usize, y: usize, mode: u8, a: Avail, fmt: Format) {+ let max = fmt.max(); let mut top = [0i32; 16]; let mut left = [0i32; 16]; if a.top { for i in 0..16 {- top[i] = plane[(y - 1) * stride + x + i] as i32;+ top[i] = plane[(y - 1) * stride + x + i].to_i32(); } } if a.left { for i in 0..16 {- left[i] = plane[(y + i) * stride + x - 1] as i32;+ left[i] = plane[(y + i) * stride + x - 1].to_i32(); } } let base = y * stride + x;@@ -243,13 +243,13 @@ pub fn pred16x16(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, 0 => { for r in 0..16 { for c in 0..16 {- plane[base + r * stride + c] = top[c] as u8;+ plane[base + r * stride + c] = S::clip(top[c], max); } } } 1 => { for r in 0..16 {- plane[base + r * stride..base + r * stride + 16].fill(left[r] as u8);+ plane[base + r * stride..base + r * stride + 16].fill(S::clip(left[r], max)); } } 2 => {@@ -259,14 +259,15 @@ pub fn pred16x16(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, (true, true) => (st + sl + 16) >> 5, (false, true) => (sl + 8) >> 4, (true, false) => (st + 8) >> 4,- _ => 128,- } as u8;+ _ => fmt.mid(),+ };+ let dc = S::clip(dc, max); for r in 0..16 { plane[base + r * stride..base + r * stride + 16].fill(dc); } } _ => {- let tl = if a.top_left { plane[(y - 1) * stride + x - 1] as i32 } else { 128 };+ let tl = if a.top_left { plane[(y - 1) * stride + x - 1].to_i32() } else { fmt.mid() }; let t = |i: i32| if i < 0 { tl } else { top[i as usize] }; let l = |i: i32| if i < 0 { tl } else { left[i as usize] }; let mut h = 0;@@ -280,34 +281,35 @@ pub fn pred16x16(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, let c = (5 * v + 32) >> 6; for r in 0..16 { for cc in 0..16 {- plane[base + r * stride + cc] = clip((aa + b * (cc as i32 - 7) + c * (r as i32 - 7) + 16) >> 5);+ plane[base + r * stride + cc] = S::clip((aa + b * (cc as i32 - 7) + c * (r as i32 - 7) + 16) >> 5, max); } } } } } -/// Chroma intra prediction for a 4:2:0 macroblock (8x8 block at chroma (x, y)), 8.3.4.-/// Modes: 0 DC, 1 horizontal, 2 vertical, 3 plane.-pub fn pred_chroma(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8, a: Avail) {+/// Chroma intra prediction (8.3.4) for the 8 x `fmt.mb_hc` chroma block of a macroblock at+/// chroma (x, y). Modes: 0 DC, 1 horizontal, 2 vertical, 3 plane.+pub fn pred_chroma<S: Sample>(plane: &mut [S], stride: usize, x: usize, y: usize, mode: u8, a: Avail, fmt: Format) { const W: usize = 8;- const H: usize = 8;+ let h = fmt.mb_hc.min(16);+ let (max, mid) = (fmt.max(), fmt.mid()); let mut top = [0i32; W];- let mut left = [0i32; H];+ let mut left = [0i32; 16]; if a.top { for i in 0..W {- top[i] = plane[(y - 1) * stride + x + i] as i32;+ top[i] = plane[(y - 1) * stride + x + i].to_i32(); } } if a.left {- for i in 0..H {- left[i] = plane[(y + i) * stride + x - 1] as i32;+ for i in 0..h {+ left[i] = plane[(y + i) * stride + x - 1].to_i32(); } } let base = y * stride + x; match mode { 0 => {- for by in 0..H / 4 {+ for by in 0..h / 4 { for bx in 0..W / 4 { let st: i32 = top[bx * 4..bx * 4 + 4].iter().sum(); let sl: i32 = left[by * 4..by * 4 + 4].iter().sum();@@ -316,7 +318,7 @@ pub fn pred_chroma(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8 (true, true) => (st + sl + 4) >> 3, (false, true) => (sl + 2) >> 2, (true, false) => (st + 2) >> 2,- _ => 128,+ _ => mid, } } else if bx > 0 { if a.top {@@ -324,50 +326,54 @@ pub fn pred_chroma(plane: &mut [u8], stride: usize, x: usize, y: usize, mode: u8 } else if a.left { (sl + 2) >> 2 } else {- 128+ mid } } else if a.left { (sl + 2) >> 2 } else if a.top { (st + 2) >> 2 } else {- 128+ mid }; for r in 0..4 { let o = base + (by * 4 + r) * stride + bx * 4;- plane[o..o + 4].fill(dc as u8);+ plane[o..o + 4].fill(S::clip(dc, max)); } } } } 1 => {- for r in 0..H {- plane[base + r * stride..base + r * stride + W].fill(left[r] as u8);+ for r in 0..h {+ plane[base + r * stride..base + r * stride + W].fill(S::clip(left[r], max)); } } 2 => {- for r in 0..H {+ for r in 0..h { for c in 0..W {- plane[base + r * stride + c] = top[c] as u8;+ plane[base + r * stride + c] = S::clip(top[c], max); } } } _ => {- let tl = if a.top_left { plane[(y - 1) * stride + x - 1] as i32 } else { 128 };+ let tl = if a.top_left { plane[(y - 1) * stride + x - 1].to_i32() } else { fmt.mid() }; let t = |i: i32| if i < 0 { tl } else { top[i as usize] }; let l = |i: i32| if i < 0 { tl } else { left[i as usize] };- let mut h = 0;+ // yCF = 4 for 4:2:2 (8-138..8-141)+ let ycf = (h as i32 - 8) / 2;+ let mut hh = 0; let mut v = 0; for k in 0..4 {- h += (k + 1) * (t(4 + k) - t(2 - k));- v += (k + 1) * (l(4 + k) - l(2 - k));+ hh += (k + 1) * (t(4 + k) - t(2 - k));+ }+ for k in 0..4 + ycf {+ v += (k + 1) * (l(4 + ycf + k) - l(2 + ycf - k)); }- let aa = 16 * (left[H - 1] + top[W - 1]);- let b = (34 * h + 32) >> 6;- let c = (34 * v + 32) >> 6;- for r in 0..H {+ let aa = 16 * (left[h - 1] + top[W - 1]);+ let b = (34 * hh + 32) >> 6;+ let c = ((34 - 29 * (h == 16) as i32) * v + 32) >> 6;+ for r in 0..h { for cc in 0..W {- plane[base + r * stride + cc] = clip((aa + b * (cc as i32 - 3) + c * (r as i32 - 3) + 16) >> 5);+ plane[base + r * stride + cc] = S::clip((aa + b * (cc as i32 - 3) + c * (r as i32 - 3 - ycf) + 16) >> 5, max); } } }@@ -392,16 +398,17 @@ mod tests { (p, stride) } const ALL: Avail = Avail { left: true, top: true, top_left: true, top_right: true };+ const F: Format = Format::YUV420_8; #[test] fn i4_vertical_horizontal_dc() { let (mut p, s) = setup();- pred4x4(&mut p, s, 4, 4, 0, ALL);+ pred4x4(&mut p, s, 4, 4, 0, ALL, F); assert_eq!(&p[4 * s + 4..4 * s + 8], &[10, 20, 30, 40]); assert_eq!(&p[7 * s + 4..7 * s + 8], &[10, 20, 30, 40]);- pred4x4(&mut p, s, 4, 4, 1, ALL);+ pred4x4(&mut p, s, 4, 4, 1, ALL, F); assert_eq!(&p[5 * s + 4..5 * s + 8], &[2, 2, 2, 2]);- pred4x4(&mut p, s, 4, 4, 2, ALL);+ pred4x4(&mut p, s, 4, 4, 2, ALL, F); // (10+20+30+40 + 1+2+3+4 + 4) >> 3 = 114 >> 3 = 14 assert_eq!(p[4 * s + 4], 14); }@@ -409,7 +416,7 @@ mod tests { #[test] fn i4_diag_down_left_hand_computed() { let (mut p, s) = setup();- pred4x4(&mut p, s, 4, 4, 3, ALL);+ pred4x4(&mut p, s, 4, 4, 3, ALL, F); // pred[0,0] = (t0 + 2 t1 + t2 + 2) >> 2 = (10 + 40 + 30 + 2) >> 2 = 20 assert_eq!(p[4 * s + 4], 20); // pred[3,3] = (t6 + 3 t7 + 2) >> 2 = (70 + 240 + 2) >> 2 = 78@@ -419,7 +426,7 @@ mod tests { #[test] fn i4_diag_down_right_hand_computed() { let (mut p, s) = setup();- pred4x4(&mut p, s, 4, 4, 4, ALL);+ pred4x4(&mut p, s, 4, 4, 4, ALL, F); // x==y: (t0 + 2 tl + l0 + 2) >> 2 = (10 + 10 + 1 + 2) >> 2 = 5 assert_eq!(p[4 * s + 4], 5); // x=1,y=0: (tl + 2 t0 + t1 + 2)>>2 = (5 + 20 + 20 + 2) >> 2 = 11@@ -432,7 +439,7 @@ mod tests { fn i4_top_right_substitution() { let (mut p, s) = setup(); let a = Avail { top_right: false, ..ALL };- pred4x4(&mut p, s, 4, 4, 3, a);+ pred4x4(&mut p, s, 4, 4, 3, a, F); // top-right replaced by t3 = 40: pred[3,3] = (40 + 120 + 2) >> 2 = 40 assert_eq!(p[7 * s + 7], 40); }@@ -441,7 +448,7 @@ mod tests { fn i16_plane_flat_is_flat() { let s = 32; let mut p = vec![77u8; s * 32];- pred16x16(&mut p, s, 8, 8, 3, ALL);+ pred16x16(&mut p, s, 8, 8, 3, ALL, F); assert!(p[8 * s + 8..8 * s + 24].iter().all(|&v| v == 77)); } @@ -453,7 +460,7 @@ mod tests { p[3 * s + 4 + i] = if i < 4 { 40 } else { 80 }; p[(4 + i) * s + 3] = if i < 4 { 8 } else { 16 }; }- pred_chroma(&mut p, s, 4, 4, 0, ALL);+ pred_chroma(&mut p, s, 4, 4, 0, ALL, F); assert_eq!(p[4 * s + 4], (160 + 32 + 4) >> 3); // top-left block: both assert_eq!(p[4 * s + 8], 80); // top-right block: top only assert_eq!(p[8 * s + 4], 16); // bottom-left: left only@@ -1,7 +1,8 @@ //! Clean-room pure-Rust H.264 / AVC decoder, implemented from ITU-T Rec. H.264 (ISO/IEC 14496-10). //! //! Supported: progressive (frame) coding, 8-bit 4:2:0, Baseline / Main / High profiles — CAVLC and CABAC,-//! I/P/B slices, 8x8 transform, custom scaling matrices, weighted prediction, spatial/temporal direct,+//! I/P/B slices; High 10 (4:2:0, 9/10-bit) and High 4:2:2 / High 4:2:2 Intra (4:2:2 intra pictures,+//! 8–10-bit: Sony XAVC Intra, AVC-Intra), 8x8 transform, custom scaling matrices, weighted prediction, spatial/temporal direct, //! multiple slices, long-term references and all MMCOs, deblocking. //! //! ```no_run@@ -35,7 +36,7 @@ mod intra; mod mbtypes; pub mod params; mod picture;-pub use picture::set_plane_allocator;+pub use picture::{set_plane_allocator, set_plane_allocator_u16}; pub mod slice; mod slicedec; #[cfg(test)]@@ -56,9 +57,23 @@ pub struct ColorInfo { pub matrix: u8, } -/// A decoded, cropped picture in 8-bit planar 4:2:0.+/// The planes of a picture with more than 8 bits per sample.+#[derive(Clone, Debug, Default)]+pub struct WidePlanes {+ pub y: Vec<u16>,+ pub u: Vec<u16>,+ pub v: Vec<u16>,+}++/// A decoded, cropped picture: planar 4:2:0 or 4:2:2, 8-bit samples in `y`, `u`, `v`, higher bit+/// depths in `wide` (`y`, `u`, `v` are then empty). #[derive(Clone, Debug)] pub struct Picture {+ /// Samples of a high bit depth picture (`bit_depth` > 8).+ pub wide: Option<WidePlanes>,+ pub bit_depth: u32,+ /// 1 (4:2:0) or 2 (4:2:2).+ pub chroma_format_idc: u32, /// Cropped luma width. pub width: u32, /// Cropped luma height.@@ -526,11 +526,14 @@ impl Sps { if !self.frame_mbs_only { return unsupported("interlaced (field / MBAFF) coding"); }- if self.chroma_array_type() != 1 {- return unsupported(format!("chroma_format_idc {} (only 4:2:0 is implemented)", self.chroma_format_idc));+ if !matches!(self.chroma_array_type(), 1 | 2) {+ return unsupported(format!("chroma_format_idc {} (only 4:2:0 and 4:2:2 are implemented)", self.chroma_format_idc)); }- if self.bit_depth_luma != 8 || self.bit_depth_chroma != 8 {- return unsupported(format!("bit depth luma {} chroma {} (only 8-bit is implemented)", self.bit_depth_luma, self.bit_depth_chroma));+ if self.bit_depth_luma != self.bit_depth_chroma || !(8..=10).contains(&self.bit_depth_luma) {+ return unsupported(format!(+ "bit depth luma {} chroma {} (8 to 10 bits, equal for luma and chroma, are implemented)",+ self.bit_depth_luma, self.bit_depth_chroma+ )); } if self.qpprime_y_zero_transform_bypass { return unsupported("qpprime_y_zero_transform_bypass (lossless)");@@ -2,25 +2,141 @@ use std::sync::{Arc, OnceLock}; -/// Planar 4:2:0 8-bit sample buffers (MB-aligned dimensions, stride == width).+/// Storage type of decoded samples: `u8` for 8-bit streams (half the memory and cache traffic of+/// 16-bit storage), `u16` for higher bit depths. Values are always within `0..1 << bit_depth`.+pub trait Sample: Copy + Default + PartialEq + Eq + std::fmt::Debug + Send + Sync + 'static {+ fn to_i32(self) -> i32;+ fn to_i16(self) -> i16;+ /// `v` clipped to `0..=max` (`max` = (1 << bit_depth) - 1; always 255 for `u8`).+ fn clip(v: i32, max: i32) -> Self;+ /// The samples of a published row if they are stored with this type.+ fn row_planes(s: &RowSamples) -> Option<&RowPlanes<Self>>;+ fn wrap_row(p: RowPlanes<Self>) -> RowSamples;+ /// An empty output plane with room for `len` samples (see [`set_plane_allocator`]).+ fn alloc_plane(len: usize) -> Vec<Self>;+}++impl Sample for u8 {+ #[inline(always)]+ fn to_i32(self) -> i32 {+ self as i32+ }+ #[inline(always)]+ fn to_i16(self) -> i16 {+ self as i16+ }+ #[inline(always)]+ fn clip(v: i32, _max: i32) -> Self {+ v.clamp(0, 255) as u8+ }+ #[inline(always)]+ fn row_planes(s: &RowSamples) -> Option<&RowPlanes<u8>> {+ match s {+ RowSamples::U8(p) => Some(p),+ RowSamples::U16(_) => None,+ }+ }+ fn wrap_row(p: RowPlanes<u8>) -> RowSamples {+ RowSamples::U8(p)+ }+ fn alloc_plane(len: usize) -> Vec<u8> {+ PLANE_ALLOCATOR.get().copied().unwrap_or(Vec::with_capacity)(len)+ }+}++impl Sample for u16 {+ #[inline(always)]+ fn to_i32(self) -> i32 {+ self as i32+ }+ #[inline(always)]+ fn to_i16(self) -> i16 {+ self as i16+ }+ #[inline(always)]+ fn clip(v: i32, max: i32) -> Self {+ v.max(0).min(max) as u16+ }+ #[inline(always)]+ fn row_planes(s: &RowSamples) -> Option<&RowPlanes<u16>> {+ match s {+ RowSamples::U16(p) => Some(p),+ RowSamples::U8(_) => None,+ }+ }+ fn wrap_row(p: RowPlanes<u16>) -> RowSamples {+ RowSamples::U16(p)+ }+ fn alloc_plane(len: usize) -> Vec<u16> {+ PLANE_ALLOCATOR_U16.get().copied().unwrap_or(Vec::with_capacity)(len)+ }+}++/// Sample layout of a picture: bit depth and chroma format (4:2:0 or 4:2:2).+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+pub struct Format {+ /// Bit depth of luma and chroma samples (8..=10).+ pub bit_depth: u32,+ /// Chroma lines per macroblock: 8 (4:2:0) or 16 (4:2:2).+ pub mb_hc: usize,+}++impl Format {+ #[cfg(test)]+ pub const YUV420_8: Format = Format { bit_depth: 8, mb_hc: 8 };++ /// Largest sample value.+ #[inline(always)]+ pub fn max(self) -> i32 {+ (1 << self.bit_depth) - 1+ }+ /// The mid-grey sample value (unavailable intra neighbours, 8.3.1.2).+ #[inline(always)]+ pub fn mid(self) -> i32 {+ 1 << (self.bit_depth - 1)+ }+ /// QpBdOffset (7.4.2.1.1).+ #[inline(always)]+ pub fn qp_bd_offset(self) -> i32 {+ 6 * (self.bit_depth as i32 - 8)+ }+ /// 4x4 chroma blocks per component and macroblock (4 or 8).+ #[inline(always)]+ pub fn chroma_blocks(self) -> usize {+ self.mb_hc / 2+ }+ /// chroma_format_idc (1 or 2).+ pub fn chroma_format_idc(self) -> u32 {+ if self.mb_hc == 16 { 2 } else { 1 }+ }+ /// Whether samples are stored as `u16`.+ pub fn wide(self) -> bool {+ self.bit_depth > 8+ }+}++/// Planar 4:2:0 / 4:2:2 sample buffers (MB-aligned dimensions, stride == width). #[derive(Clone)]-pub struct Planes {- pub y: Vec<u8>,- pub cb: Vec<u8>,- pub cr: Vec<u8>,+pub struct Planes<S> {+ pub y: Vec<S>,+ pub cb: Vec<S>,+ pub cr: Vec<S>, pub width: usize, pub height: usize, pub cwidth: usize,+ pub fmt: Format, } -impl Planes {- pub fn new(width: usize, height: usize) -> Self {- let (cw, ch) = (width / 2, height / 2);- Planes { y: vec![0; width * height], cb: vec![128; cw * ch], cr: vec![128; cw * ch], width, height, cwidth: cw }+impl<S: Sample> Planes<S> {+ /// Black luma, neutral chroma.+ pub fn new(width: usize, height: usize, fmt: Format) -> Self {+ let (cw, ch) = (width / 2, height / 16 * fmt.mb_hc);+ let mid = S::clip(fmt.mid(), fmt.max());+ Planes { y: vec![S::default(); width * height], cb: vec![mid; cw * ch], cr: vec![mid; cw * ch], width, height, cwidth: cw, fmt } }- pub fn gray(width: usize, height: usize) -> Self {- let mut p = Self::new(width, height);- p.y.fill(128);+ pub fn gray(width: usize, height: usize, fmt: Format) -> Self {+ let mut p = Self::new(width, height, fmt);+ p.y.fill(S::clip(fmt.mid(), fmt.max())); p } }@@ -57,16 +173,17 @@ pub struct MbState { pub transform_8x8: bool, /// coded_block_pattern: bits 0..3 luma, bits 4..5 chroma. pub cbp: u8,- pub qp: u8,+ /// QPY (negative values occur at high bit depths).+ pub qp: i8, /// QPc for Cb / Cr (used by deblocking).- pub qpc: [u8; 2],+ pub qpc: [i8; 2], pub intra_chroma_mode: u8, /// Intra4x4PredMode / Intra8x8PredMode per 4x4 block (raster order). pub intra_modes: [u8; 16], /// CAVLC: TotalCoeff per luma 4x4 block (raster). CABAC: coded_block_flag. pub nnz: [u8; 16],- /// Same for chroma AC blocks [Cb, Cr][raster 2x2].- pub nnz_c: [[u8; 4]; 2],+ /// Same for chroma AC blocks [Cb, Cr][raster 2 x 2 (4:2:0) or 2 x 4 (4:2:2)].+ pub nnz_c: [[u8; 8]; 2], /// CABAC coded_block_flag of DC blocks: bit0 luma (I16x16), bit1 Cb, bit2 Cr. pub cbf_dc: u8, /// Luma 4x4 blocks (raster bit) with non-zero coefficients, for deblocking bS = 2.@@ -99,7 +216,7 @@ impl Default for MbState { intra_chroma_mode: 0, intra_modes: [2; 16], nnz: [0; 16],- nnz_c: [[0; 4]; 2],+ nnz_c: [[0; 8]; 2], cbf_dc: 0, nz_mask: 0, ref_idx: [[-1; 4]; 2],@@ -110,14 +227,25 @@ impl Default for MbState { } } +/// The samples of one published macroblock row.+pub struct RowPlanes<S> {+ /// 16 luma lines.+ pub y: Box<[S]>,+ /// `mb_hc` lines per chroma component.+ pub cb: Box<[S]>,+ pub cr: Box<[S]>,+}++/// [`RowPlanes`] in the storage type of the picture.+pub enum RowSamples {+ U8(RowPlanes<u8>),+ U16(RowPlanes<u16>),+}+ /// One published macroblock row of a decoded frame: final (deblocked) samples plus the motion data /// needed as co-located information for direct prediction. pub struct FrameRow {- /// 16 luma lines.- pub y: Box<[u8]>,- /// 8 lines per chroma component.- pub cb: Box<[u8]>,- pub cr: Box<[u8]>,+ pub samples: RowSamples, /// Per 4x4 block: index mb_x * 16 + raster. pub mv: [Box<[[i16; 2]]>; 2], /// Per 8x8 block: index mb_x * 4 + b8.@@ -139,12 +267,23 @@ pub struct Frame { pub cwidth: usize, pub cheight: usize, pub mb_w: usize,+ pub fmt: Format, rows: Box<[OnceLock<FrameRow>]>, } impl Frame {- pub fn new(id: u32, poc: i32, mb_w: usize, mb_h: usize) -> Self {- Frame { id, poc, width: mb_w * 16, height: mb_h * 16, cwidth: mb_w * 8, cheight: mb_h * 8, mb_w, rows: (0..mb_h).map(|_| OnceLock::new()).collect() }+ pub fn new(id: u32, poc: i32, mb_w: usize, mb_h: usize, fmt: Format) -> Self {+ Frame {+ id,+ poc,+ width: mb_w * 16,+ height: mb_h * 16,+ cwidth: mb_w * 8,+ cheight: mb_h * fmt.mb_hc,+ mb_w,+ fmt,+ rows: (0..mb_h).map(|_| OnceLock::new()).collect(),+ } } pub fn mb_h(&self) -> usize {@@ -181,13 +320,14 @@ impl Frame { } /// Build a published row from a contiguous plane set (rows of MB row `r`) and motion data.- pub fn make_row(planes: &Planes, r: usize, mbs: &[MbState], ref_ids: &dyn Fn(&MbState, usize, i8) -> u32) -> FrameRow {+ pub fn make_row<S: Sample>(planes: &Planes<S>, r: usize, mbs: &[MbState], ref_ids: &dyn Fn(&MbState, usize, i8) -> u32) -> FrameRow { let w = planes.width; let cw = planes.cwidth;+ let hc = planes.fmt.mb_hc; let mb_w = w / 16; let y = planes.y[r * 16 * w..(r + 1) * 16 * w].into();- let cb = planes.cb[r * 8 * cw..(r + 1) * 8 * cw].into();- let cr = planes.cr[r * 8 * cw..(r + 1) * 8 * cw].into();+ let cb = planes.cb[r * hc * cw..(r + 1) * hc * cw].into();+ let cr = planes.cr[r * hc * cw..(r + 1) * hc * cw].into(); let row_mbs = &mbs[r * mb_w..(r + 1) * mb_w]; let mut mv = [vec![[0i16; 2]; mb_w * 16], vec![[0i16; 2]; mb_w * 16]]; let mut ref_idx = [vec![-1i8; mb_w * 4], vec![-1i8; mb_w * 4]];@@ -212,14 +352,20 @@ impl Frame { let [mv0, mv1] = mv; let [ri0, ri1] = ref_idx; let [id0, id1] = ref_id;- FrameRow { y, cb, cr, mv: [mv0.into(), mv1.into()], ref_idx: [ri0.into(), ri1.into()], ref_id: [id0.into(), id1.into()], intra: intra.into() }+ FrameRow {+ samples: S::wrap_row(RowPlanes { y, cb, cr }),+ mv: [mv0.into(), mv1.into()],+ ref_idx: [ri0.into(), ri1.into()],+ ref_id: [id0.into(), id1.into()],+ intra: intra.into(),+ } } /// A frame with every row published from `planes` and no motion (intra), e.g. "non-existing" /// frames inferred for frame_num gaps.- pub fn from_planes(id: u32, poc: i32, planes: &Planes) -> Self {+ pub fn from_planes<S: Sample>(id: u32, poc: i32, planes: &Planes<S>) -> Self { let (mb_w, mb_h) = (planes.width / 16, planes.height / 16);- let f = Frame::new(id, poc, mb_w, mb_h);+ let f = Frame::new(id, poc, mb_w, mb_h, planes.fmt); let blank = vec![MbState::default(); mb_w * mb_h]; for r in 0..mb_h { f.publish(r, Frame::make_row(planes, r, &blank, &|_, _, _| u32::MAX));@@ -228,68 +374,81 @@ impl Frame { } /// Copy the w x h luma window at (x0, y0) (clamped to the picture) into `out` (stride `os`).+ /// A frame stored with another sample type (only after a format change) reads as mid-grey. #[inline]- pub fn luma_window(&self, x0: i32, y0: i32, w: usize, h: usize, out: &mut [u8], os: usize) {+ pub fn luma_window<S: Sample>(&self, x0: i32, y0: i32, w: usize, h: usize, out: &mut [S], os: usize) { let (width, height) = (self.width, self.height); for r in 0..h { let yy = (y0 + r as i32).clamp(0, height as i32 - 1) as usize;- let row = self.row(yy >> 4);+ let Some(row) = S::row_planes(&self.row(yy >> 4).samples) else {+ out[r * os..r * os + w].fill(S::clip(self.fmt.mid(), self.fmt.max()));+ continue;+ }; let line = &row.y[(yy & 15) * width..(yy & 15) * width + width];- copy_line::<24>(line, x0, &mut out[r * os..], w);+ copy_line::<S, 24>(line, x0, &mut out[r * os..], w); } } /// Copy the w x h window of chroma component `c` at (x0, y0) (clamped) into `out`. #[inline]- pub fn chroma_window(&self, c: usize, x0: i32, y0: i32, w: usize, h: usize, out: &mut [u8], os: usize) {- let (cw, ch) = (self.cwidth, self.cheight);+ pub fn chroma_window<S: Sample>(&self, c: usize, x0: i32, y0: i32, w: usize, h: usize, out: &mut [S], os: usize) {+ let (cw, ch, hc) = (self.cwidth, self.cheight, self.fmt.mb_hc); for r in 0..h { let yy = (y0 + r as i32).clamp(0, ch as i32 - 1) as usize;- let row = self.row(yy >> 3);+ let Some(row) = S::row_planes(&self.row(yy / hc).samples) else {+ out[r * os..r * os + w].fill(S::clip(self.fmt.mid(), self.fmt.max()));+ continue;+ }; let plane = if c == 0 { &row.cb } else { &row.cr };- let line = &plane[(yy & 7) * cw..(yy & 7) * cw + cw];- copy_line::<16>(line, x0, &mut out[r * os..], w);+ let line = &plane[(yy % hc) * cw..(yy % hc) * cw + cw];+ copy_line::<S, 16>(line, x0, &mut out[r * os..], w); } } - /// Crop and copy the frame into planar buffers (waits for completion).- pub fn copy_cropped(&self, crop: (usize, usize, usize, usize)) -> (Vec<u8>, Vec<u8>, Vec<u8>) {+ /// Crop and copy the frame into planar buffers (waits for completion). Chroma is cropped+ /// with the frame's subsampling (vertical only for 4:2:0). None if the frame is stored with+ /// another sample type.+ pub fn copy_cropped<S: Sample>(&self, crop: (usize, usize, usize, usize)) -> Option<(Vec<S>, Vec<S>, Vec<S>)> { let (cx, cy, cw, ch) = crop;- let alloc = plane_allocator();- let mut y = alloc(cw * ch);+ let hc = self.fmt.mb_hc;+ let mut y = S::alloc_plane(cw * ch); for r in cy..cy + ch {- let row = self.row(r >> 4);+ let row = S::row_planes(&self.row(r >> 4).samples)?; y.extend_from_slice(&row.y[(r & 15) * self.width + cx..(r & 15) * self.width + cx + cw]); }- let (ccx, ccy, ccw, cch) = (cx / 2, cy / 2, cw.div_ceil(2), ch.div_ceil(2));- let mut u = alloc(ccw * cch);- let mut v = alloc(ccw * cch);+ let (ccy, cch) = if hc == 16 { (cy, ch) } else { (cy / 2, ch.div_ceil(2)) };+ let (ccx, ccw) = (cx / 2, cw.div_ceil(2));+ let mut u = S::alloc_plane(ccw * cch);+ let mut v = S::alloc_plane(ccw * cch); for r in ccy..ccy + cch {- let row = self.row(r >> 3);- let o = (r & 7) * self.cwidth + ccx;+ let row = S::row_planes(&self.row(r / hc).samples)?;+ let o = (r % hc) * self.cwidth + ccx; u.extend_from_slice(&row.cb[o..o + ccw]); v.extend_from_slice(&row.cr[o..o + ccw]); }- (y, u, v)+ Some((y, u, v)) } } -static PLANE_ALLOCATOR: std::sync::OnceLock<fn(usize) -> Vec<u8>> = std::sync::OnceLock::new();+static PLANE_ALLOCATOR: OnceLock<fn(usize) -> Vec<u8>> = OnceLock::new();+static PLANE_ALLOCATOR_U16: OnceLock<fn(usize) -> Vec<u16>> = OnceLock::new(); -/// Where output planes come from: an empty buffer with room for the given number of samples.-/// The host sets it once to recycle the planes of pictures it is done with; the first call wins.+/// Where 8-bit output planes come from: an empty buffer with room for the given number of+/// samples. The host sets it once to recycle the planes of pictures it is done with; the first+/// call wins. pub fn set_plane_allocator(alloc: fn(usize) -> Vec<u8>) { let _ = PLANE_ALLOCATOR.set(alloc); } -fn plane_allocator() -> fn(usize) -> Vec<u8> {- PLANE_ALLOCATOR.get().copied().unwrap_or(Vec::with_capacity)+/// [`set_plane_allocator`] for the 16-bit planes of high bit depth pictures.+pub fn set_plane_allocator_u16(alloc: fn(usize) -> Vec<u16>) {+ let _ = PLANE_ALLOCATOR_U16.set(alloc); } /// Copy `w` samples of `line` starting at x0 into `out`; when possible a fixed-size block of `N`-/// bytes is copied instead (cheaper than a variable-length copy; `out` must then have room for `N`).+/// samples is copied instead (cheaper than a variable-length copy; `out` must then have room for `N`). #[inline(always)]-fn copy_line<const N: usize>(line: &[u8], x0: i32, out: &mut [u8], w: usize) {+fn copy_line<S: Sample, const N: usize>(line: &[S], x0: i32, out: &mut [S], w: usize) { if w <= N && x0 >= 0 && let Some(src) = line.get(x0 as usize..).and_then(|l| l.first_chunk::<N>())@@ -303,7 +462,7 @@ fn copy_line<const N: usize>(line: &[u8], x0: i32, out: &mut [u8], w: usize) { /// Copy `out.len()` samples of `line` starting at x0, replicating edge samples outside the line. #[inline(always)]-fn copy_clamped(line: &[u8], x0: i32, out: &mut [u8]) {+fn copy_clamped<S: Sample>(line: &[S], x0: i32, out: &mut [S]) { let w = out.len(); if x0 >= 0 && x0 as usize + w <= line.len() { out.copy_from_slice(&line[x0 as usize..x0 as usize + w]);@@ -304,7 +304,8 @@ impl SliceHeader { } let slice_qp_delta = r.read_se()?; let qp = pps.pic_init_qp + slice_qp_delta;- ensure!((0..=51).contains(&qp), "slice QP {qp} out of range");+ let qp_bd_offset = 6 * sps.bit_depth_luma.saturating_sub(8) as i32;+ ensure!((-qp_bd_offset..=51).contains(&qp), "slice QP {qp} out of range"); let mut sp_for_switch = false; let mut slice_qs_delta = 0; if matches!(slice_type, SliceType::Sp | SliceType::Si) {@@ -25,8 +25,8 @@ pub struct SliceInfo { /// The picture under construction: private reconstruction buffers plus the shared [`Frame`] that /// finished (deblocked) macroblock rows are published into.-pub struct PicState {- pub planes: Planes,+pub struct PicState<S> {+ pub planes: Planes<S>, pub mbs: Vec<MbState>, pub slices: Vec<SliceInfo>, pub mb_w: usize,@@ -44,11 +44,11 @@ pub struct PicState { pub skip_deblock: bool, } -impl PicState {+impl<S: Sample> PicState<S> { pub fn new(frame: FrameRef) -> Self { let (mb_w, mb_h) = (frame.mb_w, frame.mb_h()); PicState {- planes: Planes::new(mb_w * 16, mb_h * 16),+ planes: Planes::new(mb_w * 16, mb_h * 16, frame.fmt), mbs: vec![MbState::default(); mb_w * mb_h], slices: Vec::new(), mb_w,@@ -66,7 +66,7 @@ impl PicState { /// Reuse the buffers of a finished picture of the same size for a new frame. pub fn reset(&mut self, frame: FrameRef) {- debug_assert_eq!((self.mb_w, self.mb_h), (frame.mb_w, frame.mb_h()));+ debug_assert_eq!((self.mb_w, self.mb_h, self.planes.fmt), (frame.mb_w, frame.mb_h(), frame.fmt)); // start_mb() re-initialises each macroblock before use; only availability must be reset for m in &mut self.mbs { m.slice_num = u32::MAX;@@ -158,22 +158,29 @@ pub(crate) struct MbCur { } /// Coefficient scratch for one macroblock.-pub(crate) struct Scratch {+pub(crate) struct Scratch<S> { /// Luma: 16 4x4 blocks (raster block index * 16 + raster coefficient) or 4 8x8 blocks (b8 * 64 + raster). pub coef: [i32; 256],- /// Chroma: [Cb, Cr][raster 2x2 block * 16 + raster coefficient].- pub coef_c: [[i32; 64]; 2],+ /// Chroma: [Cb, Cr][raster block (2 wide x 2 or 4 tall) * 16 + raster coefficient].+ pub coef_c: [[i32; 128]; 2], /// Luma blocks with coefficients: raster 4x4 bits, or b8 bits in 8x8 mode. pub blk_nz: u16,- /// Chroma blocks with coefficients (bit per raster 2x2 block).+ /// Chroma blocks with coefficients (bit per raster block). pub c_nz: [u8; 2],- pub pred: [[u8; 256]; 2],- pub pred_c: [[[u8; 64]; 2]; 2],+ pub pred: [[S; 256]; 2],+ pub pred_c: [[[S; 64]; 2]; 2], } -impl Scratch {+impl<S: Sample> Scratch<S> { pub fn new() -> Box<Self> {- Box::new(Scratch { coef: [0; 256], coef_c: [[0; 64]; 2], blk_nz: 0, c_nz: [0; 2], pred: [[0; 256]; 2], pred_c: [[[0; 64]; 2]; 2] })+ Box::new(Scratch {+ coef: [0; 256],+ coef_c: [[0; 128]; 2],+ blk_nz: 0,+ c_nz: [0; 2],+ pred: [[S::default(); 256]; 2],+ pred_c: [[[S::default(); 64]; 2]; 2],+ }) } } @@ -186,16 +193,19 @@ pub(crate) struct Nb { const NB_NA: Nb = Nb { avail: false, ref_idx: -1, mv: [0, 0] }; -pub struct SliceDecoder<'a> {+pub struct SliceDecoder<'a, S> { pub sh: &'a SliceHeader, pub pps: &'a Pps, pub sps: &'a Sps,- pub pic: &'a mut PicState,+ pub pic: &'a mut PicState<S>,+ /// Sample format of the picture.+ pub fmt: Format, pub refs: &'a [Vec<RefPic>; 2], pub ls: &'a LevelScale, pub slice_num: u32, pub mb_w: usize, pub mb_h: usize,+ /// QPY of the current macroblock (-QpBdOffsetY..=51). pub qp: i32, pub chroma_qp_offset: [i32; 2], // current macroblock@@ -206,7 +216,7 @@ pub struct SliceDecoder<'a> { pub nb: [Option<usize>; 4], pub mv_done: u16, pub cur: MbCur,- pub s: Box<Scratch>,+ pub s: Box<Scratch<S>>, weights: Weights, /// Temporal direct: DistScaleFactor per refIdxL0, and whether pic0 is long-term. tdirect: Vec<(i32, bool)>,@@ -218,21 +228,21 @@ pub struct SliceDecoder<'a> { /// and the corresponding chroma blocks into `c` (stride `cs`). #[allow(clippy::too_many_arguments)] #[inline]-fn predict_block(f: &Frame, mv: [i16; 2], px: usize, py: usize, bw: usize, bh: usize, y: (&mut [u8], usize), c: [&mut [u8]; 2], cs: usize) {+fn predict_block<S: Sample>(f: &Frame, mv: [i16; 2], px: usize, py: usize, bw: usize, bh: usize, y: (&mut [S], usize), c: [&mut [S]; 2], cs: usize, max: i32) { const LW: usize = 24;- let mut win = [0u8; LW * 21];+ let mut win = [S::default(); LW * 21]; let ix = px as i32 + (mv[0] as i32 >> 2); let iy = py as i32 + (mv[1] as i32 >> 2); f.luma_window(ix - 2, iy - 2, bw + 5, bh + 5, &mut win, LW);- inter::mc_luma_win(&win, LW, (mv[0] & 3) as u32, (mv[1] & 3) as u32, bw, bh, y.0, y.1);+ inter::mc_luma_win(&win, LW, (mv[0] & 3) as u32, (mv[1] & 3) as u32, bw, bh, y.0, y.1, max); let cx = (px / 2) as i32 + (mv[0] as i32 >> 3); let cy = (py / 2) as i32 + (mv[1] as i32 >> 3); let (cw, ch) = (bw / 2, bh / 2); let (fx, fy) = ((mv[0] & 7) as u32, (mv[1] & 7) as u32); for (comp, out) in c.into_iter().enumerate() {- let mut w = [0u8; 16 * 9];+ let mut w = [S::default(); 16 * 9]; f.chroma_window(comp, cx, cy, cw + 1, ch + 1, &mut w, 16);- inter::mc_chroma_win(&w, 16, fx, fy, cw, ch, out, cs);+ inter::mc_chroma_win(&w, 16, fx, fy, cw, ch, out, cs, max); } } @@ -241,8 +251,10 @@ fn median(a: i16, b: i16, c: i16) -> i16 { a.max(b).min(a.min(b).max(c)) } -impl<'a> SliceDecoder<'a> {- pub fn new(sh: &'a SliceHeader, pps: &'a Pps, sps: &'a Sps, pic: &'a mut PicState, refs: &'a [Vec<RefPic>; 2], ls: &'a LevelScale) -> Result<Self> {+impl<'a, S: Sample> SliceDecoder<'a, S> {+ pub fn new(sh: &'a SliceHeader, pps: &'a Pps, sps: &'a Sps, pic: &'a mut PicState<S>, refs: &'a [Vec<RefPic>; 2], ls: &'a LevelScale) -> Result<Self> {+ let fmt = pic.planes.fmt;+ ensure!(fmt.mb_hc == 8 || sh.slice_type.is_intra(), "inter prediction in 4:2:2 streams is not implemented"); let slice_num = pic.slices.len() as u32; let chroma_qp_offset = [pps.chroma_qp_index_offset, pps.second_chroma_qp_index_offset]; pic.slices.push(SliceInfo {@@ -251,7 +263,7 @@ impl<'a> SliceDecoder<'a> { beta_offset: sh.slice_beta_offset_div2 * 2, ref_ids: [refs[0].iter().map(|r| r.id()).collect(), refs[1].iter().map(|r| r.id()).collect()], });- let weights = Self::build_weights(sh, pps, refs, pic.poc);+ let weights = Self::build_weights(sh, pps, refs, pic.poc, fmt); let mut tdirect = Vec::new(); if sh.slice_type == SliceType::B && !sh.direct_spatial_mv_pred && !refs[1].is_empty() { let poc1 = refs[1][0].poc();@@ -272,6 +284,7 @@ impl<'a> SliceDecoder<'a> { pps, sps, pic,+ fmt, refs, ls, slice_num,@@ -292,7 +305,9 @@ impl<'a> SliceDecoder<'a> { }) } - fn build_weights(sh: &SliceHeader, pps: &Pps, refs: &[Vec<RefPic>; 2], cur_poc: i32) -> Weights {+ fn build_weights(sh: &SliceHeader, pps: &Pps, refs: &[Vec<RefPic>; 2], cur_poc: i32, fmt: Format) -> Weights {+ // explicit offsets are in units of 8-bit samples (8-301, 8-302)+ let o = |v: i32| v * (1 << (fmt.bit_depth - 8)); let mut w = Weights { mode: WpMode::Default, luma_denom: 0,@@ -309,8 +324,8 @@ impl<'a> SliceDecoder<'a> { w.luma_denom = t.luma_log2_denom as i32; w.chroma_denom = t.chroma_log2_denom as i32; for (l, entries) in [&t.l0, &t.l1].into_iter().enumerate() {- w.luma[l] = entries.iter().map(|e| (e.luma_weight, e.luma_offset)).collect();- w.chroma[l] = entries.iter().map(|e| [(e.chroma_weight[0], e.chroma_offset[0]), (e.chroma_weight[1], e.chroma_offset[1])]).collect();+ w.luma[l] = entries.iter().map(|e| (e.luma_weight, o(e.luma_offset))).collect();+ w.chroma[l] = entries.iter().map(|e| [(e.chroma_weight[0], o(e.chroma_offset[0])), (e.chroma_weight[1], o(e.chroma_offset[1]))]).collect(); } } } else if pps.weighted_bipred_idc == 2 && sh.slice_type.is_b() {@@ -374,8 +389,10 @@ impl<'a> SliceDecoder<'a> { } } + /// QPC for luma QPY `qp` (8.5.8, Table 8-15; negative below 0 at high bit depths). fn chroma_qp(&self, qp: i32, c: usize) -> i32 {- QPC_TABLE[(qp + self.chroma_qp_offset[c]).clamp(0, 51) as usize] as i32+ let qpi = (qp + self.chroma_qp_offset[c]).clamp(-self.fmt.qp_bd_offset(), 51);+ if qpi < 30 { qpi } else { QPC_TABLE[qpi as usize] as i32 } } /// nC for a luma 4x4 block (raster index) - 9.2.1.@@ -394,9 +411,10 @@ impl<'a> SliceDecoder<'a> { fn nc_chroma(&self, c: usize, raster: usize) -> i32 { let (bx, by) = (raster & 1, raster >> 1);+ let bottom = self.fmt.chroma_blocks() - 2; let cur = self.mb(); let a = if bx > 0 { Some(cur.nnz_c[c][raster - 1]) } else { self.nb[0].map(|m| self.pic.mbs[m].nnz_c[c][by * 2 + 1]) };- let b = if by > 0 { Some(cur.nnz_c[c][raster - 2]) } else { self.nb[1].map(|m| self.pic.mbs[m].nnz_c[c][2 + bx]) };+ let b = if by > 0 { Some(cur.nnz_c[c][raster - 2]) } else { self.nb[1].map(|m| self.pic.mbs[m].nnz_c[c][bottom + bx]) }; match (a, b) { (Some(a), Some(b)) => (a as i32 + b as i32 + 1) >> 1, (Some(a), None) => a as i32,@@ -739,12 +757,24 @@ impl<'a> SliceDecoder<'a> { let ys = self.pic.planes.width; let cs = self.pic.planes.cwidth; let (cx, cy) = (px / 2, py / 2);+ let max = self.fmt.max(); if use0 != use1 && wts.iter().all(|w| *w == Weight::Default) { // single-list unweighted prediction: interpolate straight into the picture let l = if use0 { 0 } else { 1 }; let f = &self.refs[l][refs[l] as usize].frame; let planes = &mut self.pic.planes;- predict_block(f, mvs[l], px, py, bw, bh, (&mut planes.y[py * ys + px..], ys), [&mut planes.cb[cy * cs + cx..], &mut planes.cr[cy * cs + cx..]], cs);+ predict_block(+ f,+ mvs[l],+ px,+ py,+ bw,+ bh,+ (&mut planes.y[py * ys + px..], ys),+ [&mut planes.cb[cy * cs + cx..], &mut planes.cr[cy * cs + cx..]],+ cs,+ max,+ ); return; } let s = &mut *self.s;@@ -754,7 +784,7 @@ impl<'a> SliceDecoder<'a> { } let Some(rp) = self.refs[l].get(refs[l] as usize) else { continue }; let [pc0, pc1] = &mut s.pred_c[l];- predict_block(&rp.frame, mvs[l], px, py, bw, bh, (&mut s.pred[l][..], 16), [&mut pc0[..], &mut pc1[..]], 8);+ predict_block(&rp.frame, mvs[l], px, py, bw, bh, (&mut s.pred[l][..], 16), [&mut pc0[..], &mut pc1[..]], 8, max); } inter::weighted_store( &mut self.pic.planes.y[py * ys + px..],@@ -765,13 +795,14 @@ impl<'a> SliceDecoder<'a> { bw, bh, wts[0],+ max, ); for c in 0..2 { let wc = wts[c + 1]; let a = if use0 { Some(&s.pred_c[0][c][..]) } else { None }; let b = if use1 { Some(&s.pred_c[1][c][..]) } else { None }; let plane = if c == 0 { &mut self.pic.planes.cb } else { &mut self.pic.planes.cr };- inter::weighted_store(&mut plane[cy * cs + cx..], cs, a, b, 8, bw / 2, bh / 2, wc);+ inter::weighted_store(&mut plane[cy * cs + cx..], cs, a, b, 8, bw / 2, bh / 2, wc, max); } } @@ -891,7 +922,7 @@ impl<'a> SliceDecoder<'a> { let Some(blk) = self.s.coef.get_mut(raster * 16..).and_then(<[i32]>::first_chunk_mut::<16>) else { return; };- transform::idct4_add(blk, &mut self.pic.planes.y[py * stride + px..], stride);+ transform::idct4_add(blk, &mut self.pic.planes.y[py * stride + px..], stride, self.fmt.max()); blk.fill(0); } @@ -905,27 +936,28 @@ impl<'a> SliceDecoder<'a> { let Some(blk) = self.s.coef.get_mut(b8 * 64..).and_then(<[i32]>::first_chunk_mut::<64>) else { return; };- transform::idct8_add(blk, &mut self.pic.planes.y[py * stride + px..], stride);+ transform::idct8_add(blk, &mut self.pic.planes.y[py * stride + px..], stride, self.fmt.max()); blk.fill(0); } fn add_chroma(&mut self) { let stride = self.pic.planes.cwidth;+ let (max, hc) = (self.fmt.max(), self.fmt.mb_hc); for c in 0..2 { if self.s.c_nz[c] == 0 { continue; }- for b in 0..4 {+ for b in 0..self.fmt.chroma_blocks() { if self.s.c_nz[c] & (1 << b) == 0 { continue; } let px = self.mb_x * 8 + (b & 1) * 4;- let py = self.mb_y * 8 + (b >> 1) * 4;+ let py = self.mb_y * hc + (b >> 1) * 4; let Some(blk) = self.s.coef_c[c].get_mut(b * 16..).and_then(<[i32]>::first_chunk_mut::<16>) else { continue; }; let plane = if c == 0 { &mut self.pic.planes.cb } else { &mut self.pic.planes.cr };- transform::idct4_add(blk, &mut plane[py * stride + px..], stride);+ transform::idct4_add(blk, &mut plane[py * stride + px..], stride, max); blk.fill(0); } self.s.c_nz[c] = 0;@@ -959,6 +991,7 @@ impl<'a> SliceDecoder<'a> { pub fn reconstruct_intra(&mut self) { let kind = self.mb().kind; let stride = self.pic.planes.width;+ let fmt = self.fmt; let (x0, y0) = (self.mb_x * 16, self.mb_y * 16); match kind { MbKind::I4x4 => {@@ -967,7 +1000,7 @@ impl<'a> SliceDecoder<'a> { let _ = blk; let av = self.luma_avail4(bx, by); let mode = self.mb().intra_modes[by * 4 + bx];- intra::pred4x4(&mut self.pic.planes.y, stride, x0 + bx * 4, y0 + by * 4, mode, av);+ intra::pred4x4(&mut self.pic.planes.y, stride, x0 + bx * 4, y0 + by * 4, mode, av, fmt); self.add_luma_block4(by * 4 + bx); } }@@ -982,13 +1015,13 @@ impl<'a> SliceDecoder<'a> { _ => false, }; let mode = self.mb().intra_modes[by * 4 + bx];- intra::pred8x8(&mut self.pic.planes.y, stride, x0 + bx * 4, y0 + by * 4, mode, av);+ intra::pred8x8(&mut self.pic.planes.y, stride, x0 + bx * 4, y0 + by * 4, mode, av, fmt); self.add_luma_block8(b8); } } MbKind::I16x16 => { let av = Avail { left: self.intra_nb(0), top: self.intra_nb(1), top_left: self.intra_nb(3), top_right: false };- intra::pred16x16(&mut self.pic.planes.y, stride, x0, y0, self.cur.info.i16_mode, av);+ intra::pred16x16(&mut self.pic.planes.y, stride, x0, y0, self.cur.info.i16_mode, av, fmt); for r in 0..16 { self.add_luma_block4(r); }@@ -998,9 +1031,9 @@ impl<'a> SliceDecoder<'a> { let av = Avail { left: self.intra_nb(0), top: self.intra_nb(1), top_left: self.intra_nb(3), top_right: false }; let cs = self.pic.planes.cwidth; let mode = self.mb().intra_chroma_mode;- let (cx, cy) = (self.mb_x * 8, self.mb_y * 8);- intra::pred_chroma(&mut self.pic.planes.cb, cs, cx, cy, mode, av);- intra::pred_chroma(&mut self.pic.planes.cr, cs, cx, cy, mode, av);+ let (cx, cy) = (self.mb_x * 8, self.mb_y * fmt.mb_hc);+ intra::pred_chroma(&mut self.pic.planes.cb, cs, cx, cy, mode, av, fmt);+ intra::pred_chroma(&mut self.pic.planes.cr, cs, cx, cy, mode, av, fmt); self.add_chroma(); self.s.blk_nz = 0; }@@ -1020,33 +1053,47 @@ impl<'a> SliceDecoder<'a> { self.s.blk_nz = 0; } - /// Write I_PCM samples.- pub fn write_pcm(&mut self, samples: &[u8]) {+ /// Bytes of the pcm samples of an I_PCM macroblock (7.3.5).+ pub fn pcm_bytes(&self) -> usize {+ (256 + 16 * self.fmt.mb_hc) * self.fmt.bit_depth as usize / 8+ }++ /// Write I_PCM samples from the byte-aligned pcm_sample_luma / pcm_sample_chroma data+ /// (`pcm_bytes()` long).+ pub fn write_pcm(&mut self, data: &[u8]) -> Result<()> {+ let bits = self.fmt.bit_depth;+ let max = self.fmt.max();+ let mut r = BitReader::new(data); let stride = self.pic.planes.width; let (x0, y0) = (self.mb_x * 16, self.mb_y * 16);- for r in 0..16 {- self.pic.planes.y[(y0 + r) * stride + x0..(y0 + r) * stride + x0 + 16].copy_from_slice(&samples[r * 16..r * 16 + 16]);+ for row in 0..16 {+ for col in 0..16 {+ self.pic.planes.y[(y0 + row) * stride + x0 + col] = S::clip(r.read_bits(bits)? as i32, max);+ } } let cs = self.pic.planes.cwidth;- let (cx, cy) = (self.mb_x * 8, self.mb_y * 8);+ let hc = self.fmt.mb_hc;+ let (cx, cy) = (self.mb_x * 8, self.mb_y * hc); for c in 0..2 {- let src = &samples[256 + c * 64..256 + c * 64 + 64]; let plane = if c == 0 { &mut self.pic.planes.cb } else { &mut self.pic.planes.cr };- for r in 0..8 {- plane[(cy + r) * cs + cx..(cy + r) * cs + cx + 8].copy_from_slice(&src[r * 8..r * 8 + 8]);+ for row in 0..hc {+ for col in 0..8 {+ plane[(cy + row) * cs + cx + col] = S::clip(r.read_bits(bits)? as i32, max);+ } } }+ Ok(()) } /// Common state for an I_PCM MB. pub fn finish_pcm(&mut self) {- let qpc = [self.chroma_qp(0, 0) as u8, self.chroma_qp(0, 1) as u8];+ let qpc = [self.chroma_qp(0, 0) as i8, self.chroma_qp(0, 1) as i8]; let st = self.mb_mut(); st.kind = MbKind::IPcm; st.qp = 0; st.qpc = qpc; st.nnz = [16; 16];- st.nnz_c = [[16; 4]; 2];+ st.nnz_c = [[16; 8]; 2]; st.cbf_dc = 7; st.cbp = 0x2f; st.nz_mask = 0xffff;@@ -1055,19 +1102,33 @@ impl<'a> SliceDecoder<'a> { /// Update QP after mb_qp_delta. pub fn apply_qp_delta(&mut self, delta: i32) -> Result<()> {- ensure!((-26..=25).contains(&delta), "mb_qp_delta {delta} out of range");- self.qp = (self.qp + delta + 52) % 52;+ // 7.4.5: mb_qp_delta in -(26 + QpBdOffsetY / 2)..=25 + QpBdOffsetY / 2; QPY wraps (7-37)+ let off = self.fmt.qp_bd_offset();+ ensure!((-(26 + off / 2)..=25 + off / 2).contains(&delta), "mb_qp_delta {delta} out of range");+ self.qp = (self.qp + delta + 52 + 2 * off).rem_euclid(52 + off) - off; Ok(()) } pub fn store_qp(&mut self) { let qp = self.qp;- let qpc = [self.chroma_qp(qp, 0) as u8, self.chroma_qp(qp, 1) as u8];+ let qpc = [self.chroma_qp(qp, 0) as i8, self.chroma_qp(qp, 1) as i8]; let st = self.mb_mut();- st.qp = qp as u8;+ st.qp = qp as i8; st.qpc = qpc; } + /// QP'Y: the quantiser used for scaling (8.5.9).+ #[inline]+ fn qp_prime(&self) -> i32 {+ self.qp + self.fmt.qp_bd_offset()+ }++ /// QP'C of component `c`.+ #[inline]+ fn qpc_prime(&self, c: usize) -> i32 {+ self.chroma_qp(self.qp, c) + self.fmt.qp_bd_offset()+ }+ // ------------------------------------------------------------------------------------------ // Residual storage helpers (shared by CAVLC and CABAC) // ------------------------------------------------------------------------------------------@@ -1078,7 +1139,7 @@ impl<'a> SliceDecoder<'a> { /// Store a 4x4 luma block given in scan order (levels[0..16]); `ac_only` skips position 0. pub fn put_luma4(&mut self, raster: usize, levels: &[i32; 16], ac_only: bool) {- let qp = self.qp;+ let qp = self.qp_prime(); let ls = &self.ls.ls4[self.luma_list4()][(qp % 6) as usize]; let blk = &mut self.s.coef[raster * 16..raster * 16 + 16]; let mut any = false;@@ -1097,7 +1158,7 @@ impl<'a> SliceDecoder<'a> { /// Store an 8x8 luma block given in scan order. pub fn put_luma8(&mut self, b8: usize, levels: &[i32; 64]) {- let qp = self.qp;+ let qp = self.qp_prime(); let list = if self.mb().kind.is_intra() { 0 } else { 1 }; let ls = &self.ls.ls8[list][(qp % 6) as usize]; let blk = &mut self.s.coef[b8 * 64..b8 * 64 + 64];@@ -1123,7 +1184,7 @@ impl<'a> SliceDecoder<'a> { for (k, &lv) in levels.iter().enumerate() { c[ZIGZAG4[k] as usize] = lv; }- let qp = self.qp;+ let qp = self.qp_prime(); let ls00 = self.ls.ls4[0][(qp % 6) as usize][0]; transform::luma_dc_dequant(&mut c, qp, ls00); for (r, &dc) in c.iter().enumerate() {@@ -1134,14 +1195,23 @@ impl<'a> SliceDecoder<'a> { } } - /// Chroma DC levels (4:2:0: 4 values in raster order) for component `c`.- pub fn put_chroma_dc(&mut self, c: usize, levels: &[i32; 4]) {- let mut v = *levels;- let qpc = self.chroma_qp(self.qp, c);+ /// Chroma DC levels (4:2:0: 4 values in raster order; 4:2:2: 8 values in scan order) for+ /// component `c`.+ pub fn put_chroma_dc(&mut self, c: usize, levels: &[i32; 8]) {+ let qpc = self.qpc_prime(c); let list = if self.mb().kind.is_intra() { 1 + c } else { 4 + c };- let ls00 = self.ls.ls4[list][(qpc % 6) as usize][0];- transform::chroma_dc_dequant_420(&mut v, qpc, ls00);- for (b, &dc) in v.iter().enumerate() {+ let mut v = *levels;+ if self.fmt.mb_hc == 16 {+ // 8.5.11.2: qP,DC = QP'c + 3+ let qp_dc = qpc + 3;+ let ls00 = self.ls.ls4[list][(qp_dc % 6) as usize][0];+ transform::chroma_dc_dequant_422(&mut v, qp_dc, ls00);+ } else {+ let ls00 = self.ls.ls4[list][(qpc % 6) as usize][0];+ let Some(dc) = v.first_chunk_mut::<4>() else { return };+ transform::chroma_dc_dequant_420(dc, qpc, ls00);+ }+ for (b, &dc) in v.iter().enumerate().take(self.fmt.chroma_blocks()) { if dc != 0 { self.s.coef_c[c][b * 16] = dc; self.s.c_nz[c] |= 1 << b;@@ -1151,7 +1221,7 @@ impl<'a> SliceDecoder<'a> { /// Chroma AC levels (scan positions 1..16 in levels[1..]) for block `b` of component `c`. pub fn put_chroma_ac(&mut self, c: usize, b: usize, levels: &[i32; 16]) {- let qpc = self.chroma_qp(self.qp, c);+ let qpc = self.qpc_prime(c); let list = if self.mb().kind.is_intra() { 1 + c } else { 4 + c }; let ls = &self.ls.ls4[list][(qpc % 6) as usize]; let blk = &mut self.s.coef_c[c][b * 16..b * 16 + 16];@@ -1220,14 +1290,13 @@ impl<'a> SliceDecoder<'a> { if info.kind == MbKind::IPcm { r.byte_align(); let pos = r.byte_pos();- let data = r.data();- ensure!(data.len() >= pos + 384, "truncated I_PCM macroblock");- let Some(&samples) = data.get(pos..).and_then(|d| d.first_chunk::<384>()) else {+ let n = self.pcm_bytes();+ let Some(samples) = r.data().get(pos..pos + n) else { return invalid("truncated I_PCM macroblock"); };- r.skip(384 * 8)?; self.finish_pcm();- self.write_pcm(&samples);+ self.write_pcm(samples)?;+ r.skip(n * 8)?; return Ok(()); } let mut transform_8x8 = false;@@ -1395,16 +1464,21 @@ impl<'a> SliceDecoder<'a> { } } let cbp_c = cbp >> 4;+ let nb = self.fmt.chroma_blocks(); if cbp_c & 3 != 0 { for c in 0..2 {- let mut lv = [0i32; 4];- cavlc::residual_block(r, &mut lv, 0, 3, 4, -1)?;+ let mut lv = [0i32; 8];+ if nb == 8 {+ cavlc::residual_block(r, &mut lv, 0, 7, 8, -2)?;+ } else {+ cavlc::residual_block(r, &mut lv, 0, 3, 4, -1)?;+ } self.put_chroma_dc(c, &lv); } } if cbp_c & 2 != 0 { for c in 0..2 {- for b in 0..4 {+ for b in 0..nb { let nc = self.nc_chroma(c, b); let mut lv = [0i32; 16]; let tc = cavlc::residual_block(r, &mut lv[1..], 0, 14, 15, nc)?;@@ -1,6 +1,7 @@ //! Scaling (dequantisation) and inverse transforms (8.5.10 - 8.5.13). use crate::params::ScalingMatrices;+use crate::picture::Sample; use crate::tables::{norm_adjust4, norm_adjust8}; /// LevelScale4x4 / LevelScale8x8 for all six scaling lists and qP % 6, raster order.@@ -83,16 +84,38 @@ pub fn chroma_dc_dequant_420(c: &mut [i32; 4], qp: i32, ls00: i32) { } } -#[inline(always)]-fn clip_u8(v: i32) -> u8 {- v.clamp(0, 255) as u8+/// 4:2:2 chroma DC (8.5.11): 2x4 transform + scaling. `c` = chroma DC levels in scan order+/// (c[0..8]); the result is dcC per 4x4 block in block order (raster, 2 wide x 4 tall). `qp_dc`+/// is QP'c + 3.+pub fn chroma_dc_dequant_422(c: &mut [i32; 8], qp_dc: i32, ls00: i32) {+ // scan order -> 4x2 matrix (rows, columns) of (8-330)+ let m = [[c[0], c[2]], [c[1], c[5]], [c[3], c[6]], [c[4], c[7]]];+ // A (4x4) * m+ let mut t = [[0i32; 2]; 4];+ for j in 0..2 {+ let (a, b, cc, d) = (m[0][j], m[1][j], m[2][j], m[3][j]);+ t[0][j] = a.wrapping_add(b).wrapping_add(cc).wrapping_add(d);+ t[1][j] = a.wrapping_add(b).wrapping_sub(cc).wrapping_sub(d);+ t[2][j] = a.wrapping_sub(b).wrapping_sub(cc).wrapping_add(d);+ t[3][j] = a.wrapping_sub(b).wrapping_add(cc).wrapping_sub(d);+ }+ for i in 0..4 {+ let f = [t[i][0].wrapping_add(t[i][1]), t[i][0].wrapping_sub(t[i][1])];+ for j in 0..2 {+ c[i * 2 + j] = if qp_dc >= 36 {+ f[j].wrapping_mul(ls00) << (qp_dc / 6 - 6).min(31)+ } else {+ f[j].wrapping_mul(ls00).wrapping_add(1 << (5 - qp_dc / 6)) >> (6 - qp_dc / 6)+ };+ }+ } } /// Inverse 4x4 transform of raster-order `d` and add to `dst` (8.5.12.2, 8.5.14). /// /// Rows first, then columns (the spec's order: the `>> 1` steps make it matter). The column /// pass and the reconstruction run across all columns at once so they vectorise.-pub fn idct4_add(d: &[i32; 16], dst: &mut [u8], stride: usize) {+pub fn idct4_add<S: Sample>(d: &[i32; 16], dst: &mut [S], stride: usize, max: i32) { let mut t = [[0i32; 4]; 4]; for (i, row) in t.iter_mut().enumerate() { let r = &d[i * 4..i * 4 + 4];@@ -118,18 +141,18 @@ pub fn idct4_add(d: &[i32; 16], dst: &mut [u8], stride: usize) { return; }; for (p, &v) in line.iter_mut().zip(row) {- *p = clip_u8(*p as i32 + ((v + 32) >> 6));+ *p = S::clip(p.to_i32() + ((v + 32) >> 6), max); } } } /// DC-only shortcut: all AC coefficients zero. #[allow(dead_code)]-pub fn idct4_dc_add(dc: i32, dst: &mut [u8], stride: usize) {+pub fn idct4_dc_add<S: Sample>(dc: i32, dst: &mut [S], stride: usize, max: i32) { let v = (dc + 32) >> 6; for i in 0..4 { for p in &mut dst[i * stride..i * stride + 4] {- *p = clip_u8(*p as i32 + v);+ *p = S::clip(p.to_i32() + v, max); } } }@@ -157,7 +180,7 @@ fn idct8_1d(d: [i32; 8]) -> [i32; 8] { /// Inverse 8x8 transform of raster-order `d` and add to `dst` (8.5.13.2): rows, then columns, /// the column pass and reconstruction across all columns at once (they vectorise).-pub fn idct8_add(d: &[i32; 64], dst: &mut [u8], stride: usize) {+pub fn idct8_add<S: Sample>(d: &[i32; 64], dst: &mut [S], stride: usize, max: i32) { let mut t = [[0i32; 8]; 8]; for (row, src) in t.iter_mut().zip(d.as_chunks::<8>().0) { *row = idct8_1d(*src);@@ -174,7 +197,7 @@ pub fn idct8_add(d: &[i32; 64], dst: &mut [u8], stride: usize) { return; }; for (p, &v) in line.iter_mut().zip(row) {- *p = clip_u8(*p as i32 + ((v + 32) >> 6));+ *p = S::clip(p.to_i32() + ((v + 32) >> 6), max); } } }@@ -219,11 +242,11 @@ mod tests { let d8: [i32; 64] = std::array::from_fn(|_| (next() % (2 * range + 1)) as i32 - range as i32); let base: Vec<u8> = (0..24 * 8).map(|_| next() as u8).collect(); let (mut a, mut b) = (base.clone(), base.clone());- idct4_add(&d4, &mut a[3..], 24);+ idct4_add(&d4, &mut a[3..], 24, 255); reference_add(&d4, 4, &mut b[3..], 24); assert_eq!(a, b, "4x4 iteration {iter}"); let (mut a, mut b) = (base.clone(), base);- idct8_add(&d8, &mut a[5..], 24);+ idct8_add(&d8, &mut a[5..], 24, 255); reference_add(&d8, 8, &mut b[5..], 24); assert_eq!(a, b, "8x8 iteration {iter}"); }@@ -235,8 +258,8 @@ mod tests { d[0] = 640; // -> +10 everywhere let mut a = [100u8; 16]; let mut b = [100u8; 16];- idct4_add(&d, &mut a, 4);- idct4_dc_add(640, &mut b, 4);+ idct4_add(&d, &mut a, 4, 255);+ idct4_dc_add(640, &mut b, 4, 255); assert_eq!(a, b); assert!(a.iter().all(|&v| v == 110)); }@@ -249,7 +272,7 @@ mod tests { d[0] = 64; d[1] = 64; let mut out = [0u8; 16];- idct4_add(&d, &mut out, 4);+ idct4_add(&d, &mut out, 4, 255); for i in 0..4 { assert_eq!(&out[i * 4..i * 4 + 4], &[2, 2, 1, 0]); }@@ -260,7 +283,7 @@ mod tests { let mut d = [0i32; 64]; d[0] = 64 * 5; let mut out = [10u8; 64];- idct8_add(&d, &mut out, 8);+ idct8_add(&d, &mut out, 8, 255); assert!(out.iter().all(|&v| v == 15)); } new file mode 100644@@ -0,0 +1,227 @@+//! High 10 / High 4:2:2 (Intra) streams: bit-exactness against ffmpeg's decoder on libx264+//! fixtures with 4:2:2 and/or 10-bit samples (skipped when ffmpeg is absent).++use crate::common;++use filmcraft_h264::{Decoder, Picture};+use std::path::PathBuf;+use std::process::Command;++struct HbdFixture {+ name: &'static str,+ width: u32,+ height: u32,+ frames: u32,+ /// libx264 / ffmpeg pixel format (yuv420p10le, yuv422p10le, yuv422p).+ pix_fmt: &'static str,+ args: &'static [&'static str],+}++const FIXTURES: &[HbdFixture] = &[+ // XAVC Intra-like: High 4:2:2 Intra, 10-bit, CABAC, 8x8 transform, several slices+ HbdFixture {+ name: "hbd_422p10_intra_cabac",+ width: 352,+ height: 288,+ frames: 6,+ pix_fmt: "yuv422p10le",+ args: &["-profile:v", "high422", "-x264-params", "keyint=1:slices=4"],+ },+ HbdFixture {+ name: "hbd_422p10_intra_cavlc",+ width: 352,+ height: 288,+ frames: 6,+ pix_fmt: "yuv422p10le",+ args: &["-profile:v", "high422", "-x264-params", "keyint=1:cabac=0"],+ },+ HbdFixture {+ name: "hbd_422p10_intra_qp_low",+ width: 176,+ height: 144,+ frames: 4,+ pix_fmt: "yuv422p10le",+ args: &["-profile:v", "high422", "-qp", "1", "-x264-params", "keyint=1"],+ },+ HbdFixture {+ name: "hbd_422p10_intra_qp_high",+ width: 176,+ height: 144,+ frames: 4,+ pix_fmt: "yuv422p10le",+ args: &["-profile:v", "high422", "-qp", "60", "-x264-params", "keyint=1:chroma-qp-offset=4"],+ },+ HbdFixture {+ name: "hbd_422p10_intra_no8x8",+ width: 176,+ height: 144,+ frames: 4,+ pix_fmt: "yuv422p10le",+ args: &["-profile:v", "high422", "-x264-params", "keyint=1:8x8dct=0:cqm=jvt"],+ },+ HbdFixture {+ name: "hbd_422p8_intra",+ width: 352,+ height: 288,+ frames: 4,+ pix_fmt: "yuv422p",+ args: &["-profile:v", "high422", "-x264-params", "keyint=1"],+ },+ HbdFixture {+ name: "hbd_422p8_intra_cavlc",+ width: 176,+ height: 144,+ frames: 4,+ pix_fmt: "yuv422p",+ args: &["-profile:v", "high422", "-x264-params", "keyint=1:cabac=0"],+ },+ // High 10: inter prediction with B-frames, weighted prediction+ HbdFixture {+ name: "hbd_420p10_inter",+ width: 352,+ height: 288,+ frames: 20,+ pix_fmt: "yuv420p10le",+ args: &["-profile:v", "high10", "-x264-params", "bframes=3:weightp=2:ref=3"],+ },+ HbdFixture {+ name: "hbd_420p10_inter_cavlc",+ width: 176,+ height: 144,+ frames: 12,+ pix_fmt: "yuv420p10le",+ args: &["-profile:v", "high10", "-x264-params", "cabac=0:bframes=2"],+ },+];++fn fixture(name: &str) -> &'static HbdFixture {+ FIXTURES.iter().find(|f| f.name == name).unwrap_or_else(|| panic!("unknown fixture {name}"))+}++/// Generate (if needed) the stream and its ffmpeg reference decode (16-bit LE above 8 bits).+fn ensure(f: &HbdFixture) -> Option<(PathBuf, PathBuf)> {+ let ff = filmcraft_testkit::ffmpeg_or_skip(f.name)?;+ let dir = common::fixtures_dir();+ let h264 = dir.join(format!("{}.h264", f.name));+ let yuv = dir.join(format!("{}.yuv", f.name));+ if !h264.exists() {+ let tmp = filmcraft_testkit::temp_path(&h264);+ let vf = format!("testsrc2=size={}x{}:rate=25,format={},noise=alls=10:allf=t+u", f.width, f.height, f.pix_fmt);+ let ok = Command::new(&ff)+ .args(["-hide_banner", "-loglevel", "error", "-y", "-f", "lavfi", "-i", &vf, "-frames:v", &f.frames.to_string(), "-c:v", "libx264"])+ .args(["-pix_fmt", f.pix_fmt])+ .args(f.args)+ .args(["-f", "h264"])+ .arg(&tmp)+ .status()+ .is_ok_and(|s| s.success());+ assert!(ok, "fixture generation failed for {}", f.name);+ std::fs::rename(&tmp, &h264).unwrap();+ }+ if !yuv.exists() {+ let tmp = filmcraft_testkit::temp_path(&yuv);+ let ok = Command::new(&ff)+ .args(["-hide_banner", "-loglevel", "error", "-y", "-i"])+ .arg(&h264)+ .args(["-fps_mode", "passthrough", "-f", "rawvideo", "-pix_fmt", f.pix_fmt])+ .arg(&tmp)+ .status()+ .is_ok_and(|s| s.success());+ assert!(ok, "reference decode failed for {}", f.name);+ std::fs::rename(&tmp, &yuv).unwrap();+ }+ Some((h264, yuv))+}++/// The picture as the bytes ffmpeg writes for its pixel format.+fn raw(p: &Picture) -> Vec<u8> {+ match &p.wide {+ Some(w) => [&w.y, &w.u, &w.v].into_iter().flat_map(|pl| pl.iter().flat_map(|v| v.to_le_bytes())).collect(),+ None => [&p.y, &p.u, &p.v].into_iter().flat_map(|pl| pl.iter().copied()).collect(),+ }+}++fn check(name: &str, threads: usize) {+ let f = fixture(name);+ let Some((h264, yuv)) = ensure(f) else { return };+ let data = std::fs::read(h264).unwrap();+ let want = std::fs::read(yuv).unwrap();+ let mut dec = Decoder::with_threads(threads);+ let mut pics = Vec::new();+ for au in common::split_access_units(&data) {+ pics.extend(dec.decode(au, 0).unwrap_or_else(|e| panic!("{name}: {e}")));+ }+ pics.extend(dec.flush());+ assert!(dec.take_error().is_none(), "{name}: decode error");+ assert_eq!(pics.len(), f.frames as usize, "{name}: frame count");+ let wide = f.pix_fmt.ends_with("10le");+ let c422 = f.pix_fmt.starts_with("yuv422");+ let frame_bytes = (f.width * f.height) as usize * if c422 { 2 } else { 3 } / if c422 { 1 } else { 2 } * if wide { 2 } else { 1 };+ for (i, p) in pics.iter().enumerate() {+ assert_eq!((p.width, p.height), (f.width, f.height));+ assert_eq!(p.bit_depth, if wide { 10 } else { 8 }, "{name}");+ assert_eq!(p.chroma_format_idc, if c422 { 2 } else { 1 }, "{name}");+ assert_eq!(p.wide.is_some(), wide, "{name}");+ let got = raw(p);+ let exp = &want[i * frame_bytes..(i + 1) * frame_bytes];+ if got != exp {+ let first = got.iter().zip(exp).position(|(a, b)| a != b).unwrap_or(0);+ panic!("{name} ({threads} threads): frame {i} differs from ffmpeg (first byte {first} of {frame_bytes})");+ }+ }+}++macro_rules! hbd_tests {+ ($($name:ident),* $(,)?) => {+ $(+ #[test]+ fn $name() {+ check(stringify!($name), 1);+ check(stringify!($name), 4);+ }+ )*+ };+}++hbd_tests!(+ hbd_422p10_intra_cabac,+ hbd_422p10_intra_cavlc,+ hbd_422p10_intra_qp_low,+ hbd_422p10_intra_qp_high,+ hbd_422p10_intra_no8x8,+ hbd_422p8_intra,+ hbd_422p8_intra_cavlc,+ hbd_420p10_inter,+ hbd_420p10_inter_cavlc,+);++/// 4:2:2 with inter prediction (High 4:2:2 long-GOP) is reported as unsupported, not decoded+/// into wrong pictures and not a crash.+#[test]+fn hbd_422_inter_is_unsupported() {+ let Some(ff) = filmcraft_testkit::ffmpeg_or_skip("hbd_422_inter") else { return };+ let path = common::fixtures_dir().join("hbd_422_inter.h264");+ if !path.exists() {+ let tmp = filmcraft_testkit::temp_path(&path);+ let ok = Command::new(&ff)+ .args(["-hide_banner", "-loglevel", "error", "-y", "-f", "lavfi", "-i", "testsrc2=size=176x144:rate=25", "-frames:v", "8"])+ .args(["-c:v", "libx264", "-pix_fmt", "yuv422p10le", "-profile:v", "high422", "-f", "h264"])+ .arg(&tmp)+ .status()+ .is_ok_and(|s| s.success());+ assert!(ok);+ std::fs::rename(&tmp, &path).unwrap();+ }+ let data = std::fs::read(path).unwrap();+ let mut dec = Decoder::with_threads(1);+ let mut err = None;+ for au in common::split_access_units(&data) {+ if let Err(e) = dec.decode(au, 0) {+ err = Some(e);+ break;+ }+ }+ dec.flush();+ let err = err.or_else(|| dec.take_error());+ assert!(err.is_some_and(|e| e.to_string().contains("4:2:2")), "4:2:2 inter slices must be refused");+}@@ -1,6 +1,7 @@ mod common; mod conformance; mod headers;+mod high_bit_depth; mod perf; mod pps_validation; mod robustness;@@ -1,6 +1,6 @@ # Codec parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (ProRes alpha final-run conformance and native macOS validation) · **Target:** Adobe Premiere Pro 2026 (26.5.2) and Media Encoder 2026+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-26 · **Change:** minor (H.264 High 10 and High 4:2:2 Intra decode) · **Target:** Adobe Premiere Pro 2026 (26.5.2) and Media Encoder 2026 Video and audio codecs, decode and encode, with fidelity and how each is tested. Containers, project and interchange files are in [file-format-parity.md](file-format-parity.md); hardware@@ -15,7 +15,7 @@ contents). Percent is ready-for-real-work for that codec: depth, profiles, speed | Codec | Premiere | Our decode | Our encode | Fidelity and tests | % | Est. | |---|---|---|---|---|---|---|-| H.264 / AVC | decode all common profiles incl. High 10, 4:2:2 (XAVC S-I, AVC-Intra); encode 8/10-bit | own decoder, **8-bit 4:2:0 only** (Baseline / Main / High, CABAC / CAVLC); no High 10 / 4:2:2 / 4:4:4, interlaced field / MBAFF, lossless | own encoder (High / Main / Baseline, B-frames, CBR / VBR / 2-pass); 8-bit 4:2:0 only; hardware VideoToolbox / NVENC | decode bit-exact on 37+ conformance streams, 500–600 fps 1080p; encode ffprobe-checked, byte-identical across machines | 65% | 20–35 h |+| H.264 / AVC | decode all common profiles incl. High 10, 4:2:2 (XAVC S-I, AVC-Intra); encode 8/10-bit | own decoder (Baseline / Main / High, CABAC / CAVLC), High 10 (9/10-bit 4:2:0) and High 4:2:2 Intra (XAVC Intra, AVC-Intra); no 4:2:2 inter (long-GOP 4:2:2), 4:4:4, interlaced field / MBAFF, lossless | own encoder (High / Main / Baseline, B-frames, CBR / VBR / 2-pass); 8-bit 4:2:0 only; hardware VideoToolbox / NVENC | decode bit-exact on 37+ conformance streams, 500–600 fps 1080p; encode ffprobe-checked, byte-identical across machines | 65% | 20–35 h | | HEVC / H.265 | decode Main, Main 10, 4:2:2 10 (XAVC HS); encode 8/10-bit, HDR | own decoder Main / Main 10 / Main Still Picture, bit-exact on 41 fixtures, ~225 fps 1080p; **no 4:2:2 / RExt** | **hardware only** (VideoToolbox 8-bit; NVENC 8-bit and Main 10 HDR on Windows); no software encoder | NVENC round trip 48.9 dB (8-bit), 71.1 dB (10-bit); ffprobe | 60% | 30–60 h | | ProRes 422 family / 4444 / XQ | decode + encode all | own decode + encode, all six profiles, alpha | yes (export UI exposes 422 flavours; 4444 / XQ request #342) | ±1 LSB, 318 fps; alpha final-run conformance and native macOS HD decode checked ([details](../crates/prores/README.md#alpha-bitstream-conformance)) | 85% | 3–6 h | | ProRes RAW | decode | none (#345) | — | — | 0% | 30–50 h (spec availability uncertain) |@@ -1,6 +1,6 @@ # Where FilmCraft falls short of Premiere Pro -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (new file: ROADMAP's "Where we are lacking" re-measured against Premiere Pro 26.5.2 and the open issue list) · **Target:** Adobe Premiere Pro 2026 (26.5.2)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-26 · **Change:** minor (G3: H.264 High 10 and High 4:2:2 Intra) · **Target:** Adobe Premiere Pro 2026 (26.5.2) Every known shortfall, one entry each, ranked by what most stops an editor using FilmCraft for real work. This is the work list: agents should prefer it over adding checklist items. Numbers are in@@ -65,8 +65,11 @@ platforms** G4, G9, G17; **performance** G16; **ecosystem** G11; **measurement** ## G3. 10-bit and 4:2:2 camera media needs hardware -- **Missing:** our H.264 decoder is 8-bit 4:2:0 only (`crates/h264/README.md`: no High 10, High- 4:2:2, 4:4:4, lossless, interlaced field / MBAFF); HEVC has no 4:2:2 / RExt. Most mirrorless and+- **Have:** H.264 High 10 (9/10-bit 4:2:0, intra and inter) and High 4:2:2 Intra (4:2:2 8–10-bit+ intra pictures: Sony XAVC Intra, AVC-Intra) in software, bit-exact with ffmpeg.+- **Missing:** H.264 4:2:2 inter prediction (long-GOP 4:2:2: XAVC S / XAVC L 4:2:2, most+ mirrorless 4:2:2 MOVs), 4:4:4, lossless, interlaced field / MBAFF (`crates/h264/README.md`);+ HEVC has no 4:2:2 / RExt. Most mirrorless and cinema cameras record exactly these (Sony XAVC S-I / XAVC HS 4:2:2 10-bit, Panasonic, Canon XF-AVC, Fujifilm). - **Evidence:** #626 (Panasonic Lumix 4K 10-bit 4:2:2 MOV). On macOS VideoToolbox decodes 10-bit@@ -74,7 +77,7 @@ platforms** G4, G9, G17; **performance** G16; **ecosystem** G11; **measurement** there is no software fallback, so the clip fails. Premiere decodes them everywhere (and in hardware on Apple silicon, Intel and NVIDIA Blackwell). - **Impact:** camera originals from most current cameras do not open on Windows or Linux.-- **Estimate:** H.264 High 10 + High 4:2:2 (chroma geometry, deblocking, transforms) 15–25 h;+- **Estimate:** H.264 4:2:2 inter (chroma motion compensation, inter chroma deblocking) 5–10 h; HEVC Main 4:2:2 10 (RExt subset) 15–25 h. Doc: [codec-parity.md](codec-parity.md). ## G4. Crashes at start-up, and no test CI on pull requests@@ -323,3 +326,4 @@ platforms** G4, G9, G17; **performance** G16; **ecosystem** G11; **measurement** | 2026-10-12 | minor | G10: MP1 (`A_MPEG/L1`) in Matroska imports (#1402) | | 2026-10-12 | minor | G10: OpenEXR PXR24 import | | 2026-10-24 | minor | G8: dragging the cut on a transition's timeline in Effect Controls moves the cut, and the transition moves with it (#577) |+| 2026-10-26 | minor | G3: H.264 High 10 and High 4:2:2 Intra decode in software (XAVC Intra, AVC-Intra); 4:2:2 inter remains |