collective
Browse / github.com/storytold/pdfcraft / patch
M4: preserve extracted artwork with indirect paint kinds
Extract and Split lose drawn images when paint kind metadata is indirect On current main `1dd2737b`, Extract Pages and Split can delete visible images when a Type 3 font or Form XObject names its Subtype through an indirect name object. A contributor-original colored Type 3 glyph paints one image through a shared XObject dictionary; ordinary page content paints a second image. The only difference between the source/control is `/Subtype 11 0 R` with object 11 containing `/Type3`, versus `/Subtype /Type3`. The actual current-source CLI reports successful `page_extract`, but: - Direct Subtype: 2,900 dark source pixels remain 2,900 after extraction. - Indirect Subty
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- KyleKassen (imported from pull request #5033, open)
- Base
e9fc890a862ea2d8c895ea32ddecda98580483cd- Size
- +993 −112 in 19 files
- Review
- pending
- Shared
- 2026-10-28
- Id
c28db826f3dab946b0bbbb9e
How it went
No other operator has reported using it yet.
Diff
raw · ATTRIBUTION.md ATTRIBUTION.toml Cargo.lock Cargo.toml ROADMAP.md crates/automation/README.md crates/automation/tests/it/extract_indirect_kinds.rs crates/automation/tests/it/extract_paint_resources.rs crates/automation/tests/it/main.rs crates/ocr/Cargo.toml crates/ocr/src/lib.rs crates/organize/README.md crates/organize/src/prune.rs crates/organize/tests/indirect_paint_kinds.rs crates/organize/tests/paint_resources.rs docs/releasing.md docs/roadmap.md packaging/windows/test-msi.ps1 parity/acrobat-features.toml
@@ -335,8 +335,8 @@ Fonts are fetched by `cargo xtask demo-pdf` into `target/demo-fonts/`, OCR model | `NotoColorEmoji-Regular.ttf` | Noto Color Emoji | Google Inc. | OFL-1.1 | https://github.com/google/fonts/tree/9710da1eacb3be272583c3224dcb70f9da6eadbb/ofl/notocoloremoji | Showcase PDF (cargo xtask demo-pdf) | | `Inter[opsz,wght].ttf` | Inter (variable) | The Inter Project Authors (Rasmus Andersson) | OFL-1.1 | https://github.com/google/fonts/tree/9710da1eacb3be272583c3224dcb70f9da6eadbb/ofl/inter | Showcase PDF (cargo xtask demo-pdf) | | `Inter-Italic[opsz,wght].ttf` | Inter Italic (variable) | The Inter Project Authors (Rasmus Andersson) | OFL-1.1 | https://github.com/google/fonts/tree/9710da1eacb3be272583c3224dcb70f9da6eadbb/ofl/inter | Showcase PDF (cargo xtask demo-pdf) |-| `text-detection.rten` | Ocrs text detection model | Robert Knight (ocrs-models; trained on HierText, CC-BY-SA-4.0) | CC-BY-SA-4.0 | https://github.com/robertknight/ocrs-models (the models ocrs downloads; model card: https://huggingface.co/robertknight/ocrs, cc-by-sa-4.0) | Scan & OCR: finds words on page images (cargo xtask models) |-| `text-recognition.rten` | Ocrs text recognition model | Robert Knight (ocrs-models; trained on HierText, CC-BY-SA-4.0) | CC-BY-SA-4.0 | https://github.com/robertknight/ocrs-models (the models ocrs downloads; model card: https://huggingface.co/robertknight/ocrs, cc-by-sa-4.0) | Scan & OCR: reads the words it finds (cargo xtask models) |+| `text-detection.onnx` | Ocrs text detection model | Robert Knight (ocrs-models; trained on HierText, CC-BY-SA-4.0) | CC-BY-SA-4.0 | https://github.com/robertknight/ocrs/blob/0c826441362a701c55560fefc17bf356b6d1037e/ocrs/examples/download-models.sh (author-published ONNX models; model card: https://huggingface.co/robertknight/ocrs/blob/df0edd170279ab971b53e094c627255a87e1a503/README.md, cc-by-sa-4.0) | Scan & OCR: finds words on page images (cargo xtask models) |+| `text-recognition.onnx` | Ocrs text recognition model | Robert Knight (ocrs-models; trained on HierText, CC-BY-SA-4.0) | CC-BY-SA-4.0 | https://github.com/robertknight/ocrs/blob/0c826441362a701c55560fefc17bf356b6d1037e/ocrs/examples/download-models.sh (author-published ONNX models; model card: https://huggingface.co/robertknight/ocrs/blob/df0edd170279ab971b53e094c627255a87e1a503/README.md, cc-by-sa-4.0) | Scan & OCR: reads the words it finds (cargo xtask models) | ## Optional build inputs (1) @@ -3356,30 +3356,30 @@ usage = "1x + 2x HiDPI 16-colour palette TIFF copied into the macOS DMG by packa sha256 = "f46881724a6621f3686a612a79dbdce4f7dbe9e2025610e44c9a094341334c2c" [[fetched]]-file = "text-detection.rten"-url = "https://ocrs-models.s3-accelerate.amazonaws.com/text-detection.rten"+file = "text-detection.onnx"+url = "https://ocrs-models.s3-accelerate.amazonaws.com/text-detection.onnx" title = "Ocrs text detection model" author = "Robert Knight (ocrs-models; trained on HierText, CC-BY-SA-4.0)"-source = "https://github.com/robertknight/ocrs-models (the models ocrs downloads; model card: https://huggingface.co/robertknight/ocrs, cc-by-sa-4.0)"+source = "https://github.com/robertknight/ocrs/blob/0c826441362a701c55560fefc17bf356b6d1037e/ocrs/examples/download-models.sh (author-published ONNX models; model card: https://huggingface.co/robertknight/ocrs/blob/df0edd170279ab971b53e094c627255a87e1a503/README.md, cc-by-sa-4.0)" licence = "CC-BY-SA-4.0" licence_url = "https://creativecommons.org/licenses/by-sa/4.0/legalcode.txt" licence_sha256 = "28a9529c7d0bb4dc51f4bf5c116a3d16ef247a052f7591466768ddf563fd1cf5" kind = "model" usage = "Scan & OCR: finds words on page images (cargo xtask models)"-sha256 = "f15cfb56bd02c4bf478a20343986504a1f01e1665c2b3a0ad66340f054b1b5ca"+sha256 = "a917b23dbd9524b465df7e922641b3ff2981623df4ded5a0234004ef2fee7cfe" [[fetched]]-file = "text-recognition.rten"-url = "https://ocrs-models.s3-accelerate.amazonaws.com/text-recognition.rten"+file = "text-recognition.onnx"+url = "https://ocrs-models.s3-accelerate.amazonaws.com/text-recognition.onnx" title = "Ocrs text recognition model" author = "Robert Knight (ocrs-models; trained on HierText, CC-BY-SA-4.0)"-source = "https://github.com/robertknight/ocrs-models (the models ocrs downloads; model card: https://huggingface.co/robertknight/ocrs, cc-by-sa-4.0)"+source = "https://github.com/robertknight/ocrs/blob/0c826441362a701c55560fefc17bf356b6d1037e/ocrs/examples/download-models.sh (author-published ONNX models; model card: https://huggingface.co/robertknight/ocrs/blob/df0edd170279ab971b53e094c627255a87e1a503/README.md, cc-by-sa-4.0)" licence = "CC-BY-SA-4.0" licence_url = "https://creativecommons.org/licenses/by-sa/4.0/legalcode.txt" licence_sha256 = "28a9529c7d0bb4dc51f4bf5c116a3d16ef247a052f7591466768ddf563fd1cf5" kind = "model" usage = "Scan & OCR: reads the words it finds (cargo xtask models)"-sha256 = "e484866d4cce403175bd8d00b128feb08ab42e208de30e42cd9889d8f1735a6e"+sha256 = "86c145c2edb96c8caed5b1ebb8f44d706408922211451c309c625157dd6061c5" [[build_input]] name = "craft-fonts"@@ -1174,7 +1174,7 @@ source = "git+https://github.com/storytold/craft-libs?rev=82a24b006eefa036b43c09 [[package]] name = "craft-keymap" version = "0.1.0"-source = "git+https://github.com/storytold/craft-libs?rev=edc41f68fa24c46651c3b5117d10f0ead75ec9cb#edc41f68fa24c46651c3b5117d10f0ead75ec9cb"+source = "git+https://github.com/storytold/craft-libs?rev=acab34c0d8d84363ea3d05885f1a15601c233a19#acab34c0d8d84363ea3d05885f1a15601c233a19" dependencies = [ "egui", "serde",@@ -1184,7 +1184,7 @@ dependencies = [ [[package]] name = "craft-menu" version = "0.1.0"-source = "git+https://github.com/storytold/craft-libs?rev=edc41f68fa24c46651c3b5117d10f0ead75ec9cb#edc41f68fa24c46651c3b5117d10f0ead75ec9cb"+source = "git+https://github.com/storytold/craft-libs?rev=acab34c0d8d84363ea3d05885f1a15601c233a19#acab34c0d8d84363ea3d05885f1a15601c233a19" dependencies = [ "craft-keymap", "craft-quit",@@ -1210,7 +1210,7 @@ dependencies = [ [[package]] name = "craft-quit" version = "0.1.0"-source = "git+https://github.com/storytold/craft-libs?rev=edc41f68fa24c46651c3b5117d10f0ead75ec9cb#edc41f68fa24c46651c3b5117d10f0ead75ec9cb"+source = "git+https://github.com/storytold/craft-libs?rev=acab34c0d8d84363ea3d05885f1a15601c233a19#acab34c0d8d84363ea3d05885f1a15601c233a19" dependencies = [ "dispatch2", "egui",@@ -1221,7 +1221,7 @@ dependencies = [ [[package]] name = "craft-ui" version = "0.1.0"-source = "git+https://github.com/storytold/craft-libs?rev=edc41f68fa24c46651c3b5117d10f0ead75ec9cb#edc41f68fa24c46651c3b5117d10f0ead75ec9cb"+source = "git+https://github.com/storytold/craft-libs?rev=acab34c0d8d84363ea3d05885f1a15601c233a19#acab34c0d8d84363ea3d05885f1a15601c233a19" dependencies = [ "egui", "log",@@ -4340,6 +4340,7 @@ dependencies = [ "ocrs", "pdfcraft-fonts", "rten",+ "tempfile", "thiserror 2.0.21", ] @@ -5473,6 +5474,7 @@ dependencies = [ "rten-base", "rten-gemm", "rten-model-file",+ "rten-onnx", "rten-parallel", "rten-shape-inference", "rten-simd",@@ -5522,6 +5524,12 @@ dependencies = [ "rten-base", ] +[[package]]+name = "rten-onnx"+version = "0.26.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "96944374568990b8d1bbc5bae5957585866f9dd8256130460405730948bfd42e"+ [[package]] name = "rten-parallel" version = "0.26.0"@@ -13,12 +13,12 @@ repository = "https://github.com/storytold/pdfcraft" [workspace.dependencies] # Shared frontend components; engines remain independent of egui.-craft-ui = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "edc41f68fa24c46651c3b5117d10f0ead75ec9cb", features = ["serde"] }+craft-ui = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "acab34c0d8d84363ea3d05885f1a15601c233a19", features = ["serde"] } # Quit the macOS way (#1136): ⌘Q asks about unsaved changes, and closing quits through `terminate:`.-craft-quit = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "edc41f68fa24c46651c3b5117d10f0ead75ec9cb" }+craft-quit = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "acab34c0d8d84363ea3d05885f1a15601c233a19" } # One menu definition for the native macOS menu bar and the in-window bar elsewhere.-craft-menu = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "edc41f68fa24c46651c3b5117d10f0ead75ec9cb" }-craft-keymap = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "edc41f68fa24c46651c3b5117d10f0ead75ec9cb", default-features = false }+craft-menu = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "acab34c0d8d84363ea3d05885f1a15601c233a19" }+craft-keymap = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "acab34c0d8d84363ea3d05885f1a15601c233a19", default-features = false } # Shared system spooler; document composition remains in pdfcraft-print. craft-print = { git = "https://github.com/storytold/craft-libs", version = "0.1.0", rev = "42e30cd5b3f4b114270dd053fccbc33bedc47f68" } # Whether a path stored in a document may be opened (GoToR link targets).@@ -2,6 +2,7 @@ **Stage: alpha** · next: beta, ~25% (ready for real work 50% → 75%) and ~400–700 h away +> **Last reviewed:** 2026-10-27 | **Last updated:** 2026-10-27 | **Change:** minor (M10: OCR model downloads use verified ONNX pairs with legacy discovery fallback; estimates unchanged) > **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M14 localization current state: Arabic logical catalog, explicit RTL label direction and mirrored stock chrome; native/browser RTL workflows unqualified. Native release Hans face via LXGW WenKai Lite; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: image optimization accounts for page-local UserUnit; estimates unchanged)@@ -199,6 +200,11 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md ## Progress log +- 2026-10-27: M4 #5031: Extract and Split resolve indirect paint-kind metadata, retaining images drawn by Type 3 glyphs and Forms; source, save/reopen and rendered controls cover both indirect and direct kinds.++- 2026-10-27: M4 #5011: Extract and Split retain resource owners shared with annotation appearances, patterns, soft masks and Type 3 fonts; caller-sensitive discovery, bounded traversal and independent-resource pruning controls cover save/reopen and rendered output.++- **2026-10-27 (M10, OCR models):** Model staging downloads and verifies the ONNX pair after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, retains complete RTEN fallback and preserves directory precedence. - **2026-10-24 (M10, #4158 partial):** Word, HTML, RTF and PowerPoint export keep a block that continues another on its line (an italic case name, a link, a superscript) as a run in its own style inside that paragraph, with the source's word space, instead of a paragraph of its own. Style changes that wrap onto the next line still split paragraphs (the reading blocks keep style boundaries). Estimates unchanged. - **2026-10-24 (M3, window move on Wayland, #3970 follow-up):** After a move or resize by PdfCraft's own title bar or edges, the button release given back to egui goes before the pointer leaving the window, so hover no longer sticks at the old place until the pointer moves. Found by CodeRabbit in review of #3970; a unit test covers it. Estimates unchanged. - **2026-10-24 (M14, #4253):** The annotation quick bar remembers Keep tool selected across restarts, including unpinning. Old and malformed settings keep safe defaults; the opt-in UI control channel reads and changes the same preference. Estimates unchanged.@@ -682,6 +688,7 @@ Newest first: the date, what moved, and the new overall estimate. | Date | Change | Summary | |---|---|---|+| 2026-10-27 | minor | M10: OCR stages SHA-256-verified ONNX models after the upstream RTEN URL brownout; discovery prefers a complete ONNX pair per directory, falls back to a complete RTEN pair, and never mixes formats or directories; language/web limits and estimates unchanged. | | 2026-10-23 | minor | M8, #3205 follow-up: phone-pattern redaction includes the +39 country code on labelled Italian geographic numbers with a slash; dates, protocol identifiers and whole-run guards remain. Estimates unchanged. | | 2026-10-23 | minor | M11: image downsampling measures physical resolution after page-local UserUnit scaling, including shared nested forms; synthetic measurement and optimization regressions. Estimates unchanged. | | 2026-10-23 | minor | M10: Word table cells use the grid columns they span, including cells after a merged cell; synthetic export regression and CLI reproduction. Estimates unchanged. |@@ -35,6 +35,14 @@ its undo/redo history. - With a root set, every read and write path must resolve inside it (symlinks and `..` included). Relative paths resolve inside the root, and `..` is resolved by name before the check. Every path outside the root gets the same refusal (`<path> is outside the allowed directory <root>`), whether or not it exists, so a confined agent can't probe the rest of the disk; on Windows another network share or device path (`\\host\share`, `\\?\UNC\…`, `\\.\…`) is refused without being contacted. "Not found" and other filesystem errors are reported only for paths inside the root. Tools that write several files into a folder name them after the document or input file, with separators, colons and control characters replaced by `_`, so a name can't lead them out of the folder. The root itself must be a folder. - `Automation::with_automation_roots` holds a read folder and a write folder (`pdfcraft-cli mcp --automation-read-root DIR --automation-write-root DIR`, see [docs/mcp.md](../../docs/mcp.md)). Reads must lie inside the read root and writes inside the write root, links followed. A root left out grants none of its access. Refusals read `automation path rejected: outside the read root <root>: <path>`. As with a root set, on Windows a path on another network share or device path is refused with that refusal without being contacted (#4003). It applies on top of `with_root`. +## Extract and Split resources++`page_extract` and `doc_split` retain resources shared by page content and annotation appearances, tiling patterns, soft-mask groups or Type 3 fonts.+Inactive appearance states are retained too.+Type 3 glyph and Form artwork retain their image resources when the paint kind is specified by an indirect name.+A bounded dependency walk conservatively skips resource pruning when it cannot finish; independent unused page images remain eligible for removal.+See [organize resource preservation](../organize/README.md#extract-and-split-resource-preservation) for limits.+ ## Text search and cache limits `text_word_count` counts the current snapshot without filling the text cache. It returns totalsnew file mode 100644@@ -0,0 +1,99 @@+//! Contributor-original colored Type 3 glyphs exercise indirect kind metadata through tools.++use pdfcraft_automation::{Automation, Content};+use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream, write_full};+use serde_json::{Value, json};++fn fixture(indirect: bool) -> Vec<u8> {+ let mut doc = Document::new_empty();+ let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap();+ let mut images = Dict::new();+ for (name, color) in [("Small", [50, 50, 50]), ("Paint", [0, 180, 0])] {+ let attrs = Dict::from_iter([+ (b"Subtype".to_vec(), Object::name("Image")),+ (b"Width".to_vec(), 1.into()),+ (b"Height".to_vec(), 1.into()),+ (b"BitsPerComponent".to_vec(), 8.into()),+ (b"ColorSpace".to_vec(), Object::name("DeviceRGB")),+ ]);+ let image = doc.add(Object::Stream(Stream::from_raw(attrs, color.to_vec())));+ images.set(name.as_bytes().to_vec(), Object::Ref(image));+ }+ let images = doc.add(images);+ let program = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"1000 0 d0 q 1000 0 0 1000 0 0 cm /Paint Do Q".to_vec())));+ let subtype = if indirect { Object::Ref(doc.add(Object::name("Type3"))) } else { Object::name("Type3") };+ let font = doc.add(Dict::from_iter([+ (b"Type".to_vec(), Object::name("Font")),+ (b"Subtype".to_vec(), subtype),+ (b"FontBBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 1000.into(), 1000.into()])),+ (b"FontMatrix".to_vec(), Object::Array(vec![Object::Real(0.001), 0.into(), 0.into(), Object::Real(0.001), 0.into(), 0.into()])),+ (b"CharProcs".to_vec(), Object::Dict(Dict::from_iter([(b"A".to_vec(), Object::Ref(program))]))),+ (b"Encoding".to_vec(), Object::Dict(Dict::from_iter([(b"Differences".to_vec(), Object::Array(vec![65.into(), Object::name("A")]))]))),+ (b"FirstChar".to_vec(), 65.into()),+ (b"LastChar".to_vec(), 65.into()),+ (b"Widths".to_vec(), Object::Array(vec![1000.into()])),+ (b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"XObject".to_vec(), Object::Ref(images))]))),+ ]));+ let resources = doc.add(Dict::from_iter([+ (b"XObject".to_vec(), Object::Ref(images)),+ (b"Font".to_vec(), Object::Dict(Dict::from_iter([(b"T3".to_vec(), Object::Ref(font))]))),+ ]));+ let contents =+ doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"q 20 0 0 20 10 10 cm /Small Do Q BT /T3 50 Tf 50 80 Td (A) Tj ET".to_vec())));+ let page = doc.add(Dict::from_iter([+ (b"Type".to_vec(), Object::name("Page")),+ (b"Parent".to_vec(), Object::Ref(pages)),+ (b"MediaBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 160.into(), 160.into()])),+ (b"Resources".to_vec(), Object::Ref(resources)),+ (b"Contents".to_vec(), Object::Ref(contents)),+ ]));+ doc.update_dict(pages, |pages| {+ pages.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)]));+ pages.set(b"Count".to_vec(), 1);+ })+ .unwrap();+ write_full(&doc, &SaveOptions::default()).unwrap()+}++fn call(a: &mut Automation, tool: &str, args: Value) -> Value {+ a.call(tool, &args).unwrap().into_iter().find_map(|content| if let Content::Json(value) = content { Some(value) } else { None }).unwrap()+}++fn render(a: &mut Automation, doc: u64) -> Vec<u8> {+ let result = a.call("page_render", &json!({"doc":doc,"page":1,"dpi":72})).unwrap();+ let Content::Png { data, .. } = &result[0] else { panic!("expected PNG") };+ data.clone()+}++#[test]+fn extract_and_split_keep_indirect_type3_artwork_after_save_and_reopen() {+ let dir = tempfile::tempdir().unwrap();+ let mut a = Automation::new().with_root(dir.path()).unwrap();+ for (name, indirect) in [("direct", false), ("indirect", true)] {+ let source = fixture(indirect);+ std::fs::write(dir.path().join(format!("{name}.pdf")), &source).unwrap();+ let doc = call(&mut a, "doc_open", json!({"path":format!("{name}.pdf")}))["doc"].as_u64().unwrap();+ let before = render(&mut a, doc);+ let mut reader = png::Decoder::new(std::io::Cursor::new(&before)).read_info().unwrap();+ let mut pixels = vec![0; reader.output_buffer_size().unwrap()];+ let info = reader.next_frame(&mut pixels).unwrap();+ let channels = info.color_type.samples();+ assert!(channels >= 3);+ let green = pixels[..info.buffer_size()].chunks_exact(channels).filter(|pixel| pixel[1] > 100 && pixel[0] < 50 && pixel[2] < 50).count();+ assert_eq!(green, 2500, "the original contains a visible colored glyph");+ let extracted =+ call(&mut a, "page_extract", json!({"doc":doc,"pages":[1],"out":format!("{name}-extracted.pdf"),"open":true}))["document"]["doc"]+ .as_u64()+ .unwrap();+ assert_eq!(render(&mut a, extracted), before, "{name} extraction");+ call(&mut a, "doc_save", json!({"doc":extracted,"path":format!("{name}-saved.pdf"),"full":true}));+ call(&mut a, "doc_close", json!({"doc":extracted}));+ let saved = call(&mut a, "doc_open", json!({"path":format!("{name}-saved.pdf")}))["doc"].as_u64().unwrap();+ assert_eq!(render(&mut a, saved), before, "{name} save/reopen");+ let parts = call(&mut a, "doc_split", json!({"doc":doc,"every":1,"out_dir":format!("{name}-parts")}));+ let split = call(&mut a, "doc_open", json!({"path":parts["files"][0]["path"]}))["doc"].as_u64().unwrap();+ assert_eq!(render(&mut a, split), before, "{name} split");+ assert_eq!(std::fs::read(dir.path().join(format!("{name}.pdf"))).unwrap(), source);+ assert_eq!(call(&mut a, "doc_info", json!({"doc":doc}))["document"]["dirty"], false);+ }+}new file mode 100644@@ -0,0 +1,95 @@+//! Contributor-original shared resources exercise the public extract and split tools.++use pdfcraft_automation::{Automation, Content};+use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream, write_full};+use serde_json::{Value, json};++fn fixture() -> Vec<u8> {+ let mut doc = Document::new_empty();+ let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap();+ let mut images = Dict::new();+ for (name, color) in [("Page", [50, 50, 50]), ("Stamp", [0, 180, 0])] {+ let dict = Dict::from_iter([+ (b"Type".to_vec(), Object::name("XObject")),+ (b"Subtype".to_vec(), Object::name("Image")),+ (b"Width".to_vec(), 1.into()),+ (b"Height".to_vec(), 1.into()),+ (b"BitsPerComponent".to_vec(), 8.into()),+ (b"ColorSpace".to_vec(), Object::name("DeviceRGB")),+ ]);+ let image = doc.add(Object::Stream(Stream::from_raw(dict, color.to_vec())));+ images.set(name.as_bytes().to_vec(), Object::Ref(image));+ }+ let resources = doc.add(Dict::from_iter([(b"XObject".to_vec(), Object::Dict(images))]));+ let appearance = doc.add(Object::Stream(Stream::from_raw(+ Dict::from_iter([+ (b"Type".to_vec(), Object::name("XObject")),+ (b"Subtype".to_vec(), Object::name("Form")),+ (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 1.into(), 1.into()])),+ (b"Resources".to_vec(), Object::Ref(resources)),+ ]),+ b"/Stamp Do".to_vec(),+ )));+ let annotation = doc.add(Dict::from_iter([+ (b"Type".to_vec(), Object::name("Annot")),+ (b"Subtype".to_vec(), Object::name("Stamp")),+ (b"Rect".to_vec(), Object::Array(vec![50.into(), 50.into(), 110.into(), 110.into()])),+ (b"F".to_vec(), 4.into()),+ (b"AP".to_vec(), Object::Dict(Dict::from_iter([(b"N".to_vec(), Object::Ref(appearance))]))),+ ]));+ let contents = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"q 20 0 0 20 10 10 cm /Page Do Q".to_vec())));+ let mut kids = Vec::new();+ for _ in 0..2 {+ let page = doc.add(Dict::from_iter([+ (b"Type".to_vec(), Object::name("Page")),+ (b"Parent".to_vec(), Object::Ref(pages)),+ (b"MediaBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 160.into(), 160.into()])),+ (b"Resources".to_vec(), Object::Ref(resources)),+ (b"Contents".to_vec(), Object::Ref(contents)),+ (b"Annots".to_vec(), Object::Array(vec![Object::Ref(annotation)])),+ ]));+ kids.push(Object::Ref(page));+ }+ doc.update_dict(pages, |pages| {+ pages.set(b"Kids".to_vec(), Object::Array(kids));+ pages.set(b"Count".to_vec(), 2);+ })+ .unwrap();+ write_full(&doc, &SaveOptions::default()).unwrap()+}++fn call(a: &mut Automation, tool: &str, args: Value) -> Value {+ a.call(tool, &args).unwrap().into_iter().find_map(|content| if let Content::Json(value) = content { Some(value) } else { None }).unwrap()+}++fn render(a: &mut Automation, doc: u64) -> Vec<u8> {+ let result = a.call("page_render", &json!({"doc":doc,"page":1,"dpi":72})).unwrap();+ let Content::Png { data, width, height } = &result[0] else { panic!("expected PNG") };+ assert_eq!((*width, *height), (160, 160));+ data.clone()+}++#[test]+fn extract_and_split_preserve_shared_stamp_pixels_after_save_and_reopen() {+ let dir = tempfile::tempdir().unwrap();+ let source = fixture();+ std::fs::write(dir.path().join("source.pdf"), &source).unwrap();+ let mut a = Automation::new().with_root(dir.path()).unwrap();+ let doc = call(&mut a, "doc_open", json!({"path":"source.pdf"}))["doc"].as_u64().unwrap();+ let before = render(&mut a, doc);+ let extracted = call(&mut a, "page_extract", json!({"doc":doc,"pages":[1],"out":"extracted.pdf","open":true}));+ let extracted = extracted["document"]["doc"].as_u64().unwrap();+ assert_eq!(render(&mut a, extracted), before);+ call(&mut a, "doc_save", json!({"doc":extracted,"path":"extracted-saved.pdf","full":true}));+ call(&mut a, "doc_close", json!({"doc":extracted}));+ let reopened = call(&mut a, "doc_open", json!({"path":"extracted-saved.pdf"}))["doc"].as_u64().unwrap();+ assert_eq!(render(&mut a, reopened), before);+ let split = call(&mut a, "doc_split", json!({"doc":doc,"every":1,"out_dir":"parts"}));+ for file in split["files"].as_array().unwrap() {+ let part = call(&mut a, "doc_open", json!({"path":file["path"]}))["doc"].as_u64().unwrap();+ assert_eq!(render(&mut a, part), before);+ }+ assert_eq!(std::fs::read(dir.path().join("source.pdf")).unwrap(), source);+ let info = call(&mut a, "doc_info", json!({"doc":doc}));+ assert_eq!(info["document"]["dirty"], false);+}@@ -5,6 +5,8 @@ mod automation; mod automation_roots; mod batch; mod button_caption;+mod extract_indirect_kinds;+mod extract_paint_resources; mod form_bulk; mod hostile_args; mod optimize_content_boundaries;@@ -13,5 +13,8 @@ workspace = true [dependencies] pdfcraft-fonts = { workspace = true } ocrs = { version = "0.13.1", default-features = false, features = ["rten"] }-rten = { version = "0.26.0", default-features = false, features = ["rten_format"] }+rten = { version = "0.26.0", default-features = false, features = ["onnx_format", "rten_format"] } thiserror.workspace = true++[dev-dependencies]+tempfile = "3.27"@@ -15,8 +15,8 @@ use std::path::{Path, PathBuf}; pub use pdfcraft_fonts::helvetica_width; /// The model files, as named in ATTRIBUTION.toml.-pub const DETECTION_MODEL: &str = "text-detection.rten";-pub const RECOGNITION_MODEL: &str = "text-recognition.rten";+pub const DETECTION_MODEL: &str = "text-detection.onnx";+pub const RECOGNITION_MODEL: &str = "text-recognition.onnx"; /// The recognition model's alphabet, as it was trained (ocrs-models' `DEFAULT_ALPHABET`): its /// output classes follow this order. The ocrs crate's own copy has an `E` where the training had@@ -48,10 +48,13 @@ pub struct Models { } impl Models {- /// The models in `dir`, if both files are there.+ /// A complete model pair in `dir`, preferring current ONNX files over legacy RTEN files.+ /// An incomplete update never mixes formats or hides an intact legacy pair. pub fn in_dir(dir: &Path) -> Option<Models> {- let m = Models { detection: dir.join(DETECTION_MODEL), recognition: dir.join(RECOGNITION_MODEL) };- (m.detection.is_file() && m.recognition.is_file()).then_some(m)+ [(DETECTION_MODEL, RECOGNITION_MODEL), ("text-detection.rten", "text-recognition.rten")].into_iter().find_map(|(detection, recognition)| {+ let m = Models { detection: dir.join(detection), recognition: dir.join(recognition) };+ (m.detection.is_file() && m.recognition.is_file()).then_some(m)+ }) } /// Look for the models: `$PDFCRAFT_MODELS`, then where the release packages install them@@ -321,6 +324,63 @@ mod tests { assert_eq!(w.up, [-10.0, 0.0]); } + /// The current ONNX pair wins when an installation also retains both legacy models.+ #[test]+ fn model_discovery_prefers_the_complete_onnx_pair() {+ let dir = tempfile::tempdir().unwrap();+ for file in ["text-detection.onnx", "text-recognition.onnx"] {+ std::fs::write(dir.path().join(file), b"model").unwrap();+ }+ let expected = Some(Models { detection: dir.path().join("text-detection.onnx"), recognition: dir.path().join("text-recognition.onnx") });+ assert_eq!(Models::in_dir(dir.path()), expected, "a complete ONNX pair is sufficient");+ for file in ["text-detection.rten", "text-recognition.rten"] {+ std::fs::write(dir.path().join(file), b"model").unwrap();+ }+ assert_eq!(Models::in_dir(dir.path()), expected, "the complete ONNX pair takes precedence over legacy models");+ }++ /// An interrupted ONNX update still permits an intact legacy pair in the same directory.+ #[test]+ fn model_discovery_falls_back_to_legacy_when_the_onnx_pair_is_incomplete() {+ for current in ["text-detection.onnx", "text-recognition.onnx"] {+ let dir = tempfile::tempdir().unwrap();+ for file in [current, "text-detection.rten", "text-recognition.rten"] {+ std::fs::write(dir.path().join(file), b"model").unwrap();+ }+ assert_eq!(+ Models::in_dir(dir.path()),+ Some(Models { detection: dir.path().join("text-detection.rten"), recognition: dir.path().join("text-recognition.rten") }),+ "{current} alone must not hide a complete legacy pair"+ );+ }+ }++ /// Detection and recognition must come from one complete format pair, never a mixed update.+ #[test]+ fn model_discovery_refuses_mixed_format_pairs() {+ for pair in [["text-detection.onnx", "text-recognition.rten"], ["text-detection.rten", "text-recognition.onnx"]] {+ let dir = tempfile::tempdir().unwrap();+ for file in pair {+ std::fs::write(dir.path().join(file), b"model").unwrap();+ }+ assert_eq!(Models::in_dir(dir.path()), None, "{pair:?} is not a complete model pair");+ }+ }++ /// Existing source builds and PDFCRAFT_MODELS overrides retain their complete RTEN pair.+ #[test]+ fn model_discovery_keeps_legacy_only_installations() {+ let dir = tempfile::tempdir().unwrap();+ assert_eq!(Models::in_dir(dir.path()), None);+ std::fs::write(dir.path().join("text-detection.rten"), b"model").unwrap();+ assert_eq!(Models::in_dir(dir.path()), None, "one legacy model is incomplete");+ std::fs::write(dir.path().join("text-recognition.rten"), b"model").unwrap();+ assert_eq!(+ Models::in_dir(dir.path()),+ Some(Models { detection: dir.path().join("text-detection.rten"), recognition: dir.path().join("text-recognition.rten") })+ );+ }+ /// #103: every release package's layout puts the models where the installed app looks. /// Mirrors packaging/{windows/package.ps1,macos/package.sh,linux/package.sh,freebsd/package.sh}. #[test]new file mode 100644@@ -0,0 +1,24 @@+# pdfcraft-organize++Page and document-structure edits over the copy-on-write PDF object graph.+Callers keep snapshots for undo; the existing engine and automation layers expose Extract Pages and Split.++## Extract and Split resource preservation++Extraction copies retained pages and prunes page/Form XObject aliases proven unused by those pages.+Annotation appearances, tiling patterns, soft-mask groups and Type 3 fonts can share those resource dictionaries.+Their paint programs are not interpreted by this pruning pass, so their reachable resource owners are retained conservatively.+All normal, rollover and pressed appearance states are protected, including inactive states and appearances on pages without page content.+Resource-less paint also protects the caller's resources; discovery caches include that caller context.+Paint-kind names and numbers are resolved when stored through indirect objects, including Type 3 and Form Subtype, soft-mask S and tiling PatternType.+Primitive probes borrow direct values and fetch only indirect objects, preserving the traversal bounds for malformed containers.++Graph protection is iterative and bounded to 10,000 resource objects/direct children per extraction, with reference cycles visited once per caller and paint role.+Exhausting this budget skips pruning for the entire output rather than guessing that an image is unused.+Independent page resource dictionaries still prune, and shared ordinary graphics-state catalogs are discovered once per owner.+A nested resource-less paint program can conservatively retain extra caller XObjects even when a nearer explicit resource dictionary would suffice.+This favors keeping artwork over reducing file size.++The regression suite constructs contributor-original fixtures in Rust source.+`cargo test -p pdfcraft-organize --test paint_resources` covers aliasing, inherited resources, malformed containers, cycles, the work cap, large shared catalogs, splitting and a Poppler pixel comparison when that external oracle is installed.+`extract_and_split_preserve_shared_stamp_pixels_after_save_and_reopen` exercises the public automation tools and rendered output through full save/reopen without changing the source file.@@ -15,7 +15,7 @@ use crate::OrganizeError; /// Form XObjects nested deeper than this are a broken file, not a reason to hang. const MAX_FORM_DEPTH: u8 = 8; /// Count every resource object and direct child; a graph beyond this is left unpruned.-const MAX_GLYPH_RESOURCE_WORK: usize = 10_000;+const MAX_PAINT_RESOURCE_WORK: usize = 10_000; /// The dictionary a set of drawn names belongs to. #[derive(Clone, Copy, PartialEq, Eq, Hash)]@@ -44,123 +44,221 @@ struct Pruner<'a> { /// past the depth limit, Type 3 glyphs drawing through them). They are left untouched: /// pruning must never drop an XObject something still draws. unsure: HashSet<XKey>,- /// An overlarge glyph-resource graph is not safe to prune at all.- glyph_budget_exhausted: bool,- glyph_work_left: usize,- glyph_seen: HashSet<ObjRef>,- /// An indirect font's explicit resources have one owner regardless of which page uses it.- glyph_fonts: HashSet<ObjRef>,+ /// An incomplete unparsed-paint resource graph is not safe to prune at all.+ paint_budget_exhausted: bool,+ paint_work_left: usize,+ /// A resource-less paint can inherit from each caller. Its appearance role also matters:+ /// an AP stream may omit Subtype, unlike opaque font/profile data reached through resources.+ paint_seen: HashSet<(ObjRef, Owner, bool)>,+ /// An owner's resource catalog is immutable during collection. Shared catalogs and the+ /// content/error paths must not repeatedly consume the protection budget for the same owner.+ resources_seen: HashSet<Owner>,+ /// Nested resource-less paint in a font can still depend on each caller's resources.+ glyph_fonts: HashSet<(ObjRef, Owner)>, /// Forms already analysed, with the owner of the resources they drew against: drawing one /// again records nothing new, and re-reading it on every `Do` is exponential in the nesting. forms_seen: HashSet<(ObjRef, Owner)>, } impl Pruner<'_> {+ /// Kind entries may reference primitive objects. Inspect direct values in place, and only+ /// fetch an indirect value, so malformed direct containers are never cloned by a probe.+ fn name_is(&self, value: Option<&Object>, expected: &[u8]) -> bool {+ let Some(value) = value else { return false };+ match value.as_ref() {+ Some(reference) => self.doc.get(reference).as_name() == Some(expected),+ None => value.as_name() == Some(expected),+ }+ }++ fn integer_is(&self, value: Option<&Object>, expected: i64) -> bool {+ let Some(value) = value else { return false };+ match value.as_ref() {+ Some(reference) => self.doc.get(reference).as_int() == Some(expected),+ None => value.as_int() == Some(expected),+ }+ }++ /// Inspect direct containers in place; resolving a direct dictionary would clone it before+ /// the protection work budget has charged its children.+ fn usable_resources(&self, object: &Object) -> bool {+ match object.as_ref() {+ Some(reference) => self.doc.get(reference).as_dict().is_some(),+ None => object.as_dict().is_some(),+ }+ }+ /// Mark the `/XObject` dictionary of `resources` as one not to prune. fn distrust(&mut self, resources: &Dict, owner: Owner) {- if let Some((_, key)) = self.xobjects(resources, owner) {+ if let Some(xobjects) = resources.get(b"XObject") {+ let key = match (xobjects.as_ref(), owner) {+ (Some(reference), _) => XKey::XObject(reference),+ (None, Owner::Resources(reference)) => XKey::Resources(reference),+ (None, Owner::Page(reference)) => XKey::Page(reference),+ }; self.unsure.insert(key); } } - fn glyph_work(&mut self, count: usize) -> bool {- if !self.glyph_budget_exhausted- && let Some(left) = self.glyph_work_left.checked_sub(count)+ fn paint_work(&mut self, count: usize) -> bool {+ if !self.paint_budget_exhausted+ && let Some(left) = self.paint_work_left.checked_sub(count) {- self.glyph_work_left = left;+ self.paint_work_left = left; return true; }- self.glyph_budget_exhausted = true;+ self.paint_budget_exhausted = true; false } /// Walk one object's direct children without cloning them or recursing. Referenced objects /// go on a separate work list, so their borrowed dictionaries stay alive during this pass.- fn protect_glyph_tree(&mut self, object: &Object, referenced: &mut Vec<ObjRef>) {- if !self.glyph_work(1) {- return;+ fn protect_paint_tree(&mut self, object: &Object, owner: Owner, appearance: bool, referenced: &mut Vec<(ObjRef, bool)>) -> bool {+ if !self.paint_work(1) {+ return false; }- let mut pending = vec![object];- while let Some(object) = pending.pop() {+ let mut inherits = false;+ let mut pending = vec![(object, appearance)];+ while let Some((object, appearance)) = pending.pop() {+ if let Some(dict) = object.as_dict() {+ let paint_stream = matches!(object, Object::Stream(_))+ && (appearance || self.name_is(dict.get(b"Subtype"), b"Form") || self.integer_is(dict.get(b"PatternType"), 1));+ if (paint_stream || self.name_is(dict.get(b"Subtype"), b"Type3"))+ && !dict.get(b"Resources").is_some_and(|resources| self.usable_resources(resources))+ {+ inherits = true;+ }+ } match object { Object::Ref(r) => {- // A glyph's forms, patterns and soft masks can share either the XObject- // dictionary itself or a resource owner with content we do interpret.+ // Unparsed paint can share either the XObject dictionary itself or a+ // resource owner with page/Form content we do interpret. self.unsure.insert(XKey::XObject(*r)); self.unsure.insert(XKey::Resources(*r)); self.unsure.insert(XKey::Page(*r));- if self.glyph_seen.insert(*r) {- referenced.push(*r);+ if self.paint_seen.insert((*r, owner, appearance)) {+ referenced.push((*r, appearance)); } } Object::Array(array) => {- if !self.glyph_work(array.len()) {- return;+ if !self.paint_work(array.len()) {+ return inherits; }- pending.extend(array);+ pending.extend(array.iter().map(|value| (value, appearance))); } Object::Dict(_) | Object::Stream(_) => { if let Some(dict) = object.as_dict() {- if !self.glyph_work(dict.len()) {- return;+ if !self.paint_work(dict.len()) {+ return inherits; }- pending.extend(dict.iter().map(|(_, value)| value));+ // Appearance dictionaries contain N/R/D and named states. Once a+ // stream is reached, its dictionary holds ordinary resource/data graphs.+ let child_appearance = appearance && !matches!(object, Object::Stream(_));+ let soft_mask = self.name_is(dict.get(b"S"), b"Luminosity") || self.name_is(dict.get(b"S"), b"Alpha");+ pending.extend(dict.iter().map(|(key, value)| (value, child_appearance || (soft_mask && key.as_slice() == b"G")))); } } _ => {} } }+ inherits } - /// Glyph programs are not interpreted here. Keep every resource owner reachable from their- /// resource graph, including nested forms, patterns and soft masks. Data bytes are never- /// decoded; references are visited once and all direct children count against the budget.- fn protect_glyph_object(&mut self, object: &Object) {+ /// Keep every owner reachable from unparsed paint, without decoding any stream bytes.+ /// Returns whether a paint program may also use its caller's resources.+ fn protect_paint_object(&mut self, object: &Object, owner: Owner, appearance: bool) -> bool { let mut referenced = Vec::new();- self.protect_glyph_tree(object, &mut referenced);- while !self.glyph_budget_exhausted {- let Some(r) = referenced.pop() else { break };+ let mut inherits = self.protect_paint_tree(object, owner, appearance, &mut referenced);+ while !self.paint_budget_exhausted {+ let Some((r, appearance)) = referenced.pop() else { break }; let object = self.doc.get(r);- self.protect_glyph_tree(&object, &mut referenced);+ inherits |= self.protect_paint_tree(&object, owner, appearance, &mut referenced);+ }+ inherits+ }++ fn protect_caller(&mut self, resources: &Dict, owner: Owner) {+ self.distrust(resources, owner);+ for (_, value) in resources.iter() {+ self.protect_paint_object(value, owner, false);+ if self.paint_budget_exhausted {+ break;+ }+ }+ }++ /// Patterns and graphics states can paint through soft masks or select Type 3 fonts.+ /// Their content is not interpreted by this pruner, so protect any aliased resource owners.+ fn protect_resources(&mut self, resources: &Dict, owner: Owner) {+ if self.paint_budget_exhausted || !self.resources_seen.insert(owner) {+ return;+ }+ self.protect_type3_resources(resources, owner);+ for key in [b"Pattern".as_slice(), b"ExtGState"] {+ if let Some(root) = resources.get(key)+ && self.protect_paint_object(root, owner, false)+ {+ self.protect_caller(resources, owner);+ }+ }+ }++ fn protect_appearances(&mut self, page: ObjRef, dict: &Dict, caller: Option<&(Dict, Owner)>) {+ let Some(annotations) = dict.get(b"Annots") else { return };+ let resolved = annotations.as_ref().map(|reference| self.doc.get(reference));+ let Some(annotations) = resolved.as_deref().unwrap_or(annotations).as_array() else { return };+ if !self.paint_work(annotations.len()) {+ return;+ }+ let owner = caller.map_or(Owner::Page(page), |(_, owner)| *owner);+ for annotation in annotations {+ let resolved = annotation.as_ref().map(|reference| self.doc.get(reference));+ let annotation = resolved.as_deref().unwrap_or(annotation);+ if let Some(appearance) = annotation.as_dict().and_then(|dict| dict.get(b"AP"))+ && self.protect_paint_object(appearance, owner, true)+ && let Some((resources, owner)) = caller+ {+ self.protect_caller(resources, *owner);+ }+ if self.paint_budget_exhausted {+ break;+ } } } fn protect_type3_resources(&mut self, resources: &Dict, owner: Owner) {- if self.glyph_budget_exhausted {+ if self.paint_budget_exhausted { return; }- let Some(fonts) = resources.get(b"Font").map(|f| self.doc.resolve(f)) else { return };- let Some(fonts) = fonts.as_dict() else { return };+ let Some(fonts) = resources.get(b"Font") else { return };+ let resolved = fonts.as_ref().map(|reference| self.doc.get(reference));+ let Some(fonts) = resolved.as_deref().unwrap_or(fonts).as_dict() else { return }; // Bound discovery too, but don't spend the glyph-graph budget repeatedly examining an // ordinary shared font catalog on every page of a large document.- if fonts.len() > MAX_GLYPH_RESOURCE_WORK {- self.glyph_budget_exhausted = true;+ if fonts.len() > MAX_PAINT_RESOURCE_WORK {+ self.paint_budget_exhausted = true; return; } for (_, font) in fonts.iter() {- if self.glyph_budget_exhausted {+ if self.paint_budget_exhausted { return; } let font_ref = font.as_ref();- let resolved = self.doc.resolve(font);- let Some(font) = resolved.as_dict().filter(|f| f.name(b"Subtype") == Some(b"Type3")) else { continue };- if let Some(glyph_resources) = font.get(b"Resources") {+ let resolved = font_ref.map(|reference| self.doc.get(reference));+ let Some(font) = resolved.as_deref().unwrap_or(font).as_dict().filter(|font| self.name_is(font.get(b"Subtype"), b"Type3")) else {+ continue;+ };+ if let Some(glyph_resources) = font.get(b"Resources").filter(|resources| self.usable_resources(resources)) { if let Some(r) = font_ref- && !self.glyph_fonts.insert(r)+ && !self.glyph_fonts.insert((r, owner)) { continue; }- self.protect_glyph_object(glyph_resources);- } else {- self.distrust(resources, owner);- // This root dictionary is already owned by the content caller. Only its- // referenced descendants can share additional owners elsewhere in the file.- for (_, value) in resources.iter() {- self.protect_glyph_object(value);- if self.glyph_budget_exhausted {- return;- }+ if self.protect_paint_object(glyph_resources, owner, false) {+ self.protect_caller(resources, owner); }+ } else {+ self.protect_caller(resources, owner); } } }@@ -180,7 +278,7 @@ impl Pruner<'_> { /// Record every XObject name `data` draws with `Do`, descending into form XObjects. fn content(&mut self, data: &[u8], resources: &Dict, owner: Owner, depth: u8) {- self.protect_type3_resources(resources, owner);+ self.protect_resources(resources, owner); let Some((xobjects, key)) = self.xobjects(resources, owner) else { return; };@@ -196,7 +294,7 @@ impl Pruner<'_> { self.used.entry(key).or_default().insert(name.to_vec()); let Some(r) = xobjects.get(name).and_then(Object::as_ref) else { continue }; let Object::Stream(s) = &*self.doc.get(r) else { continue };- if s.dict.name(b"Subtype") != Some(b"Form") {+ if !self.name_is(s.dict.get(b"Subtype"), b"Form") { continue; } let (inner, inner_owner) = match s.dict.get(b"Resources") {@@ -223,7 +321,7 @@ impl Pruner<'_> { Ok(data) => self.content(&data, &inner, inner_owner, depth + 1), // What the form draws is unknown: keep everything it could reach. Err(_) => {- self.protect_type3_resources(&inner, inner_owner);+ self.protect_resources(&inner, inner_owner); self.distrust(&inner, inner_owner); } }@@ -311,28 +409,37 @@ pub(crate) fn prune_unused_xobjects(doc: &mut Document) -> Result<usize, Organiz doc, used: HashMap::new(), unsure: HashSet::new(),- glyph_budget_exhausted: false,- glyph_work_left: MAX_GLYPH_RESOURCE_WORK,- glyph_seen: HashSet::new(),+ paint_budget_exhausted: false,+ paint_work_left: MAX_PAINT_RESOURCE_WORK,+ paint_seen: HashSet::new(),+ resources_seen: HashSet::new(), glyph_fonts: HashSet::new(), forms_seen: HashSet::new(), }; for (page, attrs) in &pages {- let Some(res) = attrs.get(b"Resources") else { continue };- let Some((res_dict, owner)) = resolve_resources(pruner.doc, res, *page) else { continue };- let Some(pd) = pruner.doc.get(*page).as_dict().cloned() else { continue };- let Some(contents) = pd.get(b"Contents").cloned() else { continue };- match content_bytes(pruner.doc, &contents) {+ let pd = pruner.doc.get(*page);+ let Some(pd) = pd.as_dict() else { continue };+ let caller = attrs.get(b"Resources").and_then(|res| resolve_resources(pruner.doc, res, *page));+ pruner.protect_appearances(*page, pd, caller.as_ref());+ if pruner.paint_budget_exhausted {+ return Ok(0);+ }+ let Some((res_dict, owner)) = caller else { continue };+ pruner.protect_resources(&res_dict, owner);+ if pruner.paint_budget_exhausted {+ return Ok(0);+ }+ let Some(contents) = pd.get(b"Contents") else { continue };+ match content_bytes(pruner.doc, contents) { Ok(data) if data.is_empty() => {} Ok(data) => pruner.content(&data, &res_dict, owner, 0), Err(_) => {- pruner.protect_type3_resources(&res_dict, owner); pruner.distrust(&res_dict, owner); } } }- let Pruner { used, unsure, glyph_budget_exhausted, .. } = pruner;- if glyph_budget_exhausted {+ let Pruner { used, unsure, paint_budget_exhausted, .. } = pruner;+ if paint_budget_exhausted { return Ok(0); } let mut removed = 0;new file mode 100644@@ -0,0 +1,176 @@+//! Paint kind entries can name their primitive values through indirect objects.++use std::sync::Arc;++use pdfcraft_cos::{Dict, Document, ObjRef, Object, SaveOptions, Stream, write_full};+use pdfcraft_organize::{SplitBy, extract_pages, pages, split};++#[derive(Clone, Copy, Debug)]+enum Kind {+ Type3,+ Form,+ Pattern,+ Mask,+}++fn primitive(doc: &mut Document, value: Object, indirect: bool) -> Object {+ if indirect { Object::Ref(doc.add(value)) } else { value }+}++fn fixture(kind: Kind, indirect: bool) -> Document {+ let mut doc = Document::new_empty();+ let tree = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap();+ let mut images = Dict::new();+ for (name, color) in [("Small", [50, 50, 50]), ("Paint", [0, 180, 0])] {+ let attrs = Dict::from_iter([+ (b"Subtype".to_vec(), Object::name("Image")),+ (b"Width".to_vec(), 1.into()),+ (b"Height".to_vec(), 1.into()),+ (b"BitsPerComponent".to_vec(), 8.into()),+ (b"ColorSpace".to_vec(), Object::name("DeviceRGB")),+ ]);+ let image = doc.add(Object::Stream(Stream::from_raw(attrs, color.to_vec())));+ images.set(name.as_bytes().to_vec(), Object::Ref(image));+ }+ let images = doc.add(images);+ let mut resources = Dict::from_iter([(b"XObject".to_vec(), Object::Ref(images))]);+ let mut content = "q 20 0 0 20 10 10 cm /Small Do Q ".to_string();+ match kind {+ Kind::Type3 => {+ let program = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"1000 0 d0 q 1000 0 0 1000 0 0 cm /Paint Do Q".to_vec())));+ let subtype = primitive(&mut doc, Object::name("Type3"), indirect);+ let font = doc.add(Dict::from_iter([+ (b"Type".to_vec(), Object::name("Font")),+ (b"Subtype".to_vec(), subtype),+ (b"FontBBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 1000.into(), 1000.into()])),+ (b"FontMatrix".to_vec(), Object::Array(vec![Object::Real(0.001), 0.into(), 0.into(), Object::Real(0.001), 0.into(), 0.into()])),+ (b"CharProcs".to_vec(), Object::Dict(Dict::from_iter([(b"A".to_vec(), Object::Ref(program))]))),+ (b"Encoding".to_vec(), Object::Dict(Dict::from_iter([(b"Differences".to_vec(), Object::Array(vec![65.into(), Object::name("A")]))]))),+ (b"FirstChar".to_vec(), 65.into()),+ (b"LastChar".to_vec(), 65.into()),+ (b"Widths".to_vec(), Object::Array(vec![1000.into()])),+ (b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"XObject".to_vec(), Object::Ref(images))]))),+ ]));+ resources.set(b"Font".to_vec(), Object::Dict(Dict::from_iter([(b"T3".to_vec(), Object::Ref(font))])));+ content.push_str("BT /T3 50 Tf 50 80 Td (A) Tj ET");+ }+ Kind::Form => {+ let subtype = primitive(&mut doc, Object::name("Form"), indirect);+ let form = doc.add(Object::Stream(Stream::from_raw(+ Dict::from_iter([+ (b"Subtype".to_vec(), subtype),+ (b"BBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 1.into(), 1.into()])),+ (b"Resources".to_vec(), Object::Dict(Dict::from_iter([(b"XObject".to_vec(), Object::Ref(images))]))),+ ]),+ b"/Paint Do".to_vec(),+ )));+ doc.update_dict(images, |images| images.set(b"Fm".to_vec(), Object::Ref(form))).unwrap();+ content.push_str("q 50 0 0 50 50 80 cm /Fm Do Q");+ }+ Kind::Pattern => {+ // Malformed resource-less paint is conservatively treated as possibly inherited,+ // whether its kind primitive is direct or indirect.+ let pattern_type = primitive(&mut doc, 1.into(), indirect);+ let pattern =+ doc.add(Object::Stream(Stream::from_raw(Dict::from_iter([(b"PatternType".to_vec(), pattern_type)]), b"/Paint Do".to_vec())));+ resources.set(b"Pattern".to_vec(), Object::Dict(Dict::from_iter([(b"Tile".to_vec(), Object::Ref(pattern))])));+ }+ Kind::Mask => {+ let group = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"/Paint Do".to_vec())));+ let mode = primitive(&mut doc, Object::name("Luminosity"), indirect);+ let mask = Object::Dict(Dict::from_iter([(b"S".to_vec(), mode), (b"G".to_vec(), Object::Ref(group))]));+ resources.set(+ b"ExtGState".to_vec(),+ Object::Dict(Dict::from_iter([(b"GS".to_vec(), Object::Dict(Dict::from_iter([(b"SMask".to_vec(), mask)])))])),+ );+ }+ }+ let resources = doc.add(resources);+ let contents = doc.add(Object::Stream(Stream::from_raw(Dict::new(), content.into_bytes())));+ let page = doc.add(Dict::from_iter([+ (b"Type".to_vec(), Object::name("Page")),+ (b"Parent".to_vec(), Object::Ref(tree)),+ (b"MediaBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 160.into(), 160.into()])),+ (b"Resources".to_vec(), Object::Ref(resources)),+ (b"Contents".to_vec(), Object::Ref(contents)),+ ]));+ doc.update_dict(tree, |tree| {+ tree.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)]));+ tree.set(b"Count".to_vec(), 1);+ })+ .unwrap();+ doc+}++fn image_names(doc: &Document) -> Vec<Vec<u8>> {+ let page = doc.get(pages(doc).unwrap()[0].obj);+ let resources = doc.resolve(page.as_dict().unwrap().get(b"Resources").unwrap());+ let images = doc.resolve(resources.as_dict().unwrap().get(b"XObject").unwrap());+ images.as_dict().unwrap().iter().map(|(name, _)| name.clone()).collect()+}++fn check(kind: Kind) {+ for indirect in [false, true] {+ let source = fixture(kind, indirect);+ let before = write_full(&source, &SaveOptions::default()).unwrap();+ for output in [extract_pages(&source, &[0]).unwrap(), split(&source, &SplitBy::PageCount(1)).unwrap().remove(0)] {+ let saved = Document::open(Arc::new(write_full(&output, &SaveOptions::default()).unwrap())).unwrap();+ assert!(image_names(&saved).iter().any(|name| name == b"Paint"), "{kind:?}, indirect={indirect}");+ }+ assert_eq!(write_full(&source, &SaveOptions::default()).unwrap(), before);+ }+}++#[test]+fn extract_keeps_images_used_by_indirect_type3_subtypes() {+ check(Kind::Type3);+}++#[test]+fn extract_keeps_images_used_by_indirect_form_subtypes() {+ check(Kind::Form);+}++#[test]+fn extract_protects_inherited_paint_with_indirect_kind_metadata() {+ check(Kind::Pattern);+ check(Kind::Mask);+}++#[test]+fn indirect_kind_extraction_matches_poppler_pixels() {+ let pdftoppm = pdfcraft_oracle::or_skip!(pdfcraft_oracle::pdftoppm());+ let dir = std::env::temp_dir().join(format!("pdfcraft-indirect-kinds-{}", std::process::id()));+ std::fs::create_dir_all(&dir).unwrap();+ for (index, kind) in [Kind::Type3, Kind::Form].into_iter().enumerate() {+ let source = fixture(kind, true);+ let render = |doc: &Document, stage: &str| {+ let path = dir.join(format!("{index}-{stage}.pdf"));+ std::fs::write(&path, write_full(doc, &SaveOptions::default()).unwrap()).unwrap();+ let result = pdftoppm.command().args(["-singlefile", "-r", "72", "-gray"]).arg(path).output().unwrap();+ assert!(result.status.success(), "{}", String::from_utf8_lossy(&result.stderr));+ assert!(result.stderr.is_empty(), "{}", String::from_utf8_lossy(&result.stderr));+ result.stdout+ };+ let before = render(&source, "source");+ assert!(before.iter().filter(|byte| **byte < 200).count() > 2500, "the source has substantial artwork");+ assert_eq!(render(&extract_pages(&source, &[0]).unwrap(), "extracted"), before);+ }+ std::fs::remove_dir_all(dir).unwrap();+}++#[test]+fn malformed_indirect_subtypes_terminate_without_panicking() {+ for value in [Some(Object::Null), Some(Object::Int(1)), Some(Object::Ref(ObjRef::new(999_999, 0))), None] {+ let mut doc = fixture(Kind::Form, true);+ let page = doc.get(pages(&doc).unwrap()[0].obj);+ let resources = doc.resolve(page.as_dict().unwrap().get(b"Resources").unwrap());+ let images = doc.resolve(resources.as_dict().unwrap().get(b"XObject").unwrap());+ let form = images.as_dict().unwrap().reference(b"Fm").unwrap();+ let subtype = doc.get(form).as_dict().unwrap().reference(b"Subtype").unwrap();+ // The final case is a self-referencing metadata object, not a name.+ doc.set(subtype, value.unwrap_or(Object::Ref(subtype)));+ let output = extract_pages(&doc, &[0]).unwrap();+ assert!(image_names(&output).iter().any(|name| name == b"Small"));+ }+}new file mode 100644@@ -0,0 +1,276 @@+//! Extraction must keep resources used by paint programs the XObject pruner does not interpret.++use std::sync::Arc;++use pdfcraft_cos::{Dict, Document, ObjRef, Object, SaveOptions, Stream, write_full};+use pdfcraft_organize::{SplitBy, extract_pages, pages, split};++fn stream(attributes: &str, content: &str) -> String {+ format!("<< {attributes} /Length {} >>\nstream\n{content}\nendstream", content.len())+}++fn build(objects: &[String]) -> Document {+ let mut out = b"%PDF-1.7\n".to_vec();+ let mut offsets = Vec::new();+ for (i, object) in objects.iter().enumerate() {+ offsets.push(out.len());+ out.extend_from_slice(format!("{} 0 obj\n{object}\nendobj\n", i + 1).as_bytes());+ }+ let xref = out.len();+ out.extend_from_slice(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).as_bytes());+ for offset in offsets {+ out.extend_from_slice(format!("{offset:010} 00000 n \n").as_bytes());+ }+ out.extend_from_slice(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objects.len() + 1).as_bytes());+ Document::open(Arc::new(out)).unwrap()+}++/// Page content draws ImA; another paint program draws ImB through the same resource owner.+/// Unused has no paint use, so separately owned resources can still be pruned.+fn fixture(extra_resources: &str, page_content: &str, annotation: &str, paint_attributes: &str, paint: &str) -> Document {+ build(&[+ "<< /Type /Catalog /Pages 2 0 R >>".into(),+ "<< /Type /Pages /Kids [3 0 R 13 0 R] /Count 2 /MediaBox [0 0 160 160] /Resources 5 0 R >>".into(),+ "<< /Type /Page /Parent 2 0 R /Contents 4 0 R /Annots [12 0 R] /Resources 5 0 R >>".into(),+ stream("", &format!("q 20 0 0 20 10 10 cm /ImA Do Q {page_content}")),+ format!("<< /XObject 6 0 R {extra_resources} >>"),+ "<< /ImA 7 0 R /ImB 8 0 R /Unused 9 0 R >>".into(),+ stream("/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", "2"),+ stream("/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", "x"),+ stream("/Type /XObject /Subtype /Image /Width 1 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", &"z".repeat(2000)),+ stream(paint_attributes, paint),+ "<< /S /Luminosity /G 10 0 R >>".into(),+ format!("<< /Type /Annot /Subtype /Stamp /Rect [50 50 110 110] /F 4 {annotation} >>"),+ "<< /Type /Page /Parent 2 0 R /Contents 14 0 R >>".into(),+ stream("", "q 20 0 0 20 10 10 cm /ImA Do Q"),+ ])+}++fn roundtrip(doc: &Document) -> Document {+ Document::open(Arc::new(write_full(doc, &SaveOptions::default()).unwrap())).unwrap()+}++fn xobjects(doc: &Document, page: usize) -> Vec<Vec<u8>> {+ let page = doc.get(pages(doc).unwrap()[page].obj);+ let resources = doc.resolve(page.as_dict().unwrap().get(b"Resources").unwrap());+ let xobjects = doc.resolve(resources.as_dict().unwrap().get(b"XObject").unwrap());+ xobjects.as_dict().unwrap().iter().map(|(name, _)| name.clone()).collect()+}++fn assert_shared_image_survives(doc: &Document, selection: &[usize]) {+ let original = write_full(doc, &SaveOptions::default()).unwrap();+ let extracted = roundtrip(&extract_pages(doc, selection).unwrap());+ assert!(xobjects(&extracted, selection.len() - 1).iter().any(|name| name == b"ImB"), "a paint program still draws ImB");+ assert_eq!(write_full(doc, &SaveOptions::default()).unwrap(), original, "source remains unchanged");+}++#[test]+fn extract_preserves_shared_annotation_appearance_resources() {+ for kind in ["N", "R", "D"] {+ for state in ["10 0 R", "<< /Off 10 0 R /On 10 0 R >>"] {+ for resources in ["/Resources 5 0 R", "/Resources << /XObject 6 0 R >>", "", "/Resources null", "/Resources 999 0 R", "/Resources []"] {+ let doc = fixture(+ "",+ "",+ &format!("/AP << /{kind} {state} >> /AS /Missing"),+ &format!("/Subtype /Form /BBox [0 0 1 1] {resources}"),+ "/ImB Do",+ );+ assert_shared_image_survives(&doc, &[0]);+ }+ }+ }+}++#[test]+fn extract_preserves_indirect_appearance_and_annotation_containers() {+ let mut doc = fixture("", "", "/AP << /N << /Off 10 0 R /On 10 0 R >> >>", "/Subtype /Form /BBox [0 0 1 1] /Resources 5 0 R", "/ImB Do");+ let ap = doc.add(doc.get(ObjRef::new(12, 0)).as_dict().unwrap().get(b"AP").unwrap().clone());+ doc.update_dict(ObjRef::new(12, 0), |annotation| annotation.set(b"AP".to_vec(), Object::Ref(ap))).unwrap();+ let annotations = doc.add(Object::Array(vec![Object::Ref(ObjRef::new(12, 0)), Object::Null, Object::Int(7)]));+ doc.update_dict(ObjRef::new(3, 0), |page| page.set(b"Annots".to_vec(), Object::Ref(annotations))).unwrap();+ assert_shared_image_survives(&doc, &[0]);+}++#[test]+fn extract_preserves_appearance_resources_on_a_page_without_contents_or_resources() {+ let mut doc = fixture("", "", "/AP << /N 10 0 R >>", "/BBox [0 0 1 1] /Resources 5 0 R", "/ImB Do");+ let page = pages(&doc).unwrap()[0].obj;+ doc.update_dict(page, |page| {+ page.remove(b"Contents");+ page.set(b"Resources".to_vec(), Object::Dict(Default::default()));+ })+ .unwrap();+ assert_shared_image_survives(&doc, &[0, 1]);+}++#[test]+fn extract_protects_each_caller_of_a_shared_resource_less_appearance() {+ let mut doc = fixture("", "", "/AP << /N 10 0 R >>", "/Subtype /Form /BBox [0 0 1 1]", "/ImB Do");+ let other_resources = doc.add((*doc.get(ObjRef::new(5, 0))).clone());+ let other_xobjects = doc.add((*doc.get(ObjRef::new(6, 0))).clone());+ doc.update_dict(other_resources, |resources| resources.set(b"XObject".to_vec(), Object::Ref(other_xobjects))).unwrap();+ doc.update_dict(ObjRef::new(13, 0), |page| {+ page.set(b"Resources".to_vec(), Object::Ref(other_resources));+ page.set(b"Annots".to_vec(), Object::Array(vec![Object::Ref(ObjRef::new(12, 0))]));+ })+ .unwrap();+ let extracted = roundtrip(&extract_pages(&doc, &[0, 1]).unwrap());+ for page in 0..2 {+ assert!(xobjects(&extracted, page).iter().any(|name| name == b"ImB"), "caller page {page}");+ }+}++#[test]+fn extract_preserves_shared_tiling_pattern_and_soft_mask_resources() {+ let pattern = fixture(+ "/Pattern << /Tile 10 0 R >>",+ "/Pattern cs /Tile scn 50 50 60 60 re f",+ "",+ "/Type /Pattern /PatternType 1 /PaintType 1 /TilingType 1 /BBox [0 0 1 1] /XStep 1 /YStep 1 /Resources 5 0 R",+ "/ImB Do",+ );+ assert_shared_image_survives(&pattern, &[0]);+ let mask = fixture(+ "/ExtGState << /GS << /SMask 11 0 R >> >>",+ "/GS gs 50 50 60 60 re f",+ "",+ "/Subtype /Form /BBox [0 0 1 1] /Group << /S /Transparency /CS /DeviceGray >> /Resources 5 0 R",+ "/ImB Do",+ );+ assert_shared_image_survives(&mask, &[0]);+ let inherited_mask = fixture("/ExtGState << /GS << /SMask 11 0 R >> >>", "/GS gs 50 50 60 60 re f", "", "/BBox [0 0 1 1]", "/ImB Do");+ assert_shared_image_survives(&inherited_mask, &[0]);+}++#[test]+fn extract_preserves_resources_of_a_type3_font_selected_by_extgstate() {+ let mut doc = fixture("", "/GS gs BT 50 80 Td (A) Tj ET", "", "", "");+ let glyph = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"1 0 0 0 1 1 d1 /ImB Do".to_vec())));+ let mut charprocs = Dict::new();+ charprocs.set(b"A".to_vec(), Object::Ref(glyph));+ let mut font = Dict::new();+ font.set(b"Subtype".to_vec(), Object::name("Type3"));+ font.set(b"CharProcs".to_vec(), Object::Dict(charprocs));+ let font = doc.add(font);+ let mut state = Dict::new();+ state.set(b"Font".to_vec(), Object::Array(vec![Object::Ref(font), Object::Int(50)]));+ let mut states = Dict::new();+ states.set(b"GS".to_vec(), Object::Dict(state));+ let states = doc.add(states);+ doc.update_dict(ObjRef::new(5, 0), |resources| resources.set(b"ExtGState".to_vec(), Object::Ref(states))).unwrap();+ assert_shared_image_survives(&doc, &[0]);+ // Discovering this catalog first through a different Type 3 font must not swallow the+ // inherited-resource requirement of the graphics-state font selected by page content.+ let mut font = Dict::new();+ font.set(b"Subtype".to_vec(), Object::name("Type3"));+ let mut font_resources = Dict::new();+ font_resources.set(b"ExtGState".to_vec(), Object::Ref(states));+ font.set(b"Resources".to_vec(), Object::Dict(font_resources));+ let font = doc.add(font);+ let mut fonts = Dict::new();+ fonts.set(b"UnusedType3".to_vec(), Object::Ref(font));+ doc.update_dict(ObjRef::new(5, 0), |resources| resources.set(b"Font".to_vec(), Object::Dict(fonts))).unwrap();+ assert_shared_image_survives(&doc, &[0]);+ // The same font graph can inherit from two different retained page resource owners.+ doc.update_dict(ObjRef::new(5, 0), |resources| {+ resources.remove(b"ExtGState");+ })+ .unwrap();+ let second_resources = doc.add((*doc.get(ObjRef::new(5, 0))).clone());+ let second_xobjects = doc.add((*doc.get(ObjRef::new(6, 0))).clone());+ doc.update_dict(second_resources, |resources| resources.set(b"XObject".to_vec(), Object::Ref(second_xobjects))).unwrap();+ doc.update_dict(ObjRef::new(13, 0), |page| page.set(b"Resources".to_vec(), Object::Ref(second_resources))).unwrap();+ let extracted = extract_pages(&doc, &[0, 1]).unwrap();+ for page in 0..2 {+ assert!(xobjects(&extracted, page).iter().any(|name| name == b"ImB"), "font's inherited caller {page}");+ }+}++#[test]+fn extract_skips_pruning_after_a_paint_graph_exceeds_the_work_budget() {+ let mut doc = fixture("", "", "/AP << /N 10 0 R >>", "/Subtype /Form /BBox [0 0 1 1] /Resources << >>", "");+ let mut ap = Dict::new();+ ap.set(b"N".to_vec(), Object::Array(vec![Object::Null; 10_001]));+ doc.update_dict(ObjRef::new(12, 0), |annotation| annotation.set(b"AP".to_vec(), Object::Dict(ap))).unwrap();+ let extracted = roundtrip(&extract_pages(&doc, &[0]).unwrap());+ assert_eq!(xobjects(&extracted, 0), xobjects(&doc, 0), "incomplete discovery cannot authorize pruning");+}++#[test]+fn extract_terminates_on_cyclic_paint_resources() {+ let mut doc = fixture("/Pattern << /Tile 10 0 R >>", "", "", "/PatternType 1 /Resources 5 0 R", "/ImB Do");+ let cycle = doc.add(Object::Null);+ let mut dict = Dict::new();+ dict.set(b"Loop".to_vec(), Object::Ref(cycle));+ dict.set(b"Shared".to_vec(), Object::Ref(ObjRef::new(6, 0)));+ doc.set(cycle, Object::Dict(dict));+ doc.update_dict(ObjRef::new(10, 0), |paint| paint.set(b"Cycle".to_vec(), Object::Ref(cycle))).unwrap();+ assert_shared_image_survives(&doc, &[0]);+}++#[test]+fn extract_still_prunes_when_appearance_resources_are_separate() {+ let mut doc = fixture("", "", "/AP << /N 10 0 R >>", "/Subtype /Form /BBox [0 0 1 1] /Resources << /XObject << /ImB 8 0 R >> >>", "/ImB Do");+ // Opaque font programs/profile streams have no Resources but are not paint programs.+ let opaque = doc.add(Object::Stream(Stream::from_raw(Dict::new(), b"opaque program bytes".to_vec())));+ doc.update_dict(ObjRef::new(10, 0), |paint| paint.set(b"FontFile".to_vec(), Object::Ref(opaque))).unwrap();+ let extracted = roundtrip(&extract_pages(&doc, &[0]).unwrap());+ assert_eq!(xobjects(&extracted, 0), [b"ImA".to_vec()]);+ let page = extracted.get(pages(&extracted).unwrap()[0].obj);+ let annotations = extracted.resolve(page.as_dict().unwrap().get(b"Annots").unwrap());+ let annotation = extracted.resolve(&annotations.as_array().unwrap()[0]);+ let ap = extracted.resolve(annotation.as_dict().unwrap().get(b"AP").unwrap());+ let normal = extracted.resolve(ap.as_dict().unwrap().get(b"N").unwrap());+ let resources = extracted.resolve(normal.as_dict().unwrap().get(b"Resources").unwrap());+ let images = extracted.resolve(resources.as_dict().unwrap().get(b"XObject").unwrap());+ assert!(images.as_dict().unwrap().contains(b"ImB"));+}++#[test]+fn split_preserves_shared_appearance_resources() {+ let doc = fixture("", "", "/AP << /N 10 0 R >>", "/Subtype /Form /BBox [0 0 1 1] /Resources 5 0 R", "/ImB Do");+ let parts = split(&doc, &SplitBy::PageCount(1)).unwrap();+ assert!(xobjects(&roundtrip(&parts[0]), 0).iter().any(|name| name == b"ImB"));+ assert_eq!(xobjects(&roundtrip(&parts[1]), 0), [b"ImA".to_vec()], "a page without the appearance still prunes");+}++#[test]+fn extract_still_prunes_when_many_pages_share_an_ordinary_graphics_state() {+ let mut doc = fixture("/ExtGState << /GS << /ca 0.5 >> >>", "/GS gs", "", "", "");+ let source_page = doc.get(ObjRef::new(3, 0));+ let mut kids = Vec::new();+ for _ in 0..4000 {+ let mut page = source_page.as_dict().unwrap().clone();+ page.remove(b"Annots");+ kids.push(Object::Ref(doc.add(page)));+ }+ doc.update_dict(ObjRef::new(2, 0), |pages| {+ pages.set(b"Kids".to_vec(), Object::Array(kids));+ pages.set(b"Count".to_vec(), 4000);+ })+ .unwrap();+ let extracted = extract_pages(&doc, &(0..4000).collect::<Vec<_>>()).unwrap();+ assert_eq!(xobjects(&extracted, 0), [b"ImA".to_vec()], "ordinary shared graphics states must not consume the paint traversal budget per page");+}++#[test]+fn extract_shared_appearance_matches_poppler_pixels() {+ let pdftoppm = pdfcraft_oracle::or_skip!(pdfcraft_oracle::pdftoppm());+ let doc = fixture("", "", "/AP << /N 10 0 R >>", "/Subtype /Form /BBox [0 0 1 1] /Resources 5 0 R", "/ImB Do");+ let dir = std::env::temp_dir().join(format!("pdfcraft-paint-resources-{}", std::process::id()));+ std::fs::create_dir_all(&dir).unwrap();+ let render = |doc: &Document, name: &str| {+ let path = dir.join(format!("{name}.pdf"));+ std::fs::write(&path, write_full(doc, &SaveOptions::default()).unwrap()).unwrap();+ let result = pdftoppm.command().args(["-f", "1", "-singlefile", "-r", "72", "-gray"]).arg(path).output().unwrap();+ assert!(result.status.success(), "{}", String::from_utf8_lossy(&result.stderr));+ assert!(result.stderr.is_empty(), "{}", String::from_utf8_lossy(&result.stderr));+ result.stdout+ };+ let before = render(&doc, "source");+ let extracted = extract_pages(&doc, &[0]).unwrap();+ let after = render(&extracted, "extracted");+ assert_eq!(after, before, "extraction must preserve the displayed stamp");+ std::fs::remove_dir_all(dir).unwrap();+}@@ -69,13 +69,20 @@ pinned in `release.yml` and builds with `CRAFT_FONTS_DIR` and `CRAFT_FONTS_REQUI embed its Japanese fonts and fail rather than ship without them (`AGENTS.md` §1.4). Bump the pin deliberately. -**OCR models:** every desktop package ships the Scan & OCR models (#103). The packaging scripts call-`stage_models` (`packaging/env.sh`; `package.ps1` on Windows), which runs `cargo xtask models` to-fetch every `kind = "model"` file in `ATTRIBUTION.toml` (verified by SHA-256) with its licence text-and an `ATTRIBUTION.txt`, straight into the package, so packaging needs the network. They go where-`pdfcraft_ocr::Models::dirs_beside_exe` looks: `models\` beside `pdfcraft.exe` (MSI, portable zip),-`PdfCraft.app/Contents/Resources/models` (macOS), and `share/pdfcraft/models` beside `bin/`-(deb, rpm, tar.gz, AppImage, Flatpak, FreeBSD). The web build doesn't include them yet.+**OCR models:** desktop packaging stages the Scan & OCR models (#103).+The packaging scripts call `stage_models` (`packaging/env.sh`; `package.ps1` on Windows), which runs `cargo xtask models` to fetch every `kind = "model"` file in `ATTRIBUTION.toml` and verify its SHA-256.+The current detection and recognition files are `text-detection.onnx` and `text-recognition.onnx`.+Each model ships with its licence text and the directory's `ATTRIBUTION.txt`, so packaging needs the network.+Upstream announced the legacy RTEN download URLs' brownout in [ocrs #282](https://github.com/robertknight/ocrs/issues/282) on 2026-10-27 and directs consumers to the ONNX models.++Runtime discovery prefers a complete ONNX detection/recognition pair within each directory, then a complete legacy `text-detection.rten` / `text-recognition.rten` pair.+An incomplete ONNX update can therefore use an intact RTEN pair in the same directory.+Discovery never combines model formats or files from different directories.+Directory precedence stays the same: `$PDFCRAFT_MODELS`, the release-package locations beside the executable, then the source tree's `assets/models/`.+The first directory with a complete supported pair wins, even if an earlier legacy pair precedes a later ONNX pair.++The release-package locations are `models\` beside `pdfcraft.exe` (MSI, portable zip), `PdfCraft.app/Contents/Resources/models` (macOS), and `share/pdfcraft/models` beside `bin/` (deb, rpm, tar.gz, AppImage, Flatpak, FreeBSD).+The web build does not include these models, and OCR remains limited to the Latin alphabet without accents. ### macOS @@ -1,5 +1,8 @@ # PdfCraft roadmap: milestones and what's next +> **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** minor (M4 #5031: resolve indirect paint-kind metadata during extraction and splitting; estimates unchanged)+> **Last reviewed:** 2026-10-27 · **Last updated:** 2026-10-27 · **Change:** minor (M4 #5011: preserve shared paint resources during Extract and Split; estimates unchanged)+> **Last reviewed:** 2026-10-27 | **Last updated:** 2026-10-27 | **Change:** minor (M10: OCR model downloads use verified ONNX pairs with legacy discovery fallback; estimates unchanged) > **Last reviewed:** 2026-10-25 | **Last updated:** 2026-10-25 | **Change:** minor (M14 localization current state: Arabic logical catalog, explicit RTL label direction and mirrored stock chrome; native/browser RTL workflows unqualified. Native release Hans face via LXGW WenKai Lite; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: image optimization accounts for page-local UserUnit; estimates unchanged)@@ -171,7 +174,7 @@ total matches the parity estimate. | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. A custom text match that ends on a reconstructed separator stops before the next glyph (#1315). Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); Lab Bradford adaptation uses LMS white-point ratios, covered by independent neutral/chromatic and direct/Indexed render oracles; inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Literal page-label prefix whitespace is preserved; nine public-inspection cases cover prefix spaces, encodings and empty-prefix controls | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). Missing: 60 fps test on a 500-page document, snapshot tests of every panel<br>Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). |-| M4 | Engine, history, save, organize | 100–180 | 95% | 8–15 | Done:<br>- command registry (menus, shortcuts and palette all use it);<br>- undo/redo; incremental, atomic and encrypted saves;<br>- autosave and crash recovery;<br>- organize, combine, extract, split and insert-from-file, with identical fonts and images stored once;<br>- CLI `edit/combine/extract/split`.<br>Done since: Organize opens directly into the page overview, with checkbox selection, an editable page-range selection (#1092), and per-page rotate/delete controls; Combine files in its own tab (sortable, resizable columns; folders; early warnings; unlocking protected files), bookmark editing, page labels (Number pages), CLI `run`, Set Page Boxes, Crop tool, Duplicate pages, Ctrl+A / Cmd+A to select all pages in Organize, Replace pages, bookmarks from tagged headings. Extract and Split preserve shared Type 3 glyph resource graphs, retaining conservative pruning limits. Undo back to the opened or last saved file is clean; Revert opens that saved file after an undo; a later save appends to it so the saved revision stays in the file (#1310, #1311, #1312). Ctrl-drag (Option on macOS) on a grid thumbnail copies the pages into the drop gap, with a "+" badge and the copies selected (#2997). Initial-view window and interface options resolve indirect boolean preferences. Done since: #3309: Stack-safe named-destination pruning visits shared/cyclic indirect nodes once, without a name-count limit. Synthetic deep-tree and 60,000-name tests verify deletion, surviving targets, undo/redo, both save modes and reopening. Organize drags cancel after document edits and Undo/Redo (#3916 partial). Missing: page transitions, recovery on the web |+| M4 | Engine, history, save, organize | 100–180 | 95% | 8–15 | Done:<br>- command registry (menus, shortcuts and palette all use it);<br>- undo/redo; incremental, atomic and encrypted saves;<br>- autosave and crash recovery;<br>- organize, combine, extract, split and insert-from-file, with identical fonts and images stored once;<br>- CLI `edit/combine/extract/split`.<br>Done since: Organize opens directly into the page overview, with checkbox selection, an editable page-range selection (#1092), and per-page rotate/delete controls; Combine files in its own tab (sortable, resizable columns; folders; early warnings; unlocking protected files), bookmark editing, page labels (Number pages), CLI `run`, Set Page Boxes, Crop tool, Duplicate pages, Ctrl+A / Cmd+A to select all pages in Organize, Replace pages, bookmarks from tagged headings. Extract and Split preserve shared Type 3 glyph resource graphs, retaining conservative pruning limits. Undo back to the opened or last saved file is clean; Revert opens that saved file after an undo; a later save appends to it so the saved revision stays in the file (#1310, #1311, #1312). Ctrl-drag (Option on macOS) on a grid thumbnail copies the pages into the drop gap, with a "+" badge and the copies selected (#2997). Initial-view window and interface options resolve indirect boolean preferences. Done since: #3309: Stack-safe named-destination pruning visits shared/cyclic indirect nodes once, without a name-count limit. Synthetic deep-tree and 60,000-name tests verify deletion, surviving targets, undo/redo, both save modes and reopening. Organize drags cancel after document edits and Undo/Redo (#3916 partial). Extract and Split preserve shared appearance, pattern, soft-mask and Type 3 resource owners with bounded caller-aware discovery (#5011). Extract and Split resolve indirect Type3/Form kind metadata before pruning image resources (#5031), with bounded mask/pattern controls. Missing: page transitions, recovery on the web | | M4 | Engine, history, save, organize | 100–180 | 95% | 8–15 | Done:<br>- command registry (menus, shortcuts and palette all use it);<br>- undo/redo; incremental, atomic and encrypted saves;<br>- autosave and crash recovery;<br>- organize, combine, extract, split and insert-from-file, with identical fonts and images stored once;<br>- CLI `edit/combine/extract/split`.<br>Done since: Organize opens directly into the page overview, with checkbox selection, an editable page-range selection (#1092), and per-page rotate/delete controls; Combine files in its own tab (sortable, resizable columns; folders; early warnings; unlocking protected files), bookmark editing, page labels (Number pages), CLI `run`, Set Page Boxes, Crop tool, Duplicate pages, Ctrl+A / Cmd+A to select all pages in Organize, Replace pages, bookmarks from tagged headings. Extract and Split preserve shared Type 3 glyph resource graphs, retaining conservative pruning limits. Undo back to the opened or last saved file is clean; Revert opens that saved file after an undo; a later save appends to it so the saved revision stays in the file (#1310, #1311, #1312). Rotation queries and edits resolve indirect values on pages and ancestors. Refreshed against upstream `99941daa` (2026-10-15). Repeated page indices accumulate rotations from the latest local value (#1919); saved/reopened regressions retain inherited angles and siblings. Missing: page transitions, recovery on the web | | M4 | Engine, history, save, organize | 100–180 | 95% | 8–15 | Done:<br>- command registry (menus, shortcuts and palette all use it);<br>- undo/redo; incremental, atomic and encrypted saves;<br>- autosave and crash recovery;<br>- organize, combine, extract, split and insert-from-file, with identical fonts and images stored once;<br>- CLI `edit/combine/extract/split`.<br>Done since: Organize opens directly into the page overview, with checkbox selection, an editable page-range selection (#1092), and per-page rotate/delete controls; Combine files in its own tab (sortable, resizable columns; folders; early warnings; unlocking protected files), bookmark editing, page labels (Number pages), CLI `run`, Set Page Boxes, Crop tool, Duplicate pages, Ctrl+A / Cmd+A to select all pages in Organize, Replace pages, bookmarks from tagged headings. Extract and Split preserve shared Type 3 glyph resource graphs, retaining conservative pruning limits. Undo back to the opened or last saved file is clean; Revert opens that saved file after an undo; a later save appends to it so the saved revision stays in the file (#1310, #1311, #1312). Rotation queries and edits resolve indirect values on pages and ancestors. Missing: page transitions, recovery on the web | | M4 | Engine, history, save, organize | 100–180 | 95% | 8–15 | Done:<br>- command registry (menus, shortcuts and palette all use it);<br>- undo/redo; incremental, atomic and encrypted saves;<br>- autosave and crash recovery;<br>- organize, combine, extract, split and insert-from-file, with identical fonts and images stored once;<br>- CLI `edit/combine/extract/split`.<br>Done since: Organize opens directly into the page overview, with checkbox selection, an editable page-range selection (#1092), and per-page rotate/delete controls; Combine files in its own tab (sortable, resizable columns; folders; early warnings; unlocking protected files), bookmark editing, page labels (Number pages), CLI `run`, Set Page Boxes, Crop tool, Duplicate pages, Ctrl+A / Cmd+A to select all pages in Organize, Replace pages, bookmarks from tagged headings. Extract and Split preserve shared Type 3 glyph resource graphs, retaining conservative pruning limits. Missing: page transitions, recovery on the web M4: Insert-from-file preserves source bookmarks; integration tests retained with current main. |@@ -193,7 +196,7 @@ total matches the parity estimate. | M7 | Content editing (text, images, header/footer, watermark) | 250–500 | 20% | 100–180 | Done: header & footer, watermarks, backgrounds, Bates; added text and images that stay editable (move, resize, format, rotate, flip, crop, replace); links (Link tool, Link Properties, create from URLs, remove all). Missing: editing existing text and images in place (the longest pole), image/PDF watermarks | | M8 | Security + redaction | 100–180 | 68% | 15–30 | Done: opening protected documents, permissions, Protect Using Password, Remove security; redaction (mark text/areas/pages, Search & Redact with patterns, apply removing glyphs, image pixels, vectors, XObject content, annotations and fields, verification, full rewrite on save); Remove hidden information and Sanitize.<br>Done since: redaction codes (U.S. FOIA and Privacy Act sets as overlay text, in Redaction properties and `redact_mark`). Redaction cleans indirect and form structure references, preserving surviving form tags. Redaction retires inherited resource dictionaries, so a parent `/Pages` can't keep a removed XObject reachable; unredacted pages are left untouched. Image redaction removes alternate images and unsupported masks only on affected images. Changed marked-content spans lose replacement descriptions; untouched spans keep theirs. Hidden-text cleanup retires unused named descriptions across inherited/shared page and form resources (64 focused tests and scoped Rust 1.96 Clippy/wasm checks pass; publication CI pending). The redaction audit counts a filled cover that hides half of any one word of a long line (#2662), applied or not. Phone-pattern redaction also matches labelled Italian geographic numbers with `+39` and a slash after the area code (#3205 follow-up); unlabelled international slash numbers remain unsupported. Missing: certificate security, custom redaction code sets and pattern locales, DCT re-encoding | | M9 | Signatures (PAdES, validation) | 160–280 | 60% | 40–70 | Done: new `sign` crate (DER, X.509, CMS, PKCS #12 on RustCrypto; aws-lc-rs for RSA private keys); PAdES B-B signing (visible, invisible, existing fields, certification with DocMDP); validation with trust store and changes-after-signing classification; self-signed digital IDs; Signatures panel, message bar, sign dialogs; agent tools. Checked with pdfsig and OpenSSL.<br>Done since: macOS Keychain and Windows Current User Personal store signing (software-backed CNG keys), certificate viewer, signed documents protected from rewrites, page validation overlays separate from original signature artwork (baked Not Verified captions remain; #79 open), snapshot tabs that reuse instead of multiplying. Missing: timestamps (B-T), LTV (DSS, OCSP, CRL), FieldMDP, smart cards, PKCS #11 |-| M10 | OCR, create, export, print | 200–350 | 33% | 120–215 | Done: create from blank/text/PNG/JPEG/TIFF (multi-page)/GIF/BMP; export PNG/JPEG/TIFF and text; Print (Acrobat's sizing, n-up, booklet, poster, comments & forms, preview, the CUPS and Windows spoolers, print-ready PDF).<br>Done since: embedded/72/custom DPI choices for image imports; export all images; OCR (searchable image for pages, ranges and multiple files); single-sided cut-and-stack imposition with cut marks, through Print and doc_print; architectural/ANSI/large ISO sheet presets, Match page at 100% and PDF-scale preview (#599 follow-up); Create PDF from multiple files (PDFs, images and text, edited as pages in the grid); OCR can also read pictures on pages that have text, without adding that text twice (#1028). Done since: Word invoice export uses borderless inferred tables, floating positioned text/artwork only where text overlaps an image or is rotated, and per-page sizes, covered by synthetic invoice automation regressions (#773); printing on the web through the browser's print dialog, sheets drawn by PdfCraft (gap 4). Lossless image export applies per-component Decode, including Indexed mapping (#1201 partial). Existing 16-bit precision and color-management limits remain. #3532 partial: vector printing retains existing appearances with absent Subtype through a print-owned Form copy; source artwork stays unchanged. #3120: Office export reads form XObjects 50 levels deep, as the renderer does, and up to 50 000 a page; pages past that are listed in `truncated_pages`. Word table cells use the grid columns they span, including cells after a merged cell. Done since: text in a table slot covered by another cell's span remains as paragraph text in Word, HTML, RTF and PowerPoint. Missing: OCR languages beyond Latin, editable-text OCR output, full Office layout fidelity/import. Done since: dotted-leader contents entries stay in separate complete paragraphs (#1757) #2156 candidate: last selected print paper persists across sessions; other print preferences unchanged. Image export applies soft-mask Decode opacity intervals before resizing, including indirect entries and inverted ranges. Vector printing gives annotation appearances unoccupied XObject names so existing page artwork survives. |+| M10 | OCR, create, export, print | 200–350 | 33% | 120–215 | Done: create from blank/text/PNG/JPEG/TIFF (multi-page)/GIF/BMP; export PNG/JPEG/TIFF and text; Print (Acrobat's sizing, n-up, booklet, poster, comments & forms, preview, the CUPS and Windows spoolers, print-ready PDF).<br>Done since: embedded/72/custom DPI choices for image imports; export all images; OCR (searchable image for pages, ranges and multiple files); single-sided cut-and-stack imposition with cut marks, through Print and doc_print; architectural/ANSI/large ISO sheet presets, Match page at 100% and PDF-scale preview (#599 follow-up); Create PDF from multiple files (PDFs, images and text, edited as pages in the grid); OCR can also read pictures on pages that have text, without adding that text twice (#1028). Model staging fetches SHA-256-verified ONNX detection/recognition files after the upstream RTEN URL brownout; runtime discovery prefers complete ONNX pairs per directory, then complete legacy RTEN pairs, without mixing formats or changing directory precedence. Done since: Word invoice export uses borderless inferred tables, floating positioned text/artwork only where text overlaps an image or is rotated, and per-page sizes, covered by synthetic invoice automation regressions (#773); printing on the web through the browser's print dialog, sheets drawn by PdfCraft (gap 4). Lossless image export applies per-component Decode, including Indexed mapping (#1201 partial). Existing 16-bit precision and color-management limits remain. #3532 partial: vector printing retains existing appearances with absent Subtype through a print-owned Form copy; source artwork stays unchanged. #3120: Office export reads form XObjects 50 levels deep, as the renderer does, and up to 50 000 a page; pages past that are listed in `truncated_pages`. Word table cells use the grid columns they span, including cells after a merged cell. Done since: text in a table slot covered by another cell's span remains as paragraph text in Word, HTML, RTF and PowerPoint. Missing: OCR languages beyond Latin, editable-text OCR output, full Office layout fidelity/import. Done since: dotted-leader contents entries stay in separate complete paragraphs (#1757) #2156 candidate: last selected print paper persists across sessions; other print preferences unchanged. Image export applies soft-mask Decode opacity intervals before resizing, including indirect entries and inverted ranges. Vector printing gives annotation appearances unoccupied XObject names so existing page artwork survives. | | M11 | Optimize, preflight, PDF/A/X/UA, print production | 200–350 | 18% | 80–140 | Done: new `optimize` crate: Reduce File Size and the PDF Optimizer (images measured where drawn, bicubic downsampling, JPEG/ZIP recompression only when smaller, discard objects and user data, Flate clean-up, resource merging, object streams). Target-version compatibility is partial: newer content is not detected or converted. Content stream boundaries preserve comments, split operands and graphics state when image fragments merge. Image downsampling includes page-local `/UserUnit` scaling when measuring physical resolution. Done since: reject invalid image and soft-mask dimensions with checked sizing (#2178). Missing: fonts and transparency panels, space audit, preflight, PDF/A/X/UA | | M12 | Accessibility, compare, measure, search, XFA | 200–380 | 30% | 90–160 | Done: new `a11y` crate with the Accessibility Checker (all 32 rules, report, Fix/Skip/Explain, options dialog and results panel, agent tools). Locatable figure, annotation and form findings highlight their page bounds. Character encoding accepts Identity-H/V with Adobe CJK collections and predefined CJK CMaps independently (#3673); per-glyph validation and embedded-font extraction remain separate.<br>Done since: 2D distance, perimeter and area measurements with persistent viewport calibration, snapping, live information and CSV export; a bounded, text-only auto-tag proposal tool for one page. Marking a figure decorative rewrites only that stream (#1324). An XFA radio item whose value contains a space is written back unchanged (#1319). Done since: #2569: Growable multiline XFA text reflows after edits, with undo/redo, save/reopen and print coverage; oversized fields still become read-only page text. Structured-data synchronization warnings retain accepted whitespace without relayout from stale data. Done since: #2572: Every repeated page-master button resolves its own field to the shared XFA click script, including print buttons. Done since: #2570: Dynamic XFA rich-text fields carry exData/maxLength into AcroForm MaxLen for reported and enforced Unicode character limits, retained after save/reopen. Done since: #2589: Active XFA skips PDF-container fallback scripts while retaining requested messages and saved scripts; ordinary/static/unlaid-out documents and explicit console/field calls keep their existing behavior. Missing: writing proposed tags and parent tree, field autotag, visual Tags/Order/Content panels, Reading Order tool, compare, geospatial/3D measurement, search index, XFA<br>Reading colours: pending page backgrounds follow the configured colour (#2059). | | M13 | Automation (MCP, Action Wizard, CLI) + AI providers | 60–120 | 45% | 25–45 | Done: MCP resources (document info, text, page images); headless tool table (123 tools incl. signing, optimizing, initial view, links, stamps, data exchange, comment review, forms authoring and scripts, redaction, sanitize, print, add content), opt-in MCP server over stdio, CLI `run`/`tools` (closed stdout pipes exit cleanly), UI control channel with drag. Missing: Action Wizard, AI providers<br>Done since: CLI run and MCP reject missing, empty, repeated and option-valued --root arguments before any tool/script execution or MCP initialization. A single-hyphen directory name remains valid; synthetic CLI regressions preserve a would-be overwritten file. `text_find` also caps the per-page match ranges it materializes at the remaining result limit plus one continuation match. CLI robustness scans visit each directory once through overlapping roots and symlink aliases. |@@ -254,6 +257,9 @@ Estimates unchanged. | Date | Change | Summary | |---|---|---|+| 2026-10-27 | minor | M4 #5031: resolve indirect paint-kind metadata before pruning shared image resources; Type3/Form rendering, inherited mask/pattern guards, malformed refs and tool save/reopen regressions. Estimates unchanged. |+| 2026-10-27 | minor | M4 #5011: preserve shared annotation appearance, pattern, soft-mask and Type 3 resources during extraction and splitting; conservative bounded traversal and rendered save/reopen regression coverage. Estimates unchanged. |+| 2026-10-27 | minor | M10: OCR stages SHA-256-verified ONNX models after the upstream RTEN URL brownout; discovery prefers a complete ONNX pair per directory, falls back to a complete RTEN pair, and never mixes formats or directories; language/web limits and estimates unchanged. | | 2026-10-24 | minor | M3, #3970 follow-up: the button release given back after a window move or resize goes before the pointer leaving, so hover doesn't stick. Estimates unchanged. | | 2026-10-23 | minor | M8, #3205 follow-up: phone-pattern redaction includes the +39 country code on labelled Italian geographic numbers with a slash; dates, protocol identifiers and whole-run guards remain. Estimates unchanged. | | 2026-10-23 | minor | M10: Word table cells use the grid columns they span, including cells after a merged cell; synthetic export regression and CLI reproduction. Estimates unchanged. |@@ -288,13 +288,16 @@ $modelsDir = Read-Row 'SELECT `Directory_Parent`, `DefaultDir` FROM `Directory` Assert-Equal $modelsDir[0] 'INSTALLFOLDER' 'OCR models folder parent' # DefaultDir is `models`, or `SHORT|models` with a generated 8.3 name. if ($modelsDir[1] -notmatch '(^|\|)models$') { throw "OCR models folder name: '$($modelsDir[1])'" }-$modelFiles = 0+$modelFiles = @() $view = $Database.OpenView('SELECT `File`.`FileName` FROM `File`, `Component` WHERE `File`.`Component_` = `Component`.`Component` AND `Component`.`Directory_` = ''ModelsFolder''') try { [void] $view.Execute()- while ($record = $view.Fetch()) { if ($record.StringData(1) -match '\.rten$') { $modelFiles++ } }+ # FileName can contain a generated 8.3 name before the long name: SHORT|long.+ while ($record = $view.Fetch()) { $modelFiles += ($record.StringData(1) -split '\|')[-1] } } finally { [void] $view.Close() }-if ($modelFiles -lt 2) { throw "expected the OCR models (*.rten) in models\, found $modelFiles" }+foreach ($model in @('text-detection.onnx', 'text-recognition.onnx')) {+ Assert-Equal (@($modelFiles | Where-Object { $_ -ceq $model }).Count) 1 "OCR model $model in models\"+} $rm = Read-Row 'SELECT `Dialog` FROM `Dialog` WHERE `Dialog` = ''MsiRMFilesInUse''' 1 Assert-Equal $rm[0] 'MsiRMFilesInUse' 'Files-in-use dialog' [void] [Runtime.InteropServices.Marshal]::FinalReleaseComObject($Database)@@ -2811,8 +2811,8 @@ milestone = "M4" status = "shipped" commands = ["page.extract"] tools = ["page_extract"]-evidence = ["crates/ui-egui/tests/it/editing.rs::extract_button_copies_selected_pages_to_a_new_tab", "crates/automation/tests/it/automation.rs::combine_extract_and_split", "crates/organize/src/tests.rs::extract_and_split_keep_the_pdfx_output_intent_and_identification", "crates/ui-egui/tests/it/pickers.rs::extract_pages_takes_the_page_ranges_typed_in_the_dialog"]-notes = "Carries the PDF/X output intent and identification (#263) in a new XMP packet; the source XMP (dates, other schemas) is not copied. The dialog's Pages field takes page ranges (\"1-5, 8, 11-13\", in the order typed, each page once) instead of the selection (#3593); page labels are not accepted there yet."+evidence = ["crates/ui-egui/tests/it/editing.rs::extract_button_copies_selected_pages_to_a_new_tab", "crates/automation/tests/it/automation.rs::combine_extract_and_split", "crates/organize/src/tests.rs::extract_and_split_keep_the_pdfx_output_intent_and_identification", "crates/ui-egui/tests/it/pickers.rs::extract_pages_takes_the_page_ranges_typed_in_the_dialog", "crates/organize/tests/paint_resources.rs::extract_preserves_shared_annotation_appearance_resources", "crates/organize/tests/paint_resources.rs::extract_preserves_shared_tiling_pattern_and_soft_mask_resources", "crates/organize/tests/paint_resources.rs::extract_preserves_resources_of_a_type3_font_selected_by_extgstate", "crates/organize/tests/paint_resources.rs::extract_still_prunes_when_many_pages_share_an_ordinary_graphics_state", "crates/automation/tests/it/extract_paint_resources.rs::extract_and_split_preserve_shared_stamp_pixels_after_save_and_reopen", "crates/organize/tests/indirect_paint_kinds.rs::extract_keeps_images_used_by_indirect_type3_subtypes", "crates/organize/tests/indirect_paint_kinds.rs::extract_keeps_images_used_by_indirect_form_subtypes", "crates/organize/tests/indirect_paint_kinds.rs::extract_protects_inherited_paint_with_indirect_kind_metadata", "crates/organize/tests/indirect_paint_kinds.rs::indirect_kind_extraction_matches_poppler_pixels", "crates/automation/tests/it/extract_indirect_kinds.rs::extract_and_split_keep_indirect_type3_artwork_after_save_and_reopen"]+notes = "Carries the PDF/X output intent and identification (#263) in a new XMP packet; the source XMP (dates, other schemas) is not copied. The dialog's Pages field takes page ranges (\"1-5, 8, 11-13\", in the order typed, each page once) instead of the selection (#3593); page labels are not accepted there yet. Shared resource owners used by annotation N/R/D states, patterns, soft-mask groups and Type 3 fonts survive pruning (#5011). The bounded dependency pass can retain extra unused images; it does not interpret these paint programs. Paint-kind primitives given by reference are resolved before resource discovery (#5031), preserving Type3/Form image uses and conservative inherited mask/pattern protection; malformed metadata and traversal limits remain bounded." [[feature]] id = "organize.extract-separate-files"@@ -2931,8 +2931,8 @@ milestone = "M4" status = "shipped" commands = ["page.split"] tools = ["doc_split"]-evidence = ["crates/ui-egui/tests/it/editing.rs::split_dialog_writes_one_file_per_part", "crates/organize/src/tests.rs::split_ranges_cover_every_page_once", "crates/organize/src/tests.rs::split_produces_standalone_documents_with_metadata", "crates/organize/src/tests.rs::extract_and_split_keep_the_pdfx_output_intent_and_identification"]-notes = "Each part carries the PDF/X output intent and identification (#263), as Extract does."+evidence = ["crates/ui-egui/tests/it/editing.rs::split_dialog_writes_one_file_per_part", "crates/organize/src/tests.rs::split_ranges_cover_every_page_once", "crates/organize/src/tests.rs::split_produces_standalone_documents_with_metadata", "crates/organize/src/tests.rs::extract_and_split_keep_the_pdfx_output_intent_and_identification", "crates/organize/tests/paint_resources.rs::extract_preserves_shared_annotation_appearance_resources", "crates/organize/tests/paint_resources.rs::extract_preserves_shared_tiling_pattern_and_soft_mask_resources", "crates/organize/tests/paint_resources.rs::extract_preserves_resources_of_a_type3_font_selected_by_extgstate", "crates/organize/tests/paint_resources.rs::extract_still_prunes_when_many_pages_share_an_ordinary_graphics_state", "crates/automation/tests/it/extract_paint_resources.rs::extract_and_split_preserve_shared_stamp_pixels_after_save_and_reopen", "crates/organize/tests/paint_resources.rs::split_preserves_shared_appearance_resources", "crates/organize/tests/indirect_paint_kinds.rs::extract_keeps_images_used_by_indirect_type3_subtypes", "crates/organize/tests/indirect_paint_kinds.rs::extract_keeps_images_used_by_indirect_form_subtypes", "crates/organize/tests/indirect_paint_kinds.rs::extract_protects_inherited_paint_with_indirect_kind_metadata", "crates/organize/tests/indirect_paint_kinds.rs::indirect_kind_extraction_matches_poppler_pixels", "crates/automation/tests/it/extract_indirect_kinds.rs::extract_and_split_keep_indirect_type3_artwork_after_save_and_reopen"]+notes = "Each part carries the PDF/X output intent and identification (#263), as Extract does. Shared resource owners used by annotation N/R/D states, patterns, soft-mask groups and Type 3 fonts survive pruning (#5011). The bounded dependency pass can retain extra unused images; it does not interpret these paint programs. Paint-kind primitives given by reference are resolved before resource discovery (#5031), preserving Type3/Form image uses and conservative inherited mask/pattern protection; malformed metadata and traversal limits remain bounded." [[feature]] id = "organize.split-before-pages"@@ -6544,8 +6544,8 @@ milestone = "M10" status = "shipped" commands = ["ocr.recognize"] tools = ["ocr_recognize", "ocr_status"]-evidence = ["crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/automation/tests/it/automation.rs::ocr_tools_make_a_scan_searchable", "crates/ui-egui/tests/it/ocr.rs::recognize_text_dialog_adds_searchable_text"]-notes = "ocrs engine (MIT/Apache) with CC-BY-SA models fetched by `cargo xtask models`; the desktop release packages ship them (#103; packaging/env.sh stage_models), the web build does not yet; runs on a worker thread with progress; one undoable step; pages with text are skipped unless asked (ocr.skip-renderable-text)."+evidence = ["crates/ocr/src/lib.rs::model_discovery_prefers_the_complete_onnx_pair", "crates/ocr/src/lib.rs::model_discovery_falls_back_to_legacy_when_the_onnx_pair_is_incomplete", "crates/ocr/src/lib.rs::model_discovery_refuses_mixed_format_pairs", "crates/ocr/src/lib.rs::model_discovery_keeps_legacy_only_installations", "crates/engine/src/tests.rs::recognize_text_makes_a_scanned_page_searchable", "crates/automation/tests/it/automation.rs::ocr_tools_make_a_scan_searchable", "crates/ui-egui/tests/it/ocr.rs::recognize_text_dialog_adds_searchable_text"]+notes = "ocrs engine (MIT/Apache) with CC-BY-SA ONNX models fetched and SHA-256 checked by `cargo xtask models`; within each search directory, a complete ONNX pair takes precedence over a complete legacy RTEN pair, incomplete and mixed-format pairs are not combined, and directory precedence is unchanged; the desktop release packages ship them (#103; packaging/env.sh stage_models), the web build does not yet; runs on a worker thread with progress; one undoable step; pages with text are skipped unless asked (ocr.skip-renderable-text)." [[feature]] id = "ocr.recognize-range"