collective
Browse / github.com/storytold/designcraft / patch
M6: preserve PDF/EPS preview types on IDML reopen
M6: preserve PDF/EPS preview types on IDML reopen Opening an INDD with unavailable PDF/EPS links can supply raster previews while retaining the original graphic type. The IDML exporter embeds those preview bytes; reopening previously sniffed the raster format and changed the placed PDF/EPS into an Image, losing its original type and subsequent PDF placement metadata. Treat embedded raster data on a PDF/EPS graphic as a stand-in: retain the declared type, page and crop, and emit an explicit warning. Actual bytes loaded from a linked replacement still determine its format. Add synthetic regressions for preview round trips and linked raster rep
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- andeecollard (imported from pull request #1626, open)
- Base
5e8ef678321cfdd2d18927e2af94d58e5f9d8836- Size
- +156 −3 in 6 files
- Review
- pending
- Shared
- 2026-10-28
- Id
3bfba3340adce3bbb77766a7
How it went
No other operator has reported using it yet.
Diff
raw · ROADMAP.md crates/idml/src/import.rs crates/idml/src/tests/assets.rs docs/gaps.md docs/private-corpus-testing.md docs/roadmap.md
@@ -1,9 +1,10 @@ # DesignCraft roadmap **Stage: pre-alpha** · next: alpha, ~5–7 h away; beta ~28 points and ~420–750 h beyond that (the core-workflow gate, not the percentage: ready for real work is already ~47%)+- 2026-10-22: local private-corpus checks cover 205 INDD direct opens, PDF/IDML exports and reopens. Embedded PDF/EPS previews retain graphic type/page/crop on IDML reopen; synthetic regression coverage and 205-file verification reduce affected documents from 36 to zero. No reference-fidelity claim or milestone change; see [evidence and limits](docs/private-corpus-testing.md). - FID1: PDF export normalizes mixed emitted gradient color spaces to RGB with an explicit warning; homogeneous device-space gradients retain their components. Linear/radial, midpoint, archival and print policies have synthetic regression coverage. -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-13 · **Change:** minor (File ▸ Open Recent; .indd and .indt open through indd-utils; text threads by hand; the alpha gate waits on Primary Text Frame page fixes; Japanese print, import and table controls localized; Live corners (#186); Optical kerning from outlines (#181); window and menu quit confirmation; failed writes preserve live save and collection state; spring-loaded Zoom navigation; orderly native exit recovery; Pages panel parents; cancelable macOS Dock Quit; text stroke behind fill; Based on Parent; object styles carry effects; table anchors keep their own paragraphs; grouped HTML/EPUB reading order; Links source paths; XLSX cached cell values retain numeric precision and text) · **Target:** Adobe InDesign 2026+> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-22 · **Change:** minor (private corpus evidence and PDF/EPS preview reopen fix; File ▸ Open Recent; .indd and .indt open through indd-utils; text threads by hand; the alpha gate waits on Primary Text Frame page fixes; Japanese print, import and table controls localized; Live corners (#186); Optical kerning from outlines (#181); window and menu quit confirmation; failed writes preserve live save and collection state; spring-loaded Zoom navigation; orderly native exit recovery; Pages panel parents; cancelable macOS Dock Quit; text stroke behind fill; Based on Parent; object styles carry effects; table anchors keep their own paragraphs; grouped HTML/EPUB reading order; Links source paths; XLSX cached cell values retain numeric precision and text) · **Target:** Adobe InDesign 2026 - AR1.1: Kashida justification tests verify space fallback without Tatweel fonts using an isolated font database, while retaining elongation assertions when fonts support it. - AR1: Arabic IDML controls now retain character direction, kashida switches, diacritic offsets/forms, paragraph policy values and independent story/table directions. Shaping preserves joining context and script/language selection; paragraph-level bidi, contextual digit conversion, mark-safe elongation, RTL columns/tables and physical column insertion have regression coverage. Vendor-specific justification and diacritic presets remain explicit preflight warnings; see docs/arabic-typography.md.@@ -3066,6 +3066,7 @@ impl<'r> Importer<'r> { }) }; let mut data = g.prop_el("Contents").map(|c| base64_decode(&c.text_content())).unwrap_or_default();+ let embedded = !data.is_empty(); let page = pdf_page_in(g); // Where the box sits on the page needs the PDF parsed: the engine finds it after import. let pdf_crop = pdf_attribute(g).and_then(|a| a.get("PDFCrop")).map_or(designcraft_doc::PdfCrop::Crop, names::pdf_crop_in);@@ -3092,6 +3093,21 @@ impl<'r> Importer<'r> { // art's bounds, which can overhang the viewBox, so they must not size the viewBox. let size = svg_viewbox_size(&data).unwrap_or(size); let (mime, px) = if preview.is_some() { (None, None) } else { sniff_image(&data) };+ // INDD previews can be embedded on IDML export while the graphic keeps+ // its original PDF/EPS link. Reopening must retain that type, page and+ // crop rather than silently converting the placement into an Image.+ // Bytes read from a relinked file still determine its actual format.+ let vector_preview = embedded && matches!(g.local(), "PDF" | "EPS") && mime.is_some_and(|m| m.starts_with("image/") && m != "image/svg+xml");+ let (mime, px) = if vector_preview {+ self.warnings.push(format!(+ "graphic {}: embedded raster stand-in for {}; original graphic type and placement retained",+ g.get("Self").unwrap_or("(unnamed)"),+ g.local()+ ));+ (None, None)+ } else {+ (mime, px)+ }; let mime = mime.map(str::to_string).unwrap_or_else(|| match link.and_then(|k| k.get("LinkResourceFormat")).unwrap_or("") { f if f.contains("EPS") || g.local() == "EPS" => "application/postscript".into(), f if f.contains("JPEG") => "image/jpeg".into(),@@ -70,6 +70,44 @@ fn package(frames: &[String]) -> Vec<u8> { ]) } +#[test]+fn embedded_vector_previews_keep_format_page_and_crop_on_reopen() {+ // Synthetic raster preview; no PDF/EPS source file is available. INDD import+ // supplies such previews while retaining the original link's format.+ let png = std::sync::Arc::new([b"\x89PNG\r\n\x1a\n".as_slice(), &[0, 0, 0, 13], b"IHDR", &4u32.to_be_bytes(), &3u32.to_be_bytes()].concat());+ let pdf = pdf_frame(1, 0.0, 3, "CropTrim");+ let eps = pdf_frame(2, 100.0, 1, "CropPDF")+ .replace("<PDF ", "<EPS ")+ .replace("</PDF>", "</EPS>")+ .replace("insert.pdf", "insert.eps")+ .replace("$ID/PDF", "$ID/EPS");+ let previews = std::collections::HashMap::from([("im1".into(), png.clone()), ("im2".into(), png.clone())]);+ let d = crate::import_idml_report_with_previews(&package(&[pdf, eps]), &|_| None, &previews).unwrap().document;+ let out = export_idml(&d);+ let imported = crate::import_idml_report_with_previews(&out, &|_| None, &Default::default()).unwrap();+ assert_eq!(imported.warnings.iter().filter(|w| w.contains("embedded raster stand-in")).count(), 2);+ let reopened = imported.document;+ let pdf = reopened.assets.values().find(|a| a.name == "insert.pdf").unwrap();+ let eps = reopened.assets.values().find(|a| a.name == "insert.eps").unwrap();+ assert_eq!(pdf.mime, "application/pdf", "a raster stand-in must not turn a placed PDF into an Image");+ assert_eq!(eps.mime, "application/postscript");+ assert_eq!(pdf.page, 2);+ assert_eq!(pdf.pdf_crop, PdfCrop::Trim);+ assert_eq!(pdf.data.as_slice(), png.as_slice());+ assert_eq!(eps.data.as_slice(), png.as_slice());+ let second = export_idml(&reopened);+ let spreads = zip_text(&second, "Spreads/");+ assert!(spreads.contains("<PDF ") && spreads.contains("<EPS "));+ assert!(spreads.contains("PageNumber=\"3\"") && spreads.contains("PDFCrop=\"CropTrim\""));+}++#[test]+fn linked_raster_replacement_keeps_its_actual_format() {+ let png = [b"\x89PNG\r\n\x1a\n".as_slice(), &[0, 0, 0, 13], b"IHDR", &4u32.to_be_bytes(), &3u32.to_be_bytes()].concat();+ let d = import_idml_with(&package(&[pdf_frame(1, 0.0, 3, "CropTrim")]), &|_| Some(png.clone())).unwrap();+ assert_eq!(d.assets.values().next().unwrap().mime, "image/png", "a linked replacement is actual artwork, not an embedded stand-in");+}+ /// (asset, crop, transform) of each graphic, left to right. fn placements(d: &Document) -> Vec<(AssetId, [f64; 4], designcraft_geom::Affine)> { let mut out: Vec<(f64, AssetId, [f64; 4], designcraft_geom::Affine)> = d.spreads[0]@@ -1,6 +1,6 @@ # Where DesignCraft falls short of InDesign -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-17 · **Change:** minor (gap 6 shrunk: Type › Reload Fonts reads fonts installed, edited or removed while running (#328); gap 3: Package links are relative, #340; gap 27 closed: the loaded text cursor threads into a frame inside a group (#1075); .indd and .indt open through indd-utils; gap 0 shrunk: text threads by hand; threading gaps added; Optical kerning from outlines (#181); gap 6: macOS lists fonts that font services and managers activate; shared print spooler; image clipping paths from IDML; Tabs panel listed as present (#343); book chapter update safety; gap 16: Based on Parent leftovers; bounded control click/drag input; swatches on placed greyscale and bitmap images) · **Target:** Adobe InDesign 2026 (21.6)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (private corpus evidence and PDF/EPS preview reopen fix; gap 6 shrunk: Type › Reload Fonts reads fonts installed, edited or removed while running (#328); gap 3: Package links are relative, #340; gap 27 closed: the loaded text cursor threads into a frame inside a group (#1075); .indd and .indt open through indd-utils; gap 0 shrunk: text threads by hand; threading gaps added; Optical kerning from outlines (#181); gap 6: macOS lists fonts that font services and managers activate; shared print spooler; image clipping paths from IDML; Tabs panel listed as present (#343); book chapter update safety; gap 16: Based on Parent leftovers; bounded control click/drag input; swatches on placed greyscale and bitmap images) · **Target:** Adobe InDesign 2026 (21.6) Every known shortfall against InDesign 2026, one entry each, ranked by how much it stops an InDesign professional from doing real work. This is the work list: agents pick from the top.@@ -49,6 +49,7 @@ Kind: **F** feature · **U** UI/UX · **T** typography · **FF** file format · DesignCraft draws from those files is unmeasured, and it inherits the IDML importer's open bugs (gap 3). - **Impact:** a client's `.indd` opens, but nobody knows how close it comes to the original.+- **Local evidence (2026-10-22):** 205 private INDDs pass direct open, PDF/IDML export and reopen; 90 further inputs are cloud-only. This is pipeline evidence, not reference fidelity or a redistributable corpus. A PDF/EPS preview type-loss regression found in 36 documents is fixed with synthetic coverage; the patched reopen check has zero such changes. See [private-corpus-testing.md](private-corpus-testing.md). The committed corpus/reference requirement stays open. - **Estimate:** 10–30 h (corpus pinned in a sibling of `storytold/photocraft-corpus`, metric, fixes in indd-utils or the IDML importer). - **Doc:** [file-format-parity.md](file-format-parity.md)@@ -62,6 +63,7 @@ Kind: **F** feature · **U** UI/UX · **T** typography · **FF** file format · verified in InDesign 2026 with synthetic files), but nothing is measured on real documents. - **Impact:** IDML is the main interchange format until INDD lands, and the only one that goes back to InDesign.+- **Local evidence (2026-10-22):** generated IDML from 205 private INDDs is measured through DesignCraft export/reopen. Original same-save IDML and reference renderings are still absent, so this does not satisfy the independent round-trip requirement. See [private-corpus-testing.md](private-corpus-testing.md). - **Estimate:** 60–100 h (corpus, metric, fixes). - **Doc:** [file-format-parity.md](file-format-parity.md) new file mode 100644@@ -0,0 +1,90 @@+# Private INDD corpus checks — 2026-10-22++Local development evidence for roadmap gaps 2, 3, 4 and 6. This is not a+redistributable corpus, an independent IDML oracle, or a visual-fidelity score.+No private documents, filenames, font files, or artwork are included here.++## Baseline++The five-document pilot passed before expansion. Of 295 discovered INDD paths,+205 had local contents and were tested (203 unique SHA-256 hashes). Ninety+cloud-only inputs were unavailable. Two directory-enumeration timeouts mean+the discovery count is not guaranteed exhaustive.++All 205 tested inputs passed the standalone conversion → IDML import → PDF and+IDML export path, and also the actual `file.open` INDD path → independent+preflight/PDF/IDML stages → reopen/export. The standalone converter was+`d249788285ac083fd29f3150695c9a21337caadc`; this DesignCraft revision's+lockfile pins its internal converter at `7f3060e5bce78d21ea70e0d5eacb96edce15dfbe`.+Those paths remain separately reported.++The checks reuse the existing CLI, engine commands, sandbox/integrity helpers,+and indd-utils' IDML element/story-text extraction. IDs are regenerated on+export, so text multisets and thread-chain shapes are compared independently+of Self IDs. Repeated story text retains multiplicity.++- Direct and separated exports had equal element counts, story-text multisets+ and text-associated thread signatures in all 205 documents.+- Converted IDML versus DesignCraft-exported IDML had exact extracted story+ text in 204/205 documents. The remaining case adds a paragraph break after+ an anchored table. It remains a serialization investigation candidate, not+ evidence of missing text. The text-associated thread signature also changes+ there because it includes the story-text hash.+- Element-count differences include defaults/resources that DesignCraft does+ not retain, and SVG → Image representation changes. Counts alone do not+ establish visible loss or equivalent styling/geometry.+- Existing preflight found missing-font issues in 47 documents, missing links+ in 176, and 123 overset issues across 28 documents. Resource candidates are+ statted without relinking: missing, cloud-only, locally present, and exact+ old-iCloud-root suffix matches remain distinguishable. These availability+ findings must be resolved before attributing reflow to composition bugs.++## Confirmed importer regression and fix++INDD opening supplies raster stand-ins for unavailable linked artwork while+retaining its declared PDF/EPS type. The default IDML exporter embeds those+stand-in bytes. Previously, reopening sniffed their raster format and changed+the placed PDF/EPS into an Image. This affected 36 of the 205 documents+(33 had PDF changes; nine had EPS changes; some had both).++The importer now recognizes embedded raster bytes on a PDF/EPS graphic as a+stand-in, retains the declared graphic type, PDF page and crop, and emits an+explicit warning. Actual bytes read from a linked replacement still determine+its format. No artwork is recovered or relinked by this fix.++The synthetic regression failed before the fix. With the fix, all 161 IDML+library tests pass, including the vector-preview round trip and a linked+raster-replacement counterexample. Reopening all 205 baseline generated IDMLs+with the patched CLI succeeds: PDF/EPS type-change documents fall from 36 to+zero, all compared element counts remain equal, and extracted story text is+equal in all 205. The corrected CLI binary is recorded separately from the+baseline; the baseline results are not overwritten.++## Remaining work++The standalone indd-utils checkout also fixes false orientation warnings caused+by exact comparison of near-identity text transforms. Seven frames in two+documents differed from identity only by 3.6e-15–7.1e-15. A tolerance of+`64 * f64::EPSILON` removes the seven warnings. Re-conversion of all 205 staged+inputs succeeds and all IDML package entries remain byte-for-byte equal to the+baseline. Synthetic tests reject meaningful scaling, shear, the opposite+rotation and non-finite values. This separate converter fix is **not** in+DesignCraft's pinned dependency until it lands upstream and the lockfile is+updated; the importer fix above is already in this local DesignCraft checkout.++Keep gaps 2–4 open: no verified reference rendering or original same-save IDML+was available, and INDT was not tested. Independent page geometry, formatting,+per-cell text, stacking order, and composition comparisons are still needed.+Font/link failures need minimal synthetic reproductions once external resource+availability is controlled. Private corpus checks stay local; public CI can+run the synthetic regression tests without private data.++The alpha Primary Text Frame/autoflow blocker and cross-platform editing gates+are unaffected. No readiness percentage or milestone is raised by this study.++The harness is a sibling of this checkout (`../development.py`,+`../validate_fix.py`, `../validate_converter.py`). It operates on previously+verified staged copies and writes to a separate, non-synced local directory.+Document tools run with OS-enforced network denial and writes confined to+their per-document output directory. Exact per-file findings and logs remain+in the local results, outside this repository.@@ -1,12 +1,18 @@ # DesignCraft roadmap (detail) -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-13 · **Change:** minor (.indd and .indt open through indd-utils; text threads by hand; the gate waits on the Primary Text Frame page fixes) · **Target:** Adobe InDesign 2026 (21.6)+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (private corpus evidence and PDF/EPS preview reopen fix; .indd and .indt open through indd-utils; text threads by hand; the gate waits on the Primary Text Frame page fixes) · **Target:** Adobe InDesign 2026 (21.6) Forward-looking plan. The one-page summary is [ROADMAP.md](../ROADMAP.md); numbers are in [target-app-parity.md](target-app-parity.md); the work list is [gaps.md](gaps.md). Hours are Opus 5.5 agent-hours (calibration in target-app-parity.md). Driving the app from agents: [agents.md](agents.md). +Local development evidence, 2026-10-22: [205 private INDD documents](private-corpus-testing.md)+now have separated and direct-opening checks, structural comparisons and reopen+tests. A preview-backed PDF/EPS type-loss bug is fixed and verified across that+set. Gaps 2–4 remain open for a reproducible reference corpus and fidelity+measurement; the alpha gate and readiness estimates are unchanged.+ ## Alpha gate InDesign's core workflows: what a layout professional does with it every day. DesignCraft is