collective
Browse / github.com/storytold/pdfcraft / patch
fix(cos): read startxref when data follows the final %%EOF
fix(cos): read startxref when data follows the final %%EOF ## What this changes A PDF with more than 4 KiB of data after its final `%%EOF` opened as damaged: ``` cross-reference data unreadable (syntax error at byte N: no startxref); rebuilding from object headers rebuilt cross-reference index from 19 object headers ``` Its own cross-reference table was intact. The reader only looked for `startxref` in the last 4096 bytes, and here it sat about 25 KB before the end. I hit this on a sales proposal produced by Telerik Reporting 17.0: the web server that served it appended its HTML print-preview page (`<!DOCTYPE html>…</html>`) after `%%EOF`. qpdf rep
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- tsw2k (imported from pull request #4983, open)
- Base
1dd2737b47a9c1c9562f0af563053d1bf510e4bf- Size
- +103 −4 in 4 files
- Review
- pending
- Shared
- 2026-10-28
- Id
aa5204ca932edad5cfd1e131
How it went
No other operator has reported using it yet.
Diff
raw · contributors/people.toml crates/cos/README.md crates/cos/src/document/reader.rs crates/cos/src/document/tests.rs
@@ -180,3 +180,8 @@ sync_github_name = true [people.dotCooCoo] real_name = "Robert Lee" sync_github_name = true++[people.tsw2k]+real_name = "Ilya Balashov"+display_name = "Ilya Balashov"+sync_github_name = false@@ -51,6 +51,8 @@ another. Edits never touch the original bytes. streams, hybrid-reference files (`/XRefStm`), and object streams. - **Repair.** A file whose xref is wrong or missing is reconstructed by scanning for `obj` headers. Repairs are *recorded*, never silent, so the UI can say the file was damaged.+ `startxref` is looked for in the last 4 KiB first and then anywhere in the file, so data after+ the final `%%EOF` (a web page glued to a served PDF) is not a reason to rebuild. - **Tolerant, never fatal.** The lexer works on a byte slice and a position; it allocates nothing for skipped data, bounds-checks every index, and caps nesting depth. Malformed input yields `CosError`, never a panic — this crate is the first thing a hostile file meets. Exhausted xref entries are skipped with a repair log; exhausted /Size hints allocate unused numbers without overflowing or replacing existing objects.@@ -23,8 +23,7 @@ impl Document { fn read_compact_xref_chain(&self, log: &mut Vec<String>) -> Result<Option<(XrefIndex, Dict, Vec<Revision>)>, CosError> { let data = &self.data;- let tail_start = data.len().saturating_sub(4096);- let sx = rfind(data, b"startxref", tail_start).ok_or(CosError::Syntax { offset: data.len(), detail: "no startxref".into() })?;+ let sx = find_startxref(data)?; let mut lx = Lexer::new(data, sx + 9); let first = match lx.object() { Ok(Object::Int(n)) if n >= 0 => n as u64,@@ -325,8 +324,7 @@ impl Document { log: &mut Vec<String>, ) -> Result<(BTreeMap<u32, XrefEntry>, Dict, Vec<Revision>, Vec<(u64, u32)>), CosError> { let data = &self.data;- let tail_start = data.len().saturating_sub(4096);- let sx = rfind(data, b"startxref", tail_start).ok_or(CosError::Syntax { offset: data.len(), detail: "no startxref".into() })?;+ let sx = find_startxref(data)?; let mut lx = Lexer::new(data, sx + 9); let first = match lx.object() { Ok(Object::Int(n)) if n >= 0 => n as u64,@@ -811,6 +809,39 @@ pub(crate) fn find(hay: &[u8], needle: &[u8], from: usize, to: usize) -> Option< hay[from..to].windows(needle.len()).position(|w| w == needle).map(|p| p + from) } +/// How far from the end of the file `startxref` is looked for first. ISO 32000-2 §7.5.5 puts it+/// on the last lines of the file.+const STARTXREF_TAIL: usize = 4096;++/// Where the last `startxref` in the file is. The tail is searched first. A file with data+/// after its final `%%EOF` (a web server that glued its HTML page to the PDF it served, a block+/// of padding) has the keyword earlier than that, and the whole file is then searched backwards,+/// so its own cross-reference data is read rather than rebuilt from object headers. The last+/// keyword wins wherever it is: an earlier one could belong to a superseded revision. The+/// trailing bytes stay in the file (an incremental save appends after them).+///+/// Objects after that `startxref` are different: an update that never got its cross-reference+/// section (a save cut short, a second file glued on), whose objects the last complete table+/// cannot reach. The error then says so, and the caller rebuilds the table from object headers+/// as it does for a file with no `startxref` at all.+pub(super) fn find_startxref(data: &[u8]) -> Result<usize, CosError> {+ let tail_start = data.len().saturating_sub(STARTXREF_TAIL);+ if let Some(sx) = rfind(data, b"startxref", tail_start) {+ return Ok(sx);+ }+ let sx = rfind(data, b"startxref", 0).ok_or(CosError::Syntax { offset: data.len(), detail: "no startxref".into() })?;+ // The trailing data starts after the `%%EOF` of that startxref (or after the keyword when+ // the marker is missing; the offset digits that follow are no object header).+ let after = find(data, b"%%EOF", sx, sx.saturating_add(64)).map_or(sx.saturating_add(9), |p| p.saturating_add(5));+ if !object_headers(data.get(after..).unwrap_or(&[])).is_empty() {+ return Err(CosError::Syntax {+ offset: sx,+ detail: format!("startxref is {} bytes before the end of the file and objects follow it", data.len().saturating_sub(sx)),+ });+ }+ Ok(sx)+}+ pub(super) fn rfind(hay: &[u8], needle: &[u8], from: usize) -> Option<usize> { if hay.len() < needle.len() { return None;@@ -1285,3 +1285,64 @@ fn a_stream_whose_length_is_itself_and_never_ends_loads_quickly() { }); rx.recv_timeout(std::time::Duration::from_secs(20)).expect("loading must not retry exponentially"); }++/// Data after the final `%%EOF` (a web server that answered with its HTML print-preview page+/// glued to the PDF, as Telerik Reporting does) pushes `startxref` out of the tail the reader+/// looks at first. The last `startxref` in the file is read instead, so the file's own+/// cross-reference table is used: nothing is rebuilt from object headers and nothing is logged.+/// The trailing bytes stay in the file, and an incremental save goes after them.+#[test]+fn a_startxref_pushed_past_the_tail_by_trailing_data_is_still_read() {+ let base = build(&["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "(three)"], "/Root 1 0 R");+ let mut bytes = base.clone();+ bytes.extend_from_slice(b"\r\n\r\n<!DOCTYPE html>\r\n<html>\r\n");+ for i in 0..400 {+ bytes.extend_from_slice(format!("<p>preview line {i}: no cross-reference keyword here</p>\r\n").as_bytes());+ }+ bytes.extend_from_slice(b"</html>\r\n");+ assert!(bytes.len() - base.len() > 2 * 4096, "the trailing data must reach past the tail window");+ let doc = Document::open(Arc::new(bytes.clone())).unwrap();+ assert!(doc.repair_log().is_empty(), "{:?}", doc.repair_log());+ assert_eq!(doc.revisions().len(), 1);+ assert!(matches!(&*doc.get(ObjRef::new(3, 0)), Object::String(_)));+ assert_compact_matches_materialized(&doc);++ let mut edited = doc.clone();+ edited.update_dict(ObjRef::new(1, 0), |dict| dict.set(b"Lang".to_vec(), Object::String(crate::PdfString::literal("en")))).unwrap();+ let saved = crate::write_incremental(&edited, &crate::SaveOptions::default()).unwrap();+ assert!(saved.starts_with(&bytes), "an incremental save keeps the original bytes, trailing data included");+ let reopened = Document::open(Arc::new(saved)).unwrap();+ assert!(reopened.repair_log().is_empty(), "{:?}", reopened.repair_log());+ assert_eq!(reopened.revisions().len(), 2);+ assert!(matches!(reopened.get(ObjRef::new(1, 0)).as_dict().and_then(|d| d.get(b"Lang")), Some(Object::String(_))));+}++/// The last `startxref` in the file still wins, wherever it is. One inside the trailing data+/// that no offset follows is a bad startxref, and the file is rebuilt from object headers as+/// before: an earlier keyword is never taken instead, since it could belong to a superseded+/// revision.+#[test]+fn a_startxref_keyword_inside_trailing_data_does_not_pick_an_earlier_one() {+ let base = build(&["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "(three)"], "/Root 1 0 R");+ let mut bytes = base.clone();+ bytes.extend_from_slice(b"\n<html><pre>");+ bytes.extend(std::iter::repeat_n(b' ', 5000));+ bytes.extend_from_slice(b"startxref is a PDF keyword</pre></html>\n");+ let doc = Document::open(Arc::new(bytes)).unwrap();+ assert!(doc.repair_log().iter().any(|l| l.contains("object headers")), "{:?}", doc.repair_log());+ assert!(matches!(&*doc.get(ObjRef::new(3, 0)), Object::String(_)));+}++/// Objects after the final `startxref` are an update that never got its cross-reference section+/// (a save cut short). The last complete table cannot reach them, so the file is rebuilt from+/// object headers as before, and the note says why.+#[test]+fn objects_after_the_last_startxref_still_rebuild_the_table() {+ let mut bytes = build(&["<< /Type /Catalog /Pages 2 0 R >>", "<< /Type /Pages /Kids [] /Count 0 >>", "(three)"], "/Root 1 0 R");+ bytes.extend(std::iter::repeat_n(b'\n', 5000));+ bytes.extend_from_slice(b"3 0 obj\n(newer)\nendobj\n");+ let doc = Document::open(Arc::new(bytes)).unwrap();+ assert!(doc.revisions().is_empty(), "rebuilt: {:?}", doc.repair_log());+ assert!(doc.repair_log().iter().any(|l| l.contains("objects follow it")), "{:?}", doc.repair_log());+ assert!(matches!(&*doc.get(ObjRef::new(3, 0)), Object::String(s) if s.bytes == b"newer"));+}