collective
Browse / github.com/storytold/cadcraft / patch
DXF 2000/2004: text in the file's code page, as AutoCAD writes it
DXF 2000/2004: text in the file's code page, as AutoCAD writes it Before 2007, a DXF file's text is in the code page its `$DWGCODEPAGE` names. CADCraft wrote pure ASCII instead, with every other character as a `\U+XXXX` escape. AutoCAD decodes those, but older readers don't. The Italian cadastre's **DOCFA**, which imports floor plans from DXF up to AutoCAD 2000, shows `Caff\U+00E8 20\U+00B0` for `Caffè 20°`. A DXF 2000 from CADCraft should simply work there, as one from AutoCAD 2000 does. ## Change - `cadcraft_dxf::write_ascii_bytes(tags, utf8)` writes the text as AutoCAD does: - UTF-8 from 2007; - before 2007, the characters the header's code page has
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- KvasFan (imported from pull request #2170, open)
- Base
dfddd13656bde4ed7f6c579d85e34874650bc31a- Size
- +192 −18 in 9 files
- Review
- pending
- Shared
- 2026-10-28
- Id
b5ed562a5378cacf7f690b53
How it went
No other operator has reported using it yet.
Diff
raw · crates/dxf/src/codepage.rs crates/dxf/src/lib.rs crates/dxf/src/version.rs crates/io/src/dxf_write.rs crates/io/src/lib.rs crates/io/src/tests.rs crates/io/tests/it/dxf_versions.rs crates/io/tests/it/followups_626.rs docs/file-format-parity.md
@@ -174,6 +174,48 @@ impl BinaryText { } } +/// How text becomes the bytes of a pre-2007 file's code page (the code pages [`decoder`] knows).+pub(crate) enum Encoder {+ Encoding(&'static Encoding),+ Table(&'static [char; 128]),+}++/// The encoder for a `$DWGCODEPAGE` value, or `None` for UTF-8 and code pages CADCraft doesn't+/// know.+pub(crate) fn encoder(code_page: &str) -> Option<Encoder> {+ match decoder(code_page)? {+ Decoder::Utf8 => None,+ Decoder::Encoding(e) => Some(Encoder::Encoding(e)),+ Decoder::Table(t) => Some(Encoder::Table(t)),+ }+}++impl Encoder {+ /// Append `c` in this code page to `out`; false, with nothing appended, when the code page+ /// has no such character, or its bytes read back as another one (Shift-JIS writes `¥` as+ /// 0x5C, which reads back as a backslash).+ pub(crate) fn push(&self, c: char, out: &mut Vec<u8>) -> bool {+ match self {+ Encoder::Table(t) => match t.iter().position(|&x| x == c).and_then(|i| u8::try_from(i).ok()) {+ Some(i) => {+ out.push(0x80 | i);+ true+ }+ None => false,+ },+ Encoder::Encoding(e) => {+ let mut buf = [0u8; 4];+ let (bytes, _, unmappable) = e.encode(c.encode_utf8(&mut buf));+ if unmappable || e.decode_without_bom_handling(&bytes).0.chars().ne(std::iter::once(c)) {+ return false;+ }+ out.extend_from_slice(&bytes);+ true+ }+ }+ }+}+ /// True for an `$ACADVER` of R2007 (AC1021) or later, whose text is UTF-8. fn is_utf8_version(acadver: &str) -> bool { acadver.trim().strip_prefix("AC").and_then(|n| n.parse::<u32>().ok()).is_some_and(|n| n >= 1021)@@ -424,6 +424,70 @@ pub fn write_ascii_as(tags: &[Tag], utf8: bool) -> String { out } +/// Write tags as ASCII DXF bytes, with text as AutoCAD writes it: UTF-8 from 2007 (`utf8`), and+/// before 2007 in the code page the tags' `$DWGCODEPAGE` names, characters outside it as+/// `\U+XXXX`. Older readers that don't decode the escapes (the Italian cadastre's DOCFA, CNC and+/// plotter front ends) then show `è`, not `\U+00E8`. Without a code page CADCraft knows, the text+/// is pure ASCII with escapes, as [`write_ascii`] writes it.+pub fn write_ascii_bytes(tags: &[Tag], utf8: bool) -> Vec<u8> {+ if utf8 {+ return write_ascii_as(tags, true).into_bytes();+ }+ let Some(enc) = header_code_page(tags).and_then(codepage::encoder) else { return write_ascii_as(tags, false).into_bytes() };+ let mut out = Vec::with_capacity(tags.len() * 16);+ for t in tags {+ match &t.value {+ Value::Str(s) if !s.is_ascii() => {+ out.extend_from_slice(format!("{:>3}\r\n", t.code).as_bytes());+ encode_string(s, &enc, &mut out);+ out.extend_from_slice(b"\r\n");+ }+ _ => out.extend_from_slice(write_ascii_as(std::slice::from_ref(t), false).as_bytes()),+ }+ }+ out+}++/// The `$DWGCODEPAGE` value among the tags (the group 3 after the group 9 that names it).+fn header_code_page(tags: &[Tag]) -> Option<&str> {+ let i = tags.iter().position(|t| t.code == 9 && matches!(&t.value, Value::Str(s) if s == "$DWGCODEPAGE"))?;+ match tags.get(i + 1) {+ Some(Tag { code: 3, value: Value::Str(s) }) => Some(s.as_str()),+ _ => None,+ }+}++/// A string in a pre-2007 file's code page: as [`escape_unicode`] writes it, but with the+/// characters the code page has as its bytes. A string whose bytes would also be valid UTF-8 is+/// escaped instead: the reader takes a file that is valid UTF-8 as UTF-8, so every line with a+/// code page byte must not be.+fn encode_string(s: &str, enc: &codepage::Encoder, out: &mut Vec<u8>) {+ let chars: Vec<char> = s.chars().collect();+ let mut bytes = Vec::with_capacity(s.len());+ for (i, &c) in chars.iter().enumerate() {+ match c {+ '\r' | '\n' => bytes.push(b' '),+ '\\' if matches!(chars.get(i + 1), Some('U' | 'M')) && chars.get(i + 2) == Some(&'+') => bytes.extend_from_slice(b"\\U+005C"),+ c if c.is_ascii() => bytes.extend_from_slice(c.encode_utf8(&mut [0u8; 4]).as_bytes()),+ c => {+ if !enc.push(c, &mut bytes) {+ let mut units = [0u16; 2];+ for u in c.encode_utf16(&mut units) {+ bytes.extend_from_slice(format!("\\U+{u:04X}").as_bytes());+ }+ }+ }+ }+ }+ if !bytes.is_ascii() && std::str::from_utf8(&bytes).is_ok() {+ let mut escaped = String::new();+ escape_unicode(s, false, &mut escaped);+ out.extend_from_slice(escaped.as_bytes());+ } else {+ out.extend_from_slice(&bytes);+ }+}+ /// Write tags as binary DXF (DXF Reference, "Binary DXF Files", R13+ layout): the sentinel, /// then per tag a 2-byte little-endian group code and the value in the binary form of the code's /// [`Kind`]: a zero-terminated string, an 8-byte real, a 2-, 4- or 8-byte integer, a 1-byte@@ -568,6 +632,54 @@ pub fn records(tags: &[Tag]) -> Vec<(String, Vec<Tag>)> { mod tests { use super::*; + /// Pre-2007 text is written in the header's code page, as AutoCAD writes it, and reads back.+ #[test]+ fn pre_2007_text_is_written_in_the_code_page() {+ let file = |code_page: &str, text: &str| {+ vec![+ Tag::s(0, "SECTION"),+ Tag::s(2, "HEADER"),+ Tag::s(9, "$ACADVER"),+ Tag::s(1, "AC1015"),+ Tag::s(9, "$DWGCODEPAGE"),+ Tag::s(3, code_page),+ Tag::s(0, "ENDSEC"),+ Tag::s(0, "SECTION"),+ Tag::s(2, "ENTITIES"),+ Tag::s(0, "TEXT"),+ Tag::s(1, text),+ Tag::s(0, "ENDSEC"),+ Tag::s(0, "EOF"),+ ]+ };+ let cases: [(&str, &str, &[u8]); 5] = [+ ("ANSI_1252", "Caffè 20° – ±1", b"Caff\xe8 20\xb0 \x96 \xb11"),+ ("ANSI_1252", "图 è", b"\\U+56FE \xe8"),+ ("ANSI_1251", "Київ", b"\xca\xe8\xbf\xe2"),+ ("DOS850", "è", b"\x8a"),+ // Bytes that would read as UTF-8 ("é" in Windows-1252 is UTF-8 for "é"): escaped.+ ("ANSI_1252", "é", b"\\U+00C3\\U+00A9"),+ ];+ for (code_page, text, expected) in cases {+ let tags = file(code_page, text);+ let bytes = write_ascii_bytes(&tags, false);+ let line = bytes.split(|&b| b == b'\n').map(<[u8]>::trim_ascii).find(|l| l.starts_with(&expected[..1]) && l.len() == expected.len());+ assert_eq!(line, Some(expected), "{code_page} {text}");+ assert_eq!(parse(&bytes).unwrap(), tags, "{code_page} {text} reads back");+ }+ // Shift-JIS encodes ¥ as 0x5C, which reads back as a backslash: a character whose bytes+ // don't decode to it again is escaped.+ let tags = file("ANSI_932", "¥図");+ assert_eq!(parse(&write_ascii_bytes(&tags, false)).unwrap(), tags, "¥ survives Shift-JIS");+ // A literal escape stays literal; no code page CADCraft knows: pure ASCII; 2007+: UTF-8.+ let tags = file("ANSI_1252", "\\U+00E9 è");+ assert_eq!(parse(&write_ascii_bytes(&tags, false)).unwrap(), tags);+ assert_eq!(write_ascii_bytes(&file("NOPE", "è"), false), write_ascii(&file("NOPE", "è")).into_bytes());+ assert_eq!(write_ascii_bytes(&file("ANSI_1252", "è"), true), write_ascii_as(&file("ANSI_1252", "è"), true).into_bytes());+ // ASCII text is written exactly as before.+ assert_eq!(write_ascii_bytes(&file("ANSI_1252", "plain"), false), write_ascii(&file("ANSI_1252", "plain")).into_bytes());+ }+ #[test] fn literal_escape_text_roundtrips() { for utf8 in [false, true] {@@ -75,7 +75,7 @@ impl Version { } /// Whether text in this version's DXF is UTF-8 (2007 and later). Older versions keep text in- /// the `$DWGCODEPAGE` code page, which CADCraft avoids by escaping non-ASCII as `\U+XXXX`.+ /// the `$DWGCODEPAGE` code page, characters outside it as `\U+XXXX` escapes. pub fn utf8(self) -> bool { self >= Version::R2007 }@@ -1234,9 +1234,9 @@ fn entity(w: &mut W, d: &Drawing, e: &Entity, owner: &str, paper: bool, cx: &Ctx } } -/// `$DWGCODEPAGE`: before 2007 the code page read from the file when CADCraft knows it (the text-/// is ASCII with `\U+XXXX` escapes, right in any code page, but the file keeps its declared one);-/// otherwise, and from 2007 (UTF-8 text), `ANSI_1252`.+/// `$DWGCODEPAGE`: before 2007 the code page read from the file when CADCraft knows it (the+/// text is written in it, characters outside it as `\U+XXXX` escapes); otherwise, and from 2007+/// (UTF-8 text), `ANSI_1252`. fn code_page(d: &Drawing, version: Version) -> String { let read = d.header.get("DWGCODEPAGE").and_then(HVal::as_str).map(str::trim).unwrap_or_default(); let known = read.is_ascii() && read.len() <= 32 && cadcraft_dxf::decode_code_page(b"", read).is_some();@@ -88,11 +88,7 @@ pub fn write_with(d: &Drawing, name: &str, opts: &SaveOptions) -> Result<Vec<u8> match ext(name).as_str() { "dxf" | "" => { let tags = dxf_write::tags(d, version);- Ok(if opts.binary {- cadcraft_dxf::write_binary(&tags, version.utf8())- } else {- cadcraft_dxf::write_ascii_as(&tags, version.utf8()).into_bytes()- })+ Ok(if opts.binary { cadcraft_dxf::write_binary(&tags, version.utf8()) } else { cadcraft_dxf::write_ascii_bytes(&tags, version.utf8()) }) } // The DWG bridge reads our R2000 DXF and writes the DWG at the version asked for. Without // one, at least 2004: R2000 DWG has no true colours, transparency, table styles or@@ -1551,6 +1551,25 @@ fn non_ascii_text_is_escaped_for_r2000_and_roundtrips() { assert_eq!(write_dxf(&back).matches("\\U+").count(), text.matches("\\U+").count()); } +/// A DXF saved at 2000 or 2004 has its text in its code page, as AutoCAD writes it: the Latin-1+/// characters of `ANSI_1252` as single bytes (older readers such as the Italian cadastre's DOCFA+/// show `\U+00C4` literally), the others as `\U+XXXX`. It reads back the same.+#[test]+fn a_2000_save_writes_text_in_its_code_page_and_roundtrips() {+ let d = intl_sample();+ for version in [crate::Version::R2000, crate::Version::R2004] {+ let bytes = crate::write_with(&d, "intl.dxf", &crate::SaveOptions { version: Some(version), binary: false }).unwrap();+ assert!(std::str::from_utf8(&bytes).is_err(), "{version}: code page bytes, not UTF-8");+ let has = |needle: &[u8]| bytes.windows(needle.len()).any(|w| w == needle);+ assert!(has(b"\\U+56FE\\U+53F7 \xc4\xb0\xf8 \xd8 \\U+D83D\\U+DE00"), "{version}: CJK and astral escaped, Latin-1 as bytes");+ assert!(!has(b"\\U+00C4"));+ assert_intl(&read_dxf(&bytes).unwrap());+ }+ // 2007 and later stay UTF-8.+ let bytes = crate::write_with(&d, "intl.dxf", &crate::SaveOptions { version: Some(crate::Version::R2018), binary: false }).unwrap();+ assert!(std::str::from_utf8(&bytes).is_ok_and(|t| t.contains("图号 Ä°ø Ø")));+}+ /// Real R2007+ files are UTF-8 and older ones are in their code page; both still read. #[test] fn reads_utf8_and_code_page_text() {@@ -87,9 +87,9 @@ fn dxf_at_every_version_reads_back_the_same() { assert_eq!(acadver(&bytes), v.code(), "{what}"); assert_eq!(cadcraft_dxf::is_binary(&bytes), binary, "{what}"); let raw = bytes.windows("图号".len()).any(|w| w == "图号".as_bytes());- assert_eq!(raw, v.utf8(), "{what}: UTF-8 text from 2007, \\U+ escapes before");+ assert_eq!(raw, v.utf8(), "{what}: UTF-8 text from 2007, the code page (CJK as \\U+ escapes) before"); if !v.utf8() && !binary {- assert!(bytes.is_ascii(), "{what}");+ assert!(std::str::from_utf8(&bytes).is_err(), "{what}: Latin-1 text as ANSI_1252 bytes, as AutoCAD writes it"); } let back = read(&bytes, "x.dxf").unwrap(); assert_eq!(first_text(&back).as_deref(), Some(INTL), "{what}");@@ -99,11 +99,16 @@ fn dxf_at_every_version_reads_back_the_same() { } } -/// R2000 ASCII is exactly the R2000 DXF CADCraft always wrote.+/// R2000 ASCII holds exactly the tags of the R2000 DXF CADCraft always wrote; only its non-ASCII+/// text is now in the code page instead of escaped. #[test] fn r2000_ascii_is_unchanged() { let d = sample();- assert_eq!(write_with(&d, "x.dxf", &opts(Version::R2000, false)).unwrap(), write_dxf(&d).into_bytes());+ let saved = write_with(&d, "x.dxf", &opts(Version::R2000, false)).unwrap();+ assert_eq!(cadcraft_dxf::parse(&saved).unwrap(), cadcraft_dxf::parse(write_dxf(&d).as_bytes()).unwrap());+ let mut ascii = Drawing::new_metric();+ ascii.add(&Space::Model, Common::default(), EntityKind::Line(Line { a: Vec3::ZERO, b: Vec3::new(1.0, 0.0, 0.0) })).unwrap();+ assert_eq!(write_with(&ascii, "x.dxf", &opts(Version::R2000, false)).unwrap(), write_dxf(&ascii).into_bytes(), "ASCII text: byte for byte"); } /// Without a version asked for: 2018 for a new drawing, otherwise the file's own version (the@@ -177,10 +177,9 @@ fn code_page_is_kept_before_2007() { .unwrap(); for v in [Version::R2000, Version::R2004] { let bytes = save(&d, "x.dxf", v);- assert!(bytes.is_ascii(), "{v:?}: ASCII with escapes");- let text = String::from_utf8(bytes).unwrap();- assert_eq!(code_page(&text), "ANSI_1251", "{v:?}");- let back = cadcraft_io::read_dxf(text.as_bytes()).unwrap();+ assert!(bytes.windows(6).any(|w| w == b"\xcf\xf0\xe8\xe2\xe5\xf2"), "{v:?}: the text in ANSI_1251, as AutoCAD writes it");+ assert_eq!(code_page(&String::from_utf8_lossy(&bytes)), "ANSI_1251", "{v:?}");+ let back = cadcraft_io::read_dxf(&bytes).unwrap(); assert_eq!(back.header.str("DWGCODEPAGE", ""), "ANSI_1251"); assert!(back.model.iter().any(|e| matches!(&e.kind, EntityKind::Text(t) if t.value == "Привет"))); }@@ -1,6 +1,6 @@ # File-format parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-22 · **Change:** minor (DIMASSOC objects for angular, diametric and radial dimensions) · **Target:** Autodesk AutoCAD 2027+> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-25 · **Change:** minor (2000/2004 DXF text in the file's code page, as AutoCAD writes it) · **Target:** Autodesk AutoCAD 2027 Every format AutoCAD 2027 reads or writes, with CADCraft's support, fidelity and tests. Work items are in [gaps.md](gaps.md#file-formats); the weighted score feeds@@ -140,6 +140,7 @@ our output in AutoCAD black-box. | Date | Change | Summary | |---|---|---|+| 2026-10-25 | minor | DXF save at 2000 and 2004: text is written in the file's `$DWGCODEPAGE` (`è`, `°` as `ANSI_1252` bytes), as AutoCAD writes it, with `\U+XXXX` only for characters outside the code page; older readers that don't decode the escapes, such as the Italian cadastre's DOCFA, showed `\U+00E8`. DWG 2000/2004 already stored the code page text | | 2026-10-22 | minor | DXF/DWG save: angular, diametric and radial dimensions get standard DIMASSOC objects (one centre reference to the circle for radial/diametric, the definition points for angular), so AutoCAD follows their geometry; the standard objects of those kinds are read back too, without CADCraft's xdata (#1651) | | 2026-10-19 | minor | DXF/DWG save: angular dimension block text honors DIMDSEP, including per-dimension overrides (#1646) | | 2026-10-19 | minor | A file's own arrowhead blocks with standard names (AutoCAD's `_Dot`, `_DotSmall`), named only by a dimension style, are kept and saved as they were instead of being replaced by CADCraft's solids; PURGE keeps arrowhead blocks dimension styles name |