diff --git a/ROADMAP.md b/ROADMAP.md --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-25 · **Change:** minor (AutoSum over a selected row puts its total to the right, and fills a selection's empty last row and column) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-27 · **Change:** minor (Linux font discovery follows fontconfig; Arabic/Persian fallback, shaping and script-only word order) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -135,6 +135,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | What landed | |---|---| +| 2026-10-27 | Linux font discovery follows fontconfig's `` and `` entries instead of only fixed paths, so a Persian/Arabic font is found wherever it is installed. The Arabic-script fallback now includes Vazirmatn, Noto Sans/Naskh/Kufi Arabic, Shabnam, Sahel, Amiri and the platform faces; HarfRust shapes and joins Persian/Arabic, and an Arabic-only cell whose chosen font lacks Arabic leads with the script face so “سلام دنیا” keeps its word order. Bidi, mixed Latin/Arabic order, wrapped RTL lines and RTL alignment remain unsupported | | 2026-10-25 | AutoSum over a selection does what Excel does: a selected row gets one total to its right (in its last cell if that is empty), not a one-cell total under each cell; a block whose last column is empty gets each row's total there, one whose last row is empty each column's, and with both the corner gets the total of the block; otherwise totals go under each column as before | | 2026-10-24 | Go To lists the named ranges of every sheet (`Sheet2!Name` for a name scoped to another sheet; hidden names, constants and formulas left out) and the references entered before, a double-click goes to an entry, and the Reference box takes several names or references separated by commas (`Price, Type`, `B14:C22,F19:G30`) | | 2026-10-24 | Status bar: a right-click opens a menu that sets which selection statistics show (Average, Count and Sum by default; Numerical Count, Minimum and Maximum are new), kept with the preferences (`view.statusBar`) | diff --git a/crates/ui-egui/src/fonts.rs b/crates/ui-egui/src/fonts.rs --- a/crates/ui-egui/src/fonts.rs +++ b/crates/ui-egui/src/fonts.rs @@ -10,7 +10,7 @@ //! Native only: on wasm there are no system fonts, every function here reports "nothing", and the //! font list is the standard [`crate::ribbon::FONTS`] set. -use std::collections::HashSet; +use std::collections::{HashMap, HashSet}; use std::path::PathBuf; use std::sync::OnceLock; @@ -41,6 +41,9 @@ pub struct Face { pub weight: f32, /// Whether the style name reads as italic or oblique. pub italic: bool, + /// Whether the face maps a common Arabic letter. A cell that names this family can then shape + /// Persian/Arabic in one face instead of splitting the line at a fallback boundary. + pub arabic: bool, } /// The font folders of the platform (the system's and the user's). Empty on wasm. @@ -70,11 +73,200 @@ pub fn system_font_dirs() -> Vec { dirs.push(h.join(".fonts")); dirs.push(h.join(".local/share/fonts")); } + // Folders fontconfig is configured with: a distribution need not keep every font under + // the standard paths above, and a user's own folder may live anywhere. Parsing + // fontconfig's configuration keeps discovery dynamic instead of guessing layouts. + dirs.extend(fontconfig_dirs()); } dirs } } +/// Font folders from fontconfig's configuration, so a font installed where fontconfig points is +/// found even when it is not under a distribution's default path (a user's font folder, a custom +/// prefix, ChromeOS's Linux container). Follows `` elements (a directory include pulls in +/// its `.conf` files) and expands `~` and `prefix="xdg"`. A missing or malformed file contributes +/// nothing, never a panic; empty on wasm. +pub fn fontconfig_dirs() -> Vec { + #[cfg(target_arch = "wasm32")] + { + Vec::new() + } + #[cfg(not(target_arch = "wasm32"))] + { + let mut out: Vec = Vec::new(); + let mut seen_dirs: HashSet = HashSet::new(); + let mut seen_files: HashSet = HashSet::new(); + for file in fontconfig_config_files() { + parse_fontconfig_file(&file, &mut out, &mut seen_dirs, &mut seen_files, 0); + } + out + } +} + +/// The fontconfig configuration files, system first then user, as fontconfig reads them. The +/// `FONTCONFIG_FILE` environment variable overrides the main configuration file when set. +#[cfg(not(target_arch = "wasm32"))] +fn fontconfig_config_files() -> Vec { + let home = std::env::var_os("HOME").map(PathBuf::from); + let mut files: Vec = Vec::new(); + match std::env::var_os("FONTCONFIG_FILE") { + Some(f) => files.push(PathBuf::from(f)), + None => files.push(PathBuf::from("/etc/fonts/fonts.conf")), + } + files.push(PathBuf::from("/etc/fonts/local.conf")); + if let Some(config) = std::env::var_os("XDG_CONFIG_HOME").map(PathBuf::from).or_else(|| home.as_ref().map(|h| h.join(".config"))) { + files.push(config.join("fontconfig/fonts.conf")); + } + if let Some(h) = &home { + files.push(h.join(".fonts.conf")); // the pre-XDG location + } + files +} + +/// How deep ``s are followed before giving up, so a cycle can't spin forever. +#[cfg(not(target_arch = "wasm32"))] +const MAX_FONTCONFIG_DEPTH: usize = 16; + +/// Read one fontconfig file: collect its `` folders and recurse into its ``s. An +/// include of a directory pulls in that directory's `.conf` files, as fontconfig does. +#[cfg(not(target_arch = "wasm32"))] +fn parse_fontconfig_file( + path: &std::path::Path, + out: &mut Vec, + seen_dirs: &mut HashSet, + seen_files: &mut HashSet, + depth: usize, +) { + if depth > MAX_FONTCONFIG_DEPTH { + return; + } + let key = std::fs::canonicalize(path).unwrap_or_else(|_| path.to_path_buf()); + if !seen_files.insert(key) { + return; + } + if path.is_dir() { + let mut entries: Vec = std::fs::read_dir(path) + .map(|rd| rd.flatten().map(|e| e.path()).filter(|p| p.extension().is_some_and(|e| e.eq_ignore_ascii_case("conf"))).collect()) + .unwrap_or_default(); + entries.sort(); + for entry in entries { + parse_fontconfig_file(&entry, out, seen_dirs, seen_files, depth + 1); + } + return; + } + let Ok(text) = std::fs::read_to_string(path) else { return }; + let base = path.parent().map(std::path::Path::to_path_buf).unwrap_or_default(); + for (attrs, inner) in xml_elements(&text, "include") { + let value = decode_xml_entities(inner).trim().to_string(); + if value.is_empty() { + continue; + } + let target = resolve_fontconfig_path(&value, xml_attr(&attrs, "prefix").as_deref(), &base); + parse_fontconfig_file(&target, out, seen_dirs, seen_files, depth + 1); + } + for (attrs, inner) in xml_elements(&text, "dir") { + let value = decode_xml_entities(inner).trim().to_string(); + if value.is_empty() { + continue; + } + let dir = resolve_fontconfig_path(&value, xml_attr(&attrs, "prefix").as_deref(), &base); + if dir.is_dir() { + let key = std::fs::canonicalize(&dir).unwrap_or_else(|_| dir.clone()); + if seen_dirs.insert(key) { + out.push(dir); + } + } + } +} + +/// Resolve a fontconfig path value: `~` is the home folder, `prefix="xdg"` means +/// `$XDG_DATA_HOME` (defaulting to `~/.local/share`), `prefix="cwd"` the working folder, and any +/// other relative value is relative to the file that named it. +#[cfg(not(target_arch = "wasm32"))] +fn resolve_fontconfig_path(value: &str, prefix: Option<&str>, base: &std::path::Path) -> PathBuf { + let home = std::env::var_os("HOME").map(PathBuf::from); + if let Some(rest) = value.strip_prefix("~/") { + return home.map(|h| h.join(rest)).unwrap_or_else(|| PathBuf::from(value)); + } + if value == "~" { + return home.unwrap_or_else(|| PathBuf::from(value)); + } + let path = PathBuf::from(value); + if path.is_absolute() { + return path; + } + match prefix { + Some("xdg") => std::env::var_os("XDG_DATA_HOME") + .map(PathBuf::from) + .or_else(|| home.map(|h| h.join(".local/share"))) + .map(|d| d.join(value)) + .unwrap_or_else(|| base.join(value)), + Some("cwd") => std::env::current_dir().map(|d| d.join(value)).unwrap_or_else(|_| base.join(value)), + _ => base.join(value), + } +} + +/// The attributes and text content of every `` element in `text`, in document order. Not a +/// full XML parser: fontconfig's configuration only uses simple elements, and malformed input is +/// ignored rather than rejected. +#[cfg(not(target_arch = "wasm32"))] +fn xml_elements<'a>(text: &'a str, name: &str) -> Vec<(String, &'a str)> { + let open = format!("<{name}"); + let close = format!(""); + let mut out: Vec<(String, &str)> = Vec::new(); + let mut rest = text; + while let Some(i) = rest.find(&open) { + rest = &rest[i + open.len()..]; + // The tag name must end here: `` is not ``. + if !rest.starts_with(|c: char| c.is_whitespace() || c == '>' || c == '/') { + continue; + } + let Some(gt) = rest.find('>') else { break }; + let attrs = rest[..gt].to_string(); + let self_closing = attrs.trim_end().ends_with('/'); + rest = &rest[gt + 1..]; + if self_closing { + out.push((attrs, "")); + continue; + } + let Some(end) = rest.find(&close) else { break }; + out.push((attrs, &rest[..end])); + rest = &rest[end + close.len()..]; + } + out +} + +/// The value of the XML attribute `name` in `attrs`, if quoted with `"` or `'`. +#[cfg(not(target_arch = "wasm32"))] +fn xml_attr(attrs: &str, name: &str) -> Option { + let key = format!("{name}="); + let mut rest = attrs; + while let Some(i) = rest.find(&key) { + if rest[..i].chars().next_back().is_none_or(char::is_whitespace) { + let after = &rest[i + key.len()..]; + let quote = after.chars().next()?; + if quote == '"' || quote == '\'' { + let value = &after[quote.len_utf8()..]; + if let Some(end) = value.find(quote) { + return Some(value[..end].to_string()); + } + } + } + rest = &rest[i + key.len()..]; + } + None +} + +/// Decode the XML entities that can appear in a fontconfig path. Anything unknown is left as is. +#[cfg(not(target_arch = "wasm32"))] +fn decode_xml_entities(text: &str) -> String { + if !text.contains('&') { + return text.to_string(); + } + text.replace("<", "<").replace(">", ">").replace(""", "\"").replace("'", "'").replace("&", "&") +} + /// Weight implied by a style name (`"Bold"` → 700, `"Light"` → 300). #[cfg(not(target_arch = "wasm32"))] fn style_weight(style: &str) -> f32 { @@ -111,10 +303,10 @@ fn style_italic(style: &str) -> bool { s.contains("italic") || s.contains("oblique") } -/// Every face in `data` as `(index, family, names, style)`, where `names` holds every family name -/// (see [`Face::names`]). Damaged data yields nothing, never a panic. +/// Every face in `data` as `(index, family, names, style, arabic)`, where `names` holds every +/// family name (see [`Face::names`]). Damaged data yields nothing, never a panic. #[cfg(not(target_arch = "wasm32"))] -fn file_faces(data: &[u8]) -> Vec<(u32, String, Vec, String)> { +fn file_faces(data: &[u8]) -> Vec<(u32, String, Vec, String, bool)> { use skrifa::raw::FileRef; use skrifa::{MetadataProvider, string::StringId}; let count = match FileRef::new(data) { @@ -141,7 +333,10 @@ fn file_faces(data: &[u8]) -> Vec<(u32, String, Vec, String)> { } } let style = name(&f, &[StringId::TYPOGRAPHIC_SUBFAMILY_NAME, StringId::SUBFAMILY_NAME]).unwrap_or_else(|| "Regular".into()); - Some((i, family, names, style)) + // A representative Arabic letter, enough to tell an Arabic-capable face from a Latin-only + // one without trusting its family name. + let arabic = f.charmap().map('\u{0633}').is_some(); + Some((i, family, names, style, arabic)) }) .collect() } @@ -173,8 +368,8 @@ pub fn scan_dirs(dirs: &[PathBuf]) -> Vec { continue; } let Ok(data) = std::fs::read(&path) else { continue }; - for (index, family, names, style) in file_faces(&data) { - out.push(Face { family, names, path: path.clone(), index, weight: style_weight(&style), italic: style_italic(&style) }); + for (index, family, names, style, arabic) in file_faces(&data) { + out.push(Face { family, names, path: path.clone(), index, weight: style_weight(&style), italic: style_italic(&style), arabic }); } } } @@ -224,6 +419,30 @@ pub fn faces_of(family: &str) -> Vec { catalog_now().map(|cat| faces_named(cat, family)).unwrap_or_default() } +/// One lookup per family and weight: cell painting asks this for every visible Arabic cell, while +/// the catalog scan is linear in installed faces. +static ARABIC_FAMILIES: OnceLock>> = OnceLock::new(); + +fn arabic_families() -> &'static std::sync::RwLock> { + ARABIC_FAMILIES.get_or_init(|| std::sync::RwLock::new(HashMap::new())) +} + +/// Whether the face the family would use for this weight/slant maps Arabic letters. `false` until +/// the background scan has found the family, and when that primary face has no Arabic coverage (its +/// regular sibling does not help: a space would resolve to the primary face and split the line). +pub fn family_supports_arabic(name: &str, bold: bool, italic: bool) -> bool { + if catalog_now().is_none() { + return false; + } + let key = (name.to_string(), bold, italic); + if let Some(answer) = arabic_families().read().unwrap_or_else(|e| e.into_inner()).get(&key) { + return *answer; + } + let answer = pick_face(&faces_of(name), bold, italic).is_some_and(|f| f.arabic); + arabic_families().write().unwrap_or_else(|e| e.into_inner()).insert(key, answer); + answer +} + /// The faces in `cat` that go by `wanted`. When two families claim one name the winner is fixed, /// not left to the scan's directory order: a family whose own (English) name it is wins, else the /// family of the first claiming face by path and index. Only the faces that carry the name are @@ -461,7 +680,7 @@ mod tests { std::fs::create_dir_all(&dir).expect("temp dir"); let path = dir.join("Ubuntu-Light.ttf"); std::fs::write(&path, epaint_default_fonts::UBUNTU_LIGHT).expect("write"); - let face = Face { family: "Ubuntu".into(), names: vec!["Ubuntu".into()], path, index: 0, weight: 300.0, italic: false }; + let face = Face { family: "Ubuntu".into(), names: vec!["Ubuntu".into()], path, index: 0, weight: 300.0, italic: false, arabic: false }; let mut fonts = egui::FontDefinitions::default(); let bound = bind_faces(&mut fonts, "Ubuntu", std::slice::from_ref(&face)); @@ -512,7 +731,15 @@ mod tests { const KO: u16 = 0x0412; fn face(family: &str, names: &[&str], path: &str, weight: f32) -> Face { - Face { family: family.into(), names: names.iter().map(|s| s.to_string()).collect(), path: path.into(), index: 0, weight, italic: false } + Face { + family: family.into(), + names: names.iter().map(|s| s.to_string()).collect(), + path: path.into(), + index: 0, + weight, + italic: false, + arabic: false, + } } #[cfg(not(target_arch = "wasm32"))] @@ -522,14 +749,14 @@ mod tests { let font = font_with_names(&[(EN, 1, "Malgun Gothic"), (EN, 2, "Regular"), (KO, 1, "맑은 고딕"), (KO, 2, "보통")]); let faces = file_faces(&font); assert_eq!(faces.len(), 1); - let (_, family, names, style) = &faces[0]; + let (_, family, names, style, _) = &faces[0]; assert_eq!(family, "Malgun Gothic", "the font list keeps the English name"); assert_eq!(names, &["Malgun Gothic", "맑은 고딕"]); assert_eq!(style, "Regular"); // Both name IDs are kept: a legacy family name (ID 1) names a single face (#1606). let font = font_with_names(&[(EN, 1, "Noto Sans KR Medium"), (EN, 2, "Regular"), (EN, 16, "Noto Sans KR"), (EN, 17, "Medium")]); - let (_, family, names, _) = &file_faces(&font)[0]; + let (_, family, names, _, _) = &file_faces(&font)[0]; assert_eq!(family, "Noto Sans KR"); assert_eq!(names, &["Noto Sans KR", "Noto Sans KR Medium"]); } @@ -539,7 +766,7 @@ mod tests { fn a_name_table_full_of_names_is_capped() { let texts: Vec = (0..100).map(|i| format!("Family {i}")).collect(); let records: Vec<(u16, u16, &str)> = texts.iter().enumerate().map(|(i, t)| (0x0400 + i as u16, 1, t.as_str())).collect(); - let (_, _, names, _) = &file_faces(&font_with_names(&records))[0]; + let (_, _, names, _, _) = &file_faces(&font_with_names(&records))[0]; assert_eq!(names.len(), MAX_FAMILY_NAMES); } @@ -599,7 +826,15 @@ mod tests { std::fs::create_dir_all(&dir).expect("temp dir"); let path = dir.join("Ubuntu-Light.ttf"); std::fs::write(&path, epaint_default_fonts::UBUNTU_LIGHT).expect("write"); - let f = Face { family: "Ubuntu".into(), names: vec!["Ubuntu".into(), "우분투".into()], path, index: 0, weight: 300.0, italic: false }; + let f = Face { + family: "Ubuntu".into(), + names: vec!["Ubuntu".into(), "우분투".into()], + path, + index: 0, + weight: 300.0, + italic: false, + arabic: false, + }; let mut fonts = egui::FontDefinitions::default(); let before = fonts.font_data.len(); @@ -611,6 +846,40 @@ mod tests { let _ = std::fs::remove_dir_all(&dir); } + #[cfg(not(target_arch = "wasm32"))] + #[test] + fn fontconfig_configuration_adds_directories_and_follows_includes() { + let base = std::env::temp_dir().join(format!("gridcraft-fontconfig-{}", std::process::id())); + let _ = std::fs::remove_dir_all(&base); + for d in ["fonts", "relative", "conf.d/fonts"] { + std::fs::create_dir_all(base.join(d)).expect("temp dir"); + } + // A directory include pulls in its .conf files, and a relative is relative to the + // file that names it. A cycle (second.conf includes the root again) must terminate. + std::fs::write(base.join("conf.d/10-extra.conf"), "fonts").expect("write"); + std::fs::write( + base.join("root.conf"), + "conf.dsecond.confrelativemissing", + ) + .expect("write"); + std::fs::write(base.join("second.conf"), "root.conf").expect("write"); + + let mut out: Vec = Vec::new(); + let mut seen_dirs: HashSet = HashSet::new(); + let mut seen_files: HashSet = HashSet::new(); + parse_fontconfig_file(&base.join("root.conf"), &mut out, &mut seen_dirs, &mut seen_files, 0); + assert!(out.iter().any(|p| p.ends_with("conf.d/fonts")), "directory include: {out:?}"); + assert!(out.iter().any(|p| p.ends_with("relative")), "relative dir: {out:?}"); + assert!(!out.iter().any(|p| p.ends_with("missing")), "a non-directory is not offered: {out:?}"); + + // Malformed input contributes nothing and never panics. + std::fs::write(base.join("broken.conf"), "unclosed").expect("write"); + let mut out = Vec::new(); + parse_fontconfig_file(&base.join("broken.conf"), &mut out, &mut HashSet::new(), &mut HashSet::new(), 0); + assert!(out.is_empty()); + let _ = std::fs::remove_dir_all(&base); + } + /// On a Windows machine with Malgun Gothic, the real file carries its Korean name. #[cfg(windows)] #[test] @@ -618,6 +887,6 @@ mod tests { let root = std::env::var_os("WINDIR").map(PathBuf::from).unwrap_or_else(|| PathBuf::from("C:\\Windows")); let Ok(data) = std::fs::read(root.join("Fonts").join("malgun.ttf")) else { return }; let faces = file_faces(&data); - assert!(faces.iter().any(|(_, family, names, _)| family == "Malgun Gothic" && names.iter().any(|n| n == "맑은 고딕")), "{faces:?}"); + assert!(faces.iter().any(|(_, family, names, _, _)| family == "Malgun Gothic" && names.iter().any(|n| n == "맑은 고딕")), "{faces:?}"); } } diff --git a/crates/ui-egui/src/formula_bar.rs b/crates/ui-egui/src/formula_bar.rs --- a/crates/ui-egui/src/formula_bar.rs +++ b/crates/ui-egui/src/formula_bar.rs @@ -111,13 +111,15 @@ pub fn show(app: &mut SheetApp, ui: &mut egui::Ui) { let id = egui::Id::unique("gridcraft.formula_bar"); // A spill child shows its anchor's formula muted and read-only. let read_only = active_spill_anchor(app).is_some(); - let info = if app.editor.is_some() { - editor_widget(app, &mut child, id, font.clone(), true, inner.width(), inner.height()) + let info = if let Some(edit) = app.editor.as_ref() { + let edit_font = theme::font_for_text(font.clone(), &edit.text); + editor_widget(app, &mut child, id, edit_font, true, inner.width(), inner.height()) } else { // Show the active cell's content; clicking starts editing in the bar. let text = active_input(app); let color = if read_only { t.text_disabled } else { t.text }; - let galley = child.painter().layout(text, font.clone(), color, inner.width()); + let text_font = theme::font_for_text(font.clone(), &text); + let galley = child.painter().layout(text, text_font, color, inner.width()); let lines = galley.rows.len(); // Each cell's view starts at its top: the offset is kept per cell, not per bar. let shown = app.session.active().map(|d| (d.uid, d.wb.active_sheet(), d.selection.active.row, d.selection.active.col)); diff --git a/crates/ui-egui/src/grid/editor.rs b/crates/ui-egui/src/grid/editor.rs --- a/crates/ui-egui/src/grid/editor.rs +++ b/crates/ui-egui/src/grid/editor.rs @@ -28,7 +28,7 @@ pub(super) fn in_cell_editor(app: &mut SheetApp, ui: &mut egui::Ui, geo: &Geo, s }; let st = wb.styles.get(sh.style_id(cell)).clone(); let size = st.font.size * 96.0 / 72.0 * geo.z; - let font = FontId::new(size, theme::cell_family(&st.font.name, st.font.bold, st.font.italic)); + let font = FontId::new(size, theme::cell_family_for_text(&st.font.name, st.font.bold, st.font.italic, &ed.text)); let lines = ed.text.lines().count().max(1) as f32 + if ed.text.ends_with('\n') { 1.0 } else { 0.0 }; // Grows with the text: the in-cell editor shows every line and never scrolls (the formula bar // is the one that stops at five lines). diff --git a/crates/ui-egui/src/grid/quadrant.rs b/crates/ui-egui/src/grid/quadrant.rs --- a/crates/ui-egui/src/grid/quadrant.rs +++ b/crates/ui-egui/src/grid/quadrant.rs @@ -231,6 +231,8 @@ pub(crate) fn paint_quadrant( if text.is_empty() { continue; } + // Arabic-only text leads with an Arabic face so egui does not split it at the spaces. + let font = FontId::new(size, theme::cell_family_for_text(&st.font.name, st.font.bold, st.font.italic, &text)); let color = ncolor.unwrap_or(lk.text_color); let halign = match st.align.h { HAlign::General => match v { diff --git a/crates/ui-egui/src/theme.rs b/crates/ui-egui/src/theme.rs --- a/crates/ui-egui/src/theme.rs +++ b/crates/ui-egui/src/theme.rs @@ -25,12 +25,13 @@ mod font_setup; pub use font_fallback::symbol_face_installed; #[cfg(test)] -use font_fallback::{cjk_han_japanese, cjk_han_simplified, cjk_hangul, fallback_faces, hebrew, lao, symbols_more, thai}; +use font_fallback::{arabic, cjk_han_japanese, cjk_han_simplified, cjk_hangul, fallback_faces, hebrew, lao, symbols_more, thai}; use font_setup::pick; pub use font_setup::{ - CELL, CELL_BOLD, CELL_BOLD_ITALIC, CELL_ITALIC, CraftFont, HanOrder, MONO, SERIF, UI, UI_BOLD, available_text, cell_family, cell_text_width, - chinese_font_loaded, craft_font_scripts, craft_fonts_cover, craft_fonts_loaded, craft_fonts_required, font_definitions, - font_definitions_for_language, font_definitions_with, pinned_font_definitions, symbol_fallback, ui_bold, ui_font, + CELL, CELL_ARABIC, CELL_BOLD, CELL_BOLD_ITALIC, CELL_ITALIC, CraftFont, HanOrder, MONO, SERIF, UI, UI_BOLD, available_text, cell_family, + cell_family_for_text, cell_text_width, chinese_font_loaded, craft_font_scripts, craft_fonts_cover, craft_fonts_loaded, craft_fonts_required, + font_definitions, font_definitions_for_language, font_definitions_with, font_for_text, pinned_font_definitions, symbol_fallback, ui_bold, + ui_font, }; #[cfg(test)] use font_setup::{candidates, craft_font_faces, try_load}; diff --git a/crates/ui-egui/src/theme/font_fallback.rs b/crates/ui-egui/src/theme/font_fallback.rs --- a/crates/ui-egui/src/theme/font_fallback.rs +++ b/crates/ui-egui/src/theme/font_fallback.rs @@ -54,6 +54,7 @@ pub(super) fn fallback_faces(han: HanOrder) -> Vec<(&'static str, Vec<(String, u ("sys-cjk-hangul", cjk_hangul()), ("sys-thai", thai()), ("sys-lao", lao()), + ("sys-arabic", arabic()), ("sys-hebrew", hebrew()), ("sys-symbols", symbols()), ("sys-symbols-2", symbols_more()), @@ -241,6 +242,70 @@ pub(super) fn hebrew() -> Vec<(String, u32)> { ) } +/// Arabic-script faces (Arabic, Persian, Urdu). Windows ships Segoe UI, Tahoma and Arial; macOS has +/// Geeza Pro and Noto Sans Arabic on newer versions; Linux usually has Noto Sans Arabic, Vazirmatn +/// or DejaVu Sans. The installed-font catalog comes first, so a face fontconfig found in a custom +/// folder is used too; the fixed paths follow for machines whose scan has not finished yet. +#[cfg(not(target_arch = "wasm32"))] +pub(super) fn arabic() -> Vec<(String, u32)> { + let mut v: Vec<(String, u32)> = Vec::new(); + // Prefer a face whose family name says it is designed for Arabic/Persian, then the broad + // Noto/DejaVu faces. One regular face per family; load_fallback stops at the first that parses. + for family in [ + "Vazirmatn", + "Noto Naskh Arabic", + "Noto Sans Arabic", + "Noto Kufi Arabic", + "Noto Naskh Arabic UI", + "Noto Sans Arabic UI", + "Shabnam", + "Sahel", + "Amiri", + "Noto Sans", + "Segoe UI", + "Tahoma", + "Arial", + "Geeza Pro", + "DejaVu Sans", + ] { + if let Some(face) = crate::fonts::faces_of(family).into_iter().find(|f| !f.italic && (f.weight - 400.0).abs() < 1.0) { + v.push((face.path.to_string_lossy().into_owned(), face.index)); + } + } + let w = windows_fonts(); + for f in ["segoeui.ttf", "tahoma.ttf", "arial.ttf", "times.ttf"] { + v.push((format!("{w}\\{f}"), 0)); + } + for p in [ + "/System/Library/Fonts/GeezaPro.ttc", + "/System/Library/Fonts/Supplemental/Geeza Pro.ttc", + "/System/Library/Fonts/Supplemental/NotoSansArabic-Regular.ttf", + "/System/Library/Fonts/Supplemental/Arial.ttf", + "/System/Library/Fonts/Supplemental/Times New Roman.ttf", + ] { + v.push((p.into(), 0)); + } + for d in [ + "/run/host/fonts", + "/run/host/local-fonts", + "/run/host/user-fonts", + "/usr/share/fonts/truetype/noto", + "/usr/share/fonts/opentype/noto", + "/usr/share/fonts/truetype", + "/usr/share/fonts", + "/usr/local/share/fonts", + ] { + v.push((format!("{d}/NotoSansArabic-Regular.ttf"), 0)); + v.push((format!("{d}/Vazirmatn-Regular.ttf"), 0)); + v.push((format!("{d}/vazirmatn/Vazirmatn-Regular.ttf"), 0)); + v.push((format!("{d}/dejavu/DejaVuSans.ttf"), 0)); + } + // Fedora's variable Noto Sans Arabic (and the mac supplemental folder path above) covers + // distributions that package only the variable font. + v.push(("/usr/share/fonts/google-noto-vf/NotoSansArabic[wght].ttf".into(), 0)); + v +} + /// Symbol faces behind the UI font. Interface fonts such as Noto Sans and Cantarell (and egui's /// built-in Ubuntu) lack arrows, check marks, geometric shapes and keyboard symbols (▾ ✓ ✕ ▲ ↳ ⇧), /// which then render as empty boxes (#197, #54). DejaVu Sans covers most of them; the paths cover diff --git a/crates/ui-egui/src/theme/font_setup.rs b/crates/ui-egui/src/theme/font_setup.rs --- a/crates/ui-egui/src/theme/font_setup.rs +++ b/crates/ui-egui/src/theme/font_setup.rs @@ -35,6 +35,10 @@ pub const CELL_ITALIC: &str = "cell-italic"; pub const CELL_BOLD_ITALIC: &str = "cell-bold-italic"; +/// Cell text in an Arabic-script face first: a whole Persian/Arabic sentence shares one face, so a +/// space cannot split it into separately shaped runs (egui has no bidi reordering between runs). +pub const CELL_ARABIC: &str = "cell-arabic"; + pub const SERIF: &str = "serif"; pub const MONO: &str = "mono"; @@ -81,16 +85,67 @@ pub fn cell_family(name: &str, bold: bool, italic: bool) -> FontFamily { ) } +/// The family for cell text: the workbook's own font when it carries Arabic glyphs, else a +/// script-leading family for Arabic-only text. egui assigns each character to the first family +/// that has it; with an Arabic-only line led by the Latin font, the spaces belong to Latin and the +/// Arabic words become separate runs, which egui draws left to right (#2384). +pub fn cell_family_for_text(name: &str, bold: bool, italic: bool, text: &str) -> FontFamily { + if is_arabic_only(text) { + let role = crate::fonts::role_key(name, bold, italic); + if crate::fonts::is_bound(&role) && crate::fonts::family_supports_arabic(name, bold, italic) { + return FontFamily::Name(role.into()); + } + if crate::fonts::is_bound(CELL_ARABIC) { + return FontFamily::Name(CELL_ARABIC.into()); + } + } + cell_family(name, bold, italic) +} + +/// [`FontId`] for a single line of interface text (formula bar, in-cell editor), in the Arabic +/// script-leading family when the line is Arabic-only. +pub fn font_for_text(font: FontId, text: &str) -> FontId { + if is_arabic_only(text) && crate::fonts::is_bound(CELL_ARABIC) { FontId::new(font.size, FontFamily::Name(CELL_ARABIC.into())) } else { font } +} + +/// Whether `text` is made only of Arabic-script letters, Arabic punctuation/digits and neutral +/// separators. Mixed Latin/Arabic text is deliberately not redirected: egui cannot reorder its +/// runs, and forcing the whole line into an Arabic face would change the Latin glyph shapes too. +fn is_arabic_only(text: &str) -> bool { + let mut arabic = false; + for ch in text.chars() { + if ch.is_whitespace() || ch.is_ascii_digit() || ch.is_ascii_punctuation() { + continue; + } + let c = ch as u32; + let neutral = matches!(c, 0x200C..=0x200F | 0x202A..=0x202E | 0x2066..=0x2069 | 0x00AB | 0x00BB | 0x061C); + if neutral { + continue; + } + if (0x0600..=0x06FF).contains(&c) + || (0x0750..=0x077F).contains(&c) + || (0x08A0..=0x08FF).contains(&c) + || (0xFB50..=0xFDFF).contains(&c) + || (0xFE70..=0xFEFF).contains(&c) + { + arabic = true; + } else { + return false; + } + } + arabic +} + /// Width in points, at 100% zoom, of one line of cell text drawn as the grid draws it: in the -/// face [`cell_family`] picks, which on a machine without the named font is a wider or narrower -/// substitute (#1316). A family the active font set doesn't have yet (bound fonts apply from the +/// face [`cell_family_for_text`] picks, which on a machine without the named font is a wider or +/// narrower substitute (#1316). A family the active font set doesn't have yet (bound fonts apply from the /// next frame) falls back to the engine's estimate rather than letting egui panic on it. /// /// `exact` lays the text out (a galley kept until the end of the frame); otherwise the glyph /// advances are summed without kerning, which is cheap and never narrower. This takes the /// Context lock (`fonts_mut`): never call it, or run a command, while an egui lock is held. pub fn cell_text_width(ctx: &egui::Context, text: &str, font: &gridcraft_engine::model::Font, exact: bool) -> f32 { - let family = cell_family(&font.name, font.bold, font.italic); + let family = cell_family_for_text(&font.name, font.bold, font.italic, text); let id = FontId::new(font.size * 96.0 / 72.0, family); ctx.fonts_mut(|f| { f.families().contains(&id.family).then(|| { @@ -464,6 +519,23 @@ fn font_definitions_impl(language: crate::i18n::Language, used: &[String], syste // Chinese comes from the native-only OFL LXGW face when embedded; a system Han face still // leads, and the barred Noto CJK collection never enters. append_craft_faces(&mut fonts); + // An Arabic-first chain for Arabic-only cell text. A system Arabic face is preferred, then the + // permitted craft-fonts Arabic face, so the same rule works on the web and on hosts without a + // system Arabic font. Without either, the role families behave as before. + let craft = craft_font_faces(); + let arabic = fallback.iter().find(|k| k.as_str() == "sys-arabic").cloned(); + let arabic = arabic.or_else(|| craft.iter().find(|(_, f)| f.scripts.contains(&"Arab")).map(|(k, _)| k.clone())); + if let Some(arabic) = arabic { + let mut chain: Vec = Vec::new(); + for key in std::iter::once(&arabic).chain(base_prop.iter()).chain(fallback.iter()).chain(craft.iter().map(|(k, _)| k)) { + if !chain.contains(key) { + chain.push(key.clone()); + } + } + chain.retain(|k| fonts.font_data.contains_key(k)); + fonts.families.insert(FontFamily::Name(CELL_ARABIC.into()), chain); + bound.push(CELL_ARABIC.to_string()); + } drop_unparseable_fonts(&mut fonts); // Only families egui still has data for count as bound (a dropped face can empty a chain). bound.retain(|f| fonts.families.get(&FontFamily::Name(f.as_str().into())).is_some_and(|c| !c.is_empty())); @@ -558,6 +630,18 @@ mod font_binding_tests { [UI, UI_BOLD, CELL, CELL_BOLD, CELL_ITALIC, CELL_BOLD_ITALIC, SERIF, MONO].iter().map(|s| s.to_string()).collect() } + #[test] + fn arabic_only_cell_text_uses_the_arabic_family_when_it_is_bound() { + let _g = crate::fonts::TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner()); + crate::fonts::set_bound(always_bound().into_iter().chain([CELL_ARABIC.to_string()])); + assert_eq!(cell_family_for_text("Calibri", false, false, "سلام دنیا"), FontFamily::Name(CELL_ARABIC.into())); + assert_eq!(cell_family_for_text("Calibri", false, false, "سلام."), FontFamily::Name(CELL_ARABIC.into())); + assert_eq!(font_for_text(ui_font(14.0), "سلام دنیا").family, FontFamily::Name(CELL_ARABIC.into())); + // Mixed text and Latin-only text keep their ordinary role (egui cannot reorder mixed runs). + assert_eq!(cell_family_for_text("Calibri", false, false, "GridCraft فارسی"), FontFamily::Name(CELL.into())); + assert_eq!(font_for_text(ui_font(14.0), "GridCraft"), ui_font(14.0)); + } + #[test] fn a_name_that_is_not_an_installed_font_gets_a_built_in_role() { let _g = crate::fonts::TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner()); diff --git a/crates/ui-egui/src/theme/tests.rs b/crates/ui-egui/src/theme/tests.rs --- a/crates/ui-egui/src/theme/tests.rs +++ b/crates/ui-egui/src/theme/tests.rs @@ -121,11 +121,11 @@ fn language_reorders_only_the_han_faces() { let keys = |han| fallback_faces(han).into_iter().map(|(k, _)| k).collect::>(); assert_eq!( keys(HanOrder::Japanese), - ["sys-cjk-han-ja", "sys-cjk-han-zh", "sys-cjk-hangul", "sys-thai", "sys-lao", "sys-hebrew", "sys-symbols", "sys-symbols-2"] + ["sys-cjk-han-ja", "sys-cjk-han-zh", "sys-cjk-hangul", "sys-thai", "sys-lao", "sys-arabic", "sys-hebrew", "sys-symbols", "sys-symbols-2"] ); assert_eq!( keys(HanOrder::Chinese), - ["sys-cjk-han-zh", "sys-cjk-han-ja", "sys-cjk-hangul", "sys-thai", "sys-lao", "sys-hebrew", "sys-symbols", "sys-symbols-2"] + ["sys-cjk-han-zh", "sys-cjk-han-ja", "sys-cjk-hangul", "sys-thai", "sys-lao", "sys-arabic", "sys-hebrew", "sys-symbols", "sys-symbols-2"] ); assert_eq!(HanOrder::of(Language::Ja), HanOrder::Japanese); for l in [Language::En, Language::Zh, Language::Ko, Language::Ru, Language::PtBr, Language::De, Language::Cs, Language::Fr, Language::Tr] { @@ -183,7 +183,7 @@ fn fallback_covers_its_script() { // have a glyph for every non-space character in the sample. A script runs when a system // face or the optional craft-fonts input provides one; the printed skip is a recorded // gate, and a present face that is not wired in fails. - let cases: [(&str, Vec<(String, u32)>, &str); 10] = [ + let cases: [(&str, Vec<(String, u32)>, &str); 11] = [ ("sys-cjk-han-ja", cjk_han_japanese(), "カテゴリ"), // Chinese uses a system Han face or the native-only permitted LXGW craft face; the // barred Noto CJK collection never supplies it. @@ -191,6 +191,7 @@ fn fallback_covers_its_script() { ("sys-cjk-hangul", cjk_hangul(), "한글"), ("sys-thai", thai(), "ทดสอบ"), ("sys-lao", lao(), "ພາສາລາວ"), + ("sys-arabic", arabic(), "مرحبا"), ("sys-hebrew", hebrew(), "עברית"), ("craft-noto-sans-arabic", Vec::new(), "مرحبا"), ("craft-noto-sans-georgian", Vec::new(), "ქართული"), diff --git a/crates/ui-egui/tests/persian.rs b/crates/ui-egui/tests/persian.rs new file mode 100644 --- /dev/null +++ b/crates/ui-egui/tests/persian.rs @@ -0,0 +1,231 @@ +//! Persian and Arabic (right-to-left) text must render with real glyphs, not tofu boxes, in both +//! the interface and the cell fonts, and a workbook must keep it through an XLSX round-trip. +//! +//! A separate test binary: it runs the process-wide installed-font scan, whose faces bind from the +//! next frame and would leak into the other integration tests (see `it/main.rs`). + +use egui::{Color32, FontFamily, FontId}; +use gridcraft_engine::Session; +use gridcraft_engine::core::{CellRef, Value}; +use gridcraft_ui_egui::{fonts, theme}; +use serde_json::json; +use std::sync::{Mutex, MutexGuard}; + +/// `font_definitions*` records the bound role families process-wide. Tests in this binary build +/// different font sets, so serialize them; the app has one font set and never hits this race. +static FONT_LOCK: Mutex<()> = Mutex::new(()); + +fn lock_fonts() -> MutexGuard<'static, ()> { + FONT_LOCK.lock().unwrap_or_else(|e| e.into_inner()) +} + +/// One sample per goal: a Persian phrase, mixed Persian/English, Arabic with punctuation, and a +/// Persian sentence holding a Latin URL and Latin digits (all common in real sheets). +const PERSIAN: &str = "سلام دنیا"; +const MIXED: &str = "GridCraft فارسی 123"; +const ARABIC_PUNCT: &str = "مرحبا، كيف حالك؟ (١٢٣)"; +const PERSIAN_URL: &str = "وب‌سایت https://example.com؟ 42"; + +/// A context with the same font set the app installs, built after the scan so dynamically-found +/// faces (not just the hardcoded paths) are bound. +fn context() -> egui::Context { + let _ = fonts::catalog(); // finish the scan before building the font set + let ctx = egui::Context::default(); + ctx.set_fonts(theme::font_definitions()); + let mut out = ctx.run_ui(egui::RawInput::default(), |_ui| {}); + out.textures_delta.clear(); // nothing consumes the atlas in a headless test + ctx +} + +/// Every drawn glyph of `text` in `family`. +fn drawn(ctx: &egui::Context, family: &FontFamily, text: &str) -> Vec { + let font = FontId::new(14.0, family.clone()); + let galley = ctx.fonts_mut(|f| f.layout_no_wrap(text.to_owned(), font, Color32::BLACK)); + galley.rows.iter().flat_map(|r| r.glyphs.iter().copied()).collect() +} + +/// Whether the font set has an Arabic-script face for the fallback chain: a system face or the +/// permitted craft-fonts Arabic face. Without one there is nothing to shape with, and the tests +/// skip instead of failing a minimal install (the CI gate builds craft fonts). +fn arabic_face_loaded() -> bool { + let fonts = theme::font_definitions(); + fonts.font_data.contains_key("sys-arabic") || fonts.families.contains_key(&FontFamily::Name(theme::CELL_ARABIC.into())) +} + +/// The characters of `text` that have no glyph in `family`. Fonts may draw a missing character +/// as a zero-width component (egui 0.37's replacement glyph), so ask the font chain directly. +fn tofu_chars(ctx: &egui::Context, family: &FontFamily, text: &str) -> String { + let font = FontId::new(14.0, family.clone()); + ctx.fonts_mut(|f| text.chars().filter(|ch| *ch != ' ' && !f.has_glyph(&font, *ch)).collect()) +} + +#[test] +fn persian_renders_real_glyphs_in_the_interface_and_cell_fonts() { + let _g = lock_fonts(); + let ctx = context(); + if !arabic_face_loaded() { + eprintln!("skipped: no Arabic-script font installed"); + return; + } + if !fonts::faces_of("Vazirmatn").is_empty() || !fonts::faces_of("Noto Sans Arabic").is_empty() { + // The scan found an Arabic-script face; the fallback must have loaded one of them. + assert!( + theme::font_definitions().font_data.contains_key("sys-arabic"), + "an Arabic font is installed but no Arabic-script fallback face was bound" + ); + } + for family in [FontFamily::Proportional, FontFamily::Name(theme::CELL.into())] { + for sample in [PERSIAN, MIXED, ARABIC_PUNCT, PERSIAN_URL] { + let missing = tofu_chars(&ctx, &family, sample); + assert!(missing.is_empty(), "{sample:?} renders as boxes in {family:?}: {missing:?}"); + } + } +} + +#[test] +fn bold_and_italic_cell_roles_still_have_arabic() { + let _g = lock_fonts(); + if !arabic_face_loaded() { + eprintln!("skipped: no Arabic-script font installed"); + return; + } + // Bold cells use a different role chain; it must carry `sys-arabic` too, or a bold Persian cell + // would fall to tofu even though the regular one renders. + let ctx = context(); + for role in [theme::CELL_BOLD, theme::CELL_ITALIC, theme::CELL_BOLD_ITALIC] { + let family = FontFamily::Name(role.into()); + let missing = tofu_chars(&ctx, &family, PERSIAN); + assert!(missing.is_empty(), "bold/italic {role} renders Persian as boxes: {missing:?}"); + } +} + +#[test] +fn mixed_latin_and_persian_keeps_the_persian_shaped_and_the_latin_readable() { + let _g = lock_fonts(); + let ctx = context(); + if !arabic_face_loaded() { + eprintln!("skipped: no Arabic-script font installed"); + return; + } + let family = FontFamily::Name(theme::CELL.into()); + // Every character (Latin, Persian, digits, space) has a real glyph. + assert!(tofu_chars(&ctx, &family, MIXED).is_empty()); + // The galley is wider than its Latin part alone, so the Persian is actually laid out. + let width = |text: &str| ctx.fonts_mut(|f| f.layout_no_wrap(text.to_owned(), FontId::new(14.0, family.clone()), Color32::BLACK)).size().x; + let (full, latin) = (width(MIXED), width("GridCraft 123")); + assert!(full > latin, "mixed text ({full}) must be wider than its Latin part ({latin})"); +} + +/// Within one shaping run, HarfRust lays a right-to-left script's glyphs out in visual order: the +/// last logical character comes first, drawn at the left. That is what makes the letters join and +/// read right to left. epaint has no bidi reordering *between* runs (`TODO(emilk): heed bidi +/// characters`): a space can be owned by a different face, splitting a sentence into one run per +/// word, and those runs are concatenated in logical order. So this checks a single word only; the +/// limitation is recorded in `docs/localization-parity.md`. +#[test] +fn a_right_to_left_word_is_shaped_in_visual_order() { + let _g = lock_fonts(); + let ctx = context(); + if !arabic_face_loaded() { + eprintln!("skipped: no Arabic-script font installed"); + return; + } + let family = FontFamily::Name(theme::CELL.into()); + let glyphs = drawn(&ctx, &family, "اب"); + let at = |ch: char| glyphs.iter().find(|g| g.chr == ch && g.advance_width > 0.0).map(|g| g.pos.x); + let (Some(alef), Some(beh)) = (at('ا'), at('ب')) else { + panic!("both letters should be shaped: {glyphs:?}"); + }; + assert!(beh < alef, "the last logical letter (ب) must be left of the first (ا), got ا={alef}, ب={beh}"); +} + +/// A Persian sentence is two right-to-left runs if a space belongs to a Latin face. The Arabic cell +/// family leads with the script face instead, so the whole sentence is one run and its words are in +/// reading order. (Mixed Latin/Arabic still has no bidi; see `docs/localization-parity.md`.) +#[test] +fn a_persian_sentence_keeps_its_word_order_in_the_arabic_cell_family() { + let _g = lock_fonts(); + let ctx = context(); + if !arabic_face_loaded() || !fonts::is_bound(theme::CELL_ARABIC) { + eprintln!("skipped: no Arabic-script font installed"); + return; + } + assert_eq!( + theme::cell_family_for_text("No Such Font", false, false, PERSIAN), + FontFamily::Name(theme::CELL_ARABIC.into()), + "a font without Arabic must use the script-leading family" + ); + let glyphs = drawn(&ctx, &FontFamily::Name(theme::CELL_ARABIC.into()), PERSIAN); + let at = |ch: char| glyphs.iter().find(|g| g.chr == ch && g.advance_width > 0.0).map(|g| g.pos.x); + let (first_word, second_word) = (at('س'), at('د')); + let (Some(salam), Some(donya)) = (first_word, second_word) else { + panic!("both words should be shaped: {glyphs:?}"); + }; + assert!(donya < salam, "دنیا must be left of سلام in a right-to-left sentence"); +} + +#[test] +fn an_installed_arabic_font_keeps_its_own_family_for_arabic_cells() { + let _g = lock_fonts(); + let _ = fonts::catalog(); + let installed = ["Vazirmatn", "Noto Naskh Arabic", "Noto Sans Arabic"].into_iter().find(|name| !fonts::faces_of(name).is_empty()); + let Some(name) = installed else { + eprintln!("skipped: no named Arabic-script font installed"); + return; + }; + let ctx = egui::Context::default(); + ctx.set_fonts(theme::font_definitions_with(gridcraft_ui_egui::i18n::Language::En, &[name.to_string()])); + let mut out = ctx.run_ui(egui::RawInput::default(), |_| {}); + out.textures_delta.clear(); + let family = theme::cell_family_for_text(name, false, false, PERSIAN); + let FontFamily::Name(role) = &family else { panic!("an installed font must name its own family, got {family:?}") }; + assert!(fonts::is_bound(role), "the chosen Arabic font's role must be bound"); + assert_ne!(&**role, theme::CELL_ARABIC, "a font with Arabic coverage keeps its own typeface"); + assert!(tofu_chars(&ctx, &family, PERSIAN).is_empty(), "the chosen font {name} must draw Persian"); +} + +#[test] +fn a_workbook_keeps_persian_and_its_font_through_an_xlsx_round_trip() { + let _g = lock_fonts(); + let _ = fonts::catalog(); + if !arabic_face_loaded() { + eprintln!("skipped: no Arabic-script font installed"); + return; + } + let mut session = Session::new(); + session.new_workbook(); + session.execute("range.setValues", json!({"range": "A1:A2", "values": [[PERSIAN], [MIXED]]})).expect("set values"); + // A Persian font selected for the cell, as a user would, when it is installed. + let vazirmatn = !fonts::faces_of("Vazirmatn").is_empty(); + if vazirmatn { + session.execute("home.fontName", json!({"range": "A1", "name": "Vazirmatn"})).expect("set font"); + } + let saved = session.execute("file.saveBytes", json!({"format": "xlsx"})).expect("save"); + let base64 = saved.get("base64").and_then(|b| b.as_str()).expect("base64"); + let name = saved.get("name").and_then(|n| n.as_str()).unwrap_or("Book.xlsx"); + session.execute("file.open", json!({"name": name, "base64": base64})).expect("reopen"); + + let (font, family_name) = { + let doc = session.active().expect("document"); + let sh = doc.wb.active().expect("sheet"); + assert_eq!(sh.value(CellRef::new(0, 0)), Value::Text(PERSIAN.into()), "A1 survives the round-trip"); + assert_eq!(sh.value(CellRef::new(1, 0)), Value::Text(MIXED.into()), "A2 survives the round-trip"); + let st = doc.wb.styles.get(sh.style_id(CellRef::new(0, 0))); + (st.font.clone(), st.font.name.clone()) + }; + if vazirmatn { + assert_eq!(font.name, "Vazirmatn", "the cell font survives the round-trip"); + } + // Build the reopened workbook's font set and render the cell text in its own family. + let used = vec![font.name.clone()]; + let ctx = egui::Context::default(); + ctx.set_fonts(theme::font_definitions_with(gridcraft_ui_egui::i18n::Language::En, &used)); + let mut out = ctx.run_ui(egui::RawInput::default(), |_| {}); + out.textures_delta.clear(); + let family = theme::cell_family(&family_name, font.bold, font.italic); + if let FontFamily::Name(role) = &family { + assert!(fonts::is_bound(role), "the cell font role {role:?} must be bound to font data"); + } + let missing = tofu_chars(&ctx, &family, PERSIAN); + assert!(missing.is_empty(), "the reopened cell's Persian renders as boxes: {missing:?}"); +} diff --git a/docs/gaps.md b/docs/gaps.md --- a/docs/gaps.md +++ b/docs/gaps.md @@ -1,6 +1,6 @@ # Gaps -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-24 · **Change:** minor (§8: PivotTable Enable show details; earlier: PivotTable Show Details) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-27 · **Change:** minor (§5: fontconfig discovery, Arabic/Persian fallback and script-only word order; bidi remains) · **Target:** Microsoft Excel (Microsoft 365) Every known shortfall against Excel, one entry each, ranked by user impact (what blocks someone using GridCraft for real work first). This is the work list: pick from the top, check the open @@ -53,10 +53,15 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha- - **Estimate:** 25–35 h for beta (40–60 h full). [chart-parity.md](chart-parity.md) ### 5. Text in non-Latin scripts doesn't render or export (L) -- **Missing:** CJK font fallback, complex shaping (Arabic, Devanagari, Thai), bidi, non-Latin PDF. -- **Evidence:** system-font fallback for CJK, Thai and Hebrew landed in #38 (closed #32); #142 - (Chinese text) awaits confirmation; #54 (symbols as boxes on Fedora); no shaping for Arabic or - Devanagari; PDF writer uses 14 standard fonts only. +- **Missing:** bidi (mixed and wrapped right-to-left runs, RTL alignment), Devanagari/Thai shaping, + non-Latin PDF. Arabic/Persian shaping and font fallback now work. +- **Evidence:** system-font fallback for CJK, Thai, Hebrew and Arabic landed (#38 and the 2026-10-27 + Arabic work); the scan reads fontconfig ``/`` entries, so installed Vazirmatn, Noto + Naskh Arabic, Shabnam or Sahel faces are found outside the standard paths; HarfRust shapes and + joins Arabic/Persian, and an Arabic-only cell leads with its script face so its words stay in + reading order. Mixed Latin/Arabic runs, wrapped RTL lines and RTL alignment still use logical + order; #142 (Chinese text) awaits confirmation; #54 (symbols as boxes on Fedora); PDF writer + uses 14 standard fonts only. - **Impact:** blocks most of the world's users even with an English UI. - **Estimate:** 12–20 h. [localization-parity.md](localization-parity.md) @@ -183,6 +188,7 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha- | Date | Change | Summary | |---|---|---| +| 2026-10-27 | minor | §5: fontconfig ``/`` discovery finds Arabic/Persian faces outside fixed paths; the script fallback shapes and joins Arabic-only cell text in reading order. Bidi, mixed Latin/Arabic order, wrapped RTL lines and RTL alignment remain | | 2026-10-24 | minor | §8: PivotTable Enable show details: `pivot.options` `enableShowDetails` turns Show Details off for a PivotTable, read and written in XLSX as `enableDrill` | | 2026-10-23 | minor | §8: PivotTable Show Details: double-clicking a value cell lists its source records on a new sheet (#2364) | | 2026-10-22 | minor | §20: a native window per workbook window, New Window on the same workbook and Arrange All on the desktop; left: Arrange All across monitors, the web's one window, pen pressure (estimate 10–14 h → 5–8 h) | diff --git a/docs/localization-parity.md b/docs/localization-parity.md --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-23 · **Change:** minor (missing translations are listed by `cargo xtask i18n-report` and never fail a check) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-27 · **Change:** minor (Linux font discovery follows fontconfig; Arabic/Persian fallback finds installed faces and shapes a script-only cell in reading order; bidi and mixed-run order remain) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Per-language status against Excel, which ships its UI in dozens of languages (Windows language packs cover 100+), with localized function names, argument separators, number and date formats, @@ -44,10 +44,16 @@ right-to-left sheets and full complex-script support. Part of - **Script support.** System-font fallback for CJK, Thai and Hebrew landed in #38 (closes #32; #142 is still open pending confirmation). Native CJK fallback also probes Flatpak's read-only `/run/host/fonts`, `/run/host/local-fonts` and `/run/host/user-fonts` layouts, so installed - Noto CJK and WenQuanYi faces can render Chinese without a broad filesystem grant. This does - not change the installed-font catalogue or add bundled fonts. Lao falls back to Lao Sangam MN / Lao MN (macOS), - Leelawadee UI / DokChampa (Windows) or Noto Sans Lao / Phetsarath OT (Linux). There is no complex-text shaping: Arabic and - Devanagari don't join or reorder, and there is no bidirectional layout. **Sheet Right-to-Left** + Noto CJK and WenQuanYi faces can render Chinese without a broad filesystem grant; that probe + adds no bundled fonts. The general font scan now also follows fontconfig's ``/`` + configuration, so the installed-font catalogue includes folders fontconfig names. Lao falls back to Lao Sangam MN / Lao MN (macOS), + Leelawadee UI / DokChampa (Windows) or Noto Sans Lao / Phetsarath OT (Linux). Arabic-script + fonts (Vazirmatn, Noto Sans/Naskh/Kufi Arabic, Shabnam, Sahel, Amiri…) are found by family + wherever the font scan reached, including folders only fontconfig names; HarfRust shapes and + joins Arabic/Persian, and an Arabic-script-only cell whose chosen font lacks Arabic leads with + that script face so its words stay in reading order. There is still no bidirectional layout: + mixed Latin/Arabic runs, wrapping and Excel's right-to-left alignment are not reordered, and + Devanagari/Thai shaping is not covered. **Sheet Right-to-Left** (`view.rightToLeft`, Page Layout › Sheet Options, XLSX `rightToLeft`, `.xls` WINDOW2 fRightToLeft) mirrors the grid (#146): column A and the row headers on the right, the vertical scrollbar on the left; hit-testing, arrow keys, editor, popups and floating objects follow; text stays left-to-right inside its box @@ -72,7 +78,7 @@ separators, number/date formats, machine translation with agent review, and scri | Simplified Chinese (Mandarin) | zh-Hans | 266 (~12%) | no | CJK system-font fallback (#38); IME unverified; PDF can't export | no | menus only | 8–13 h | | Spanish | es | 0 (0%) | no | Latin: yes | no | none | 6–10 h | | Hindi | hi | 0 (0%) | no | Devanagari: **no shaping** | no | none | 12–18 h | -| Arabic | ar | 0 (0%) | no | **no shaping, no RTL UI**; Sheet Right-to-Left mirrors the grid (#146) | no | none | 25–35 h | +| Arabic | ar | 0 (0%) | no | Arabic/Persian shaping and script-only word order; **no bidi/mixed-run order, no RTL UI**; Sheet Right-to-Left mirrors the grid (#146) | no | none | 25–35 h | | French | fr | 0 (0%) | no | Latin: yes | no | none | 6–10 h | | Portuguese | pt-BR | 943 (~43%) | dialogs and tooltips (dialogs take effect with #669) | Latin: yes | no | menus, File view, dialogs | 2–4 h | | Indonesian | id | 0 (0%) | no | Latin: yes | no | none | 5–8 h | @@ -102,14 +108,15 @@ language (none so far) and of localized function names. 2. Locale-aware formulas: localized function-name tables and separators at the formula-bar boundary only (files keep en-US invariant names), locale number and date formats, list separators in CSV. -3. Text shaping and fallback: a shaping engine for grid, editor and PDF text (Arabic, Devanagari, - Thai), system font fallback for CJK, bidirectional runs, right-to-left sheet tabs and printing; - embedded subset fonts in PDF. +3. Text shaping and fallback: Arabic-script shaping and font fallback exist; add bidi (mixed and + wrapped RTL runs), Devanagari/Thai shaping, PDF text (embedded subset fonts), right-to-left + sheet tabs and printing. ## Revision history | Date | Change | Summary | |---|---|---| +| 2026-10-27 | minor | Linux font discovery reads fontconfig's `` and `` entries, so an Arabic/Persian font is found wherever it is installed (not only at one distribution path). The Arabic-script fallback (Vazirmatn, Noto Sans/Naskh/Kufi Arabic, Shabnam, Sahel, Amiri…) renders and joins Persian/Arabic; an Arabic-only cell whose chosen font lacks Arabic uses a script-leading family, keeping “سلام دنیا” in reading order where a Latin-owned space used to split it into separately ordered runs. Bidi, mixed Latin/Arabic order, wrapped RTL lines and RTL alignment remain unsupported and are documented | | 2026-10-23 | minor | Missing translations are reported, never a failure: `cargo xtask i18n-report` prints, per language, the strings other catalogs translate and it lacks (plus Spanish function help rows keyed by English the function catalog no longer has) and always passes; `cargo xtask ci` runs it first. Coverage and per-language parity tests and tests of one language's exact wording are gone; UI tests take the expected text from the catalog. Duplicate-key, placeholder, context-prefix and line-shape checks stay | | 2026-10-21 | minor | Romanian interface (`ro`, Română): 970 labels covering every key of the other catalogs (`crates/ui-egui/src/i18n/ro.rs`), `ro-*` system locales, the View-tab switcher and `app.language.set`; context keys show no prefix (File › Options test). English formulas and formats; AI-assisted draft pending native-speaker review | | 2026-10-20 | minor | Japanese: furigana (`rPh` phonetic runs and `` in `xl/sharedStrings.xml`) is no longer deleted by an open + save; it is kept on the workbook and written back (a `` without runs is kept too, while duplicate texts with different readings keep only the first, reported on open). Display of the readings is still missing (`crates/xlsx/src/tests/phonetic_tests.rs`) |