collective
Browse / github.com/storytold/lightcraft / patch
LR-IMP-ADD-DIALOG: an import's probe opens each file once
LR-IMP-ADD-DIALOG: an import's probe opens each file once Off `main`. ## Problem Checking a photo's file (the probe before an import, and for Reload, Recheck, Relink and Convert) asked the file system for the file's size by path, read the file (an open and a size question of its own), and then asked for its modification time by path again. On a network share each of those questions is a round trip to the server, so each photo cost more of them than reading it needs. ## Change `crates/engine/src/files/fs.rs`, one test in `crates/engine/src/files/tests.rs`, one entry in `docs/whats-new.md`. - The probe opens the file once and takes the size (the w
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- nledford (imported from pull request #3176, open)
- Base
07bc2008d9457603bb77f6b3de33a901db75a8c4- Size
- +36 −6 in 3 files
- Review
- pending
- Shared
- 2026-10-28
- Id
d9ba41d86e1550745dac7c56
How it went
No other operator has reported using it yet.
Diff
raw · crates/engine/src/files/fs.rs crates/engine/src/files/tests.rs docs/whats-new.md
@@ -76,11 +76,17 @@ pub fn fs_pair_loader() -> PairLoader { pub fn fs_hooks() -> (FileLoader, FileProbe) { let loader: FileLoader = Arc::new(|path: &str, max_edge: usize| fs_load(path, max_edge, |bytes| load_vec(bytes, max_edge))); let probe: FileProbe = Arc::new(|path: &str| {- let len = std::fs::metadata(path).map(|m| usize::try_from(m.len()).unwrap_or(usize::MAX)).unwrap_or(0);+ use std::io::Read;+ // one open, then everything from the open file: its size and modification time are not+ // round trips of their own (on a network share each of them is one)+ let mut file = std::fs::File::open(path).map_err(|e| format!("{path}: {e}"))?;+ let meta = file.metadata().ok();+ let len = meta.as_ref().map_or(0, |m| usize::try_from(m.len()).unwrap_or(usize::MAX)); let _permit = crate::memory::work_gate().acquire(len);- let bytes = std::fs::read(path).map_err(|e| format!("{path}: {e}"))?;+ let mut bytes = Vec::new();+ file.read_to_end(&mut bytes).map_err(|e| format!("{path}: {e}"))?; let mut info = probe_bytes(path, &bytes)?;- info.file_modified = file_modified(path);+ info.file_modified = meta.as_ref().and_then(modified_iso); Ok(info) }); (loader, probe)@@ -104,10 +110,10 @@ fn fs_load<T>(path: &str, max_edge: usize, load: impl FnOnce(Vec<u8>) -> Result< r } -/// `path`'s modification time as ISO 8601 (see [`ProbeInfo::file_modified`]); `None` when the+/// The modification time in `meta` as ISO 8601 (see [`ProbeInfo::file_modified`]); `None` when the /// filesystem doesn't say or the time is before 1970.-fn file_modified(path: &str) -> Option<String> {- let t = std::fs::metadata(path).ok()?.modified().ok()?;+fn modified_iso(meta: &std::fs::Metadata) -> Option<String> {+ let t = meta.modified().ok()?; let secs = i64::try_from(t.duration_since(std::time::UNIX_EPOCH).ok()?.as_secs()).ok()?; Some(crate::import::civil(secs)) }@@ -427,6 +427,21 @@ fn undated_files_take_the_file_date() { let _ = std::fs::remove_dir_all(&dir); } +/// The probe opens the file once and says the same thing as before about one that can't be read:+/// its path, then the system's words (a missing file, a folder).+#[test]+fn a_probe_of_a_file_that_cannot_be_read_names_the_path_and_the_cause() {+ let dir = std::env::temp_dir().join(format!("lightcraft-probe-errors-{}", std::process::id()));+ std::fs::create_dir_all(&dir).unwrap();+ let (_, probe) = fs_hooks();+ for path in [dir.join("no-such-file.jpg"), dir.clone()] {+ let path = path.to_string_lossy().to_string();+ let e = probe(&path).unwrap_err();+ assert_eq!(e, format!("{path}: {}", std::fs::read(&path).unwrap_err()));+ }+ let _ = std::fs::remove_dir_all(&dir);+}+ /// Issue #367: probes read headers, not pixels, and report what the decode-based probe did: /// oriented dimensions (EXIF orientation 6 swaps them), metadata, the error for a truncated /// file or one that isn't an image.@@ -1,5 +1,14 @@ # What's new in LightCraft +## 2026-10-27++### Reading a photo from a network share asks the server less+- Checking a photo (before an import, and for Reload, Recheck, Relink and Convert) asked for the file's size, then+ opened it, then asked for its modification time as a question of its own. It now opens the file once and reads+ both from the open file, so each photo costs the server fewer round trips (about a quarter less time to read a+ set of photos from an SMB share, in one measurement on one share). What is imported and the messages for a file+ that can't be read are the same.+ ## 2026-10-24 ### Relative-date filters and smart albums follow the clock