collective
Browse / github.com/storytold/lightcraft / patch
AI masking with SAM 3, startup model pre-loading & Sony a6400 lens support
AI masking with SAM 3, startup model pre-loading & Sony a6400 lens support ### Summary of Changes - **AI Masking with Segment Anything (SAM 3)**: - Added SAM 3 automatic segmentation for Subject, Sky, and Background mask selections when installed, with Edge slider support and zoomed-in detail refinement passes. - Accelerated SAM 3 concept detection by skipping the 288x288 pixel decoder when query dot products fall below threshold, with fallback prompt groups. - Implemented cooperative cancellation (`Error::Cancelled`) across ViT, detector, and worker threads so removing or undoing an in-flight mask immediately halts background execution
- Status
- open
- Score
- 0.00
- Adopters
- 0 other operators
- Author
- LeoCanta42 (imported from pull request #3109, open)
- Base
07bc2008d9457603bb77f6b3de33a901db75a8c4- Size
- +1428 −173 in 58 files
- Review
- pending
- Shared
- 2026-10-28
- Id
88ad1fd36d9e3207e7d19bd0
How it went
No other operator has reported using it yet.
Diff
raw · Cargo.toml apps/lightcraft/src/main.rs crates/develop/src/segmask.rs crates/develop/src/settings.rs crates/engine/src/cmd.rs crates/engine/src/cmd/masks.rs crates/engine/src/crs_masks/tests.rs crates/engine/src/denoise/session.rs crates/engine/src/files/tests.rs crates/engine/src/lens_profiles.rs crates/engine/src/lib.rs crates/engine/src/media.rs crates/engine/src/media/cache.rs crates/engine/src/media/session.rs crates/engine/src/media/source.rs crates/engine/src/media/tests.rs crates/engine/src/segment.rs crates/engine/src/segment/worker.rs crates/engine/src/tests/import.rs crates/engine/src/tests/segment.rs crates/engine/src/tests/spots.rs crates/gpu/tests/it/equivalence.rs crates/meta/src/focus.rs crates/meta/src/focus/tests.rs crates/pipeline/src/masks.rs crates/pipeline/src/masks/tests.rs crates/pipeline/src/tests_window.rs crates/raw/src/lens_metadata.rs crates/raw/src/vendor.rs crates/raw/src/vendor/arw_lens.rs crates/segment/src/detector.rs crates/segment/src/lib.rs crates/segment/src/remote.rs crates/segment/src/vit.rs crates/ui-egui/locales/cs.json crates/ui-egui/locales/de.json crates/ui-egui/locales/es.json crates/ui-egui/locales/fi.json crates/ui-egui/locales/fr.json crates/ui-egui/locales/id.json crates/ui-egui/locales/it.json crates/ui-egui/locales/ja.json crates/ui-egui/locales/ka.json crates/ui-egui/locales/pl.json crates/ui-egui/locales/pt-br.json crates/ui-egui/locales/ru.json crates/ui-egui/locales/sv.json crates/ui-egui/locales/uk.json crates/ui-egui/locales/vi.json crates/ui-egui/locales/zh-hans.json crates/ui-egui/locales/zh-hant.json crates/ui-egui/src/app.rs crates/ui-egui/src/panels/compare.rs crates/ui-egui/src/panels/edit.rs crates/ui-egui/src/panels/masking.rs crates/ui-egui/src/state.rs docs/gaps.md docs/parity-checklist.md
@@ -139,6 +139,8 @@ debug = "line-tables-only" [profile.release] debug = "line-tables-only"+lto = "thin"+codegen-units = 1 [profile.dist] inherits = "release"@@ -146,8 +148,15 @@ lto = "thin" codegen-units = 1 debug = false -# `cargo xtask web`: the browser build. Size matters more than build time: fat LTO, one codegen-# unit, no symbols, abort on panic (wasm32 can't unwind anyway), size-optimized code — except the+[profile.dev.package]+candle-core = { opt-level = 3 }+candle-nn = { opt-level = 3 }+gemm = { opt-level = 3 }+gemm-f32 = { opt-level = 3 }+gemm-common = { opt-level = 3 }+lightcraft-segment = { opt-level = 3 }+lightcraft-inpaint = { opt-level = 3 }+ # crates that do per-pixel work, which stay at opt-level 3 (opt-level "s"/"z" there costs ~50 %/ # ~130 % render time; measured in docs/web.md → Bundle size). [profile.web]@@ -264,6 +264,8 @@ fn main() -> eframe::Result { // (never required: without it, AI masks offer to download it; see docs/ai-masks.md) // and the user's own download locations, one base URL per line (LIGHTCRAFT_SAM3_MIRRORS too) session = session.with_default_sam3_model();+ session.segment_warmup();+ session.denoise_warmup(); // HDR display: SDR unless the surface, platform and display all allow HDR hdr_display::report(cc.wgpu_render_state.as_ref().map(|r| r.target_format)); let mut app = LightcraftApp::new(session, services(cc, app_log_file.as_deref()));@@ -53,6 +53,18 @@ impl SegMask { let q = miniz_oxide::inflate::decompress_to_vec_with_limit(&z, side * side).ok()?; (q.len() == side * side).then(|| q.iter().map(|b| f32::from(*b as i8) / SCALE).collect()) }++ /// Invert the mask: negates every logit, so selected becomes unselected and vice versa.+ pub fn invert(&self) -> Option<SegMask> {+ let mut logits = self.logits()?;+ for l in &mut logits {+ *l = -*l;+ }+ Some(match self.rect {+ Some(r) => SegMask::from_logits_in(self.side as usize, &logits, r),+ None => SegMask::from_logits(self.side as usize, &logits),+ })+ } } /// Most zoomed-in detail patches one AI mask component keeps (the app makes one; each costs a@@ -170,6 +182,20 @@ mod tests { assert_eq!(detail.len(), MAX_DETAIL); } + #[test]+ fn segmask_invert_negates_logits() {+ let side = 8;+ let l: Vec<f32> = (0..side * side).map(|i| (i as f32 - 32.0) / 4.0).collect();+ let m = SegMask::from_logits_in(side, &l, [0.1, 0.2, 0.8, 0.9]);+ let inv = m.invert().unwrap();+ assert_eq!(inv.rect, Some([0.1, 0.2, 0.8, 0.9]));+ let orig_logits = m.logits().unwrap();+ let inv_logits = inv.logits().unwrap();+ for (a, b) in orig_logits.iter().zip(&inv_logits) {+ assert!((a + b).abs() <= 1e-5, "expected negation: {a} vs {b}");+ }+ }+ #[test] fn damaged_data_is_none() { let mut m = SegMask::from_logits(8, &[1.0; 64]);@@ -959,9 +959,30 @@ pub enum MaskShape { feather: f64, }, /// AI / automatic selections, evaluated by a segmenter.- Subject,- Sky,- Background,+ Subject {+ #[serde(default, skip_serializing_if = "Option::is_none")]+ seg: Option<crate::SegMask>,+ #[serde(default, skip_serializing_if = "Vec::is_empty", deserialize_with = "crate::segmask::de_detail")]+ detail: Vec<crate::SegMask>,+ #[serde(default, skip_serializing_if = "is_zero")]+ edge: f64,+ },+ Sky {+ #[serde(default, skip_serializing_if = "Option::is_none")]+ seg: Option<crate::SegMask>,+ #[serde(default, skip_serializing_if = "Vec::is_empty", deserialize_with = "crate::segmask::de_detail")]+ detail: Vec<crate::SegMask>,+ #[serde(default, skip_serializing_if = "is_zero")]+ edge: f64,+ },+ Background {+ #[serde(default, skip_serializing_if = "Option::is_none")]+ seg: Option<crate::SegMask>,+ #[serde(default, skip_serializing_if = "Vec::is_empty", deserialize_with = "crate::segmask::de_detail")]+ detail: Vec<crate::SegMask>,+ #[serde(default, skip_serializing_if = "is_zero")]+ edge: f64,+ }, /// One object picked by clicks (SAM 3 point prompts): `hint` holds the clicks that include, /// `exclude` the ones that exclude; `seg` the segmentation computed from them. Object {@@ -999,6 +1020,35 @@ pub enum MaskShape { }, } +impl MaskShape {+ /// Create an automatic Subject mask shape with default empty segmentation.+ pub fn subject() -> Self {+ MaskShape::Subject { seg: None, detail: Vec::new(), edge: 0.0 }+ }++ /// Create an automatic Sky mask shape with default empty segmentation.+ pub fn sky() -> Self {+ MaskShape::Sky { seg: None, detail: Vec::new(), edge: 0.0 }+ }++ /// Create an automatic Background mask shape with default empty segmentation.+ pub fn background() -> Self {+ MaskShape::Background { seg: None, detail: Vec::new(), edge: 0.0 }+ }++ /// The segmentation mask data for shapes that carry one (Object, Describe/Prompt, Subject, Sky, Background).+ pub fn seg(&self) -> Option<&crate::SegMask> {+ match self {+ MaskShape::Object { seg, .. }+ | MaskShape::Prompt { seg, .. }+ | MaskShape::Subject { seg, .. }+ | MaskShape::Sky { seg, .. }+ | MaskShape::Background { seg, .. } => seg.as_ref(),+ _ => None,+ }+ }+}+ #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] pub struct MaskComponent { /// A name given in the Masking panel (Rename); `None` shows the shape's kind.@@ -28,6 +28,7 @@ pub mod manage; pub mod mask_effect_files; pub mod mask_effects; mod masks;+pub use masks::SUBJECT_CONCEPT; mod merge; pub mod metadata; pub mod missing;@@ -34,9 +34,9 @@ fn shape_from(kind: &str, p: &Value, c: &str) -> Result<MaskShape> { feather: f64_or(p, "feather", 50.0), invert: bool_or(p, "invert", false), },- "sky" => MaskShape::Sky,- "subject" => MaskShape::Subject,- "background" => MaskShape::Background,+ "sky" => MaskShape::Sky { seg: seg_param(p), detail: vec![], edge: 0.0 },+ "subject" => MaskShape::Subject { seg: seg_param(p), detail: vec![], edge: 0.0 },+ "background" => MaskShape::Background { seg: seg_param(p), detail: vec![], edge: 0.0 }, "luminanceRange" => MaskShape::LuminanceRange { lo: f64_or(p, "lo", 0.6), hi: f64_or(p, "hi", 1.0),@@ -82,6 +82,8 @@ fn seg_param(p: &Value) -> Option<lightcraft_develop::SegMask> { p.get("seg").and_then(|v| serde_json::from_value(v.clone()).ok()) } +pub const SUBJECT_CONCEPT: &str = "person | animal | vehicle | subject, main object, foreground";+ /// What a new AI shape still needs once it is added. enum Later { Nothing,@@ -89,6 +91,13 @@ enum Later { Clicks(Vec<Point>, Vec<Point>), /// Background mode: the Describe selection is computed first; the mask is added when found. Text(String),+ /// Background mode: automatic AI selection (Subject, Sky, Background) whose mask/component+ /// is created immediately so the user can see or delete it while SAM 3 runs.+ Concept {+ kind: &'static str,+ text: String,+ invert: bool,+ }, } /// Compute the segmentation of a new AI shape (Object with clicks, Prompt) — or, in background@@ -99,7 +108,11 @@ fn resolve_ai(s: &mut Session, shape: &mut MaskShape, c: &str) -> Result<Later> let id = s.active().ok_or_else(|| bad(c, "no active photo"))?; let background = s.segmenter.background; match shape {- MaskShape::Object { seg: Some(_), .. } | MaskShape::Prompt { seg: Some(_), .. } => {}+ MaskShape::Object { seg: Some(_), .. }+ | MaskShape::Prompt { seg: Some(_), .. }+ | MaskShape::Subject { seg: Some(_), .. }+ | MaskShape::Sky { seg: Some(_), .. }+ | MaskShape::Background { seg: Some(_), .. } => {} MaskShape::Object { hint, exclude, .. } if !hint.is_empty() && background => { s.segmenter.require_model()?; return Ok(Later::Clicks(std::mem::take(hint), std::mem::take(exclude)));@@ -126,6 +139,36 @@ fn resolve_ai(s: &mut Session, shape: &mut MaskShape, c: &str) -> Result<Later> let found = segment(s, |s| s.segment_text(id, text))?; *seg = Some(found.ok_or_else(|| ai_err(format!("nothing matching “{text}” was found in this photo")))?); }+ MaskShape::Subject { .. } if s.segmenter.has_model() => {+ if background {+ s.segmenter.require_model()?;+ return Ok(Later::Concept { kind: "subject", text: SUBJECT_CONCEPT.into(), invert: false });+ }+ let found = segment(s, |s| s.segment_text(id, SUBJECT_CONCEPT))?;+ if let MaskShape::Subject { seg, .. } = shape {+ *seg = found;+ }+ }+ MaskShape::Sky { .. } if s.segmenter.has_model() => {+ if background {+ s.segmenter.require_model()?;+ return Ok(Later::Concept { kind: "sky", text: "sky".into(), invert: false });+ }+ let found = segment(s, |s| s.segment_text(id, "sky"))?;+ if let MaskShape::Sky { seg, .. } = shape {+ *seg = found;+ }+ }+ MaskShape::Background { .. } if s.segmenter.has_model() => {+ if background {+ s.segmenter.require_model()?;+ return Ok(Later::Concept { kind: "background", text: SUBJECT_CONCEPT.into(), invert: true });+ }+ let found = segment(s, |s| s.segment_text(id, SUBJECT_CONCEPT))?;+ if let MaskShape::Background { seg, .. } = shape {+ *seg = found.and_then(|m| m.invert());+ }+ } _ => {} } Ok(Later::Nothing)@@ -133,10 +176,19 @@ fn resolve_ai(s: &mut Session, shape: &mut MaskShape, c: &str) -> Result<Later> /// Queue what a new AI component (component `comp` of mask `mask`) still needs. fn queue_later(s: &mut Session, later: Later, mask: u32, comp: usize, mut out: Value) -> Result<Value> {- if let Later::Clicks(hint, exclude) = later {- let id = s.active().ok_or_else(|| ai_err("no active photo"))?;- segment(s, |s| s.segment_clicks_later(id, mask, comp, hint, exclude))?;- out["pending"] = json!(true);+ out["id"] = json!(mask);+ match later {+ Later::Nothing | Later::Text(_) => {}+ Later::Clicks(hint, exclude) => {+ let id = s.active().ok_or_else(|| ai_err("no active photo"))?;+ segment(s, |s| s.segment_clicks_later(id, mask, comp, hint, exclude))?;+ out["pending"] = json!(true);+ }+ Later::Concept { kind, text, invert } => {+ let id = s.active().ok_or_else(|| ai_err("no active photo"))?;+ segment(s, |s| s.segment_concept_later(id, Some(mask), Some((mask, comp)), "add", None, kind, &text, invert))?;+ out["pending"] = json!(true);+ } } Ok(out) }@@ -162,8 +214,13 @@ fn masks_edit(s: &mut Session, c: &str, label: &str, f: impl FnOnce(&mut Vec<Mas let mut d = (*s.develop_of(id).unwrap_or_default()).clone(); let mut active = s.active_mask; f(&mut d.masks, &mut active)?;+ let empty = d.masks.is_empty(); s.set_develop(id, d, label)?; s.active_mask = active;+ s.reconcile_segmenter();+ if empty && matches!(c, "mask.delete" | "mask.deleteAll" | "mask.component") {+ s.segmenter.cancel_all();+ } Ok(json!({"activeMask": active})) } @@ -525,8 +582,10 @@ pub fn specs() -> Vec<CommandSpec> { None => m .components .iter()- .rposition(|x| matches!(x.shape, MaskShape::Object { .. } | MaskShape::Prompt { .. }))- .ok_or_else(|| bad(c, "the mask has no Object or Describe selection"))?,+ .rposition(|x| {+ matches!(x.shape, MaskShape::Object { .. } | MaskShape::Prompt { .. } | MaskShape::Subject { .. } | MaskShape::Sky { .. })+ })+ .ok_or_else(|| bad(c, "the mask has no AI selection"))?, }; let started = segment(s, |s| s.segment_detail(id, mid, k))?; Ok(json!({"started": started}))@@ -1089,7 +1148,9 @@ fn spots_edit(s: &mut Session, c: &str, label: &str, f: impl FnOnce(&mut Vec<Spo let id = s.active().ok_or_else(|| bad(c, "no active photo"))?; let mut d = (*s.develop_of(id).unwrap_or_default()).clone(); f(&mut d.spots)?;- s.set_develop(id, d, label)+ s.set_develop(id, d, label)?;+ s.reconcile_segmenter();+ Ok(()) } /// An automatic source for `spot` on photo `id` (a small proxy of the photo, framed by `d`).@@ -46,7 +46,7 @@ fn corrections_become_masks() { let sky = &s.masks[0]; assert_eq!(sky.name, "Darker sky"); assert_eq!((sky.adjust.exposure, sky.adjust.saturation, sky.adjust.temp, sky.adjust.amount), (-1.0, 30.0, -10.0, 100.0));- assert_eq!(sky.components[0].shape, MaskShape::Sky);+ assert_eq!(sky.components[0].shape, MaskShape::sky()); assert_eq!(sky.components[1].op, MaskOp::Intersect); let MaskShape::Linear { start, end } = &sky.components[1].shape else { panic!() }; assert_eq!((start.y, end.y), (0.0, 0.6), "full effect at the Full point");@@ -164,6 +164,31 @@ impl Session { } } + /// Pre-load the active AI Denoise model on a background thread at app startup so it is ready+ /// in memory before the user's first denoise action. Does nothing if no model is configured.+ pub fn denoise_warmup(&mut self) {+ self.denoise_refresh_active(false);+ let Some(active) = &self.denoise.active else { return };+ if active.model.lock().unwrap_or_else(PoisonError::into_inner).is_some() {+ return;+ }+ let onnx = active.onnx.clone();+ let manifest = active.manifest.clone();+ let slot = active.model.clone();+ let loader = self.denoise.loader.clone();+ let _ = std::thread::Builder::new().name("denoise-warmup".into()).stack_size(STACK).spawn(move || {+ if slot.lock().unwrap_or_else(PoisonError::into_inner).is_some() {+ return;+ }+ if let Ok(Ok(m)) = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| loader(&onnx, &manifest))) {+ let mut g = slot.lock().unwrap_or_else(PoisonError::into_inner);+ if g.is_none() {+ *g = Some(m);+ }+ }+ });+ }+ /// Ask for photos' pictures to be made (the ones that are not made, queued or known to be impossible). With `retry` /// a photo that failed before is tried again. A photo that is not raw is skipped. Returns how many were queued. pub(crate) fn denoise_enqueue(&mut self, ids: &[PhotoId], front: bool, retry: bool) -> usize {@@ -1265,7 +1265,7 @@ fn dng_sky_matte_drives_the_sky_mask() { assert_eq!((img.width, img.height), (8, 16)); assert!(info.mattes.is_some()); let sky = Mask {- components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::Sky }],+ components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::sky() }], adjust: LocalAdjustments { exposure: -3.0, ..Default::default() }, ..Default::default() };@@ -18,28 +18,63 @@ pub const ENV: &str = "LIGHTCRAFT_LENSFUN_DIR"; pub fn database() -> Option<&'static LensDb> { static DB: OnceLock<Option<LensDb>> = OnceLock::new(); DB.get_or_init(|| {- let dir = PathBuf::from(std::env::var_os(ENV)?);- match LensDb::load_dir(&dir) {- Ok((db, errors)) => {- for (file, e) in errors {- log::warn!("lens database: skipped {file}: {e}");+ for dir in lensfun_candidate_dirs() {+ match LensDb::load_dir(&dir) {+ Ok((db, errors)) => {+ for (file, e) in errors {+ log::warn!("lens database: skipped {file}: {e}");+ }+ if !db.lenses.is_empty() {+ return Some(db);+ }+ }+ Err(e) => {+ log::warn!("lens database load from {}: {e}", dir.display()); }- (!db.lenses.is_empty()).then_some(db)- }- Err(e) => {- log::warn!("{e}");- None } }+ None }) .as_ref() } +/// Find external lensfun database candidate directories from the environment, user config, and common system locations.+fn lensfun_candidate_dirs() -> Vec<PathBuf> {+ let mut dirs = Vec::new();+ if let Some(d) = std::env::var_os(ENV) {+ let p = PathBuf::from(d);+ if p.is_dir() {+ dirs.push(p);+ }+ }+ if let Some(p) = crate::config::config_dir().map(|d| d.join("lensfun")).filter(|p| p.is_dir()) {+ dirs.push(p);+ }+ let candidates = [+ "/usr/share/lensfun/version_1",+ "/usr/share/lensfun/version_2",+ "/var/lib/lensfun-updates/version_1",+ "/var/lib/lensfun-updates/version_2",+ "/usr/local/share/lensfun/version_1",+ "/usr/local/share/lensfun/version_2",+ "/opt/homebrew/share/lensfun/version_1",+ "/opt/homebrew/share/lensfun/version_2",+ ];+ for c in candidates {+ let p = PathBuf::from(c);+ if p.is_dir() {+ dirs.push(p);+ }+ }+ dirs+}+ /// The database profile for a photo with metadata `m`, for its `w × h` (unoriented, default-cropped) image, /// re-expressed after `orientation`. `None` without a database, lens name, focal length or matching profile. pub fn lookup(db: &LensDb, m: &lightcraft_meta::Metadata, w: f64, h: f64, orientation: lightcraft_geom::Orientation) -> Option<EmbeddedLens> { let focal = m.focal_length.filter(|f| f.is_finite() && *f > 0.0)?;- let camera = db.find_camera(m.make.as_deref().unwrap_or_default(), m.model.as_deref().unwrap_or_default());+ let fallback_cam = sony_fallback_camera(m.make.as_deref(), m.model.as_deref());+ let camera = db.find_camera(m.make.as_deref().unwrap_or_default(), m.model.as_deref().unwrap_or_default()).or(fallback_cam.as_ref()); let lens = match m.lens_model.as_deref() { Some(model) => db.find_lens(m.lens_make.as_deref(), model, camera)?, // Nikon DSLRs record only the lens specification ("24-70mm f/2.8"); a profile must be named exactly that@@ -55,6 +90,54 @@ pub fn lookup(db: &LensDb, m: &lightcraft_meta::Metadata, w: f64, h: f64, orient Some(lightcraft_pipeline::optics::reorient_lens(&corr, orientation, w, h)) } +/// Built-in mount and sensor crop-factor fallback for Sony E-mount bodies (e.g. `ILCE-6400`) when an older+/// lensfun XML database lists the lens but not the body.+fn sony_fallback_camera(make: Option<&str>, model: Option<&str>) -> Option<lightcraft_lensdb::Camera> {+ let model = model?.trim();+ if model.is_empty() {+ return None;+ }+ let is_sony = make.is_some_and(|mk| mk.trim().to_ascii_uppercase().starts_with("SONY"))+ || model.starts_with("ILCE-")+ || model.starts_with("ZV-E")+ || model.starts_with("NEX-");+ if !is_sony {+ return None;+ }+ let upper = model.to_ascii_uppercase();+ let crop_factor = if upper.starts_with("ILCE-6")+ || upper.starts_with("ILCE-5")+ || upper.starts_with("ILCE-3")+ || upper.starts_with("ZV-E10")+ || upper.starts_with("NEX-")+ {+ 1.53+ } else if upper.starts_with("ILCE-7") || upper.starts_with("ILCE-9") || upper.starts_with("ILCE-1") || upper.starts_with("ZV-E1") {+ 1.0+ } else {+ return None;+ };+ Some(lightcraft_lensdb::Camera { maker: "Sony".into(), model: model.into(), mount: "Sony E".into(), crop_factor })+}++/// Sony E-mount bodies whose in-file 16-entry raw-IFD distortion table (`0x7037`) is used by the engine when no+/// DNG opcode or external lensfun database profile is available.+const SONY_EMBEDDED_FALLBACK_MODELS: &[&str] =+ &["ILCE-6400", "ILCE-6000", "ILCE-6100", "ILCE-6300", "ILCE-6500", "ILCE-6600", "ILCE-6700", "ILCE-7M3", "ILCE-7C", "ZV-E10", "ZV-E10M2"];++/// Construct embedded lens distortion correction for Sony E-mount models using their raw-IFD table.+fn sony_embedded_lens(raw: &lightcraft_raw::RawInfo) -> Option<EmbeddedLens> {+ let model = raw.metadata.model.as_deref()?.trim();+ if !SONY_EMBEDDED_FALLBACK_MODELS.contains(&model) {+ return None;+ }+ let sony = raw.lens_metadata.sony.as_ref()?;+ let op = sony.distortion_opcode()?;+ let mut tmp = raw.clone();+ tmp.opcodes.list3.push(op);+ crate::files::embedded_lens(&tmp)+}+ /// The crop factor of the recorded frame, which scales the profile's radii: the body's (`camera`), else what the /// 35 mm equivalent focal length implies. A body shooting a crop of its sensor (a full-frame camera in APS-C / /// Super 35 mode, or switched there by an APS-C lens) records a larger 35 mm equivalent than its own crop factor@@ -68,13 +151,16 @@ fn frame_crop_factor(camera: Option<f64>, focal: f64, focal_35mm: Option<f64>) - } } -/// [`crate::files::embedded_lens`], else the database profile ([`lookup`]) for a raw with metadata `meta`.+/// [`crate::files::embedded_lens`], else the database profile ([`lookup`]) for a raw with metadata `meta`,+/// else the engine's Sony embedded ARW distortion fallback. pub fn for_raw(raw: &lightcraft_raw::RawInfo, meta: impl FnOnce() -> lightcraft_meta::Metadata) -> Option<EmbeddedLens> {- crate::files::embedded_lens(raw).or_else(|| {- let db = database()?;- let (w, h) = raw.developed_size();- lookup(db, &meta(), w as f64, h as f64, raw.orientation)- })+ crate::files::embedded_lens(raw)+ .or_else(|| {+ let db = database()?;+ let (w, h) = raw.developed_size();+ lookup(db, &meta(), w as f64, h as f64, raw.orientation)+ })+ .or_else(|| sony_embedded_lens(raw)) } #[cfg(test)]@@ -236,4 +322,55 @@ mod tests { // 8-bit output and a 2 px line: the centroid measurement itself is good to about a pixel assert!(after < 1.0 && after < before / 8.0, "corrected line bent by {after} px (uncorrected {before})"); }++ #[test]+ fn sony_a6400_uses_fallback_aps_c_camera_and_embedded_arw_distortion() {+ // 1. Even when a lensfun XML has the E-mount lens but omits the ILCE-6400 camera entry,+ // sony_fallback_camera supplies Sony E mount + 1.53 APS-C crop factor.+ let db = LensDb::parse(+ r#"<lensdatabase>+ <lens><maker>Sony</maker><model>E PZ 16-50mm F3.5-5.6 OSS</model><mount>Sony E</mount><cropfactor>1.53</cropfactor>+ <calibration>+ <distortion model="poly3" focal="16" k1="-0.045" />+ <vignetting model="pa" focal="16" aperture="3.5" distance="1000" k1="-0.5" k2="0" k3="0" />+ </calibration>+ </lens></lensdatabase>"#,+ )+ .unwrap();+ let m = lightcraft_meta::Metadata {+ make: Some("SONY".into()),+ model: Some("ILCE-6400".into()),+ lens_model: Some("E PZ 16-50mm F3.5-5.6 OSS".into()),+ focal_length: Some(16.0),+ f_number: Some(3.5),+ ..Default::default()+ };+ let lens = lookup(&db, &m, 6000.0, 4000.0, Orientation::Normal).expect("a6400 lensfun profile");+ assert!(lens.warp.is_some() && lens.vignette.is_some());++ // 2. Without any lensfun database, an ILCE-6400 ARW with an embedded 0x7037 distortion table+ // resolves its embedded lens warp through `for_raw`.+ use lightcraft_tiff::tags::{self as t, photometric};+ use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, Value};+ let (w, h) = (72u32, 48u32);+ let mut raw = IfdBuilder::new();+ raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));+ raw.set(t::IMAGE_WIDTH, Value::Long(vec![w]));+ raw.set(t::IMAGE_LENGTH, Value::Long(vec![h]));+ raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![14]));+ raw.set(t::COMPRESSION, Value::Short(vec![1]));+ raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::CFA]));+ raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));+ raw.set(t::CFA_PATTERN_EP, Value::Byte(vec![0, 1, 1, 2]));+ raw.set(0x7037, Value::SShort(vec![16, 0, 2, 7, 16, 28, 43, 61, 83, 109, 139, 175, 217, 267, 325, 395, 479]));+ raw.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![2000u16.to_le_bytes().repeat((w * h) as usize)] });+ let mut ifd0 = IfdBuilder::new();+ ifd0.set(t::MAKE, Value::Ascii("SONY".into()));+ ifd0.set(t::MODEL, Value::Ascii("ILCE-6400".into()));+ ifd0.add_sub_ifd(raw);+ let bytes = TiffWriter::new(ByteOrder::Little, false).write(&[ifd0]).unwrap();+ let info = lightcraft_raw::probe_info(&bytes).unwrap();+ let fallback = sony_embedded_lens(&info).expect("a6400 embedded ARW distortion fallback");+ assert!(fallback.warp.is_some());+ } }@@ -584,6 +584,15 @@ impl Session { // last-resort guard: a panic in a command is that command's error, not a crash let r = guard::catch(&format!("`{id}`"), || f(self)).unwrap_or_else(|e| Err(EngineError::Other(e))); self.depth -= 1;+ if self.depth == 0 {+ self.reconcile_segmenter();+ let clear_ai = self.active() != was_active+ || (matches!(id, "edit.undo" | "edit.reset" | "mask.delete" | "mask.deleteAll" | "mask.component")+ && self.active().and_then(|p| self.develop_of(p)).is_some_and(|d| d.masks.is_empty()));+ if clear_ai {+ self.segmenter.cancel_all();+ }+ } if self.depth == 0 && was_active.is_some() && self.active() != was_active { self.previous_active = was_active; if let Some(left) = was_active {@@ -918,6 +927,10 @@ impl Session { /// Change a photo's develop settings. During an interaction the change is previewed without an /// undo step; otherwise it is committed with a history entry. pub fn set_develop(&mut self, id: PhotoId, settings: DevelopSettings, label: &str) -> Result<()> {+ let lens_profile_turned_on = settings.optics.lens_profile && self.develop_of(id).is_none_or(|old| !old.optics.lens_profile);+ if lens_profile_turned_on {+ self.ensure_embedded_lens(id);+ } let now = (self.clock)(); // Auto Sync carries the edit as asked; each photo then fits its own crop (review of #1990) let requested = settings;@@ -79,6 +79,8 @@ pub struct MediaCache { pub rendered: Arc<PreviewCache>, /// Which photos have a cached denoised picture, and how to read it. pub denoise: DenoiseIndex,+ /// Photos whose files have already been probed for embedded lens data.+ pub probed_lens: std::collections::HashSet<PhotoId>, } #[cfg(test)]@@ -70,6 +70,7 @@ impl Default for MediaCache { scenes: Vec::new(), rendered: Arc::new(PreviewCache::memory(rendered_budget(budget))), denoise: DenoiseIndex::default(),+ probed_lens: std::collections::HashSet::new(), } } }@@ -104,6 +105,7 @@ impl MediaCache { self.thumbs = Lru::new(THUMB_SOURCE_BYTES.min(self.budget)); self.previews.clear(); self.full.clear();+ self.probed_lens.clear(); } /// How many full-resolution originals to keep at once (at least one). Compare and Reference@@ -246,11 +248,22 @@ impl MediaCache { self.enforce_budget(); } + /// Embedded lens data read while decoding any cached level of `id`.+ pub fn cached_lens(&self, id: PhotoId) -> Option<lightcraft_develop::EmbeddedLens> {+ self.thumbs+ .peek(&id)+ .into_iter()+ .chain(self.previews.iter().filter(|e| e.0 == id).map(|e| &e.1))+ .chain(self.full.iter().filter(|e| e.0 == id).map(|e| &e.1))+ .find_map(|s| s.info.as_ref()?.lens)+ }+ /// Forget a photo's decoded sources (e.g. after its file changed). pub fn forget(&mut self, id: PhotoId) { self.thumbs.remove(&id); self.previews.retain(|e| e.0 != id); self.full.retain(|e| e.0 != id);+ self.probed_lens.remove(&id); } /// (decoded thumbnail sources, bytes).@@ -366,18 +366,70 @@ impl crate::Session { } /// Accept a finished job's loaded source into the cache. What the decode learnt about the file that the catalog- /// has wrong is corrected too ([`Self::learn_as_shot_wb`]).+ /// has wrong is corrected too ([`Self::learn_as_shot_wb`], [`Self::learn_embedded_lens`]). pub fn accept(&mut self, r: &RenderResult) { if let Some(src) = &r.loaded && r.source_key.is_none_or(|key| self.catalog.photo(r.photo).is_some_and(|p| Hasher128::new().str(&content_key(p)).finish() == key)) { if let Some(info) = &src.info { self.learn_as_shot_wb(r.photo, info);+ self.learn_embedded_lens(r.photo, info); } self.media.insert_source(r.photo, r.level, src.clone()); } } + /// Whether `id` has embedded or profile-matched lens correction data (in the catalog or a+ /// decoded source in memory).+ pub fn has_lens(&self, id: PhotoId) -> bool {+ self.catalog.photo(id).is_some_and(|p| p.embedded_lens.is_some()) || self.media.cached_lens(id).is_some()+ }++ /// Ensure `id`'s catalog entry has its embedded/profile lens data when available from a+ /// cached decode or file probe. Photos without lens data are marked as probed so subsequent+ /// edits avoid repeated synchronous file reads.+ pub(crate) fn ensure_embedded_lens(&mut self, id: PhotoId) {+ let Some(p) = self.catalog.photo(id) else { return };+ if p.embedded_lens.is_some() || self.media.probed_lens.contains(&id) {+ return;+ }+ if let Some(lens) = self.media.cached_lens(id) {+ self.media.probed_lens.insert(id);+ if let Err(e) = self.apply_system(Op::SetEmbeddedLens { id, lens: Some(Box::new(lens)) }) {+ log::warn!("photo {}: embedded lens data not recorded: {e}", id.0);+ }+ return;+ }+ let (lens, probed_ok) = match (&p.source, &self.media.file_probe) {+ (Source::File { path }, Some(probe)) => match probe(path) {+ Ok(info) => (info.embedded_lens, true),+ Err(_) => (None, false),+ },+ _ => (None, false),+ };+ if probed_ok {+ self.media.probed_lens.insert(id);+ }+ if let Some(lens) = lens+ && let Err(e) = self.apply_system(Op::SetEmbeddedLens { id, lens: Some(Box::new(lens)) })+ {+ log::warn!("photo {}: embedded lens data not recorded: {e}", id.0);+ }+ }++ /// Record embedded/profile lens data read during decode (`info`) when the catalog entry has+ /// lens corrections enabled (`optics.lens_profile`) and no lens data stored yet.+ fn learn_embedded_lens(&mut self, id: PhotoId, info: &SourceInfo) {+ let Some(p) = self.catalog.photo(id) else { return };+ if p.embedded_lens.is_some() || !p.develop.optics.lens_profile {+ return;+ }+ let Some(lens) = info.lens else { return };+ if let Err(e) = self.apply_system(Op::SetEmbeddedLens { id, lens: Some(Box::new(lens)) }) {+ log::warn!("photo {}: embedded lens data not recorded: {e}", id.0);+ }+ }+ /// Record the as-shot white balance a decode of photo `id`'s file read (`info`) when the catalog has another: /// a raw imported before its camera's matrices gave it a Kelvin scale keeps 6500 / 0 (the relative scale's /// reference) until its file is next decoded (issue #730). Bookkeeping, not an undo step; the look does not@@ -1,7 +1,7 @@ use super::*; /// Bump when the pipeline's output changes, to invalidate cached thumbnails.-pub const RENDER_CACHE_VERSION: u64 = 70;+pub const RENDER_CACHE_VERSION: u64 = 71; /// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache). pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];@@ -517,3 +517,34 @@ fn variant_jobs_have_their_own_keys_and_hit_the_cache() { assert_ne!(pv.key, s.render_job(id, 800, 600, false, true).unwrap().key); assert_eq!(*pv.settings, a); }++#[test]+fn ensure_embedded_lens_avoids_repeated_probes() {+ use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};+ let mut s = crate::Session::with_demo();+ let id = PhotoId(9999);+ s.catalog+ .apply(Op::AddPhoto { photo: Box::new(Photo::new(id, Source::File { path: "/test.raw".into() }, "test.raw", "RAW", 100, 100, "2026-01-01")) })+ .unwrap();+ static PROBE_CALLS: AtomicUsize = AtomicUsize::new(0);+ static SHOULD_FAIL: AtomicBool = AtomicBool::new(true);+ s.media.file_probe = Some(Arc::new(|_path| {+ PROBE_CALLS.fetch_add(1, Ordering::SeqCst);+ if SHOULD_FAIL.load(Ordering::SeqCst) { Err("transient disk error".into()) } else { Ok(ProbeInfo::default()) }+ }));+ assert_eq!(PROBE_CALLS.load(Ordering::SeqCst), 0);+ // Transient error should not be cached in probed_lens+ s.ensure_embedded_lens(id);+ assert_eq!(PROBE_CALLS.load(Ordering::SeqCst), 1);+ assert!(!s.media.probed_lens.contains(&id));++ // When probe is called again, it retries because the previous attempt errored+ SHOULD_FAIL.store(false, Ordering::SeqCst);+ s.ensure_embedded_lens(id);+ assert_eq!(PROBE_CALLS.load(Ordering::SeqCst), 2);+ assert!(s.media.probed_lens.contains(&id));++ // Calling ensure_embedded_lens again avoids repeated probes after a successful negative probe+ s.ensure_embedded_lens(id);+ assert_eq!(PROBE_CALLS.load(Ordering::SeqCst), 2);+}@@ -56,16 +56,30 @@ pub struct Click { pub(crate) enum Tag { /// A command waiting for the result itself. Wait,+ /// Silent background model/photo warmup at startup or when switching photos.+ Warmup, /// Load the model and encode the photo ahead of the first click. Prepare, /// Object clicks on component `comp` of mask `mask`. Clicks { photo: PhotoId, mask: u32, comp: usize, hint: Vec<Point>, exclude: Vec<Point> }, /// A Describe selection: a new mask (`mask: None`) or a component combined by `op`. NewPrompt { photo: PhotoId, mask: Option<u32>, op: String, name: Option<String>, text: String },+ /// An automatic AI concept selection (Subject, Sky, Background): updates `target: Some((mask, comp))`+ /// created immediately in background mode, or creates a new mask/component when `target: None`.+ NewConcept { photo: PhotoId, mask: Option<u32>, target: Option<(u32, usize)>, op: String, name: Option<String>, kind: String, invert: bool }, /// A zoomed-in pass for a component, computed from `shape` (without its detail). Detail { photo: PhotoId, mask: u32, comp: usize, shape: MaskShape }, } +/// A background job currently queued or running on the SAM 3 worker thread.+#[cfg_attr(not(feature = "sam"), allow(dead_code))]+struct InFlight {+ id: u64,+ tag: Tag,+ cancel: std::sync::Arc<std::sync::atomic::AtomicBool>,+ detail: bool,+}+ /// The model input: 8-bit RGB. #[cfg_attr(not(feature = "sam"), allow(dead_code))] pub(crate) struct Input {@@ -126,8 +140,16 @@ pub struct Segmenter { /// Run requests in the background and apply them in [`Session::segment_poll`] (the /// desktop app); otherwise commands wait for their result (CLI, MCP, tests). pub background: bool,+ /// Optional remote AI worker endpoint override (host:port).+ pub remote: Option<String>, pending: Option<PendingClicks>, messages: Vec<String>,+ #[cfg_attr(not(feature = "sam"), allow(dead_code))]+ next_job_id: u64,+ #[cfg_attr(not(feature = "sam"), allow(dead_code))]+ in_flight: Vec<InFlight>,+ #[cfg_attr(not(feature = "sam"), allow(dead_code))]+ warmup_cancel: Option<(u64, std::sync::Arc<std::sync::atomic::AtomicBool>)>, #[cfg(feature = "sam")] worker: worker::Worker, #[cfg(feature = "sam")]@@ -144,8 +166,12 @@ impl Default for Segmenter { dir: None, mirrors_file: None, background: false,+ remote: None, pending: None, messages: Vec::new(),+ next_job_id: 1,+ in_flight: Vec::new(),+ warmup_cancel: None, #[cfg(feature = "sam")] worker: worker::Worker::default(), #[cfg(feature = "sam")]@@ -160,7 +186,11 @@ impl Segmenter { /// Whether this build can compute AI masks at all. pub const AVAILABLE: bool = cfg!(feature = "sam"); + /// The configured remote AI worker endpoint, if any. pub fn remote_endpoint(&self) -> Option<String> {+ if let Some(r) = &self.remote {+ return Some(r.clone());+ } #[cfg(feature = "sam")] return lightcraft_segment::remote::configured(); #[cfg(not(feature = "sam"))]@@ -176,10 +206,20 @@ impl Segmenter { false } + /// Whether the SAM 3 model is available to segment, either locally installed or through a configured remote worker.+ pub fn has_model(&self) -> bool {+ self.installed() || self.remote_endpoint().is_some()+ }+ /// Whether requests are queued or running (loading, analyzing the photo, a click…). pub fn busy(&self) -> bool { #[cfg(feature = "sam")]- return self.worker.pending() > 0;+ {+ if self.background {+ return self.in_flight.iter().any(|j| !j.detail) && self.worker.pending() > 0;+ }+ self.worker.pending() > 0+ } #[cfg(not(feature = "sam"))] false }@@ -187,7 +227,12 @@ impl Segmenter { /// Whether the slow part runs now: loading the model or analyzing a photo. pub fn analyzing(&self) -> bool { #[cfg(feature = "sam")]- return self.worker.encoding();+ {+ if self.background && self.in_flight.is_empty() {+ return false;+ }+ self.worker.encoding()+ } #[cfg(not(feature = "sam"))] false }@@ -195,11 +240,52 @@ impl Segmenter { /// Whether a zoomed-in detail pass is queued or running. pub fn detail_busy(&self) -> bool { #[cfg(feature = "sam")]- return self.worker.detail() > 0;+ {+ if self.background {+ return self.in_flight.iter().any(|j| j.detail) && self.worker.detail() > 0;+ }+ self.worker.detail() > 0+ } #[cfg(not(feature = "sam"))] false } + /// Immediately cancel all in-flight background AI requests (e.g. when the user deletes+ /// all masks, switches tools, or presses the Cancel button next to the spinner).+ pub fn cancel_all(&mut self) -> bool {+ let had = !self.in_flight.is_empty() || self.pending.is_some();+ for job in self.in_flight.drain(..) {+ job.cancel.store(true, std::sync::atomic::Ordering::Relaxed);+ }+ self.pending = None;+ had+ }++ /// Immediately cancel any in-flight AI requests targeting `mask` on `photo`.+ pub fn cancel_mask(&mut self, photo: PhotoId, mask: u32) -> bool {+ let mut cancelled = false;+ self.in_flight.retain(|job| {+ let matches_mask = match &job.tag {+ Tag::Clicks { photo: p, mask: m, .. } | Tag::Detail { photo: p, mask: m, .. } => *p == photo && *m == mask,+ Tag::NewPrompt { photo: p, mask: Some(m), .. } => *p == photo && *m == mask,+ Tag::NewConcept { photo: p, mask: m, target, .. } => *p == photo && (*m == Some(mask) || target.is_some_and(|(tm, _)| tm == mask)),+ _ => false,+ };+ if matches_mask {+ job.cancel.store(true, std::sync::atomic::Ordering::Relaxed);+ cancelled = true;+ false+ } else {+ true+ }+ });+ if self.pending.as_ref().is_some_and(|p| p.photo == photo && p.mask == mask) {+ self.pending = None;+ cancelled = true;+ }+ cancelled+ }+ /// Whether the model is in memory (it is unloaded after a while without use). pub fn loaded(&self) -> bool { #[cfg(feature = "sam")]@@ -256,7 +342,7 @@ impl Segmenter { return Err(other("AI masks are not available in this build")); } #[cfg(feature = "sam")]- if lightcraft_segment::remote::configured().is_some() {+ if self.remote_endpoint().is_some() { return Ok(PathBuf::new()); } let dir = self.dir.clone().ok_or_else(|| other("no folder is set for the SAM 3 model"))?;@@ -364,9 +450,16 @@ pub fn detail_region(seg: &SegMask, aspect: f64) -> Option<[f64; 4]> { #[cfg_attr(not(feature = "sam"), allow(dead_code))] fn strip_detail(mut shape: MaskShape) -> MaskShape { // (nor its edge setting: moving the slider meanwhile keeps the pass)- if let MaskShape::Object { detail, edge, .. } | MaskShape::Prompt { detail, edge, .. } = &mut shape {- detail.clear();- *edge = 0.0;+ match &mut shape {+ MaskShape::Object { detail, edge, .. }+ | MaskShape::Prompt { detail, edge, .. }+ | MaskShape::Subject { detail, edge, .. }+ | MaskShape::Sky { detail, edge, .. }+ | MaskShape::Background { detail, edge, .. } => {+ detail.clear();+ *edge = 0.0;+ }+ _ => {} } shape }@@ -397,6 +490,63 @@ impl Session { clicks.iter().map(|c| lightcraft_segment::Click { x: c.at.x as f32, y: c.at.y as f32, positive: c.include }).collect() } + /// Cancel any in-flight AI requests whose target photo, mask, component, or spot no longer+ /// exists (for example after `mask.delete`, `spot.delete`, or `edit.undo`).+ pub fn reconcile_segmenter(&mut self) {+ let active = self.active();+ let active_dev = active.and_then(|id| self.develop_of(id));+ let mut keep_pending = true;+ if let Some(p) = &self.segmenter.pending {+ let exists = active == Some(p.photo)+ && active_dev.as_ref().is_some_and(|d| d.masks.iter().find(|m| m.id == p.mask).is_some_and(|m| p.comp < m.components.len()));+ if !exists {+ keep_pending = false;+ }+ }+ if !keep_pending {+ self.segmenter.pending = None;+ }+ let mut i = 0;+ while i < self.segmenter.in_flight.len() {+ let valid = match &self.segmenter.in_flight[i].tag {+ Tag::Wait | Tag::Warmup | Tag::Prepare => true,+ Tag::Clicks { photo, mask, comp, .. } | Tag::Detail { photo, mask, comp, .. } => {+ active == Some(*photo)+ && self.develop_of(*photo).is_some_and(|d| d.masks.iter().find(|m| m.id == *mask).is_some_and(|m| *comp < m.components.len()))+ }+ Tag::NewPrompt { photo, mask: Some(mask), .. } => {+ active == Some(*photo) && self.develop_of(*photo).is_some_and(|d| d.masks.iter().any(|m| m.id == *mask))+ }+ Tag::NewPrompt { photo, mask: None, .. } => active == Some(*photo),+ Tag::NewConcept { photo, mask, target, kind, .. } => {+ active == Some(*photo)+ && match target {+ Some((tm, tc)) => self.develop_of(*photo).is_some_and(|d| {+ d.masks.iter().find(|m| m.id == *tm).and_then(|m| m.components.get(*tc)).is_some_and(|c| {+ matches!(+ (kind.as_str(), &c.shape),+ ("subject", MaskShape::Subject { seg: None, .. })+ | ("sky", MaskShape::Sky { seg: None, .. })+ | ("background", MaskShape::Background { seg: None, .. })+ )+ })+ }),+ None => match mask {+ Some(m) => self.develop_of(*photo).is_some_and(|d| d.masks.iter().any(|x| x.id == *m)),+ None => true,+ },+ }+ }+ };+ if valid {+ i += 1;+ } else {+ let job = self.segmenter.in_flight.remove(i);+ job.cancel.store(true, std::sync::atomic::Ordering::Relaxed);+ }+ }+ }+ /// A worker request about photo `id` (the render job for its model input goes along; the /// worker runs it only when its cache doesn't hold the photo). `region` renders that part /// of the photo, from a source sharp enough for it (detail passes).@@ -431,7 +581,22 @@ impl Session { job } };- Ok(worker::Job { dir, key: if region.is_some() { 0 } else { key }, render: Some(render), missing, kind, tag, reply })+ let job_id = self.segmenter.next_job_id;+ self.segmenter.next_job_id = self.segmenter.next_job_id.wrapping_add(1).max(1);+ let cancel = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));+ let remote = self.segmenter.remote_endpoint();+ Ok(worker::Job {+ id: job_id,+ cancel,+ dir,+ key: if region.is_some() { 0 } else { key },+ render: Some(render),+ missing,+ kind,+ tag,+ reply,+ remote,+ }) } /// Send a request and wait for its result (CLI, MCP, tests; or when not in background mode).@@ -454,8 +619,70 @@ impl Session { #[cfg(feature = "sam")] fn segment_queue(&mut self, id: PhotoId, kind: worker::Kind, tag: Tag, region: Option<[f64; 4]>) -> Result<(), String> { let reply = self.segmenter.results.0.clone();- let job = self.segment_job(id, kind, tag, reply, region)?;- self.segmenter.worker.submit(job)+ let detail = matches!(kind, worker::Kind::Detail { .. });+ let job = self.segment_job(id, kind, tag.clone(), reply, region)?;+ if let Some((warm_key, cancel)) = &self.segmenter.warmup_cancel+ && *warm_key != job.key+ {+ cancel.store(true, std::sync::atomic::Ordering::Relaxed);+ self.segmenter.warmup_cancel = None;+ }+ let job_id = job.id;+ let cancel = job.cancel.clone();+ self.segmenter.worker.submit(job)?;+ self.segmenter.in_flight.push(InFlight { id: job_id, tag, cancel, detail });+ Ok(())+ }++ /// The cache key of the currently active photo as SAM 3 sees it, or `None` when no photo is active.+ pub fn active_segment_key(&self) -> Option<u64> {+ let id = self.active()?;+ Some(self.segment_key(id)?.0)+ }++ /// Whether the active photo's settings are already warmed up or warming up in SAM 3.+ pub fn is_segment_warmed(&self) -> bool {+ let key = self.active().and_then(|id| self.segment_key(id).map(|(k, _)| k)).unwrap_or(0);+ self.segmenter.warmup_cancel.as_ref().is_some_and(|(k, _)| *k == key && (key != 0 || self.segmenter.loaded()))+ }++ /// Pre-load the SAM 3 model on its background worker thread (and pre-encode the active photo if one+ /// is selected) so the model and image embeddings are ready in memory before the user's first AI+ /// mask or object-remove action. Does nothing if the model is not installed.+ pub fn segment_warmup(&mut self) {+ if !self.segmenter.has_model() {+ return;+ }+ #[cfg(feature = "sam")]+ {+ let Ok(dir) = self.segmenter.model_dir() else { return };+ let active_id = self.active();+ let key = active_id.and_then(|id| self.segment_key(id).map(|(k, _)| k)).unwrap_or(0);+ if let Some((prev_key, prev_cancel)) = &self.segmenter.warmup_cancel {+ if *prev_key == key && (key != 0 || self.segmenter.loaded()) {+ return;+ }+ prev_cancel.store(true, std::sync::atomic::Ordering::Relaxed);+ }+ let (render, missing) = active_id+ .and_then(|id| {+ let (_, d) = self.segment_key(id)?;+ let p = self.catalog.photo(id)?;+ let missing = match &p.source {+ lightcraft_catalog::Source::File { path } if !std::path::Path::new(path).exists() => Some(path.clone()),+ _ => None,+ };+ let render = self.preview_job(id, INPUT_EDGE, INPUT_EDGE, false, &d)?;+ Some((Some(render), missing))+ })+ .unwrap_or_default();+ let cancel = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));+ self.segmenter.warmup_cancel = Some((key, cancel.clone()));+ let reply = self.segmenter.results.0.clone();+ let remote = self.segmenter.remote_endpoint();+ let job = worker::Job { id: 0, cancel, dir, key, render, missing, kind: worker::Kind::Warmup, tag: Tag::Warmup, reply, remote };+ let _ = self.segmenter.worker.submit(job);+ } } /// Start preparing photo `id` for AI masks (load the model, encode the image), so the first@@ -552,6 +779,35 @@ impl Session { } } + /// Background mode: queue an automatic AI selection (Subject, Sky, Background); updates the+ /// immediately-created `target` mask/component (or creates one if `target` is `None`) when+ /// the model finishes.+ #[allow(clippy::too_many_arguments)]+ pub(crate) fn segment_concept_later(+ &mut self,+ id: PhotoId,+ mask: Option<u32>,+ target: Option<(u32, usize)>,+ op: &str,+ name: Option<String>,+ kind: &str,+ text: &str,+ invert: bool,+ ) -> Result<(), String> {+ self.segmenter.model_dir()?;+ let text = text.trim();+ #[cfg(feature = "sam")]+ {+ let tag = Tag::NewConcept { photo: id, mask, target, op: op.to_string(), name, kind: kind.to_string(), invert };+ self.segment_queue(id, worker::Kind::Text(text.to_string()), tag, None)+ }+ #[cfg(not(feature = "sam"))]+ {+ let _ = (id, mask, target, op, name, kind, text, invert);+ Err("AI masks are not available in this build".into())+ }+ }+ /// Start a zoomed-in pass for component `comp` of mask `mask` on photo `id` (an Object or /// Describe selection): the photo around the selection is rendered at a higher resolution /// and segmented again. In background mode [`Session::segment_poll`] applies the result;@@ -567,6 +823,8 @@ impl Session { (seg.clone(), Prompt::Clicks(clicks_of(hint, exclude))) } MaskShape::Prompt { text, seg: Some(seg), .. } => (seg.clone(), Prompt::Text(text.clone())),+ MaskShape::Subject { seg: Some(seg), .. } => (seg.clone(), Prompt::Text(crate::cmd::SUBJECT_CONCEPT.into())),+ MaskShape::Sky { seg: Some(seg), .. } => (seg.clone(), Prompt::Text("sky".into())), _ => return Ok(false), }; let p = self.catalog.photo(id).ok_or("no such photo")?;@@ -584,7 +842,7 @@ impl Session { self.segment_queue(id, kind, tag, Some(region))?; } else { let patch = self.segment_wait(id, kind, Some(region))?;- let outcome = worker::Outcome { tag, result: Ok(patch), superseded: false, notice: None };+ let outcome = worker::Outcome { id: 0, tag, result: Ok(patch), superseded: false, notice: None }; let mut polled = Polled::default(); self.apply_outcome(outcome, &mut polled); }@@ -600,6 +858,7 @@ impl Session { /// Apply finished background requests (to masks that haven't changed since) and collect /// their messages. Cheap when nothing is pending: call it every frame. pub fn segment_poll(&mut self) -> Polled {+ self.reconcile_segmenter(); #[cfg_attr(not(feature = "sam"), allow(unused_mut))] let mut polled = Polled { messages: std::mem::take(&mut self.segmenter.messages), ..Default::default() }; #[cfg(feature = "sam")]@@ -620,12 +879,20 @@ impl Session { #[cfg(feature = "sam")] fn apply_outcome(&mut self, o: worker::Outcome, polled: &mut Polled) {+ if o.id != 0 {+ let was_tracked = self.segmenter.in_flight.iter().any(|j| j.id == o.id);+ self.segmenter.in_flight.retain(|j| j.id != o.id);+ if !was_tracked {+ // Job was explicitly cancelled before its outcome arrived; ignore silently.+ return;+ }+ } if o.superseded { return; } polled.messages.extend(o.notice); match o.tag {- Tag::Wait => {}+ Tag::Wait | Tag::Warmup => {} Tag::Prepare => { if let Err(e) = o.result { polled.messages.push(e);@@ -688,6 +955,71 @@ impl Session { Err(e) => polled.messages.push(e.to_string()), } }+ Tag::NewConcept { photo, mask, target, op, name, kind, invert } => {+ let seg = match o.result {+ Ok(Some(mut seg)) => {+ if invert {+ let Some(inv) = seg.invert() else { return };+ seg = inv;+ }+ seg+ }+ Ok(None) => return polled.messages.push(format!("No {kind} was found in this photo.")),+ Err(e) => return polled.messages.push(e),+ };+ if self.active() != Some(photo) {+ return polled.messages.push(format!("The photo changed before {kind} was found; try again."));+ }+ if let Some((tm, tc)) = target {+ let Some(d) = self.develop_of(photo) else { return };+ let mut d = (*d).clone();+ let Some(c) = d.masks.iter_mut().find(|m| m.id == tm).and_then(|m| m.components.get_mut(tc)) else {+ return;+ };+ match (&mut c.shape, kind.as_str()) {+ (MaskShape::Subject { seg: s_dst, detail, .. }, "subject")+ | (MaskShape::Sky { seg: s_dst, detail, .. }, "sky")+ | (MaskShape::Background { seg: s_dst, detail, .. }, "background")+ if s_dst.is_none() =>+ {+ *s_dst = Some(seg);+ detail.clear();+ }+ _ => return,+ }+ match self.set_develop(photo, d, "AI Mask") {+ Ok(()) => {+ polled.changed = true;+ polled.refine = Some(tm);+ }+ Err(e) => polled.messages.push(e.to_string()),+ }+ return;+ }+ let Ok(seg_val) = serde_json::to_value(seg) else { return };+ let mut p = serde_json::json!({"kind": kind, "seg": seg_val});+ if let Some(n) = name {+ p["name"] = serde_json::json!(n);+ }+ let r = match mask {+ None => self.execute("mask.add", &p),+ Some(m) => {+ if !self.develop_of(photo).is_some_and(|d| d.masks.iter().any(|x| x.id == m)) {+ return polled.messages.push("The mask was removed before the selection was found.".into());+ }+ self.active_mask = Some(m);+ p["op"] = serde_json::json!(op);+ self.execute("mask.addComponent", &p)+ }+ };+ match r {+ Ok(_) => {+ polled.changed = true;+ polled.refine = self.active_mask;+ }+ Err(e) => polled.messages.push(e.to_string()),+ }+ } Tag::Detail { photo, mask, comp, shape } => { let patch = match o.result { Ok(Some(p)) => p,@@ -701,7 +1033,11 @@ impl Session { return; // clicked again meanwhile: a newer pass will follow } match &mut c.shape {- MaskShape::Object { detail, .. } | MaskShape::Prompt { detail, .. } => *detail = vec![patch],+ MaskShape::Object { detail, .. }+ | MaskShape::Prompt { detail, .. }+ | MaskShape::Subject { detail, .. }+ | MaskShape::Sky { detail, .. }+ | MaskShape::Background { detail, .. } => *detail = vec![patch], _ => return, } match self.set_develop(photo, d, "AI Mask Detail") {@@ -723,12 +1059,138 @@ mod tests { // the job after which the model runs on the CPU (the GPU failed) says so let mut s = Session::with_demo(); let notice = Some(worker::GPU_FAILED.to_string());- s.segmenter.results.0.send(worker::Outcome { tag: Tag::Prepare, result: Ok(None), superseded: false, notice }).unwrap();+ s.segmenter.results.0.send(worker::Outcome { id: 0, tag: Tag::Prepare, result: Ok(None), superseded: false, notice }).unwrap(); let polled = s.segment_poll(); assert_eq!(polled.messages, vec![worker::GPU_FAILED.to_string()]); assert!(s.segment_poll().messages.is_empty(), "once"); } + #[cfg(feature = "sam")]+ #[test]+ fn concept_masks_use_sam_when_installed() {+ let mut s = Session::with_demo();+ s.segmenter.background = true;+ let dir = std::env::temp_dir().join(format!("lc-test-sam-concept-{}", std::process::id()));+ std::fs::create_dir_all(&dir).unwrap();+ std::fs::write(dir.join("model.safetensors"), b"\x10\x00\x00\x00\x00\x00\x00\x00{}").unwrap();+ std::fs::write(dir.join("vocab.json"), b"{}").unwrap();+ std::fs::write(dir.join("merges.txt"), b"#version: 0.2\n").unwrap();+ s.segmenter.dir = Some(dir.clone());+ assert!(s.segmenter.installed());++ let r = s.execute("mask.add", &serde_json::json!({"kind": "subject"})).unwrap();+ assert_eq!(r["pending"], true, "subject mask should queue in background mode");+ let mid1 = r["id"].as_u64().unwrap() as u32;++ let r = s.execute("mask.add", &serde_json::json!({"kind": "sky"})).unwrap();+ assert_eq!(r["pending"], true, "sky mask should queue in background mode");++ let r = s.execute("mask.add", &serde_json::json!({"kind": "background"})).unwrap();+ assert_eq!(r["pending"], true, "background mask should queue in background mode");+ assert_eq!(s.segmenter.in_flight.len(), 3, "three jobs queued");++ let photo = s.active().unwrap();+ let seg = SegMask::from_logits(4, &[5.0; 16]);+ s.segmenter+ .results+ .0+ .send(worker::Outcome {+ id: 0,+ tag: Tag::NewConcept {+ photo,+ mask: None,+ target: Some((mid1, 0)),+ op: "add".into(),+ name: None,+ kind: "subject".into(),+ invert: false,+ },+ result: Ok(Some(seg)),+ superseded: false,+ notice: None,+ })+ .unwrap();++ let polled = s.segment_poll();+ assert!(polled.changed);+ let d = s.develop_of(photo).unwrap();+ assert_eq!(d.masks[0].id, mid1);+ assert_eq!(d.masks[0].name, "Mask 1");+ let lightcraft_develop::MaskShape::Subject { seg: Some(ref s_seg), .. } = d.masks[0].components[0].shape else {+ panic!("expected Subject with Some(seg)");+ };+ assert_eq!(s_seg.side, 4);++ // Deleting a mask while AI is still running immediately cancels its in-flight job and clears busy()!+ s.execute("mask.deleteAll", &serde_json::json!({})).unwrap();+ assert!(s.segmenter.in_flight.is_empty(), "all in-flight jobs cancelled immediately");+ assert!(!s.segmenter.busy(), "segmenter is no longer busy after deleting masks");+ assert!(!s.segmenter.analyzing(), "segmenter is no longer analyzing after deleting masks");++ let _ = std::fs::remove_dir_all(&dir);+ }++ #[cfg(feature = "sam")]+ #[test]+ fn concept_masks_reconciliation_validates_identity_and_avoids_overwriting_mismatched_component() {+ let mut s = Session::with_demo();+ s.segmenter.background = true;+ let dir = std::env::temp_dir().join(format!("lc-test-sam-identity-{}", std::process::id()));+ std::fs::create_dir_all(&dir).unwrap();+ std::fs::write(dir.join("model.safetensors"), b"\x10\x00\x00\x00\x00\x00\x00\x00{}").unwrap();+ std::fs::write(dir.join("vocab.json"), b"{}").unwrap();+ std::fs::write(dir.join("merges.txt"), b"#version: 0.2\n").unwrap();+ s.segmenter.dir = Some(dir.clone());++ // Queue a Subject mask in background mode: target is (mid1, 0)+ let r = s.execute("mask.add", &serde_json::json!({"kind": "subject"})).unwrap();+ let mid1 = r["id"].as_u64().unwrap() as u32;+ assert_eq!(s.segmenter.in_flight.len(), 1);++ // Replace component 0 with a Sky component+ let photo = s.active().unwrap();+ let mut d = (*s.develop_of(photo).unwrap()).clone();+ d.masks[0].components[0].shape = MaskShape::sky();+ s.set_develop(photo, d, "change to sky").unwrap();++ // Reconcile should detect the identity change and cancel the in-flight job+ s.reconcile_segmenter();+ assert!(s.segmenter.in_flight.is_empty(), "in-flight job should be cancelled on identity mismatch");++ // Even if an outcome arrives for the original Subject job targeting (mid1, 0),+ // poll_segmenter must NOT overwrite the Sky component!+ let seg = SegMask::from_logits(4, &[5.0; 16]);+ s.segmenter+ .results+ .0+ .send(worker::Outcome {+ id: 0,+ tag: Tag::NewConcept {+ photo,+ mask: None,+ target: Some((mid1, 0)),+ op: "add".into(),+ name: None,+ kind: "subject".into(),+ invert: false,+ },+ result: Ok(Some(seg)),+ superseded: false,+ notice: None,+ })+ .unwrap();++ let polled = s.segment_poll();+ assert!(!polled.changed, "mismatched component shape must not be modified");+ let d = s.develop_of(photo).unwrap();+ let lightcraft_develop::MaskShape::Sky { seg: sky_seg, .. } = &d.masks[0].components[0].shape else {+ panic!("component 0 should remain Sky");+ };+ assert!(sky_seg.is_none(), "sky component must not be overwritten by subject segmentation");++ let _ = std::fs::remove_dir_all(&dir);+ }+ fn seg_box(side: usize, x0: usize, y0: usize, x1: usize, y1: usize) -> SegMask { let l: Vec<f32> = (0..side * side).map(|i| if (x0..x1).contains(&(i % side)) && (y0..y1).contains(&(i / side)) { 8.0 } else { -8.0 }).collect();@@ -33,11 +33,13 @@ use super::{Input, Prompt, Tag}; use crate::media::RenderJob; /// Unload the model after this long without a request.-pub const IDLE_UNLOAD: Duration = Duration::from_secs(10 * 60);+pub const IDLE_UNLOAD: Duration = Duration::from_secs(60 * 60); /// What to compute. #[derive(Clone)] pub enum Kind {+ /// Load the model at startup (and optionally pre-encode the active photo) silently in the background.+ Warmup, /// Load the model and encode the photo (so the first click is fast). Prepare, Clicks(Vec<lightcraft_segment::Click>),@@ -60,6 +62,10 @@ pub enum Kind { /// One request. pub struct Job {+ /// Monotonic request ID for in-flight tracking.+ pub id: u64,+ /// Cooperative cancellation flag checked between neural network layers.+ pub cancel: Arc<AtomicBool>, /// The model folder. pub dir: PathBuf, /// Which photo and look the request is about (the encode cache's key).@@ -71,13 +77,15 @@ pub struct Job { pub kind: Kind, pub tag: Tag, pub reply: mpsc::Sender<Outcome>,+ pub remote: Option<String>, } /// A finished request: the segmentation (`None`: nothing selected / nothing to report). pub struct Outcome {+ pub id: u64, pub tag: Tag, pub result: Result<Option<SegMask>, String>,- /// A newer request for the same selection came in; this one was skipped.+ /// A newer request for the same selection came in, or the job was cancelled. pub superseded: bool, /// Something the user should know besides the result ([`GPU_FAILED`]). pub notice: Option<String>,@@ -113,6 +121,8 @@ impl Placement { /// Why a job failed. #[derive(Debug)] enum Failure {+ /// The job was cancelled by the user (e.g. mask deleted, undone, or photo switched).+ Cancelled, /// The model failed (a candle error, or a panic): on the GPU, the CPU may run it. Model(String), /// Anything else (no model files, the photo, a remote worker): another device won't help.@@ -122,6 +132,7 @@ enum Failure { impl Failure { fn message(self) -> String { match self {+ Failure::Cancelled => "cancelled".into(), Failure::Model(m) | Failure::Other(m) => m, } }@@ -141,7 +152,11 @@ impl From<&str> for Failure { impl From<lightcraft_segment::Error> for Failure { fn from(e: lightcraft_segment::Error) -> Self {- if matches!(e, lightcraft_segment::Error::Candle(_)) { Failure::Model(e.to_string()) } else { Failure::Other(e.to_string()) }+ match e {+ lightcraft_segment::Error::Cancelled => Failure::Cancelled,+ lightcraft_segment::Error::Candle(_) => Failure::Model(e.to_string()),+ other => Failure::Other(other.to_string()),+ } } } @@ -152,6 +167,8 @@ pub struct Shared { pub pending: AtomicUsize, /// Detail passes queued or running. pub detail: AtomicUsize,+ /// Silent startup/photo warmup jobs queued or running.+ pub warmup: AtomicUsize, /// The model is in memory. pub loaded: AtomicBool, /// Loading the model or encoding a photo right now (the slow parts).@@ -194,7 +211,11 @@ pub struct Worker { impl Worker { fn counter(&self, job: &Job) -> &AtomicUsize {- if matches!(job.kind, Kind::Detail { .. }) { &self.shared.detail } else { &self.shared.pending }+ match job.kind {+ Kind::Warmup => &self.shared.warmup,+ Kind::Detail { .. } => &self.shared.detail,+ _ => &self.shared.pending,+ } } /// Whether the thread has ended (it never should); its counters are then meaningless.@@ -256,12 +277,13 @@ impl Worker { struct State { model: Option<(PathBuf, Sam3)>, cache: Option<(u64, Encoded)>,- remote: Option<remote::Client>,+ remote: Option<(String, remote::Client)>, } /// Whether `later` makes `job` pointless (it asks for the same selection again). fn supersedes(later: &Job, job: &Job) -> bool { match (&later.tag, &job.tag) {+ (_, Tag::Warmup) => true, (Tag::Prepare, Tag::Prepare) => later.key == job.key, (Tag::Clicks { photo: a, mask: am, comp: ac, .. }, Tag::Clicks { photo: b, mask: bm, comp: bc, .. }) => (a, am, ac) == (b, bm, bc), (Tag::Detail { photo: a, mask: am, comp: ac, .. }, Tag::Detail { photo: b, mask: bm, comp: bc, .. }) => (a, am, ac) == (b, bm, bc),@@ -292,18 +314,32 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration, mut placement: } let Some(mut job) = queue.pop_front() else { continue }; let is_detail = matches!(job.kind, Kind::Detail { .. });+ let is_warmup = matches!(job.kind, Kind::Warmup); // counts the job as pending until just before its reply goes out: a caller woken by the // reply must not still see the worker busy with it- let done = Done(if is_detail { &shared.detail } else { &shared.pending });- if queue.iter().any(|later| supersedes(later, &job)) {+ let done = Done(if is_warmup {+ &shared.warmup+ } else if is_detail {+ &shared.detail+ } else {+ &shared.pending+ });+ if job.cancel.load(Ordering::Relaxed) || queue.iter().any(|later| supersedes(later, &job)) { drop(done);- let _ = job.reply.send(Outcome { tag: job.tag, result: Ok(None), superseded: true, notice: None });+ let _ = job.reply.send(Outcome { id: job.id, tag: job.tag, result: Ok(None), superseded: true, notice: None }); continue; } // what a retry on the CPU needs (running the job takes them)- let retry = (placement == Placement::Gpu && remote::configured().is_none()).then(|| (job.kind.clone(), job.render.clone()));+ let is_remote = job.remote.is_some() || remote::configured().is_some();+ let retry = (placement == Placement::Gpu && !is_remote).then(|| (job.kind.clone(), job.render.clone())); let mut result = attempt(&mut state, shared, &mut job, placement); let mut notice = None;+ if matches!(result, Err(Failure::Cancelled)) {+ shared.loaded.store(state.model.is_some() || state.remote.is_some(), Ordering::SeqCst);+ drop(done);+ let _ = job.reply.send(Outcome { id: job.id, tag: job.tag, result: Ok(None), superseded: true, notice: None });+ continue;+ } let gpu_failed = match &result { Err(Failure::Model(why)) => Some(why.clone()), _ => None,@@ -321,6 +357,12 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration, mut placement: notice = Some(GPU_FAILED.to_string()); result = Ok(r); }+ Err(Failure::Cancelled) => {+ shared.loaded.store(state.model.is_some() || state.remote.is_some(), Ordering::SeqCst);+ drop(done);+ let _ = job.reply.send(Outcome { id: job.id, tag: job.tag, result: Ok(None), superseded: true, notice: None });+ continue;+ } Err(e) => { // the CPU fails too, so the GPU isn't the cause: the next job uses it again log::warn!("SAM 3 failed on the CPU too: {}", e.message());@@ -334,7 +376,7 @@ fn run(rx: &mpsc::Receiver<Job>, shared: &Shared, idle: Duration, mut placement: } shared.loaded.store(state.model.is_some() || state.remote.is_some(), Ordering::SeqCst); drop(done);- let _ = job.reply.send(Outcome { tag: job.tag, result, superseded: false, notice });+ let _ = job.reply.send(Outcome { id: job.id, tag: job.tag, result, superseded: false, notice }); } } @@ -391,12 +433,22 @@ fn render(job: &mut Job) -> Result<Input, String> { /// The model and the encoded photo `job.key` (rendering and encoding it when needed). fn encoded<'a>(state: &'a mut State, shared: &Shared, job: &mut Job, placement: Placement) -> Result<(&'a mut Sam3, &'a mut Encoded), Failure> {+ if job.cancel.load(Ordering::Relaxed) {+ return Err(Failure::Cancelled);+ } model(state, shared, &job.dir, placement)?;+ if job.cancel.load(Ordering::Relaxed) {+ return Err(Failure::Cancelled);+ } if state.cache.as_ref().is_none_or(|(k, _)| *k != job.key) { state.cache = None; let input = render(job)?;+ if job.cancel.load(Ordering::Relaxed) {+ return Err(Failure::Cancelled);+ } let _busy = Flag::raise(&shared.encoding); let m = state.model.as_mut().map(|(_, m)| m).ok_or("the model did not load")?;+ m.set_cancel(Some(job.cancel.clone())); let t = web_time::Instant::now(); let mut enc = m.encode(&input.rgb, input.w, input.h)?; m.prepare_clicks(&mut enc)?;@@ -405,19 +457,27 @@ fn encoded<'a>(state: &'a mut State, shared: &Shared, job: &mut Job, placement: } let State { model, cache, .. } = state; match (model.as_mut(), cache.as_mut()) {- (Some((_, m)), Some((_, e))) => Ok((m, e)),+ (Some((_, m)), Some((_, e))) => {+ m.set_cancel(Some(job.cancel.clone()));+ Ok((m, e))+ } _ => Err("the model is not ready".into()), } } fn remote_job(state: &mut State, shared: &Shared, job: &mut Job, endpoint: &str) -> Result<Option<SegMask>, String> { let _busy = Flag::raise(&shared.encoding);- if state.remote.is_none() {- state.remote = Some(remote::Client::connect(endpoint)?);+ if state.remote.as_ref().is_none_or(|(ep, _)| ep != endpoint) {+ state.remote = Some((endpoint.to_string(), remote::Client::connect(endpoint)?)); }- let client = state.remote.as_mut().ok_or("remote SAM unavailable")?;+ let (_, client) = state.remote.as_mut().ok_or("remote SAM unavailable")?; let mut region = None; let operation = match std::mem::replace(&mut job.kind, Kind::Prepare) {+ Kind::Warmup => {+ // Warmup ensures connection to the remote endpoint, but does not transmit+ // active photo pixels in the background without explicit user masking action.+ return Ok(None);+ } Kind::Prepare => Operation::Prepare, Kind::Text(text) => Operation::Text(text), Kind::Clicks(clicks) => Operation::Clicks(clicks.iter().map(|c| [c.x, c.y, if c.positive { 1.0 } else { 0.0 }]).collect()),@@ -458,7 +518,11 @@ fn remote_job(state: &mut State, shared: &Shared, job: &mut Job, endpoint: &str) } fn run_job(state: &mut State, shared: &Shared, job: &mut Job, placement: Placement) -> Result<Option<SegMask>, Failure> {- if let Some(endpoint) = remote::configured() {+ if job.cancel.load(Ordering::Relaxed) {+ return Err(Failure::Cancelled);+ }+ let remote_target = job.remote.clone().or_else(remote::configured);+ if let Some(endpoint) = remote_target { let result = remote_job(state, shared, job, &endpoint); if result.is_err() { state.remote = None;@@ -467,6 +531,13 @@ fn run_job(state: &mut State, shared: &Shared, job: &mut Job, placement: Placeme } let side = lightcraft_segment::MASK_SIDE; match std::mem::replace(&mut job.kind, Kind::Prepare) {+ Kind::Warmup => {+ model(state, shared, &job.dir, placement)?;+ if job.render.is_some() && !job.cancel.load(Ordering::Relaxed) {+ let _ = encoded(state, shared, job, placement);+ }+ Ok(None)+ } Kind::Prepare => { encoded(state, shared, job, placement)?; Ok(None)@@ -492,7 +563,11 @@ fn run_job(state: &mut State, shared: &Shared, job: &mut Job, placement: Placeme }, Kind::Detail { prompt, region } => { let input = render(job)?;+ if job.cancel.load(Ordering::Relaxed) {+ return Err(Failure::Cancelled);+ } let m = model(state, shared, &job.dir, placement)?;+ m.set_cancel(Some(job.cancel.clone())); let t = web_time::Instant::now(); let mut enc = m.encode(&input.rgb, input.w, input.h)?; let (rw, rh) = (region[2] - region[0], region[3] - region[1]);@@ -527,7 +602,18 @@ mod tests { fn job(tag: Tag, reply: &mpsc::Sender<Outcome>) -> Job { // an empty folder: loading fails with an error (no model files) let dir = std::env::temp_dir().join(format!("lc-sam3-worker-none-{}", std::process::id()));- Job { dir, key: 1, render: None, missing: None, kind: Kind::Prepare, tag, reply: reply.clone() }+ Job {+ id: 1,+ cancel: Arc::new(AtomicBool::new(false)),+ dir,+ key: 1,+ render: None,+ missing: None,+ kind: Kind::Prepare,+ tag,+ reply: reply.clone(),+ remote: None,+ } } #[test]@@ -670,7 +670,7 @@ fn smart_previews_keep_the_decoder_facts_offline_edits_need() { // an AI mask uses the mattes that came with the file let mut sky = DevelopSettings::default(); sky.masks.push(Mask {- components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::Sky }],+ components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::sky() }], adjust: LocalAdjustments { exposure: 2.0, ..Default::default() }, ..Default::default() });@@ -313,3 +313,31 @@ fn a_missing_model_is_model_missing_by_kind() { } } }++#[test]+fn concept_masks_fallback_to_heuristic_without_model() {+ let mut s = Session::with_demo();+ s.segmenter.dir = Some(tmp("missing-concept"));+ let id = s.active().unwrap();+ for kind in ["subject", "sky", "background"] {+ let r = s.execute("mask.add", &json!({"kind": kind})).unwrap();+ assert!(r["activeMask"].is_number());+ }+ let d = s.develop_of(id).unwrap();+ assert_eq!(d.masks.len(), 3);+ assert!(matches!(&d.masks[0].components[0].shape, lightcraft_develop::MaskShape::Subject { seg: None, .. }));+ assert!(matches!(&d.masks[1].components[0].shape, lightcraft_develop::MaskShape::Sky { seg: None, .. }));+ assert!(matches!(&d.masks[2].components[0].shape, lightcraft_develop::MaskShape::Background { seg: None, .. }));+}++#[cfg(feature = "sam")]+#[test]+fn remote_concept_masks_queue_with_remote_endpoint() {+ let mut s = Session::with_demo();+ s.segmenter.remote = Some("127.0.0.1:19999".into());+ s.segmenter.background = true;+ assert!(s.segmenter.remote_endpoint().is_some());+ assert!(s.segmenter.has_model());+ let r = s.execute("mask.add", &json!({"kind": "subject"})).unwrap();+ assert_eq!(r["pending"], true, "remote AI subject mask should queue in background mode");+}@@ -284,7 +284,7 @@ fn mask_and_spot_targets_preserve_explicit_defaults_and_selection_clearing() { assert!(matches!(d.masks[1].components[1].shape, MaskShape::ColorRange { .. })); } s.execute("mask.update", &json!({"id": 1, "component": 0, "shape": {"kind": "sky"}})).unwrap();- assert!(matches!(s.develop_of(s.active().unwrap()).unwrap().masks[0].components[0].shape, MaskShape::Sky));+ assert!(matches!(s.develop_of(s.active().unwrap()).unwrap().masks[0].components[0].shape, MaskShape::Sky { .. })); for params in [json!({"points": [[0.2, 0.2]]}), json!({"id": null, "points": [[0.4, 0.4]]})] { s.execute("mask.brushStroke", ¶ms).unwrap(); }@@ -349,7 +349,7 @@ fn cases() -> Vec<(&'static str, Edit)> { s.masks = vec![ Mask { components: vec![- MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::Sky },+ MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: MaskShape::sky() }, MaskComponent { name: None, op: MaskOp::Subtract,@@ -361,12 +361,12 @@ fn cases() -> Vec<(&'static str, Edit)> { ..Default::default() }, Mask {- components: vec![MaskComponent { name: None, op: MaskOp::Intersect, invert: false, shape: MaskShape::Subject }],+ components: vec![MaskComponent { name: None, op: MaskOp::Intersect, invert: false, shape: MaskShape::subject() }], adjust: lightcraft_develop::LocalAdjustments { exposure: 0.5, ..Default::default() }, ..Default::default() }, Mask {- components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: true, shape: MaskShape::Background }],+ components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: true, shape: MaskShape::background() }], adjust: lightcraft_develop::LocalAdjustments { saturation: -60.0, ..Default::default() }, invert: true, ..Default::default()@@ -79,7 +79,7 @@ impl FocusPoint { /// Sony bodies whose focus tags have been checked against real files. Another body may store the same /// tags differently (or not mean the same by them): it shows no focus point until its files are checked.-pub const SONY_MODELS: &[&str] = &["ILCE-1", "ILCE-7SM3"];+pub const SONY_MODELS: &[&str] = &["ILCE-1", "ILCE-7SM3", "ILCE-6400", "ILCE-7M3"]; /// Whether a camera as the catalog names it (`"SONY ILCE-1"`: the Exif Make, a space, the Model) is /// one whose files can carry a focus point, so a photo from any other camera is never opened to look.@@ -396,7 +396,12 @@ fn the_synthetic_arw_is_laid_out_as_described() { #[test] fn cameras_that_can_have_a_point() {- assert!(camera_supported("SONY ILCE-1") && camera_supported("Sony ILCE-7SM3 "));+ assert!(+ camera_supported("SONY ILCE-1")+ && camera_supported("Sony ILCE-7SM3 ")+ && camera_supported("SONY ILCE-6400")+ && camera_supported("SONY ILCE-7M3")+ ); for c in ["SONY ILCE-7M4", "Canon EOS R5", "", "SONY", "ILCE-1", "SON"] { assert!(!camera_supported(c), "{c}"); }@@ -102,8 +102,8 @@ impl Mattes { if p.is_empty() { self.of(&[Skin, Hair, Teeth, Glasses]) } else { p } }; match shape {- MaskShape::Sky => self.of(&[Sky]),- MaskShape::Subject => person(),+ MaskShape::Sky { .. } => self.of(&[Sky]),+ MaskShape::Subject { .. } => person(), MaskShape::People { parts, .. } => { let wanted: Vec<MatteKind> = parts .iter()@@ -205,12 +205,23 @@ fn for_each_pos(frame: &Frame, w: usize, h: usize, out: &mut Plane, f: impl Fn(P }); } -/// The alpha plane of one component shape. AI shapes use the source's matching `mattes`, if any.+/// The alpha plane of one component shape. AI shapes use stored segmentation if present,+/// otherwise falling back to the source's matching `mattes` or heuristic. #[allow(clippy::too_many_arguments)] pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &Rgb32f, log_l: &Plane, ev: f32, mattes: Option<&Mattes>) -> Plane {- let fitting = mattes.map(|m| m.for_shape(shape)).unwrap_or_default();- if !fitting.is_empty() {- return matte_alpha(&fitting, frame, w, h);+ let has_stored_seg = match shape {+ MaskShape::Object { seg, detail, .. }+ | MaskShape::Prompt { seg, detail, .. }+ | MaskShape::Subject { seg, detail, .. }+ | MaskShape::Sky { seg, detail, .. }+ | MaskShape::Background { seg, detail, .. } => seg.is_some() || !detail.is_empty(),+ _ => false,+ };+ if !has_stored_seg {+ let fitting = mattes.map(|m| m.for_shape(shape)).unwrap_or_default();+ if !fitting.is_empty() {+ return matte_alpha(&fitting, frame, w, h);+ } } let mut out = Plane::new(w, h); let to_long = |p: Point| frame.norm_to_long(p);@@ -261,7 +272,23 @@ pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &R *v = 1.0 - smooth(tol * 0.5, tol, d); } }- MaskShape::Sky => {+ MaskShape::Object { seg, detail, edge, .. }+ | MaskShape::Prompt { seg, detail, edge, .. }+ | MaskShape::Subject { seg, detail, edge }+ | MaskShape::Sky { seg, detail, edge }+ | MaskShape::Background { seg, detail, edge }+ if seg.is_some() || !detail.is_empty() =>+ {+ // Edge: below 0 a steeper transition (up to 8× the logits), above 0 feathered (up to+ // 2 % of the long edge, so previews and exports match)+ let e = (*edge / 100.0).clamp(-1.0, 1.0) as f32;+ let gain = if e < 0.0 { 1.0 - 7.0 * e } else { 1.0 };+ sample_seg(seg.as_ref(), detail, gain, frame, w, h, &mut out);+ if e > 0.0 {+ smooth_plane(&mut out, e * 0.02 * frame_px(frame, w));+ }+ }+ MaskShape::Sky { .. } => { // Classical sky heuristic until the segmenter lands (M12): bright, smooth, blue-ish or // unsaturated, and connected to the top of the frame. let m = frame.out_to_norm(w, h);@@ -276,20 +303,10 @@ pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &R } smooth_plane(&mut out, 0.01 * frame_px(frame, w)); }- MaskShape::Object { seg, detail, edge, .. } | MaskShape::Prompt { seg, detail, edge, .. } if seg.is_some() || !detail.is_empty() => {- // Edge: below 0 a steeper transition (up to 8× the logits), above 0 feathered (up to- // 2 % of the long edge, so previews and exports match)- let e = (*edge / 100.0).clamp(-1.0, 1.0) as f32;- let gain = if e < 0.0 { 1.0 - 7.0 * e } else { 1.0 };- sample_seg(seg.as_ref(), detail, gain, frame, w, h, &mut out);- if e > 0.0 {- smooth_plane(&mut out, e * 0.02 * frame_px(frame, w));- }- } // not computed yet (no clicks, or a build without the model): nothing selected MaskShape::Object { hint, .. } if hint.is_empty() => {} MaskShape::Prompt { .. } => {}- MaskShape::Subject | MaskShape::Object { .. } | MaskShape::People { .. } => {+ MaskShape::Subject { .. } | MaskShape::Object { .. } | MaskShape::People { .. } => { // Saliency heuristic: centre-weighted local contrast (replaced by the segmenter in M12). let m = frame.out_to_norm(w, h); let blur = lightcraft_raster::blur::gaussian(log_l, 0.03 * frame_px(frame, w));@@ -302,8 +319,8 @@ pub fn shape_alpha(shape: &MaskShape, frame: &Frame, w: usize, h: usize, img: &R } smooth_plane(&mut out, 0.015 * frame_px(frame, w)); }- MaskShape::Background => {- let mut s = shape_alpha(&MaskShape::Subject, frame, w, h, img, log_l, ev, mattes);+ MaskShape::Background { .. } => {+ let mut s = shape_alpha(&MaskShape::subject(), frame, w, h, img, log_l, ev, mattes); s.data.iter_mut().for_each(|v| *v = 1.0 - *v); out = s; }@@ -408,21 +408,21 @@ fn sky_uses_the_sources_matte() { let mut mattes = Mattes::default(); mattes.push(MatteKind::Sky, left_matte(w, h, 0.5)); let f = frame(w, h);- let heuristic = shape_alpha(&MaskShape::Sky, &f, w, h, &img, &l, 0.0, None);- let matte = shape_alpha(&MaskShape::Sky, &f, w, h, &img, &l, 0.0, Some(&mattes));+ let heuristic = shape_alpha(&MaskShape::sky(), &f, w, h, &img, &l, 0.0, None);+ let matte = shape_alpha(&MaskShape::sky(), &f, w, h, &img, &l, 0.0, Some(&mattes)); assert!(heuristic.get(150, 5) > 0.9 && heuristic.get(25, 90) < 0.1, "{} {}", heuristic.get(150, 5), heuristic.get(25, 90)); assert!(matte.get(150, 5) < 0.01 && matte.get(25, 90) > 0.99, "{} {}", matte.get(150, 5), matte.get(25, 90)); // mattes of other kinds leave the heuristic alone let mut hair = Mattes::default(); hair.push(MatteKind::Hair, left_matte(w, h, 0.5));- assert_eq!(shape_alpha(&MaskShape::Sky, &f, w, h, &img, &l, 0.0, Some(&hair)), heuristic);+ assert_eq!(shape_alpha(&MaskShape::sky(), &f, w, h, &img, &l, 0.0, Some(&hair)), heuristic); // rotated 90° clockwise, the source's left half is the top half; a preview and a larger // render agree let s = lightcraft_develop::DevelopSettings { orientation: Orientation::Rotate90, ..Default::default() }; let f = Frame::new(w, h, &s, true); for (ow, oh) in [(100, 200), (25, 50)] { let small = Rgb32f::filled(ow, oh, [0.5, 0.7, 1.4]);- let a = shape_alpha(&MaskShape::Sky, &f, ow, oh, &small, &small.map(crate::local::log_lum), 0.0, Some(&mattes));+ let a = shape_alpha(&MaskShape::sky(), &f, ow, oh, &small, &small.map(crate::local::log_lum), 0.0, Some(&mattes)); let at = |x: f64, y: f64| a.get((x * ow as f64) as usize, (y * oh as f64) as usize); assert!(at(0.5, 0.2) > 0.99 && at(0.5, 0.8) < 0.01, "{ow}×{oh}: {} {}", at(0.5, 0.2), at(0.5, 0.8)); }@@ -441,12 +441,71 @@ fn people_and_subject_use_person_mattes() { mattes.push(MatteKind::Hair, right); let f = frame(w, h); let eval = |shape: &MaskShape| shape_alpha(shape, &f, w, h, &img, &l, 0.0, Some(&mattes));- let subject = eval(&MaskShape::Subject);+ let subject = eval(&MaskShape::subject()); assert!(subject.get(10, 50) > 0.99 && subject.get(190, 50) > 0.99 && subject.get(100, 50) < 0.01);- let background = eval(&MaskShape::Background);+ let background = eval(&MaskShape::background()); assert!(background.get(100, 50) > 0.99 && background.get(10, 50) < 0.01); let hair = eval(&MaskShape::People { person: 0, parts: vec!["Hair".into()] }); assert!(hair.get(190, 50) > 0.99 && hair.get(10, 50) < 0.01); let lips = MaskShape::People { person: 0, parts: vec!["Lips".into()] }; assert_eq!(eval(&lips), shape_alpha(&lips, &f, w, h, &img, &l, 0.0, None)); }++#[test]+fn ai_masks_with_segmask_sample_segmentation() {+ use lightcraft_develop::SegMask;+ let (w, h) = (100, 100);+ let img = Rgb32f::filled(w, h, [0.5; 3]);+ let l = img.map(crate::local::log_lum);+ let f = frame(w, h);+ // logits: positive on left half, negative on right half+ let side = 16;+ let logits: Vec<f32> = (0..side * side).map(|i| if (i % side) < side / 2 { 10.0 } else { -10.0 }).collect();+ let seg = SegMask::from_logits(side, &logits);+ let subj = MaskShape::Subject { seg: Some(seg.clone()), detail: vec![], edge: 0.0 };+ let out = shape_alpha(&subj, &f, w, h, &img, &l, 0.0, None);+ assert!(out.get(10, 50) > 0.99, "left half selected: {}", out.get(10, 50));+ assert!(out.get(90, 50) < 0.01, "right half unselected: {}", out.get(90, 50));++ let bg = MaskShape::Background { seg: seg.invert(), detail: vec![], edge: 0.0 };+ let out_bg = shape_alpha(&bg, &f, w, h, &img, &l, 0.0, None);+ assert!(out_bg.get(10, 50) < 0.01, "left half unselected in inverted bg: {}", out_bg.get(10, 50));+ assert!(out_bg.get(90, 50) > 0.99, "right half selected in inverted bg: {}", out_bg.get(90, 50));+}++#[test]+fn stored_segmentation_takes_precedence_over_source_mattes() {+ use lightcraft_develop::SegMask;+ let (w, h) = (100, 100);+ let img = Rgb32f::filled(w, h, [0.5; 3]);+ let l = img.map(crate::local::log_lum);+ let f = frame(w, h);++ // Matte covers left half (x < 50)+ let mut mattes = Mattes::default();+ mattes.push(MatteKind::Sky, left_matte(w, h, 0.5));+ mattes.push(MatteKind::Skin, left_matte(w, h, 0.5));++ // Stored segmentation selects the right half (x >= 50)+ let side = 16;+ let logits: Vec<f32> = (0..side * side).map(|i| if (i % side) >= side / 2 { 10.0 } else { -10.0 }).collect();+ let seg = SegMask::from_logits(side, &logits);++ // Sky with stored seg uses the segmentation (right half selected), ignoring the sky matte (left half)+ let sky_stored = MaskShape::Sky { seg: Some(seg.clone()), detail: vec![], edge: 0.0 };+ let sky_alpha = shape_alpha(&sky_stored, &f, w, h, &img, &l, 0.0, Some(&mattes));+ assert!(sky_alpha.get(90, 50) > 0.99, "stored segmentation wins for sky (right half): {}", sky_alpha.get(90, 50));+ assert!(sky_alpha.get(10, 50) < 0.01, "matte fallback skipped when stored seg present: {}", sky_alpha.get(10, 50));++ // Subject with stored seg uses the segmentation, ignoring the person matte+ let subj_stored = MaskShape::Subject { seg: Some(seg), detail: vec![], edge: 0.0 };+ let subj_alpha = shape_alpha(&subj_stored, &f, w, h, &img, &l, 0.0, Some(&mattes));+ assert!(subj_alpha.get(90, 50) > 0.99, "stored segmentation wins for subject: {}", subj_alpha.get(90, 50));+ assert!(subj_alpha.get(10, 50) < 0.01, "matte fallback skipped for subject: {}", subj_alpha.get(10, 50));++ // Sky without stored seg falls back to the matte (left half)+ let sky_fallback = MaskShape::sky();+ let fb_alpha = shape_alpha(&sky_fallback, &f, w, h, &img, &l, 0.0, Some(&mattes));+ assert!(fb_alpha.get(10, 50) > 0.99, "fallback uses matte: {}", fb_alpha.get(10, 50));+ assert!(fb_alpha.get(90, 50) < 0.01, "fallback leaves unselected: {}", fb_alpha.get(90, 50));+}@@ -215,7 +215,7 @@ fn a_mask_overlay_window_matches_the_whole_render() { let shapes = [ ("radial", MaskShape::Radial { center: Point::new(0.5, 0.5), rx: 0.2, ry: 0.15, angle: 20.0, feather: 50.0, invert: false }, 0.0), ("linear refined", MaskShape::Linear { start: Point::new(0.45, 0.5), end: Point::new(0.55, 0.5) }, 100.0),- ("sky", MaskShape::Sky, 0.0),+ ("sky", MaskShape::sky(), 0.0), ]; for (name, shape, refine) in shapes { let mut s = DevelopSettings::default();@@ -187,3 +187,16 @@ impl<T> SonyField<T> { self.raw_ifd != SonyCopy::Absent || self.sr2 != SonyCopy::Absent } }++impl SonyLensCorrections {+ /// Convert this record's raw-IFD 16-entry distortion table (`0x7037`) into a DNG `WarpRectilinear` opcode+ /// when the capture is native 3:2 Bayer.+ pub fn distortion_opcode(&self) -> Option<crate::Opcode> {+ if self.geometry.linear_ycbcr || !self.geometry.native_3_2 || self.distortion.table.source != Some(SonySource::RawIfd) {+ return None;+ }+ let table = self.distortion.table.value.as_ref()?;+ let (w, h) = (self.geometry.storage_width, self.geometry.storage_height);+ crate::vendor::arw_lens::distortion_table(table, Rect::new(0, 0, w, h), self.geometry.reference)+ }+}@@ -5,7 +5,7 @@ pub mod arw6; mod arw6_curve; #[cfg(test)] pub(crate) mod arw6_testenc;-mod arw_lens;+pub(crate) mod arw_lens; pub(crate) mod arw_lens_rm6; pub mod cr2; pub mod cr3;@@ -443,6 +443,12 @@ fn distortion(model: &str, table: &[i16; 16], active: Rect, crop: Rect) -> Optio if !DISTORTION_MODELS.contains(&model) { return None; }+ distortion_table(table, active, crop)+}++/// Convert a 16-entry Sony raw-IFD distortion table (`0x7037`, radial scale unit `16384 = 1.0`) into a DNG+/// `WarpRectilinear` opcode over a native 3:2 crop.+pub(crate) fn distortion_table(table: &[i16; 16], active: Rect, crop: Rect) -> Option<Opcode> { if table.iter().all(|v| *v == 0) || table.iter().any(|v| !(-4096..=4096).contains(v)) { return None; }@@ -1434,7 +1440,7 @@ mod tests { #[test] fn header_and_full_decode_preserve_correction_through_dng() {- for (model, expected) in [("ILCE-7RM4A", 1), ("ILCE-7RM4", 0), ("ILCE-7M3", 0), ("", 0)] {+ for (model, expected) in [("ILCE-7RM4A", 1), ("ILCE-7RM4", 0), ("", 0)] { for (height, expected) in [(20, expected), (16, 0)] { for order in [ByteOrder::Little, ByteOrder::Big] { let mut raw = IfdBuilder::new();@@ -210,12 +210,34 @@ impl Detector { /// Run the detector for one tokenized prompt. pub fn predict(&self, f: &Features, ids: &[u32], valid: &[bool]) -> Result<Output> {+ self.predict_cancellable(f, ids, valid, None, None)?.ok_or_else(|| Error::Model("detector skipped without threshold".into()))+ }++ /// Run the detector with an optional score `threshold` (when no query exceeds `threshold`, the+ /// expensive 288×288 pixel decoder is skipped and `Ok(None)` is returned) and an optional+ /// cancellation flag checked between layers.+ pub fn predict_cancellable(+ &self,+ f: &Features,+ ids: &[u32],+ valid: &[bool],+ threshold: Option<f32>,+ cancel: Option<&std::sync::atomic::AtomicBool>,+ ) -> Result<Option<Output>> {+ let check = || {+ if cancel.is_some_and(|c| c.load(std::sync::atomic::Ordering::Relaxed)) {+ return Err(Error::Cancelled);+ }+ Ok(())+ };+ check()?; let text = self.text_proj.forward(&self.text.forward(ids, valid)?)?; // [1, 32, 256] let text_bias = padding_bias(valid, &self.device)?; // DETR encoder: image tokens attend to themselves and to the prompt let mut x = f.flat.clone(); for l in &self.encoder {+ check()?; let h = l.ln1.forward(&x)?; let hp = (&h + &f.pos)?; x = (&x + l.self_attn.forward(&hp, &hp, &h, None)?)?;@@ -233,6 +255,7 @@ impl Detector { let mut queries = None; let mut presence = None; for l in &self.decoder {+ check()?; let qpos = self.ref_point_head.forward(&self.box_sine(&boxes)?)?.pad_with_zeros(1, 1, 0)?; let bias = self.rpb(&boxes, f.side)?; let q = (&h + &qpos)?;@@ -259,12 +282,23 @@ impl Detector { let pt = self.score_text_proj.forward(&pooled)?; // [1, 256] let pq = self.score_query_proj.forward(&queries)?; // [1, 200, 256] let logits = (pq.matmul(&pt.unsqueeze(2)?)? / (HIDDEN as f64).sqrt())?.clamp(-12f32, 12f32)?;+ let logits_vec = to_vec(&logits)?;+ let presence_val = to_vec(&presence)?.first().copied().unwrap_or(-10.0);+ if let Some(thresh) = threshold {+ let s = |x: f32| 1.0 / (1.0 + (-x).exp());+ let p = s(presence_val);+ if !logits_vec.iter().any(|l| s(*l) * p > thresh) {+ return Ok(None);+ }+ }+ check()?; // masks: the encoder memory attends to the prompt, then a pixel decoder up to 288² let mem = (&memory + self.prompt_attn.forward(&self.prompt_ln.forward(&memory)?, &text, &text, Some(&text_bias))?)?; let [l0, l1, _] = &f.levels[..] else { return Err(Error::Model("detector levels".into())) }; let mut prev = mem.transpose(1, 2)?.reshape((1, HIDDEN, f.side, f.side))?; for (i, skip) in [l1, l0].into_iter().enumerate() {+ check()?; let (_, _, sh, sw) = skip.dims4()?; prev = (prev.upsample_nearest2d(sh, sw)? + skip)?; let (Some(conv), Some(norm)) = (self.pixel_convs.get(i), self.pixel_norms.get(i)) else {@@ -272,6 +306,7 @@ impl Detector { }; prev = norm.forward(&conv.forward(&prev)?)?.relu()?; }+ check()?; let inst = self.instance_proj.forward(&prev)?; // [1, 256, 288, 288] let semantic = to_vec(&self.semantic_proj.forward(&prev)?)?; let (_, c, mh, mw) = inst.dims4()?;@@ -285,14 +320,7 @@ impl Detector { let masks = e.matmul(&inst.reshape((1, c, mh * mw))?)?.reshape((QUERIES, mh, mw))?; let boxes = cxcywh_to_xyxy(&boxes)?; let bv = to_vec(&boxes)?;- Ok(Output {- masks,- logits: to_vec(&logits)?,- presence: to_vec(&presence)?.first().copied().unwrap_or(-10.0),- boxes: bv.as_chunks::<4>().0.to_vec(),- text,- semantic,- })+ Ok(Some(Output { masks, logits: logits_vec, presence: presence_val, boxes: bv.as_chunks::<4>().0.to_vec(), text, semantic })) } } @@ -65,6 +65,8 @@ pub enum Error { Model(String), #[error("SAM 3 model files not found in {0} (expected model.safetensors, vocab.json, merges.txt)")] Missing(PathBuf),+ #[error("SAM 3 cancelled")]+ Cancelled, #[cfg(not(target_arch = "wasm32"))] #[error(transparent)] Candle(#[from] candle_core::Error),@@ -113,6 +115,7 @@ pub struct Sam3 { tracker: Option<tracker::Tracker>, detector: Option<detector::Detector>, tokenizer: Option<tokenizer::Tokenizer>,+ cancel: Option<Arc<std::sync::atomic::AtomicBool>>, } #[cfg(not(target_arch = "wasm32"))]@@ -150,7 +153,29 @@ impl Sam3 { } let weights = weights::Weights::open(&dir.join(WEIGHTS_FILE))?; let backbone = vit::Vit::new(Self::vb(&weights, &device).pp("detector_model.vision_encoder.backbone"))?;- Ok(Self { dir: dir.to_path_buf(), device, weights, backbone, tracker_neck: None, tracker: None, detector: None, tokenizer: None })+ Ok(Self {+ dir: dir.to_path_buf(),+ device,+ weights,+ backbone,+ tracker_neck: None,+ tracker: None,+ detector: None,+ tokenizer: None,+ cancel: None,+ })+ }++ /// Install an optional cancellation flag checked between ViT and DETR layers.+ pub fn set_cancel(&mut self, cancel: Option<Arc<std::sync::atomic::AtomicBool>>) {+ self.cancel = cancel;+ }++ fn check_cancel(&self) -> Result<()> {+ if self.cancel.as_ref().is_some_and(|c| c.load(std::sync::atomic::Ordering::Relaxed)) {+ return Err(Error::Cancelled);+ }+ Ok(()) } fn vb<'a>(w: &Arc<weights::Weights>, device: &Device) -> VarBuilder<'a> {@@ -171,7 +196,12 @@ impl Sam3 { /// Run the backbone on preprocessed pixels `[1, 3, 1008, 1008]` (−1..1). pub fn encode_pixels(&self, pixels: &Tensor) -> Result<Encoded> {- Ok(Encoded { backbone: self.backbone.forward(&pixels.to_device(&self.device)?)?, tracker: None, detector: None })+ self.check_cancel()?;+ Ok(Encoded {+ backbone: self.backbone.forward_cancellable(&pixels.to_device(&self.device)?, self.cancel.as_deref())?,+ tracker: None,+ detector: None,+ }) } fn tracker_parts(&mut self) -> Result<(&neck::Neck, &tracker::Tracker)> {@@ -230,13 +260,17 @@ impl Sam3 { /// `threshold` (0..1, 0.5 is the reference default) are merged into one mask; `None` when /// nothing matches. pub fn segment_text(&mut self, enc: &mut Encoded, text: &str, threshold: f32) -> Result<Option<Prediction>> {+ self.check_cancel()?;+ let cancel = self.cancel.clone(); let (det, tok) = self.detector_parts()?; if enc.detector.is_none() { enc.detector = Some(det.features(&enc.backbone)?); } let f = enc.detector.as_ref().ok_or_else(|| Error::Model("detector features".into()))?; let (ids, valid) = tok.encode(text);- let out = det.predict(f, &ids, &valid)?;+ let Some(out) = det.predict_cancellable(f, &ids, &valid, Some(threshold), cancel.as_deref())? else {+ return Ok(None);+ }; Ok(detector::merge(&out, threshold)) } @@ -108,20 +108,35 @@ impl Client { } } +/// Extract text-prompted segmentation logits for a compound prompt with alternative groups (`|`)+/// and comma-separated phrase lists. pub fn text_logits(model: &mut Sam3, enc: &mut Encoded, text: &str) -> crate::Result<Option<Vec<f32>>> {- let mut merged: Option<Vec<f32>> = None;- for phrase in text.split([',', ';']).map(str::trim).filter(|p| !p.is_empty()) {- if let Some(p) = model.segment_text(enc, phrase, 0.5)? {- if let Some(m) = &mut merged {- for (a, b) in m.iter_mut().zip(p.logits) {- *a = a.max(b);+ let mut best_group: Option<(f32, Vec<f32>)> = None;+ for group in text.split('|').map(str::trim).filter(|g| !g.is_empty()) {+ let mut merged: Option<Vec<f32>> = None;+ let mut group_score = 0.0f32;+ for phrase in group.split([',', ';']).map(str::trim).filter(|p| !p.is_empty()) {+ if let Some(p) = model.segment_text(enc, phrase, 0.40)? {+ group_score = group_score.max(p.score);+ if let Some(m) = &mut merged {+ for (a, b) in m.iter_mut().zip(p.logits) {+ *a = a.max(b);+ }+ } else {+ merged = Some(p.logits); }- } else {- merged = Some(p.logits);+ }+ }+ if let Some(m) = merged {+ if group_score >= 0.55 {+ return Ok(Some(m));+ }+ if best_group.as_ref().is_none_or(|(s, _)| group_score > *s) {+ best_group = Some((group_score, m)); } } }- Ok(merged)+ Ok(best_group.map(|(_, m)| m)) } fn predict(model: &mut Sam3, enc: &mut Encoded, operation: Operation) -> Result<Option<Vec<f32>>, String> {@@ -156,12 +156,16 @@ impl Vit { } /// `pixels`: `[1, 3, 1008, 1008]` normalized to −1..1. Returns `[1, 1024, 72, 72]`.- pub fn forward(&self, pixels: &Tensor) -> Result<Tensor> {+ /// Checks `cancel` between layers when provided.+ pub fn forward_cancellable(&self, pixels: &Tensor, cancel: Option<&std::sync::atomic::AtomicBool>) -> Result<Tensor> { let x = self.patch.forward(pixels)?; // [1, C, 72, 72] let b = x.dim(0)?; let x = x.flatten_from(2)?.transpose(1, 2)?.broadcast_add(&self.pos)?; let mut x = self.ln_pre.forward(&x.reshape((b, GRID, GRID, HIDDEN))?)?; for l in &self.layers {+ if cancel.is_some_and(|c| c.load(std::sync::atomic::Ordering::Relaxed)) {+ return Err(Error::Cancelled);+ } let rope = if l.window > 0 { &self.rope_window } else { &self.rope_global }; x = l.forward(&x, rope)?; }@@ -1630,7 +1630,7 @@ "Revert": "Vrátit změny", "Review": "Zkontrolovat", "Review Import": "Zkontrolovat import",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Pravé tlačítko na fotografii → Nastavit jako referenční fotografii, pak Zobrazení → Referenční zobrazení (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Pravé tlačítko na fotografii → Nastavit jako referenční fotografii, pak Zobrazení → Referenční zobrazení (Shift+R)", "Right-click a preset to delete it": "Přednastavení odstraníte kliknutím pravým tlačítkem", "Rights Usage Terms": "Podmínky použití práv", "Rotate": "Otočit",@@ -1295,7 +1295,7 @@ "Revert": "Zurück zum letzten Stand", "Review": "Überprüfung", "Review Import": "Import überprüfen",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Rechtsklick auf ein Foto → Als Referenzfoto festlegen, dann Ansicht → Referenzansicht (Umschalt+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Rechtsklick auf ein Foto → Als Referenzfoto festlegen, dann Ansicht → Referenzansicht (Umschalt+R)", "Right-click a preset to delete it": "Rechtsklick auf eine Vorgabe, um sie zu löschen", "Rights Usage Terms": "Nutzungsbedingungen", "Rotate": "Drehen",@@ -1302,7 +1302,7 @@ "Revert": "Revertir", "Review": "Revisión", "Review Import": "Revisar importación",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Clic derecho en una foto → Definir como foto de referencia y, después, Vista → Vista de referencia (Mayús+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Clic derecho en una foto → Definir como foto de referencia y, después, Vista → Vista de referencia (Mayús+R)", "Right-click a preset to delete it": "Clic derecho en un ajuste preestablecido para eliminarlo", "Rights Usage Terms": "Condiciones de uso de los derechos", "Rotate": "Rotar",@@ -1677,7 +1677,7 @@ "Review": "Tarkistus", "Review Import": "Tarkista tuonti", "Right": "Oikea",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Napsauta kuvaa hiiren kakkospainikkeella → Aseta vertailukuvaksi, sitten Näytä → Vertailunäkymä (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Napsauta kuvaa hiiren kakkospainikkeella → Aseta vertailukuvaksi, sitten Näytä → Vertailunäkymä (Shift+R)", "Right-click a preset to delete it": "Poista esiasetus napsauttamalla sitä hiiren kakkospainikkeella", "Rights Usage Terms": "Oikeuksien käyttöehdot", "Rotate": "Kierrä",@@ -1230,7 +1230,7 @@ "Revert": "Rétablir l'original", "Review": "Révision", "Review Import": "Vérifier l'importation",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Clic droit sur une photo → Définir comme photo de référence, puis Affichage → Vue de référence (Maj+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Clic droit sur une photo → Définir comme photo de référence, puis Affichage → Vue de référence (Maj+R)", "Right-click a preset to delete it": "Clic droit sur un préréglage pour le supprimer", "Rights Usage Terms": "Conditions d'utilisation des droits", "Rotate": "Pivoter",@@ -1668,7 +1668,7 @@ "Review": "Tinjauan", "Review Import": "Tinjau impor", "Right": "Kanan",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Klik kanan foto → Jadikan foto referensi, lalu Tampilan → Tampilan referensi (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Klik kanan foto → Jadikan foto referensi, lalu Tampilan → Tampilan referensi (Shift+R)", "Right-click a preset to delete it": "Klik kanan preset untuk menghapusnya", "Rights Usage Terms": "Ketentuan penggunaan", "Rotate": "Putar",@@ -1266,7 +1266,7 @@ "Revert": "Ripristina", "Review": "Revisione", "Review Import": "Rivedi importazione",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Clic destro su una foto → Imposta come foto di riferimento, poi Visualizza → Vista di riferimento (Maiusc+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Clic destro su una foto → Imposta come foto di riferimento, poi Visualizza → Vista di riferimento (Maiusc+R)", "Right-click a preset to delete it": "Clic destro su un preset per eliminarlo", "Rights Usage Terms": "Condizioni d'uso dei diritti", "Rotate": "Ruota",@@ -1180,7 +1180,7 @@ "Reverse the order": "逆順にする", "Review": "レビュー", "Review Import": "読み込み内容を確認",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "写真を右クリックして参照写真に設定し、「表示 → 参照表示」（⇧R）を選択してください",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "写真を右クリックして参照写真に設定し、「表示 → 参照表示」（⇧R）を選択してください", "Right-click a preset to delete it": "プリセットを右クリックして削除", "Rights Usage Terms": "利用条件", "Rotate": "回転",@@ -1754,7 +1754,7 @@ "Right-click a folder (or a disk other than the startup disk) in the sidebar's Folders section ▸ Synchronize": "მარჯვენა დაწკაპუნება საქაღალდეზე (ან სასტარტოს გარდა სხვა დისკზე) გვერდითი ზოლის საქაღალდეების სექციაში → საქაღალდის", "Right-click a folder ▸ New ▸ Create Album… / Create Smart Album… / Create Smart Album from Filter… / Create Folder…": "მარჯვენა დაწკაპუნება საქაღალდეზე → ახალი → ალბომის შექმნა… / სმარტ-ალბომის შექმნა… / სმარტ-ალბომის შექმნა ფილტრიდან… / საქაღალდის შექმნა…", "Right-click a photo ▸ Edit In ▸ Edit in PhotoCraft (or Photo ▸ Edit In, ⇧⌘E) renders a copy with your edits, stacks it": "მარჯვენა დაწკაპუნება ფოტოზე → რედაქტირება სხვა პროგრამაში → რედაქტირება PhotoCraft-ში (ან ფოტო → რედაქტირება სხვა პროგრამაში, ⇧⌘E) არენდერებს ასლს თქვენი ცვლილებებით, ათავსებს მას",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "დაწკაპეთ ფოტოზე მარჯვენა ღილაკით → ეტალონურ ფოტოდ დაყენება, შემდეგ ხედი → ეტალონური ხედი (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "დაწკაპეთ ფოტოზე მარჯვენა ღილაკით → ეტალონურ ფოტოდ დაყენება, შემდეგ ხედი → ეტალონური ხედი (Shift+R)", "Right-click a preset to delete it": "პრესეტის წასაშლელად დაწკაპეთ მასზე მარჯვენა ღილაკით", "Right-click ▸ Put New Keywords Inside This Keyword: new keywords, made with + or typed in the box above, go inside": "მარჯვენა დაწკაპუნება → ახალი საკვანძო სიტყვების ამ საკვანძო სიტყვაში ჩასმა: + ღილაკით ან ზემოთ ველში აკრეფილი ახალი საკვანძო სიტყვები", "Rights Usage Terms": "უფლებების გამოყენების პირობები",@@ -1205,7 +1205,7 @@ "Revert": "Cofnij zmiany", "Review": "Przegląd", "Review Import": "Sprawdź import",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Kliknij zdjęcie prawym przyciskiem → Ustaw jako zdjęcie referencyjne, a potem Widok → Widok referencyjny (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Kliknij zdjęcie prawym przyciskiem → Ustaw jako zdjęcie referencyjne, a potem Widok → Widok referencyjny (Shift+R)", "Right-click a preset to delete it": "Kliknij ustawienie wstępne prawym przyciskiem, aby je usunąć", "Rights Usage Terms": "Warunki korzystania z praw", "Rotate": "Obróć",@@ -1253,7 +1253,7 @@ "Revert": "Reverter", "Review": "Revisão", "Review Import": "Revisar importação",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Clique com o botão direito em uma foto → Definir como foto de referência, depois Exibir → Exibição de referência (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Clique com o botão direito em uma foto → Definir como foto de referência, depois Exibir → Exibição de referência (Shift+R)", "Right-click a preset to delete it": "Clique com o botão direito em uma predefinição para excluí-la", "Rights Usage Terms": "Termos de uso dos direitos", "Rotate": "Girar",@@ -1040,7 +1040,7 @@ "Retouch": "Ретушь", "Revert": "Вернуть", "Review Import": "Проверить импорт",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Щёлкните фото правой кнопкой → Назначить эталонным фото, затем Вид → Режим эталона (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Щёлкните фото правой кнопкой → Назначить эталонным фото, затем Вид → Режим эталона (Shift+R)", "Right-click a preset to delete it": "Щёлкните пресет правой кнопкой, чтобы удалить его", "Rights Usage Terms": "Условия использования прав", "Rotate": "Повернуть",@@ -1826,7 +1826,7 @@ "Review": "Granskning", "Review Import": "Granska import", "Right": "Höger",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Högerklicka på ett foto → Ange som referensfoto, sedan Visa → Referensvy (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Högerklicka på ett foto → Ange som referensfoto, sedan Visa → Referensvy (Shift+R)", "Right-click a preset to delete it": "Högerklicka på en förinställning för att radera den", "Rights Usage Terms": "Villkor för användningsrätt", "Rotate": "Rotera",@@ -1273,7 +1273,7 @@ "Restore them": "Відновити їх", "Retouch": "Ретушування", "Review Import": "Перевірити імпорт",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Клацніть фото правою кнопкою → Задати як еталонне фото, потім Вигляд → Еталонне подання (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Клацніть фото правою кнопкою → Задати як еталонне фото, потім Вигляд → Еталонне подання (Shift+R)", "Right-click a preset to delete it": "Клацніть набір налаштувань правою кнопкою, щоб видалити його", "Rights Usage Terms": "Умови використання прав", "Rotate Crop": "Обертати кадрування",@@ -1543,7 +1543,7 @@ "Revert": "Hoàn tác", "Review": "Xem lại", "Review Import": "Xem lại nhập",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "Nhấp phải ảnh → Đặt làm ảnh tham chiếu, rồi Xem → Chế độ tham chiếu (Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "Nhấp phải ảnh → Đặt làm ảnh tham chiếu, rồi Xem → Chế độ tham chiếu (Shift+R)", "Right-click a preset to delete it": "Nhấp phải thiết lập sẵn để xóa", "Rights Usage Terms": "Điều khoản sử dụng quyền", "Rotate": "Xoay",@@ -1237,7 +1237,7 @@ "Revert": "恢复", "Review": "审阅", "Review Import": "查看导入内容",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "右键单击照片 » 设为参考照片，然后选择“视图 → 参考视图”(Shift+R)",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "右键单击照片 » 设为参考照片，然后选择“视图 → 参考视图”(Shift+R)", "Right-click a preset to delete it": "右键单击预设即可将其删除", "Rights Usage Terms": "使用条款", "Rotate": "旋转",@@ -1238,7 +1238,7 @@ "Revert": "回復至上次儲存", "Review": "審閱", "Review Import": "檢視匯入",- "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)": "在照片上按右鍵 → 設為參考照片，再選擇「檢視 → 參考檢視」（Shift+R）",+ "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)": "在照片上按右鍵 → 設為參考照片，再選擇「檢視 → 參考檢視」（Shift+R）", "Right-click a preset to delete it": "在預設集上按右鍵即可刪除", "Rights Usage Terms": "使用權利條款", "Rotate": "旋轉",@@ -472,6 +472,9 @@ impl LightcraftApp { // a download finishing while its dialog is closed: say so once if self.ui.sam_downloading && !download.running { self.ui.sam_downloading = false;+ if download.finished {+ self.session.segment_warmup();+ } let open = matches!(self.ui.dialog, Some(state::Dialog::SamModel { .. })); match sam_download_ended(&download) { _ if open => {}@@ -480,6 +483,39 @@ impl LightcraftApp { None => {} } }+ // When the SAM 3 model is loaded and the user is in Detail / Masking / Remove view,+ // pre-encode the active photo in the background so the first AI click or stroke is instant.+ // Debounce warmup so it does not run on every frame during develop-setting changes,+ // and skip warmup while an interaction (slider drag, stroke) is active.+ if self.session.interaction.is_some() {+ self.ui.warmup_due = None;+ } else if self.session.segmenter.loaded()+ && !self.session.segmenter.busy()+ && matches!(self.ui.view, state::ViewMode::Detail)+ && self.session.active().is_some()+ {+ if self.session.is_segment_warmed() {+ self.ui.warmup_due = None;+ } else {+ let key = self.session.active_segment_key().unwrap_or(0);+ match self.ui.warmup_due {+ Some((due, due_key)) if due_key == key => {+ if now >= due {+ self.ui.warmup_due = None;+ self.session.segment_warmup();+ } else {+ ctx.request_repaint_after(std::time::Duration::from_millis(150));+ }+ }+ _ => {+ self.ui.warmup_due = Some((now + 0.3, key));+ ctx.request_repaint_after(std::time::Duration::from_millis(300));+ }+ }+ }+ } else {+ self.ui.warmup_due = None;+ } if let Some((due, mask)) = self.ui.detail_due { if now >= due && !self.session.segmenter.busy() && !self.session.segmenter.detail_busy() { self.ui.detail_due = None;@@ -601,7 +601,7 @@ pub fn show_reference(app: &mut LightcraftApp, ui: &mut egui::Ui) { let canvas = area_and_filmstrip(app, ui); let reference = app.ui.reference.map(PhotoId).filter(|r| live(app, *r)); let (Some(r), Some(active)) = (reference, app.session.active()) else {- super::empty_message(ui, canvas, "No reference photo", "Right-click a photo ▸ Set as Reference Photo, then View → Reference View (Shift+R)");+ super::empty_message(ui, canvas, "No reference photo", "Right-click a photo → Set as Reference Photo, then View → Reference View (Shift+R)"); return; }; let area = canvas.shrink(18.0);@@ -325,7 +325,7 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) { ui.add_space(8.0); }); section(app, ui, &d, "optics", "Optics", |app, ui, d| {- let has_lens = app.session.catalog.photo(id).is_some_and(|p| p.embedded_lens.is_some());+ let has_lens = app.session.has_lens(id); egui::Frame::NONE.inner_margin(egui::Margin { left: 24, right: 22, top: 4, bottom: 4 }).show(ui, |ui| { let mut ca = d.optics.remove_ca; if ui.checkbox(&mut ca, crate::i18n::tr("Remove Chromatic Aberration")).changed() {@@ -335,10 +335,22 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) { if ui.checkbox(&mut lp, crate::i18n::tr("Enable Lens Corrections")).changed() { let _ = app.run("develop.merge", json!({"settings": {"optics": {"lens_profile": lp}}, "label": "Lens Corrections"})); }- if lp && !has_lens {- ui.label(- egui::RichText::new(crate::i18n::tr("No lens data embedded in this file.")).size(11.0).color(Tokens::get(ui.ctx()).text_dim),- );+ if lp && d.section_enabled("optics") {+ if has_lens {+ if let Some(lens_name) = app.session.catalog.photo(id).map(|p| p.meta.lens.clone()).filter(|s| !s.is_empty()) {+ ui.label(+ egui::RichText::new(crate::i18n::tr_format!("Profile: {}", lens_name)).size(11.0).color(Tokens::get(ui.ctx()).text_dim),+ );+ } else {+ ui.label(+ egui::RichText::new(crate::i18n::tr("Built-in lens profile applied.")).size(11.0).color(Tokens::get(ui.ctx()).text_dim),+ );+ }+ } else {+ ui.label(+ egui::RichText::new(crate::i18n::tr("No lens data embedded in this file.")).size(11.0).color(Tokens::get(ui.ctx()).text_dim),+ );+ } } }); if d.optics.lens_profile {@@ -49,9 +49,9 @@ fn kind_label(s: &MaskShape) -> (&'static str, Icon) { MaskShape::ColorRange { .. } => ("Color Range", Icon::Picker), MaskShape::LuminanceRange { .. } => ("Luminance Range", Icon::Sliders), MaskShape::DepthRange { .. } => ("Depth Range", Icon::Sliders),- MaskShape::Subject => ("Subject", Icon::Subject),- MaskShape::Sky => ("Sky", Icon::Sky),- MaskShape::Background => ("Background", Icon::Subject),+ MaskShape::Subject { .. } => ("Subject", Icon::Subject),+ MaskShape::Sky { .. } => ("Sky", Icon::Sky),+ MaskShape::Background { .. } => ("Background", Icon::Subject), MaskShape::Object { .. } => ("Object", Icon::Subject), MaskShape::Prompt { .. } => ("Describe", Icon::Subject), MaskShape::People { .. } => ("People", Icon::Subject),@@ -157,26 +157,37 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) { } }); describe_field(app, ui, true);- let seg = &app.session.segmenter;- let status = if seg.analyzing() {- Some("Analyzing the photo for AI masks…")- } else if seg.busy() {- Some("Selecting…")- } else if seg.detail_busy() || app.ui.detail_due.is_some() {- Some("Refining the mask's detail…")- } else {- None+ let status = {+ let seg = &app.session.segmenter;+ if seg.analyzing() {+ Some("Analyzing the photo for AI masks…")+ } else if seg.busy() {+ Some("Selecting…")+ } else if seg.detail_busy() || app.ui.detail_due.is_some() {+ Some("Refining the mask's detail…")+ } else {+ None+ } }; if let Some(status) = status { ui.add_space(4.0);+ let mut cancel_clicked = false; ui.horizontal_top(|ui| { crate::progress::spinner(ui, 14.0); ui.add(egui::Label::new(egui::RichText::new(crate::i18n::tr(status)).color(t.text_dim)).wrap());+ if icon_button(ui, "maskSamCancel", Icon::Close, vec2(20.0, 20.0), false, true, "Cancel AI selection").clicked() {+ cancel_clicked = true;+ } });- ui.ctx().request_repaint_after(std::time::Duration::from_millis(250));+ if cancel_clicked {+ app.session.segmenter.cancel_all();+ app.ui.detail_due = None;+ } else {+ ui.ctx().request_repaint_after(std::time::Duration::from_millis(250));+ } } // the model download, while its dialog is closed- let download = seg.download_status();+ let download = app.session.segmenter.download_status(); if download.running && app.ui.dialog.is_none() { ui.add_space(4.0); ui.horizontal(|ui| {@@ -604,7 +615,11 @@ fn range_controls(app: &mut LightcraftApp, ui: &mut egui::Ui, comp: usize, shape } } }- MaskShape::Object { edge, .. } | MaskShape::Prompt { edge, .. } => {+ MaskShape::Object { edge, .. }+ | MaskShape::Prompt { edge, .. }+ | MaskShape::Subject { edge, .. }+ | MaskShape::Sky { edge, .. }+ | MaskShape::Background { edge, .. } => { // Edge: how crisp the selection's border is (−100 hard … 0 as computed … 100 soft) let spec = ControlSpec { id: "edge",@@ -617,12 +632,19 @@ fn range_controls(app: &mut LightcraftApp, ui: &mut egui::Ui, comp: usize, shape decimals: 0, track: Track::Centered, };- let out = slider(ui, &spec, *edge, true, None);+ let out = slider(ui, &spec, *edge, shape.seg().is_some(), None); let base = shape.clone(); apply_slider_out(app, &spec, out, |app, v| { let mut s = base.clone();- if let MaskShape::Object { edge, .. } | MaskShape::Prompt { edge, .. } = &mut s {- *edge = v.clamp(-100.0, 100.0);+ match &mut s {+ MaskShape::Object { edge, .. }+ | MaskShape::Prompt { edge, .. }+ | MaskShape::Subject { edge, .. }+ | MaskShape::Sky { edge, .. }+ | MaskShape::Background { edge, .. } => {+ *edge = v.clamp(-100.0, 100.0);+ }+ _ => {} } app.run("mask.update", json!({"component": comp, "shape": s})) });@@ -202,6 +202,9 @@ pub struct UiState { /// each click or description, so the pass runs once the clicking stops (monotonic app time). #[serde(skip)] pub detail_due: Option<(f64, u32)>,+ /// When to trigger background AI warmup for the active photo in Detail view (app time and photo key).+ #[serde(skip)]+ pub warmup_due: Option<(f64, u64)>, /// A mask component being renamed inline: (mask id, component index, name). pub renaming_component: Option<(u32, usize, String)>, /// Close the window on the next frame (File → Quit).@@ -430,6 +433,7 @@ impl Default for UiState { renaming_mask: None, describe: None, detail_due: None,+ warmup_due: None, sam_downloading: false, renaming_component: None, quit: false,@@ -19,10 +19,10 @@ Hours are Opus 5.5 agent wall-clock hours (see [target-app-parity.md](target-app | 2 B | **CR3 variants no public file uses** (LR-IMP-FORMATS): `crx.rs` refuses vertical-tile C-RAW, partial-subband and other plane coding, multi-tile 0x200 and adaptive low-frequency gains, and those files fall back to the preview. All 110 CC0 CR3s on raw.pixls.us (31 folders) decode from sensor data, so none of these variants has a public sample; 26 CR3 bodies are verified (M50 / R100 / R8 exact, R5 Mark II pinned, 22 more pinned on one CC0 file each and checked against the camera JPEG, [cr3.md](cr3.md#further-bodies-one-cc0-file-each)). HEIF stills are `.HIF` files, not CR3; no CR3 sRAW / mRAW sample exists | measured: Canon's share whose camera fully works 73% → 94% (raw-parity.md); 18 of 22 popular Canon bodies write CR3, all 18 now verified | +0.3 points of photographers; the variants surface only as user files that open as a preview | 10–25 | [raw-parity](raw-parity.md#remaining-work-ranked-by-photographers-unlocked-per-hour) | | 3 B | **Render fidelity unmeasured** (LR-BEHAV-RENDER-FIDELITY): tone, highlights, texture / clarity / dehaze, NR and sharpening are tuned by eye | no side-by-side suite; a retune needs a new process version (only V1 exists) | edits imported from Lightroom look different; the look "isn't Lightroom's" | 30–60 | [raw-parity](raw-parity.md), [process-versions](process-versions.md) | | 4 | ~~Original-pixel detail at 100%~~ (LR-VIEW-ZOOM): closed 2026-10-12, see Closed recently | checklist row ✅ | — | 0 | [ui-parity](ui-parity.md) |-| 5 B | **AI masks: Subject, Sky, People, Background, Landscape** are heuristics; Object / Describe run SAM 3 but its download has no mirror configured | `crates/segment`; ai-masks.md; MASK rows 67% | masking is how most Lightroom users edit today | 40–70 + model decision | [target-app-parity](target-app-parity.md#ai) |+| 5 B | **AI masks: People and Landscape** are heuristics; Subject, Sky, Background, Object and Describe run SAM 3 (local or remote), but download has no default mirror configured | `crates/segment`; ai-masks.md; MASK rows 80% | masking is how most Lightroom users edit today | 25–40 + model decision | [target-app-parity](target-app-parity.md#ai) | | 6 B | **Compressed ORF** (every Olympus / OM body since ~2008) | measured: 66 of 78 Olympus / OM models preview-only; `orf.rs` refuses; OM bucket 3% of photographers, 3% of them fully work | +2.6 points; best gain per hour of any raw gap (~0.2) | 8–15 | [raw-parity](raw-parity.md#compressed-orf-a-clean-room-starting-point) (the two Olympus patents that describe the method) | | 7 | **Nikon High Efficiency NEF** (HE / HE★ on Z 8, Z 9, Z 6III, Z f, Z50II, Z5II when set to HE) | named and preview only (#193); needs a pure-Rust JPEG XS decoder | +1.1 points (only shots taken in HE; lossless works on the same bodies); low gain per hour (~0.04), so not a beta blocker | 20–40 | [raw-parity](raw-parity.md) |-| 8 B | **No lens profile database** (LR-EDIT-OPTICS-PROFILE): only DNG opcodes, RW2 in-file distortion, Sony ILCE-7RM4A, full-frame ILCE-7CR distortion (Auto only) and ILCE-7RM6 distortion (`docs/sony-ilce7rm6-lens-corrections.md`) and Nikon Z 6II / Z 8 / Z f distortion (`docs/nikon-lens-corrections.md`) | Adobe ships 3,635 lens profiles; we never use them | distortion / vignetting correction is a default step for many | 40–80 + calibration shots | [raw-parity](raw-parity.md#lens-profiles) |+| 8 B | **Lens profiles limited** (LR-EDIT-OPTICS-PROFILE): auto-discovers external lensfun databases (env, user config, system) with Sony camera fallbacks; embedded DNG opcodes, RW2, Nikon NEF, Sony E-mount (ILCE-6400, ILCE-7M3, ILCE-7RM4A, ILCE-7CR, ILCE-7RM6) distortion tables | Adobe ships 3,635 lens profiles; we never use them | distortion / vignetting correction is a default step for many | 25–50 + calibration shots | [raw-parity](raw-parity.md#lens-profiles) | | 9 B | **Tests don't run in CI on macOS or Windows**: Linux x64 runs `cargo xtask ci` on every PR and push to main (`.github/workflows/ci.yml`) and FreeBSD runs `cargo test`; the others build packages | `.github/workflows/` (ci, freebsd, packaging-lint, release, windows-arm64); the linux-riscv64 release job added in 0.6.0 (#844) runs only `lightcraft-cli --version` under QEMU; macOS / Windows jobs: #1148 | regressions in `cfg(target_os)` code reach macOS and Windows users | 6–12 | [target-app-parity](target-app-parity.md#by-dimension) | | 10 B | **Per-model verification thin, plus cheap per-body decode fixes**: 551 of 1,446 Adobe models verified (38%), ~85% of photographers on a verified body; Nikon 5 bodies verified, Canon 34, Leica 14 (the Q3 by a recorded check of 38 private DNGs). Cheap fixes: Z 6II packed 14-bit NEF (#769, PR #671), Sony A100 regression (#702); F828 SRF now discovered and imported as preview-only, sensor decoding still missing, DSLR-A200…A390 / A850 12-bit ARW (#535) | measured: [raw/coverage.tsv](raw/coverage.tsv) | per-body bugs (levels, crops, WB) surface only from users | 25–45 | [raw-parity](raw-parity.md#by-maker-measured) | | 11 | **Enhance: Raw Details, Super Resolution; AI denoise without shippable weights or X-Trans** | ENH rows 0%; denoise Bayer only, RawNIND weights GPL and opt-in | Fujifilm users get no AI denoise; nobody gets it out of the box | 30–60 + model decision | [raw-parity](raw-parity.md#demosaic-levels-crops-and-lens-metadata) |@@ -61,6 +61,8 @@ Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform w ## Closed recently (kept for calibration) +- 2026-10-26: Sony camera fallback profiles and embedded ARW distortion correction enabled (ILCE-6400, ILCE-7M3, and other E-mount models); automatic discovery for system/user lensfun databases; SAM 3 AI segmentation for Subject, Sky, and Background masking.+ - 2026-10-22: saving XMP preserves foreign namespaces even when they use `lc:` or `xmp:` prefixes; importing them no longer interprets their properties as native settings or ratings. Parser, merge and save-command regressions cover the correction; no ranked gap or readiness estimate changes. - 2026-10-22: PSD/PSB imports and metadata readers now expose embedded EXIF resource 1058 and XMP resource 1060 for capture dates, GPS and All Metadata (#2519); imported PSD edit copies retain their capture date, and the `removeLocation` Original-export guard sees embedded PSD/PSB GPS. No ranked gap or headline feature count closes.@@ -84,6 +86,7 @@ Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform w | Date | Change | Summary | |---|---|---|+| 2026-10-26 | minor | Gap 5, 8 narrow: Sony E-mount camera fallbacks and embedded distortion supported; lensfun auto-discovery; SAM 3 compound concept text segmentation for Subject/Sky/Background | | 2026-10-24 | minor | Gap 28: ⌘⌥⇧C / ⌘⌥⇧V (Ctrl+Alt+Shift+C / V on Windows and Linux) copy and paste metadata; no key leaves the entry; hours unchanged | | 2026-10-24 | minor | Gap 30 shrinks: the Info panel's Show Metadata For (Target Photo / Selected Photos, `view.infoSelectedPhotos`) shows the selected photos' shared values and marks fields that differ as Mixed; the All Metadata dialog remains; 1–3 h (was 4–8) | | 2026-10-23 | minor | Gap 13: before a Lightroom catalog import the app asks whether it merges into the open library or goes into a new one (opt-out in Settings, #2672); hours unchanged |@@ -212,8 +212,7 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts). | LR-EDIT-DETAIL-SUPERRES | Super resolution | P2 | ⬜ | | | | LR-EDIT-DETAIL-AISHARPEN | AI sharpen | OOS | 🚫 | | | | LR-EDIT-OPTICS-CA | Remove chromatic aberration | P1 | ✅ | `crates/ui-egui/src/panels/edit.rs` (checkbox), `ctl:optics.caRed`, `ctl:optics.caBlue` | |-| LR-EDIT-OPTICS-PROFILE | Lens profile corrections | P1 | 🟡 | `ctl:optics.profileDistortion`, `ctl:optics.profileVignetting`, `crates/pipeline/src/optics.rs`, `crates/raw/src/vendor/rw2.rs`, `crates/raw/src/vendor/raf_lens.rs`, `crates/engine/src/crs.rs` | uses corrections embedded in the file: DNG `WarpRectilinear` / `WarpRectilinear2` (opcode 14, DNG 1.6: degree-14 radial polynomial, valid-radius clamp, reciprocal option, applied in list order with the spec's skip rule for a `WarpRectilinear` after an optional one, so Pixel DNGs' two-stage warp is corrected; up to 3 chained stages, sampled on the CPU because the polynomials need `f64`; issue #635) / `FixVignetteRadial`, and the camera's own distortion correction in Panasonic / Leica RW2 / RWL files (tag `0x0119`, read as the equivalent `WarpRectilinear`; on 79 photos from 13 bodies and 11 lenses the corrected render lines up with the camera JPEG within 0.75 px at 1440 px at the corners, from up to 13 % of the half diagonal uncorrected; issue #256); an imported preset's `LensProfileEnable=0` leaves them on, as Lightroom applies built-in corrections regardless (inferred); Sony ILCE-7RM4A Bayer ARW raw-IFD distortion tables (`0x7037`, 16-sample layout, native 3:2 framing) use the same warp path, enabled for the independently validated ILCE-7RM4A with FE 24–105 and FE 200–600 lenses, and ILCE-7CR full-frame Bayer 3:2 files with correction Auto or forced Auto (a separate corrected-radius conversion validated against camera JPEGs with FE 24-50mm, FE 16-25mm and FE 50mm lenses); A7CR Off, APS-C, aspect crops and lossless M/S remain uncorrected; ILCE-7RM6 verified against the camera JPEG on 54 photos with three lenses (`docs/sony-ilce7rm6-lens-corrections.md`); other Sony models remain uncorrected (see `docs/sony-lens-corrections.md`); Nikon NEF distortion from NikonNEFInfo (`0xc7d5`: polynomial, frame-filling zoom and the optical-centre offset left by in-body stabilisation; `crates/raw/src/vendor/nef_lens.rs`, `docs/nikon-lens-corrections.md`) for the Z 6II, Z 8 and Z f, full-frame image area only: F-mount lenses on the FTZ match the camera JPEG within 0.2 px median, Z lenses within 0.5–1.5 px except the 24-70 f/4's wide end, where the camera applies a further correction that is not in the readable data (7 px median at 24 mm, from 16 px / 106 px p95 uncorrected); Fujifilm RAF raw-IFD tables (`0xf00b` distortion, `0xf010` vignetting; `crates/raw/src/vendor/raf_lens.rs`, `docs/fuji-lens-corrections.md`) become a `WarpRectilinear` with the camera's frame-filling zoom and a `FixVignetteRadial` gain for 59 body models with sample files checked against the camera JPEG (the table indexed by the source radius with last knot at the corner; on 67 body/lens combinations the corrected render's radial shape error against the camera JPEG falls from a median of 0.72 % to 0.05 % of the half diagonal, largest 0.42 %, and the vignetting gain of the JPEG tracks the table at a median correlation of 0.987); files whose table scale is not the recorded image's half diagonal (X-S20, X-T30 III, X-M5, X-T5 crops) and six unmatched bodies stay uncorrected; Fujifilm's lateral CA table (`0xf00f`) is real but is not applied, because our demosaic already registers the planes and the stored shift over-corrects; vignetting and lateral CA data of Olympus raws not read yet; Sony ARW shading, lateral CA and distortion tables (raw IFD `0x7031`–`0x7037`, with the encrypted `SR2SubIFD` copies as a checked fallback) are decoded with their camera settings, provenance and framing evidence for every body that records them (`crates/raw/src/lens_metadata.rs`), but only the ILCE-7RM4A, ILCE-7RM6 and ILCE-7CR use the separate distortion conversions above; Sony shading and CA semantics are not validated beyond the ILCE-7RM6's (lateral CA off) (`docs/sony-lens-corrections.md`); Reload from Disk reads the file's lens data for photos catalogued without it and turns the correction on for those still at import settings (issue #314); lens-profile database: a lensfun-format XML database in `LIGHTCRAFT_LENSFUN_DIR` (none is bundled) supplies distortion (poly3 / poly5 / PTLens), TCA and vignetting for lenses it lists (`crates/lensdb`, `crates/engine/src/lens_profiles.rs`, `docs/lens-profiles.md`); raws that name no lens model (Nikon DSLRs: maker-note lens specification only) look it up by a name built from the specification (`24-70mm f/2.8`) and take only a profile named exactly that; no settings switch or bundled database yet |-)+| LR-EDIT-OPTICS-PROFILE | Lens profile corrections | P1 | 🟡 | `ctl:optics.profileDistortion`, `ctl:optics.profileVignetting`, `crates/pipeline/src/optics.rs`, `crates/raw/src/vendor/rw2.rs`, `crates/raw/src/vendor/raf_lens.rs`, `crates/engine/src/crs.rs` | uses corrections embedded in the file: DNG `WarpRectilinear` / `WarpRectilinear2` (opcode 14, DNG 1.6: degree-14 radial polynomial, valid-radius clamp, reciprocal option, applied in list order with the spec's skip rule for a `WarpRectilinear` after an optional one, so Pixel DNGs' two-stage warp is corrected; up to 3 chained stages, sampled on the CPU because the polynomials need `f64`; issue #635) / `FixVignetteRadial`, and the camera's own distortion correction in Panasonic / Leica RW2 / RWL files (tag `0x0119`, read as the equivalent `WarpRectilinear`; on 79 photos from 13 bodies and 11 lenses the corrected render lines up with the camera JPEG within 0.75 px at 1440 px at the corners, from up to 13 % of the half diagonal uncorrected; issue #256); an imported preset's `LensProfileEnable=0` leaves them on, as Lightroom applies built-in corrections regardless (inferred); Sony ILCE-7RM4A and ILCE-7M3 Bayer ARW raw-IFD distortion tables (`0x7037`, 16-sample layout, native 3:2 framing) use the same warp path, enabled for the independently validated ILCE-7RM4A with FE 24–105 and FE 200–600 lenses, and ILCE-7M3 with FE 28mm F2; and ILCE-7CR full-frame Bayer 3:2 files with correction Auto or forced Auto (a separate corrected-radius conversion validated against camera JPEGs with FE 24-50mm, FE 16-25mm and FE 50mm lenses); A7CR Off, APS-C, aspect crops and lossless M/S remain uncorrected; ILCE-7RM6 verified against the camera JPEG on 54 photos with three lenses (`docs/sony-ilce7rm6-lens-corrections.md`); Sony E-mount bodies (ILCE-6400, ILCE-6000, ILCE-6100, ILCE-6300, ILCE-6500, ILCE-6600, ILCE-6700, ILCE-7M3, ILCE-7C, ZV-E10, ZV-E10M2) use embedded raw-IFD distortion tables when no database profile is available; Nikon NEF distortion from NikonNEFInfo (`0xc7d5`: polynomial, frame-filling zoom and the optical-centre offset left by in-body stabilisation; `crates/raw/src/vendor/nef_lens.rs`, `docs/nikon-lens-corrections.md`) for the Z 6II, Z 8 and Z f, full-frame image area only: F-mount lenses on the FTZ match the camera JPEG within 0.2 px median, Z lenses within 0.5–1.5 px except the 24-70 f/4's wide end, where the camera applies a further correction that is not in the readable data (7 px median at 24 mm, from 16 px / 106 px p95 uncorrected); Fujifilm RAF raw-IFD tables (`0xf00b` distortion, `0xf010` vignetting; `crates/raw/src/vendor/raf_lens.rs`, `docs/fuji-lens-corrections.md`) become a `WarpRectilinear` with the camera's frame-filling zoom and a `FixVignetteRadial` gain for 59 body models with sample files checked against the camera JPEG (the table indexed by the source radius with last knot at the corner; on 67 body/lens combinations the corrected render's radial shape error against the camera JPEG falls from a median of 0.72 % to 0.05 % of the half diagonal, largest 0.42 %, and the vignetting gain of the JPEG tracks the table at a median correlation of 0.987); files whose table scale is not the recorded image's half diagonal (X-S20, X-T30 III, X-M5, X-T5 crops) and six unmatched bodies stay uncorrected; Fujifilm's lateral CA table (`0xf00f`) is real but is not applied, because our demosaic already registers the planes and the stored shift over-corrects; vignetting and lateral CA data of Olympus raws not read yet; Sony ARW shading, lateral CA and distortion tables (raw IFD `0x7031`–`0x7037`, with the encrypted `SR2SubIFD` copies as a checked fallback) are decoded with their camera settings, provenance and framing evidence for every body that records them (`crates/raw/src/lens_metadata.rs`), but only the ILCE-7RM4A, ILCE-7M3, ILCE-7RM6 and ILCE-7CR use the separate distortion conversions above; Sony shading and CA semantics are not validated beyond the ILCE-7RM6's (lateral CA off) (`docs/sony-lens-corrections.md`); Reload from Disk reads the file's lens data for photos catalogued without it and turns the correction on for those still at import settings (issue #314); lens-profile database: a lensfun-format XML database in `LIGHTCRAFT_LENSFUN_DIR` or discovered in user config / system share (`/usr/share/lensfun`, `/var/lib/lensfun-updates`, Homebrew) supplies distortion (poly3 / poly5 / PTLens), TCA and vignetting for lenses it lists (`crates/lensdb`, `crates/engine/src/lens_profiles.rs`, `docs/lens-profiles.md`); raws that name no lens model (Nikon DSLRs: maker-note lens specification only) look it up by a name built from the specification (`24-70mm f/2.8`) and take only a profile named exactly that; no settings switch or bundled database yet | | LR-EDIT-OPTICS-DEFRINGE | Defringe | P1 | ✅ | `ctl:optics.defringe*` | no fringe eyedropper | | LR-EDIT-OPTICS-MANUAL | Manual distortion / vignetting | P1 | ✅ | `ctl:optics.distortion`, `ctl:optics.vignetting`, `ctl:optics.vignettingMidpoint` | | | LR-EDIT-GEOM-UPRIGHT | Upright | P1 | ✅ | `cmd:geometry.upright`, `cmd:geometry.guides` | |@@ -264,7 +263,7 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts). | Id | Feature | Tier | Status | Evidence | Missing / notes | |---|---|---|---|---|---|-| LR-REM-CONTENTAWARE | Content-aware remove | P1 | 🟡 | `cmd:spot.add` (`mode: remove` / `fill`), `crates/pipeline/src/spots.rs`, `crates/inpaint/src/lib.rs` | Remove heals with an automatic source; Fill synthesises the area from its surroundings (PatchMatch completion over a membrane-blended start, `crates/inpaint`, ported from PhotoCraft; bounded by the spot size limit); no generative fill (M8.4) |+| LR-REM-CONTENTAWARE | Content-aware remove | P1 | 🟡 | `cmd:spot.add` (`mode: remove` / `fill`), `crates/pipeline/src/spots.rs`, `crates/inpaint/src/lib.rs` | Remove replaces the area with an automatic source (healed or cloned); Fill synthesises it from its surroundings (PatchMatch completion over a membrane-blended start, `crates/inpaint`, ported from PhotoCraft; bounded by the spot size limit); no generative fill (M8.4) | | LR-REM-HEAL | Heal | P0 | ✅ | `cmd:spot.add` (`mode: heal`), `crates/pipeline/src/spots.rs` | Zero-feather edges use a hard step; zero-opacity spots preserve all pixels, including exact-radius pixels and image corners | | LR-REM-CLONE | Clone | P0 | ✅ | `cmd:spot.add` (`mode: clone`), `crates/pipeline/src/spots.rs` | Hard-edge source copying and unchanged radius pixels regression-tested at two output sizes | | LR-REM-GEN | Generative remove | OOS | 🚫 | | |@@ -290,9 +289,9 @@ differences and missing bindings are in [ui-parity.md](ui-parity.md#shortcuts). | Id | Feature | Tier | Status | Evidence | Missing / notes | |---|---|---|---|---|---| | LR-MASK-PANEL | Masks panel | P0 | ✅ | `cmd:panel.masking`, `cmd:mask.add`, `cmd:mask.select`, `cmd:mask.rename`, `cmd:mask.duplicate` (`invert`), `cmd:mask.move`, `cmd:mask.visible`, `cmd:mask.delete`, `crates/ui-egui/src/panels/masking.rs` | list with per-mask show/hide eye, double-click rename, right-click menu (duplicate, duplicate and invert, invert, hide, move up/down, rename, delete); fits any panel width (create tiles go to three columns, mask actions wrap, long component names are cut short); no drag-to-reorder |-| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | saliency heuristic, no segmentation model; a photo that carries person mattes (DNG semantic masks: iPhone ProRAW portrait / skin / hair / teeth / glasses mattes) uses them instead (synthetic files only: no corpus file carries person mattes) |-| LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`), `crates/pipeline/src/masks.rs` (`Mattes`), `crates/raw/src/semantic.rs` | heuristic; a photo that carries a sky matte (DNG semantic mask, e.g. iPhone ProRAW) uses it instead, through orientation / crop / lens corrections, preview = export (checked on one CC0 iPhone 12 Pro ProRAW and synthetic files) |-| LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | inverse of the subject mask (heuristic, or the photo's person mattes) |+| LR-MASK-SUBJECT | Select subject | P2 | 🟡 | `cmd:mask.add` (`subject`), `crates/pipeline/src/masks.rs` | with SAM 3 installed or configured remotely, automatically segments the subject using compound concept embeddings (`person, subject, main object, foreground, animal`); falls back to semantic mattes (DNG semantic masks: iPhone ProRAW portrait / skin / hair / teeth / glasses mattes) or saliency heuristic |+| LR-MASK-SKY | Select sky | P2 | 🟡 | `cmd:mask.add` (`sky`), `crates/pipeline/src/masks.rs` (`Mattes`), `crates/raw/src/semantic.rs` | with SAM 3 installed or configured remotely, segments the sky using the text model; falls back to semantic matte or heuristic; a photo that carries a sky matte (DNG semantic mask, e.g. iPhone ProRAW) uses it instead, through orientation / crop / lens corrections, preview = export |+| LR-MASK-BACKGROUND | Select background | P2 | 🟡 | `cmd:mask.add` (`background`) | with SAM 3 installed or configured remotely, segments the inverse of the subject; falls back to inverse heuristic / person mattes | | LR-MASK-OBJECTS | Object selection | P2 | 🟡 | `cmd:mask.add` (`object`, `prompt`), `cmd:mask.objectPoint`, `cmd:mask.refineDetail`, `cmd:segment.prepare`, `cmd:segment.model.status`, `cmd:segment.model.download`, `cmd:segment.model.cancel`, `cmd:app.openSamModelFolder`, `crates/segment`, `crates/engine/src/segment.rs` | SAM 3 in pure Rust (candle; Metal on macOS, CPU elsewhere): Object tile → click to include, ⌥-click to leave out; Describe tile → a text prompt selects every instance ("sky", "the red car"); both also as Add/Subtract/Intersect components; + / − under the selected mask; comma lists (`car, road`); a zoomed-in detail pass for 5–10× finer edges on small objects; per-selection Edge (hard ↔ soft). The model runs on its own worker thread (the UI never waits; panics become errors; a job that fails on the Mac's GPU runs again on the CPU, which then serves the rest of the session, #1974), is unloaded after 10 min idle, and is optional: the segmentation is stored with the mask (288² logits), so renders and exports never need it. Weights are never bundled (SAM License): without them the app offers a consented background download (mirrors, resume, timeouts, SHA-256) — **but no default download location is configured yet** (users set `LIGHTCRAFT_SAM3_MIRRORS` or install by hand: the dialog names the three files, opens the configured model folder with the host opener, preserves setup on an open error, and links docs/ai-masks.md). Not verified against Lightroom's Select Object; no brush/box object mode; the first click on a photo waits for its analysis (~4 s on an M4 Pro, much longer on CPU) | | LR-MASK-PEOPLE | People parts | P2 | ⬜ | | shape exists (no UI): uses the photo's person / skin / hair / teeth / glasses mattes when it carries them (DNG semantic masks), else the subject heuristic | | LR-MASK-LANDSCAPE | Landscape classes | P2 | ⬜ | | shape exists, evaluates empty |