docs: align M1 design spec with ROADMAP (M1.5, shared/ai -> M3)

docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md

@@ -51,7 +51,7 @@ Both apps currently **duplicate** near-identical code (`immich.py`, `config.py`,
 
 ## Roadmap
 
-The full multi-milestone roadmap (M1–M6), end-state, and cross-cutting decisions live in the authoritative **[`docs/ROADMAP.md`](../../ROADMAP.md)**. In brief: **this spec is M1** — Foundation + `trip-cluster`, the POC that proves the shared foundation. M2 (`tag-verify`), M3 (`enrich`), M4/M5 (migrate the existing apps onto the foundation), and M6 (non-trip categorization) follow, each with its own spec→plan→build cycle.
+The full multi-milestone roadmap (M1–M6), end-state, and cross-cutting decisions live in the authoritative **[`docs/ROADMAP.md`](../../ROADMAP.md)**. In brief: **this spec is M1** — Foundation + `trip-cluster`, the POC that proves the shared foundation. M1.5 (visual-similarity clustering, dependency spiked in M1), M2 (`tag-verify`), M3 (`enrich`, which also introduces `shared/ai`), M4/M5 (migrate the existing apps onto the foundation), and M6 (non-trip categorization) follow, each with its own spec→plan→build cycle.
 
 ---
 
@@ -69,8 +69,8 @@ immich-photo-flow/
   shared/
     immich/               # the one true Immich client
     core/                 # SQLite store + domain models + persistence
-    ai/                   # Anthropic batch wrapper (scaffolded; light/unused in M1)
     ui/                   # base.html, Tailwind/DaisyUI/Alpine/HTMX, Jinja macros, app.js
+                          # (shared/ai is deferred to M3 — see Shared packages)
   apps/
     trip-cluster/
       Dockerfile
@@ -108,8 +108,8 @@ SQLite store + domain dataclasses + persistence (connection, schema/migrations,
 
 **SQLite tables:** `assets`, `asset_tags`, `tags`, `clusters`, `cluster_members`, `writeback_log`, `meta`. Single DB file on the mounted volume.
 
-### `shared/ai`
-Anthropic **batch** Messages API wrapper using the official `anthropic` SDK, carrying over image-rater's proven conventions (per-criterion judgments, `confidence`, score floors, chunking ≤50/batch, default model `claude-haiku-4-5`). **Scaffolded but barely used in M1** — trip-cluster is algorithm-first (near-zero AI cost). It earns its keep in M3 (enrich).
+### `shared/ai` — deferred to M3
+**Not built in M1.** trip-cluster is algorithm-first (near-zero AI cost), so M1 has no consumer for an Anthropic wrapper; building it now would freeze its interface before the M3 enrich requirements that actually shape it. `shared/ai` is introduced in **M3 (enrich)** — an Anthropic **batch** Messages API wrapper using the official `anthropic` SDK, carrying over image-rater's proven conventions (per-criterion judgments, `confidence`, score floors, chunking ≤50/batch, default model `claude-haiku-4-5`). The `ANTHROPIC_API_KEY` plumbing stays optional in M1.
 
 ### `shared/ui`
 The shared visual foundation, extracted from the proven travel-memories/image-rater templates:
@@ -141,28 +141,36 @@ Pull assets from Immich via `shared/immich`, upsert metadata into SQLite, downlo
 - `--tag NAME` → a single already-tagged trip;
 - `--subset N` → a cap.
 
-This lets the whole tool be validated on the last trip for near-zero cost before widening to the backlog over time.
+This lets the tool be validated cheaply before widening to the backlog over time. **Validation gate (M1):** validate on a deliberately *hard* sample — a GPS-poor, multi-year, low-density slice (e.g. a well-remembered old trip plus its surrounding everyday photos), **not** the most recent trip (the easy case: phone-era, GPS-rich, already tagged). Widening is gated on a **quantified acceptance bar** measured against a small hand-labelled set: trip-boundary precision/recall, coverage-flag recall, and an acceptable over-split / false-cluster rate. Until that bar is met on the hard sample, the backlog is not widened.
 
 ### 2. Cluster (automatic) — signal hierarchy
 Pure, algorithmic, **no API calls** (keeps the POC nearly free). Produces candidate trips, each with a `suggested_name`, `confidence`, and `kind_guess`:
 
 1. **Existing trip tag** → authoritative seed; its assets form a confirmed cluster (still shown, to verify *completeness*). Respects the user's existing trip-tag convention.
-2. **Timestamp gap clustering** → primary structure: sort by `taken_at`, split where the inter-photo gap exceeds a tunable threshold.
+2. **Timestamp gap clustering** → primary structure: sort by `taken_at`, split where the inter-photo gap is large relative to local cadence. A single global threshold fails across a 15-year density gradient (a sparse old trip has multi-day intra-trip gaps; a dense recent everyday period has hour-scale inter-day gaps), so the split is **density-adaptive** — threshold relative to local photo cadence / a per-era percentile — validated with unit fixtures spanning both a sparse-old and a dense-recent regime.
 3. **Location anchors** (existing location tags like "Kiev" + GPS when present) → refine boundaries, propose names.
 4. **Coverage detection** → assets *inside* a confirmed trip's time window but *missing* its tag are flagged "likely belongs here" (the completeness gap); assets carrying a trip tag but *outside* their cluster are flagged as outliers.
-5. **Visual similarity** → **deferred from M1; leading post-M1 enhancement.** Trips are defined by time + place, not visual likeness, so the signals above resolve the large majority; visual similarity only helps a narrow case (an ambiguous time gap where two bursts may be one trip). pHash is the wrong tool (it finds near-duplicates, not trip-level similarity). CLIP is the right tool, but Immich's REST API does not cleanly expose raw embedding vectors. **Viable path: read-only access to Immich's Postgres pgvector embeddings** (do nearest-neighbor/clustering ourselves), pending a feasibility check against the live Immich version. Especially valuable here because the old library is GPS-poor, so visual continuity may be one of the few secondary signals for those photos.
+5. **Visual similarity** → **clustering deferred to M1.5; dependency verified in M1.** Trips are defined by time + place, not visual likeness, so the signals above resolve the large majority; visual similarity helps the narrow-but-important case of the GPS-poor old library, where visual continuity may be one of the few secondary signals. pHash is the wrong tool (near-duplicates, not trip-level similarity); CLIP is right, but Immich's REST API does not cleanly expose raw embedding vectors. **Viable path: read-only access to Immich's Postgres pgvector embeddings** (do nearest-neighbor/clustering ourselves). Because this is the rescue signal for the hardest case, **M1 includes a read-only feasibility spike** — confirm the embeddings are readable and pin the table/column shape and DB access/credential against the live Immich version (no clustering build). The clustering itself is **M1.5** (see roadmap), so it sits in a scheduled near-term milestone rather than floating indefinitely.
 
 **Confidence & kind_guess** (echoing image-rater's confidence/floor approach): tight time window + existing trip tag + consistent location → high confidence; sparse, untagged, no GPS → low ("needs your eye"). Low-volume scattered clusters → `kind_guess = everyday` (suggested non-trip).
 
 ### 3. Review (human, cluster-level)
-A review screen lists **candidate trips sorted by "needs attention"** (low confidence first). Per cluster: thumbnail grid (with the shared grid+lightbox / arrow-key / full-screen component), editable suggested name, and actions:
+**Layout — master/detail.** A left rail lists **candidate clusters sorted by "needs attention"** (low confidence first), each with confidence / status / count badges; selecting one opens a detail pane with its editable suggested name, action controls, and the canonical **grid + lightbox** (ring-selection, arrow-key navigation *within the grid*, full-screen view, Esc to close). Moving between clusters uses its own shortcut so arrow keys stay bound to the grid.
+
+Per-cluster actions:
 - **Approve trip** (confirm + tweak name)
 - **Mark non-trip**
-- **Split** (break one cluster into two)
-- **Merge adjacent** (combine with a neighbor)
+- **Split** — select a boundary asset in the chronologically-ordered grid and choose "split before here"; the cluster partitions at that timestamp into two clusters, both with editable names. Reuses the existing ring-selection / arrow-key focus.
+- **Merge** — surfaces the cluster's **chronological** neighbour(s) (prev/next by time, computed from a temporal index, *independent of the needs-attention list sort*) with a preview of the combined range; the user confirms which to absorb.
 - **Skip**
 
-To keep it fast: high-confidence clusters arrive **pre-filled**, and an **"approve all high-confidence"** bulk action lets the user rubber-stamp the obvious cases, concentrating attention on the fuzzy ones. Every decision **persists immediately** to SQLite (resumable: closing and reopening resumes exactly where the user left off).
+**Per-asset refinement (coverage flags).** Within a cluster's grid, assets flagged "likely belongs here" (the completeness gap) appear with a distinct badge and an **include** toggle; flagged **outliers** appear with an **exclude** toggle. Include/exclude updates `cluster_members` before write-back, so the algorithm's completeness work is actionable rather than informational.
+
+**Keyboard-first.** Cluster-level actions mirror the sibling convention (e.g. `A` approve, `N` non-trip, `S` split, `M` merge, `X` skip; arrows navigate within the grid; `[` / `]` move between clusters), listed in a help footer as image-rater/travel-memories do. *(First-class requirement.)*
+
+**Empty states.** Because ingest is scopeable and assets can already be `_pipeline/processed`, a `cluster` run can legitimately yield nothing. `serve` distinguishes three cases, each naming the next step: *no clusters yet* (run ingest/cluster), *all clusters reviewed*, and *this scope produced no clusters*.
+
+To keep it fast: high-confidence clusters arrive **pre-filled**, and an **"approve all high-confidence"** bulk action lets the user rubber-stamp the obvious cases, concentrating attention on the fuzzy ones. Every decision **persists immediately** to SQLite, so review is resumable across sessions (see Write-back for what survives loss of the SQLite layer itself).
 
 ### 4. Write-back (to Immich, idempotent)
 On approval (per-cluster or batch `apply`):
@@ -170,8 +178,9 @@ On approval (per-cluster or batch `apply`):
 - **Mark non-trip** → apply `_pipeline/non-trip` so those assets are filtered out and never resurface as unreviewed.
 - **Mark processed** → every adjudicated asset (trip-assigned, non-trip, **or** reviewed-and-skipped) gets `_pipeline/processed`. This is the durable "done" flag in the source of truth: it captures the reviewed-but-untagged case, and lets a fresh `ingest` re-derive what's already handled even if the SQLite working DB is lost (see Ingest read-back).
 - **Idempotent**: `writeback_log` records applied changes; re-runs skip what's already done. **Explicit confirmation before any write** (mirrors image-rater's export safety).
+- **Partial-failure reporting**: a batch "apply all approved" reports **per-cluster results** (succeeded / failed with reason) from `writeback_log`; clusters that fail stay in their approved-but-unapplied state so a re-run retries only those. The UI surfaces this as inline status badges plus a summary, so the user always knows what landed in the library.
 
-All durable state lands in **Immich**; SQLite remains the working/review layer that can be rebuilt from Immich tags.
+All durable state lands in **Immich**; SQLite is the working/review layer. **Only applied (tag-written) decisions are rebuildable from Immich** — approved-but-unapplied clusters, in-progress split/merge, edited names and notes live only in SQLite. So the resumability guarantee holds against loss of SQLite only for applied work: decisions should be **applied promptly on approval** (or the SQLite working DB backed up), and the rebuild-from-Immich path recovers everything already written as tags.
 
 ### Tag conventions (shared across all apps)
 Two clearly separated kinds of tags:
@@ -206,3 +215,11 @@ M1 is algorithm-first, so trip-cluster makes **essentially no Anthropic calls**
 - Migrating the existing apps onto the shared foundation (M4/M5).
 - Geocoding / GPS backfill / AI captioning (M3).
 - Narrative text drafting (human-owned; downstream).
+
+---
+
+## Deferred / Open Questions
+
+### From 2026-06-27 ce-doc-review
+
+- **Cluster-count blow-up from everyday photos (F4).** Cluster-level QA assumes "a few dozen" candidate trips, but clustering everyday/non-trip photos with per-gap splitting could mint hundreds–thousands of low-confidence clusters that "approve all high-confidence" won't relieve. The concern is real, but the reviewer's proposed pre-pass (collapsing everyday spans into bulk buckets) is unconvincing; a candidate alternative is ordering/filtering clusters by **date-span** (a multi-week span ranks above a 2-day span). Crucially, the actual cluster output on real Immich data is unknown to both reviewer and author — resolve by **observing real behaviour on a representative subset first**, then choose the surfacing/collapsing strategy.