immich-photo-flow/docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md

# immich-photo-flow — Design (M1: Foundation + Trip-Cluster)

- **Date:** 2026-06-27
- **Status:** Approved design, pending implementation plan
- **Scope of this spec:** Milestone M1 only (shared foundation + the `trip-cluster` app). Later milestones are documented as a roadmap, each to get its own spec→plan→build cycle.

## Purpose

A monorepo of small, independent tools that clean up and structure a ~40k-image, 15-year [Immich](https://immich.app) library so it can become travel "memories." This is the **upstream** stage of an existing pipeline:

```
[NEW categorizer: trips + tags]  →  image-rater (pick best → 4+ album)  →  travel-memories (album → Grav blog)
```

Every handoff is through **Immich itself** (tags, albums, ratings) — no app talks to another directly. The categorizer's job is to leave clean **trip tags** (and explicit **non-trip** marks) in Immich for `image-rater` to consume by tag.

### Division of labor (the core principle)

AI/algorithms do what they do well; the human does QA and (later, downstream) content creation.

- The machine **proposes** groupings with a confidence score.
- The human **reviews at the cluster level** (a few dozen candidate trips), not photo-by-photo, mostly in bulk, with attention concentrated where confidence is low.
- Nothing is silently auto-applied to the library. Trust matters with 15 years of memories.

### What M1 explicitly does NOT do

- It does **not** decide which photos *within* a trip are keepers — that is `image-rater`'s job downstream.
- It does **not** fully categorize everyday/non-trip photos beyond marking them as non-trip (deferred to a future milestone).
- It does **not** draft any narrative text (the user owns content creation).

## Context & constraints (from the existing ecosystem)

The two existing apps (`claude-image-rater`, `claude-travel-memories`, both under `/home/mischa/Projects/`) establish strong, consistent conventions this project follows and builds upon:

- **Stack:** Python 3.12 + Flask (factory `create_app()`) + an argparse CLI sharing one package.
- **No-build frontend:** Jinja templates extending `base.html`, **DaisyUI 4 + Tailwind + Alpine.js** (CDN), HTMX for navigation, one `static/app.js`.
- **Single-responsibility modules:** one module is the *only* thing that talks to Immich; one is the *only* thing that talks to Anthropic.
- **Testing:** pytest + `pytest-httpserver` (mock Immich) + Playwright UI tests, **TDD** ("failing test first").
- **Docker:** one container per app, distinct port, `user: ${UID}:${GID}`, state on a mounted volume.
- **Proven UI:** travel-memories' triage screen (grid + ring-selection + keyboard navigation + fullscreen lightbox) was reused by image-rater. This becomes the canonical shared component (see §UI).

Both apps currently **duplicate** near-identical code (`immich.py`, `config.py`, `slugify`, `base.html`, thumbnail proxy route, JSON-state pattern). That duplication is the evidence that extracting a shared foundation is high-value, and that migrating those apps onto it later (M4/M5) is largely mechanical.

### Decisions locked during brainstorming

- **State store: SQLite** (not the JSON-file pattern of the existing apps). A single DB makes the cross-trip filtering this project needs ("all assets in a trip window but missing the trip tag", "everything still unreviewed") far easier. `shared/core` is designed cleanly enough that the other apps *could* migrate onto it later.
- **Immich is the source of truth.** SQLite is the working/review layer; all durable results are written back to Immich as tags.
- **Monorepo with shared packages + per-app containers** (Approach A on a shared foundation). Build the categorizer first as the POC that proves the foundation, then migrate the other apps onto it in later milestones (Option 2 end-state), in milestones — not all at once.
- **GPS is a minority signal.** Most old camera photos lack GPS, so trip detection is **timestamp-first**, anchored by existing tags, refined by GPS where present.
- **Everyday/non-trip photos:** assign trips AND positively mark non-trip so they stop resurfacing as "unreviewed." Full non-trip categorization is deferred.

## Roadmap

The full multi-milestone roadmap (M1–M6), end-state, and cross-cutting decisions live in the authoritative **[`docs/ROADMAP.md`](../../ROADMAP.md)**. In brief: **this spec is M1** — Foundation + `trip-cluster`, the POC that proves the shared foundation. M1.5 (visual-similarity clustering, dependency spiked in M1), M2 (`tag-verify`), M3 (`enrich`, which also introduces `shared/ai`), M4/M5 (migrate the existing apps onto the foundation), and M6 (non-trip categorization) follow, each with its own spec→plan→build cycle.

---

# M1 Design

## Monorepo layout

```
immich-photo-flow/
  pyproject.toml          # workspace; shared/* as editable path deps
  docker-compose.yml      # one service per app (just trip-cluster for M1)
  .env.example            # IMMICH_URL, IMMICH_API_KEY, ANTHROPIC_API_KEY, UID, GID
  README.md
  CLAUDE.md
  shared/
    immich/               # the one true Immich client
    core/                 # SQLite store + domain models + persistence
    ui/                   # base.html, Tailwind/DaisyUI/Alpine/HTMX, Jinja macros, app.js
                          # (shared/ai is deferred to M3 — see Shared packages)
  apps/
    trip-cluster/
      Dockerfile
      categorize.py       # CLI entry point -> app.cli.main()
      app/                # Flask factory, routes, app-specific logic
      tests/
  docs/superpowers/specs/ ...
```

- `shared/*` are local, editable Python packages so apps `import` them directly. Packaging via a workspace (`pyproject.toml` path deps; `uv` workspace optional). This stays close to the existing pip+venv practice while enabling the monorepo.
- Each app owns its Dockerfile and a distinct port. **trip-cluster serves on 8084** (8082 = travel-memories, 8083 = image-rater).
- The SQLite DB and downloaded thumbnails live on a mounted volume so state persists across container restarts and is reachable from the host. `user: ${UID}:${GID}`.

## Shared packages

### `shared/immich`
The single Immich REST client (consolidates the duplicated `immich.py` from both existing apps). `requests.Session` with `x-api-key`. Capabilities needed for M1:
- `resolve_tag_id(name)`, list tags / tag inventory.
- Search assets (paginated `POST /api/search/metadata`) by date range, by tag, and broadly — returning at least: `id`, `originalFileName`, `localDateTime`, GPS lat/lon (when present), city/country (when present), `type`, existing tags, rating.
- `download_thumbnail(asset_id)` (`…/thumbnail?size=preview`).
- `upsert_tag(name)`, `tag_assets(tag_id, asset_ids)` (write-back of trip / `_pipeline/*` tags), plus a helper for the shared `_pipeline/` meta-tag namespace (see Tag conventions).

### `shared/core`
SQLite store + domain dataclasses + persistence (connection, schema/migrations, data-access helpers). Owns the on-disk schema. Designed as a clean data-access layer so other apps can adopt it later.

**Domain model (central unit = Cluster / candidate trip):**

- **Asset** — mirror of an Immich asset: `immich_id`, `taken_at`, `gps_lat`/`gps_lon` (nullable), `place_city`/`place_country` (nullable), `type`, `has_gps`, `thumb_path`, `ingested_at`.
- **Tag** — inventory of Immich tags: `name`, `immich_tag_id`, usage `count` (ingested now; used heavily in M2).
- **AssetTag** — existing Immich tags per asset (denormalized for filtering).
- **Cluster** — a candidate trip: `id`, `start_at`, `end_at`, `count`, `suggested_name`, `confidence`, `kind_guess` (`trip`|`everyday`), `status` (`pending`|`approved`|`non_trip`|`merged`|`split`|`skipped`), `decided_name`, `reviewed_at`, `notes`.
- **ClusterMember** — asset↔cluster link with `member_confidence` and `is_outlier`.
- **WritebackLog** — every change pushed to Immich (`asset_id`, `action`, `tag`, `result`, `applied_at`) for idempotency.
- **Meta** — last-ingest timestamp, run params, schema version.

**SQLite tables:** `assets`, `asset_tags`, `tags`, `clusters`, `cluster_members`, `writeback_log`, `meta`. Single DB file on the mounted volume.

### `shared/ai` — deferred to M3
**Not built in M1.** trip-cluster is algorithm-first (near-zero AI cost), so M1 has no consumer for an Anthropic wrapper; building it now would freeze its interface before the M3 enrich requirements that actually shape it. `shared/ai` is introduced in **M3 (enrich)** — an Anthropic **batch** Messages API wrapper using the official `anthropic` SDK, carrying over image-rater's proven conventions (per-criterion judgments, `confidence`, score floors, chunking ≤50/batch, default model `claude-haiku-4-5`). The `ANTHROPIC_API_KEY` plumbing stays optional in M1.

### `shared/ui`
The shared visual foundation, extracted from the proven travel-memories/image-rater templates:
- `base.html` (DaisyUI 4 + Tailwind + Alpine.js + HTMX via CDN, navbar, content block).
- The canonical **grid + fullscreen lightbox** component: thumbnail grid with ring-selection, **arrow-key navigation**, **full-screen view** (arrows = prev/next, Esc = close). *(First-class requirement.)*
- Reusable Jinja macros (image grid, lightbox, confidence/status badges, approve/reject controls) and shared `app.js` Alpine components.

## The `trip-cluster` app (M1 deliverable)

### Workflow

```
categorize ingest [--from DATE --to DATE | --tag NAME | --subset N]   # 1. pull → SQLite + thumbs
categorize cluster [--gap-threshold ...]                              # 2. build candidate trips
categorize serve                                                      # 3. review UI on :8084
                                                                      #    (approve / non-trip / split / merge)
categorize apply                                                      # 4. write approved tags back to Immich
```

(`serve` may also trigger `apply` per-cluster via a UI button, plus a batch "apply all approved" action; CLI `apply` is the headless equivalent.)

### 1. Ingest (automatic, scopeable)
Pull assets from Immich via `shared/immich`, upsert metadata into SQLite, download thumbnails. **Incremental & idempotent** (fetch only changed-since-last via Immich `updatedAt`; upsert by `immich_id`).

**Read-back of pipeline state:** ingest also reads each asset's existing tags, including `_pipeline/*`. Assets carrying `_pipeline/processed` are marked already-adjudicated in SQLite, so the working DB can be **rebuilt from Immich** after a loss and review resumes without redoing finished work.

**Scopeable from day one** — the mechanism for the vet-first plan. Instead of forcing a full 40k pull, `ingest` accepts a bounded scope:
- `--from / --to` (date window) → e.g. the last trip plus surrounding everyday photos;
- `--tag NAME` → a single already-tagged trip;
- `--subset N` → a cap.

This lets the tool be validated cheaply before widening to the backlog over time. **Validation gate (M1):** validate on a deliberately *hard* sample — a GPS-poor, multi-year, low-density slice (e.g. a well-remembered old trip plus its surrounding everyday photos), **not** the most recent trip (the easy case: phone-era, GPS-rich, already tagged). Widening is gated on a **quantified acceptance bar** measured against a small hand-labelled set: trip-boundary precision/recall, coverage-flag recall, and an acceptable over-split / false-cluster rate. Until that bar is met on the hard sample, the backlog is not widened.

### 2. Cluster (automatic) — signal hierarchy
Pure, algorithmic, **no API calls** (keeps the POC nearly free). Produces candidate trips, each with a `suggested_name`, `confidence`, and `kind_guess`:

1. **Existing trip tag** → authoritative seed; its assets form a confirmed cluster (still shown, to verify *completeness*). Respects the user's existing trip-tag convention.
2. **Timestamp gap clustering** → primary structure: sort by `taken_at`, split where the inter-photo gap is large relative to local cadence. A single global threshold fails across a 15-year density gradient (a sparse old trip has multi-day intra-trip gaps; a dense recent everyday period has hour-scale inter-day gaps), so the split is **density-adaptive** — threshold relative to local photo cadence / a per-era percentile — validated with unit fixtures spanning both a sparse-old and a dense-recent regime.
3. **Location anchors** (existing location tags like "Kiev" + GPS when present) → refine boundaries, propose names.
4. **Coverage detection** → assets *inside* a confirmed trip's time window but *missing* its tag are flagged "likely belongs here" (the completeness gap); assets carrying a trip tag but *outside* their cluster are flagged as outliers.
5. **Visual similarity** → **clustering deferred to M1.5; dependency verified in M1.** Trips are defined by time + place, not visual likeness, so the signals above resolve the large majority; visual similarity helps the narrow-but-important case of the GPS-poor old library, where visual continuity may be one of the few secondary signals. pHash is the wrong tool (near-duplicates, not trip-level similarity); CLIP is right, but Immich's REST API does not cleanly expose raw embedding vectors. **Viable path: read-only access to Immich's Postgres pgvector embeddings** (do nearest-neighbor/clustering ourselves). Because this is the rescue signal for the hardest case, **M1 includes a read-only feasibility spike** — confirm the embeddings are readable and pin the table/column shape and DB access/credential against the live Immich version (no clustering build). The clustering itself is **M1.5** (see roadmap), so it sits in a scheduled near-term milestone rather than floating indefinitely.

**Confidence & kind_guess** (echoing image-rater's confidence/floor approach): tight time window + existing trip tag + consistent location → high confidence; sparse, untagged, no GPS → low ("needs your eye"). Low-volume scattered clusters → `kind_guess = everyday` (suggested non-trip).

### 3. Review (human, cluster-level)
**Layout — master/detail.** A left rail lists **candidate clusters sorted by "needs attention"** (low confidence first), each with confidence / status / count badges; selecting one opens a detail pane with its editable suggested name, action controls, and the canonical **grid + lightbox** (ring-selection, arrow-key navigation *within the grid*, full-screen view, Esc to close). Moving between clusters uses its own shortcut so arrow keys stay bound to the grid.

Per-cluster actions:
- **Approve trip** (confirm + tweak name)
- **Mark non-trip**
- **Split** — select a boundary asset in the chronologically-ordered grid and choose "split before here"; the cluster partitions at that timestamp into two clusters, both with editable names. Reuses the existing ring-selection / arrow-key focus.
- **Merge** — surfaces the cluster's **chronological** neighbour(s) (prev/next by time, computed from a temporal index, *independent of the needs-attention list sort*) with a preview of the combined range; the user confirms which to absorb.
- **Skip**

**Per-asset refinement (coverage flags).** Within a cluster's grid, assets flagged "likely belongs here" (the completeness gap) appear with a distinct badge and an **include** toggle; flagged **outliers** appear with an **exclude** toggle. Include/exclude updates `cluster_members` before write-back, so the algorithm's completeness work is actionable rather than informational.

**Keyboard-first.** Cluster-level actions mirror the sibling convention (e.g. `A` approve, `N` non-trip, `S` split, `M` merge, `X` skip; arrows navigate within the grid; `[` / `]` move between clusters), listed in a help footer as image-rater/travel-memories do. *(First-class requirement.)*

**Empty states.** Because ingest is scopeable and assets can already be `_pipeline/processed`, a `cluster` run can legitimately yield nothing. `serve` distinguishes three cases, each naming the next step: *no clusters yet* (run ingest/cluster), *all clusters reviewed*, and *this scope produced no clusters*.

To keep it fast: high-confidence clusters arrive **pre-filled**, and an **"approve all high-confidence"** bulk action lets the user rubber-stamp the obvious cases, concentrating attention on the fuzzy ones. Every decision **persists immediately** to SQLite, so review is resumable across sessions (see Write-back for what survives loss of the SQLite layer itself).

### 4. Write-back (to Immich, idempotent)
On approval (per-cluster or batch `apply`):
- **Approve** → write the trip tag (via `upsert_tag` + `tag_assets`) to all member assets, respecting the existing **content** trip-tag convention (trip names are user-facing, not namespaced).
- **Mark non-trip** → apply `_pipeline/non-trip` so those assets are filtered out and never resurface as unreviewed.
- **Mark processed** → every adjudicated asset (trip-assigned, non-trip, **or** reviewed-and-skipped) gets `_pipeline/processed`. This is the durable "done" flag in the source of truth: it captures the reviewed-but-untagged case, and lets a fresh `ingest` re-derive what's already handled even if the SQLite working DB is lost (see Ingest read-back).
- **Idempotent**: `writeback_log` records applied changes; re-runs skip what's already done. **Explicit confirmation before any write** (mirrors image-rater's export safety).
- **Partial-failure reporting**: a batch "apply all approved" reports **per-cluster results** (succeeded / failed with reason) from `writeback_log`; clusters that fail stay in their approved-but-unapplied state so a re-run retries only those. The UI surfaces this as inline status badges plus a summary, so the user always knows what landed in the library.

All durable state lands in **Immich**; SQLite is the working/review layer. **Only applied (tag-written) decisions are rebuildable from Immich** — approved-but-unapplied clusters, in-progress split/merge, edited names and notes live only in SQLite. So the resumability guarantee holds against loss of SQLite only for applied work: decisions should be **applied promptly on approval** (or the SQLite working DB backed up), and the rebuild-from-Immich path recovers everything already written as tags.

### Tag conventions (shared across all apps)
Two clearly separated kinds of tags:

- **Content / organizational tags** — trip names ("Italy 2019"), locations ("Kiev"), people. User-facing, part of the existing convention, **never namespaced**. trip-cluster must **read and respect** the existing trip-tag convention rather than invent a parallel one.
- **Pipeline meta-tags** — everything the tooling generates as machinery, nested under a single parent **`_pipeline/`** so the whole set can be removed by deleting the parent and never clutters the tag list. Defined as a **shared convention in `shared/immich`** and used by every app in the monorepo:
  - `_pipeline/processed` — categorizer: asset adjudicated (any outcome)
  - `_pipeline/non-trip` — categorizer: everyday/noise
  - `_pipeline/ai-rating/<0-5>` — image-rater's `ai-rating/<n>` moves under this root when migrated (M4)
  - room for future meta-tags (review-state, etc.)

**Reconciliation caveat:** before writing, **inspect the live Immich instance** and reconcile with anything already present (e.g. image-rater's existing un-namespaced `ai-rating/<n>`) rather than blindly creating duplicates.

## App shape & conventions
- `categorize` CLI (argparse subcommands: `ingest` · `cluster` · `serve` · `apply`) + Flask factory `create_app()`.
- Jinja + DaisyUI/Tailwind/Alpine/HTMX via `shared/ui`, **no build step**, Immich thumbnail proxy route, serves on **8084**.
- Env vars (same names as existing apps): `IMMICH_URL`, `IMMICH_API_KEY`, `ANTHROPIC_API_KEY` (Anthropic optional for M1), loaded from `.env`.

## Testing strategy
TDD throughout, mirroring the existing apps:
- **Unit:** clustering algorithm as a pure function (timestamps/tags/GPS → clusters) — deterministic and easy to assert; SQLite store; config; Immich client (mocked via `pytest-httpserver`).
- **Route/UI:** the review flow (approve / non-trip / split / merge, immediate persistence, idempotent write-back) with Immich mocked.
- **Playwright UI tests:** grid + arrow-key navigation + full-screen lightbox behavior, mirroring travel-memories/image-rater.
- Shared packages tested independently of the apps.

## Cost notes
M1 is algorithm-first, so trip-cluster makes **essentially no Anthropic calls** — clustering is local timestamp/tag/GPS math. AI cost becomes relevant only in M3 (enrich), where image-input tokens dominate and the batch API (~50% cheaper, chunked ≤50) plus the Haiku default keep it modest (image-rater reference: ~700 images on Haiku ≈ $1).

## Out of scope for M1
- Choosing keepers within a trip (image-rater).
- Full non-trip/everyday categorization (M6).
- Migrating the existing apps onto the shared foundation (M4/M5).
- Geocoding / GPS backfill / AI captioning (M3).
- Narrative text drafting (human-owned; downstream).

---

## Deferred / Open Questions

### From 2026-06-27 ce-doc-review

- **Cluster-count blow-up from everyday photos (F4).** Cluster-level QA assumes "a few dozen" candidate trips, but clustering everyday/non-trip photos with per-gap splitting could mint hundreds–thousands of low-confidence clusters that "approve all high-confidence" won't relieve. The concern is real, but the reviewer's proposed pre-pass (collapsing everyday spans into bulk buckets) is unconvincing; a candidate alternative is ordering/filtering clusters by **date-span** (a multi-week span ranks above a 2-day span). Crucially, the actual cluster output on real Immich data is unknown to both reviewer and author — resolve by **observing real behaviour on a representative subset first**, then choose the surfacing/collapsing strategy.