immich-photo-flow/docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md

immich-photo-flow — Design (M1: Foundation + Trip-Cluster)

• Date: 2026-06-27
• Status: Approved design, pending implementation plan
• Scope of this spec: Milestone M1 only (shared foundation + the trip-cluster app). Later milestones are documented as a roadmap, each to get its own spec→plan→build cycle.

Purpose

A monorepo of small, independent tools that clean up and structure a ~40k-image, 15-year Immich library so it can become travel "memories." This is the upstream stage of an existing pipeline:

[NEW categorizer: trips + tags]  →  image-rater (pick best → 4+ album)  →  travel-memories (album → Grav blog)

Every handoff is through Immich itself (tags, albums, ratings) — no app talks to another directly. The categorizer's job is to leave clean trip tags (and explicit non-trip marks) in Immich for image-rater to consume by tag.

Division of labor (the core principle)

AI/algorithms do what they do well; the human does QA and (later, downstream) content creation.

• The machine proposes groupings with a confidence score.
• The human reviews at the cluster level (a few dozen candidate trips), not photo-by-photo, mostly in bulk, with attention concentrated where confidence is low.
• Nothing is silently auto-applied to the library. Trust matters with 15 years of memories.

What M1 explicitly does NOT do

• It does not decide which photos within a trip are keepers — that is image-rater's job downstream.
• It does not fully categorize everyday/non-trip photos beyond marking them as non-trip (deferred to a future milestone).
• It does not draft any narrative text (the user owns content creation).

Context & constraints (from the existing ecosystem)

The two existing apps (claude-image-rater, claude-travel-memories, both under /home/mischa/Projects/) establish strong, consistent conventions this project follows and builds upon:

• Stack: Python 3.12 + Flask (factory create_app()) + an argparse CLI sharing one package.
• No-build frontend: Jinja templates extending base.html, DaisyUI 4 + Tailwind + Alpine.js (CDN), HTMX for navigation, one static/app.js.
• Single-responsibility modules: one module is the only thing that talks to Immich; one is the only thing that talks to Anthropic.
• Testing: pytest + pytest-httpserver (mock Immich) + Playwright UI tests, TDD ("failing test first").
• Docker: one container per app, distinct port, user: ${UID}:${GID}, state on a mounted volume.
• Proven UI: travel-memories' triage screen (grid + ring-selection + keyboard navigation + fullscreen lightbox) was reused by image-rater. This becomes the canonical shared component (see §UI).

Both apps currently duplicate near-identical code (immich.py, config.py, slugify, base.html, thumbnail proxy route, JSON-state pattern). That duplication is the evidence that extracting a shared foundation is high-value, and that migrating those apps onto it later (M4/M5) is largely mechanical.

Decisions locked during brainstorming

• State store: SQLite (not the JSON-file pattern of the existing apps). A single DB makes the cross-trip filtering this project needs ("all assets in a trip window but missing the trip tag", "everything still unreviewed") far easier. shared/core is designed cleanly enough that the other apps could migrate onto it later.
• Immich is the source of truth. SQLite is the working/review layer; all durable results are written back to Immich as tags.
• Monorepo with shared packages + per-app containers (Approach A on a shared foundation). Build the categorizer first as the POC that proves the foundation, then migrate the other apps onto it in later milestones (Option 2 end-state), in milestones — not all at once.
• GPS is a minority signal. Most old camera photos lack GPS, so trip detection is timestamp-first, anchored by existing tags, refined by GPS where present.
• Everyday/non-trip photos: assign trips AND positively mark non-trip so they stop resurfacing as "unreviewed." Full non-trip categorization is deferred.

Roadmap

The full multi-milestone roadmap (M1–M6), end-state, and cross-cutting decisions live in the authoritative docs/ROADMAP.md. In brief: this spec is M1 — Foundation + trip-cluster, the POC that proves the shared foundation. M1.5 (visual-similarity clustering, dependency spiked in M1), M2 (tag-verify), M3 (enrich, which also introduces shared/ai), M4/M5 (migrate the existing apps onto the foundation), and M6 (non-trip categorization) follow, each with its own spec→plan→build cycle.

---

M1 Design

Monorepo layout

immich-photo-flow/
  pyproject.toml          # workspace; shared/* as editable path deps
  docker-compose.yml      # one service per app (just trip-cluster for M1)
  .env.example            # IMMICH_URL, IMMICH_API_KEY, ANTHROPIC_API_KEY, UID, GID
  README.md
  CLAUDE.md
  shared/
    immich/               # the one true Immich client
    core/                 # SQLite store + domain models + persistence
    ui/                   # base.html, Tailwind/DaisyUI/Alpine/HTMX, Jinja macros, app.js
                          # (shared/ai is deferred to M3 — see Shared packages)
  apps/
    trip-cluster/
      Dockerfile
      categorize.py       # CLI entry point -> app.cli.main()
      app/                # Flask factory, routes, app-specific logic
      tests/
  docs/superpowers/specs/ ...

• shared/* are local, editable Python packages so apps import them directly. Packaging via a workspace (pyproject.toml path deps; uv workspace optional). This stays close to the existing pip+venv practice while enabling the monorepo.
• Each app owns its Dockerfile and a distinct port. trip-cluster serves on 8084 (8082 = travel-memories, 8083 = image-rater).
• The SQLite DB and downloaded thumbnails live on a mounted volume so state persists across container restarts and is reachable from the host. user: ${UID}:${GID}.

Shared packages

shared/immich

The single Immich REST client (consolidates the duplicated immich.py from both existing apps). requests.Session with x-api-key. Capabilities needed for M1:

• resolve_tag_id(name), list tags / tag inventory.
• Search assets (paginated POST /api/search/metadata) by date range, by tag, and broadly — returning at least: id, originalFileName, localDateTime, GPS lat/lon (when present), city/country (when present), type, existing tags, rating.
• download_thumbnail(asset_id) (…/thumbnail?size=preview).
• upsert_tag(name), tag_assets(tag_id, asset_ids) (write-back of trip / _pipeline/* tags), plus a helper for the shared _pipeline/ meta-tag namespace (see Tag conventions).

shared/core

SQLite store + domain dataclasses + persistence (connection, schema/migrations, data-access helpers). Owns the on-disk schema. Designed as a clean data-access layer so other apps can adopt it later.

Domain model (central unit = Cluster / candidate trip):

• Asset — mirror of an Immich asset: immich_id, taken_at, gps_lat/gps_lon (nullable), place_city/place_country (nullable), type, has_gps, thumb_path, ingested_at.
• Tag — inventory of Immich tags: name, immich_tag_id, usage count (ingested now; used heavily in M2).
• AssetTag — existing Immich tags per asset (denormalized for filtering).
• Cluster — a candidate trip: id, start_at, end_at, count, suggested_name, confidence, kind_guess (trip|everyday), status (pending|approved|non_trip|merged|split|skipped), decided_name, reviewed_at, notes.
• ClusterMember — asset↔cluster link with member_confidence and is_outlier.
• WritebackLog — every change pushed to Immich (asset_id, action, tag, result, applied_at) for idempotency.
• Meta — last-ingest timestamp, run params, schema version.

SQLite tables: assets, asset_tags, tags, clusters, cluster_members, writeback_log, meta. Single DB file on the mounted volume.

shared/ai — deferred to M3

Not built in M1. trip-cluster is algorithm-first (near-zero AI cost), so M1 has no consumer for an Anthropic wrapper; building it now would freeze its interface before the M3 enrich requirements that actually shape it. shared/ai is introduced in M3 (enrich) — an Anthropic batch Messages API wrapper using the official anthropic SDK, carrying over image-rater's proven conventions (per-criterion judgments, confidence, score floors, chunking ≤50/batch, default model claude-haiku-4-5). The ANTHROPIC_API_KEY plumbing stays optional in M1.

shared/ui

The shared visual foundation, extracted from the proven travel-memories/image-rater templates:

• base.html (DaisyUI 4 + Tailwind + Alpine.js + HTMX via CDN, navbar, content block).
• The canonical grid + fullscreen lightbox component: thumbnail grid with ring-selection, arrow-key navigation, full-screen view (arrows = prev/next, Esc = close). (First-class requirement.)
• Reusable Jinja macros (image grid, lightbox, confidence/status badges, approve/reject controls) and shared app.js Alpine components.

The trip-cluster app (M1 deliverable)

Workflow

categorize ingest [--from DATE --to DATE | --tag NAME | --subset N]   # 1. pull → SQLite + thumbs
categorize cluster [--gap-threshold ...]                              # 2. build candidate trips
categorize serve                                                      # 3. review UI on :8084
                                                                      #    (approve / non-trip / split / merge)
categorize apply                                                      # 4. write approved tags back to Immich

(serve may also trigger apply per-cluster via a UI button, plus a batch "apply all approved" action; CLI apply is the headless equivalent.)

1. Ingest (automatic, scopeable)

Pull assets from Immich via shared/immich, upsert metadata into SQLite, download thumbnails. Incremental & idempotent (fetch only changed-since-last via Immich updatedAt; upsert by immich_id).

Read-back of pipeline state: ingest also reads each asset's existing tags, including _pipeline/*. Assets carrying _pipeline/processed are marked already-adjudicated in SQLite, so the working DB can be rebuilt from Immich after a loss and review resumes without redoing finished work.

Scopeable from day one — the mechanism for the vet-first plan. Instead of forcing a full 40k pull, ingest accepts a bounded scope:

• --from / --to (date window) → e.g. the last trip plus surrounding everyday photos;
• --tag NAME → a single already-tagged trip;
• --subset N → a cap.

This lets the tool be validated cheaply before widening to the backlog over time. Validation gate (M1): validate on a deliberately hard sample — a GPS-poor, multi-year, low-density slice (e.g. a well-remembered old trip plus its surrounding everyday photos), not the most recent trip (the easy case: phone-era, GPS-rich, already tagged). Widening is gated on a quantified acceptance bar measured against a small hand-labelled set: trip-boundary precision/recall, coverage-flag recall, and an acceptable over-split / false-cluster rate. Until that bar is met on the hard sample, the backlog is not widened.

2. Cluster (automatic) — signal hierarchy

Pure, algorithmic, no API calls (keeps the POC nearly free). Produces candidate trips, each with a suggested_name, confidence, and kind_guess:

1. Existing trip tag → authoritative seed; its assets form a confirmed cluster (still shown, to verify completeness). Respects the user's existing trip-tag convention.
2. Timestamp gap clustering → primary structure: sort by taken_at, split where the inter-photo gap is large relative to local cadence. A single global threshold fails across a 15-year density gradient (a sparse old trip has multi-day intra-trip gaps; a dense recent everyday period has hour-scale inter-day gaps), so the split is density-adaptive — threshold relative to local photo cadence / a per-era percentile — validated with unit fixtures spanning both a sparse-old and a dense-recent regime.
3. Location anchors (existing location tags like "Kiev" + GPS when present) → refine boundaries, propose names.
4. Coverage detection → assets inside a confirmed trip's time window but missing its tag are flagged "likely belongs here" (the completeness gap); assets carrying a trip tag but outside their cluster are flagged as outliers.
5. Visual similarity → clustering deferred to M1.5; dependency verified in M1. Trips are defined by time + place, not visual likeness, so the signals above resolve the large majority; visual similarity helps the narrow-but-important case of the GPS-poor old library, where visual continuity may be one of the few secondary signals. pHash is the wrong tool (near-duplicates, not trip-level similarity); CLIP is right, but Immich's REST API does not cleanly expose raw embedding vectors. Viable path: read-only access to Immich's Postgres pgvector embeddings (do nearest-neighbor/clustering ourselves). Because this is the rescue signal for the hardest case, M1 includes a read-only feasibility spike — confirm the embeddings are readable and pin the table/column shape and DB access/credential against the live Immich version (no clustering build). The clustering itself is M1.5 (see roadmap), so it sits in a scheduled near-term milestone rather than floating indefinitely.

Confidence & kind_guess (echoing image-rater's confidence/floor approach): tight time window + existing trip tag + consistent location → high confidence; sparse, untagged, no GPS → low ("needs your eye"). Low-volume scattered clusters → kind_guess = everyday (suggested non-trip).

3. Review (human, cluster-level)

Layout — master/detail. A left rail lists candidate clusters sorted by "needs attention" (low confidence first), each with confidence / status / count badges; selecting one opens a detail pane with its editable suggested name, action controls, and the canonical grid + lightbox (ring-selection, arrow-key navigation within the grid, full-screen view, Esc to close). Moving between clusters uses its own shortcut so arrow keys stay bound to the grid.

Per-cluster actions:

• Approve trip (confirm + tweak name)
• Mark non-trip
• Split — select a boundary asset in the chronologically-ordered grid and choose "split before here"; the cluster partitions at that timestamp into two clusters, both with editable names. Reuses the existing ring-selection / arrow-key focus.
• Merge — surfaces the cluster's chronological neighbour(s) (prev/next by time, computed from a temporal index, independent of the needs-attention list sort) with a preview of the combined range; the user confirms which to absorb.
• Skip

Per-asset refinement (coverage flags). Within a cluster's grid, assets flagged "likely belongs here" (the completeness gap) appear with a distinct badge and an include toggle; flagged outliers appear with an exclude toggle. Include/exclude updates cluster_members before write-back, so the algorithm's completeness work is actionable rather than informational.

Keyboard-first. Cluster-level actions mirror the sibling convention (e.g. A approve, N non-trip, S split, M merge, X skip; arrows navigate within the grid; [ / ] move between clusters), listed in a help footer as image-rater/travel-memories do. (First-class requirement.)

Empty states. Because ingest is scopeable and assets can already be _pipeline/processed, a cluster run can legitimately yield nothing. serve distinguishes three cases, each naming the next step: no clusters yet (run ingest/cluster), all clusters reviewed, and this scope produced no clusters.

To keep it fast: high-confidence clusters arrive pre-filled, and an "approve all high-confidence" bulk action lets the user rubber-stamp the obvious cases, concentrating attention on the fuzzy ones. Every decision persists immediately to SQLite, so review is resumable across sessions (see Write-back for what survives loss of the SQLite layer itself).

4. Write-back (to Immich, idempotent)

On approval (per-cluster or batch apply):

• Approve → write the trip tag (via upsert_tag + tag_assets) to all member assets, respecting the existing content trip-tag convention (trip names are user-facing, not namespaced).
• Mark non-trip → apply _pipeline/non-trip so those assets are filtered out and never resurface as unreviewed.
• Mark processed → every adjudicated asset (trip-assigned, non-trip, or reviewed-and-skipped) gets _pipeline/processed. This is the durable "done" flag in the source of truth: it captures the reviewed-but-untagged case, and lets a fresh ingest re-derive what's already handled even if the SQLite working DB is lost (see Ingest read-back).
• Idempotent: writeback_log records applied changes; re-runs skip what's already done. Explicit confirmation before any write (mirrors image-rater's export safety).
• Partial-failure reporting: a batch "apply all approved" reports per-cluster results (succeeded / failed with reason) from writeback_log; clusters that fail stay in their approved-but-unapplied state so a re-run retries only those. The UI surfaces this as inline status badges plus a summary, so the user always knows what landed in the library.

All durable state lands in Immich; SQLite is the working/review layer. Only applied (tag-written) decisions are rebuildable from Immich — approved-but-unapplied clusters, in-progress split/merge, edited names and notes live only in SQLite. So the resumability guarantee holds against loss of SQLite only for applied work: decisions should be applied promptly on approval (or the SQLite working DB backed up), and the rebuild-from-Immich path recovers everything already written as tags.

Tag conventions (shared across all apps)

Two clearly separated kinds of tags:

• Content / organizational tags — trip names ("Italy 2019"), locations ("Kiev"), people. User-facing, part of the existing convention, never namespaced. trip-cluster must read and respect the existing trip-tag convention rather than invent a parallel one.
• Pipeline meta-tags — everything the tooling generates as machinery, nested under a single parent _pipeline/ so the whole set can be removed by deleting the parent and never clutters the tag list. Defined as a shared convention in shared/immich and used by every app in the monorepo:
  • _pipeline/processed — categorizer: asset adjudicated (any outcome)
  • _pipeline/non-trip — categorizer: everyday/noise
  • _pipeline/ai-rating/<0-5> — image-rater's ai-rating/<n> moves under this root when migrated (M4)
  • room for future meta-tags (review-state, etc.)

Reconciliation caveat: before writing, inspect the live Immich instance and reconcile with anything already present (e.g. image-rater's existing un-namespaced ai-rating/<n>) rather than blindly creating duplicates.

App shape & conventions

• categorize CLI (argparse subcommands: ingest · cluster · serve · apply) + Flask factory create_app().
• Jinja + DaisyUI/Tailwind/Alpine/HTMX via shared/ui, no build step, Immich thumbnail proxy route, serves on 8084.
• Env vars (same names as existing apps): IMMICH_URL, IMMICH_API_KEY, ANTHROPIC_API_KEY (Anthropic optional for M1), loaded from .env.

Testing strategy

TDD throughout, mirroring the existing apps:

• Unit: clustering algorithm as a pure function (timestamps/tags/GPS → clusters) — deterministic and easy to assert; SQLite store; config; Immich client (mocked via pytest-httpserver).
• Route/UI: the review flow (approve / non-trip / split / merge, immediate persistence, idempotent write-back) with Immich mocked.
• Playwright UI tests: grid + arrow-key navigation + full-screen lightbox behavior, mirroring travel-memories/image-rater.
• Shared packages tested independently of the apps.

Cost notes

M1 is algorithm-first, so trip-cluster makes essentially no Anthropic calls — clustering is local timestamp/tag/GPS math. AI cost becomes relevant only in M3 (enrich), where image-input tokens dominate and the batch API (~50% cheaper, chunked ≤50) plus the Haiku default keep it modest (image-rater reference: ~700 images on Haiku ≈ $1).

Out of scope for M1

• Choosing keepers within a trip (image-rater).
• Full non-trip/everyday categorization (M6).
• Migrating the existing apps onto the shared foundation (M4/M5).
• Geocoding / GPS backfill / AI captioning (M3).
• Narrative text drafting (human-owned; downstream).

---

Deferred / Open Questions

From 2026-06-27 ce-doc-review

• Cluster-count blow-up from everyday photos (F4). Cluster-level QA assumes "a few dozen" candidate trips, but clustering everyday/non-trip photos with per-gap splitting could mint hundreds–thousands of low-confidence clusters that "approve all high-confidence" won't relieve. The concern is real, but the reviewer's proposed pre-pass (collapsing everyday spans into bulk buckets) is unconvincing; a candidate alternative is ordering/filtering clusters by date-span (a multi-week span ranks above a 2-day span). Crucially, the actual cluster output on real Immich data is unknown to both reviewer and author — resolve by observing real behaviour on a representative subset first, then choose the surfacing/collapsing strategy.