commit 0cb0bcd6a787f0d855a9d2940b10130f9924f814 Author: mischa Date: Sat Jun 27 14:17:46 2026 +0200 docs: M1 design spec — image categorizer foundation + trip-cluster Co-Authored-By: Claude Opus 4.8 diff --git a/docs/superpowers/specs/2026-06-27-image-categorizer-design.md b/docs/superpowers/specs/2026-06-27-image-categorizer-design.md new file mode 100644 index 0000000..ef0b60b --- /dev/null +++ b/docs/superpowers/specs/2026-06-27-image-categorizer-design.md @@ -0,0 +1,203 @@ +# Image Categorizer — Design (M1: Foundation + Trip-Cluster) + +**Date:** 2026-06-27 +**Status:** Approved design, pending implementation plan +**Scope of this spec:** Milestone M1 only (shared foundation + the `trip-cluster` app). Later milestones are documented as a roadmap, each to get its own spec→plan→build cycle. + +## Purpose + +A monorepo of small, independent tools that clean up and structure a ~40k-image, 15-year [Immich](https://immich.app) library so it can become travel "memories." This is the **upstream** stage of an existing pipeline: + +``` +[NEW categorizer: trips + tags] → image-rater (pick best → 4+ album) → travel-memories (album → Grav blog) +``` + +Every handoff is through **Immich itself** (tags, albums, ratings) — no app talks to another directly. The categorizer's job is to leave clean **trip tags** (and explicit **non-trip** marks) in Immich for `image-rater` to consume by tag. + +### Division of labor (the core principle) + +AI/algorithms do what they do well; the human does QA and (later, downstream) content creation. + +- The machine **proposes** groupings with a confidence score. +- The human **reviews at the cluster level** (a few dozen candidate trips), not photo-by-photo, mostly in bulk, with attention concentrated where confidence is low. +- Nothing is silently auto-applied to the library. Trust matters with 15 years of memories. + +### What M1 explicitly does NOT do + +- It does **not** decide which photos *within* a trip are keepers — that is `image-rater`'s job downstream. +- It does **not** fully categorize everyday/non-trip photos beyond marking them as non-trip (deferred to a future milestone). +- It does **not** draft any narrative text (the user owns content creation). + +## Context & constraints (from the existing ecosystem) + +The two existing apps (`claude-image-rater`, `claude-travel-memories`, both under `/home/mischa/Projects/`) establish strong, consistent conventions this project follows and builds upon: + +- **Stack:** Python 3.12 + Flask (factory `create_app()`) + an argparse CLI sharing one package. +- **No-build frontend:** Jinja templates extending `base.html`, **DaisyUI 4 + Tailwind + Alpine.js** (CDN), HTMX for navigation, one `static/app.js`. +- **Single-responsibility modules:** one module is the *only* thing that talks to Immich; one is the *only* thing that talks to Anthropic. +- **Testing:** pytest + `pytest-httpserver` (mock Immich) + Playwright UI tests, **TDD** ("failing test first"). +- **Docker:** one container per app, distinct port, `user: ${UID}:${GID}`, state on a mounted volume. +- **Proven UI:** travel-memories' triage screen (grid + ring-selection + keyboard navigation + fullscreen lightbox) was reused by image-rater. This becomes the canonical shared component (see §UI). + +Both apps currently **duplicate** near-identical code (`immich.py`, `config.py`, `slugify`, `base.html`, thumbnail proxy route, JSON-state pattern). That duplication is the evidence that extracting a shared foundation is high-value, and that migrating those apps onto it later (M4/M5) is largely mechanical. + +### Decisions locked during brainstorming + +- **State store: SQLite** (not the JSON-file pattern of the existing apps). A single DB makes the cross-trip filtering this project needs ("all assets in a trip window but missing the trip tag", "everything still unreviewed") far easier. `shared/core` is designed cleanly enough that the other apps *could* migrate onto it later. +- **Immich is the source of truth.** SQLite is the working/review layer; all durable results are written back to Immich as tags. +- **Monorepo with shared packages + per-app containers** (Approach A on a shared foundation). Build the categorizer first as the POC that proves the foundation, then migrate the other apps onto it in later milestones (Option 2 end-state), in milestones — not all at once. +- **GPS is a minority signal.** Most old camera photos lack GPS, so trip detection is **timestamp-first**, anchored by existing tags, refined by GPS where present. +- **Everyday/non-trip photos:** assign trips AND positively mark non-trip so they stop resurfacing as "unreviewed." Full non-trip categorization is deferred. + +## Roadmap (sequenced; each milestone independently shippable) + +**End state:** one monorepo — `shared/{immich, core, ai, ui}` + `apps/{trip-cluster, tag-verify, enrich, image-rater, travel-memories}`, each its own container. + +- **M1 — Foundation + trip-cluster (THIS SPEC).** Stand up the monorepo, shared packages, SQLite store, Immich ingest, and the `trip-cluster` app end-to-end. Validate on the last trip's photos. This is the POC that proves the foundation. +- **M2 — tag-verify** app on the proven foundation (verify/normalize existing tags, dedupe vocabulary, find outliers). +- **M3 — enrich** app (geocode location tags like "Kiev" → coordinates and optionally backfill GPS into Immich to improve future trip detection; AI captioning; content tags; noise detection). +- **M4 — migrate image-rater** onto shared packages (+ optional SQLite). +- **M5 — migrate travel-memories** onto shared packages (+ optional SQLite). +- **M6 (future/optional) — non-trip categorization** (the deferred full-library taxonomy). + +--- + +# M1 Design + +## Monorepo layout + +``` +claude-image-categorizer/ + pyproject.toml # workspace; shared/* as editable path deps + docker-compose.yml # one service per app (just trip-cluster for M1) + .env.example # IMMICH_URL, IMMICH_API_KEY, ANTHROPIC_API_KEY, UID, GID + README.md + CLAUDE.md + shared/ + immich/ # the one true Immich client + core/ # SQLite store + domain models + persistence + ai/ # Anthropic batch wrapper (scaffolded; light/unused in M1) + ui/ # base.html, Tailwind/DaisyUI/Alpine/HTMX, Jinja macros, app.js + apps/ + trip-cluster/ + Dockerfile + categorize.py # CLI entry point -> app.cli.main() + app/ # Flask factory, routes, app-specific logic + tests/ + docs/superpowers/specs/ ... +``` + +- `shared/*` are local, editable Python packages so apps `import` them directly. Packaging via a workspace (`pyproject.toml` path deps; `uv` workspace optional). This stays close to the existing pip+venv practice while enabling the monorepo. +- Each app owns its Dockerfile and a distinct port. **trip-cluster serves on 8084** (8082 = travel-memories, 8083 = image-rater). +- The SQLite DB and downloaded thumbnails live on a mounted volume so state persists across container restarts and is reachable from the host. `user: ${UID}:${GID}`. + +## Shared packages + +### `shared/immich` +The single Immich REST client (consolidates the duplicated `immich.py` from both existing apps). `requests.Session` with `x-api-key`. Capabilities needed for M1: +- `resolve_tag_id(name)`, list tags / tag inventory. +- Search assets (paginated `POST /api/search/metadata`) by date range, by tag, and broadly — returning at least: `id`, `originalFileName`, `localDateTime`, GPS lat/lon (when present), city/country (when present), `type`, existing tags, rating. +- `download_thumbnail(asset_id)` (`…/thumbnail?size=preview`). +- `upsert_tag(name)`, `tag_assets(tag_id, asset_ids)` (write-back of trip / non-trip tags). + +### `shared/core` +SQLite store + domain dataclasses + persistence (connection, schema/migrations, data-access helpers). Owns the on-disk schema. Designed as a clean data-access layer so other apps can adopt it later. + +**Domain model (central unit = Cluster / candidate trip):** + +- **Asset** — mirror of an Immich asset: `immich_id`, `taken_at`, `gps_lat`/`gps_lon` (nullable), `place_city`/`place_country` (nullable), `type`, `has_gps`, `thumb_path`, `ingested_at`. +- **Tag** — inventory of Immich tags: `name`, `immich_tag_id`, usage `count` (ingested now; used heavily in M2). +- **AssetTag** — existing Immich tags per asset (denormalized for filtering). +- **Cluster** — a candidate trip: `id`, `start_at`, `end_at`, `count`, `suggested_name`, `confidence`, `kind_guess` (`trip`|`everyday`), `status` (`pending`|`approved`|`non_trip`|`merged`|`split`|`skipped`), `decided_name`, `reviewed_at`, `notes`. +- **ClusterMember** — asset↔cluster link with `member_confidence` and `is_outlier`. +- **WritebackLog** — every change pushed to Immich (`asset_id`, `action`, `tag`, `result`, `applied_at`) for idempotency. +- **Meta** — last-ingest timestamp, run params, schema version. + +**SQLite tables:** `assets`, `asset_tags`, `tags`, `clusters`, `cluster_members`, `writeback_log`, `meta`. Single DB file on the mounted volume. + +### `shared/ai` +Anthropic **batch** Messages API wrapper using the official `anthropic` SDK, carrying over image-rater's proven conventions (per-criterion judgments, `confidence`, score floors, chunking ≤50/batch, default model `claude-haiku-4-5`). **Scaffolded but barely used in M1** — trip-cluster is algorithm-first (near-zero AI cost). It earns its keep in M3 (enrich). + +### `shared/ui` +The shared visual foundation, extracted from the proven travel-memories/image-rater templates: +- `base.html` (DaisyUI 4 + Tailwind + Alpine.js + HTMX via CDN, navbar, content block). +- The canonical **grid + fullscreen lightbox** component: thumbnail grid with ring-selection, **arrow-key navigation**, **full-screen view** (arrows = prev/next, Esc = close). *(First-class requirement.)* +- Reusable Jinja macros (image grid, lightbox, confidence/status badges, approve/reject controls) and shared `app.js` Alpine components. + +## The `trip-cluster` app (M1 deliverable) + +### Workflow + +``` +categorize ingest [--from DATE --to DATE | --tag NAME | --subset N] # 1. pull → SQLite + thumbs +categorize cluster [--gap-threshold ...] # 2. build candidate trips +categorize serve # 3. review UI on :8084 + # (approve / non-trip / split / merge) +categorize apply # 4. write approved tags back to Immich +``` + +(`serve` may also trigger `apply` per-cluster via a UI button, plus a batch "apply all approved" action; CLI `apply` is the headless equivalent.) + +### 1. Ingest (automatic, scopeable) +Pull assets from Immich via `shared/immich`, upsert metadata into SQLite, download thumbnails. **Incremental & idempotent** (fetch only changed-since-last via Immich `updatedAt`; upsert by `immich_id`). + +**Scopeable from day one** — the mechanism for the vet-first plan. Instead of forcing a full 40k pull, `ingest` accepts a bounded scope: +- `--from / --to` (date window) → e.g. the last trip plus surrounding everyday photos; +- `--tag NAME` → a single already-tagged trip; +- `--subset N` → a cap. + +This lets the whole tool be validated on the last trip for near-zero cost before widening to the backlog over time. + +### 2. Cluster (automatic) — signal hierarchy +Pure, algorithmic, **no API calls** (keeps the POC nearly free). Produces candidate trips, each with a `suggested_name`, `confidence`, and `kind_guess`: + +1. **Existing trip tag** → authoritative seed; its assets form a confirmed cluster (still shown, to verify *completeness*). Respects the user's existing trip-tag convention. +2. **Timestamp gap clustering** → primary structure: sort by `taken_at`, split where the inter-photo gap exceeds a tunable threshold. +3. **Location anchors** (existing location tags like "Kiev" + GPS when present) → refine boundaries, propose names. +4. **Coverage detection** → assets *inside* a confirmed trip's time window but *missing* its tag are flagged "likely belongs here" (the completeness gap); assets carrying a trip tag but *outside* their cluster are flagged as outliers. +5. **Visual similarity** (Immich CLIP / local pHash) → *optional* tie-breaker for ambiguous boundaries; marked optional so it can never block the slice. + +**Confidence & kind_guess** (echoing image-rater's confidence/floor approach): tight time window + existing trip tag + consistent location → high confidence; sparse, untagged, no GPS → low ("needs your eye"). Low-volume scattered clusters → `kind_guess = everyday` (suggested non-trip). + +### 3. Review (human, cluster-level) +A review screen lists **candidate trips sorted by "needs attention"** (low confidence first). Per cluster: thumbnail grid (with the shared grid+lightbox / arrow-key / full-screen component), editable suggested name, and actions: +- **Approve trip** (confirm + tweak name) +- **Mark non-trip** +- **Split** (break one cluster into two) +- **Merge adjacent** (combine with a neighbor) +- **Skip** + +To keep it fast: high-confidence clusters arrive **pre-filled**, and an **"approve all high-confidence"** bulk action lets the user rubber-stamp the obvious cases, concentrating attention on the fuzzy ones. Every decision **persists immediately** to SQLite (resumable: closing and reopening resumes exactly where the user left off). + +### 4. Write-back (to Immich, idempotent) +On approval (per-cluster or batch `apply`): +- **Approve** → write the trip tag (via `upsert_tag` + `tag_assets`) to all member assets, respecting the existing trip-tag convention. +- **Mark non-trip** → apply a `non-trip` tag (named to fit the user's existing scheme) so those assets are filtered out and never resurface as unreviewed. +- **Idempotent**: `writeback_log` records applied changes; re-runs skip what's already done. **Explicit confirmation before any write** (mirrors image-rater's export safety). + +All durable state lands in **Immich**; SQLite remains the working/review layer. + +### Tag-scheme reconciliation (important) +Before introducing any tag convention (trip names, the `non-trip` mark, or any future `ai-*` scheme), **inspect the live Immich instance and reconcile with what already exists** rather than inventing a parallel scheme. The user's trips already follow a convention — trip-cluster must read and respect it. + +## App shape & conventions +- `categorize` CLI (argparse subcommands: `ingest` · `cluster` · `serve` · `apply`) + Flask factory `create_app()`. +- Jinja + DaisyUI/Tailwind/Alpine/HTMX via `shared/ui`, **no build step**, Immich thumbnail proxy route, serves on **8084**. +- Env vars (same names as existing apps): `IMMICH_URL`, `IMMICH_API_KEY`, `ANTHROPIC_API_KEY` (Anthropic optional for M1), loaded from `.env`. + +## Testing strategy +TDD throughout, mirroring the existing apps: +- **Unit:** clustering algorithm as a pure function (timestamps/tags/GPS → clusters) — deterministic and easy to assert; SQLite store; config; Immich client (mocked via `pytest-httpserver`). +- **Route/UI:** the review flow (approve / non-trip / split / merge, immediate persistence, idempotent write-back) with Immich mocked. +- **Playwright UI tests:** grid + arrow-key navigation + full-screen lightbox behavior, mirroring travel-memories/image-rater. +- Shared packages tested independently of the apps. + +## Cost notes +M1 is algorithm-first, so trip-cluster makes **essentially no Anthropic calls** — clustering is local timestamp/tag/GPS math. AI cost becomes relevant only in M3 (enrich), where image-input tokens dominate and the batch API (~50% cheaper, chunked ≤50) plus the Haiku default keep it modest (image-rater reference: ~700 images on Haiku ≈ $1). + +## Out of scope for M1 +- Choosing keepers within a trip (image-rater). +- Full non-trip/everyday categorization (M6). +- Migrating the existing apps onto the shared foundation (M4/M5). +- Geocoding / GPS backfill / AI captioning (M3). +- Narrative text drafting (human-owned; downstream).