From c406b567bde16fdd840828b409d1bb510e8d543c Mon Sep 17 00:00:00 2001 From: mischa Date: Sat, 27 Jun 2026 15:04:44 +0200 Subject: [PATCH] docs: extract authoritative ROADMAP.md; spec links to it Co-Authored-By: Claude Opus 4.8 --- docs/ROADMAP.md | 65 +++++++++++++++++++ .../2026-06-27-immich-photo-flow-design.md | 11 +--- 2 files changed, 67 insertions(+), 9 deletions(-) create mode 100644 docs/ROADMAP.md diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md new file mode 100644 index 0000000..0871806 --- /dev/null +++ b/docs/ROADMAP.md @@ -0,0 +1,65 @@ +# immich-photo-flow — Roadmap + +**Last updated:** 2026-06-27 + +This is the authoritative, living roadmap for the project. Each milestone gets its own design spec → implementation plan → build cycle under `docs/superpowers/`. This document is the map; the specs are the territory. + +## Vision + +A monorepo of small, independent Python/Flask tools that clean up and structure a ~40k-image, 15-year [Immich](https://immich.app) library into travel "memories." It is the **upstream** stage of an existing pipeline (all repos under `/home/mischa/Projects/`): + +``` +immich-photo-flow (trips + tags) → image-rater (pick best → 4+ album) → travel-memories (album → Grav blog) +``` + +Every handoff is through **Immich itself** (tags / albums / ratings) — no app calls another directly. + +**Core principle — division of labor:** AI/algorithms propose; the human does QA at the cluster/group level (mostly in bulk, attention concentrated where confidence is low) and owns content creation. Nothing is silently auto-applied to a 15-year library of memories. + +## End state + +One monorepo: + +``` +shared/{immich, core (SQLite), ai, ui} +apps/{trip-cluster, tag-verify, enrich, image-rater, travel-memories} +``` + +Each app is independently containerized (its own Dockerfile + port). `shared/*` are local editable packages providing the common foundation. + +## Milestones + +| # | Name | Goal | Status | +|---|------|------|--------| +| **M1** | Foundation + `trip-cluster` | Stand up the monorepo, shared packages, SQLite store, Immich ingest, and the `trip-cluster` app end-to-end. Validate on the last trip's photos. **This is the POC that proves the foundation.** | Design approved; spec written; plan pending | +| **M2** | `tag-verify` | Verify/normalize existing tags, dedupe the tag vocabulary, find outliers — on the proven foundation. | Not started | +| **M3** | `enrich` | Geocode existing location tags ("Kiev" → coordinates) and backfill GPS into Immich to improve future trip detection; AI captioning; content tags; noise detection. | Not started | +| **M4** | Migrate `image-rater` | Move the existing app onto the shared packages (+ optional SQLite). Largely mechanical (delete local copies, import shared). | Not started | +| **M5** | Migrate `travel-memories` | Same migration for the blog-export app. | Not started | +| **M6** | Non-trip categorization (future/optional) | The deferred full-library taxonomy for everyday/non-trip photos. | Deferred | + +**Sequencing rationale:** Build the new categorizer first as the POC that *earns* the shared foundation (Option 1), then migrate the existing apps onto it (Option 2 end-state) — in milestones, not all at once. Doing `shared/ui` + `shared/immich` well in M1 makes M4/M5 mostly mechanical; the existing apps already duplicate exactly what those packages absorb. + +## Cross-cutting decisions & conventions + +These apply across all milestones (decided during the 2026-06-27 brainstorm): + +- **Immich is the source of truth.** SQLite is a rebuildable working/review layer; all durable results are written back to Immich as tags. +- **State store: SQLite** (`shared/core`), not the JSON-file pattern of the existing apps — chosen for easy cross-trip filtering at 40k scale. Designed cleanly enough that the other apps can migrate onto it. +- **Stack/conventions mirror the sibling apps:** Python 3.12 + Flask + argparse CLI; no-build frontend (Jinja + DaisyUI/Tailwind/Alpine/HTMX); one module talks to Immich, one to Anthropic; pytest + pytest-httpserver + Playwright; TDD; one Docker container per app; `user: ${UID}:${GID}`; state on a mounted volume. +- **Ports:** travel-memories 8082, image-rater 8083, trip-cluster 8084. +- **Tag conventions:** + - *Content/organizational tags* (trip names, locations, people) are user-facing, follow the existing convention, and are **never namespaced**. + - *Pipeline meta-tags* are nested under a single parent **`_pipeline/`** so they can be removed wholesale and never clutter the tag list: `_pipeline/processed`, `_pipeline/non-trip`, `_pipeline/ai-rating/<0-5>` (image-rater after M4), etc. Defined as a shared convention in `shared/immich`. + - Always **reconcile with the live Immich instance** before writing tags (e.g. image-rater's existing un-namespaced `ai-rating/`). +- **Trip detection is timestamp-first**, anchored by existing trip/location tags, refined by GPS where present. GPS is a minority signal (old camera photos lack it). +- **Resumable / idempotent / scopeable** throughout: this is a long-running, vet-first, work-over-time effort; `_pipeline/processed` + `writeback_log` make work re-derivable and re-runnable; ingest is scopeable (date range / tag / subset) so tools are validated on one trip before the backlog. + +## Enablement notes (things to set up to unlock later milestones) + +- **Visual similarity (post-M1, leading enhancement):** Immich's REST API doesn't cleanly expose CLIP embeddings. Viable path = **read-only access to Immich's Postgres pgvector** embeddings (verify table/column against the live version). User can provide DB access; user is re-running CLIP with a stronger model — both pure upside for trip-level similarity, especially valuable for the GPS-poor old library. +- **GPS facts:** Immich's reverse-geocoding only *labels* coordinates a photo already has; it does **not** invent GPS, and Immich cannot infer GPS from image content. For GPS-less old photos, coordinates come from manual map placement or the **M3 `enrich`** step (geocode location tags → write back as GPS). Reverse-geocoding + metadata-extraction jobs can be run anytime to strengthen location anchors for the GPS-having subset. + +## Related specs + +- M1: `docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md` diff --git a/docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md b/docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md index 2d44a34..1c05626 100644 --- a/docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md +++ b/docs/superpowers/specs/2026-06-27-immich-photo-flow-design.md @@ -49,16 +49,9 @@ Both apps currently **duplicate** near-identical code (`immich.py`, `config.py`, - **GPS is a minority signal.** Most old camera photos lack GPS, so trip detection is **timestamp-first**, anchored by existing tags, refined by GPS where present. - **Everyday/non-trip photos:** assign trips AND positively mark non-trip so they stop resurfacing as "unreviewed." Full non-trip categorization is deferred. -## Roadmap (sequenced; each milestone independently shippable) +## Roadmap -**End state:** one monorepo — `shared/{immich, core, ai, ui}` + `apps/{trip-cluster, tag-verify, enrich, image-rater, travel-memories}`, each its own container. - -- **M1 — Foundation + trip-cluster (THIS SPEC).** Stand up the monorepo, shared packages, SQLite store, Immich ingest, and the `trip-cluster` app end-to-end. Validate on the last trip's photos. This is the POC that proves the foundation. -- **M2 — tag-verify** app on the proven foundation (verify/normalize existing tags, dedupe vocabulary, find outliers). -- **M3 — enrich** app (geocode location tags like "Kiev" → coordinates and optionally backfill GPS into Immich to improve future trip detection; AI captioning; content tags; noise detection). -- **M4 — migrate image-rater** onto shared packages (+ optional SQLite). -- **M5 — migrate travel-memories** onto shared packages (+ optional SQLite). -- **M6 (future/optional) — non-trip categorization** (the deferred full-library taxonomy). +The full multi-milestone roadmap (M1–M6), end-state, and cross-cutting decisions live in the authoritative **[`docs/ROADMAP.md`](../../ROADMAP.md)**. In brief: **this spec is M1** — Foundation + `trip-cluster`, the POC that proves the shared foundation. M2 (`tag-verify`), M3 (`enrich`), M4/M5 (migrate the existing apps onto the foundation), and M6 (non-trip categorization) follow, each with its own spec→plan→build cycle. ---