From e5923317de72ad54c4bad75dc66cf7981e60b649 Mon Sep 17 00:00:00 2001 From: mischa Date: Sat, 27 Jun 2026 19:41:59 +0200 Subject: [PATCH] docs(spec): pgvector embedding feasibility spike (M1.5 dependency) Defines the read-only Postgres/pgvector feasibility spike that the roadmap claims ran in M1 but the M1 plan deferred. Pins the four pass/fail requirements, the disposable-probe approach, and the findings-doc contract that unblocks M1.5. Co-Authored-By: Claude Opus 4.8 --- ...6-06-27-pgvector-embedding-spike-design.md | 116 ++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md diff --git a/docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md b/docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md new file mode 100644 index 0000000..69a3965 --- /dev/null +++ b/docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md @@ -0,0 +1,116 @@ +# pgvector embedding feasibility spike — design + +**Date:** 2026-06-27 +**Milestone dependency:** unblocks **M1.5** (visual similarity). See `docs/ROADMAP.md`. +**Status:** design + +## Why this exists + +The roadmap narrative claims M1 "runs a read-only feasibility spike" and that M1.5's +dependency is "verified in M1." It is not: the M1 plan explicitly deferred the spike +(`docs/superpowers/plans/2026-06-27-immich-photo-flow-m1.md:3710` — *"pgvector spike → +out of this plan by decision (tracked separately; M1.5 dependency)"*). So M1.5 is blocked +on a prerequisite that never ran. This spec defines that prerequisite. + +M1.5 wants to cluster photos by visual similarity using the CLIP embeddings Immich already +computes — the rescue signal for the GPS-poor old library where timestamp/GPS are weakest. +Immich's REST API does not cleanly expose raw embedding vectors, so the viable path is +**read-only access to Immich's Postgres pgvector embeddings**. Before M1.5 can build +clustering on that path, this spike must prove the path exists and pin its shape. + +## Scope + +**Feasibility only.** Read-only. Throwaway probe. The spike: + +- does **not** build clustering, a tested module, or any `shared/` reader; +- does **not** write to Immich or its database (only `SELECT`); +- exists to turn unknowns into **pinned facts** and hand M1.5 a verified contract. + +## Requirements (pass/fail) + +The spike succeeds when it answers all four against the **live** database and records the +answers in the findings doc. Phrased as plain questions: + +1. **Can we read the CLIP embeddings at all?** Connect read-only over the LAN and read the + embedding vectors Immich stores. *This is the real go/no-go: if the embeddings are not + readable, M1.5 needs a different plan.* +2. **Can we join each embedding back to a photo we already track?** Our SQLite store keys + assets by Immich asset ID. Confirm each embedding row carries that same ID so an + embedding can be matched to a known asset. *Without the join the vectors are unusable.* +3. **What are the exact shapes?** Record the table name, embedding column, **vector + dimension**, and the correct pgvector **distance operator** for similarity. These have + drifted across Immich versions, so they must be read from *this* DB, not assumed. +4. **What is the coverage?** What fraction of library assets currently have an embedding + (the user is re-running CLIP with a stronger model — has it finished?). *Tells M1.5 how + much it can lean on the signal.* + +## What we already know (to verify, not assume) + +Immich historically stores CLIP vectors in a **`smart_search`** table with an **`embedding`** +column of pgvector type `vector`, keyed by `assetId` referencing `assets.id`; similarity uses +**cosine** distance (`<=>`); the legacy default model (`ViT-B-32`) produced **512-dim** +vectors. Names and dimension have changed across versions and the user's re-run uses a +stronger model, so the probe **discovers** these rather than trusting them — the list above is +only the set of candidates to probe first. + +## Approach + +Disposable read-only probe + two durable artifacts (chosen over a throwaway-only or a +build-the-module-now approach: feasibility-only honors the roadmap, but capturing the DSN and +a written contract is the cheap part that saves M1.5 from guessing). + +### Connection + +- New optional env var **`IMMICH_DB_URL`** (a Postgres DSN), added to `.env.example` and + loaded by `config.py` as an **optional** field (REST creds stay required; the DSN is only + needed for the spike / M1.5). +- Simplest credentials that work: Immich's existing Postgres user, kept read-only by the + probe issuing **only `SELECT`**. A dedicated least-privilege read-only role is documented as + recommended hardening for M1.5 but is **not required** to prove feasibility. +- Driver: `psycopg` (psycopg3). Add to the spike's dependencies; M1.5 will formalize it. + +### The probe + +A single disposable script, **`scripts/pgvector_spike.py`**, that is idempotent and +read-only. It: + +1. connects using `IMMICH_DB_URL`; +2. discovers candidate embedding tables/columns from the catalog (probe `smart_search` / + `embedding` first, then fall back to scanning `information_schema` for `vector`-typed + columns) — so it survives version drift; +3. reads one embedding and reports its **dimension** and a sample of the **distance operator** + working (e.g. a `... ORDER BY embedding <=> embedding LIMIT 5` self-similarity sanity + check); +4. verifies the **asset-ID join**: the embedding row's key matches `assets.id`, and that same + ID is one we store in SQLite (`shared/photoflow/core`); +5. computes **coverage**: `count(embeddings) / count(assets)` (and, if cheap, broken down by + whether the asset is an image vs video); +6. prints a human-readable report **and** writes/refreshes the findings doc. + +It hardcodes nothing destructive, takes no write path, and is safe to re-run. + +## Deliverables + +1. `scripts/pgvector_spike.py` — disposable read-only probe (removed or left as a documented + one-off after M1.5 internalizes its findings). +2. `IMMICH_DB_URL` in `.env.example`; optional `immich_db_url` field in `config.py`. +3. **`docs/superpowers/specs/2026-06-27-pgvector-embedding-findings.md`** — the one-page pinned + schema that becomes M1.5's contract: table, embedding column, vector dimension, distance + operator, the embedding → `assetId` → SQLite `asset` join, coverage %, Immich version, and + the (optional) read-only-role recipe. +4. Roadmap correction: M1.5's status note updated to "dependency spike pending/this spec" + instead of "verified in M1." + +## Out of scope (explicitly M1.5) + +CLIP clustering; a tested `shared/photoflow/immich/db.py` reader; nearest-neighbor queries at +scale; any use of the embeddings beyond the feasibility checks above. + +## Risks + +- **Embeddings unreadable / not joinable** → M1.5 blocked; spike's whole point is to surface + this early and cheaply. +- **Schema differs from the known shape** → expected and handled by catalog discovery, not + hardcoded names. +- **Coverage near zero** (re-run unfinished) → not a spike failure; recorded as a fact so M1.5 + can sequence around it.