immich-photo-flow/docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md

pgvector embedding feasibility spike — design

Date: 2026-06-27 Milestone dependency: unblocks M1.5 (visual similarity). See docs/ROADMAP.md. Status: design

Why this exists

The roadmap narrative claims M1 "runs a read-only feasibility spike" and that M1.5's dependency is "verified in M1." It is not: the M1 plan explicitly deferred the spike (docs/superpowers/plans/2026-06-27-immich-photo-flow-m1.md:3710 — "pgvector spike → out of this plan by decision (tracked separately; M1.5 dependency)"). So M1.5 is blocked on a prerequisite that never ran. This spec defines that prerequisite.

M1.5 wants to cluster photos by visual similarity using the CLIP embeddings Immich already computes — the rescue signal for the GPS-poor old library where timestamp/GPS are weakest. Immich's REST API does not cleanly expose raw embedding vectors, so the viable path is read-only access to Immich's Postgres pgvector embeddings. Before M1.5 can build clustering on that path, this spike must prove the path exists and pin its shape.

Scope

Feasibility only. Read-only. Throwaway probe. The spike:

• does not build clustering, a tested module, or any shared/ reader;
• does not write to Immich or its database (only SELECT);
• exists to turn unknowns into pinned facts and hand M1.5 a verified contract.

Requirements (pass/fail)

The spike succeeds when it answers all four against the live database and records the answers in the findings doc. Phrased as plain questions:

1. Can we read the CLIP embeddings at all? Connect read-only over the LAN and read the embedding vectors Immich stores. This is the real go/no-go: if the embeddings are not readable, M1.5 needs a different plan.
2. Can we join each embedding back to a photo we already track? Our SQLite store keys assets by Immich asset ID. Confirm each embedding row carries that same ID so an embedding can be matched to a known asset. Without the join the vectors are unusable.
3. What are the exact shapes? Record the table name, embedding column, vector dimension, and the correct pgvector distance operator for similarity. These have drifted across Immich versions, so they must be read from this DB, not assumed.
4. What is the coverage? What fraction of library assets currently have an embedding (the user is re-running CLIP with a stronger model — has it finished?). Tells M1.5 how much it can lean on the signal.

What we already know (to verify, not assume)

Immich historically stores CLIP vectors in a smart_search table with an embedding column of pgvector type vector, keyed by assetId referencing assets.id; similarity uses cosine distance (<=>); the legacy default model (ViT-B-32) produced 512-dim vectors. Names and dimension have changed across versions and the user's re-run uses a stronger model, so the probe discovers these rather than trusting them — the list above is only the set of candidates to probe first.

Approach

Disposable read-only probe + two durable artifacts (chosen over a throwaway-only or a build-the-module-now approach: feasibility-only honors the roadmap, but capturing the DSN and a written contract is the cheap part that saves M1.5 from guessing).

Connection

• New optional env var IMMICH_DB_URL (a Postgres DSN), added to .env.example and loaded by config.py as an optional field (REST creds stay required; the DSN is only needed for the spike / M1.5).
• Simplest credentials that work: Immich's existing Postgres user, kept read-only by the probe issuing only SELECT. A dedicated least-privilege read-only role is documented as recommended hardening for M1.5 but is not required to prove feasibility.
• Driver: psycopg (psycopg3). Add to the spike's dependencies; M1.5 will formalize it.

The probe

A single disposable script, scripts/pgvector_spike.py, that is idempotent and read-only. It:

1. connects using IMMICH_DB_URL;
2. discovers candidate embedding tables/columns from the catalog (probe smart_search / embedding first, then fall back to scanning information_schema for vector-typed columns) — so it survives version drift;
3. reads one embedding and reports its dimension and a sample of the distance operator working (e.g. a ... ORDER BY embedding <=> embedding LIMIT 5 self-similarity sanity check);
4. verifies the asset-ID join: the embedding row's key matches assets.id, and that same ID is one we store in SQLite (shared/photoflow/core);
5. computes coverage: count(embeddings) / count(assets) (and, if cheap, broken down by whether the asset is an image vs video);
6. prints a human-readable report and writes/refreshes the findings doc.

It hardcodes nothing destructive, takes no write path, and is safe to re-run.

Deliverables

1. scripts/pgvector_spike.py — disposable read-only probe (removed or left as a documented one-off after M1.5 internalizes its findings).
2. IMMICH_DB_URL in .env.example; optional immich_db_url field in config.py.
3. docs/superpowers/specs/2026-06-27-pgvector-embedding-findings.md — the one-page pinned schema that becomes M1.5's contract: table, embedding column, vector dimension, distance operator, the embedding → assetId → SQLite asset join, coverage %, Immich version, and the (optional) read-only-role recipe.
4. Roadmap correction: M1.5's status note updated to "dependency spike pending/this spec" instead of "verified in M1."

Out of scope (explicitly M1.5)

CLIP clustering; a tested shared/photoflow/immich/db.py reader; nearest-neighbor queries at scale; any use of the embeddings beyond the feasibility checks above.

Risks

• Embeddings unreadable / not joinable → M1.5 blocked; spike's whole point is to surface this early and cheaply.
• Schema differs from the known shape → expected and handled by catalog discovery, not hardcoded names.
• Coverage near zero (re-run unfinished) → not a spike failure; recorded as a fact so M1.5 can sequence around it.