immich-photo-flow/docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md

# pgvector embedding feasibility spike — design

**Date:** 2026-06-27
**Milestone dependency:** unblocks **M1.5** (visual similarity). See `docs/ROADMAP.md`.
**Status:** design

## Why this exists

The roadmap narrative claims M1 "runs a read-only feasibility spike" and that M1.5's
dependency is "verified in M1." It is not: the M1 plan explicitly deferred the spike
(`docs/superpowers/plans/2026-06-27-immich-photo-flow-m1.md:3710` — *"pgvector spike →
out of this plan by decision (tracked separately; M1.5 dependency)"*). So M1.5 is blocked
on a prerequisite that never ran. This spec defines that prerequisite.

M1.5 wants to cluster photos by visual similarity using the CLIP embeddings Immich already
computes — the rescue signal for the GPS-poor old library where timestamp/GPS are weakest.
Immich's REST API does not cleanly expose raw embedding vectors, so the viable path is
**read-only access to Immich's Postgres pgvector embeddings**. Before M1.5 can build
clustering on that path, this spike must prove the path exists and pin its shape.

## Scope

**Feasibility only.** Read-only. Throwaway probe. The spike:

- does **not** build clustering, a tested module, or any `shared/` reader;
- does **not** write to Immich or its database (only `SELECT`);
- exists to turn unknowns into **pinned facts** and hand M1.5 a verified contract.

## Requirements (pass/fail)

The spike succeeds when it answers all four against the **live** database and records the
answers in the findings doc. Phrased as plain questions:

1. **Can we read the CLIP embeddings at all?** Connect read-only over the LAN and read the
   embedding vectors Immich stores. *This is the real go/no-go: if the embeddings are not
   readable, M1.5 needs a different plan.*
2. **Can we join each embedding back to a photo we already track?** Our SQLite store keys
   assets by Immich asset ID. Confirm each embedding row carries that same ID so an
   embedding can be matched to a known asset. *Without the join the vectors are unusable.*
3. **What are the exact shapes?** Record the table name, embedding column, **vector
   dimension**, and the correct pgvector **distance operator** for similarity. These have
   drifted across Immich versions, so they must be read from *this* DB, not assumed.
4. **What is the coverage?** What fraction of library assets currently have an embedding
   (the user is re-running CLIP with a stronger model — has it finished?). *Tells M1.5 how
   much it can lean on the signal.*

## What we already know (to verify, not assume)

Immich historically stores CLIP vectors in a **`smart_search`** table with an **`embedding`**
column of pgvector type `vector`, keyed by `assetId` referencing `assets.id`; similarity uses
**cosine** distance (`<=>`); the legacy default model (`ViT-B-32`) produced **512-dim**
vectors. Names and dimension have changed across versions and the user's re-run uses a
stronger model, so the probe **discovers** these rather than trusting them — the list above is
only the set of candidates to probe first.

## Approach

Disposable read-only probe + two durable artifacts (chosen over a throwaway-only or a
build-the-module-now approach: feasibility-only honors the roadmap, but capturing the DSN and
a written contract is the cheap part that saves M1.5 from guessing).

### Connection

- New optional env var **`IMMICH_DB_URL`** (a Postgres DSN), added to `.env.example` and
  loaded by `config.py` as an **optional** field (REST creds stay required; the DSN is only
  needed for the spike / M1.5).
- Simplest credentials that work: Immich's existing Postgres user, kept read-only by the
  probe issuing **only `SELECT`**. A dedicated least-privilege read-only role is documented as
  recommended hardening for M1.5 but is **not required** to prove feasibility.
- Driver: `psycopg` (psycopg3). Add to the spike's dependencies; M1.5 will formalize it.

### The probe

A single disposable script, **`scripts/pgvector_spike.py`**, that is idempotent and
read-only. It:

1. connects using `IMMICH_DB_URL`;
2. discovers candidate embedding tables/columns from the catalog (probe `smart_search` /
   `embedding` first, then fall back to scanning `information_schema` for `vector`-typed
   columns) — so it survives version drift;
3. reads one embedding and reports its **dimension** and a sample of the **distance operator**
   working (e.g. a `... ORDER BY embedding <=> embedding LIMIT 5` self-similarity sanity
   check);
4. verifies the **asset-ID join**: the embedding row's key matches `assets.id`, and that same
   ID is one we store in SQLite (`shared/photoflow/core`);
5. computes **coverage**: `count(embeddings) / count(assets)` (and, if cheap, broken down by
   whether the asset is an image vs video);
6. prints a human-readable report **and** writes/refreshes the findings doc.

It hardcodes nothing destructive, takes no write path, and is safe to re-run.

## Deliverables

1. `scripts/pgvector_spike.py` — disposable read-only probe (removed or left as a documented
   one-off after M1.5 internalizes its findings).
2. `IMMICH_DB_URL` in `.env.example`; optional `immich_db_url` field in `config.py`.
3. **`docs/superpowers/specs/2026-06-27-pgvector-embedding-findings.md`** — the one-page pinned
   schema that becomes M1.5's contract: table, embedding column, vector dimension, distance
   operator, the embedding → `assetId` → SQLite `asset` join, coverage %, Immich version, and
   the (optional) read-only-role recipe.
4. Roadmap correction: M1.5's status note updated to "dependency spike pending/this spec"
   instead of "verified in M1."

## Out of scope (explicitly M1.5)

CLIP clustering; a tested `shared/photoflow/immich/db.py` reader; nearest-neighbor queries at
scale; any use of the embeddings beyond the feasibility checks above.

## Risks

- **Embeddings unreadable / not joinable** → M1.5 blocked; spike's whole point is to surface
  this early and cheaply.
- **Schema differs from the known shape** → expected and handled by catalog discovery, not
  hardcoded names.
- **Coverage near zero** (re-run unfinished) → not a spike failure; recorded as a fact so M1.5
  can sequence around it.