docs(spec): harden pgvector spike from ce-doc-review (11 findings)

Multi-persona review (coherence, feasibility, product-lens, security-lens,
adversarial) surfaced that the spike proved DB access but not that DB access
was the right path or that the signal was useful, and pinned a contract
against a private, in-flight-changing schema. Applied 11 fixes:

- Requirement 0: test the "REST can't expose embeddings" premise instead of
  asserting it; record which endpoints were checked and why insufficient.
- Reframe Req 1 "real go/no-go" to access-only; signal-usefulness is M1.5's
  first task, not this spike's.
- Mark the schema unsupported/internal, version-pinned; require an M1.5
  re-probe/version-guard per Immich upgrade; tie shape to recorded model+ver.
- Fix probe correctness: pgvector adapter / server-side vector_dims (psycopg3
  returns vector as string); Postgres-internal join with optional SQLite
  cross-check; coverage over the image/embeddable population (both ratios).
- DB-enforced read-only session; standalone .env loading; psycopg+pgvector
  added to Deliverables; roadmap correction now fixes both false claims.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md

@@ -14,9 +14,10 @@ on a prerequisite that never ran. This spec defines that prerequisite.
 
 M1.5 wants to cluster photos by visual similarity using the CLIP embeddings Immich already
 computes — the rescue signal for the GPS-poor old library where timestamp/GPS are weakest.
-Immich's REST API does not cleanly expose raw embedding vectors, so the viable path is
+The working assumption is that Immich's REST API does not cleanly expose raw embedding
-**read-only access to Immich's Postgres pgvector embeddings**. Before M1.5 can build
+vectors, so the viable path is **read-only access to Immich's Postgres pgvector embeddings**.
-clustering on that path, this spike must prove the path exists and pin its shape.
+That assumption is load-bearing — it must be *tested*, not asserted (see Requirement 0). Before
+M1.5 can build clustering on that path, this spike must prove the path exists and pin its shape.
 
 ## Scope
 
@@ -28,21 +29,43 @@ clustering on that path, this spike must prove the path exists and pin its shape
 
 ## Requirements (pass/fail)
 
-The spike succeeds when it answers all four against the **live** database and records the
+The spike succeeds when it answers all of the following against the **live** database and
-answers in the findings doc. Phrased as plain questions:
+records the answers in the findings doc. Phrased as plain questions:
 
+0. **Does M1.5 actually need raw vectors (vs. a REST similarity query)?** Before relying on
+   DB access, state what M1.5's clustering needs — raw embedding vectors or a pairwise
+   similarity-neighbor query — and record which Immich REST endpoints were checked (e.g.
+   `/api/search/smart`, any asset-similarity/duplicate endpoint) and *why each is
+   insufficient*. This turns the load-bearing "REST can't expose embeddings" premise into a
+   documented finding; if a REST path suffices, the DB-access path below is unnecessary.
 1. **Can we read the CLIP embeddings at all?** Connect read-only over the LAN and read the
-   embedding vectors Immich stores. *This is the real go/no-go: if the embeddings are not
+   embedding vectors Immich stores. *This de-risks **access** only: if the embeddings are not
-   readable, M1.5 needs a different plan.*
+   readable, M1.5 needs a different plan. It does **not** prove the signal is useful (see the
-2. **Can we join each embedding back to a photo we already track?** Our SQLite store keys
+   signal-usefulness note below).*
-   assets by Immich asset ID. Confirm each embedding row carries that same ID so an
+2. **Can we join each embedding back to a photo we already track?** Confirm **inside
-   embedding can be matched to a known asset. *Without the join the vectors are unusable.*
+   Postgres** that each embedding row's key references `assets.id` — the join that must hold
+   for the vectors to be usable. That asset ID is the same key our SQLite store uses, so a
+   cross-check against SQLite (`shared/photoflow/core`) is *optional* confirmation and must
+   degrade gracefully when the store is unpopulated (a fresh checkout has no `ingest` run),
+   rather than failing this requirement for the wrong reason.
 3. **What are the exact shapes?** Record the table name, embedding column, **vector
-   dimension**, and the correct pgvector **distance operator** for similarity. These have
+   dimension**, and the correct pgvector **distance operator** for similarity. These are
-   drifted across Immich versions, so they must be read from *this* DB, not assumed.
+   **model- and version-dependent** (and have drifted across Immich versions), so they must be
-4. **What is the coverage?** What fraction of library assets currently have an embedding
+   read from *this* DB, not assumed — and the recorded shape is valid only for the model +
-   (the user is re-running CLIP with a stronger model — has it finished?). *Tells M1.5 how
+   Immich version observed (see Requirement 4 / Risks).
-   much it can lean on the signal.*
+4. **What is the coverage?** What fraction of the **embeddable** library currently has an
+   embedding. Compute against the image/embeddable population, not all assets —
+   `count(distinct embedding.assetId) / count(assets WHERE type = 'IMAGE')` (or Immich's
+   equivalent asset-type filter) — since videos and other non-image rows CLIP never embeds
+   would otherwise deflate the ratio. Record both the raw and image-only ratios. *Tells M1.5
+   how much it can lean on the signal — but the user is re-running CLIP with a stronger model,
+   so coverage (and the dimension in Requirement 3) is a snapshot of whichever model is live
+   when the probe runs.*
+
+**Signal-usefulness is M1.5's first task, not this spike's.** Passing Requirements 1–4 proves
+the embeddings are reachable and well-shaped; it does **not** prove visual similarity actually
+rescues trip detection in the GPS-poor library. Before building clustering, M1.5 must validate
+the signal (e.g. eyeball nearest-neighbour quality on a sample of the GPS-poor set).
 
 ## What we already know (to verify, not assume)
 
@@ -53,6 +76,11 @@ vectors. Names and dimension have changed across versions and the user's re-run
 stronger model, so the probe **discovers** these rather than trusting them — the list above is
 only the set of candidates to probe first.
 
+This is Immich's **undocumented, internal** schema with no deprecation contract: it can be
+renamed or restructured on any Immich upgrade. The probe therefore records the exact Immich
+version observed alongside the shape, and M1.5 must budget a re-probe / version-guard on every
+Immich upgrade — the "contract" the findings doc hands M1.5 is version-pinned, not durable.
+
 ## Approach
 
 Disposable read-only probe + two durable artifacts (chosen over a throwaway-only or a
@@ -63,11 +91,23 @@ a written contract is the cheap part that saves M1.5 from guessing).
 
 - New optional env var **`IMMICH_DB_URL`** (a Postgres DSN), added to `.env.example` and
   loaded by `config.py` as an **optional** field (REST creds stay required; the DSN is only
-  needed for the spike / M1.5).
+  needed for the spike / M1.5). `config.py` reads only `os.environ`, and the repo loads `.env`
-- Simplest credentials that work: Immich's existing Postgres user, kept read-only by the
+  solely via docker-compose's `env_file`, so a standalone host-run probe must populate the
-  probe issuing **only `SELECT`**. A dedicated least-privilege read-only role is documented as
+  environment itself — either run it via `docker compose run` (so `env_file` applies) or load
-  recommended hardening for M1.5 but is **not required** to prove feasibility.
+  `.env` explicitly first (e.g. `set -a; source .env; set +a`). If run in-container, confirm
-- Driver: `psycopg` (psycopg3). Add to the spike's dependencies; M1.5 will formalize it.
+  container-to-Immich-Postgres network reachability.
+- Read-only must be **enforced at the DB/session layer**, not just by which statements the
+  script issues: open the session read-only (`SET default_transaction_read_only = on` /
+  `SET TRANSACTION READ ONLY`) or connect via a role granted only `SELECT`. The simplest
+  credentials that work are Immich's existing Postgres user, but that user is write/DDL-capable
+  against the **source-of-truth** DB, so the session-level guard is required to remove write
+  capability while the spike runs. A dedicated least-privilege read-only role remains
+  recommended hardening for M1.5.
+- Driver: `psycopg` (psycopg3), plus the **`pgvector`** Python package. psycopg3 returns a
+  `vector` column as a *string* unless the adapter is registered, so call
+  `pgvector.psycopg.register_vector(conn)` after connect — or read the dimension server-side
+  (`SELECT vector_dims(embedding) …` / catalog `atttypmod`) and run the `<=>` check in SQL,
+  which needs no adapter. Add these to the spike's dependencies; M1.5 will formalize them.
 
 ### The probe
 
@@ -78,13 +118,17 @@ read-only. It:
 2. discovers candidate embedding tables/columns from the catalog (probe `smart_search` /
    `embedding` first, then fall back to scanning `information_schema` for `vector`-typed
    columns) — so it survives version drift;
-3. reads one embedding and reports its **dimension** and a sample of the **distance operator**
+3. reads one embedding and reports its **dimension** — via the registered `pgvector` adapter
-   working (e.g. a `... ORDER BY embedding <=> embedding LIMIT 5` self-similarity sanity
+   or server-side `vector_dims(embedding)`, **not** the length of a raw string — and a sample
-   check);
+   of the **distance operator** working (e.g. a `... ORDER BY embedding <=> embedding LIMIT 5`
-4. verifies the **asset-ID join**: the embedding row's key matches `assets.id`, and that same
+   self-similarity sanity check, casting literals to `::vector` as needed);
-   ID is one we store in SQLite (`shared/photoflow/core`);
+4. verifies the **asset-ID join inside Postgres**: the embedding row's key references
-5. computes **coverage**: `count(embeddings) / count(assets)` (and, if cheap, broken down by
+   `assets.id`. As *optional* confirmation it also checks whether that ID is one we store in
-   whether the asset is an image vs video);
+   SQLite (`shared/photoflow/core`), degrading gracefully (not failing the check) when the
+   store is unpopulated;
+5. computes **coverage** against the embeddable population:
+   `count(distinct embedding.assetId) / count(assets WHERE type = 'IMAGE')` (or Immich's
+   equivalent asset-type filter), recording **both** the raw and image-only ratios;
 6. prints a human-readable report **and** writes/refreshes the findings doc.
 
 It hardcodes nothing destructive, takes no write path, and is safe to re-run.
@@ -93,13 +137,17 @@ It hardcodes nothing destructive, takes no write path, and is safe to re-run.
 
 1. `scripts/pgvector_spike.py` — disposable read-only probe (removed or left as a documented
    one-off after M1.5 internalizes its findings).
-2. `IMMICH_DB_URL` in `.env.example`; optional `immich_db_url` field in `config.py`.
+2. `IMMICH_DB_URL` in `.env.example`; optional `immich_db_url` field in `config.py`. Spike
-3. **`docs/superpowers/specs/2026-06-27-pgvector-embedding-findings.md`** — the one-page pinned
+   dependencies added: `psycopg` (psycopg3) and the `pgvector` Python package.
+3. **`docs/superpowers/specs/2026-06-27-pgvector-embedding-findings.md`** — the one-page
    schema that becomes M1.5's contract: table, embedding column, vector dimension, distance
-   operator, the embedding → `assetId` → SQLite `asset` join, coverage %, Immich version, and
+   operator, the embedding → `assetId` → SQLite `asset` join, coverage % (raw and image-only),
-   the (optional) read-only-role recipe.
+   the model **and** Immich version observed, and the (optional) read-only-role recipe. The doc
-4. Roadmap correction: M1.5's status note updated to "dependency spike pending/this spec"
+   must state that this is Immich's **unsupported internal schema**, valid only for the recorded
-   instead of "verified in M1."
+   model + version, and that M1.5 re-probes / version-guards on every Immich upgrade.
+4. Roadmap correction: fix **both** false claims named in "Why this exists" — the narrative
+   that M1 "runs a read-only feasibility spike" (→ "spike defined/pending per this spec") **and**
+   M1.5's status note "verified in M1" (→ "dependency spike pending/this spec").
 
 ## Out of scope (explicitly M1.5)