m038
1a6afe86fc
Code-review follow-ups on the pgvector spike: - self-similarity sanity check now compares to the seed's own self-distance (0 for cosine/L2, ~-1 for inner product <#>) instead of a hardcoded ~0, so it no longer misfires if Immich ever uses a vector_ip_ops index (correctness P3). - mark the deliberate raw-SQLite read in the probe as a spike-only exception and add an M1.5 TODO that the real pgvector reader belongs in shared/photoflow/immich and SQLite access in shared/photoflow/core (project-standards P2 x2). - document join_and_coverage's return shape; extract a _pct() helper to dedupe the coverage-percentage formatting (maintainability P3 x2). Findings doc refreshed from the latest live run (coverage now ~46%, re-run ongoing).
85 lines
4.1 KiB
Markdown
85 lines
4.1 KiB
Markdown
# pgvector embedding feasibility spike — findings
|
||
|
||
**Generated:** 2026-06-27 18:49 UTC by `scripts/pgvector_spike.py`
|
||
**Status:** machine-probed against the live Immich Postgres.
|
||
**This is M1.5's contract.** See the design spec:
|
||
`docs/superpowers/specs/2026-06-27-pgvector-embedding-spike-design.md`.
|
||
|
||
> ⚠️ **Immich's undocumented, internal schema — no deprecation contract.**
|
||
> Everything below is valid **only** for the model + Immich version observed and
|
||
> can be renamed/restructured on any Immich upgrade. M1.5 must re-run this probe
|
||
> (or version-guard) on every Immich upgrade. The "contract" is version-pinned,
|
||
> not durable.
|
||
|
||
## Requirement 0 — does M1.5 need raw vectors (vs. a REST query)?
|
||
|
||
M1.5 clusters photos by *visual similarity* at trip level — it needs either the
|
||
raw CLIP embedding vectors or an arbitrary asset→asset nearest-neighbour query.
|
||
Immich's REST surface provides neither:
|
||
|
||
- `POST /api/search/smart` — **text→image** CLIP search: takes a text query,
|
||
returns assets. It never returns embedding vectors and cannot do asset→asset
|
||
similarity without a text prompt. Insufficient.
|
||
- `POST /api/search/metadata`, `/api/search/random` — metadata/random only; no
|
||
embeddings, no similarity.
|
||
- Duplicate detection (`/api/duplicates`) consumes embeddings *internally* but only
|
||
surfaces near-duplicate groups above Immich's own threshold — not a tunable
|
||
pairwise similarity usable for trip-level clustering. Insufficient.
|
||
- No documented endpoint returns raw CLIP vectors or arbitrary k-NN neighbours.
|
||
|
||
**Conclusion:** the load-bearing "REST can't expose embeddings" premise holds for
|
||
Immich's documented API → **read-only Postgres access (below) is the viable path.**
|
||
(Re-confirm against the OpenAPI of the running version on upgrade.)
|
||
|
||
## Requirements 1–4 — probed facts
|
||
|
||
| # | Question | Answer |
|
||
|---|----------|--------|
|
||
| 1 | Embeddings readable? | **yes — read 20727 distinct embedding(s)** |
|
||
| 2 | Join to asset table? | **20727/20727 embeddings join to asset via assetId** |
|
||
| 3 | Table / column | `public.smart_search` / `embedding` |
|
||
| 3 | Vector dimension | **1152** (live sample=1152, declared typmod=1152) |
|
||
| 3 | Distance operator | **<=> (cosine) — from index opclass vector_cosine_ops** |
|
||
| 3 | Operator sanity check | OK — nearest neighbour (a duplicate (identical-vector) asset) at distance 0 (seed self-distance 0) |
|
||
| 4 | Coverage (raw) | **45.2%** (20727/45854 assets) |
|
||
| 4 | Coverage (image-only) | **46.6%** (20323/43601 IMAGE assets) |
|
||
|
||
- **Asset-key join:** FK smart_search.assetId -> asset.id
|
||
- **Orphan embeddings** (no matching asset): 0
|
||
- **SQLite cross-check (optional):** skipped — no SQLite store at data/trip-cluster.db (fresh checkout; not a failure)
|
||
|
||
## The join M1.5 relies on
|
||
|
||
`smart_search.assetId` → `asset.id` →
|
||
SQLite `assets.immich_id` (`shared/photoflow/core`). The Immich asset UUID is the
|
||
same key our store uses, so embeddings index straight onto tracked photos.
|
||
|
||
> ⚠️ Immich's asset table is **`asset`** in the observed version
|
||
> (it was `assets` in older versions). The probe discovers this from the FK; M1.5's
|
||
> reader must not hardcode the name.
|
||
|
||
## Environment observed
|
||
|
||
- **Postgres:** 18.2 (Debian 18.2-1.pgdg12+1)
|
||
- **pgvector extension:** 0.8.1
|
||
- **Schema age proxy (latest migration):** unknown
|
||
- **Immich server version:** _record from the Immich UI / `GET /api/server/version`_
|
||
— not reliably available in the DB.
|
||
- **CLIP model:** _record the model from Immich's Machine-Learning settings_ — the
|
||
user is re-running CLIP with a stronger model, so dimension + coverage above are a
|
||
snapshot of whichever model was live at probe time.
|
||
|
||
## Read-only access (recommended hardening for M1.5)
|
||
|
||
The probe enforces read-only at the session layer
|
||
(`SET default_transaction_read_only = on` + psycopg `read_only`), which neutralises
|
||
write capability even with Immich's write/DDL-capable `postgres` user. For M1.5,
|
||
provision a dedicated least-privilege role instead:
|
||
|
||
```sql
|
||
CREATE ROLE photoflow_ro LOGIN PASSWORD '...';
|
||
GRANT CONNECT ON DATABASE immich TO photoflow_ro;
|
||
GRANT USAGE ON SCHEMA public TO photoflow_ro;
|
||
GRANT SELECT ON public.smart_search, public.asset TO photoflow_ro;
|
||
```
|