Code-review follow-ups on the pgvector spike:
- self-similarity sanity check now compares to the seed's own self-distance
(0 for cosine/L2, ~-1 for inner product <#>) instead of a hardcoded ~0, so it
no longer misfires if Immich ever uses a vector_ip_ops index (correctness P3).
- mark the deliberate raw-SQLite read in the probe as a spike-only exception and
add an M1.5 TODO that the real pgvector reader belongs in shared/photoflow/immich
and SQLite access in shared/photoflow/core (project-standards P2 x2).
- document join_and_coverage's return shape; extract a _pct() helper to dedupe the
coverage-percentage formatting (maintainability P3 x2).
Findings doc refreshed from the latest live run (coverage now ~46%, re-run ongoing).
scripts/pgvector_spike.py probes Immich's Postgres read-only (session-level
read-only guard) and answers the M1.5 prerequisites against the live DB:
catalog-discovered embedding table/column, vector dimension, distance operator
(from the index opclass), the embedding->asset FK join, and coverage over the
IMAGE population. Discovers the asset table name from the FK (asset, not the
legacy assets) rather than hardcoding it, so it survives Immich version drift.
Findings (this Immich version): smart_search.embedding, 1152-dim, cosine <=>
(vector_cosine_ops), assetId->asset.id join clean (0 orphans), ~44% image
coverage (CLIP re-run in progress). Doc is M1.5's version-pinned contract.
Multi-persona review (coherence, feasibility, product-lens, security-lens,
adversarial) surfaced that the spike proved DB access but not that DB access
was the right path or that the signal was useful, and pinned a contract
against a private, in-flight-changing schema. Applied 11 fixes:
- Requirement 0: test the "REST can't expose embeddings" premise instead of
asserting it; record which endpoints were checked and why insufficient.
- Reframe Req 1 "real go/no-go" to access-only; signal-usefulness is M1.5's
first task, not this spike's.
- Mark the schema unsupported/internal, version-pinned; require an M1.5
re-probe/version-guard per Immich upgrade; tie shape to recorded model+ver.
- Fix probe correctness: pgvector adapter / server-side vector_dims (psycopg3
returns vector as string); Postgres-internal join with optional SQLite
cross-check; coverage over the image/embeddable population (both ratios).
- DB-enforced read-only session; standalone .env loading; psycopg+pgvector
added to Deliverables; roadmap correction now fixes both false claims.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Defines the read-only Postgres/pgvector feasibility spike that the
roadmap claims ran in M1 but the M1 plan deferred. Pins the four
pass/fail requirements, the disposable-probe approach, and the
findings-doc contract that unblocks M1.5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
m038
