diff --git a/CLAUDE.md b/CLAUDE.md index 6355263..5777af9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -132,6 +132,8 @@ After updating, also create the new trip's page tree under `user/pages/01.trips/ Always use `make` commands for anything on the production server (`make remote-install-plugins`, `make remote-clean`, etc.) — never SSH directly since credentials live in `.env`. If a remote operation isn't covered by an existing `make` command, either ask the user to run it manually or suggest adding a new `make` command if it seems reusable. +For a full upgrade/deploy through local → test → prod (ordered steps, smoke checklist, rollback), follow the runbook at [`docs/guides/deploy-cycle.md`](docs/guides/deploy-cycle.md). + ### Content sync - `make content-push` — commit and push `user/` to Gitea (triggers production pull via webhook) diff --git a/deploy/env/prod/system.yaml b/deploy/env/prod/system.yaml index a859bf9..efef65f 100644 --- a/deploy/env/prod/system.yaml +++ b/deploy/env/prod/system.yaml @@ -1,4 +1,8 @@ -# Production-only Grav config overrides. +# Deployed-environment Grav config overrides (test AND prod). +# +# Both server environments share this one file so test stays a faithful dress +# rehearsal of prod: deploy/env/test/system.yaml is a symlink to this file. +# Edit here and both environments move together — never let them drift. # # Deep-merged OVER the committed user/config/system.yaml via Grav's # per-environment config mechanism: on the server this file is deployed to @@ -8,9 +12,10 @@ # # These values are deliberately NOT in the committed system.yaml because they # would break local development (see CLAUDE.md §1 — dev keeps twig.cache:false -# so theme edits take effect immediately). Prod is the only place they apply. +# so theme edits take effect immediately). They apply only on the deployed +# hosts, never on a local dev checkout. # -# Deploy with: make remote-apply-env-prod +# Deploy with: make remote-apply-env-test / make remote-apply-env-prod # The user/env/ tree is outside the content repo's tracked folders, so it is # NOT restored by content-push / git-sync / remote-fetch-content — re-run the # target above after any fresh install. diff --git a/deploy/env/test/system.yaml b/deploy/env/test/system.yaml new file mode 120000 index 0000000..022f162 --- /dev/null +++ b/deploy/env/test/system.yaml @@ -0,0 +1 @@ +../prod/system.yaml \ No newline at end of file diff --git a/docs/guides/deploy-cycle.md b/docs/guides/deploy-cycle.md new file mode 100644 index 0000000..ba26b63 --- /dev/null +++ b/docs/guides/deploy-cycle.md @@ -0,0 +1,133 @@ +# Upgrade & Deploy Cycle: local → test → prod + +This runbook is the repeatable procedure for shipping a Grav upgrade or any +server-affecting change (core version, plugins, config, theme) through the three +environments. It was distilled from the 2026-07 Grav 2.0.4→2.0.7 cutover, where +every production surprise traced back to one of the desyncs this procedure now +forces you to check. + +**Governing principle:** `test` is a **full dress rehearsal of `prod`** — same +config, same `-test`/`-prod` make targets, same order. A gotcha only gets caught +on test if test is a faithful mirror of prod. Do not shortcut test. + +All server operations go through `make remote-*` targets (never raw SSH — the +targets build the SSH connection from `.env.`, which must never be read +directly). Every `remote-*` target has `-test` and `-prod` variants; a bare +target fails via `guard-env`. + +--- + +## The mental model: three places state lives + +Every failure in the reference cutover was a desync between these three layers. +Before and after each deploy step, ask: *are they in sync?* + +| Layer | Location | Synced by | Failure mode | +|---|---|---|---| +| Plugin **code** | `user/plugins//` | GPM only (gitignored `/plugins/*`) | can vanish while config remains → plugin won't enable | +| **Repo config** | `user/config/…` | `content-push` / git-sync | holds GPM channel + is where the version floor bites | +| **Host config** | `user/env//config/…` | nothing — server-only | not restored on fresh install; must be re-applied; **must be gitignored** | + +Referenced gotcha docs: +- `docs/solutions/integration-issues/grav-plugin-config-without-code-wont-enable.md` — code-vs-config desync. +- `docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md` — stale `GRAV_VERSION` / version floor. +- `docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md` — gitignore is the sync boundary; env-tree leak. +- `docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md` — plugin config must live in the tracked override. + +--- + +## Phase 0 — Local (author + prove the change) + +1. Make the change in the repo: + - GPM channel: `gpm.releases: stable` in `user/config/system.yaml` (authoritative; reaches servers via content pull, so it must be right **before** any server GPM op). + - `plugins.txt` — the GPM-managed set only. **Never** add `git-sync` (it is remote-only). + - Prod-only overrides (Twig cache/debug, `debugger.shutdown.close_connection: false`) in `deploy/env/prod/system.yaml` — **never** commit prod values into `user/config/system.yaml`. + - **Bump `GRAV_VERSION` in `.env.test` and `.env.prod`** to the target version. A stale value here installs the wrong core (an rc), which then blocks the `api` plugin and 404s admin. +2. `make build-assets` if you touched `js/src/*` (never hand-edit the bundled `js/*.js`). +3. Run the dev server (`docker compose … up`) and the Playwright suite. +4. Pre-flight assertions: + - `gpm.releases` is `stable`. + - `plugins.txt` is correct and does **not** contain `git-sync`. + - No prod Twig values leaked into the committed `system.yaml`. +5. Commit. `make content-push`. + +--- + +## Phase 1 — Test (the rehearsal — catch things here) + +### Pre-flight + +- `make remote-git-sync-disable-test` **before any content reset.** This is the safety catch for the whole window: it stops a half-migrated state (e.g. a fresh install-time `versions.yaml`) from auto-committing and pushing on the first sync. + +### Apply — in this fixed order + +``` +make remote-fetch-content-test # 1. clean-reset synced folders to repo state +make remote-upgrade-grav-test # 2. gpm self-upgrade (rewrites schema — expect drift) +make remote-update-plugins-test # 3. gpm update the plugins.txt set +make remote-gpm-install-test PKG=git-sync # 4. EXPLICITLY (re)install each remote-only plugin +make remote-apply-env-test # 5. re-deploy the env override (not synced; gone after install) +``` + +Why each matters: +- **Step 4** is non-optional even if git-sync "was already there" — remote-only plugins are not in `plugins.txt`, so nothing in steps 1–3 restores them. If the code is missing, the plugin is inert despite valid config. +- **Step 5** re-writes `user/env//config/…` from `deploy/env//`. The env tree is not synced by anything, so a fresh install loses it until you re-apply. + +### Verify (smoke checklist — this is the payoff) + +- **Code present, not just config:** `ls user/plugins//` for every expected plugin (especially `git-sync`). An empty/absent dir = reinstall (step 4). *(Do this via an ssh one-liner you run, or `make remote-diag-test`.)* +- **HTTP:** `/` → 200, `/admin` → 200, `/api/v1/pages` → 401, `/gpx-manager` → 200. Watch for the double-`Content-Encoding` garbage page (fix: `debugger.shutdown.close_connection: false` in the env override — already in `deploy/env/prod/system.yaml`). +- **Post smoke test:** submit one entry via `/post` and confirm it appears in the trip feed immediately. This proves the `cache-on-save` plugin works with prod caching on. +- **Config drift:** `make remote-diag-test` — diff server config against the repo. Fold any *intended* schema migration (e.g. the Twig-3 `strict_mode` flags a `self-upgrade` writes) back into `user/config/system.yaml`, or the next `fetch-content` reverts it. + +### Re-enable + prove sync + +- `make remote-git-sync-enable-test`. +- Confirm a content push round-trips to the server, **and** that no secret/boomerang commit lands on Gitea. Verify `/env/` is gitignored so the env tree (which holds the token, JWT, CSRF salt) can never enter the sync add-set. + +--- + +## Phase 2 — Prod (repeat identically — should be mechanical) + +Run the **exact same sequence** with `-prod` targets. Because test rehearsed it, +prod holds no surprises. Differences to layer on: + +- Optional: `make remote-maintenance-on-prod` at the start, `remote-maintenance-off-prod` at the end, for a clean window. +- Confirm secrets are valid/rotated and `/env/` is gitignored **before** `remote-git-sync-enable-prod`. Re-enable git-sync **last**. +- After a clean cutover, bump the outer-repo submodule pin to the finished `user/` commit — and **push `user/` before the outer repo** (the superproject references a child SHA that must already exist upstream). + +``` +make remote-git-sync-disable-prod +make remote-fetch-content-prod +make remote-upgrade-grav-prod +make remote-update-plugins-prod +make remote-gpm-install-prod PKG=git-sync +make remote-apply-env-prod +# ── smoke checklist (same as test) ── +make remote-git-sync-enable-prod +``` + +For a first-time / from-scratch prod bring-up, `make remote-install-prod` does the +full install; then still run `remote-apply-env-prod` and the smoke checklist, and +reinstall remote-only plugins explicitly. + +--- + +## Rollback & safety + +- **git-sync stays disabled through the whole apply window** on each host — it is the catch that prevents a half-migrated state from auto-pushing. +- **Content** is a git repo: a bad content deploy is recoverable with `make remote-fetch-content-` back to a known commit. +- **Core + plugins** are GPM-reinstallable (`remote-upgrade-grav`, `remote-update-plugins`, `remote-gpm-install PKG=…`). +- The one thing tooling cannot regenerate is the un-synced `user/env//` tree — its source of truth is `deploy/env//`, so keep that current and re-apply with `remote-apply-env-`. + +--- + +## One-line invariants (the through-line) + +1. `test` is config-identical to `prod`, run with the same targets in the same order. +2. Verify the **code layer** (`ls user/plugins//`), not just config, on every deploy. +3. Reinstall **remote-only** plugins (git-sync) explicitly — nothing else restores them. +4. `GRAV_VERSION` in `.env.` and `gpm.releases: stable` are correct **before** any server GPM op. +5. Re-apply the **env override** after every install; keep `/env/` **gitignored**. +6. git-sync **off** during the window, **on** last; confirm the round-trip carries no secrets. +7. Diagnose actual state before changing config — an `ls` or `remote-diag` beats a guess. diff --git a/docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md b/docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md new file mode 100644 index 0000000..451a3ac --- /dev/null +++ b/docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md @@ -0,0 +1,243 @@ +--- +title: "Secret exposure under bidirectional Grav git-sync: gitignore is the only boundary (and the tracked-file boomerang trap)" +date: 2026-07-05 +module: git-sync +problem_type: architecture_pattern +component: tooling +severity: high +category: architecture-patterns +applies_when: + - "Enabling bidirectional Grav git-sync (direction: both, on_save: true) so prod can push content back to Gitea" + - "Auditing whether the server can leak secrets (tokens, password hashes, signing salts) off the production host" + - "A per-install runtime-generated value is being persisted into a tracked config file that also carries functional config" + - "Deciding where a per-install secret must live so it never round-trips" + - "A config value keeps ping-ponging or re-committing itself across environments after each sync" +tags: + - git-sync + - secret-exposure + - gitignore + - grav + - popularity-salt + - config-boundary + - bidirectional-sync + - per-install-secret + - env-tree-leak +--- + +# Secret exposure under bidirectional Grav git-sync: gitignore is the only boundary (and the tracked-file boomerang trap) + +> **Correction (2026-07-05):** An earlier version of this doc claimed the sync +> add-set was *scoped to the configured `folders`*, and concluded that +> `accounts/` and `user/env/` were "safe by construction" because they sit +> outside `pages/config/themes`. **That model is wrong and caused a live secret +> leak.** git-sync's auto-commit stages files **outside** the configured folders; +> the only reliable exclusion is `.gitignore`. The corrected model is below. + +## Context + +The intotheeast.com Grav site runs the `git-sync` plugin on production in +**bidirectional** mode: `direction: both`, `on_save: true`, a webhook at +`/_git-sync`, and configured `folders: pages, config, themes`. Bidirectional +means the server both *pulls* content authored elsewhere **and** *pushes* +content authored on the server (Admin edits, `/post` submissions, uploads) back +to the shared Gitea repo. + +The operator's question was: **"Can I safely enable bidirectional prod → Gitea +sync without leaking secrets?"** Production holds things that must never reach a +shared repo — API tokens, JWT signing secrets, CSRF salts, the git-sync token +itself. + +The dangerous, tempting answer is "the plugin only syncs `pages/config/themes`, +so anything outside those folders is safe." **This is false, and acting on it +leaked secrets.** See the incident below. + +## The incident (what actually happened) + +Production's git-sync auto-commit `9337003` ("(Grav GitSync) Automatic Commit +...") pushed the **entire `user/env/intotheeast.com/config/` tree** to Gitea — +including `api-private.php` (JWT secret), `security-private.php` (CSRF salt), and +`git-sync.yaml` (the sync **token + webhook secret**) — plus `accounts/mischa.yaml` +(password hash), `system.yaml`, and `post-form.md`. + +**None of those paths is under the configured `folders: pages, config, themes`.** +`user/env/` is a sibling of `user/config/`, not a subfolder of it. Yet git-sync +staged and pushed them anyway. That single fact refutes the "folders scope the +add-set" model empirically: **the `folders` setting does not scope what the +auto-commit stages.** Whatever git-sync's exact `git add` invocation, the +operational truth is that its commit sweeps the whole `user/` working tree. + +Remediation: disable sync → `.gitignore` `/env/` and `git rm --cached` it → +`reset --hard` prod to the gitignored state → regenerate the leaked JWT/CSRF +salts (delete the `*-private.php` files; Grav regenerates them) → **rotate the +Gitea token and webhook secret** (they were exposed in `git-sync.yaml`). Rotation +is what actually neutralizes the leak; the history rewrite is optional for a +private repo. + +## Guidance + +The safety question reduces to one predicate — but **not** the one the original +doc used: + +> **A file round-trips to the shared repo if and only if it is NOT gitignored.** +> The configured `folders` setting does **not** narrow this. Treat the +> round-trippable set as *everything under `user/` that git will track* — i.e. +> everything not matched by `user/.gitignore`. + +Consequences, corrected: + +1. **`.gitignore` is the only reliable boundary.** Do not rely on a file being + "outside the synced folders." If it is under `user/` and not gitignored, a + bidirectional sync can push it. Design exclusions with `.gitignore`, and + verify with `git -C user status` / `git -C user check-ignore `. + +2. **`accounts/` is NOT structurally excluded.** `user/accounts/*.yaml` (bcrypt + password hashes) is a *tracked* content folder and is not gitignored, so it + **does** round-trip — `accounts/mischa.yaml` was in the leak commit. If you + need an account file to stay server-local, it must be gitignored explicitly + (as `accounts/testrunner.yaml` already is). The earlier "password hashes never + leave the server" claim was wrong. + +3. **The per-environment tree `user/env//` MUST be gitignored** — it is + **not** inherently safe. It holds the live git-sync token, JWT secret, and + CSRF salt (Grav writes all server-side Admin config there once the env dir + exists). Because it is not under `user/config/` people assumed it was outside + the sync scope; the incident proved it is not. It is now gitignored + (`/env/` in `user/.gitignore`, commit `6e8eadb`). Keep it that way. + + > Side effect worth remembering: once `user/env//` exists, Grav's Admin + > writes **all** config changes there (system and plugin), not into + > `user/config/`. So server-side Admin edits are server-only — but "server-only" + > now depends entirely on `/env/` being gitignored, not on folder scope. When + > auditing, check **both** `user/config/...` and `user/env//config/...` + > (env wins at runtime). See `docs/working/git-sync-notes.md`. + +4. **Per-install secrets go in gitignored companion files.** Grav's convention + splits a per-install secret out of the functional YAML into a sibling that is + gitignored: the JWT secret in `api-private.php`, the CSRF/nonce + rate-limit + salt in `security-private.php`, plus `security.yaml` and `versions.yaml`. + These are safe **because they are gitignored**, not because of where they sit. + +Run every secret through predicate #1 (is it gitignored?) and the answer falls +out — but you must actually enumerate what is *not* gitignored, not what is +"outside the folders." + +### The tracked-file boomerang (a separate trap) + +Independently of the folder-scope error above, there is a second trap that the +original doc got right and that still holds: **a per-install value that a plugin +regenerates at runtime and persists into a tracked, functional config file.** + +The concrete case: the `api` plugin's *popularity* feature generates +`popularity.salt` and writes it **into `user/config/plugins/api.yaml`** — a file +that also carries must-be-shared functional config. That file is tracked, so on +prod the popularity feature regenerates the salt, git-sync stages the change, +commits, and **pushes prod's salt back to the shared repo**. Another environment +pulls it, regenerates *its own* salt, pushes again. The value **ping-pongs across +installs**, producing endless noise commits. + +The critical realization: **you cannot gitignore a single key inside a file that +also carries functional config.** `.gitignore` operates on whole files. `api.yaml` +must be tracked because the rest of it must be shared; therefore the salt inside +it is tracked too; therefore it boomerangs. + +### The rules + +For any per-install runtime-generated value, pick one of exactly three +resolutions — and do **not** reach for the fourth (stripping the line), which +cannot work under sync: + +- **Isolate the value into a gitignored companion `-private.php`** — the + pattern Grav uses for the JWT secret via `api-private.php`. +- **Disable the feature that generates it** (e.g. `popularity.enabled: false`). +- **Consciously accept the churn** when the value is genuinely low-stakes + (`popularity.salt` is an IP-hashing salt, not a credential). + +Do **not** keep stripping the value from the tracked file — bidirectional sync +brings it right back on the next save. + +### Before / after: what sticks and what doesn't + +**A standalone file gitignored + untracked sticks.** For `security-private.php` +(a file that contains *only* the secret) or the whole `user/env/` tree: + +```bash +git -C user rm --cached -r config/security-private.php # or: rm --cached -r env +printf '/config/security-private.php\n/env/\n' >> user/.gitignore +``` + +This works permanently. The path is no longer tracked, so the sync's add-set +skips it forever. The fix sticks because the secret owns its own gitignored path. + +**A key inside a tracked functional file — stripping the line does NOT stick.** +For `popularity.salt` inside `api.yaml`, deleting just the `salt:` line and +committing looks clean locally, but Grav re-appends it at runtime and the next +sync re-commits and re-pushes it. The only durable fixes are the +companion-private-file pattern or disabling the feature. + +## Why This Matters + +Two quiet, cross-environmental failure modes: + +1. **The folder-scope illusion.** Assuming "only `pages/config/themes` sync" is a + security control leads you to leave secrets in `env/` or `accounts/` unignored + — and a single Admin save on prod pushes them to a shared repo. This actually + happened here. The only defensible mental model is *gitignore is the boundary*; + enumerate the un-ignored set, not the "un-foldered" set. + +2. **The boomerang.** A "cleanup" commit that strips a secret from a *tracked* + file looks done locally but silently reappears upstream on the next content + save, because the plugin regenerates it and the sync re-commits it. + +Getting both right is what lets you answer "is bidirectional sync safe?" honestly. +The answer is **yes, once `user/.gitignore` actually excludes every sensitive +path** — `env/`, the per-install `*-private.php` files, `security.yaml`, +`versions.yaml`, and any account file that must stay server-local — and once every +runtime-regenerated value either lives in its own gitignored file or is a +consciously-accepted low-stakes churn. It is emphatically **not** safe on the +strength of folder scoping alone. + +## When to Apply + +- **Enabling or auditing bidirectional git-sync** on a server that authors + content. Enumerate the round-trippable set as *everything under `user/` not + matched by `.gitignore`* — then confirm no secret is in it. +- **Deciding where a new secret or per-install generated value should live.** + Standalone gitignored file for anything sensitive; never a key inside a shared + functional YAML; never "outside the folders" as the sole justification. +- **Reviewing a "stop tracking this secret" cleanup** for whether it will stick: + is the secret in its own gitignored path (holds) or a line inside a tracked + functional file that something regenerates (boomerangs)? + +## Examples + +**The leak (what "folder scope is safe" cost).** With `folders: pages, config, +themes` and `/env/` **not** gitignored, prod's auto-commit `9337003` pushed +`user/env/intotheeast.com/config/**` (JWT, CSRF salt, git-sync token + webhook +secret), `accounts/mischa.yaml`, `system.yaml`, and `post-form.md` to Gitea — +all outside the configured folders. Fix: gitignore + untrack `/env/`, regenerate +the JWT/CSRF salts, **rotate the token and webhook secret**. + +**Safe after remediation.** Same bidirectional config, but now `user/.gitignore` +excludes `/env/`, `config/plugins/git-sync.yaml`, `config/plugins/api-private.php`, +`config/security.yaml`, `config/security-private.php`, `config/versions.yaml`. +Running each secret through *is-it-gitignored*: all sensitive paths are excluded → +none is in the round-trippable set. Verified: prod's `git status` shows only the +intended tracked content, and no boomerang/secret commit lands on the remote. + +**Boomerang example.** `popularity.salt` in the tracked `api.yaml` regenerates +per-install and re-commits under sync. The fix that *sticks* is the +companion-private-file pattern or `popularity.enabled: false` — **not** stripping +the `salt:` line. + +## Related + +- `docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md` — the + origin of the "functional plugin config in tracked `user/config/plugins/`, + secrets/per-install values in gitignored `*-private.php`" rule. That doc covers + *where config must live to deploy*; this doc covers *why gitignore — not folder + scope — is the sync boundary, and why a runtime-written tracked value boomerangs*. +- `docs/working/git-sync-notes.md` — operational notes on git-sync's synced + folders and the per-environment tree. Corrected in the same 2026-07-05 pass to + drop the "env/ is outside the sync scope so it's safe" claim. +- `docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md` + — adjacent context from the same Grav production cutover, different failure mode. diff --git a/docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md b/docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md index 36df1f7..731d0b0 100644 --- a/docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md +++ b/docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md @@ -82,4 +82,5 @@ The failure is silent and per-environment: it works on the developer's machine ( - `docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md` — the config gap documented here was the *other* latent problem surfaced in that same investigation: the `api` plugin also had to be *installed* first before any config could take effect. The install gap (GPM version floor) and this config-tracking gap compounded each other on the fresh prod environment. - `docs/working/git-sync-notes.md` — the related third config location: on prod, Grav Admin saves config into the per-environment tree `user/env//config/`, which is *also* untracked. Same "config that doesn't reach the repo" family. +- `docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md` — the sync-boomerang consequence of this rule: a per-install value that a plugin regenerates into a *tracked* functional config file (e.g. `popularity.salt` in `api.yaml`) re-commits itself and ping-pongs across environments under bidirectional git-sync. The `*-private.php` companion pattern this doc establishes is exactly the durable fix. - **CLAUDE.md §0 (plugin-management model):** only `pages/`, `config/`, `accounts/`, `themes/` are tracked in the `user/` repo; `plugins/` and `data/` are gitignored and GPM-managed. That tracking boundary is exactly why functional config must live under `config/plugins/`, not in the plugin's own folder. diff --git a/docs/solutions/integration-issues/grav-plugin-config-without-code-wont-enable.md b/docs/solutions/integration-issues/grav-plugin-config-without-code-wont-enable.md new file mode 100644 index 0000000..37bca85 --- /dev/null +++ b/docs/solutions/integration-issues/grav-plugin-config-without-code-wont-enable.md @@ -0,0 +1,100 @@ +--- +title: "Grav plugin won't enable because its code is missing while its config persists in the env tree" +date: 2026-07-05 +category: integration-issues +module: git-sync +problem_type: integration_issue +component: tooling +severity: high +symptoms: + - "git-sync plugin will not enable on prod despite enabled: true in its config" + - "Plugin does not appear / cannot be toggled on in the Grav Admin UI" + - "Config-level fix attempts (editing plugin YAML) have no effect" + - "make remote-gpm-install-prod PKG=git-sync reports a FRESH install, not 'already installed'" +root_cause: incomplete_setup +resolution_type: dependency_update +related_components: + - documentation + - development_workflow +tags: + - grav + - git-sync + - gpm + - plugin-management + - env-config + - config-without-code + - troubleshooting-order + - remote-only-plugin +--- + +# Grav plugin won't enable because its code is missing while its config persists in the env tree + +## Problem + +After remediating an unrelated git-sync secret leak on prod (intotheeast.com, Grav 2.0.7 on a DirectAdmin/Apache shared host), the operator went to re-enable the `git-sync` plugin. Setting `enabled: true` in its config had no effect: the plugin would not appear as enabled in the Grav Admin plugins UI, and toggling it on manually in the browser did not take either. It looked fully "configured" — the config file was right there — but the plugin was inert and sync never ran. + +The trap is that the plugin's config file existed (in the per-host env tree at `user/env/intotheeast.com/config/plugins/git-sync.yaml`), so any inspection that reads only config concluded the plugin was present and just needed enabling. The actual problem was one layer down: the plugin's **code** was missing from `user/plugins/git-sync/`. A Grav plugin cannot load or enable without its code on disk, no matter what its config says. + +## Symptoms + +- The git-sync plugin does not appear as enabled, and cannot be enabled, in the Grav Admin UI — despite `enabled: true` being present in its config. +- Toggling `enabled` in config, or flipping the toggle in the Admin UI, produces no working plugin. Sync does not run. +- The plugin's config file DOES exist (in the per-environment tree `user/env/intotheeast.com/config/plugins/git-sync.yaml`), so the plugin appears "present" whenever only the config is inspected — masking the real state. + +## What Didn't Work + +1. **Setting / ensuring `enabled: true` in the git-sync config.** No effect. Config was never the problem. +2. **Enabling the plugin manually in the Admin UI.** The toggle wouldn't take. + +Both failed attempts operate on the **config** layer. But the plugin's **code** was absent from `user/plugins/git-sync/`, and Grav can't load a plugin without its code. Grav (and any tooling that reads the config tree) reports a plugin as "configured" purely from the presence of its config file, which masks the absence of code. Diagnosing and poking at the config layer could never fix a missing-code problem — and guessing at config changes before running a simple `ls` on the plugin directory cost real time here. + +## Solution + +First, run the decisive diagnostic on the server — confirm whether the plugin code actually exists *before* touching config: + +``` +ls -la $WEBROOT/user/plugins/git-sync/ # empty/absent => missing code, reinstall +``` + +(In this project, do that via a make target or an ssh one-liner the user runs — never raw SSH by the assistant. All server ops go through `make remote-*`.) + +With the directory confirmed empty/absent, reinstall the plugin's code via GPM: + +``` +make remote-gpm-install-prod PKG=git-sync # GPM fresh-installs Git Sync v3.4.4 +``` + +That make target runs, on the server: + +``` +php bin/gpm index -f && php bin/gpm install git-sync -y && php bin/grav clearcache +``` + +The install output read **"Preparing to install Git Sync [v3.4.4] ... Success!"** — a **fresh** install, not "already installed." That fresh-install line is exactly what confirmed the code had been absent all along. After the reinstall plus cache clear, the plugin enabled and sync worked. + +## Why This Works + +Grav resolves a plugin from **two independent locations**: + +- **Code** at `user/plugins//` — installed by GPM. Note `user/plugins/` is gitignored (`/plugins/*`) and is NOT tracked by the content repo. +- **Config** — the tracked `user/config/plugins/.yaml` and/or the per-host `user/env//config/plugins/.yaml`. + +These two can **desync**: config can exist with no code behind it. Config alone makes the plugin look present to any tool that only reads config, but the plugin stays inert until its code is on disk. GPM reinstall restores the code; `clearcache` makes Grav re-scan and pick it up. + +A project-specific amplifier made this worse: `git-sync` is a **remote-only, GPM-managed** plugin. It is deliberately NOT in `plugins.txt`, so `make install-plugins` and the normal `make remote-install` flow do **not** restore it. Only an explicit `php bin/gpm install git-sync` (via `make remote-gpm-install-prod PKG=git-sync`) does. So when its code goes missing, it does not self-heal through the standard install path — you must reinstall it explicitly. + +How the code went missing here is **unconfirmed**. It happened around the git-sync secret-leak remediation, but the exact step that wiped `user/plugins/git-sync/` was not established — don't assume a specific cause. + +## Prevention + +- **Check the code layer before the config layer.** When a Grav plugin "won't enable" and config toggles do nothing, FIRST verify the code exists: `ls user/plugins//` on the server. Config-without-code is the failure class; the empty directory is the tell. +- **Enumerate both layers in all locations when diagnosing.** Plugins have a code layer (`user/plugins//`) and a config layer, and on prod the config can live in the env tree (`user/env//config/plugins/.yaml`) and persist completely independently of the code. Remember: once `user/env//` exists, Grav Admin writes ALL config there, so always check both `user/config/...` and the env path (env wins). +- **Know which plugins are remote-only.** The 3-category model: GPM-via-`plugins.txt` (admin2 / api / flex-objects), custom-in-repo (cache-on-save / story-blocks), and remote-only (git-sync — never in `plugins.txt`). Remote-only plugins are NOT restored by the standard install/content flows, so reinstall them explicitly via GPM after any operation that could have wiped `user/plugins/`. +- **Diagnose actual state before proposing config fixes.** An `ls` is cheaper than a guess. Establishing that the code was missing would have pointed straight at the reinstall instead of a round of config poking. + +## Related + +- `docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md` — **closest sibling.** Same family (a plugin non-functional on prod, resolved by a GPM install + cache clear), same 2026-07-04/05 cutover context, same `plugins.txt` / `make remote-*` / GPM machinery. **Distinct trigger:** there, GPM refuses to *offer* the plugin because the installed core is below the version floor; here, the plugin's *code folder is simply missing* while its config persists (config-without-code desync). Two different ways a plugin ends up absent/inert on prod. +- `docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md` — same module (git-sync) and explains **why the config survived without code**: Grav Admin writes `git-sync.yaml` into the per-environment tree `user/env//config/plugins/`, which is untracked/gitignored and not part of the plugin package. The orphaned config here is the flip side of that env-tree behavior. +- `docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md` — establishes the plugin **code (GPM/gitignored) vs config (tracked override / env tree)** split that this bug exploits. This doc is a concrete failure of that split going the other way: config present (in the env tree), code absent. +- `docs/working/git-sync-notes.md` — operational notes on git-sync's per-environment tree, where `git-sync.yaml` lives server-only. Context for where the orphaned config resided. diff --git a/docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md b/docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md index 231bd18..abd6989 100644 --- a/docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md +++ b/docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md @@ -81,7 +81,7 @@ GPM (Grav Package Manager) only offers a plugin version whose declared Grav requ - **Keep `.env.` `GRAV_VERSION` current.** It is the version a *fresh* `make remote-install-` bakes in; a stale value silently installs an old core. After any core upgrade, bump the env file too — self-upgrade only moves the running server. Note this is a *third* version-authority surface alongside `user/config/system.yaml` `gpm.releases` (channel) and `plugins.txt` — they must stay in sync. - **When GPM says "package not found" for a package you know is on your channel, check the target's Grav core version first** (`php bin/grav --version` on the server, or `make remote-diag-`). GPM filters by core compatibility; "not found" often means "no version compatible with your core," not "missing from the index." `gpm index -f` will not help. -- **Don't trust a top-level install "Success" to mean dependencies installed.** A fresh install can leave a plugin's declared dependency unsatisfied (here `admin2` installed but its `api` dependency didn't). Verify with `ls user/plugins/`. +- **Don't trust a top-level install "Success" to mean dependencies installed.** A fresh install can leave a plugin's declared dependency unsatisfied (here `admin2` installed but its `api` dependency didn't). Verify with `ls user/plugins/`. The same `ls` guards a *second*, distinct way a plugin ends up non-functional: its **code folder can be missing while its config persists** (e.g. in the per-host env tree), so it looks configured but never loads. Checking `ls user/plugins/` catches both the missing-dependency and the config-without-code cases — see `grav-plugin-config-without-code-wont-enable.md`. - **Know the Admin2 ⇄ api coupling.** Admin2 authenticates via the api plugin's `/api/v1` endpoints; a missing or unrouted api plugin makes admin login fail *silently* (login POST 404s, nothing logged). A quick `curl /api/v1/pages` expecting `401` (not `404`) is a good post-deploy smoke check. ## Related @@ -92,3 +92,7 @@ This is one of three independent gotchas from the same **2026-07-04 Grav 2.0.4 p - `docs/solutions/test-failures/new-user-grants-api-not-admin-on-admin2.md` — sibling: an authenticated account is denied an admin-gated page because `login new-user` auto-detect granted `api.*` but not `admin.*`. Different root cause (permission provisioning), same admin2/api area. - `docs/working/plans/2026-07-04-grav-2.0.4-upgrade.md` — the upgrade plan whose Global Constraints spell out the GPM version floors (`grav >=2.0.4`, `api >=1.0.6`) that cause the "package not found" on an rc core. - `docs/solutions/conventions/grav-plugin-config-must-be-tracked-override.md` — the *other* latent problem from this same investigation: the api plugin's functional config (`enabled` / `route` / `session_enabled`) must live in the tracked `user/config/plugins/api.yaml` to deploy at all. Necessary but not sufficient here (the plugin must be installed first), but a durable convention in its own right. + +A closely-related **sibling in the "plugin absent/non-functional on prod" family** (from the 2026-07-05 follow-up, not one of the three cutover gotchas above): + +- `docs/solutions/integration-issues/grav-plugin-config-without-code-wont-enable.md` — same outcome (a plugin inert on prod, fixed by a GPM install + cache clear), **different trigger**: there, GPM won't *offer* the plugin because the core is below the version floor; there, the plugin's *code folder is simply missing* while its config persists in the env tree (config-without-code desync). Same `ls user/plugins/` smoke check flushes both out. diff --git a/docs/working/git-sync-notes.md b/docs/working/git-sync-notes.md index b1c5045..473a427 100644 --- a/docs/working/git-sync-notes.md +++ b/docs/working/git-sync-notes.md @@ -16,12 +16,17 @@ user/config/plugins/git-sync.yaml ← NOT here Why this matters: -- **Server-only, not synced, not committed.** `user/env/` is outside the - content repo's tracked folders (`pages`/`config`/`accounts`/`themes`) and is - not one of git-sync's synced folders (`pages`/`config`/`themes`). So config - saved via Admin *on the server* never reaches Gitea or local. This is ideal - for the git-sync token (it stays server-only) but means **prod Admin config - edits silently diverge** — author durable config in the repo, not prod Admin. +- **⚠️ `user/env/` is NOT safe unless gitignored — it is NOT scoped out by the + `folders` setting.** An earlier version of this note claimed `user/env/` + "never reaches Gitea" because it is outside git-sync's synced folders. **That + is wrong and caused a live secret leak (2026-07-05).** git-sync's auto-commit + stages files *outside* the configured `folders`; on prod it pushed the whole + `user/env/intotheeast.com/config/` tree — JWT secret, CSRF salt, **and the + git-sync token + webhook secret** — to Gitea. The fix was to **gitignore + `/env/`** (commit `6e8eadb`). So: prod Admin config edits stay server-only + *only because `/env/` is now gitignored*, not because of folder scope. Author + durable config in the repo, not prod Admin. Full analysis: + `docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md`. - **Look in both places.** When inspecting/toggling server config, check `user/config/plugins/.yaml` **and** `user/env//config/plugins/.yaml` (env wins). @@ -48,19 +53,25 @@ If you see the comma-string form on an older version, fix it by editing `git-sync.yaml` directly (at whichever path it lives — see above); do not re-save folders via the Admin UI on the buggy version. -## Per-install / secret files — must be gitignored (never synced) +## Per-install / secret files — must be gitignored (gitignore is the boundary) -git-sync syncs the `config/` folder, so any per-install or secret file tracked -there would get pushed to Gitea and pollute every environment. Keep these out -of the content repo (all in `user/.gitignore`): +git-sync's auto-commit stages **everything under `user/` that is not +gitignored** — the `folders` setting does *not* scope the commit add-set (a +2026-07-05 leak proved this by pushing `user/env/**`, outside the configured +folders). So `.gitignore` — not folder scope — is the only thing keeping a +per-install or secret file off Gitea. Keep all of these gitignored in +`user/.gitignore`: -| File | Why | +| Path | Why | |---|---| +| `env/` | **whole per-host env tree** — holds the live git-sync token, JWT secret, CSRF salt + all server-side Admin config. Gitignored + untracked 2026-07-05 (commit `6e8eadb`) after it leaked to Gitea. NOT safe on folder scope alone. | | `config/plugins/git-sync.yaml` | encrypted token; server-specific (also lives at env path on prod) | | `config/plugins/api-private.php` | API JWT secret | | `config/security.yaml` | Grav nonces/salts (legacy location) | | `config/versions.yaml` | per-install Grav schema-migration state — differs per env (dev 2.0.4, prod 2.0.7); Grav regenerates it. Untracked 2026-07-04. | -| `config/security-private.php` | **TODO:** committed salt secret; should be gitignored like `api-private.php` (deferred — untracking resets server sessions) | +| `config/security-private.php` | CSRF/nonce + admin rate-limit signing salt; gitignored + untracked 2026-07-05 (commit 2840018). Each env keeps its own; untracking regenerates prod's salt (one-time admin re-login). | + +> **Why a key inside a *tracked* config file (e.g. `popularity.salt` in `api.yaml`) can't just be stripped** — it regenerates at runtime and boomerangs back via git-sync's `git add -A`. See `docs/solutions/architecture-patterns/git-sync-secret-exposure-and-tracked-file-boomerang.md` for the full round-trippable-set model. ## git-sync config summary (prod, 2026-07-04)