From 3d9d3ecb85af710bae2798fc7cf25f0e7531ab70 Mon Sep 17 00:00:00 2001 From: Mischa Date: Wed, 8 Jul 2026 12:15:01 +0200 Subject: [PATCH] docs(solutions): capture docker exec root-owned bind-mount fix New learning: docker exec defaults to root, so make targets writing into the ./user bind mount (esp. install-plugins -> gpm) created root-owned files (11,624 accumulated), breaking worktree-rm. Fix: HOST_UID/HOST_GID + `-u` on file-writing execs while the grav container still boots as root. Cross-linked reciprocally with the sibling docker-dev-env upgrade doc. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RDS6t8wcpbwKvvrxykVQ5K --- ...docker-exec-root-owned-bind-mount-files.md | 124 ++++++++++++++++++ ...de-local-grav-core-rebuild-docker-image.md | 1 + 2 files changed, 125 insertions(+) create mode 100644 docs/solutions/integration-issues/docker-exec-root-owned-bind-mount-files.md diff --git a/docs/solutions/integration-issues/docker-exec-root-owned-bind-mount-files.md b/docs/solutions/integration-issues/docker-exec-root-owned-bind-mount-files.md new file mode 100644 index 0000000..ea9ae4a --- /dev/null +++ b/docs/solutions/integration-issues/docker-exec-root-owned-bind-mount-files.md @@ -0,0 +1,124 @@ +--- +title: docker exec defaults to root, writing root-owned files into the host bind mount +date: 2026-07-08 +problem_type: integration_issue +category: integration-issues +module: docker-dev-environment +component: development_workflow +severity: high +symptoms: + - "11,624 root-owned (uid 0) files accumulated under the host ./user bind mount" + - "make worktree-rm fails: cannot rm root-owned plugin files without sudo" + - "files stay root-owned even though UID/GID env vars were set to the host user" + - "install-plugins writes the entire plugin tree as root via php bin/gpm install" +root_cause: config_error +resolution_type: config_change +related_components: + - tooling + - docker-compose + - grav-cms +tags: + - docker + - docker-exec + - bind-mount + - file-permissions + - uid-gid + - makefile + - grav + - gpm +--- + +## Problem + +In the Grav CMS travel-blog Docker dev environment, `make` targets that shelled into the `grav` container were silently creating **root-owned (uid 0)** files inside the host `./user` bind mount. The `grav` service (based on `getgrav/grav`) bind-mounts host `./user` → `/var/www/html/user`, so anything the container writes there lands on the host filesystem with whatever ownership the writing process had. + +Over time this accumulated **11,624** root-owned files under `./user`. The immediate breakage: `make worktree-rm` could no longer delete a worktree's plugin tree, because a non-root host user cannot remove root-owned files without `sudo`. The working tree became unmanageable, and the per-worktree isolated-container workflow (which is what surfaced the accumulation) left root-owned debris behind on every teardown. + +The root of the surprise: the developer had already set `UID`/`GID` env vars to their own user and reasonably assumed that covered container file ownership. It did not — those vars never reached the `grav` service. + +## Symptoms + +- `ls -la user/plugins/...` shows files owned by `root root` instead of the host user. +- `make worktree-rm` (and a plain `rm -rf` on a worktree) fails with `Permission denied` on plugin files. +- Thousands of root-owned files pile up under `./user` — `find ./user -uid 0` counted **11,624**. +- Confusing because `UID`/`GID` were already set to the developer's own user, yet ownership was still root. + +## What Didn't Work + +Several plausible fixes were tried or considered and rejected: + +- **Setting `UID`/`GID` env vars.** These only reached the `travel-memories` service, which consumes them via its compose `user: "${UID}:${GID}"` directive. The `grav` service has no such directive, so it never consumed them. +- **`APACHE_RUN_USER=#1000` / `APACHE_RUN_GROUP=#1000` on the grav service.** These only affect the Apache **worker** processes. They do nothing for `docker exec` CLI invocations or for the entrypoint — which are what the make targets actually run. +- **Adding `user: "${UID}:${GID}"` to the grav service in compose.** Not viable. The `getgrav/grav` base-image entrypoint must boot as root to bind port `:80` and set up cron. Pinning the whole container to a non-root user breaks boot. +- **Hardening `worktree-rm` to delete root files via a throwaway root container.** Rejected by the user: no make command should require or use root privileges. The correct fix is to stop *creating* root-owned files, not to add a privileged cleanup step. + +**The symptom was noticed for weeks before it was diagnosed.** (session history) During the earlier Grav 2.0.4/2.0.7 upgrade work, container-written files repeatedly surfaced as root-owned — the API plugin's generated `config/plugins/api-private.php` was flagged as "owned by the container, permission-denied to me", and worktree teardown already required `git worktree remove --force` to get past files it couldn't cleanly remove. Each instance was treated as a one-off annoyance rather than traced to `docker exec` defaulting to uid 0. Consolidating plugin management onto `make install-plugins` / `gpm install` during that upgrade actually *enlarged* the problem surface, because it increased how often the container writes into the host mount as root. + +## Root Cause + +`docker exec` defaults to running as root (uid 0). Because the grav container must boot as root, and `docker exec` inherits that default unless `-u` is passed explicitly, every make target that did `docker exec ` without `-u` wrote root-owned files into the `./user` bind mount. + +The worst offender was `install-plugins`, which runs `php bin/gpm install` and writes the entire plugin tree into `./user/plugins`. + +## Solution + +Derive the host identity once in the Makefile and drop privileges on the specific exec that writes to the bind mount (commit `209b804`). + +Add host-user vars: + +```makefile +HOST_UID := $(shell id -u) +HOST_GID := $(shell id -g) +``` + +Rewrite `install-plugins`. + +**Before** (wrote root-owned plugins): + +```makefile +install-plugins: + docker exec -w /var/www/html $(GRAV_CONTAINER) php bin/gpm install $(shell cat plugins.txt | tr '\n' ' ') -y + $(MAKE) apply-plugin-patches +``` + +**After** (plugins owned by host user): + +```makefile +install-plugins: + # cache/ and tmp/ are root-owned in the image, so make them writable first + # (container-internal chown — never touches the host) so gpm can run AS YOU. + docker exec $(GRAV_CONTAINER) chown -R $(HOST_UID):$(HOST_GID) /var/www/html/cache /var/www/html/tmp + # gpm runs as the host user, so the plugins it writes into ./user/plugins are + # owned by you, not root — no post-hoc chown, no root files to clean up later. + docker exec -u $(HOST_UID):$(HOST_GID) -w /var/www/html $(GRAV_CONTAINER) php bin/gpm install $(shell cat plugins.txt | tr '\n' ' ') -y + $(MAKE) apply-plugin-patches +``` + +## Why This Works + +The container still *boots* as root — which it needs, to bind `:80` and set up cron. But the individual `docker exec` that writes into the bind mount now runs as the host uid/gid via `-u $(HOST_UID):$(HOST_GID)`. Files that exec creates on the host are therefore owned by the developer, not root. No post-hoc chown, no cleanup debt. + +The preliminary chown of `cache/` and `tmp/` is container-internal: those paths are root-owned in the base image and are not host-managed content in the same way. gpm needs them writable to run as a non-root user; without making them writable first, gpm exits 1. Chowning them inside the container never touches the host filesystem. + +**Empirical validation.** A minimal touch/stat test isolates the mechanism: `docker exec -u 1000:1000 touch /var/www/html/user/probe` produces a host file owned by `1000`, while the same command without `-u` produces one owned by `0`. After applying the fix, `make fix-perms` cleared the backlog (11,624 → 0) and a real `make install-plugins` ran clean: gpm exit 0, zero root-owned files created, `api`/`admin2` plugins owned by the host user, and the site healthy (`/` and `/admin` → 200). + +## Prevention + +The reusable principle, worth internalizing beyond this one repo: + +- **Any make/CI target that writes files into a host bind mount via `docker exec` must pass `-u $(HOST_UID):$(HOST_GID)`.** A container booting as root does *not* mean your exec commands must run as root. Drop privileges per-exec. +- **Derive host identity once in the Makefile and reuse it:** `HOST_UID := $(shell id -u)` / `HOST_GID := $(shell id -g)`. +- **Don't rely on `APACHE_RUN_USER` or compose-level `UID`/`GID` env vars to fix exec ownership** — they don't apply to `docker exec`. `APACHE_RUN_USER` only affects Apache workers; compose `user:`/env vars only affect services wired to consume them. +- **You can't just add `user:` to a service whose entrypoint needs root** (to bind privileged ports, set up cron, etc.). Drop privileges per-exec instead of per-container. +- **If a tool run as non-root needs writable scratch dirs that are root-owned in the image, chown them container-internally first.** That doesn't touch the host. +- **Root-owned files accumulate invisibly.** (session history) Plugin code under `user/plugins//` is gitignored by project convention (only `cache-on-save` and `story-blocks` are tracked), so root-owned files pile up in the bind mount without ever appearing in `git status` — they only bite at worktree-removal time. Don't wait for `git status` to reveal them; `find ./user -uid 0 | wc -l` is the real detector. +- **Keep a `make fix-perms` escape hatch** (`find ./user -uid 0 ... chown`) for residual root files — notably first-boot files the base-image entrypoint writes as root (`config/security.yaml`, `data/api-keys.yaml`), which no `-u` on a make target can reach. After this fix it's a rare mop-up, not a routine step. +- **Verification recipe:** `docker exec -u 1000:1000 touch /mnt/f && stat -c '%u' host/f` should print your uid, not `0`. + +This lives in the Makefile because make targets are the only sanctioned container interface in this project — the fix belongs there, not in ad-hoc docker commands. + +## Related + +- [`tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md`](../tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md) — the sibling docker-dev-env doc. It documents `make install-plugins` → `docker exec … php bin/gpm install` as a routine local step but never addresses *who* those execs run as. This doc is its complement: it explains why the exec must drop to the host user. +- [`architecture-patterns/dual-repo-submodule-workflow.md`](../architecture-patterns/dual-repo-submodule-workflow.md) — worktrees + the `./user` submodule/bind mount, including the persistent `M user` dirty-state warning. Root-owned files landing in `./user` from root-default execs are a concrete cause of unexpected permission/dirty state in worktree dev servers. +- `docs/guides/deploy-cycle.md` — the three-layer state model (plugin code / repo config / host env tree); the host env tree is the layer across which these root-owned files land. diff --git a/docs/solutions/tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md b/docs/solutions/tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md index 7f16b64..397da87 100644 --- a/docs/solutions/tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md +++ b/docs/solutions/tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md @@ -113,3 +113,4 @@ $ docker exec intotheeast_grav sh -c \ - `docs/guides/local-setup.md` — "Upgrading to a newer Grav" section documents the same bump-and-rebuild procedure, but with stale "RC bundle" wording and without the `docker rm -f`, version-verify, plugin-refresh, or non-durability details. **Refresh candidate** — fold these operational steps in and drop the "RC" language (`Dockerfile` now pins stable `grav-admin-v2.0.7.zip`). - `docs/solutions/integration-issues/stale-grav-version-blocks-api-plugin-install.md` — the server/`.env`/fresh-install counterpart. That doc frames `GRAV_VERSION` as a version-authority surface for *remote* installs; this doc adds the **fourth** authority surface (the hardcoded release-zip URL in `Dockerfile`) and clarifies that `.env* GRAV_VERSION` never touches the local Docker core. Servers correctly self-upgrade because they have no image; local Docker cannot. - `docs/guides/deploy-cycle.md` — the local→test→prod runbook. Its Phase 0 (Local) covers bumping `GRAV_VERSION` for remote installs but not upgrading the local Docker core; its "where state lives" table omits that the local core lives in the Docker image. +- `docs/solutions/integration-issues/docker-exec-root-owned-bind-mount-files.md` — the ownership counterpart to the `make install-plugins` step above (line 54). That `docker exec … php bin/gpm install` writes the plugin tree into the `./user` bind mount **as root** unless `-u $(HOST_UID):$(HOST_GID)` is passed; that doc explains the fix and why the container still boots as root.