New learning: docker exec defaults to root, so make targets writing into the ./user bind mount (esp. install-plugins -> gpm) created root-owned files (11,624 accumulated), breaking worktree-rm. Fix: HOST_UID/HOST_GID + `-u` on file-writing execs while the grav container still boots as root. Cross-linked reciprocally with the sibling docker-dev-env upgrade doc. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RDS6t8wcpbwKvvrxykVQ5K
9.7 KiB
title, date, problem_type, category, module, component, severity, symptoms, root_cause, resolution_type, related_components, tags
| title | date | problem_type | category | module | component | severity | symptoms | root_cause | resolution_type | related_components | tags | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| docker exec defaults to root, writing root-owned files into the host bind mount | 2026-07-08 | integration_issue | integration-issues | docker-dev-environment | development_workflow | high |
|
config_error | config_change |
|
|
Problem
In the Grav CMS travel-blog Docker dev environment, make targets that shelled into the grav container were silently creating root-owned (uid 0) files inside the host ./user bind mount. The grav service (based on getgrav/grav) bind-mounts host ./user → /var/www/html/user, so anything the container writes there lands on the host filesystem with whatever ownership the writing process had.
Over time this accumulated 11,624 root-owned files under ./user. The immediate breakage: make worktree-rm could no longer delete a worktree's plugin tree, because a non-root host user cannot remove root-owned files without sudo. The working tree became unmanageable, and the per-worktree isolated-container workflow (which is what surfaced the accumulation) left root-owned debris behind on every teardown.
The root of the surprise: the developer had already set UID/GID env vars to their own user and reasonably assumed that covered container file ownership. It did not — those vars never reached the grav service.
Symptoms
ls -la user/plugins/...shows files owned byroot rootinstead of the host user.make worktree-rm(and a plainrm -rfon a worktree) fails withPermission deniedon plugin files.- Thousands of root-owned files pile up under
./user—find ./user -uid 0counted 11,624. - Confusing because
UID/GIDwere already set to the developer's own user, yet ownership was still root.
What Didn't Work
Several plausible fixes were tried or considered and rejected:
- Setting
UID/GIDenv vars. These only reached thetravel-memoriesservice, which consumes them via its composeuser: "${UID}:${GID}"directive. Thegravservice has no such directive, so it never consumed them. APACHE_RUN_USER=#1000/APACHE_RUN_GROUP=#1000on the grav service. These only affect the Apache worker processes. They do nothing fordocker execCLI invocations or for the entrypoint — which are what the make targets actually run.- Adding
user: "${UID}:${GID}"to the grav service in compose. Not viable. Thegetgrav/gravbase-image entrypoint must boot as root to bind port:80and set up cron. Pinning the whole container to a non-root user breaks boot. - Hardening
worktree-rmto delete root files via a throwaway root container. Rejected by the user: no make command should require or use root privileges. The correct fix is to stop creating root-owned files, not to add a privileged cleanup step.
The symptom was noticed for weeks before it was diagnosed. (session history) During the earlier Grav 2.0.4/2.0.7 upgrade work, container-written files repeatedly surfaced as root-owned — the API plugin's generated config/plugins/api-private.php was flagged as "owned by the container, permission-denied to me", and worktree teardown already required git worktree remove --force to get past files it couldn't cleanly remove. Each instance was treated as a one-off annoyance rather than traced to docker exec defaulting to uid 0. Consolidating plugin management onto make install-plugins / gpm install during that upgrade actually enlarged the problem surface, because it increased how often the container writes into the host mount as root.
Root Cause
docker exec defaults to running as root (uid 0). Because the grav container must boot as root, and docker exec inherits that default unless -u is passed explicitly, every make target that did docker exec <container> <cmd> without -u wrote root-owned files into the ./user bind mount.
The worst offender was install-plugins, which runs php bin/gpm install and writes the entire plugin tree into ./user/plugins.
Solution
Derive the host identity once in the Makefile and drop privileges on the specific exec that writes to the bind mount (commit 209b804).
Add host-user vars:
HOST_UID := $(shell id -u)
HOST_GID := $(shell id -g)
Rewrite install-plugins.
Before (wrote root-owned plugins):
install-plugins:
docker exec -w /var/www/html $(GRAV_CONTAINER) php bin/gpm install $(shell cat plugins.txt | tr '\n' ' ') -y
$(MAKE) apply-plugin-patches
After (plugins owned by host user):
install-plugins:
# cache/ and tmp/ are root-owned in the image, so make them writable first
# (container-internal chown — never touches the host) so gpm can run AS YOU.
docker exec $(GRAV_CONTAINER) chown -R $(HOST_UID):$(HOST_GID) /var/www/html/cache /var/www/html/tmp
# gpm runs as the host user, so the plugins it writes into ./user/plugins are
# owned by you, not root — no post-hoc chown, no root files to clean up later.
docker exec -u $(HOST_UID):$(HOST_GID) -w /var/www/html $(GRAV_CONTAINER) php bin/gpm install $(shell cat plugins.txt | tr '\n' ' ') -y
$(MAKE) apply-plugin-patches
Why This Works
The container still boots as root — which it needs, to bind :80 and set up cron. But the individual docker exec that writes into the bind mount now runs as the host uid/gid via -u $(HOST_UID):$(HOST_GID). Files that exec creates on the host are therefore owned by the developer, not root. No post-hoc chown, no cleanup debt.
The preliminary chown of cache/ and tmp/ is container-internal: those paths are root-owned in the base image and are not host-managed content in the same way. gpm needs them writable to run as a non-root user; without making them writable first, gpm exits 1. Chowning them inside the container never touches the host filesystem.
Empirical validation. A minimal touch/stat test isolates the mechanism: docker exec -u 1000:1000 <container> touch /var/www/html/user/probe produces a host file owned by 1000, while the same command without -u produces one owned by 0. After applying the fix, make fix-perms cleared the backlog (11,624 → 0) and a real make install-plugins ran clean: gpm exit 0, zero root-owned files created, api/admin2 plugins owned by the host user, and the site healthy (/ and /admin → 200).
Prevention
The reusable principle, worth internalizing beyond this one repo:
- Any make/CI target that writes files into a host bind mount via
docker execmust pass-u $(HOST_UID):$(HOST_GID). A container booting as root does not mean your exec commands must run as root. Drop privileges per-exec. - Derive host identity once in the Makefile and reuse it:
HOST_UID := $(shell id -u)/HOST_GID := $(shell id -g). - Don't rely on
APACHE_RUN_USERor compose-levelUID/GIDenv vars to fix exec ownership — they don't apply todocker exec.APACHE_RUN_USERonly affects Apache workers; composeuser:/env vars only affect services wired to consume them. - You can't just add
user:to a service whose entrypoint needs root (to bind privileged ports, set up cron, etc.). Drop privileges per-exec instead of per-container. - If a tool run as non-root needs writable scratch dirs that are root-owned in the image, chown them container-internally first. That doesn't touch the host.
- Root-owned files accumulate invisibly. (session history) Plugin code under
user/plugins/<name>/is gitignored by project convention (onlycache-on-saveandstory-blocksare tracked), so root-owned files pile up in the bind mount without ever appearing ingit status— they only bite at worktree-removal time. Don't wait forgit statusto reveal them;find ./user -uid 0 | wc -lis the real detector. - Keep a
make fix-permsescape hatch (find ./user -uid 0 ... chown) for residual root files — notably first-boot files the base-image entrypoint writes as root (config/security.yaml,data/api-keys.yaml), which no-uon a make target can reach. After this fix it's a rare mop-up, not a routine step. - Verification recipe:
docker exec -u 1000:1000 <container> touch /mnt/f && stat -c '%u' host/fshould print your uid, not0.
This lives in the Makefile because make targets are the only sanctioned container interface in this project — the fix belongs there, not in ad-hoc docker commands.
Related
tooling-decisions/upgrade-local-grav-core-rebuild-docker-image.md— the sibling docker-dev-env doc. It documentsmake install-plugins→docker exec … php bin/gpm installas a routine local step but never addresses who those execs run as. This doc is its complement: it explains why the exec must drop to the host user.architecture-patterns/dual-repo-submodule-workflow.md— worktrees + the./usersubmodule/bind mount, including the persistentM userdirty-state warning. Root-owned files landing in./userfrom root-default execs are a concrete cause of unexpected permission/dirty state in worktree dev servers.docs/guides/deploy-cycle.md— the three-layer state model (plugin code / repo config / host env tree); the host env tree is the layer across which these root-owned files land.