Files
intotheeast-com/docs/solutions/architecture-patterns/dual-repo-submodule-workflow.md
T
m038andClaude Fable 5 24867524a1 docs(solutions): compound refresh — fix reference drift, grow CONCEPTS.md
Refresh audit of all 13 docs/solutions learnings against the current
codebase. Core guidance verified accurate everywhere; three docs had
reference drift:

- dual-repo-submodule-workflow: point worktree setup/teardown at the
  make worktree-new/worktree-rm targets (manual procedure misses
  .worktree-env isolation)
- docker-exec-root-owned-bind-mount-files: tracked-plugin list now
  includes entry-actions; fix-perms description matches actual target
- grav-plugin-config-without-code-wont-enable: 3-category model's
  custom-in-repo list now includes entry-actions

CONCEPTS.md: add Container, Content repo, Outer repo, Pin, Env tree,
Remote-only plugin; refresh Active Trip (switching is one setting now).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195b3cDdMeize2Mm1FgC2aU
2026-07-08 23:37:56 +02:00

100 lines
7.6 KiB
Markdown

---
title: Dual-repo submodule workflow — outer dev-env repo + user/ content submodule
date: 2026-07-04
category: architecture-patterns
module: Repo structure — outer repo + user/ content repo
problem_type: architecture_pattern
component: git
severity: medium
applies_when:
- Starting a feature that touches both the outer repo and user/ (theme, plugins, pages)
- Setting up a git worktree for long-running work while doing other work in parallel
- Deciding when to bump the user/ submodule pointer in the outer repo
- A git worktree of the outer repo shows an empty or broken user/ directory
- Seeing a persistent "M user" / "m user" dirty state in the outer repo
tags: [git, submodule, worktree, dual-repo, user-repo, docker, content-sync, pointer-bump]
---
# Dual-repo submodule workflow
## Context
This project is **two independent git repositories** that happen to be nested:
- **Outer repo** (`intotheeast-com.git`) — the Grav dev environment: `tests/`, `scripts/`, `docs/`, `docker-compose.yml`, `Dockerfile`, `Makefile`, `CLAUDE.md`.
- **`user/` repo** (`intotheeast-com-content.git`) — all site content and the theme: `pages/`, `config/`, `accounts/`, `themes/`. It has its own remote (the Gitea content mirror) and its own release cadence (`make content-push` → webhook → production pull).
As of 2026-07-04, the outer repo tracks `user/` as a **proper git submodule** (`.gitmodules` at the outer root, git dir absorbed into `.git/modules/user`). Before that it was an *orphaned gitlink* — a `160000` tree entry with no `.gitmodules`, so git had no URL to populate or update it. That broke worktrees (a fresh outer worktree got an empty `user/`) and offered no supported sync path.
## Why a submodule (and not untracking)
Two options were weighed: make it a real submodule, or stop tracking `user/` in the outer repo entirely (gitignore it, symlink the real checkout in).
The submodule was chosen deliberately, for one reason that outweighs its ceremony:
- **Routine content churn is benign** — day-to-day entries/stories change `user/` constantly and never break the dev environment. Those changes do **not** need to be reflected in the outer repo.
- **Cross-repo *features* must be tracked together.** A feature like the journal post-form touches both repos (a plugin + theme JS/CSS in `user/`, and tests/docs in the outer repo). The outer repo pinning an exact `user/` commit records *"this dev-env state expects this content/theme state"* — so checking out the outer feature also gets the matching `user/` code. That coupling is real and worth having.
- It enables **per-worktree `user/` checkouts**, which is what makes true parallel work across both repos possible (see below). This was a hard requirement.
The cost accepted: a persistent `M user` dirty signal (intrinsic to submodules under active development) and the possibility of gitlink merge conflicts between outer branches. Neither is removed by the submodule; they are the price of version pinning.
## The pointer-bump convention
The outer repo's `user` gitlink stores an exact `user/` commit SHA. **When to bump it:**
- **Routine content changes → do not bump.** Push content with `make content-push` and leave the outer pin where it is. A stale pin during normal content work is expected and harmless.
- **At the end of a cross-repo feature → bump once.** When the feature's `user/` work is finalized, update the outer pin to the finished `user/` commit, as the final step of the feature (its own `chore: bump user pointer to <sha>` commit, or folded into the final integration commit).
Two rules keep the pin from dangling for other machines/clones:
1. **Pin a commit reachable from `user/`'s published `main`.** Prefer the **merge-to-main commit**. Pinning a feature-branch tip is safe *only* if that exact commit survives onto `main` (fast-forward / no-squash merge); a squashed-away tip becomes an orphaned SHA and `git submodule update` fails elsewhere.
2. **Push `user/` before the outer repo.** The submodule golden rule: the superproject references a child SHA, so the child must already be pushed. `make content-push` handles the `user/` push — just do it before pushing the outer branch.
Production is unaffected either way: prod pulls `user/` directly via the content-remote webhook, independent of the outer repo's pin. The pin is **dev-side coordination only**.
## Parallel work: worktree + its own dev server
The payoff. Because `docker-compose.yml` mounts `./user` **relative to the compose file**, and a worktree is a full copy of the outer tree (compose file included), each worktree serves *its own* `user/`. Two worktrees = two independent sites, no gitlink collisions.
**Use the make targets — don't do the steps by hand.** From the main checkout:
```bash
make worktree-new NAME=<feature> # create + start its own dev server
make worktree-rm NAME=<feature> # tear down cleanly
```
`worktree-new` does, in order: `git worktree add .worktrees/<feature> -b feat/<feature> main`, `git submodule update --init user`, branches `user/` onto `feat/<feature>`, writes a git-ignored `.worktree-env` (own compose project name, container name, auto-assigned port `8090+`) so every `make`/compose command run inside that worktree targets its own server, and starts the Grav service. The manual equivalent misses `.worktree-env` — without it, make commands in the worktree hit the main checkout's container on `:8081`.
`.worktrees/` is kept out of git via `.git/info/exclude` (local, shared across worktrees — no committed `.gitignore` change needed).
### Teardown
A submodule inside a linked worktree stores its git dir under `.git/modules/user/worktrees/<name>`, so teardown needs a submodule-deinit step before the worktree can be removed — skipping it is what leaves orphaned `.worktrees/` dirs. `make worktree-rm NAME=<feature>` runs the full sequence:
```bash
# what worktree-rm does internally
make -C .worktrees/<feature> stop # compose down (its own server)
git -C .worktrees/<feature> submodule deinit -f user # detach the submodule worktree
git worktree remove --force .worktrees/<feature>
git worktree prune
git branch -d feat/<feature> # manual, if merged
```
### Landing a commit on main without disturbing the main checkout
When the main checkout is mid-work on another branch, add a commit to `main` through a throwaway worktree instead of `git checkout main` (which would yank branches out from under an open IDE):
```bash
git worktree add .worktrees/main-tmp main
git -C .worktrees/main-tmp cherry-pick <sha> # or edit + commit
git worktree remove .worktrees/main-tmp
```
## Gotchas
- **`M user` / `m user` is normal.** Uppercase `M` = the pin differs from `user/` HEAD (bump pending or intentional). Lowercase `m` = the submodule working tree is dirty (e.g. an uncommitted `config/site.yaml` used for local testing). Neither is an error.
- **Gitlink merge conflicts still happen.** If two outer branches pin different `user/` SHAs, merging them conflicts on the `user` entry. Resolve by choosing the correct (usually newer, merged) SHA, then `git add user`.
- **Worktrees need `submodule update --init`.** A fresh outer worktree has an empty `user/` until you run it — it is not automatic.
- **The submodule git dir was absorbed** (`git submodule absorbgitdirs user`) so all worktrees share `.git/modules/user`. `user/.git` is now a gitfile (`gitdir: ../.git/modules/user`), not a directory. `make content-push`/`content-pull` still operate on `user/` normally.
- **Access requires the content remote** (SSH over Tailscale). A machine that cannot reach it cannot `submodule update` — but it could never clone `user/` anyway, so this is not a regression.