From fb3a656db5a70453b3295beb871a48ad163dd005 Mon Sep 17 00:00:00 2001 From: Mischa Date: Sat, 20 Jun 2026 15:07:37 +0200 Subject: [PATCH] docs: add Pixelfed import implementation plan --- .../plans/2026-06-20-pixelfed-import.md | 546 ++++++++++++++++++ 1 file changed, 546 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-20-pixelfed-import.md diff --git a/docs/superpowers/plans/2026-06-20-pixelfed-import.md b/docs/superpowers/plans/2026-06-20-pixelfed-import.md new file mode 100644 index 0000000..0d2f690 --- /dev/null +++ b/docs/superpowers/plans/2026-06-20-pixelfed-import.md @@ -0,0 +1,546 @@ +# Pixelfed Import & Demo Reorganisation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Import 36 Pixelfed posts into three permanent trips and reorganise the demo system so Italy demo content moves to a clearly-labelled `italy-2026-demo` trip and Japan demo content is retired. + +**Architecture:** Three independent tasks — demo cleanup first, then real trip scaffolding, then the Python import script that routes posts by year and downloads photos. All user-facing content lives in `user/pages/` and is committed to the `user/` git repo; the Makefile and import script are committed to the main repo. + +**Tech Stack:** Bash (file operations), Python 3 stdlib only (json, os, urllib.request, datetime), Grav Flat-File CMS YAML frontmatter. + +## Global Constraints + +- Dev server: `http://localhost:8081` — must be running (`make start`) to test cache clears +- `user/` is a separate git repo — all commits to `user/pages/`, `user/docs/`, `user/themes/` use `git -C user commit`; Makefile and `scripts/` use the root `git commit` +- Never read `.env` directly +- All new trip pages use `template: trip` for `trip.md`, `template: dailies` for the dailies index, `template: map` for map, `template: stats` for stats, `template: stories` for stories — matching existing trips exactly +- Input JSON: `/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json` (36 posts) +- Trip routing by `created_at` year: 2023 → `central-asia-2023`, 2024 → `us-canada-mex-2024`, 2025 → `italy-2025` + +--- + +## File Map + +| File | Change | Repo | +|---|---|---| +| `user/docs/demo/trips/italy-2026-demo/` | New — copy of italy-2025 demo source with updated `trip.md` | user | +| `user/pages/01.trips/italy-2026-demo/` | Not committed — created at runtime by `demo-load` | — | +| `user/pages/01.trips/italy-2025/trip.md` | Update title to `Cycling Tuscany 2025` | user | +| `user/pages/01.trips/italy-2025/01.dailies/dailies.md` | New — missing index page | user | +| `user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/` | Delete demo story | user | +| `user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino/` | Delete demo story | user | +| `user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena/` | Delete demo story | user | +| `user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/` | Delete 5 demo entries | user | +| `user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry` + `2026-04-*.entry` | Delete 9 demo entries | user | +| `user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/` | Delete demo story | user | +| `user/pages/01.trips/central-asia-2023/` | New permanent trip page tree | user | +| `user/pages/01.trips/us-canada-mex-2024/` | New permanent trip page tree | user | +| `Makefile` | Replace `demo-load` and `demo-reset` targets | main | +| `scripts/pixelfed-import.py` | New one-time import script | main | + +--- + +## Task 1: Demo reorganisation + +**Files:** +- Create: `user/docs/demo/trips/italy-2026-demo/` (copy + edit) +- Modify: `user/pages/01.trips/italy-2025/trip.md` +- Create: `user/pages/01.trips/italy-2025/01.dailies/dailies.md` +- Delete: `user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/`, `02.long-climb-montalcino/`, `03.one-evening-siena/` +- Delete: `user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/` (5 demo entries) +- Delete: `user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry` + `2026-04-*.entry` (9 demo entries) +- Delete: `user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/` +- Modify: `Makefile` + +**Interfaces:** +- Produces: `demo-load` and `demo-reset` targets that only touch `italy-2026-demo`; `italy-2025` is clean and ready for real content; `japan-korea-2026` has only the real `2026-06-17.entry` + +- [ ] **Step 1: Copy italy-2025 demo source to italy-2026-demo** + +```bash +cp -r user/docs/demo/trips/italy-2025 user/docs/demo/trips/italy-2026-demo +``` + +- [ ] **Step 2: Update the trip.md in the new demo source** + +Edit `user/docs/demo/trips/italy-2026-demo/trip.md` to read: + +```yaml +--- +title: 'Italy 2026 (Demo)' +template: trip +date: '2026-09-01' +date_start: '2026-09-01' +date_end: '2026-09-08' +cover_image: '' +--- +``` + +- [ ] **Step 3: Remove italy-2025 demo stories from pages** + +```bash +rm -rf user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn +rm -rf user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino +rm -rf user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena +``` + +- [ ] **Step 4: Remove italy-2025 demo dailies entries from pages** + +```bash +rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-0800-rolling-through-val-dorcia.entry +rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-1900-siena-at-dusk.entry +rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1200-towers-of-san-gimignano.entry +rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1800-into-florence.entry +rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-08-0900-tyrrhenian-coast.entry +``` + +Check actual folder names first in case any differ: + +```bash +ls user/pages/01.trips/italy-2025/01.dailies/ +``` + +Remove all folders listed (they are all demo content). + +- [ ] **Step 5: Add missing dailies.md to italy-2025** + +Create `user/pages/01.trips/italy-2025/01.dailies/dailies.md`: + +```yaml +--- +title: 'The Journey' +template: dailies +content: + items: '@self.children' + order: + by: date + dir: desc + filter: + published: true +--- +``` + +- [ ] **Step 6: Update italy-2025 trip title** + +Edit `user/pages/01.trips/italy-2025/trip.md`: + +```yaml +--- +title: 'Cycling Tuscany 2025' +template: trip +date: '2025-10-11' +date_start: '2025-10-11' +date_end: '2025-10-16' +cover_image: '' +--- +``` + +- [ ] **Step 7: Remove japan demo content from pages** + +```bash +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-25-1540-wheels-down-narita.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-26-1000-sakura-in-ueno-park.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-27-0715-summit-clouds-and-snow.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-28-1130-thousand-torii-gates.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-29-1400-deer-of-nara.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-30-1800-dotonbori-after-dark.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-31-0730-last-morning-in-arashiyama.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-01-0900-seoul-calling.entry +rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-02-1100-gyeongbokgung-and-beyond.entry +rm -rf user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates +``` + +- [ ] **Step 8: Update the Makefile demo targets** + +Replace the entire `demo-load` and `demo-reset` blocks (lines 42–64) with: + +```makefile +demo-load: + # Load italy-2026-demo trip (create pages if absent) + mkdir -p user/pages/01.trips/italy-2026-demo/01.dailies user/pages/01.trips/italy-2026-demo/02.map user/pages/01.trips/italy-2026-demo/03.stats user/pages/01.trips/italy-2026-demo/04.stories + cp user/docs/demo/trips/italy-2026-demo/trip.md user/pages/01.trips/italy-2026-demo/trip.md 2>/dev/null || true + cp user/docs/demo/trips/italy-2026-demo/map.md user/pages/01.trips/italy-2026-demo/02.map/map.md 2>/dev/null || true + cp user/docs/demo/trips/italy-2026-demo/stats.md user/pages/01.trips/italy-2026-demo/03.stats/stats.md 2>/dev/null || true + cp user/docs/demo/trips/italy-2026-demo/stories.md user/pages/01.trips/italy-2026-demo/04.stories/stories.md 2>/dev/null || true + cp -r user/docs/demo/trips/italy-2026-demo/04.stories/. user/pages/01.trips/italy-2026-demo/04.stories/ 2>/dev/null || true + cp -r user/docs/demo/trips/italy-2026-demo/dailies/. user/pages/01.trips/italy-2026-demo/01.dailies/ + cp user/docs/demo/trips/italy-2026-demo/*.gpx user/pages/01.trips/italy-2026-demo/ 2>/dev/null || true + docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache" + +demo-reset: + rm -rf user/pages/01.trips/italy-2026-demo + docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache" +``` + +- [ ] **Step 9: Verify demo-load and demo-reset work** + +```bash +make demo-load +``` + +Expected: `italy-2026-demo` appears at `http://localhost:8081` trips listing. Confirm stories and GPX map load. + +```bash +make demo-reset +``` + +Expected: `italy-2026-demo` disappears from the trips listing. `italy-2025` and `japan-korea-2026` are unaffected. + +- [ ] **Step 10: Commit user repo changes** + +Includes the new demo source, cleaned pages, updated italy-2025 title, and new dailies.md: + +```bash +git -C user add -A +git -C user commit -m "chore: move italy demo to italy-2026-demo; clean japan and italy-2025 demo content" +``` + +- [ ] **Step 11: Commit main repo changes (Makefile only)** + +```bash +git add Makefile +git commit -m "chore: update demo-load/demo-reset for italy-2026-demo; retire japan demo" +``` + +--- + +## Task 2: Create real trip page trees + +**Files:** +- Create: `user/pages/01.trips/central-asia-2023/` (trip.md + 4 subfolders with index pages) +- Create: `user/pages/01.trips/us-canada-mex-2024/` (trip.md + 4 subfolders with index pages) + +**Interfaces:** +- Produces: `central-asia-2023` and `us-canada-mex-2024` trip folder trees with `01.dailies/dailies.md` present — required for the import script to write entries into them + +- [ ] **Step 1: Create Central Asia 2023 trip tree** + +```bash +mkdir -p user/pages/01.trips/central-asia-2023/01.dailies +mkdir -p user/pages/01.trips/central-asia-2023/02.map +mkdir -p user/pages/01.trips/central-asia-2023/03.stats +mkdir -p user/pages/01.trips/central-asia-2023/04.stories +``` + +Create `user/pages/01.trips/central-asia-2023/trip.md`: + +```yaml +--- +title: 'Central Asia 2023' +template: trip +date: '2023-08-28' +date_start: '2023-08-28' +date_end: '2023-10-18' +cover_image: '' +--- +``` + +Create `user/pages/01.trips/central-asia-2023/01.dailies/dailies.md`: + +```yaml +--- +title: 'The Journey' +template: dailies +content: + items: '@self.children' + order: + by: date + dir: desc + filter: + published: true +--- +``` + +Create `user/pages/01.trips/central-asia-2023/02.map/map.md`: + +```yaml +--- +title: 'Trip Map' +template: map +--- +``` + +Create `user/pages/01.trips/central-asia-2023/03.stats/stats.md`: + +```yaml +--- +title: 'Trip Stats' +template: stats +--- +``` + +Create `user/pages/01.trips/central-asia-2023/04.stories/stories.md`: + +```yaml +--- +title: Stories +template: stories +published: true +--- +``` + +- [ ] **Step 2: Create Northern America 2024 trip tree** + +```bash +mkdir -p user/pages/01.trips/us-canada-mex-2024/01.dailies +mkdir -p user/pages/01.trips/us-canada-mex-2024/02.map +mkdir -p user/pages/01.trips/us-canada-mex-2024/03.stats +mkdir -p user/pages/01.trips/us-canada-mex-2024/04.stories +``` + +Create `user/pages/01.trips/us-canada-mex-2024/trip.md`: + +```yaml +--- +title: 'Northern America 2024' +template: trip +date: '2024-05-28' +date_start: '2024-05-28' +date_end: '2024-08-07' +cover_image: '' +--- +``` + +Create `user/pages/01.trips/us-canada-mex-2024/01.dailies/dailies.md`: + +```yaml +--- +title: 'The Journey' +template: dailies +content: + items: '@self.children' + order: + by: date + dir: desc + filter: + published: true +--- +``` + +Create `user/pages/01.trips/us-canada-mex-2024/02.map/map.md`: + +```yaml +--- +title: 'Trip Map' +template: map +--- +``` + +Create `user/pages/01.trips/us-canada-mex-2024/03.stats/stats.md`: + +```yaml +--- +title: 'Trip Stats' +template: stats +--- +``` + +Create `user/pages/01.trips/us-canada-mex-2024/04.stories/stories.md`: + +```yaml +--- +title: Stories +template: stories +published: true +--- +``` + +- [ ] **Step 3: Verify trips appear in the site** + +Open `http://localhost:8081` — the Past Trips section should list Central Asia 2023 and Northern America 2024. + +- [ ] **Step 4: Commit to user repo** + +```bash +git -C user add user/pages/01.trips/central-asia-2023 user/pages/01.trips/us-canada-mex-2024 +git -C user commit -m "feat: add central-asia-2023 and us-canada-mex-2024 trip page trees" +``` + +--- + +## Task 3: Pixelfed import script + +**Files:** +- Create: `scripts/pixelfed-import.py` +- Modify: `Makefile` (add `pixelfed-import` target) + +**Interfaces:** +- Consumes: `user/pages/01.trips/{trip}/01.dailies/` folders from Task 2 and existing `italy-2025` +- Produces: `{date}-pixelfed-{N}.entry/` folders with `entry.md` + downloaded photo files + +- [ ] **Step 1: Write the import script** + +Create `scripts/pixelfed-import.py`: + +```python +#!/usr/bin/env python3 +"""One-time import of Pixelfed statuses into Grav entry pages.""" + +import json +import os +import urllib.request +from datetime import datetime, timezone + +INPUT_FILE = '/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json' +USER_PAGES = 'user/pages/01.trips' + +TRIP_MAP = { + '2023': 'central-asia-2023', + '2024': 'us-canada-mex-2024', + '2025': 'italy-2025', +} + +EXT_MAP = { + 'image/jpeg': 'jpg', + 'image/png': 'png', + 'image/gif': 'gif', + 'image/webp': 'webp', +} + +ENTRY_TEMPLATE = """\ +--- +title: '{title}' +date: '{date}' +template: entry +published: true +hero_image: '{hero_image}' +lat: '' +lng: '' +location_city: '{location_city}' +location_country: '{location_country}' +weather_temp_c: '' +weather_desc: '' +--- + +{body} +""" + + +def download(url, dest): + try: + urllib.request.urlretrieve(url, dest) + return True + except Exception as exc: + print(f' Warning: download failed {url}: {exc}') + return False + + +def main(): + with open(INPUT_FILE) as f: + posts = json.load(f) + + counters = {} + + for post in posts: + year = post['created_at'][:4] + trip = TRIP_MAP.get(year) + if not trip: + print(f"Skip: no trip mapping for year {year} (post {post['id']})") + continue + + counters[trip] = counters.get(trip, 0) + 1 + n = counters[trip] + + date_str = post['created_at'][:10] # YYYY-MM-DD + folder = f'{date_str}-pixelfed-{n}.entry' + path = os.path.join(USER_PAGES, trip, '01.dailies', folder) + + if os.path.exists(path): + print(f'Skip: {folder} already exists') + continue + + os.makedirs(path) + print(f'Creating {trip}/{folder}') + + hero_image = '' + for i, att in enumerate(post.get('media_attachments', []), 1): + ext = EXT_MAP.get(att.get('mime', ''), 'jpg') + filename = f'photo-{i}.{ext}' + if download(att['url'], os.path.join(path, filename)) and i == 1: + hero_image = filename + + place = post.get('place') or {} + dt = datetime.fromisoformat(post['created_at'].replace('Z', '+00:00')) + date_fmt = dt.strftime('%Y-%m-%d %H:%M') + + entry_md = ENTRY_TEMPLATE.format( + title=f'Pixelfed Import {n}', + date=date_fmt, + hero_image=hero_image, + location_city=place.get('name', ''), + location_country=place.get('country', ''), + body=post.get('content_text', '').strip(), + ) + + with open(os.path.join(path, 'entry.md'), 'w') as f: + f.write(entry_md) + + print(f'\nDone. Posts per trip: {counters}') + + +if __name__ == '__main__': + main() +``` + +- [ ] **Step 2: Add make target** + +In `Makefile`, after the `demo-reset` block, add: + +```makefile +pixelfed-import: + python3 scripts/pixelfed-import.py +``` + +- [ ] **Step 3: Run the import** + +```bash +make pixelfed-import +``` + +Expected output (approximately): + +``` +Creating central-asia-2023/2023-08-28-pixelfed-1.entry +Creating central-asia-2023/2023-08-29-pixelfed-2.entry +... +Creating us-canada-mex-2024/2024-05-28-pixelfed-1.entry +... +Creating italy-2025/2025-10-11-pixelfed-1.entry +Creating italy-2025/2025-10-16-pixelfed-2.entry + +Done. Posts per trip: {'central-asia-2023': 22, 'us-canada-mex-2024': 12, 'italy-2025': 2} +``` + +- [ ] **Step 4: Verify entries in the browser** + +Open `http://localhost:8081/trips/central-asia-2023/dailies` — confirm entries appear in reverse-date order with photos. + +Open one entry (e.g. the first Central Asia post) — confirm the hero image displays and the body text is readable. + +- [ ] **Step 5: Verify entries in Admin2** + +Log in at `http://localhost:8081/admin`. Navigate to Pages → find one of the new entry pages. Confirm the media tab shows the downloaded photos. + +- [ ] **Step 6: Commit main repo (script + Makefile)** + +```bash +git add scripts/pixelfed-import.py Makefile +git commit -m "feat: add pixelfed-import script and make target" +``` + +- [ ] **Step 7: Commit user repo (imported entries)** + +```bash +git -C user add user/pages/01.trips/central-asia-2023/01.dailies +git -C user add user/pages/01.trips/us-canada-mex-2024/01.dailies +git -C user add user/pages/01.trips/italy-2025/01.dailies +git -C user commit -m "feat: import 36 Pixelfed posts into central-asia-2023, us-canada-mex-2024, italy-2025" +``` + +- [ ] **Step 8: Push to Gitea** + +```bash +make content-push +``` + +Expected: push completes, production webhook fires.