m038/intotheeast-com
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: add Pixelfed import implementation plan

Browse Source
This commit is contained in:
Mischa
2026-06-20 15:07:37 +02:00
parent 9402594eb8
commit fb3a656db5
1 changed files with 546 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/plans/2026-06-20-pixelfed-import.md
+546
View File
@@ -0,0 +1,546 @@
# Pixelfed Import & Demo Reorganisation Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Import 36 Pixelfed posts into three permanent trips and reorganise the demo system so Italy demo content moves to a clearly-labelled `italy-2026-demo` trip and Japan demo content is retired.
**Architecture:** Three independent tasks — demo cleanup first, then real trip scaffolding, then the Python import script that routes posts by year and downloads photos. All user-facing content lives in `user/pages/` and is committed to the `user/` git repo; the Makefile and import script are committed to the main repo.
**Tech Stack:** Bash (file operations), Python 3 stdlib only (json, os, urllib.request, datetime), Grav Flat-File CMS YAML frontmatter.
## Global Constraints
- Dev server: `http://localhost:8081` — must be running (`make start`) to test cache clears
- `user/` is a separate git repo — all commits to `user/pages/`, `user/docs/`, `user/themes/` use `git -C user commit`; Makefile and `scripts/` use the root `git commit`
- Never read `.env` directly
- All new trip pages use `template: trip` for `trip.md`, `template: dailies` for the dailies index, `template: map` for map, `template: stats` for stats, `template: stories` for stories — matching existing trips exactly
- Input JSON: `/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json` (36 posts)
- Trip routing by `created_at` year: 2023 → `central-asia-2023`, 2024 → `us-canada-mex-2024`, 2025 → `italy-2025`
---
## File Map
| File | Change | Repo |
|---|---|---|
| `user/docs/demo/trips/italy-2026-demo/` | New — copy of italy-2025 demo source with updated `trip.md` | user |
| `user/pages/01.trips/italy-2026-demo/` | Not committed — created at runtime by `demo-load` | — |
| `user/pages/01.trips/italy-2025/trip.md` | Update title to `Cycling Tuscany 2025` | user |
| `user/pages/01.trips/italy-2025/01.dailies/dailies.md` | New — missing index page | user |
| `user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/` | Delete demo story | user |
| `user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino/` | Delete demo story | user |
| `user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena/` | Delete demo story | user |
| `user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/` | Delete 5 demo entries | user |
| `user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry` + `2026-04-*.entry` | Delete 9 demo entries | user |
| `user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/` | Delete demo story | user |
| `user/pages/01.trips/central-asia-2023/` | New permanent trip page tree | user |
| `user/pages/01.trips/us-canada-mex-2024/` | New permanent trip page tree | user |
| `Makefile` | Replace `demo-load` and `demo-reset` targets | main |
| `scripts/pixelfed-import.py` | New one-time import script | main |
---
## Task 1: Demo reorganisation
**Files:**
- Create: `user/docs/demo/trips/italy-2026-demo/` (copy + edit)
- Modify: `user/pages/01.trips/italy-2025/trip.md`
- Create: `user/pages/01.trips/italy-2025/01.dailies/dailies.md`
- Delete: `user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/`, `02.long-climb-montalcino/`, `03.one-evening-siena/`
- Delete: `user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/` (5 demo entries)
- Delete: `user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry` + `2026-04-*.entry` (9 demo entries)
- Delete: `user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/`
- Modify: `Makefile`
**Interfaces:**
- Produces: `demo-load` and `demo-reset` targets that only touch `italy-2026-demo`; `italy-2025` is clean and ready for real content; `japan-korea-2026` has only the real `2026-06-17.entry`
- [ ] **Step 1: Copy italy-2025 demo source to italy-2026-demo**
```bash
cp -r user/docs/demo/trips/italy-2025 user/docs/demo/trips/italy-2026-demo
```
- [ ] **Step 2: Update the trip.md in the new demo source**
Edit `user/docs/demo/trips/italy-2026-demo/trip.md` to read:
```yaml
---
title: 'Italy 2026 (Demo)'
template: trip
date: '2026-09-01'
date_start: '2026-09-01'
date_end: '2026-09-08'
cover_image: ''
---
```
- [ ] **Step 3: Remove italy-2025 demo stories from pages**
```bash
rm -rf user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn
rm -rf user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino
rm -rf user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena
```
- [ ] **Step 4: Remove italy-2025 demo dailies entries from pages**
```bash
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-0800-rolling-through-val-dorcia.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-1900-siena-at-dusk.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1200-towers-of-san-gimignano.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1800-into-florence.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-08-0900-tyrrhenian-coast.entry
```
Check actual folder names first in case any differ:
```bash
ls user/pages/01.trips/italy-2025/01.dailies/
```
Remove all folders listed (they are all demo content).
- [ ] **Step 5: Add missing dailies.md to italy-2025**
Create `user/pages/01.trips/italy-2025/01.dailies/dailies.md`:
```yaml
---
title: 'The Journey'
template: dailies
content:
items: '@self.children'
order:
by: date
dir: desc
filter:
published: true
---
```
- [ ] **Step 6: Update italy-2025 trip title**
Edit `user/pages/01.trips/italy-2025/trip.md`:
```yaml
---
title: 'Cycling Tuscany 2025'
template: trip
date: '2025-10-11'
date_start: '2025-10-11'
date_end: '2025-10-16'
cover_image: ''
---
```
- [ ] **Step 7: Remove japan demo content from pages**
```bash
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-25-1540-wheels-down-narita.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-26-1000-sakura-in-ueno-park.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-27-0715-summit-clouds-and-snow.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-28-1130-thousand-torii-gates.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-29-1400-deer-of-nara.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-30-1800-dotonbori-after-dark.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-31-0730-last-morning-in-arashiyama.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-01-0900-seoul-calling.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-02-1100-gyeongbokgung-and-beyond.entry
rm -rf user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates
```
- [ ] **Step 8: Update the Makefile demo targets**
Replace the entire `demo-load` and `demo-reset` blocks (lines 42–64) with:
```makefile
demo-load:
# Load italy-2026-demo trip (create pages if absent)
mkdir -p user/pages/01.trips/italy-2026-demo/01.dailies user/pages/01.trips/italy-2026-demo/02.map user/pages/01.trips/italy-2026-demo/03.stats user/pages/01.trips/italy-2026-demo/04.stories
cp user/docs/demo/trips/italy-2026-demo/trip.md user/pages/01.trips/italy-2026-demo/trip.md 2>/dev/null || true
cp user/docs/demo/trips/italy-2026-demo/map.md user/pages/01.trips/italy-2026-demo/02.map/map.md 2>/dev/null || true
cp user/docs/demo/trips/italy-2026-demo/stats.md user/pages/01.trips/italy-2026-demo/03.stats/stats.md 2>/dev/null || true
cp user/docs/demo/trips/italy-2026-demo/stories.md user/pages/01.trips/italy-2026-demo/04.stories/stories.md 2>/dev/null || true
cp -r user/docs/demo/trips/italy-2026-demo/04.stories/. user/pages/01.trips/italy-2026-demo/04.stories/ 2>/dev/null || true
cp -r user/docs/demo/trips/italy-2026-demo/dailies/. user/pages/01.trips/italy-2026-demo/01.dailies/
cp user/docs/demo/trips/italy-2026-demo/*.gpx user/pages/01.trips/italy-2026-demo/ 2>/dev/null || true
docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache"
demo-reset:
rm -rf user/pages/01.trips/italy-2026-demo
docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache"
```
- [ ] **Step 9: Verify demo-load and demo-reset work**
```bash
make demo-load
```
Expected: `italy-2026-demo` appears at `http://localhost:8081` trips listing. Confirm stories and GPX map load.
```bash
make demo-reset
```
Expected: `italy-2026-demo` disappears from the trips listing. `italy-2025` and `japan-korea-2026` are unaffected.
- [ ] **Step 10: Commit user repo changes**
Includes the new demo source, cleaned pages, updated italy-2025 title, and new dailies.md:
```bash
git -C user add -A
git -C user commit -m "chore: move italy demo to italy-2026-demo; clean japan and italy-2025 demo content"
```
- [ ] **Step 11: Commit main repo changes (Makefile only)**
```bash
git add Makefile
git commit -m "chore: update demo-load/demo-reset for italy-2026-demo; retire japan demo"
```
---
## Task 2: Create real trip page trees
**Files:**
- Create: `user/pages/01.trips/central-asia-2023/` (trip.md + 4 subfolders with index pages)
- Create: `user/pages/01.trips/us-canada-mex-2024/` (trip.md + 4 subfolders with index pages)
**Interfaces:**
- Produces: `central-asia-2023` and `us-canada-mex-2024` trip folder trees with `01.dailies/dailies.md` present — required for the import script to write entries into them
- [ ] **Step 1: Create Central Asia 2023 trip tree**
```bash
mkdir -p user/pages/01.trips/central-asia-2023/01.dailies
mkdir -p user/pages/01.trips/central-asia-2023/02.map
mkdir -p user/pages/01.trips/central-asia-2023/03.stats
mkdir -p user/pages/01.trips/central-asia-2023/04.stories
```
Create `user/pages/01.trips/central-asia-2023/trip.md`:
```yaml
---
title: 'Central Asia 2023'
template: trip
date: '2023-08-28'
date_start: '2023-08-28'
date_end: '2023-10-18'
cover_image: ''
---
```
Create `user/pages/01.trips/central-asia-2023/01.dailies/dailies.md`:
```yaml
---
title: 'The Journey'
template: dailies
content:
items: '@self.children'
order:
by: date
dir: desc
filter:
published: true
---
```
Create `user/pages/01.trips/central-asia-2023/02.map/map.md`:
```yaml
---
title: 'Trip Map'
template: map
---
```
Create `user/pages/01.trips/central-asia-2023/03.stats/stats.md`:
```yaml
---
title: 'Trip Stats'
template: stats
---
```
Create `user/pages/01.trips/central-asia-2023/04.stories/stories.md`:
```yaml
---
title: Stories
template: stories
published: true
---
```
- [ ] **Step 2: Create Northern America 2024 trip tree**
```bash
mkdir -p user/pages/01.trips/us-canada-mex-2024/01.dailies
mkdir -p user/pages/01.trips/us-canada-mex-2024/02.map
mkdir -p user/pages/01.trips/us-canada-mex-2024/03.stats
mkdir -p user/pages/01.trips/us-canada-mex-2024/04.stories
```
Create `user/pages/01.trips/us-canada-mex-2024/trip.md`:
```yaml
---
title: 'Northern America 2024'
template: trip
date: '2024-05-28'
date_start: '2024-05-28'
date_end: '2024-08-07'
cover_image: ''
---
```
Create `user/pages/01.trips/us-canada-mex-2024/01.dailies/dailies.md`:
```yaml
---
title: 'The Journey'
template: dailies
content:
items: '@self.children'
order:
by: date
dir: desc
filter:
published: true
---
```
Create `user/pages/01.trips/us-canada-mex-2024/02.map/map.md`:
```yaml
---
title: 'Trip Map'
template: map
---
```
Create `user/pages/01.trips/us-canada-mex-2024/03.stats/stats.md`:
```yaml
---
title: 'Trip Stats'
template: stats
---
```
Create `user/pages/01.trips/us-canada-mex-2024/04.stories/stories.md`:
```yaml
---
title: Stories
template: stories
published: true
---
```
- [ ] **Step 3: Verify trips appear in the site**
Open `http://localhost:8081` — the Past Trips section should list Central Asia 2023 and Northern America 2024.
- [ ] **Step 4: Commit to user repo**
```bash
git -C user add user/pages/01.trips/central-asia-2023 user/pages/01.trips/us-canada-mex-2024
git -C user commit -m "feat: add central-asia-2023 and us-canada-mex-2024 trip page trees"
```
---
## Task 3: Pixelfed import script
**Files:**
- Create: `scripts/pixelfed-import.py`
- Modify: `Makefile` (add `pixelfed-import` target)
**Interfaces:**
- Consumes: `user/pages/01.trips/{trip}/01.dailies/` folders from Task 2 and existing `italy-2025`
- Produces: `{date}-pixelfed-{N}.entry/` folders with `entry.md` + downloaded photo files
- [ ] **Step 1: Write the import script**
Create `scripts/pixelfed-import.py`:
```python
#!/usr/bin/env python3
"""One-time import of Pixelfed statuses into Grav entry pages."""
import json
import os
import urllib.request
from datetime import datetime, timezone
INPUT_FILE = '/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json'
USER_PAGES = 'user/pages/01.trips'
TRIP_MAP = {
'2023': 'central-asia-2023',
'2024': 'us-canada-mex-2024',
'2025': 'italy-2025',
}
EXT_MAP = {
'image/jpeg': 'jpg',
'image/png': 'png',
'image/gif': 'gif',
'image/webp': 'webp',
}
ENTRY_TEMPLATE = """\
---
title: '{title}'
date: '{date}'
template: entry
published: true
hero_image: '{hero_image}'
lat: ''
lng: ''
location_city: '{location_city}'
location_country: '{location_country}'
weather_temp_c: ''
weather_desc: ''
---
{body}
"""
def download(url, dest):
try:
urllib.request.urlretrieve(url, dest)
return True
except Exception as exc:
print(f' Warning: download failed {url}: {exc}')
return False
def main():
with open(INPUT_FILE) as f:
posts = json.load(f)
counters = {}
for post in posts:
year = post['created_at'][:4]
trip = TRIP_MAP.get(year)
if not trip:
print(f"Skip: no trip mapping for year {year} (post {post['id']})")
continue
counters[trip] = counters.get(trip, 0) + 1
n = counters[trip]
date_str = post['created_at'][:10] # YYYY-MM-DD
folder = f'{date_str}-pixelfed-{n}.entry'
path = os.path.join(USER_PAGES, trip, '01.dailies', folder)
if os.path.exists(path):
print(f'Skip: {folder} already exists')
continue
os.makedirs(path)
print(f'Creating {trip}/{folder}')
hero_image = ''
for i, att in enumerate(post.get('media_attachments', []), 1):
ext = EXT_MAP.get(att.get('mime', ''), 'jpg')
filename = f'photo-{i}.{ext}'
if download(att['url'], os.path.join(path, filename)) and i == 1:
hero_image = filename
place = post.get('place') or {}
dt = datetime.fromisoformat(post['created_at'].replace('Z', '+00:00'))
date_fmt = dt.strftime('%Y-%m-%d %H:%M')
entry_md = ENTRY_TEMPLATE.format(
title=f'Pixelfed Import {n}',
date=date_fmt,
hero_image=hero_image,
location_city=place.get('name', ''),
location_country=place.get('country', ''),
body=post.get('content_text', '').strip(),
)
with open(os.path.join(path, 'entry.md'), 'w') as f:
f.write(entry_md)
print(f'\nDone. Posts per trip: {counters}')
if __name__ == '__main__':
main()
```
- [ ] **Step 2: Add make target**
In `Makefile`, after the `demo-reset` block, add:
```makefile
pixelfed-import:
python3 scripts/pixelfed-import.py
```
- [ ] **Step 3: Run the import**
```bash
make pixelfed-import
```
Expected output (approximately):
```
Creating central-asia-2023/2023-08-28-pixelfed-1.entry
Creating central-asia-2023/2023-08-29-pixelfed-2.entry
...
Creating us-canada-mex-2024/2024-05-28-pixelfed-1.entry
...
Creating italy-2025/2025-10-11-pixelfed-1.entry
Creating italy-2025/2025-10-16-pixelfed-2.entry
Done. Posts per trip: {'central-asia-2023': 22, 'us-canada-mex-2024': 12, 'italy-2025': 2}
```
- [ ] **Step 4: Verify entries in the browser**
Open `http://localhost:8081/trips/central-asia-2023/dailies` — confirm entries appear in reverse-date order with photos.
Open one entry (e.g. the first Central Asia post) — confirm the hero image displays and the body text is readable.
- [ ] **Step 5: Verify entries in Admin2**
Log in at `http://localhost:8081/admin`. Navigate to Pages → find one of the new entry pages. Confirm the media tab shows the downloaded photos.
- [ ] **Step 6: Commit main repo (script + Makefile)**
```bash
git add scripts/pixelfed-import.py Makefile
git commit -m "feat: add pixelfed-import script and make target"
```
- [ ] **Step 7: Commit user repo (imported entries)**
```bash
git -C user add user/pages/01.trips/central-asia-2023/01.dailies
git -C user add user/pages/01.trips/us-canada-mex-2024/01.dailies
git -C user add user/pages/01.trips/italy-2025/01.dailies
git -C user commit -m "feat: import 36 Pixelfed posts into central-asia-2023, us-canada-mex-2024, italy-2025"
```
- [ ] **Step 8: Push to Gitea**
```bash
make content-push
```
Expected: push completes, production webhook fires.