547 lines
17 KiB
Markdown
547 lines
17 KiB
Markdown
# Pixelfed Import & Demo Reorganisation Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Import 36 Pixelfed posts into three permanent trips and reorganise the demo system so Italy demo content moves to a clearly-labelled `italy-2026-demo` trip and Japan demo content is retired.
|
||
|
||
**Architecture:** Three independent tasks — demo cleanup first, then real trip scaffolding, then the Python import script that routes posts by year and downloads photos. All user-facing content lives in `user/pages/` and is committed to the `user/` git repo; the Makefile and import script are committed to the main repo.
|
||
|
||
**Tech Stack:** Bash (file operations), Python 3 stdlib only (json, os, urllib.request, datetime), Grav Flat-File CMS YAML frontmatter.
|
||
|
||
## Global Constraints
|
||
|
||
- Dev server: `http://localhost:8081` — must be running (`make start`) to test cache clears
|
||
- `user/` is a separate git repo — all commits to `user/pages/`, `user/docs/`, `user/themes/` use `git -C user commit`; Makefile and `scripts/` use the root `git commit`
|
||
- Never read `.env` directly
|
||
- All new trip pages use `template: trip` for `trip.md`, `template: dailies` for the dailies index, `template: map` for map, `template: stats` for stats, `template: stories` for stories — matching existing trips exactly
|
||
- Input JSON: `/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json` (36 posts)
|
||
- Trip routing by `created_at` year: 2023 → `central-asia-2023`, 2024 → `us-canada-mex-2024`, 2025 → `italy-2025`
|
||
|
||
---
|
||
|
||
## File Map
|
||
|
||
| File | Change | Repo |
|
||
|---|---|---|
|
||
| `user/docs/demo/trips/italy-2026-demo/` | New — copy of italy-2025 demo source with updated `trip.md` | user |
|
||
| `user/pages/01.trips/italy-2026-demo/` | Not committed — created at runtime by `demo-load` | — |
|
||
| `user/pages/01.trips/italy-2025/trip.md` | Update title to `Cycling Tuscany 2025` | user |
|
||
| `user/pages/01.trips/italy-2025/01.dailies/dailies.md` | New — missing index page | user |
|
||
| `user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/` | Delete demo story | user |
|
||
| `user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino/` | Delete demo story | user |
|
||
| `user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena/` | Delete demo story | user |
|
||
| `user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/` | Delete 5 demo entries | user |
|
||
| `user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry` + `2026-04-*.entry` | Delete 9 demo entries | user |
|
||
| `user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/` | Delete demo story | user |
|
||
| `user/pages/01.trips/central-asia-2023/` | New permanent trip page tree | user |
|
||
| `user/pages/01.trips/us-canada-mex-2024/` | New permanent trip page tree | user |
|
||
| `Makefile` | Replace `demo-load` and `demo-reset` targets | main |
|
||
| `scripts/pixelfed-import.py` | New one-time import script | main |
|
||
|
||
---
|
||
|
||
## Task 1: Demo reorganisation
|
||
|
||
**Files:**
|
||
- Create: `user/docs/demo/trips/italy-2026-demo/` (copy + edit)
|
||
- Modify: `user/pages/01.trips/italy-2025/trip.md`
|
||
- Create: `user/pages/01.trips/italy-2025/01.dailies/dailies.md`
|
||
- Delete: `user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/`, `02.long-climb-montalcino/`, `03.one-evening-siena/`
|
||
- Delete: `user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/` (5 demo entries)
|
||
- Delete: `user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry` + `2026-04-*.entry` (9 demo entries)
|
||
- Delete: `user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/`
|
||
- Modify: `Makefile`
|
||
|
||
**Interfaces:**
|
||
- Produces: `demo-load` and `demo-reset` targets that only touch `italy-2026-demo`; `italy-2025` is clean and ready for real content; `japan-korea-2026` has only the real `2026-06-17.entry`
|
||
|
||
- [ ] **Step 1: Copy italy-2025 demo source to italy-2026-demo**
|
||
|
||
```bash
|
||
cp -r user/docs/demo/trips/italy-2025 user/docs/demo/trips/italy-2026-demo
|
||
```
|
||
|
||
- [ ] **Step 2: Update the trip.md in the new demo source**
|
||
|
||
Edit `user/docs/demo/trips/italy-2026-demo/trip.md` to read:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Italy 2026 (Demo)'
|
||
template: trip
|
||
date: '2026-09-01'
|
||
date_start: '2026-09-01'
|
||
date_end: '2026-09-08'
|
||
cover_image: ''
|
||
---
|
||
```
|
||
|
||
- [ ] **Step 3: Remove italy-2025 demo stories from pages**
|
||
|
||
```bash
|
||
rm -rf user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn
|
||
rm -rf user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino
|
||
rm -rf user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena
|
||
```
|
||
|
||
- [ ] **Step 4: Remove italy-2025 demo dailies entries from pages**
|
||
|
||
```bash
|
||
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-0800-rolling-through-val-dorcia.entry
|
||
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-1900-siena-at-dusk.entry
|
||
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1200-towers-of-san-gimignano.entry
|
||
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1800-into-florence.entry
|
||
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-08-0900-tyrrhenian-coast.entry
|
||
```
|
||
|
||
Check actual folder names first in case any differ:
|
||
|
||
```bash
|
||
ls user/pages/01.trips/italy-2025/01.dailies/
|
||
```
|
||
|
||
Remove all folders listed (they are all demo content).
|
||
|
||
- [ ] **Step 5: Add missing dailies.md to italy-2025**
|
||
|
||
Create `user/pages/01.trips/italy-2025/01.dailies/dailies.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'The Journey'
|
||
template: dailies
|
||
content:
|
||
items: '@self.children'
|
||
order:
|
||
by: date
|
||
dir: desc
|
||
filter:
|
||
published: true
|
||
---
|
||
```
|
||
|
||
- [ ] **Step 6: Update italy-2025 trip title**
|
||
|
||
Edit `user/pages/01.trips/italy-2025/trip.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Cycling Tuscany 2025'
|
||
template: trip
|
||
date: '2025-10-11'
|
||
date_start: '2025-10-11'
|
||
date_end: '2025-10-16'
|
||
cover_image: ''
|
||
---
|
||
```
|
||
|
||
- [ ] **Step 7: Remove japan demo content from pages**
|
||
|
||
```bash
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-25-1540-wheels-down-narita.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-26-1000-sakura-in-ueno-park.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-27-0715-summit-clouds-and-snow.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-28-1130-thousand-torii-gates.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-29-1400-deer-of-nara.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-30-1800-dotonbori-after-dark.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-31-0730-last-morning-in-arashiyama.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-01-0900-seoul-calling.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-02-1100-gyeongbokgung-and-beyond.entry
|
||
rm -rf user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates
|
||
```
|
||
|
||
- [ ] **Step 8: Update the Makefile demo targets**
|
||
|
||
Replace the entire `demo-load` and `demo-reset` blocks (lines 42–64) with:
|
||
|
||
```makefile
|
||
demo-load:
|
||
# Load italy-2026-demo trip (create pages if absent)
|
||
mkdir -p user/pages/01.trips/italy-2026-demo/01.dailies user/pages/01.trips/italy-2026-demo/02.map user/pages/01.trips/italy-2026-demo/03.stats user/pages/01.trips/italy-2026-demo/04.stories
|
||
cp user/docs/demo/trips/italy-2026-demo/trip.md user/pages/01.trips/italy-2026-demo/trip.md 2>/dev/null || true
|
||
cp user/docs/demo/trips/italy-2026-demo/map.md user/pages/01.trips/italy-2026-demo/02.map/map.md 2>/dev/null || true
|
||
cp user/docs/demo/trips/italy-2026-demo/stats.md user/pages/01.trips/italy-2026-demo/03.stats/stats.md 2>/dev/null || true
|
||
cp user/docs/demo/trips/italy-2026-demo/stories.md user/pages/01.trips/italy-2026-demo/04.stories/stories.md 2>/dev/null || true
|
||
cp -r user/docs/demo/trips/italy-2026-demo/04.stories/. user/pages/01.trips/italy-2026-demo/04.stories/ 2>/dev/null || true
|
||
cp -r user/docs/demo/trips/italy-2026-demo/dailies/. user/pages/01.trips/italy-2026-demo/01.dailies/
|
||
cp user/docs/demo/trips/italy-2026-demo/*.gpx user/pages/01.trips/italy-2026-demo/ 2>/dev/null || true
|
||
docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache"
|
||
|
||
demo-reset:
|
||
rm -rf user/pages/01.trips/italy-2026-demo
|
||
docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache"
|
||
```
|
||
|
||
- [ ] **Step 9: Verify demo-load and demo-reset work**
|
||
|
||
```bash
|
||
make demo-load
|
||
```
|
||
|
||
Expected: `italy-2026-demo` appears at `http://localhost:8081` trips listing. Confirm stories and GPX map load.
|
||
|
||
```bash
|
||
make demo-reset
|
||
```
|
||
|
||
Expected: `italy-2026-demo` disappears from the trips listing. `italy-2025` and `japan-korea-2026` are unaffected.
|
||
|
||
- [ ] **Step 10: Commit user repo changes**
|
||
|
||
Includes the new demo source, cleaned pages, updated italy-2025 title, and new dailies.md:
|
||
|
||
```bash
|
||
git -C user add -A
|
||
git -C user commit -m "chore: move italy demo to italy-2026-demo; clean japan and italy-2025 demo content"
|
||
```
|
||
|
||
- [ ] **Step 11: Commit main repo changes (Makefile only)**
|
||
|
||
```bash
|
||
git add Makefile
|
||
git commit -m "chore: update demo-load/demo-reset for italy-2026-demo; retire japan demo"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 2: Create real trip page trees
|
||
|
||
**Files:**
|
||
- Create: `user/pages/01.trips/central-asia-2023/` (trip.md + 4 subfolders with index pages)
|
||
- Create: `user/pages/01.trips/us-canada-mex-2024/` (trip.md + 4 subfolders with index pages)
|
||
|
||
**Interfaces:**
|
||
- Produces: `central-asia-2023` and `us-canada-mex-2024` trip folder trees with `01.dailies/dailies.md` present — required for the import script to write entries into them
|
||
|
||
- [ ] **Step 1: Create Central Asia 2023 trip tree**
|
||
|
||
```bash
|
||
mkdir -p user/pages/01.trips/central-asia-2023/01.dailies
|
||
mkdir -p user/pages/01.trips/central-asia-2023/02.map
|
||
mkdir -p user/pages/01.trips/central-asia-2023/03.stats
|
||
mkdir -p user/pages/01.trips/central-asia-2023/04.stories
|
||
```
|
||
|
||
Create `user/pages/01.trips/central-asia-2023/trip.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Central Asia 2023'
|
||
template: trip
|
||
date: '2023-08-28'
|
||
date_start: '2023-08-28'
|
||
date_end: '2023-10-18'
|
||
cover_image: ''
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/central-asia-2023/01.dailies/dailies.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'The Journey'
|
||
template: dailies
|
||
content:
|
||
items: '@self.children'
|
||
order:
|
||
by: date
|
||
dir: desc
|
||
filter:
|
||
published: true
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/central-asia-2023/02.map/map.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Trip Map'
|
||
template: map
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/central-asia-2023/03.stats/stats.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Trip Stats'
|
||
template: stats
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/central-asia-2023/04.stories/stories.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: Stories
|
||
template: stories
|
||
published: true
|
||
---
|
||
```
|
||
|
||
- [ ] **Step 2: Create Northern America 2024 trip tree**
|
||
|
||
```bash
|
||
mkdir -p user/pages/01.trips/us-canada-mex-2024/01.dailies
|
||
mkdir -p user/pages/01.trips/us-canada-mex-2024/02.map
|
||
mkdir -p user/pages/01.trips/us-canada-mex-2024/03.stats
|
||
mkdir -p user/pages/01.trips/us-canada-mex-2024/04.stories
|
||
```
|
||
|
||
Create `user/pages/01.trips/us-canada-mex-2024/trip.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Northern America 2024'
|
||
template: trip
|
||
date: '2024-05-28'
|
||
date_start: '2024-05-28'
|
||
date_end: '2024-08-07'
|
||
cover_image: ''
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/us-canada-mex-2024/01.dailies/dailies.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'The Journey'
|
||
template: dailies
|
||
content:
|
||
items: '@self.children'
|
||
order:
|
||
by: date
|
||
dir: desc
|
||
filter:
|
||
published: true
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/us-canada-mex-2024/02.map/map.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Trip Map'
|
||
template: map
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/us-canada-mex-2024/03.stats/stats.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: 'Trip Stats'
|
||
template: stats
|
||
---
|
||
```
|
||
|
||
Create `user/pages/01.trips/us-canada-mex-2024/04.stories/stories.md`:
|
||
|
||
```yaml
|
||
---
|
||
title: Stories
|
||
template: stories
|
||
published: true
|
||
---
|
||
```
|
||
|
||
- [ ] **Step 3: Verify trips appear in the site**
|
||
|
||
Open `http://localhost:8081` — the Past Trips section should list Central Asia 2023 and Northern America 2024.
|
||
|
||
- [ ] **Step 4: Commit to user repo**
|
||
|
||
```bash
|
||
git -C user add user/pages/01.trips/central-asia-2023 user/pages/01.trips/us-canada-mex-2024
|
||
git -C user commit -m "feat: add central-asia-2023 and us-canada-mex-2024 trip page trees"
|
||
```
|
||
|
||
---
|
||
|
||
## Task 3: Pixelfed import script
|
||
|
||
**Files:**
|
||
- Create: `scripts/pixelfed-import.py`
|
||
- Modify: `Makefile` (add `pixelfed-import` target)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `user/pages/01.trips/{trip}/01.dailies/` folders from Task 2 and existing `italy-2025`
|
||
- Produces: `{date}-pixelfed-{N}.entry/` folders with `entry.md` + downloaded photo files
|
||
|
||
- [ ] **Step 1: Write the import script**
|
||
|
||
Create `scripts/pixelfed-import.py`:
|
||
|
||
```python
|
||
#!/usr/bin/env python3
|
||
"""One-time import of Pixelfed statuses into Grav entry pages."""
|
||
|
||
import json
|
||
import os
|
||
import urllib.request
|
||
from datetime import datetime, timezone
|
||
|
||
INPUT_FILE = '/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json'
|
||
USER_PAGES = 'user/pages/01.trips'
|
||
|
||
TRIP_MAP = {
|
||
'2023': 'central-asia-2023',
|
||
'2024': 'us-canada-mex-2024',
|
||
'2025': 'italy-2025',
|
||
}
|
||
|
||
EXT_MAP = {
|
||
'image/jpeg': 'jpg',
|
||
'image/png': 'png',
|
||
'image/gif': 'gif',
|
||
'image/webp': 'webp',
|
||
}
|
||
|
||
ENTRY_TEMPLATE = """\
|
||
---
|
||
title: '{title}'
|
||
date: '{date}'
|
||
template: entry
|
||
published: true
|
||
hero_image: '{hero_image}'
|
||
lat: ''
|
||
lng: ''
|
||
location_city: '{location_city}'
|
||
location_country: '{location_country}'
|
||
weather_temp_c: ''
|
||
weather_desc: ''
|
||
---
|
||
|
||
{body}
|
||
"""
|
||
|
||
|
||
def download(url, dest):
|
||
try:
|
||
urllib.request.urlretrieve(url, dest)
|
||
return True
|
||
except Exception as exc:
|
||
print(f' Warning: download failed {url}: {exc}')
|
||
return False
|
||
|
||
|
||
def main():
|
||
with open(INPUT_FILE) as f:
|
||
posts = json.load(f)
|
||
|
||
counters = {}
|
||
|
||
for post in posts:
|
||
year = post['created_at'][:4]
|
||
trip = TRIP_MAP.get(year)
|
||
if not trip:
|
||
print(f"Skip: no trip mapping for year {year} (post {post['id']})")
|
||
continue
|
||
|
||
counters[trip] = counters.get(trip, 0) + 1
|
||
n = counters[trip]
|
||
|
||
date_str = post['created_at'][:10] # YYYY-MM-DD
|
||
folder = f'{date_str}-pixelfed-{n}.entry'
|
||
path = os.path.join(USER_PAGES, trip, '01.dailies', folder)
|
||
|
||
if os.path.exists(path):
|
||
print(f'Skip: {folder} already exists')
|
||
continue
|
||
|
||
os.makedirs(path)
|
||
print(f'Creating {trip}/{folder}')
|
||
|
||
hero_image = ''
|
||
for i, att in enumerate(post.get('media_attachments', []), 1):
|
||
ext = EXT_MAP.get(att.get('mime', ''), 'jpg')
|
||
filename = f'photo-{i}.{ext}'
|
||
if download(att['url'], os.path.join(path, filename)) and i == 1:
|
||
hero_image = filename
|
||
|
||
place = post.get('place') or {}
|
||
dt = datetime.fromisoformat(post['created_at'].replace('Z', '+00:00'))
|
||
date_fmt = dt.strftime('%Y-%m-%d %H:%M')
|
||
|
||
entry_md = ENTRY_TEMPLATE.format(
|
||
title=f'Pixelfed Import {n}',
|
||
date=date_fmt,
|
||
hero_image=hero_image,
|
||
location_city=place.get('name', ''),
|
||
location_country=place.get('country', ''),
|
||
body=post.get('content_text', '').strip(),
|
||
)
|
||
|
||
with open(os.path.join(path, 'entry.md'), 'w') as f:
|
||
f.write(entry_md)
|
||
|
||
print(f'\nDone. Posts per trip: {counters}')
|
||
|
||
|
||
if __name__ == '__main__':
|
||
main()
|
||
```
|
||
|
||
- [ ] **Step 2: Add make target**
|
||
|
||
In `Makefile`, after the `demo-reset` block, add:
|
||
|
||
```makefile
|
||
pixelfed-import:
|
||
python3 scripts/pixelfed-import.py
|
||
```
|
||
|
||
- [ ] **Step 3: Run the import**
|
||
|
||
```bash
|
||
make pixelfed-import
|
||
```
|
||
|
||
Expected output (approximately):
|
||
|
||
```
|
||
Creating central-asia-2023/2023-08-28-pixelfed-1.entry
|
||
Creating central-asia-2023/2023-08-29-pixelfed-2.entry
|
||
...
|
||
Creating us-canada-mex-2024/2024-05-28-pixelfed-1.entry
|
||
...
|
||
Creating italy-2025/2025-10-11-pixelfed-1.entry
|
||
Creating italy-2025/2025-10-16-pixelfed-2.entry
|
||
|
||
Done. Posts per trip: {'central-asia-2023': 22, 'us-canada-mex-2024': 12, 'italy-2025': 2}
|
||
```
|
||
|
||
- [ ] **Step 4: Verify entries in the browser**
|
||
|
||
Open `http://localhost:8081/trips/central-asia-2023/dailies` — confirm entries appear in reverse-date order with photos.
|
||
|
||
Open one entry (e.g. the first Central Asia post) — confirm the hero image displays and the body text is readable.
|
||
|
||
- [ ] **Step 5: Verify entries in Admin2**
|
||
|
||
Log in at `http://localhost:8081/admin`. Navigate to Pages → find one of the new entry pages. Confirm the media tab shows the downloaded photos.
|
||
|
||
- [ ] **Step 6: Commit main repo (script + Makefile)**
|
||
|
||
```bash
|
||
git add scripts/pixelfed-import.py Makefile
|
||
git commit -m "feat: add pixelfed-import script and make target"
|
||
```
|
||
|
||
- [ ] **Step 7: Commit user repo (imported entries)**
|
||
|
||
```bash
|
||
git -C user add user/pages/01.trips/central-asia-2023/01.dailies
|
||
git -C user add user/pages/01.trips/us-canada-mex-2024/01.dailies
|
||
git -C user add user/pages/01.trips/italy-2025/01.dailies
|
||
git -C user commit -m "feat: import 36 Pixelfed posts into central-asia-2023, us-canada-mex-2024, italy-2025"
|
||
```
|
||
|
||
- [ ] **Step 8: Push to Gitea**
|
||
|
||
```bash
|
||
make content-push
|
||
```
|
||
|
||
Expected: push completes, production webhook fires.
|