Files

T

m038 28008da922 docs: restructure docs/ into guides/ reference/ working/ research/

2026-06-21 12:37:55 +02:00

17 KiB

Raw Blame History

Pixelfed Import & Demo Reorganisation Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Import 36 Pixelfed posts into three permanent trips and reorganise the demo system so Italy demo content moves to a clearly-labelled italy-2026-demo trip and Japan demo content is retired.

Architecture: Three independent tasks — demo cleanup first, then real trip scaffolding, then the Python import script that routes posts by year and downloads photos. All user-facing content lives in user/pages/ and is committed to the user/ git repo; the Makefile and import script are committed to the main repo.

Tech Stack: Bash (file operations), Python 3 stdlib only (json, os, urllib.request, datetime), Grav Flat-File CMS YAML frontmatter.

Global Constraints

Dev server: http://localhost:8081 — must be running (make start) to test cache clears
user/ is a separate git repo — all commits to user/pages/, user/docs/, user/themes/ use git -C user commit; Makefile and scripts/ use the root git commit
Never read .env directly
All new trip pages use template: trip for trip.md, template: dailies for the dailies index, template: map for map, template: stats for stats, template: stories for stories — matching existing trips exactly
Input JSON: /home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json (36 posts)
Trip routing by created_at year: 2023 → central-asia-2023, 2024 → us-canada-mex-2024, 2025 → italy-2025

File Map

File	Change	Repo
`user/docs/demo/trips/italy-2026-demo/`	New — copy of italy-2025 demo source with updated `trip.md`	user
`user/pages/01.trips/italy-2026-demo/`	Not committed — created at runtime by `demo-load`	—
`user/pages/01.trips/italy-2025/trip.md`	Update title to `Cycling Tuscany 2025`	user
`user/pages/01.trips/italy-2025/01.dailies/dailies.md`	New — missing index page	user
`user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/`	Delete demo story	user
`user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino/`	Delete demo story	user
`user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena/`	Delete demo story	user
`user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/`	Delete 5 demo entries	user
`user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-.entry` + `2026-04-.entry`	Delete 9 demo entries	user
`user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/`	Delete demo story	user
`user/pages/01.trips/central-asia-2023/`	New permanent trip page tree	user
`user/pages/01.trips/us-canada-mex-2024/`	New permanent trip page tree	user
`Makefile`	Replace `demo-load` and `demo-reset` targets	main
`scripts/pixelfed-import.py`	New one-time import script	main

Task 1: Demo reorganisation

Files:

Create: user/docs/demo/trips/italy-2026-demo/ (copy + edit)
Modify: user/pages/01.trips/italy-2025/trip.md
Create: user/pages/01.trips/italy-2025/01.dailies/dailies.md
Delete: user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn/, 02.long-climb-montalcino/, 03.one-evening-siena/
Delete: user/pages/01.trips/italy-2025/01.dailies/2025-09-*.entry/ (5 demo entries)
Delete: user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-*.entry + 2026-04-*.entry (9 demo entries)
Delete: user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates/
Modify: Makefile

Interfaces:

Produces: demo-load and demo-reset targets that only touch italy-2026-demo; italy-2025 is clean and ready for real content; japan-korea-2026 has only the real 2026-06-17.entry
Step 1: Copy italy-2025 demo source to italy-2026-demo

cp -r user/docs/demo/trips/italy-2025 user/docs/demo/trips/italy-2026-demo

Step 2: Update the trip.md in the new demo source

Edit user/docs/demo/trips/italy-2026-demo/trip.md to read:

---
title: 'Italy 2026 (Demo)'
template: trip
date: '2026-09-01'
date_start: '2026-09-01'
date_end: '2026-09-08'
cover_image: ''
---

Step 3: Remove italy-2025 demo stories from pages

rm -rf user/pages/01.trips/italy-2025/04.stories/01.val-dorcia-dawn
rm -rf user/pages/01.trips/italy-2025/04.stories/02.long-climb-montalcino
rm -rf user/pages/01.trips/italy-2025/04.stories/03.one-evening-siena

Step 4: Remove italy-2025 demo dailies entries from pages

rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-0800-rolling-through-val-dorcia.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-05-1900-siena-at-dusk.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1200-towers-of-san-gimignano.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-06-1800-into-florence.entry
rm -rf user/pages/01.trips/italy-2025/01.dailies/2025-09-08-0900-tyrrhenian-coast.entry

Check actual folder names first in case any differ:

ls user/pages/01.trips/italy-2025/01.dailies/

Remove all folders listed (they are all demo content).

Step 5: Add missing dailies.md to italy-2025

Create user/pages/01.trips/italy-2025/01.dailies/dailies.md:

---
title: 'The Journey'
template: dailies
content:
    items: '@self.children'
    order:
        by: date
        dir: desc
    filter:
        published: true
---

Step 6: Update italy-2025 trip title

Edit user/pages/01.trips/italy-2025/trip.md:

---
title: 'Cycling Tuscany 2025'
template: trip
date: '2025-10-11'
date_start: '2025-10-11'
date_end: '2025-10-16'
cover_image: ''
---

Step 7: Remove japan demo content from pages

rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-25-1540-wheels-down-narita.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-26-1000-sakura-in-ueno-park.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-27-0715-summit-clouds-and-snow.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-28-1130-thousand-torii-gates.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-29-1400-deer-of-nara.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-30-1800-dotonbori-after-dark.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-03-31-0730-last-morning-in-arashiyama.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-01-0900-seoul-calling.entry
rm -rf user/pages/01.trips/japan-korea-2026/01.dailies/2026-04-02-1100-gyeongbokgung-and-beyond.entry
rm -rf user/pages/01.trips/japan-korea-2026/04.stories/01.the-thousand-gates

Step 8: Update the Makefile demo targets

Replace the entire demo-load and demo-reset blocks (lines 42–64) with:

demo-load:
	# Load italy-2026-demo trip (create pages if absent)
	mkdir -p user/pages/01.trips/italy-2026-demo/01.dailies user/pages/01.trips/italy-2026-demo/02.map user/pages/01.trips/italy-2026-demo/03.stats user/pages/01.trips/italy-2026-demo/04.stories
	cp user/docs/demo/trips/italy-2026-demo/trip.md user/pages/01.trips/italy-2026-demo/trip.md 2>/dev/null || true
	cp user/docs/demo/trips/italy-2026-demo/map.md user/pages/01.trips/italy-2026-demo/02.map/map.md 2>/dev/null || true
	cp user/docs/demo/trips/italy-2026-demo/stats.md user/pages/01.trips/italy-2026-demo/03.stats/stats.md 2>/dev/null || true
	cp user/docs/demo/trips/italy-2026-demo/stories.md user/pages/01.trips/italy-2026-demo/04.stories/stories.md 2>/dev/null || true
	cp -r user/docs/demo/trips/italy-2026-demo/04.stories/. user/pages/01.trips/italy-2026-demo/04.stories/ 2>/dev/null || true
	cp -r user/docs/demo/trips/italy-2026-demo/dailies/. user/pages/01.trips/italy-2026-demo/01.dailies/
	cp user/docs/demo/trips/italy-2026-demo/*.gpx user/pages/01.trips/italy-2026-demo/ 2>/dev/null || true
	docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache"

demo-reset:
	rm -rf user/pages/01.trips/italy-2026-demo
	docker exec intotheeast_grav bash -c "cd /var/www/html && php bin/grav clearcache"

Step 9: Verify demo-load and demo-reset work

make demo-load

Expected: italy-2026-demo appears at http://localhost:8081 trips listing. Confirm stories and GPX map load.

make demo-reset

Expected: italy-2026-demo disappears from the trips listing. italy-2025 and japan-korea-2026 are unaffected.

Step 10: Commit user repo changes

Includes the new demo source, cleaned pages, updated italy-2025 title, and new dailies.md:

git -C user add -A
git -C user commit -m "chore: move italy demo to italy-2026-demo; clean japan and italy-2025 demo content"

Step 11: Commit main repo changes (Makefile only)

git add Makefile
git commit -m "chore: update demo-load/demo-reset for italy-2026-demo; retire japan demo"

Task 2: Create real trip page trees

Files:

Create: user/pages/01.trips/central-asia-2023/ (trip.md + 4 subfolders with index pages)
Create: user/pages/01.trips/us-canada-mex-2024/ (trip.md + 4 subfolders with index pages)

Interfaces:

Produces: central-asia-2023 and us-canada-mex-2024 trip folder trees with 01.dailies/dailies.md present — required for the import script to write entries into them
Step 1: Create Central Asia 2023 trip tree

mkdir -p user/pages/01.trips/central-asia-2023/01.dailies
mkdir -p user/pages/01.trips/central-asia-2023/02.map
mkdir -p user/pages/01.trips/central-asia-2023/03.stats
mkdir -p user/pages/01.trips/central-asia-2023/04.stories

Create user/pages/01.trips/central-asia-2023/trip.md:

---
title: 'Central Asia 2023'
template: trip
date: '2023-08-28'
date_start: '2023-08-28'
date_end: '2023-10-18'
cover_image: ''
---

Create user/pages/01.trips/central-asia-2023/01.dailies/dailies.md:

---
title: 'The Journey'
template: dailies
content:
    items: '@self.children'
    order:
        by: date
        dir: desc
    filter:
        published: true
---

Create user/pages/01.trips/central-asia-2023/02.map/map.md:

---
title: 'Trip Map'
template: map
---

Create user/pages/01.trips/central-asia-2023/03.stats/stats.md:

---
title: 'Trip Stats'
template: stats
---

Create user/pages/01.trips/central-asia-2023/04.stories/stories.md:

---
title: Stories
template: stories
published: true
---

Step 2: Create Northern America 2024 trip tree

mkdir -p user/pages/01.trips/us-canada-mex-2024/01.dailies
mkdir -p user/pages/01.trips/us-canada-mex-2024/02.map
mkdir -p user/pages/01.trips/us-canada-mex-2024/03.stats
mkdir -p user/pages/01.trips/us-canada-mex-2024/04.stories

Create user/pages/01.trips/us-canada-mex-2024/trip.md:

---
title: 'Northern America 2024'
template: trip
date: '2024-05-28'
date_start: '2024-05-28'
date_end: '2024-08-07'
cover_image: ''
---

Create user/pages/01.trips/us-canada-mex-2024/01.dailies/dailies.md:

---
title: 'The Journey'
template: dailies
content:
    items: '@self.children'
    order:
        by: date
        dir: desc
    filter:
        published: true
---

Create user/pages/01.trips/us-canada-mex-2024/02.map/map.md:

---
title: 'Trip Map'
template: map
---

Create user/pages/01.trips/us-canada-mex-2024/03.stats/stats.md:

---
title: 'Trip Stats'
template: stats
---

Create user/pages/01.trips/us-canada-mex-2024/04.stories/stories.md:

---
title: Stories
template: stories
published: true
---

Step 3: Verify trips appear in the site

Open http://localhost:8081 — the Past Trips section should list Central Asia 2023 and Northern America 2024.

Step 4: Commit to user repo

git -C user add user/pages/01.trips/central-asia-2023 user/pages/01.trips/us-canada-mex-2024
git -C user commit -m "feat: add central-asia-2023 and us-canada-mex-2024 trip page trees"

Task 3: Pixelfed import script

Files:

Create: scripts/pixelfed-import.py
Modify: Makefile (add pixelfed-import target)

Interfaces:

Consumes: user/pages/01.trips/{trip}/01.dailies/ folders from Task 2 and existing italy-2025
Produces: {date}-pixelfed-{N}.entry/ folders with entry.md + downloaded photo files
Step 1: Write the import script

Create scripts/pixelfed-import.py:

#!/usr/bin/env python3
"""One-time import of Pixelfed statuses into Grav entry pages."""

import json
import os
import urllib.request
from datetime import datetime, timezone

INPUT_FILE = '/home/mischa/Nextcloud/Downloads/pixelfed/pixelfed-statuses.json'
USER_PAGES = 'user/pages/01.trips'

TRIP_MAP = {
    '2023': 'central-asia-2023',
    '2024': 'us-canada-mex-2024',
    '2025': 'italy-2025',
}

EXT_MAP = {
    'image/jpeg': 'jpg',
    'image/png':  'png',
    'image/gif':  'gif',
    'image/webp': 'webp',
}

ENTRY_TEMPLATE = """\
---
title: '{title}'
date: '{date}'
template: entry
published: true
hero_image: '{hero_image}'
lat: ''
lng: ''
location_city: '{location_city}'
location_country: '{location_country}'
weather_temp_c: ''
weather_desc: ''
---

{body}
"""


def download(url, dest):
    try:
        urllib.request.urlretrieve(url, dest)
        return True
    except Exception as exc:
        print(f'  Warning: download failed {url}: {exc}')
        return False


def main():
    with open(INPUT_FILE) as f:
        posts = json.load(f)

    counters = {}

    for post in posts:
        year = post['created_at'][:4]
        trip = TRIP_MAP.get(year)
        if not trip:
            print(f"Skip: no trip mapping for year {year} (post {post['id']})")
            continue

        counters[trip] = counters.get(trip, 0) + 1
        n = counters[trip]

        date_str = post['created_at'][:10]  # YYYY-MM-DD
        folder = f'{date_str}-pixelfed-{n}.entry'
        path = os.path.join(USER_PAGES, trip, '01.dailies', folder)

        if os.path.exists(path):
            print(f'Skip: {folder} already exists')
            continue

        os.makedirs(path)
        print(f'Creating {trip}/{folder}')

        hero_image = ''
        for i, att in enumerate(post.get('media_attachments', []), 1):
            ext = EXT_MAP.get(att.get('mime', ''), 'jpg')
            filename = f'photo-{i}.{ext}'
            if download(att['url'], os.path.join(path, filename)) and i == 1:
                hero_image = filename

        place = post.get('place') or {}
        dt = datetime.fromisoformat(post['created_at'].replace('Z', '+00:00'))
        date_fmt = dt.strftime('%Y-%m-%d %H:%M')

        entry_md = ENTRY_TEMPLATE.format(
            title=f'Pixelfed Import {n}',
            date=date_fmt,
            hero_image=hero_image,
            location_city=place.get('name', ''),
            location_country=place.get('country', ''),
            body=post.get('content_text', '').strip(),
        )

        with open(os.path.join(path, 'entry.md'), 'w') as f:
            f.write(entry_md)

    print(f'\nDone. Posts per trip: {counters}')


if __name__ == '__main__':
    main()

Step 2: Add make target

In Makefile, after the demo-reset block, add:

pixelfed-import:
	python3 scripts/pixelfed-import.py

Step 3: Run the import

make pixelfed-import

Expected output (approximately):

Creating central-asia-2023/2023-08-28-pixelfed-1.entry
Creating central-asia-2023/2023-08-29-pixelfed-2.entry
...
Creating us-canada-mex-2024/2024-05-28-pixelfed-1.entry
...
Creating italy-2025/2025-10-11-pixelfed-1.entry
Creating italy-2025/2025-10-16-pixelfed-2.entry

Done. Posts per trip: {'central-asia-2023': 22, 'us-canada-mex-2024': 12, 'italy-2025': 2}

Step 4: Verify entries in the browser

Open http://localhost:8081/trips/central-asia-2023/dailies — confirm entries appear in reverse-date order with photos.

Open one entry (e.g. the first Central Asia post) — confirm the hero image displays and the body text is readable.

Step 5: Verify entries in Admin2

Log in at http://localhost:8081/admin. Navigate to Pages → find one of the new entry pages. Confirm the media tab shows the downloaded photos.

Step 6: Commit main repo (script + Makefile)

git add scripts/pixelfed-import.py Makefile
git commit -m "feat: add pixelfed-import script and make target"

Step 7: Commit user repo (imported entries)

git -C user add user/pages/01.trips/central-asia-2023/01.dailies
git -C user add user/pages/01.trips/us-canada-mex-2024/01.dailies
git -C user add user/pages/01.trips/italy-2025/01.dailies
git -C user commit -m "feat: import 36 Pixelfed posts into central-asia-2023, us-canada-mex-2024, italy-2025"

Step 8: Push to Gitea

make content-push

Expected: push completes, production webhook fires.

17 KiB Raw Blame History Unescape Escape

Pixelfed Import & Demo Reorganisation Implementation Plan

Global Constraints

File Map

Task 1: Demo reorganisation

Task 2: Create real trip page trees

Task 3: Pixelfed import script

17 KiB

Raw Blame History