docs(claude): rewrite CLAUDE.md as agent working guide

Rewrote from scratch based on a full read of README.md. Focus: actionable rules and pitfalls for the AI agent, not human overview. Covers: patch pattern + rules, two-jail architecture, internal URL rewriting, nginx/Fedify requirements, MongoDB collections, post-type discovery table, three compose paths and their checkbox mechanics, ap_timeline insertion timing, status ID format, JF2→AS2 mapping, visibility mapping, fork update commands, debugging starting points (12 symptoms), and full env var reference. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 08:55:31 +02:00
parent 5e50d7aceb
commit 1628b47cc8
1 changed files with 126 additions and 84 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1,124 +1,160 @@
 # CLAUDE.md — indiekit-blog

 Personal [Indiekit](https://getindiekit.com/) deployment for [blog.giersig.eu](https://blog.giersig.eu).
-Built on the [rmdes/indiekit](https://github.com/rmdes/indiekit) fork ecosystem with a set of
-patch scripts that fix issues that cannot be upstreamed.

-## Memory files
+## Always read memory files first

-Detailed architecture and lessons learned live in `memory/`:
+Before investigating or modifying anything:

- [`memory/project_activitypub.md`](memory/project_activitypub.md) — AP data flows, own-post timeline, compose paths, all AP patches
- [`memory/project_architecture.md`](memory/project_architecture.md) — FreeBSD jails, nginx, MongoDB collections, actor URLs, patch infrastructure
- [`memory/feedback_patches.md`](memory/feedback_patches.md) — Patch authoring patterns, known fragile points, compose-path gotchas
-
-**Always read the relevant memory file before investigating or modifying anything in this repo.**
+| File | When to read |
+|---|---|
+| [`memory/project_activitypub.md`](memory/project_activitypub.md) | Any AP / fediverse / reply threading work |
+| [`memory/project_architecture.md`](memory/project_architecture.md) | Server layout, MongoDB, nginx, internal URLs |
+| [`memory/feedback_patches.md`](memory/feedback_patches.md) | Writing or debugging patch scripts |

 ---

 ## Running

 ```sh
-npm run serve          # apply all patches then start Indiekit
-npm run postinstall    # re-apply patches after npm install (postinstall runs automatically)
+npm run serve          # preflights + all patches + start Indiekit
+npm run postinstall    # re-apply patches after npm install
 ```

-Do **not** restart with `node` directly — patches must run first.
+Never start with `node` directly — patches must run first.

 ---

 ## Patch system

-All fixes to `node_modules` live in `scripts/patch-*.mjs`.
-Both `postinstall` and `serve` in `package.json` run them in order.
+All node_modules fixes live in `scripts/patch-*.mjs`. Both `postinstall` and `serve` run them in order.

-### Writing a patch
+### Pattern

 ```js
-const MARKER = "// [patch] my-patch-name";   // idempotency check
-const OLD_SNIPPET = `exact source text`;
-const NEW_SNIPPET = `replacement ${MARKER}`;
+const MARKER = "// [patch] my-patch-name";
+const OLD_SNIPPET = `exact source text (spaces not tabs, exact line endings)`;
+const NEW_SNIPPET = `replacement text ${MARKER}`;

-// 1. Skip if MARKER already present
+// 1. Read file — skip if MARKER already present
 // 2. Warn if OLD_SNIPPET not found (upstream changed)
 // 3. Replace + writeFile
 ```

- Match node_modules text **byte-for-byte** (spaces, not tabs; exact line endings).
- Always include both candidate paths: `node_modules/@rmdes/...` and
-  `node_modules/@indiekit/indiekit/node_modules/@rmdes/...`.
- Template literals in patch strings: escape `` \` `` and `\${}`.
- Append new AP patches **after** `patch-ap-federation-bridge-base-url` in both
-  `postinstall` and `serve` in `package.json`.
- Some patches are `serve`-only (not `postinstall`) — check both when adding microsub patches.
+### Rules
+
+- Include **both** candidate paths: `node_modules/@rmdes/...` and `node_modules/@indiekit/indiekit/node_modules/@rmdes/...`
+- Escape template literals: `` \` `` and `\${}`
+- Append new AP patches **after** `patch-ap-federation-bridge-base-url` in both `postinstall` and `serve`
+- `patch-microsub-reader-ap-dispatch` is `serve`-only — check both scripts for microsub patches
+- After writing a patch script, run it immediately (`node scripts/patch-*.mjs`) to verify it applies cleanly

 ---

-## ActivityPub actor
+## Architecture — things that affect code

-| Field | Value |
+### Two-jail setup
+
+```
+Internet → nginx (web jail 10.100.0.10) → Indiekit (node jail 10.100.0.20:3000)
+```
+
+The node jail **cannot reach its own public HTTPS URL**. Internal self-fetches must use `INTERNAL_FETCH_URL=http://10.100.0.20:3000` directly. All such fetches go through `_toInternalUrl()` (injected by `patch-micropub-fetch-internal-url`).
+
+### nginx / Fedify
+
+nginx must forward `Host: blog.giersig.eu` and `X-Forwarded-Proto: https` or AP lookups 302-redirect to the login page. See `patch-ap-federation-bridge-base-url`.
+
+`createFederation()` requires `allowPrivateAddress: true` (blog resolves to a LAN IP) and `signatureTimeWindow: { hours: 12 }` (Mastodon retries with old signatures).
+
+### MongoDB collections
+
+| Collection | Contents |
 |---|---|
-| Handle | `svemagie` (env `AP_HANDLE`) |
-| Full handle | `@svemagie@blog.giersig.eu` |
-| Actor URL | `https://blog.giersig.eu/activitypub/users/svemagie` |
-| Mastodon migration alias | `@svemagie@troet.cafe` (`AP_ALSO_KNOWN_AS`) |
-
-AP syndicator `info.uid = publicationUrl` (`https://blog.giersig.eu/`), `info.checked = true`.
+| `posts` | Micropub post data — `properties.url` is the lookup key |
+| `ap_timeline` | AP posts (inbound + outbound) — keyed by `uid` |
+| `ap_notifications` | Mentions, replies, likes, boosts |
+| `ap_followers` / `ap_following` | Actor URLs |
+| `ap_activities` | Outbound/inbound activity log |
+| `ap_profile` | Own actor (name, icon, url) |
+| `ap_interactions` | Own likes/boosts |
+| `ap_keys` | RSA + Ed25519 key pairs |
+| `ap_featured` | Pinned posts |

 ---

 ## Post type discovery

-`getPostType(postTypes, properties)` in `post-type-discovery.js`:
+`getPostType(postTypes, properties)` checks **key presence only** — value doesn't matter:

-| Property present | Type |
-|---|---|
-| `in-reply-to` | `reply` → `/replies/{slug}/` |
-| `like-of` | `like` → `/likes/{slug}/` |
-| `repost-of` | `repost` → `/reposts/{slug}/` |
-| `photo` | `photo` → `/photos/{slug}/` |
-| _(none)_ | `note` → `/notes/{slug}/` |
+| Key present | Type | Saved at |
+|---|---|---|
+| `in-reply-to` | `reply` | `/replies/{slug}/` |
+| `like-of` | `like` | `/likes/{slug}/` |
+| `repost-of` | `repost` | `/reposts/{slug}/` |
+| `photo` | `photo` | `/photos/{slug}/` |
+| _(none)_ | `note` | `/notes/{slug}/` |

-Only KEY presence matters, not value. If `in-reply-to` is silently missing from the JF2
-object, the post becomes `"note"` with no error — this is a common threading bug root cause.
+If `in-reply-to` is silently absent, the post becomes a note **with no error**. This is the most common threading bug root cause.

 ---

-## Reply threading — two compose paths
+## Reply threading — compose paths

-| Path | How AP target is pre-checked |
+Three paths, different syndication mechanics:
+
+| Path | AP checkbox mechanism |
 |---|---|
-| AP reader `/activitypub/admin/reader/compose` | `target.defaultChecked = target.checked === true` (patched) |
-| Microsub reader `/microsub/admin/reader/compose` | `target.checked` from Micropub `q=config` (already `true` in config) |
-| Mastodon client API `POST /api/v1/statuses` | `mp-syndicate-to` hardcoded to `publicationUrl` (always AP) |
+| AP reader `/activitypub/admin/reader/compose` | `target.defaultChecked = target.checked === true` *(patched by `patch-ap-compose-default-checked`)* |
+| Microsub reader `/microsub/admin/reader/compose` | `target.checked` from Micropub `q=config` — already `true` for AP syndicator |
+| Mastodon client API `POST /api/v1/statuses` | `mp-syndicate-to` hardcoded to `publicationUrl` — always AP |

-The AP reader template uses `target.defaultChecked`, NOT `target.checked` — these are different.
-`patch-ap-compose-default-checked` maps the config's `checked: true` through to `defaultChecked`.
+The AP reader template uses `target.defaultChecked`, **not** `target.checked`. These are different fields.

-### Mastodon API reply-to-reply (patched)
+### ap_timeline insertion timing

-`POST /api/v1/statuses` resolves `in_reply_to_id` via `findTimelineItemById(ap_timeline, id)`.
-Own posts were previously not inserted into `ap_timeline` until the Eleventy build webhook fired
-(30–120 s). `patch-ap-mastodon-reply-threading` inserts a provisional item immediately after
-`postContent.create()` using `addTimelineItem()` (`$setOnInsert` — idempotent).
+Own posts reach `ap_timeline` via two paths:
+- **Mastodon API**: inserted immediately after `postContent.create()` *(patched by `patch-ap-mastodon-reply-threading`)*
+- **Micropub + syndication webhook**: inserted by syndicator after Eleventy build (30–120 s)
+
+Any new code path that creates posts should insert to `ap_timeline` immediately — otherwise `in_reply_to_id` lookups fail during the build window.
+
+### Status ID format
+
+`encodeCursor(published)` = ms-since-epoch string. `findTimelineItemById` resolves this with a ±1 s range query using MongoDB `$dateFromString` to handle TZ-offset ISO strings.

 ---

-## ap_timeline
+## ActivityPub syndicator

-Key collection. Documents keyed by `uid`.
+`syndicator.syndicate(properties)` does **not** filter by post type. A note and a reply both become `Create(Note)`. The difference is whether `inReplyTo` is set (from `properties["in-reply-to"]`).

- **Inbound posts** — inserted by `inbox-handlers.js` on Create/Update/Announce activities
- **Own posts** — inserted by `syndicator.js` after delivery (after build) AND now immediately
-  by `POST /api/v1/statuses` (via patch)
- **Status IDs** — `encodeCursor(published)` = ms-since-epoch string; `findTimelineItemById`
-  resolves them with a ±1 s MongoDB `$dateFromString` range query to handle TZ-offset strings
+JF2 → AS2 mapping:
+
+| Post type | Activity | Notes |
+|---|---|---|
+| `note` / `reply` | `Create(Note)` | reply has `inReplyTo` |
+| `like` | `Create(Note)` | bookmark framing (🔖 emoji) |
+| `repost` | `Announce` | |
+| `article` | `Create(Article)` | has `name` |
+
+Visibility:
+
+| Value | `to` | `cc` |
+|---|---|---|
+| `public` | `as:Public` | followers |
+| `unlisted` | followers | `as:Public` |
+| `followers` | followers | — |

 ---

 ## Fork dependencies

-Four packages come from GitHub forks (not npm):
+```sh
+# Pull latest commit from a fork:
+npm install github:svemagie/<package-name>
+npm install github:svemagie/indiekit-endpoint-activitypub
+```

 | Package | Fork |
 |---|---|
@@ -127,31 +163,37 @@ Four packages come from GitHub forks (not npm):
 | `@rmdes/indiekit-endpoint-microsub` | `github:svemagie/indiekit-endpoint-microsub#bookmarks-import` |
 | `@rmdes/indiekit-endpoint-youtube` | `github:svemagie/indiekit-endpoint-youtube` |

-To pull latest fork commits: `npm install github:svemagie/<package-name>`.
+---
+
+## Debugging — starting points
+
+| Symptom | First check |
+|---|---|
+| Reply created as "note" not "reply" | Is `in-reply-to` in the Micropub request? Check: form hidden field, `submitComposeController`, `findTimelineItemById` return value, `formEncodedToJf2` |
+| Reply not federated to AP | Is `mp-syndicate-to` set? Check `target.defaultChecked` / `target.checked`, `getSyndicateToProperty` in `jf2.js` |
+| AP lookup returns 302 / auth redirect | nginx not forwarding `Host`/`X-Forwarded-Proto` — see `patch-ap-federation-bridge-base-url` |
+| `findTimelineItemById` returns null | Item not yet in `ap_timeline` (build not finished) or TZ-offset date mismatch — `$dateFromString` range query should catch offsets |
+| Favourite/reblog hangs in Mastodon client | `resolveAuthor` timeout — `Promise.race` 5 s cap should prevent this |
+| "Empty reply from server" on webmention poller | Poller routing through nginx (returns 444 for wrong Host) — must use `INDIEKIT_DIRECT_URL` |
+| HTTP Signature verify errors flooding logs | Expected for deleted/migrated actors — suppressed to `fatal` level in federation-setup.js |
+| "OAuth callback failed. Missing parameters." | `state` parameter not echoed — fixed in fork (`b54146c`) |
+| AP object 410 / Tombstone | Post was deleted — correct, served by FEP-4f05 |

 ---

-## Environment variables (key ones)
+## Environment variables

 | Var | Purpose |
 |---|---|
-| `AP_HANDLE` | ActivityPub handle (default: `svemagie` from `GITHUB_USERNAME`) |
-| `PUBLICATION_URL` | Canonical blog URL (`https://blog.giersig.eu`) |
-| `INDIEKIT_URL` | Application URL (same as publication URL here) |
+| `AP_HANDLE` | AP handle (`svemagie`) |
+| `AP_ALSO_KNOWN_AS` | Migration alias (`https://troet.cafe/users/svemagie`) |
+| `AP_LOG_LEVEL` | Fedify log level (`info` default; `debug` for delivery tracing) |
+| `AP_DEBUG` | `1` to enable Fedify debug dashboard at `/activitypub/__debug__/` |
+| `PUBLICATION_URL` | Canonical blog URL |
+| `INDIEKIT_URL` | Application URL (same as publication URL) |
+| `INTERNAL_FETCH_URL` | Direct node jail URL for self-fetches (`http://10.100.0.20:3000`) |
+| `INDIEKIT_BIND_HOST` | Jail IP for webmention poller direct connect |
+| `REDIS_URL` | Redis for AP message queue + KV (production; without this, queue lost on restart) |
 | `MONGO_HOST` / `MONGO_URL` | MongoDB connection |
-| `REDIS_URL` | Redis for AP message queue and KV store (production) |
-| `AP_ALSO_KNOWN_AS` | Migration alias (old Mastodon handle) |
 | `GH_CONTENT_TOKEN` | GitHub token for writing posts to the `blog` repo |
-
---
-
-## Common debugging starting points
-
- **Post created as "note" instead of "reply"** → `in-reply-to` missing from JF2. Check: form
-  hidden field, `submitComposeController`, Mastodon API `findTimelineItemById`, `formEncodedToJf2`.
- **Reply not federated to AP** → `mp-syndicate-to` not set. Check: `target.defaultChecked` /
-  `target.checked` in compose form, `getSyndicateToProperty` in `jf2.js`.
- **AP object lookup 302 redirect** → nginx not forwarding `Host` / `X-Forwarded-Proto`; see
-  `patch-ap-federation-bridge-base-url`.
- **Timeline item not found by Mastodon client** → `findTimelineItemById` date mismatch; stored
-  date is TZ-offset string, lookup decodes cursor to UTC ISO — relies on `$dateFromString` range.
+| `SECRET` | JWT signing secret (webmention poller auth) |