Charlie Root
373e0c4be8
All checks were successful
Deploy Indiekit Server / deploy (push) Successful in 1m16s
276 lines
11 KiB
Markdown
276 lines
11 KiB
Markdown
# CLAUDE.md — indiekit-blog
|
||
|
||
Personal [Indiekit](https://getindiekit.com/) deployment for [blog.giersig.eu](https://blog.giersig.eu).
|
||
|
||
## Always read memory files first
|
||
|
||
Before investigating or modifying anything:
|
||
|
||
| File | When to read |
|
||
|---|---|
|
||
| [`memory/project_activitypub.md`](memory/project_activitypub.md) | Any AP / fediverse / reply threading work |
|
||
| [`memory/project_architecture.md`](memory/project_architecture.md) | Server layout, MongoDB, nginx, internal URLs |
|
||
| [`memory/feedback_patches.md`](memory/feedback_patches.md) | Writing or debugging patch scripts |
|
||
|
||
---
|
||
|
||
## Running
|
||
|
||
```sh
|
||
npm run serve # preflights + all patches + start Indiekit
|
||
npm run postinstall # re-apply patches after npm install
|
||
```
|
||
|
||
Never start with `node` directly — patches must run first.
|
||
|
||
---
|
||
|
||
## Patch system
|
||
|
||
All node_modules fixes live in `scripts/patch-*.mjs`. Both `postinstall` and `serve` run them in order.
|
||
|
||
### Pattern
|
||
|
||
```js
|
||
const MARKER = "// [patch] my-patch-name";
|
||
const OLD_SNIPPET = `exact source text (spaces not tabs, exact line endings)`;
|
||
const NEW_SNIPPET = `replacement text ${MARKER}`;
|
||
|
||
// 1. Read file — skip if MARKER already present
|
||
// 2. Warn if OLD_SNIPPET not found (upstream changed)
|
||
// 3. Replace + writeFile
|
||
```
|
||
|
||
### Rules
|
||
|
||
- Include **both** candidate paths: `node_modules/@rmdes/...` and `node_modules/@indiekit/indiekit/node_modules/@rmdes/...`
|
||
- Escape template literals: `` \` `` and `\${}`
|
||
- Append new AP patches **after** `patch-ap-federation-bridge-base-url` in both `postinstall` and `serve`
|
||
- `patch-microsub-reader-ap-dispatch` is `serve`-only — check both scripts for microsub patches
|
||
- After writing a patch script, run it immediately (`node scripts/patch-*.mjs`) to verify it applies cleanly
|
||
|
||
---
|
||
|
||
## Architecture — things that affect code
|
||
|
||
### Two-jail setup
|
||
|
||
```
|
||
Internet → nginx (web jail 10.100.0.10) → Indiekit (node jail 10.100.0.20:3000)
|
||
```
|
||
|
||
The node jail **cannot reach its own public HTTPS URL**. Internal self-fetches must use `INTERNAL_FETCH_URL=http://10.100.0.20:3000` directly. All such fetches go through `_toInternalUrl()` (injected by `patch-micropub-fetch-internal-url`).
|
||
|
||
### nginx / Fedify
|
||
|
||
nginx must forward `Host: blog.giersig.eu` and `X-Forwarded-Proto: https` or AP lookups 302-redirect to the login page. See `patch-ap-federation-bridge-base-url`.
|
||
|
||
`createFederation()` requires `allowPrivateAddress: true` (blog resolves to a LAN IP) and `signatureTimeWindow: { hours: 12 }` (Mastodon retries with old signatures).
|
||
|
||
### MongoDB collections
|
||
|
||
| Collection | Contents |
|
||
|---|---|
|
||
| `posts` | Micropub post data — `properties.url` is the lookup key |
|
||
| `ap_timeline` | AP posts (inbound + outbound) — keyed by `uid` |
|
||
| `ap_notifications` | Mentions, replies, likes, boosts |
|
||
| `ap_followers` / `ap_following` | Actor URLs |
|
||
| `ap_activities` | Outbound/inbound activity log |
|
||
| `ap_profile` | Own actor (name, icon, url) |
|
||
| `ap_interactions` | Own likes/boosts |
|
||
| `ap_keys` | RSA + Ed25519 key pairs |
|
||
| `ap_featured` | Pinned posts |
|
||
|
||
---
|
||
|
||
## Post type discovery
|
||
|
||
`getPostType(postTypes, properties)` checks **key presence only** — value doesn't matter:
|
||
|
||
| Key present | Type | Saved at |
|
||
|---|---|---|
|
||
| `in-reply-to` | `reply` | `/replies/{slug}/` |
|
||
| `like-of` | `like` | `/likes/{slug}/` |
|
||
| `repost-of` | `repost` | `/reposts/{slug}/` |
|
||
| `photo` | `photo` | `/photos/{slug}/` |
|
||
| _(none)_ | `note` | `/notes/{slug}/` |
|
||
|
||
If `in-reply-to` is silently absent, the post becomes a note **with no error**. This is the most common threading bug root cause.
|
||
|
||
---
|
||
|
||
## Reply threading — compose paths
|
||
|
||
Three paths, different syndication mechanics:
|
||
|
||
| Path | AP checkbox mechanism |
|
||
|---|---|
|
||
| AP reader `/activitypub/admin/reader/compose` | `target.defaultChecked = target.checked === true` *(patched by `patch-ap-compose-default-checked`)* |
|
||
| Microsub reader `/microsub/admin/reader/compose` | `target.checked` from Micropub `q=config` — already `true` for AP syndicator |
|
||
| Mastodon client API `POST /api/v1/statuses` | `mp-syndicate-to` hardcoded to `publicationUrl` — always AP |
|
||
|
||
The AP reader template uses `target.defaultChecked`, **not** `target.checked`. These are different fields.
|
||
|
||
### ap_timeline insertion timing
|
||
|
||
Own posts reach `ap_timeline` via two paths:
|
||
- **Mastodon API**: inserted immediately after `postContent.create()` *(patched by `patch-ap-mastodon-reply-threading`)*
|
||
- **Micropub + syndication webhook**: inserted by syndicator after Eleventy build (30–120 s)
|
||
|
||
Any new code path that creates posts should insert to `ap_timeline` immediately — otherwise `in_reply_to_id` lookups fail during the build window.
|
||
|
||
### Status ID format
|
||
|
||
`encodeCursor(published)` = ms-since-epoch string. `findTimelineItemById` resolves this with a ±1 s range query using MongoDB `$dateFromString` to handle TZ-offset ISO strings.
|
||
|
||
---
|
||
|
||
## ActivityPub syndicator
|
||
|
||
`syndicator.syndicate(properties)` does **not** filter by post type. A note and a reply both become `Create(Note)`. The difference is whether `inReplyTo` is set (from `properties["in-reply-to"]`).
|
||
|
||
JF2 → AS2 mapping:
|
||
|
||
| Post type | Activity | Notes |
|
||
|---|---|---|
|
||
| `note` / `reply` | `Create(Note)` | reply has `inReplyTo` |
|
||
| `like` | `Create(Note)` | bookmark framing (🔖 emoji) |
|
||
| `repost` | `Announce` | |
|
||
| `article` | `Create(Article)` | has `name` |
|
||
|
||
Visibility:
|
||
|
||
| Value | `to` | `cc` |
|
||
|---|---|---|
|
||
| `public` | `as:Public` | followers |
|
||
| `unlisted` | followers | `as:Public` |
|
||
| `followers` | followers | — |
|
||
|
||
---
|
||
|
||
## Fork dependencies
|
||
|
||
```sh
|
||
# Pull latest commit from a fork:
|
||
npm install git+https://gitea.giersig.eu/svemagie/<package-name>
|
||
npm install git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-activitypub
|
||
```
|
||
|
||
| Package | Fork |
|
||
|---|---|
|
||
| `@rmdes/indiekit-endpoint-activitypub` | `git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-activitypub` |
|
||
| `@rmdes/indiekit-endpoint-blogroll` | `git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-blogroll` |
|
||
| `@rmdes/indiekit-endpoint-microsub` | `git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-microsub` |
|
||
| `@rmdes/indiekit-endpoint-youtube` | `git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-youtube` |
|
||
|
||
---
|
||
|
||
## Debugging — starting points
|
||
|
||
| Symptom | First check |
|
||
|---|---|
|
||
| Reply created as "note" not "reply" | Is `in-reply-to` in the Micropub request? Check: form hidden field, `submitComposeController`, `findTimelineItemById` return value, `formEncodedToJf2` |
|
||
| Reply not federated to AP | Is `mp-syndicate-to` set? Check `target.defaultChecked` / `target.checked`, `getSyndicateToProperty` in `jf2.js` |
|
||
| AP lookup returns 302 / auth redirect | nginx not forwarding `Host`/`X-Forwarded-Proto` — see `patch-ap-federation-bridge-base-url` |
|
||
| `findTimelineItemById` returns null | Item not yet in `ap_timeline` (build not finished) or TZ-offset date mismatch — `$dateFromString` range query should catch offsets |
|
||
| Favourite/reblog hangs in Mastodon client | `resolveAuthor` timeout — `Promise.race` 5 s cap should prevent this |
|
||
| "Empty reply from server" on webmention poller | Poller routing through nginx (returns 444 for wrong Host) — must use `INDIEKIT_DIRECT_URL` |
|
||
| HTTP Signature verify errors flooding logs | Expected for deleted/migrated actors — suppressed to `fatal` level in federation-setup.js |
|
||
| "OAuth callback failed. Missing parameters." | `state` parameter not echoed — fixed in fork (`b54146c`) |
|
||
| AP object 410 / Tombstone | Post was deleted — correct, served by FEP-4f05 |
|
||
|
||
---
|
||
|
||
## Environment variables
|
||
|
||
| Var | Purpose |
|
||
|---|---|
|
||
| `AP_HANDLE` | AP handle (`svemagie`) |
|
||
| `AP_ALSO_KNOWN_AS` | Migration alias (`https://troet.cafe/users/svemagie`) |
|
||
| `AP_LOG_LEVEL` | Fedify log level (`info` default; `debug` for delivery tracing) |
|
||
| `AP_DEBUG` | `1` to enable Fedify debug dashboard at `/activitypub/__debug__/` |
|
||
| `PUBLICATION_URL` | Canonical blog URL |
|
||
| `INDIEKIT_URL` | Application URL (same as publication URL) |
|
||
| `INTERNAL_FETCH_URL` | Direct node jail URL for self-fetches (`http://10.100.0.20:3000`) |
|
||
| `INDIEKIT_BIND_HOST` | Jail IP for webmention poller direct connect |
|
||
| `REDIS_URL` | Redis for AP message queue + KV (production; without this, queue lost on restart) |
|
||
| `MONGO_HOST` / `MONGO_URL` | MongoDB connection |
|
||
| `GH_CONTENT_TOKEN` | Gitea PAT for writing posts to the indiekit-blog repo |
|
||
| `SECRET` | JWT signing secret (webmention poller auth) |
|
||
|
||
---
|
||
|
||
## Content store (Gitea)
|
||
|
||
`@indiekit/store-github` is pointed at the self-hosted Gitea instance instead of GitHub. Key config in `indiekit.config.mjs`:
|
||
|
||
```js
|
||
"@indiekit/store-github": {
|
||
baseUrl: giteaBaseUrl, // GITEA_BASE_URL from .env
|
||
user: process.env.GITEA_CONTENT_USER,
|
||
repo: process.env.GITEA_CONTENT_REPO,
|
||
branch: "main",
|
||
token: githubContentToken, // GH_CONTENT_TOKEN from .env
|
||
}
|
||
```
|
||
|
||
**`GITEA_BASE_URL`** must end with a trailing slash: `http://10.100.0.90:3000/api/v1/`
|
||
Without it, `new URL(apiPath, baseUrl)` silently strips the `v1` segment → 404 on all writes.
|
||
|
||
**`GH_CONTENT_TOKEN`** — the Gitea PAT for `svemagie`. `start.sh` rejects startup if neither `GH_CONTENT_TOKEN` nor `GITHUB_TOKEN` is present. The token must have repo read/write scope on `giersig.eu/indiekit-blog`.
|
||
|
||
**`GITEA_CONTENT_USER`** = `giersig.eu` (the org, not the personal username)
|
||
**`GITEA_CONTENT_REPO`** = `indiekit-blog`
|
||
|
||
---
|
||
|
||
## Micropub → Gitea build dispatch
|
||
|
||
Gitea Contents API commits (what `store-github` does) do **not** trigger `on: push` CI workflows. `patch-micropub-gitea-dispatch.mjs` patches the Micropub endpoint to fire a `workflow_dispatch` event to `giersig.eu/indiekit-blog` after each create/update, so the blog rebuilds immediately after a post is published.
|
||
|
||
---
|
||
|
||
## Pushing changes from the server
|
||
|
||
The node jail shell is tcsh, which mangles multi-line `echo`/`printf` and inline heredocs. To push file changes to Gitea from the server, use a Python script:
|
||
|
||
```python
|
||
python3 << 'PYEOF'
|
||
import urllib.request, json, base64
|
||
|
||
TOKEN = "your-gitea-pat"
|
||
REPO = "giersig.eu/indiekit-blog"
|
||
PATH = ".github/workflows/deploy.yml"
|
||
BASE = "http://10.100.0.90:3000/api/v1"
|
||
|
||
# 1. Get current SHA
|
||
req = urllib.request.Request(f"{BASE}/repos/{REPO}/contents/{PATH}",
|
||
headers={"Authorization": f"token {TOKEN}"})
|
||
info = json.loads(urllib.request.urlopen(req).read())
|
||
sha = info["sha"]
|
||
|
||
# 2. Read new content and encode
|
||
with open("/path/to/local/file") as f:
|
||
content = base64.b64encode(f.read().encode()).decode()
|
||
|
||
# 3. PUT new content
|
||
data = json.dumps({"message": "update file", "content": content, "sha": sha}).encode()
|
||
req2 = urllib.request.Request(f"{BASE}/repos/{REPO}/contents/{PATH}",
|
||
data=data, method="PUT",
|
||
headers={"Authorization": f"token {TOKEN}", "Content-Type": "application/json"})
|
||
urllib.request.urlopen(req2)
|
||
print("done")
|
||
PYEOF
|
||
```
|
||
|
||
Always generate base64 from the actual file — never copy b64 strings from session history (they can be silently corrupted by terminal line wrapping).
|
||
|
||
For Node.js scripts passed via `bastille cmd node sh -c '...'`, use base64 to avoid quoting issues:
|
||
|
||
```sh
|
||
# On local machine: encode the script
|
||
cat script.js | base64 | tr -d '\n'
|
||
# On server: decode and run
|
||
echo <b64> | b64decode -r > /tmp/script.js && node /tmp/script.js
|
||
```
|