doc: update
All checks were successful
Deploy Indiekit Server / deploy (push) Successful in 1m15s

This commit is contained in:
Sven
2026-04-01 17:42:18 +02:00
parent 65c1813b06
commit 2e35c5bd40
4 changed files with 159 additions and 3 deletions

1
.gitignore vendored
View File

@@ -2,6 +2,7 @@
start.sh
indiekit.config.mjs.orig
node_modules/
memory/
.cache/
.lesshst
.mongodb/

View File

@@ -129,6 +129,10 @@ Any new code path that creates posts should insert to `ap_timeline` immediately
`syndicator.syndicate(properties)` does **not** filter by post type. A note and a reply both become `Create(Note)`. The difference is whether `inReplyTo` is set (from `properties["in-reply-to"]`).
**Deduplication** (`patch-ap-syndicate-dedup`): at the start of `syndicate()`, queries `ap_activities` for an existing outbound Create/Announce/Update for `properties.url`. If found, returns the existing URL without re-federating. Prevents duplicate activities from CI webhooks triggering syndication twice (the Gitea commit that saves the syndication URL triggers a second build → second webhook call).
**Delete propagation** (`patch-micropub-delete-propagation` + `patch-bluesky-syndicator-delete`): `action=delete` in Micropub now iterates `publication.syndicationTargets` and calls `syndicator.delete(url, syndication)` fire-and-forget for any syndicator that exposes `.delete()`. The AP syndicator broadcasts a `Delete(Note)` via `broadcastDelete(url)`. The Bluesky syndicator deletes the bsky.app post via `com.atproto.repo.deleteRecord`, resolving the URL from `_deletedProperties`.
JF2 → AS2 mapping:
| Post type | Activity | Notes |
@@ -175,7 +179,8 @@ npm install git+https://gitea.giersig.eu/svemagie/indiekit-endpoint-activitypub
| `findTimelineItemById` returns null | Item not yet in `ap_timeline` (build not finished) or TZ-offset date mismatch — `$dateFromString` range query should catch offsets |
| Favourite/reblog hangs in Mastodon client | `resolveAuthor` timeout — `Promise.race` 5 s cap should prevent this |
| "Empty reply from server" on webmention poller | Poller routing through nginx (returns 444 for wrong Host) — must use `INDIEKIT_DIRECT_URL` |
| HTTP Signature verify errors flooding logs | Expected for deleted/migrated actors — suppressed to `fatal` level in federation-setup.js |
| HTTP Signature 401 errors on all inbound activities | nginx forwarding wrong `Host` header — fixed by `patch-ap-signature-host-header` (overrides to `blog.giersig.eu`) |
| HTTP Signature verify errors flooding logs for deleted/migrated actors | Expected noise — `patch-ap-inbox-delivery-debug` suppresses to `fatal`; real errors surface at `error` level |
| "OAuth callback failed. Missing parameters." | `state` parameter not echoed — fixed in fork (`b54146c`) |
| AP object 410 / Tombstone | Post was deleted — correct, served by FEP-4f05 |
@@ -226,7 +231,7 @@ Without it, `new URL(apiPath, baseUrl)` silently strips the `v1` segment → 404
## Micropub → Gitea build dispatch
Gitea Contents API commits (what `store-github` does) do **not** trigger `on: push` CI workflows. `patch-micropub-gitea-dispatch.mjs` patches the Micropub endpoint to fire a `workflow_dispatch` event to `giersig.eu/indiekit-blog` after each create/update, so the blog rebuilds immediately after a post is published.
Gitea Contents API commits (what `store-github` does) do **not** trigger `on: push` CI workflows. `patch-micropub-gitea-dispatch-conditional.mjs` patches the Micropub endpoint to fire a `workflow_dispatch` event to `giersig.eu/indiekit-blog` after each create/update, so the blog rebuilds immediately after a post is published.
---

View File

@@ -188,8 +188,11 @@ The patch replaces the broken date-from-URL regex with a simple last-path-segmen
### Troubleshooting
**All inbound AP activities return 401 / remote servers stop delivering**
The root cause is usually the `host` header being forwarded as the nginx upstream IP instead of the canonical `blog.giersig.eu`. Fedify includes `host` in the signed-string for Cavage HTTP Signatures; if it doesn't match what the remote server signed, every inbox POST fails verification. Fixed by `patch-ap-signature-host-header`: overrides `"host"` with `new URL(publicationUrl).host` in `fromExpressRequest()` after copying headers from the Express request.
**`ERR fedify·federation·inbox Failed to verify the request's HTTP Signatures`**
This message is expected at low volume (deleted actors, migrated servers with gone keys) and is suppressed to `fatal` level via a dedicated LogTape logger for `["fedify", "federation", "inbox"]` in `federation-setup.js` (`9b6db98`). If you see it flooding logs, check that the LogTape configuration applied. The body buffering patch must also preserve raw bytes in `req._rawBody` — if `JSON.stringify(req.body)` is used instead, the Digest header won't match.
At low volume this is expected (deleted actors, migrated servers with stale keys). `patch-ap-inbox-delivery-debug` changes the log level for `["fedify","federation","inbox"]` from `"fatal"` to `"error"` so real delivery failures are visible. If you see it flooding, the most common cause is the `host` header mismatch above — check that `patch-ap-signature-host-header` is applied. The body buffering patch must also preserve raw bytes in `req._rawBody` — if `JSON.stringify(req.body)` is used instead, the Digest header won't match.
**Mastodon client OAuth fails with "OAuth callback failed. Missing parameters."**
The OAuth 2.0 spec requires the server to echo the `state` parameter back in the authorization redirect. Mastodon clients (e.g. murmel.social) send a random `state` value for CSRF protection and fail if it is absent from the callback. Fixed in `b54146c`: `state` is now threaded through GET query → session store (surviving the IndieAuth login redirect) → hidden form field → POST body → callback URL (both approve and deny paths).
@@ -419,6 +422,30 @@ In the AP reader compose form (`/activitypub/admin/reader/compose`), the upstrea
**`patch-ap-mastodon-reply-threading.mjs`**
After `POST /api/v1/statuses` (Phanpy/Elk creates a post), the handler intentionally did not insert the new post into `ap_timeline` — it relied on the Eleventy build webhook firing 30120 s later. If the user replies to their own new post during that window, `findTimelineItemById` returns null, `in_reply_to_id` is silently dropped, and the follow-up reply is classified as a "note" (wrong post type, wrong URL path, no `inReplyTo` in AP output, broken thread on Mastodon). Fix: immediately after `postContent.create()`, inserts a provisional timeline item via `addTimelineItem()` using `$setOnInsert` (idempotent — syndicator's later upsert is a no-op).
**`patch-ap-mastodon-status-id.mjs`**
`POST /api/v1/statuses` previously returned `id: String(Date.now())` — the wall-clock time of the HTTP response. But the `ap_timeline` item is stored with `published: data.properties.published`, which is set before the Gitea write (515 s earlier). When a client immediately replies to the freshly created post, it sends `in_reply_to_id` equal to `Date.now()`, which is 515 s later than the stored `published`. The ±1 s range query in `findTimelineItemById` misses, so `inReplyTo = null` and the reply is saved as a note. Fix: use `encodeCursor(data.properties.published)` as the status ID in the creation response (falls back to `String(Date.now())` if `published` is missing). The returned ID now matches what `findTimelineItemById` will resolve.
**`patch-ap-status-reply-id.mjs`**
Two-part fix for `in_reply_to_id` always being `null` in the Mastodon status serializer. (A) `status.js`: the field was a tautological `item.inReplyTo ? null : null` (unfilled TODO) — changed to `item.inReplyToId || null`. (B) `statuses.js` POST handler: when pre-inserting own posts into `ap_timeline` (reply-threading patch), also stores `inReplyToId: inReplyToId || null` — the raw `in_reply_to_id` cursor from the client is already a valid `encodeCursor` value. Inbound AP replies from remote servers will still have `inReplyToId = null` until a separate patch populates it from `ap_timeline` lookups.
**`patch-ap-interactions-send-guard.mjs`**
`likePost` and `boostPost` in `lib/mastodon/helpers/interactions.js` called `ctx.sendActivity()` without try/catch. Any Fedify or Redis error propagated to the caller → 500 response → the `ap_interactions` DB write never ran → interaction not recorded locally. Fix: wrap both `sendActivity` calls in try/catch so delivery failures are non-fatal. Interaction is still recorded in `ap_interactions`; client sees correct UI state.
**`patch-ap-syndicate-dedup.mjs`**
The CI webhook calls `/syndicate?source_url=X&force=true` after every Eleventy build. When `syndicateToTargets()` saves the syndication URL it commits to Gitea → triggers another build → second CI call also hits the syndicate endpoint → duplicate `Create(Note)` activity sent. Root cause: the AP syndicator UID (`publicationUrl`) shares the same origin as the syndication URL (`properties.url`), so `force` mode re-selects it. Fix: at the start of `syndicate()`, queries `ap_activities` for an existing outbound Create/Announce/Update for `properties.url`. If found, returns the existing URL without re-federating.
**`patch-ap-mastodon-delete-fix.mjs`**
Two bugs in the Mastodon API delete route. Bug 1 (ReferenceError): the route used `objectId` (undefined) instead of `item._id` from `findTimelineItemById` → every delete threw ReferenceError → 500 → timeline entry never removed. Bug 2 (no AP broadcast): the route called `postContent.delete()` directly, bypassing the Indiekit syndicator framework → no `Delete(Note)` activity sent to followers → post persisted on Mastodon. Fix: (a) adds `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to `mastodonPluginOptions` in `index.js`; (b) calls `broadcastDelete(postUrl)` after removing the timeline entry.
**`patch-ap-inbox-publication-url.mjs`**
`collections._publicationUrl` was never set in `federation-setup.js`, so every `pubUrl && objectId.startsWith(pubUrl)` guard in `handleCreate`/`handleAnnounce` always evaluated to `undefined`. Result: no reply notifications, no boost notifications for own content, replies from non-followers not stored in `ap_timeline`. Fix: sets `collections._publicationUrl = publicationUrl` before `registerInboxListeners()`. Also adds an else-if branch in `handleCreate` to store replies to own posts in `ap_timeline` even when the sender is not in `ap_following`.
**`patch-ap-inbox-delivery-debug.mjs`**
The LogTape logger for `["fedify","federation","inbox"]` was hardcoded to `"fatal"` in `federation-setup.js`, suppressing all inbox errors including genuine delivery failures. Fix: changes the log level to `"error"` so real failures are visible. Expected noise from deleted/migrated actors (whose keys no longer resolve) still floods at `"error"` but can be filtered at the log aggregator level.
**`patch-ap-signature-host-header.mjs`**
`patch-ap-federation-bridge-base-url` fixed Fedify URL routing to use the canonical `publicationUrl`, but left the `host` header in the copied Headers object untouched. nginx forwards an internal host (e.g. `10.100.0.20`) which Fedify reads from `request.headers.get("host")` when reconstructing the signed-string for Cavage HTTP Signatures. Signed-string mismatch → every inbox POST returns 401 → remote servers exhaust retries and stop delivering. Fix: after the header-copy loop in `fromExpressRequest()`, overrides `"host"` with `new URL(publicationUrl).host` (`"blog.giersig.eu"`) when `publicationUrl` is provided.
### Conversations
**`patch-conversations-collection-guards.mjs`**
@@ -471,6 +498,9 @@ Defaults OwnYourSwarm `/where` check-in notes to `visibility: unlisted` unless t
**`patch-micropub-ai-block-resync.mjs`**
Detects stale AI-disclosure block files and re-generates them on next post save. Fixes posts that had MongoDB AI fields set but missing or empty `_ai-block.md` sidecar files (caused by a previous bug where `supportsAiDisclosure` always returned false).
**`patch-micropub-delete-propagation.mjs`**
Micropub `action=delete` only deleted the post from the content store. AP and Bluesky syndications persisted. Fix: after `postContent.delete()`, iterates `publication.syndicationTargets` and calls `syndicator.delete(url, syndication)` fire-and-forget for any syndicator that exposes `.delete()`. AP syndicator broadcasts `Delete(Note)`; Bluesky syndicator deletes the bsky.app post (see `patch-bluesky-syndicator-delete`).
### Posts
**`patch-endpoint-posts-ai-fields.mjs`**
@@ -567,6 +597,9 @@ One-time migration (guarded by a `migrations` MongoDB collection entry, currentl
**`patch-bluesky-syndicator-internal-url.mjs`**
Rewrites own-domain fetch URLs in the Bluesky syndicator to `INTERNAL_FETCH_URL` for jailed setups. Covers `uploadMedia()` (photo uploads), `uploadImageFromUrl()` (OG image thumbnails), and `fetchOpenGraphData()` (OG metadata extraction).
**`patch-bluesky-syndicator-delete.mjs`**
Extends the Bluesky syndicator with a `delete(url, syndication)` method that deletes the corresponding bsky.app post via `com.atproto.repo.deleteRecord`. The bsky.app URL is resolved from `_deletedProperties` (the post properties preserved by Indiekit during `action=delete`). Triggered by `patch-micropub-delete-propagation`.
### Internal URL rewriting
**`patch-micropub-fetch-internal-url.mjs`**
@@ -843,6 +876,38 @@ A CommonJS preload module that runs a Prometheus scrape endpoint on port 9209 (c
## Changelog
### 2026-04-01
**chore(deps): upgrade @fedify/* 2.1.1 → 2.1.3** (`65c1813`)
Fedify patch releases. No API changes; bug fixes in delivery and signature handling.
**fix(ap): HTTP Signature verification fails for all inbound activities** (`55923be`)
nginx forwards the upstream IP as the `Host` header. Fedify includes `host` in the Cavage signed-string; mismatched value → 401 on every inbox POST → remote servers exhaust retries and stop delivering. Fixed by `patch-ap-signature-host-header`: overrides `"host"` in `fromExpressRequest()` with `new URL(publicationUrl).host`.
**fix(ap): add AP inbox diagnostics — surface signature errors and request logging** (`8b1b5d9`)
`patch-ap-inbox-delivery-debug` changes the inbox LogTape level from `"fatal"` to `"error"`. Setting `AP_DEBUG=1` enables per-request body logging before Fedify's signature check.
**feat(ap/bsky): propagate Micropub deletes to ActivityPub and Bluesky** (`e791c06`)
`patch-micropub-delete-propagation`: after `postContent.delete()`, calls `syndicator.delete()` on all syndicators that expose `.delete()`. `patch-bluesky-syndicator-delete`: new Bluesky `.delete()` method via `com.atproto.repo.deleteRecord`. AP syndicator broadcasts `Delete(Note)` via `broadcastDelete()`.
**fix(activitypub): inbound replies/notifications broken — publicationUrl missing in inbox handlers** (`63bc41e`)
`patch-ap-inbox-publication-url`: sets `collections._publicationUrl` in `federation-setup.js` before inbox listener registration, enabling `handleCreate`/`handleAnnounce`/`handleLike` notifications for own content. Also stores replies to own posts from non-followers in `ap_timeline`.
**fix(ap): status creation response id mismatch breaks immediate reply threading** (`patch-ap-mastodon-status-id`)
`POST /api/v1/statuses` returned `id: String(Date.now())` (response time), but `ap_timeline` stores `published` (set pre-Gitea-write, 515 s earlier). Immediate follow-up replies sent `in_reply_to_id = Date.now()`, missing the ±1 s range query. Fixed: use `encodeCursor(data.properties.published)` as the response ID.
**fix(ap): in_reply_to_id always null in Mastodon status serializer** (`patch-ap-status-reply-id`)
Tautological `item.inReplyTo ? null : null` in `status.js` fixed to `item.inReplyToId || null`. POST handler now also stores `inReplyToId` in the timeline item so own replies are threaded in Phanpy/Elk.
**fix(ap): favourite/reblog error crashes interaction recording** (`patch-ap-interactions-send-guard`)
`sendActivity` in `likePost`/`boostPost` lacked try/catch; any Redis/Fedify error caused a 500 and skipped the `ap_interactions` DB write. Wrapped in try/catch so delivery failures are non-fatal.
**fix(ap): duplicate Create/Announce activities on double CI webhook** (`patch-ap-syndicate-dedup`)
CI webhook triggers twice per post (Gitea commit from syndication URL save triggers another build). `patch-ap-syndicate-dedup` checks `ap_activities` for an existing outbound activity for the URL and short-circuits if found.
**fix(ap): Mastodon API delete: ReferenceError + no AP broadcast** (`patch-ap-mastodon-delete-fix`)
Delete route used undefined `objectId` (should be `item._id`) → ReferenceError → 500. Also called `postContent.delete()` directly → no `Delete(Note)` sent to followers. Fixed: corrected variable + wired `broadcastDelete` through `mastodonPluginOptions`.
### 2026-03-27
**merge: upstream v3.9.x — Fedify 2.1.0, 5 FEPs, security/perf audit** (`230bfd1` in svemagie/indiekit-endpoint-activitypub)

View File

@@ -142,6 +142,91 @@ HTTP Signature Digest check passes.
- With inbox log level now `"error"`: signature failures show as Fedify error logs.
- Queue processing failures: `[inbox-queue] Failed processing ...` — always logged.
### `patch-ap-signature-host-header` *(2026-04-01)*
**File:** `lib/controllers/federation-bridge.js``fromExpressRequest()`
**Problem:** `patch-ap-federation-bridge-base-url` fixed Fedify URL routing to use the canonical
`publicationUrl`, but left the `host` header in the copied Headers object untouched. nginx forwards
an internal host (e.g. `10.100.0.20`) which Fedify reads from `request.headers.get("host")` when
reconstructing the signed-string for Cavage HTTP Signatures. Signed-string mismatch → every inbox
POST returns 401 → remote servers exhaust retries and stop delivering.
**Fix:** After the header-copy loop in `fromExpressRequest()`, override `"host"` with
`new URL(publicationUrl).host` (`"blog.giersig.eu"`) when `publicationUrl` is provided.
**Effect:** HTTP Signature verification now succeeds for all inbound AP activities.
### `patch-ap-mastodon-status-id` *(2026-04-01)*
**File:** `lib/mastodon/routes/statuses.js`
**Problem:** `POST /api/v1/statuses` returned `id: String(Date.now())` — the wall-clock time of the
response. The `ap_timeline` item uses `published: data.properties.published`, set before the Gitea
write (which can take 515 s). When the client replies to the freshly created post, it sends
`in_reply_to_id: <Date.now() id>`, which is 515 s later than the stored `published` → the ±1 s
range query misses → `inReplyTo = null` → reply saved as note.
**Fix:** Use `encodeCursor(data.properties.published)` as the status ID in the creation response
(falls back to `String(Date.now())` if published is missing). Response ID now matches what
`findTimelineItemById` will resolve.
### `patch-ap-interactions-send-guard` *(2026-04-01)*
**File:** `lib/mastodon/helpers/interactions.js`
**Problem:** `likePost` and `boostPost` call `ctx.sendActivity(...)` without try/catch. Any Fedify
or Redis error propagates → 500 response → the `ap_interactions` DB write never runs → interaction
not recorded locally.
**Fix:** Wrap both `sendActivity` calls in try/catch so delivery failures are non-fatal. Interaction
still recorded in `ap_interactions`; client sees correct UI state.
### `patch-ap-syndicate-dedup` *(2026-04-01)*
**File:** `lib/syndicator.js``syndicate()`
**Problem:** The CI webhook calls `/syndicate?source_url=X&force=true` after every Eleventy build.
When `syndicateToTargets()` saves the syndication URL it commits to Gitea → triggers another build
→ second CI call also hits the syndicate endpoint → duplicate `Create(Note)` activity sent.
Root cause: the AP syndicator UID (`publicationUrl`) shares the same origin as the syndication URL
(`properties.url`), so `force` mode re-selects it.
**Fix:** At the start of `syndicate()`, query `ap_activities` for an existing outbound
Create/Announce/Update for `properties.url`. If found, return the existing URL without re-federating.
### `patch-ap-mastodon-delete-fix` *(2026-04-01)*
**File:** `lib/mastodon/routes/statuses.js` (delete route) + `index.js`
**Bug 1 (ReferenceError):** Delete route used `objectId` (undefined) instead of `item._id` from
`findTimelineItemById` → every delete threw ReferenceError → 500 → timeline entry never removed.
**Bug 2 (no AP broadcast):** Route called `postContent.delete()` directly, bypassing the Indiekit
syndicator framework → no `Delete(Note)` activity sent to followers → post persists on Mastodon.
**Fix:** (a) Add `broadcastDelete: (url) => pluginRef.broadcastDelete(url)` to
`mastodonPluginOptions` in `index.js`. (b) Call `req.app.locals.mastodonPluginOptions.broadcastDelete(postUrl)`
after removing the timeline entry.
### `patch-micropub-delete-propagation` + `patch-bluesky-syndicator-delete` *(2026-04-01)*
**Files:** `node_modules/@indiekit/endpoint-micropub/lib/action.js` + Bluesky syndicator
**Problem:** Micropub `action=delete` only deleted the post from the content store. AP and Bluesky
syndications persisted.
**Fix:** After `postContent.delete()`, iterate `publication.syndicationTargets` and call
`syndicator.delete(url, syndication)` fire-and-forget for any syndicator exposing `.delete()`.
Bluesky syndicator extended with `deletePost(bskyUrl)` (via `com.atproto.repo.deleteRecord`) and
`delete(url, syndication)` that resolves the bsky.app URL from the preserved `_deletedProperties`.
### `patch-ap-inbox-publication-url` *(via 63bc41ebb, 2026-04-01)*
**File:** `lib/controllers/federation-setup.js`
**Problem:** `collections._publicationUrl` was never set in `federation-setup.js`, so every
`pubUrl && objectId.startsWith(pubUrl)` guard in `handleCreate`/`handleAnnounce` always evaluated
to `undefined` → no reply notifications, no boost notifications for own content, replies from
non-followers not stored in `ap_timeline`.
**Fix:** Set `collections._publicationUrl = publicationUrl` before `registerInboxListeners()`.
Also added else-if branch in `handleCreate` to store replies to own posts in `ap_timeline` even
when sender is not in `ap_following`.
### `patch-ap-status-reply-id` *(2026-04-01)*
**Files:** `lib/mastodon/entities/status.js` + `lib/mastodon/routes/statuses.js`
**Problem:** `in_reply_to_id` in the status serializer was a tautological `item.inReplyTo ? null : null`
(unfilled TODO) — always `null`. Mastodon clients (Phanpy/Elk) use this field to display reply
threading; without it, own replies appear as standalone posts.
**Fix (two parts):**
(A) `status.js`: return `item.inReplyToId || null` instead of the tautological null.
(B) `statuses.js` POST handler: when pre-inserting own posts into `ap_timeline` (reply-threading
patch), also store `inReplyToId: inReplyToId || null` — the raw `in_reply_to_id` cursor from
the client is already a valid `encodeCursor` value.
**Note:** Inbound AP replies from remote servers still have `inReplyToId = null` (separate patch
needed). Own replies via the Mastodon client API are fully fixed.
**Effect:** Own replies are threaded correctly in Phanpy/Elk.
---
## detectProtocol() in Microsub Reader
`detectProtocol(url)` in `reader.js` classifies URLs for syndication auto-selection: