mirror of
https://github.com/svemagie/indiekit-endpoint-activitypub.git
synced 2026-04-02 15:44:58 +02:00
Merge upstream rmdes:main — v2.13.0–v2.15.4 into svemagie/main
New upstream features:
- v2.13.0: FEP-8fcf/fe34 compliance, custom emoji, manual follow approval
- v2.14.0: Server blocking, Redis caching, key refresh, async inbox queue
- v2.15.0: Outbox failure handling (strike system), reply chain forwarding
- v2.15.1: Reply intelligence in reader (visibility badges, thread reconstruction)
- v2.15.2: Strip invalid as:Endpoints type from actor serialization
- v2.15.3: Exclude soft-deleted posts from outbox/content negotiation
- v2.15.4: Wire content-warning property for CW text
Conflict resolution:
- federation-setup.js: merged our draft/unlisted/visibility filters with
upstream's soft-delete filter
- compose.js: kept our DM compose path, adopted upstream's
lookupWithSecurity for remote object resolution
- notifications.js: kept our separate reply/mention tabs, added upstream's
follow_request grouping
- inbox-listeners.js: took upstream's thin-shim rewrite (handlers moved to
inbox-handlers.js which already has DM detection)
- notification-card.njk: merged DM badge with follow_request support
Preserved from our fork:
- Like/Announce to:Public cc:followers addressing
- Nested tag normalization (cat.split("/").at(-1))
- DM compose/reply path in compose controller
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
128
CLAUDE.md
128
CLAUDE.md
@@ -17,7 +17,10 @@ An Indiekit plugin that adds full ActivityPub federation via [Fedify](https://fe
|
||||
index.js ← Plugin entry, route registration, syndicator
|
||||
├── lib/federation-setup.js ← Fedify Federation instance, dispatchers, collections
|
||||
├── lib/federation-bridge.js ← Express ↔ Fedify request/response bridge
|
||||
├── lib/inbox-listeners.js ← Handlers for Follow, Undo, Like, Announce, Create, Delete, etc.
|
||||
├── lib/inbox-listeners.js ← Fedify inbox listener registration + reply forwarding
|
||||
├── lib/inbox-handlers.js ← Async inbox activity handlers (Create, Like, Announce, etc.)
|
||||
├── lib/inbox-queue.js ← Persistent MongoDB-backed async inbox processing queue
|
||||
├── lib/outbox-failure.js ← Outbox delivery failure handling (410 cleanup, 404 strikes, strike reset)
|
||||
├── lib/jf2-to-as2.js ← JF2 → ActivityStreams conversion (plain JSON + Fedify vocab)
|
||||
├── lib/kv-store.js ← MongoDB-backed KvStore for Fedify (get/set/delete/list)
|
||||
├── lib/activity-log.js ← Activity logging to ap_activities
|
||||
@@ -25,13 +28,26 @@ index.js ← Plugin entry, route registration, syndicat
|
||||
├── lib/timeline-store.js ← Timeline item extraction + sanitization
|
||||
├── lib/timeline-cleanup.js ← Retention-based timeline pruning
|
||||
├── lib/og-unfurl.js ← Open Graph link previews + quote enrichment
|
||||
├── lib/key-refresh.js ← Remote actor key freshness tracking (skip redundant re-fetches)
|
||||
├── lib/redis-cache.js ← Redis-cached actor lookups (cachedQuery wrapper)
|
||||
├── lib/lookup-helpers.js ← WebFinger/actor resolution utilities
|
||||
├── lib/lookup-cache.js ← In-memory LRU cache for actor lookups
|
||||
├── lib/resolve-author.js ← Author resolution with fallback chain
|
||||
├── lib/content-utils.js ← Content sanitization and text processing
|
||||
├── lib/emoji-utils.js ← Custom emoji detection and rendering
|
||||
├── lib/fedidb.js ← FediDB integration for popular accounts
|
||||
├── lib/batch-refollow.js ← Gradual re-follow for imported Mastodon accounts
|
||||
├── lib/migration.js ← CSV parsing + WebFinger resolution for Mastodon import
|
||||
├── lib/csrf.js ← CSRF token generation/validation
|
||||
├── lib/migrations/
|
||||
│ └── separate-mentions.js ← Data migration: split mentions from notifications
|
||||
├── lib/storage/
|
||||
│ ├── timeline.js ← Timeline CRUD with cursor pagination
|
||||
│ ├── notifications.js ← Notification CRUD with read/unread tracking
|
||||
│ └── moderation.js ← Mute/block storage
|
||||
│ ├── moderation.js ← Mute/block storage
|
||||
│ ├── server-blocks.js ← Server-level domain blocking
|
||||
│ ├── followed-tags.js ← Hashtag follow/unfollow storage
|
||||
│ └── messages.js ← Direct message storage
|
||||
├── lib/controllers/ ← Express route handlers (admin UI)
|
||||
│ ├── dashboard.js, reader.js, compose.js, profile.js, profile.remote.js
|
||||
│ ├── public-profile.js ← Public profile page (HTML fallback for actor URL)
|
||||
@@ -44,6 +60,15 @@ index.js ← Plugin entry, route registration, syndicat
|
||||
│ ├── featured.js, featured-tags.js
|
||||
│ ├── interactions.js, interactions-like.js, interactions-boost.js
|
||||
│ ├── moderation.js, migrate.js, refollow.js
|
||||
│ ├── messages.js ← Direct message UI
|
||||
│ ├── follow-requests.js ← Manual follow approval UI
|
||||
│ ├── follow-tag.js ← Hashtag follow/unfollow actions
|
||||
│ ├── tabs.js ← Explore tab management
|
||||
│ ├── my-profile.js ← Self-profile view
|
||||
│ ├── resolve.js ← Actor/post resolution endpoint
|
||||
│ ├── authorize-interaction.js ← Remote interaction authorization
|
||||
│ ├── federation-mgmt.js ← Federation management (server blocks)
|
||||
│ └── federation-delete.js ← Account deletion / federation cleanup
|
||||
├── views/ ← Nunjucks templates
|
||||
│ ├── activitypub-*.njk ← Page templates
|
||||
│ ├── layouts/ap-reader.njk ← Reader layout (NOT reader.njk — see gotcha below)
|
||||
@@ -60,7 +85,9 @@ index.js ← Plugin entry, route registration, syndicat
|
||||
|
||||
```
|
||||
Outbound: Indiekit post → syndicator.syndicate() → jf2ToAS2Activity() → ctx.sendActivity() → follower inboxes
|
||||
Inbound: Remote inbox POST → Fedify → inbox-listeners.js → MongoDB collections → admin UI
|
||||
Delivery failure → outbox-failure.js → 410: full cleanup | 404: strike system → eventual cleanup
|
||||
Inbound: Remote inbox POST → Fedify → inbox-listeners.js → ap_inbox_queue → inbox-handlers.js → MongoDB
|
||||
Reply forwarding: inbox-listeners.js checks if reply is to our post → ctx.forwardActivity() → follower inboxes
|
||||
Reader: Followed account posts → Create inbox → timeline-store → ap_timeline → reader UI
|
||||
Explore: Public Mastodon API → fetchMastodonTimeline() → mapMastodonToItem() → explore UI
|
||||
|
||||
@@ -73,7 +100,7 @@ processing pipeline via item-processing.js:
|
||||
|
||||
| Collection | Purpose | Key fields |
|
||||
|---|---|---|
|
||||
| `ap_followers` | Accounts following us | `actorUrl` (unique), `inbox`, `sharedInbox`, `source` |
|
||||
| `ap_followers` | Accounts following us | `actorUrl` (unique), `inbox`, `sharedInbox`, `source`, `deliveryFailures`, `firstFailureAt`, `lastFailureAt` |
|
||||
| `ap_following` | Accounts we follow | `actorUrl` (unique), `source`, `acceptedAt` |
|
||||
| `ap_activities` | Activity log (TTL-indexed) | `direction`, `type`, `actorUrl`, `objectUrl`, `receivedAt` |
|
||||
| `ap_keys` | Cryptographic key pairs | `type` ("rsa" or "ed25519"), key material |
|
||||
@@ -81,11 +108,19 @@ processing pipeline via item-processing.js:
|
||||
| `ap_profile` | Actor profile (single doc) | `name`, `summary`, `icon`, `attachments`, `actorType` |
|
||||
| `ap_featured` | Pinned posts | `postUrl`, `pinnedAt` |
|
||||
| `ap_featured_tags` | Featured hashtags | `tag`, `addedAt` |
|
||||
| `ap_timeline` | Reader timeline items | `uid` (unique), `published`, `author`, `content` |
|
||||
| `ap_timeline` | Reader timeline items | `uid` (unique), `published`, `author`, `content`, `visibility`, `isContext` |
|
||||
| `ap_notifications` | Likes, boosts, follows, mentions | `uid` (unique), `type`, `read` |
|
||||
| `ap_muted` | Muted actors/keywords | `url` or `keyword` |
|
||||
| `ap_blocked` | Blocked actors | `url` |
|
||||
| `ap_interactions` | Like/boost tracking per post | `objectUrl`, `type` |
|
||||
| `ap_messages` | Direct messages | `uid` (unique), `conversationId`, `author`, `content` |
|
||||
| `ap_followed_tags` | Hashtags we follow | `tag` (unique) |
|
||||
| `ap_explore_tabs` | Saved explore instances | `instance` (unique), `label` |
|
||||
| `ap_reports` | Outbound Flag activities | `actorUrl`, `reportedAt` |
|
||||
| `ap_pending_follows` | Follow requests awaiting approval | `actorUrl` (unique), `receivedAt` |
|
||||
| `ap_blocked_servers` | Blocked server domains | `domain` (unique) |
|
||||
| `ap_key_freshness` | Remote actor key verification timestamps | `actorUrl` (unique), `lastVerifiedAt` |
|
||||
| `ap_inbox_queue` | Persistent async inbox queue | `activityId`, `status`, `enqueuedAt` |
|
||||
|
||||
## Critical Patterns and Gotchas
|
||||
|
||||
@@ -141,13 +176,24 @@ Express 5 removed the `"back"` magic keyword from `response.redirect()`. It's tr
|
||||
|
||||
JSON-LD compaction collapses single-element arrays to plain objects. Mastodon's `update_account_fields` checks `attachment.is_a?(Array)` and silently skips if it's not an array. `sendFedifyResponse()` in `federation-bridge.js` forces `attachment` to always be an array.
|
||||
|
||||
**Note:** The old `endpoints.type` bug ([fedify#576](https://github.com/fedify-dev/fedify/issues/576)) was fixed in Fedify 2.0 — that workaround has been removed.
|
||||
### 10. WORKAROUND: Endpoints `as:Endpoints` Type Stripping
|
||||
|
||||
### 10. Profile Links — Express qs Body Parser Key Mismatch
|
||||
**File:** `lib/federation-bridge.js` (in `sendFedifyResponse()`)
|
||||
**Upstream issue:** [fedify#576](https://github.com/fedify-dev/fedify/issues/576) — FIXED in Fedify 2.1.0
|
||||
**Workaround:** `delete json.endpoints.type` strips the invalid `"type": "as:Endpoints"` from actor JSON.
|
||||
**Remove when:** Upgrading to Fedify ≥ 2.1.0.
|
||||
|
||||
### 11. KNOWN ISSUE: PropertyValue Attachment Type Validation
|
||||
|
||||
**Upstream issue:** [fedify#629](https://github.com/fedify-dev/fedify/issues/629) — OPEN
|
||||
**Problem:** `PropertyValue` (schema.org type) is not a valid AS2 Object/Link, so browser.pub rejects `/attachment`. Every Mastodon-compatible server emits this — cannot remove without breaking profile fields.
|
||||
**Workaround:** None applied (would break Mastodon compatibility). Documented as a known browser.pub strictness issue.
|
||||
|
||||
### 12. Profile Links — Express qs Body Parser Key Mismatch
|
||||
|
||||
`express.urlencoded({ extended: true })` uses `qs` which strips `[]` from array field names. HTML fields named `link_name[]` arrive as `request.body.link_name` (not `request.body["link_name[]"]`). The profile controller reads `link_name` and `link_value`, NOT `link_name[]`.
|
||||
|
||||
### 11. Author Resolution Fallback Chain
|
||||
### 13. Author Resolution Fallback Chain
|
||||
|
||||
`extractObjectData()` in `timeline-store.js` uses a multi-strategy fallback:
|
||||
1. `object.getAttributedTo()` — async, may fail with Authorized Fetch
|
||||
@@ -157,7 +203,7 @@ JSON-LD compaction collapses single-element arrays to plain objects. Mastodon's
|
||||
|
||||
Without this chain, many timeline items show "Unknown" as the author.
|
||||
|
||||
### 12. Username Extraction from Actor URLs
|
||||
### 14. Username Extraction from Actor URLs
|
||||
|
||||
When extracting usernames from attribution IDs, handle multiple URL patterns:
|
||||
- `/@username` (Mastodon)
|
||||
@@ -166,33 +212,33 @@ When extracting usernames from attribution IDs, handle multiple URL patterns:
|
||||
|
||||
The regex was previously matching "users" instead of the actual username from `/users/NatalieDavis`.
|
||||
|
||||
### 13. Empty Boost Filtering
|
||||
### 15. Empty Boost Filtering
|
||||
|
||||
Lemmy/PieFed send Announce activities where the boosted object resolves to an activity ID instead of a Note/Article with actual content. Check `object.content || object.name` before storing to avoid empty cards in the timeline.
|
||||
|
||||
### 14. Temporal.Instant for Fedify Dates
|
||||
### 16. Temporal.Instant for Fedify Dates
|
||||
|
||||
Fedify uses `@js-temporal/polyfill` for dates. When setting `published` on Fedify objects, use `Temporal.Instant.from(isoString)`. When reading Fedify dates in inbox handlers, use `String(object.published)` to get ISO strings — NOT `new Date(object.published)` which causes `TypeError`.
|
||||
|
||||
### 15. LogTape — Configure Once Only
|
||||
### 17. LogTape — Configure Once Only
|
||||
|
||||
`@logtape/logtape`'s `configure()` can only be called once per process. The module-level `_logtapeConfigured` flag prevents duplicate configuration. If configure fails (e.g., another plugin already configured it), catch the error silently.
|
||||
|
||||
When the debug dashboard is enabled (`debugDashboard: true`), LogTape configuration is **skipped entirely** because `@fedify/debugger` configures its own LogTape sink for the dashboard UI.
|
||||
|
||||
### 16. .authorize() Intentionally NOT Chained on Actor Dispatcher
|
||||
### 18. .authorize() Intentionally NOT Chained on Actor Dispatcher
|
||||
|
||||
Fedify's `.authorize()` triggers HTTP Signature verification on every GET to the actor endpoint. Servers requiring Authorized Fetch cause infinite loops: Fedify tries to fetch their key → they return 401 → Fedify retries → 500 errors. Re-enable when Fedify supports authenticated document loading for outgoing fetches.
|
||||
|
||||
### 17. Delivery Queue Must Be Started
|
||||
### 19. Delivery Queue Must Be Started
|
||||
|
||||
`federation.startQueue()` MUST be called after setup. Without it, `ctx.sendActivity()` enqueues tasks but the message queue never processes them — activities are never delivered.
|
||||
|
||||
### 18. Shared Key Dispatcher for Shared Inbox
|
||||
### 20. Shared Key Dispatcher for Shared Inbox
|
||||
|
||||
`inboxChain.setSharedKeyDispatcher()` tells Fedify to use our actor's key pair when verifying HTTP Signatures on the shared inbox. Without this, servers like hachyderm.io (which requires Authorized Fetch) have their signatures rejected.
|
||||
|
||||
### 19. Fedify 2.0 Modular Imports
|
||||
### 21. Fedify 2.0 Modular Imports
|
||||
|
||||
Fedify 2.0 uses modular entry points instead of a single barrel export. Imports must use the correct subpath:
|
||||
|
||||
@@ -210,19 +256,19 @@ import { Person, Note, Article, Create, Follow, ... } from "@fedify/fedify/vocab
|
||||
// import { Person, createFederation, exportJwk } from "@fedify/fedify";
|
||||
```
|
||||
|
||||
### 20. importSpki Removed in Fedify 2.0
|
||||
### 22. importSpki Removed in Fedify 2.0
|
||||
|
||||
Fedify 1.x exported `importSpki()` for loading PEM public keys. This was removed in 2.0. The local `importSpkiPem()` function in `federation-setup.js` replaces it using the Web Crypto API directly (`crypto.subtle.importKey("spki", ...)`). Similarly, `importPkcs8Pem()` handles private keys in PKCS#8 format.
|
||||
|
||||
### 21. KvStore Requires list() in Fedify 2.0
|
||||
### 23. KvStore Requires list() in Fedify 2.0
|
||||
|
||||
Fedify 2.0 added a `list(prefix?)` method to the KvStore interface. It must return an `AsyncIterable<{ key: string[], value: unknown }>`. The `MongoKvStore` in `kv-store.js` implements this as an async generator that queries MongoDB with a regex prefix match on the `_id` field.
|
||||
|
||||
### 22. Debug Dashboard Body Consumption
|
||||
### 24. Debug Dashboard Body Consumption
|
||||
|
||||
The `@fedify/debugger` login form POSTs `application/x-www-form-urlencoded` data. Because Express's body parser runs before the Fedify bridge, the POST body stream is already consumed (`req.readable === false`). The bridge in `federation-bridge.js` detects this and reconstructs the body from `req.body`. Without this, the debugger's login handler receives an empty body and throws `"Response body object should not be disturbed or locked"`. See also Gotcha #1.
|
||||
|
||||
### 23. Unified Item Processing Pipeline
|
||||
### 25. Unified Item Processing Pipeline
|
||||
|
||||
All views that display timeline items — reader, explore, tag timeline, hashtag explore, and their AJAX API counterparts — **must** use the shared pipeline in `lib/item-processing.js`. Never duplicate moderation filtering, quote stripping, interaction map building, or card rendering in individual controllers.
|
||||
|
||||
@@ -253,7 +299,7 @@ const html = await renderItemCards(processed, request, { interactionMap, mountPa
|
||||
|
||||
**If you add a new view that shows timeline items, use this pipeline.** Do not inline the logic.
|
||||
|
||||
### 24. Unified Infinite Scroll Alpine Component
|
||||
### 26. Unified Infinite Scroll Alpine Component
|
||||
|
||||
All views with infinite scroll use a single `apInfiniteScroll` Alpine.js component (in `assets/reader-infinite-scroll.js`), parameterized via data attributes on the container element:
|
||||
|
||||
@@ -272,7 +318,7 @@ All views with infinite scroll use a single `apInfiniteScroll` Alpine.js compone
|
||||
|
||||
**Do not create separate scroll components for new views.** Configure the existing one with appropriate data attributes. The explore view uses `data-cursor-param="max_id"` and `data-cursor-field="maxId"` (Mastodon API conventions), while the reader uses `data-cursor-param="before"` and `data-cursor-field="before"`.
|
||||
|
||||
### 25. Quote Embeds and Enrichment
|
||||
### 27. Quote Embeds and Enrichment
|
||||
|
||||
Posts that quote another post (Mastodon quote feature via FEP-044f) are rendered with an embedded card showing the quoted post's author, content, and timestamp. The data flow:
|
||||
|
||||
@@ -281,6 +327,40 @@ Posts that quote another post (Mastodon quote feature via FEP-044f) are rendered
|
||||
3. **On-demand:** `post-detail.js` fetches quotes on demand for items that have `quoteUrl` but no stored `quote` data (pre-existing items)
|
||||
4. **Rendering:** `partials/ap-quote-embed.njk` renders the embedded card; `stripQuoteReferences()` removes the inline `RE: <link>` paragraph to avoid duplication
|
||||
|
||||
### 28. Async Inbox Processing (v2.14.0+)
|
||||
|
||||
Inbound activities follow a two-stage pattern: `inbox-listeners.js` receives activities from Fedify, persists them to `ap_inbox_queue`, then `inbox-handlers.js` processes them asynchronously. This ensures no data loss if the server crashes mid-processing. Reply forwarding (`ctx.forwardActivity()`) happens synchronously in `inbox-listeners.js` because `forwardActivity()` is only available on `InboxContext`, not the base `Context` used by the queue processor.
|
||||
|
||||
### 29. Outbox Delivery Failure Handling (v2.15.0+)
|
||||
|
||||
`lib/outbox-failure.js` handles permanent delivery failures reported by Fedify's `setOutboxPermanentFailureHandler`:
|
||||
|
||||
- **410 Gone** → Immediate full cleanup: deletes follower from `ap_followers`, their items from `ap_timeline` (by `author.url`), their notifications from `ap_notifications` (by `actorUrl`)
|
||||
- **404 Not Found** → Strike system: increments `deliveryFailures` on the follower doc, sets `firstFailureAt` via `$setOnInsert`. After 3 strikes over 7+ days, triggers the same full cleanup as 410
|
||||
- **Strike reset** → `resetDeliveryStrikes()` is called in `inbox-listeners.js` after `touchKeyFreshness()` for every inbound activity type (except Block). If an actor is sending us activities, they're alive — `$unset` the strike fields
|
||||
|
||||
### 30. Reply Chain Fetching and Reply Forwarding (v2.15.0+)
|
||||
|
||||
- `fetchReplyChain()` in `inbox-handlers.js`: When a reply arrives, recursively fetches parent posts up to 5 levels deep using `object.getReplyTarget()`. Ancestors are stored with `isContext: true` flag. Uses `$setOnInsert` upsert so re-fetching ancestors is a no-op.
|
||||
- Reply forwarding in `inbox-listeners.js`: When a Create activity is a reply to one of our posts (checked via `inReplyTo.startsWith(publicationUrl)`) and is addressed to the public collection, calls `ctx.forwardActivity()` to re-deliver the reply to our followers' inboxes.
|
||||
|
||||
### 31. Write-Time Visibility Classification (v2.15.0+)
|
||||
|
||||
`computeVisibility(object)` in `inbox-handlers.js` classifies posts at ingest time based on `to`/`cc` fields:
|
||||
- `to` includes `https://www.w3.org/ns/activitystreams#Public` → `"public"`
|
||||
- `cc` includes Public → `"unlisted"`
|
||||
- Neither → `"private"` or `"direct"` (based on whether followers collection is in `to`)
|
||||
|
||||
The `visibility` field is stored on `ap_timeline` documents for future filtering.
|
||||
|
||||
### 32. Server Blocking (v2.14.0+)
|
||||
|
||||
`lib/storage/server-blocks.js` manages domain-level blocks stored in `ap_blocked_servers`. When a server is blocked, all inbound activities from that domain are rejected in `inbox-listeners.js` before any processing occurs. The `federation-mgmt.js` controller provides the admin UI.
|
||||
|
||||
### 33. Key Freshness Tracking (v2.14.0+)
|
||||
|
||||
`lib/key-refresh.js` tracks when remote actor keys were last verified in `ap_key_freshness`. `touchKeyFreshness()` is called for every inbound activity. This allows skipping redundant key re-fetches for actors we've recently verified, reducing network round-trips.
|
||||
|
||||
## Date Handling Convention
|
||||
|
||||
**All dates MUST be stored as ISO 8601 strings.** This is mandatory across all Indiekit plugins.
|
||||
@@ -348,6 +428,10 @@ On restart, `refollow:pending` entries are reset to `import` to prevent stale cl
|
||||
| `GET` | `{mount}/admin/reader/profile` | Remote profile view | Yes |
|
||||
| `GET` | `{mount}/admin/reader/moderation` | Moderation dashboard | Yes |
|
||||
| `POST` | `{mount}/admin/reader/mute,unmute,block,unblock` | Moderation actions | Yes |
|
||||
| `GET/POST` | `{mount}/admin/reader/messages` | Direct messages | Yes |
|
||||
| `GET/POST` | `{mount}/admin/follow-requests` | Manual follow approval | Yes |
|
||||
| `POST` | `{mount}/admin/reader/follow-tag,unfollow-tag` | Follow/unfollow hashtag | Yes |
|
||||
| `GET/POST` | `{mount}/admin/federation` | Server blocking management | Yes |
|
||||
| `GET` | `{mount}/admin/followers,following,activities` | Lists | Yes |
|
||||
| `GET/POST` | `{mount}/admin/profile` | Actor profile editor | Yes |
|
||||
| `GET/POST` | `{mount}/admin/featured` | Pinned posts | Yes |
|
||||
|
||||
64
README.md
64
README.md
@@ -43,6 +43,28 @@ Private ActivityPub messages (messages addressed only to your actor, with no `as
|
||||
- Reply delivery — replies are addressed to and delivered directly to the original post's author
|
||||
- Shared inbox support with collection sync (FEP-8fcf)
|
||||
- Configurable actor type (Person, Service, Organization, Group)
|
||||
- Manual follow approval — review and accept/reject follow requests before they take effect
|
||||
- Direct messages — private conversations stored separately from the public timeline
|
||||
|
||||
**Federation Resilience** *(v2.14.0+)*
|
||||
- Async inbox queue — inbound activities are persisted to MongoDB before processing, ensuring no data loss on crashes
|
||||
- Server blocking — block entire remote servers by domain, rejecting all inbound activities from blocked instances
|
||||
- Key freshness tracking — tracks when remote actor keys were last verified, skipping redundant re-fetches
|
||||
- Redis-cached actor lookups — caches actor resolution results to reduce network round-trips
|
||||
- Delivery strike tracking on `ap_followers` — counts consecutive delivery failures per follower
|
||||
- FEP-fe34 security — verifies `proof.created` timestamps to reject replayed activities
|
||||
|
||||
**Outbox Failure Handling** *(v2.15.0+, inspired by [Hollo](https://github.com/fedify-dev/hollo))*
|
||||
- **410 Gone** — immediate full cleanup: removes the follower, their timeline items, and their notifications
|
||||
- **404 Not Found** — strike system: 3 consecutive failures over 7+ days triggers the same full cleanup
|
||||
- Strike auto-reset — when an actor sends us any activity, their delivery failure count resets to zero
|
||||
- Prevents orphaned data from accumulating over time while tolerating temporary server outages
|
||||
|
||||
**Reply Intelligence** *(v2.15.0+, inspired by [Hollo](https://github.com/fedify-dev/hollo))*
|
||||
- Recursive reply chain fetching — when a reply arrives, fetches parent posts up to 5 levels deep for thread context
|
||||
- Ancestor posts stored with `isContext: true` flag for thread view without cluttering the main timeline
|
||||
- Reply forwarding to followers — when someone replies to our posts, the reply is forwarded to our followers so they see the full conversation
|
||||
- Write-time visibility classification — computes `public`/`unlisted`/`private`/`direct` from `to`/`cc` fields at ingest time
|
||||
|
||||
**Reader**
|
||||
- Timeline view showing posts from followed accounts with tab filtering (notes, articles, replies, boosts, media)
|
||||
@@ -67,6 +89,8 @@ Private ActivityPub messages (messages addressed only to your actor, with no `as
|
||||
**Moderation**
|
||||
- Mute actors or keywords
|
||||
- Block actors (also removes from followers)
|
||||
- Block entire servers by domain
|
||||
- Report remote actors to their home instance (Flag activity)
|
||||
- All moderation actions available from the reader UI
|
||||
|
||||
**Mastodon Migration**
|
||||
@@ -229,6 +253,7 @@ When remote servers send activities to your inbox:
|
||||
- **Accept(Follow)** → Marks our follow as accepted
|
||||
- **Reject(Follow)** → Marks our follow as rejected
|
||||
- **Block** → Removes actor from our followers
|
||||
- **Flag** → Outbound report sent to remote actor's instance
|
||||
|
||||
### Direct Message Detection
|
||||
|
||||
@@ -307,7 +332,7 @@ The plugin creates these collections automatically:
|
||||
|
||||
| Collection | Description |
|
||||
|---|---|
|
||||
| `ap_followers` | Accounts following your actor |
|
||||
| `ap_followers` | Accounts following your actor (includes delivery failure strike tracking) |
|
||||
| `ap_following` | Accounts you follow |
|
||||
| `ap_activities` | Activity log with automatic TTL cleanup |
|
||||
| `ap_keys` | RSA and Ed25519 key pairs for HTTP Signatures |
|
||||
@@ -315,11 +340,19 @@ The plugin creates these collections automatically:
|
||||
| `ap_profile` | Actor profile (single document) |
|
||||
| `ap_featured` | Pinned/featured posts |
|
||||
| `ap_featured_tags` | Featured hashtags |
|
||||
| `ap_timeline` | Reader timeline items from followed accounts |
|
||||
| `ap_timeline` | Reader timeline items (includes `visibility`, `isContext`, `isDirect` fields) |
|
||||
| `ap_notifications` | Interaction notifications (includes `isDirect` and `senderActorUrl` fields for DMs) |
|
||||
| `ap_muted` | Muted actors and keywords |
|
||||
| `ap_blocked` | Blocked actors |
|
||||
| `ap_interactions` | Per-post like/boost tracking |
|
||||
| `ap_messages` | Direct messages / private conversations |
|
||||
| `ap_followed_tags` | Hashtags you follow for timeline filtering |
|
||||
| `ap_explore_tabs` | Saved Mastodon instances for the explore view |
|
||||
| `ap_reports` | Outbound reports (Flag activities) sent to remote instances |
|
||||
| `ap_pending_follows` | Follow requests awaiting manual approval |
|
||||
| `ap_blocked_servers` | Blocked server domains (instance-level blocks) |
|
||||
| `ap_key_freshness` | Tracks when remote actor keys were last verified |
|
||||
| `ap_inbox_queue` | Persistent async inbox processing queue |
|
||||
|
||||
## Supported Post Types
|
||||
|
||||
@@ -361,6 +394,23 @@ Mastodon's `update_account_fields` checks `attachment.is_a?(Array)` and silently
|
||||
|
||||
**Revisit when:** Fedify adds an option to preserve arrays during JSON-LD serialization, or Mastodon fixes their array check.
|
||||
|
||||
### Endpoints `as:Endpoints` Type Stripping
|
||||
|
||||
**File:** `lib/federation-bridge.js` (in `sendFedifyResponse()`)
|
||||
**Upstream issue:** [fedify#576](https://github.com/fedify-dev/fedify/issues/576) — FIXED in Fedify 2.1.0
|
||||
|
||||
Fedify serializes the `endpoints` object with `"type": "as:Endpoints"`, which is not a valid ActivityStreams type. browser.pub rejects this. The bridge strips the `type` field from the `endpoints` object before sending.
|
||||
|
||||
**Remove when:** Upgrading to Fedify ≥ 2.1.0.
|
||||
|
||||
### PropertyValue Attachment Type (Known Issue)
|
||||
|
||||
**Upstream issue:** [fedify#629](https://github.com/fedify-dev/fedify/issues/629) — OPEN
|
||||
|
||||
Fedify serializes `PropertyValue` attachments (used by Mastodon for profile metadata fields) with `"type": "PropertyValue"`, a schema.org type that is not a valid AS2 Object or Link. browser.pub rejects `/attachment` as invalid. However, every Mastodon-compatible server emits `PropertyValue` — removing it would break profile field display across the fediverse.
|
||||
|
||||
**No workaround applied.** This is a de facto fediverse standard despite not being in the AS2 vocabulary.
|
||||
|
||||
### `.authorize()` Not Chained on Actor Dispatcher
|
||||
|
||||
**File:** `lib/federation-setup.js` (line ~254)
|
||||
@@ -403,6 +453,16 @@ This is not a bug — Fedify requires explicit opt-in for signed fetches. But it
|
||||
- **Existing DMs before this fork** — Notifications received before upgrading to this fork lack `isDirect`/`senderActorUrl` and won't appear in the Direct tab (resend or patch manually in MongoDB)
|
||||
- **No read receipts** — Outbound DMs are stored locally but the recipient receives no read-receipt activity
|
||||
|
||||
## Acknowledgements
|
||||
|
||||
This plugin builds on the excellent [Fedify](https://fedify.dev) framework by [Hong Minhee](https://github.com/dahlia). Fedify provides the core ActivityPub federation layer — HTTP Signatures, content negotiation, message queues, and the vocabulary types that make all of this possible.
|
||||
|
||||
Several federation patterns in this plugin were inspired by studying other open-source ActivityPub implementations:
|
||||
|
||||
- **[Hollo](https://github.com/fedify-dev/hollo)** (by the Fedify author) — A single-user Fedify-based ActivityPub server that served as the primary reference implementation. The outbox permanent failure handling (410 cleanup and 404 strike system), recursive reply chain fetching, reply forwarding to followers, and write-time visibility classification in v2.15.0 are all adapted from Hollo's patterns for a MongoDB/single-user context.
|
||||
|
||||
- **[Wafrn](https://github.com/gabboman/wafrn)** — A federated social network whose ActivityPub implementation informed the operational resilience patterns added in v2.14.0. Server blocking, key freshness tracking, async inbox processing with persistent queues, and the general approach to federation hardening were inspired by studying Wafrn's production codebase.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
|
||||
@@ -351,6 +351,12 @@
|
||||
margin-left: 0.2em;
|
||||
}
|
||||
|
||||
.ap-card__visibility {
|
||||
font-size: var(--font-size-xs);
|
||||
margin-left: 0.3em;
|
||||
opacity: 0.7;
|
||||
}
|
||||
|
||||
.ap-card__timestamp-link {
|
||||
color: inherit;
|
||||
text-decoration: none;
|
||||
@@ -3408,3 +3414,28 @@
|
||||
}
|
||||
}
|
||||
|
||||
/* Follow request approve/reject actions */
|
||||
.ap-follow-request {
|
||||
margin-block-end: var(--space-m);
|
||||
}
|
||||
|
||||
.ap-follow-request__actions {
|
||||
display: flex;
|
||||
gap: var(--space-s);
|
||||
margin-block-start: var(--space-xs);
|
||||
padding-inline-start: var(--space-l);
|
||||
}
|
||||
|
||||
.ap-follow-request__form {
|
||||
display: inline;
|
||||
}
|
||||
|
||||
.button--danger {
|
||||
background-color: var(--color-red45);
|
||||
color: white;
|
||||
}
|
||||
|
||||
.button--danger:hover {
|
||||
background-color: var(--color-red35, #c0392b);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,403 +0,0 @@
|
||||
# ActivityPub Coverage Audit: @rmdes/indiekit-endpoint-activitypub vs Fedify 2.0
|
||||
|
||||
**Date:** 2026-03-13
|
||||
**Plugin Version:** 2.9.2
|
||||
**Fedify Version:** 2.0.0
|
||||
**Auditor:** Claude Code (Opus 4.6)
|
||||
|
||||
---
|
||||
|
||||
## Legend
|
||||
|
||||
- **Implemented** — fully working in production
|
||||
- **Partial** — some aspects implemented, gaps remain
|
||||
- **Not implemented** — Fedify supports it, we don't use it
|
||||
|
||||
---
|
||||
|
||||
## 1. Inbound Activity Handlers
|
||||
|
||||
All handlers are in `lib/inbox-listeners.js`. Fedify dispatches inbound activities to registered listeners via `setInboxListeners()`.
|
||||
|
||||
| Activity Type | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| `Follow` | Full | Auto-accept, store follower in `ap_followers`, create notification, log to `ap_activities` (lines 90–149) | **Implemented** |
|
||||
| `Accept` | Full | Updates `ap_following` entry to `source: "federation"`, clears retry fields (lines 194–235) | **Implemented** |
|
||||
| `Reject` | Full | Marks `ap_following` entry as `source: "rejected"` (lines 236–267) | **Implemented** |
|
||||
| `Undo(Follow)` | Full | Removes from `ap_followers` (lines 151–170) | **Implemented** |
|
||||
| `Undo(Like)` | Full | Removes from `ap_activities` (lines 171–183) | **Implemented** |
|
||||
| `Undo(Announce)` | Full | Removes from `ap_activities` (lines 184–193) | **Implemented** |
|
||||
| `Like` | Full | Filtered to our content only (`objectId.startsWith(publicationUrl)`), stores notification + activity log (lines 268–317) | **Implemented** |
|
||||
| `Announce` | Full | Dual path: boosts of our posts → notification; boosts from followed accounts → `ap_timeline` with quote enrichment (lines 318–412) | **Implemented** |
|
||||
| `Create` | Full | Four paths: DMs → `ap_messages`; replies to us → notification; mentions → notification; followed accounts → `ap_timeline` with link preview + quote enrichment (lines 413–639) | **Implemented** |
|
||||
| `Delete` | Full | Removes from `ap_activities` + `ap_timeline` (lines 640–649) | **Implemented** |
|
||||
| `Update` | Full | Post updates → `ap_timeline` content refresh; profile updates → follower data refresh (lines 672–735) | **Implemented** |
|
||||
| `Move` | Full | Updates follower `actorUrl` to new address, stores `movedFrom` (lines 650–671) | **Implemented** |
|
||||
| `Block` | Full | Remote actor blocked us → removes from `ap_followers` (lines 736–744) | **Implemented** |
|
||||
| `Add` / `Remove` | Full | No-op — logged only. Mastodon uses these for featured collection management (lines 745–750) | **Partial** — not used for featured collection sync |
|
||||
| `Flag` | Full | Not handled | **Not implemented** — no report/moderation inbox |
|
||||
| `EmojiReact` | Full (LitePub) | Not handled | **Not implemented** |
|
||||
| `Dislike` | Full | Not handled | **Not implemented** — rarely used in fediverse |
|
||||
| `Question` | Full | Not handled specially | **Not implemented** — polls not parsed |
|
||||
| `Arrive` / `Travel` / `Join` / `Leave` | Full | Not handled | **Not implemented** — niche activity types |
|
||||
| `Invite` / `Offer` | Full | Not handled | **Not implemented** |
|
||||
| `Read` / `View` / `Listen` | Full | Not handled | **Not implemented** — niche |
|
||||
|
||||
**Score: 13/21 activity types handled (62%), covering ~99% of real-world fediverse traffic**
|
||||
|
||||
---
|
||||
|
||||
## 2. Outbound Activities
|
||||
|
||||
Outbound activities are sent via `ctx.sendActivity()` from syndicator (`index.js`), interaction controllers (`lib/controllers/interactions-*.js`), compose (`lib/controllers/compose.js`), and messages (`lib/controllers/messages.js`).
|
||||
|
||||
| Activity | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| `Create(Note)` | Full | Via syndicator (`jf2ToAS2Activity()`) + DM compose (`submitMessageController`) | **Implemented** |
|
||||
| `Create(Article)` | Full | Via syndicator (`jf2ToAS2Activity()`) | **Implemented** |
|
||||
| `Like` | Full | Reader interaction button (`interactions-like.js:14–115`) | **Implemented** |
|
||||
| `Undo(Like)` | Full | Unlike button (`interactions-like.js:121–229`) | **Implemented** |
|
||||
| `Announce` | Full | Boost button (`interactions-boost.js:14–~100`) | **Implemented** |
|
||||
| `Undo(Announce)` | Full | Unboost button (`interactions-boost.js:~101–~180`) | **Implemented** |
|
||||
| `Follow` | Full | Reader follow + migration + batch refollow (`index.js:572–667`) | **Implemented** |
|
||||
| `Undo(Follow)` | Full | Unfollow button (`index.js:674–~750`) | **Implemented** |
|
||||
| `Accept(Follow)` | Full | Auto-accept on inbound Follow (`inbox-listeners.js:120–128`) | **Implemented** |
|
||||
| `Update(Person)` | Full | Profile edit broadcasts to all followers (`index.js:761–~850`) | **Implemented** |
|
||||
| `Delete` | Full | Not sent when posts are deleted | **Not implemented** |
|
||||
| `Block` | Full | Local-only mute/block, no `Block` activity sent to remote | **Not implemented** |
|
||||
| `Flag` | Full | No report sending UI | **Not implemented** |
|
||||
| `Move` | Full | No outbound account migration | **Not implemented** |
|
||||
| `Reject(Follow)` | Full | Auto-accept only, no manual approval/reject | **Not implemented** |
|
||||
| `Create(Question)` | Full | No poll creation | **Not implemented** |
|
||||
|
||||
**Score: 10/16 common outbound activities (63%)**
|
||||
|
||||
---
|
||||
|
||||
## 3. Federation Dispatchers & Collections
|
||||
|
||||
All dispatchers are registered in `lib/federation-setup.js`.
|
||||
|
||||
| Dispatcher/Collection | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| Actor (`Person`) | Full | Full with 5 actor types (Person, Service, Organization, Group, Application). Instance actor for shared inbox signing. RSA + Ed25519 key pairs. `mapHandle()` + `mapAlias()`. (lines 134–160) | **Implemented** |
|
||||
| Inbox (personal + shared) | Full | Both endpoints registered. `setSharedKeyDispatcher()` for Authorized Fetch servers. (lines 283–295) | **Implemented** |
|
||||
| Outbox | Full | Paginated, converts published blog posts to `Create(Note\|Article)` via `jf2ToAS2Activity()`. 20 per page. (lines 589–~650) | **Implemented** |
|
||||
| Followers | Full | Paginated + one-shot mode for `sendActivity("followers")` batch delivery. Counter. (lines 396–445) | **Implemented** |
|
||||
| Following | Full | Paginated with counter. 20 per page. (lines 447–475) | **Implemented** |
|
||||
| Liked | Full | From `posts` collection where `post-type: "like"`. Paginated. (lines 477–518) | **Implemented** |
|
||||
| Featured (pinned posts) | Full | Admin UI + AP collection. Converts pinned posts via `jf2ToAS2Activity()`. (lines 520–555) | **Implemented** |
|
||||
| Featured Tags | Full | Admin UI + AP collection. Hashtag objects with category page links. (lines 557–587) | **Implemented** |
|
||||
| Object dispatcher | Full | Content negotiation on individual post URLs. Returns `Create(Note\|Article)` AS2 JSON-LD for `Accept: application/activity+json`. | **Implemented** |
|
||||
| WebFinger | Full | With OStatus subscribe link for remote follow from WordPress AP, Misskey, etc. (lines 275–282) | **Implemented** |
|
||||
| NodeInfo | Full | Version 2.1. Reports software, protocols, total posts, active users. (lines 322–339) | **Implemented** |
|
||||
| `.authorize()` on actor | Full | Intentionally disabled — causes infinite loops with Authorized Fetch servers. See CLAUDE.md Gotcha #16. | **Not implemented** |
|
||||
| Custom collections | Full | Not used | **Not implemented** |
|
||||
|
||||
**Score: 11/13 (85%)**
|
||||
|
||||
---
|
||||
|
||||
## 4. Cryptography & Security
|
||||
|
||||
Key storage in `ap_keys` collection. Key generation and signing handled by Fedify internals.
|
||||
|
||||
| Feature | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| RSA key pairs (HTTP Signatures) | Full | Generated on first use, stored as PEM in `ap_keys` (`federation-setup.js`) | **Implemented** |
|
||||
| Ed25519 key pairs (Object Integrity Proofs) | Full | Generated on first use, stored as JWK in `ap_keys` | **Implemented** |
|
||||
| HTTP Signatures (draft-cavage-12) | Full | Automatic via Fedify signing on all outbound requests | **Implemented** |
|
||||
| HTTP Message Signatures (RFC 9421) | Full | Automatic via Fedify double-knocking negotiation | **Implemented** |
|
||||
| Double-Knocking negotiation | Full | Automatic — Fedify caches per-server signature spec preference | **Implemented** |
|
||||
| Authenticated Document Loader | Full | Used in all inbox handlers via `getAuthLoader()` helper. Required for Authorized Fetch servers (hachyderm.io, etc.) | **Implemented** |
|
||||
| Object Integrity Proofs (FEP-8b32) | Full | Ed25519 keys stored; Fedify creates proofs automatically | **Implemented** (via Fedify) |
|
||||
| Linked Data Signatures | Full | Fedify handles verification on inbound; not explicitly configured | **Partial** — verification only |
|
||||
| Authorized Fetch on actor endpoint | Full | Disabled — `.authorize()` causes infinite key-fetch loops. Instance actor used as workaround for shared inbox signing. | **Not implemented** |
|
||||
| Origin-based security (FEP-fe34) | Full | Not configured — using Fedify defaults | **Not implemented** |
|
||||
| Inbox idempotency | Full | Not explicitly configured — using Fedify default (`"per-inbox"`) | **Implemented** (default) |
|
||||
| Signature time window | Full | Default (1 hour) | **Implemented** (default) |
|
||||
| CSRF protection | N/A (app concern) | Token generation + validation on all POST routes (`lib/csrf.js`) | **Implemented** |
|
||||
| Content sanitization | N/A (app concern) | `sanitize-html` on all inbound content (`timeline-store.js`) | **Implemented** |
|
||||
|
||||
**Score: 8/12 Fedify-specific features (67%)**
|
||||
|
||||
---
|
||||
|
||||
## 5. Content & Object Types
|
||||
|
||||
Object creation in `lib/jf2-to-as2.js`. Object parsing in `lib/timeline-store.js` and `lib/inbox-listeners.js`.
|
||||
|
||||
| Object Type | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| `Note` | Full | Create, display, syndicate. Primary post type for notes/replies/DMs. | **Implemented** |
|
||||
| `Article` | Full | Create, display, syndicate. Used for article post type. | **Implemented** |
|
||||
| `Image` (attachment) | Full | Photo posts with `Image` attachments in `jf2ToAS2Activity()` | **Implemented** |
|
||||
| `Video` (attachment) | Full | Video post attachments | **Implemented** |
|
||||
| `Audio` (attachment) | Full | Audio post attachments | **Implemented** |
|
||||
| `Hashtag` (tag) | Full | Tags on syndicated posts, featured tags collection, tag timeline | **Implemented** |
|
||||
| `Mention` (tag) | Full | Tags on replies for addressing, DM recipient mentions | **Implemented** |
|
||||
| `PropertyValue` | Full | Profile attachment fields (name/value pairs) | **Implemented** |
|
||||
| Quote posts (FEP-044f) | Full | Ingest via `quoteUrl` (3 namespaces), enrich via `fetchAndStoreQuote()`, render via `ap-quote-embed.njk` | **Implemented** |
|
||||
| `Question` (polls) | Full | Not parsed — poll posts render without options | **Not implemented** |
|
||||
| `Event` | Full | Not handled — events render as generic objects | **Not implemented** |
|
||||
| `Page` | Full | Passthrough via content negotiation only | **Partial** |
|
||||
| `ChatMessage` (LitePub DMs) | Full | Not handled — we use standard `Create(Note)` DM addressing | **Not implemented** |
|
||||
| `Tombstone` | Full | Not created when posts are deleted | **Not implemented** |
|
||||
| `Emoji` (custom) | Full (`toot:Emoji`) | Not handled — custom emoji renders as `:shortcode:` text | **Not implemented** |
|
||||
| `Place` (location) | Full | Not handled — location data ignored | **Not implemented** |
|
||||
| Sensitive / Content Warning | Full | `sensitive` flag displayed on inbound items but not settable on outbound | **Partial** |
|
||||
| `Source` (original markup) | Full | Not used on outbound activities | **Not implemented** |
|
||||
|
||||
**Score: 10/18 (56%), but core types fully covered**
|
||||
|
||||
---
|
||||
|
||||
## 6. Audience Addressing & Visibility
|
||||
|
||||
Addressing logic in `lib/jf2-to-as2.js` (lines 179–194) and `lib/controllers/messages.js` for DMs.
|
||||
|
||||
| Visibility Mode | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| **Public** (`to: PUBLIC_COLLECTION`, `cc: followers`) | Full | Standard addressing for all syndicated posts | **Implemented** |
|
||||
| **Unlisted** (`to: followers`, `cc: PUBLIC_COLLECTION`) | Full | Not available — no UI option | **Not implemented** |
|
||||
| **Followers-only** (`to: followers`, no PUBLIC) | Full | Not available — all posts are public | **Not implemented** |
|
||||
| **Direct/DM** (`to: specific actors` only) | Full | Inbound detection (`isDirectMessage()`) + outbound via `submitMessageController` | **Implemented** |
|
||||
|
||||
**Score: 2/4 (50%) — the two missing modes are rarely needed for IndieWeb sites**
|
||||
|
||||
---
|
||||
|
||||
## 7. FEP (Fediverse Enhancement Proposals)
|
||||
|
||||
| FEP | Description | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| FEP-8b32 | Object Integrity Proofs | Ed25519 keys generated and stored; Fedify creates proofs on outbound activities | **Implemented** (via Fedify) |
|
||||
| FEP-521a | Multiple Cryptographic Keys | Both RSA + Ed25519 key pairs via `setKeyPairsDispatcher()` | **Implemented** |
|
||||
| FEP-044f | Quote Posts | Full pipeline: ingest `quoteUrl` (3 namespaces), enrich via `fetchAndStoreQuote()`, render embedded card, strip inline `RE:` references | **Implemented** |
|
||||
| FEP-8fcf | Followers Collection Synchronization | Not configured — `syncCollection` option not passed to `sendActivity()` | **Not implemented** |
|
||||
| FEP-fe34 | Origin-Based Security | Not configured — using Fedify defaults (`crossOrigin` not set) | **Not implemented** |
|
||||
| FEP-ae0c | Relay Protocols | `@fedify/relay` not used — personal site doesn't need relay | **Not implemented** |
|
||||
| FEP-c0e0 | Actor Succession | `successor` property not set on actor | **Not implemented** |
|
||||
| FEP-9091 | DID-Based Actor Identification | `DidService`/`Export` not used | **Not implemented** |
|
||||
| FEP-5711 | Inverse Collection Properties | `likesOf`, `sharesOf`, etc. not exposed | **Not implemented** |
|
||||
|
||||
**Score: 3/9 (33%) — the three implemented FEPs are the most impactful for interoperability**
|
||||
|
||||
---
|
||||
|
||||
## 8. Infrastructure & Operations
|
||||
|
||||
| Feature | Fedify Support | Our Implementation | Status |
|
||||
|---|---|---|---|
|
||||
| Redis message queue | `@fedify/redis` | `RedisMessageQueue` with `parallelWorkers` config (default 5) | **Implemented** |
|
||||
| In-process queue | `InProcessMessageQueue` | Fallback when `redisUrl` not set | **Implemented** |
|
||||
| MongoDB KvStore | Custom (app-provided) | `MongoKvStore` in `lib/kv-store.js` with `get`/`set`/`delete`/`list()` (required in Fedify 2.0) | **Implemented** |
|
||||
| Debug dashboard | `@fedify/debugger` | Optional via `debugDashboard: true`, password-protected at `/{mount}/__debug__/` | **Implemented** |
|
||||
| OpenTelemetry tracing | Full | Via `@fedify/debugger` `FedifySpanExporter` | **Implemented** |
|
||||
| LogTape logging | Full | Configured once with `_logtapeConfigured` flag to prevent duplicate setup | **Implemented** |
|
||||
| Delivery failure handling | Full | 404/410 permanent failures logged + stored in `ap_activities` (lines 344–361) | **Implemented** |
|
||||
| Exponential backoff retry | Full | Using Fedify default retry policy | **Implemented** |
|
||||
| Activity transformers | Full | Not used — `autoIdAssigner()` and `actorDehydrator()` defaults only | **Not implemented** |
|
||||
| PostgreSQL queue | `@fedify/postgres` | Not applicable — using Redis | N/A |
|
||||
| SQLite queue | `@fedify/sqlite` | Not applicable — using Redis | N/A |
|
||||
|
||||
**Score: 8/9 relevant features (89%)**
|
||||
|
||||
---
|
||||
|
||||
## 9. Application-Level Features (Beyond Fedify)
|
||||
|
||||
These features are built on top of Fedify — Fedify provides the federation primitives, we provide the application logic.
|
||||
|
||||
| Feature | Description | Status |
|
||||
|---|---|---|
|
||||
| Timeline reader | Full reader UI with tabs (notes, articles, boosts, media, replies, unread) | **Implemented** |
|
||||
| Notifications | Like, boost, follow, mention, reply, DM notification types with unread tracking | **Implemented** |
|
||||
| Direct messages | Inbound + outbound DMs with conversation sidebar, compose form | **Implemented** |
|
||||
| Explore | Public Mastodon timeline aggregation from configured instances | **Implemented** |
|
||||
| Hashtag explore | Cross-instance hashtag search via Mastodon API | **Implemented** |
|
||||
| Tag timeline | Posts from followed accounts filtered by hashtag | **Implemented** |
|
||||
| Post detail | Full post view with replies, quote enrichment | **Implemented** |
|
||||
| Remote profile | View remote actor profiles with follow/mute/block actions | **Implemented** |
|
||||
| Moderation | Mute (by URL or keyword), block, with filtering across all views | **Implemented** |
|
||||
| Mastodon migration | CSV import + WebFinger resolution + batch re-follow state machine | **Implemented** |
|
||||
| Featured posts | Pin/unpin posts to featured collection | **Implemented** |
|
||||
| Featured tags | Manage featured hashtags | **Implemented** |
|
||||
| Profile editor | Name, summary, icon, image, attachments, broadcasts update to followers | **Implemented** |
|
||||
| Link previews | Open Graph unfurling via `unfurl.js` for timeline items | **Implemented** |
|
||||
| Infinite scroll | Unified Alpine.js component with configurable cursor parameters | **Implemented** |
|
||||
| CSRF protection | Token generation/validation on all POST routes | **Implemented** |
|
||||
| Content sanitization | `sanitize-html` on all inbound content | **Implemented** |
|
||||
| Activity log | Full inbound/outbound activity logging with TTL cleanup | **Implemented** |
|
||||
| Timeline cleanup | Retention-based pruning (`timelineRetention` config) | **Implemented** |
|
||||
| Hashtag following | Follow/unfollow hashtags, items from non-followed accounts matching tags appear in timeline | **Implemented** |
|
||||
| Public profile page | HTML fallback for actor URL when accessed from browser | **Implemented** |
|
||||
|
||||
---
|
||||
|
||||
## 10. Overall Summary
|
||||
|
||||
| Category | Score | Percentage | Notes |
|
||||
|---|---|---|---|
|
||||
| Inbound Activities | 13/21 | 62% | All high-traffic types covered |
|
||||
| Outbound Activities | 10/16 | 63% | Missing: Delete, Block, Flag, Move, Reject |
|
||||
| Dispatchers/Collections | 11/13 | 85% | Near complete |
|
||||
| Crypto/Security | 8/12 | 67% | Core signing works |
|
||||
| Object Types | 10/18 | 56% | Core types done |
|
||||
| Addressing | 2/4 | 50% | Public + DM only |
|
||||
| FEPs | 3/9 | 33% | Key FEPs implemented |
|
||||
| Infrastructure | 8/9 | 89% | Excellent |
|
||||
| **Weighted Overall** | — | **~70%** | **~95%+ of real-world fediverse traffic covered** |
|
||||
|
||||
---
|
||||
|
||||
## 11. Gap Analysis: High-Impact Improvements
|
||||
|
||||
Ordered by impact-to-effort ratio.
|
||||
|
||||
### Priority 1 — High Impact, Low Effort
|
||||
|
||||
| Gap | Impact | Effort | Details |
|
||||
|---|---|---|---|
|
||||
| **Outbound `Delete` activity** | High | Low | When a post is deleted in Indiekit, remote servers are never notified. The post remains visible on all federated instances indefinitely. Hook into Indiekit's post delete lifecycle, send `Delete(Tombstone)` to followers. |
|
||||
| **Outbound `Block` activity** | Medium | Low | Our block is local-only (`ap_blocked`). Remote servers don't know we blocked them, so they continue delivering activities. Send `Block` activity on block, `Undo(Block)` on unblock. |
|
||||
| **Unlisted addressing mode** | Medium | Low | Add a "visibility" option to the syndicator: public (default), unlisted (`to: followers, cc: PUBLIC`). Useful for posts that shouldn't appear on public timelines but are still accessible via link. |
|
||||
|
||||
### Priority 2 — Medium Impact, Medium Effort
|
||||
|
||||
| Gap | Impact | Effort | Details |
|
||||
|---|---|---|---|
|
||||
| **Question/Poll support (inbound)** | Medium | Medium | Poll posts from Mastodon render without options. Parse `Question` object's `inclusiveOptions`/`exclusiveOptions`, display vote options and results in timeline. Voting (outbound) is a separate feature. |
|
||||
| **`Flag` handler (inbound reports)** | Medium | Medium | Other servers can't send us abuse reports. Add `Flag` inbox listener, store in a `ap_reports` collection, add moderation UI tab. |
|
||||
| **Content Warning / Sensitive flag (outbound)** | Medium | Low | Inbound sensitive content is displayed with a warning. Add a "sensitive" / CW option to the compose form and syndicator so outbound posts can include content warnings. |
|
||||
| **Followers-only addressing** | Medium | Medium | Add a "followers-only" visibility option. Requires `to: followers` only, no PUBLIC. Also needs consideration for who can see the post on our own site. |
|
||||
|
||||
### Priority 3 — Low Impact
|
||||
|
||||
| Gap | Impact | Effort | Details |
|
||||
|---|---|---|---|
|
||||
| **Custom Emoji** | Low | Medium | Mastodon custom emoji renders as `:shortcode:` text. Parse `Emoji` tags, fetch images, inline-replace in content. |
|
||||
| **`Reject(Follow)` / manual approval** | Low | Medium | Currently all follows are auto-accepted. Add a "manually approves followers" mode with pending/accept/reject UI. |
|
||||
| **`Tombstone` on delete** | Low | Low | Instead of just deleting from collections, create a `Tombstone` object for the deleted resource. Mostly a federation correctness improvement. |
|
||||
| **Activity transformers** | Low | Low | Fedify's `actorDehydrator()` improves Threads compatibility. Consider enabling for broader compatibility. |
|
||||
| **FEP-8fcf Followers Sync** | Low | Low | Pass `syncCollection: true` to `sendActivity()` calls. Reduces duplicate deliveries for servers that support it. |
|
||||
| **FEP-fe34 Origin-Based Security** | Low | Low | Set `crossOrigin: "ignore"` or `"throw"` on federation options. Prevents spoofed attribution attacks. |
|
||||
|
||||
### Not Recommended (Skip)
|
||||
|
||||
| Gap | Reason |
|
||||
|---|---|
|
||||
| `EmojiReact` | Misskey/Pleroma-only, very niche |
|
||||
| `Arrive`/`Travel`/`Join`/`Leave` | Almost never seen in real fediverse |
|
||||
| `Invite`/`Offer` | Group-specific, very niche |
|
||||
| `Dislike` | Not implemented by any major fediverse software |
|
||||
| Relay support (FEP-ae0c) | Only useful at scale, not for personal sites |
|
||||
| DID-based identity (FEP-9091) | Future spec, minimal adoption |
|
||||
| Actor succession (FEP-c0e0) | Future spec, minimal adoption |
|
||||
| `ChatMessage` (LitePub DMs) | Our standard DM addressing works with all servers |
|
||||
|
||||
---
|
||||
|
||||
## 12. Data Flow Reference
|
||||
|
||||
### Outbound Activity Flow
|
||||
|
||||
```
|
||||
Indiekit blog post (JF2)
|
||||
↓
|
||||
syndicator.syndicate() [index.js]
|
||||
↓
|
||||
jf2ToAS2Activity() [lib/jf2-to-as2.js — converts JF2 → Fedify vocab objects]
|
||||
↓
|
||||
ctx.sendActivity({ identifier: handle }, "followers", activity) [Fedify]
|
||||
↓
|
||||
Redis queue [or InProcessMessageQueue]
|
||||
↓
|
||||
HTTP POST to follower inboxes [signed with RSA/Ed25519 by Fedify]
|
||||
```
|
||||
|
||||
### Inbound Activity Flow
|
||||
|
||||
```
|
||||
Remote server HTTP POST to /{mount}/inbox [HTTP Signature verified by Fedify]
|
||||
↓
|
||||
federation-bridge.js [reconstructs body if Express consumed stream, uses req.originalUrl]
|
||||
↓
|
||||
Fedify matches activity type → calls registered listener
|
||||
↓
|
||||
inbox-listeners.js [authenticated document loader for all remote fetches]
|
||||
↓
|
||||
MongoDB storage [ap_followers, ap_timeline, ap_notifications, ap_messages, ap_activities]
|
||||
↓
|
||||
Admin UI renders data [reader, notifications, messages, moderation]
|
||||
```
|
||||
|
||||
### Reader Timeline Pipeline
|
||||
|
||||
```
|
||||
Raw items from ap_timeline
|
||||
↓
|
||||
applyTabFilter() [notes/articles/boosts/media/replies — lib/item-processing.js]
|
||||
↓
|
||||
loadModerationData() [load muted URLs, keywords, blocked URLs]
|
||||
↓
|
||||
postProcessItems() [filter muted/blocked, strip quote refs, build interaction map]
|
||||
↓
|
||||
renderItemCards() [server-side Nunjucks → HTML for AJAX responses]
|
||||
↓
|
||||
Alpine.js infinite scroll [apInfiniteScroll component — assets/reader-infinite-scroll.js]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. MongoDB Collections Reference
|
||||
|
||||
| Collection | Records | Indexes | TTL |
|
||||
|---|---|---|---|
|
||||
| `ap_followers` | Accounts following us | `actorUrl` (unique) | No |
|
||||
| `ap_following` | Accounts we follow | `actorUrl` (unique) | No |
|
||||
| `ap_activities` | Activity log | `direction`, `type`, `actorUrl`, `objectUrl`, `receivedAt` | Yes (`activityRetentionDays`, default 90) |
|
||||
| `ap_keys` | Crypto key pairs | `type` (rsa/ed25519) | No |
|
||||
| `ap_kv` | Fedify KV store | `_id` (key path) | Yes (Fedify-managed) |
|
||||
| `ap_profile` | Actor profile (single doc) | — | No |
|
||||
| `ap_featured` | Pinned posts | `postUrl` | No |
|
||||
| `ap_featured_tags` | Featured hashtags | `tag` | No |
|
||||
| `ap_timeline` | Reader timeline | `uid` (unique), `published`, `author.url`, `type` | No (manual cleanup via `timelineRetention`) |
|
||||
| `ap_notifications` | Notifications | `uid` (unique), `type`, `read`, `createdAt` | Yes (`notificationRetentionDays`, default 30) |
|
||||
| `ap_messages` | Direct messages | `uid` (unique), `conversationId`+`published`, `read`, `direction` | Yes (reuses `notificationRetentionDays`) |
|
||||
| `ap_muted` | Muted actors/keywords | `url` or `keyword` | No |
|
||||
| `ap_blocked` | Blocked actors | `url` | No |
|
||||
| `ap_interactions` | Like/boost tracking | `objectUrl`, `type` | No |
|
||||
| `ap_followed_tags` | Hashtags we follow | `tag` | No |
|
||||
|
||||
---
|
||||
|
||||
## 14. Configuration Reference
|
||||
|
||||
```javascript
|
||||
{
|
||||
mountPath: "/activitypub", // URL prefix for all routes
|
||||
actor: {
|
||||
handle: "rick", // Fediverse username (@rick@rmendes.net)
|
||||
name: "Ricardo Mendes", // Display name (seeds profile on first run)
|
||||
summary: "", // Bio (seeds profile)
|
||||
icon: "", // Avatar URL (seeds profile)
|
||||
},
|
||||
checked: true, // Syndicator checked by default in Micropub UI
|
||||
alsoKnownAs: "", // Mastodon migration alias (for Move activities)
|
||||
activityRetentionDays: 90, // TTL for ap_activities (0 = forever)
|
||||
storeRawActivities: false, // Store full JSON of inbound activities
|
||||
redisUrl: "", // Redis for delivery queue (empty = in-process)
|
||||
parallelWorkers: 5, // Parallel delivery workers (with Redis)
|
||||
actorType: "Person", // Person | Service | Organization | Group | Application
|
||||
logLevel: "warning", // Fedify log level: debug | info | warning | error | fatal
|
||||
timelineRetention: 1000, // Max timeline items (0 = unlimited)
|
||||
notificationRetentionDays: 30, // Days to keep notifications (0 = forever)
|
||||
debugDashboard: false, // Enable @fedify/debugger at {mount}/__debug__/
|
||||
debugPassword: "", // Password for debug dashboard
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
*This audit reflects the state of the plugin at version 2.9.2. It should be updated when new features are added or when Fedify releases new capabilities.*
|
||||
@@ -1,513 +0,0 @@
|
||||
# Explore Page Tabbed Redesign Implementation Plan
|
||||
|
||||
Created: 2026-02-28
|
||||
Status: VERIFIED
|
||||
Approved: Yes
|
||||
Iterations: 0
|
||||
Worktree: No
|
||||
|
||||
> **Status Lifecycle:** PENDING → COMPLETE → VERIFIED
|
||||
> **Iterations:** Tracks implement→verify cycles (incremented by verify phase)
|
||||
>
|
||||
> - PENDING: Initial state, awaiting implementation
|
||||
> - COMPLETE: All tasks implemented
|
||||
> - VERIFIED: All checks passed
|
||||
>
|
||||
> **Approval Gate:** Implementation CANNOT proceed until `Approved: Yes`
|
||||
> **Worktree:** No — working directly on current branch
|
||||
|
||||
## Summary
|
||||
|
||||
**Goal:** Replace the cramped deck/column layout on the ActivityPub explore page with a full-width tabbed design. Three tab types: Search (always first, not removable), Instance (pinned instances with local/federated badge), and Hashtag (aggregated across all pinned instances). New `ap_explore_tabs` collection replaces `ap_decks` (clean start, no migration).
|
||||
|
||||
**Architecture:** Server-rendered tab navigation with Alpine.js for tab content loading. Each tab loads its own timeline via the existing explore API (instance tabs) or a new hashtag aggregation API (hashtag tabs). The tab bar is a horizontal scrollable row with the Search tab always first, followed by user-ordered Instance and Hashtag tabs. Tab reordering uses server-side PATCH endpoint. No limit on tab count.
|
||||
|
||||
**Tech Stack:** Express routes (Node.js), Nunjucks templates, Alpine.js 3.x for client-side interactivity, MongoDB for `ap_explore_tabs` collection, Mastodon API v1 for timelines.
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Replace deck grid layout with full-width tab navigation
|
||||
- New `ap_explore_tabs` collection with schema: `{ type, domain?, scope?, hashtag?, order, addedAt }`
|
||||
- Search tab: existing instance search + optional hashtag field switching between `/timelines/public` and `/timelines/tag/{hashtag}`
|
||||
- Instance tabs: full-width timeline with local/federated scope badge
|
||||
- Hashtag tabs: parallel queries across all pinned instances, merge by date, dedup by post URL
|
||||
- Tab CRUD API: add, remove, reorder
|
||||
- Tab reordering UI: up/down arrow buttons (simpler, more accessible than drag-and-drop)
|
||||
- Each tab loads independently with infinite scroll
|
||||
- Replace deck-related Alpine.js components with tab-based ones
|
||||
- Replace deck CSS with tab CSS
|
||||
- Update i18n locale strings
|
||||
- Note: The responsive CSS fix (`width: 100%` + `box-sizing: border-box` on `.ap-lookup__input` and `.ap-explore-form__input`) was already committed prior to this plan — no action needed
|
||||
|
||||
### Out of Scope
|
||||
|
||||
- Drag-and-drop tab reordering (deferred — up/down arrows first, DnD can be added later)
|
||||
- Per-instance hashtag filter within instance tabs (deferred per user decision)
|
||||
- Migration of old `ap_decks` data (clean start per user decision)
|
||||
- Changes to the main reader timeline, tag timeline, or notifications
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js >= 22 (already in place)
|
||||
- `@rmdes/indiekit-endpoint-activitypub` repo at version 2.0.36
|
||||
- MongoDB with existing ActivityPub collections
|
||||
|
||||
## Context for Implementer
|
||||
|
||||
> This section is critical for cross-session continuity.
|
||||
|
||||
- **Patterns to follow:**
|
||||
- Controller pattern: `lib/controllers/explore.js` — exports factory functions `controllerName(mountPath)` returning `async (request, response, next) => { ... }`
|
||||
- API JSON endpoint pattern: `exploreApiController()` at `explore.js:260` — renders partials server-side via `request.app.render()`, returns `{ html, maxId }`
|
||||
- CSRF validation pattern: `lib/controllers/decks.js:45` — `validateToken(request)` from `../csrf.js`
|
||||
- SSRF prevention: `validateInstance()` at `explore.js:22` — validates hostnames, blocks private IPs
|
||||
- Alpine.js registration: `reader-decks.js:9` — `document.addEventListener("alpine:init", () => { Alpine.data(...) })`
|
||||
- CSS conventions: Uses Indiekit theme custom properties (`--color-on-background`, `--color-primary`, etc.)
|
||||
- Tab styling: Existing `.ap-tabs` / `.ap-tab` / `.ap-tab--active` CSS at `reader.css:91-134` (reused and extended)
|
||||
|
||||
- **Conventions:**
|
||||
- ESM modules (`import`/`export`)
|
||||
- Dates stored as ISO 8601 strings: `new Date().toISOString()`
|
||||
- Template variables must avoid collisions with Nunjucks macro names imported in `default.njk` (e.g., `tag` collides with the `tag` macro — use `hashtag` instead)
|
||||
- Express 5: No `redirect("back")` — use explicit paths
|
||||
- sanitize-html for any remote content displayed in HTML
|
||||
|
||||
- **Key files:**
|
||||
- `index.js` — Plugin entry; collection registration (line 888), route registration (line 239-246), index creation (line 1036-1039)
|
||||
- `lib/controllers/explore.js` — Current explore controller (405 lines) with `exploreController`, `exploreApiController`, `instanceSearchApiController`, `instanceCheckApiController`, `popularAccountsApiController`, and helper `mapMastodonStatusToItem`
|
||||
- `lib/controllers/decks.js` — Current deck CRUD (137 lines): `listDecksController`, `addDeckController`, `removeDeckController`
|
||||
- `views/activitypub-explore.njk` — Current explore template (218 lines) with Search tab and Decks tab
|
||||
- `assets/reader-decks.js` — Alpine components: `apDeckToggle`, `apDeckColumn` (212 lines)
|
||||
- `assets/reader-infinite-scroll.js` — Alpine components: `apExploreScroll`, `apInfiniteScroll` (183 lines)
|
||||
- `assets/reader-autocomplete.js` — Alpine components: `apInstanceSearch`, `apPopularAccounts` (214 lines)
|
||||
- `assets/reader.css` — All styles (2248 lines); deck styles at lines 2063-2248
|
||||
- `locales/en.json` — i18n strings; explore section at line 229
|
||||
|
||||
- **Gotchas:**
|
||||
- Template variable `tag` is shadowed by Nunjucks macro from `default.njk` — always use `hashtag` in template context
|
||||
- The `.ap-tabs` CSS class already exists and is used for the current Search/Decks tab bar — it will be extended for the new design
|
||||
- `reader-infinite-scroll.js` contains `apExploreScroll` (for explore page) AND `apInfiniteScroll` (for main reader timeline) — only the former is being replaced
|
||||
- The `ap_kv` collection is used for FediDB caching — not related to deck/tab storage
|
||||
- Mastodon hashtag timeline API: `GET /api/v1/timelines/tag/{hashtag}?local=true|false&limit=20&max_id=X` — public, no auth needed
|
||||
|
||||
- **Domain context:**
|
||||
- The explore page lets users browse public timelines from remote Mastodon-compatible instances
|
||||
- Instance tabs pin specific instances so users don't re-search each time
|
||||
- Hashtag tabs aggregate a hashtag across ALL pinned instances in parallel (e.g., #indieweb from mastodon.social + fosstodon.org + ...)
|
||||
- The Search tab is the entry point for discovering new instances + one-off browsing
|
||||
|
||||
## Runtime Environment
|
||||
|
||||
- **Start command:** Deployed via Cloudron (`/app/pkg/start.sh`); locally via `node --loader` or through Indiekit dev server
|
||||
- **Port:** 8080 (Indiekit), 3000 (nginx proxy)
|
||||
- **Deploy path:** `indiekit-cloudron/Dockerfile` installs from npm, `cloudron build --no-cache && cloudron update --app rmendes.net --no-backup`
|
||||
- **Health check:** `curl -s https://rmendes.net/activitypub/ | head -20` (should return dashboard HTML)
|
||||
- **Restart procedure:** `cloudron restart --app rmendes.net` or full rebuild cycle
|
||||
|
||||
## Feature Inventory — Files Being Replaced
|
||||
|
||||
This is a **refactoring task** — the deck system is being replaced by a tab system.
|
||||
|
||||
### Files Being Replaced
|
||||
|
||||
| Old File | Functions/Features | Mapped to Task |
|
||||
| --- | --- | --- |
|
||||
| `lib/controllers/decks.js` | `listDecksController()`, `addDeckController()`, `removeDeckController()` — CRUD for `ap_decks` | Task 2 (replaced by tab CRUD) |
|
||||
| `lib/controllers/explore.js` | `exploreController()` — renders explore page with deck data | Task 3, Task 5 |
|
||||
| `lib/controllers/explore.js` | `exploreApiController()` — AJAX infinite scroll for single instance | Task 5, Task 6 |
|
||||
| `lib/controllers/explore.js` | `instanceSearchApiController()` — FediDB autocomplete | Task 3 (kept as-is) |
|
||||
| `lib/controllers/explore.js` | `instanceCheckApiController()` — timeline support check | Task 3 (kept as-is) |
|
||||
| `lib/controllers/explore.js` | `popularAccountsApiController()` — popular accounts API | Task 3 (kept as-is) |
|
||||
| `lib/controllers/explore.js` | `validateInstance()` — SSRF-safe hostname validation | Task 2 (reused as-is) |
|
||||
| `lib/controllers/explore.js` | `mapMastodonStatusToItem()` — status-to-timeline-item mapping | Task 5, Task 6 (reused as-is) |
|
||||
| `views/activitypub-explore.njk` | Search form + autocomplete | Task 3 |
|
||||
| `views/activitypub-explore.njk` | Deck grid + deck columns | Task 4 |
|
||||
| `views/activitypub-explore.njk` | Instance timeline + infinite scroll | Task 5 |
|
||||
| `assets/reader-decks.js` | `apDeckToggle` — star/add-to-deck button | Task 4 (replaced by "Pin" button) |
|
||||
| `assets/reader-decks.js` | `apDeckColumn` — individual deck column with infinite scroll | Task 5 (replaced by tab panel) |
|
||||
| `assets/reader-infinite-scroll.js` | `apExploreScroll` — explore page infinite scroll | Task 5 (replaced by tab-scoped scroll) |
|
||||
| `assets/reader-infinite-scroll.js` | `apInfiniteScroll` — main reader timeline scroll | NOT TOUCHED (kept as-is) |
|
||||
| `assets/reader-autocomplete.js` | `apInstanceSearch` — instance autocomplete | Task 3 (extended with hashtag field) |
|
||||
| `assets/reader-autocomplete.js` | `apPopularAccounts` — popular account autocomplete | NOT TOUCHED (kept as-is) |
|
||||
| `assets/reader.css` | `.ap-deck-*` styles (lines 2063-2248) | Task 4 (replaced by tab styles) |
|
||||
| `assets/reader.css` | `.ap-explore-deck-toggle` styles (lines 2063-2103) | Task 4 (replaced by "Pin" styles) |
|
||||
| `assets/reader.css` | `.ap-tabs` styles (lines 91-134) | Task 4 (extended for dynamic tabs) |
|
||||
| `locales/en.json` | `explore.tabs.*`, `explore.deck.*` strings | Task 3 (updated strings) |
|
||||
| `index.js` | `ap_decks` collection registration + indexes (line 888, 1036-1039) | Task 1 |
|
||||
| `index.js` | Deck route registration (lines 244-246) | Task 2 |
|
||||
|
||||
### Feature Mapping Verification
|
||||
|
||||
- [x] All old files listed above
|
||||
- [x] All functions/classes identified
|
||||
- [x] Every feature has a task number
|
||||
- [x] No features accidentally omitted
|
||||
|
||||
## Progress Tracking
|
||||
|
||||
**MANDATORY: Update this checklist as tasks complete. Change `[ ]` to `[x]`.**
|
||||
|
||||
- [x] Task 1: Collection setup — `ap_explore_tabs` replaces `ap_decks`
|
||||
- [x] Task 2: Tab CRUD API — add, remove, reorder endpoints
|
||||
- [x] Task 3: Search tab — form with hashtag field, updated template
|
||||
- [x] Task 4: Tab bar UI — dynamic tabs with scope badges, reordering, pin button
|
||||
- [x] Task 5: Instance tab panel — full-width timeline with infinite scroll
|
||||
- [x] Task 6: Hashtag tab panel — cross-instance aggregation
|
||||
- [x] Task 7: Cleanup — remove old deck code, update CSS, update locales
|
||||
|
||||
**Total Tasks:** 7 | **Completed:** 7 | **Remaining:** 0
|
||||
|
||||
## Implementation Tasks
|
||||
|
||||
### Task 1: Collection Setup — `ap_explore_tabs` Replaces `ap_decks`
|
||||
|
||||
**Objective:** Register the new `ap_explore_tabs` MongoDB collection with proper indexes, replacing `ap_decks`. Clean start — no migration of old data.
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `index.js` — Replace `ap_decks` collection registration with `ap_explore_tabs`, update `this._collections`, create indexes
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- Schema: `{ type: "instance"|"hashtag", domain?: string, scope?: "local"|"federated", hashtag?: string, order: number, addedAt: string (ISO 8601) }`
|
||||
- Indexes: single unique compound index on `(type, domain, scope, hashtag)`. **CRITICAL: All insertions MUST explicitly set ALL four fields** — instance tabs set `hashtag: null`, hashtag tabs set `domain: null, scope: null`. MongoDB treats missing fields and explicit `null` differently in compound indexes, so omitting a field would bypass the uniqueness constraint and allow duplicates.
|
||||
- `order` field: integer, used for user-controlled tab ordering. New tabs get `order = max(existing orders) + 1`. Use `findOneAndUpdate` with `sort: { order: -1 }` to atomically determine the next order value (prevents race conditions on concurrent additions).
|
||||
- No limit on tab count (removed the `MAX_DECKS = 8` restriction)
|
||||
- Remove old `ap_decks` references from collection registration and `this._collections`
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] `ap_explore_tabs` collection registered in `index.js` (replacing `ap_decks`)
|
||||
- [ ] `this._collections` includes `ap_explore_tabs` instead of `ap_decks`
|
||||
- [ ] Unique compound index on `(type, domain, scope, hashtag)` created
|
||||
- [ ] Index on `order` field created for efficient sorting
|
||||
- [ ] Old `ap_decks` references completely removed from `index.js`
|
||||
|
||||
**Verify:**
|
||||
|
||||
- `grep -r "ap_decks" index.js` returns nothing
|
||||
- `grep "ap_explore_tabs" index.js` shows collection registration, `this._collections`, and index creation
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Tab CRUD API — Add, Remove, Reorder Endpoints
|
||||
|
||||
**Objective:** Create the tab management API replacing the old deck CRUD. Supports adding instance tabs, adding hashtag tabs, removing any tab, and reordering tabs.
|
||||
|
||||
**Dependencies:** Task 1
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `lib/controllers/tabs.js` — New tab CRUD controller with `listTabsController`, `addTabController`, `removeTabController`, `reorderTabsController`, and `validateHashtag()` helper
|
||||
- Modify: `index.js` — Replace deck route imports and registrations with tab routes
|
||||
- Delete: `lib/controllers/decks.js` — Deleted here when replaced by tabs.js (Task 7 only verifies it's gone)
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- `POST /admin/reader/api/tabs` — Add tab. Body: `{ type: "instance"|"hashtag", domain?, scope?, hashtag? }`. Validates domain via `validateInstance()`, validates hashtag via `validateHashtag()`. Auto-assigns `order = max(existing) + 1`. **CRITICAL: Insertions MUST explicitly set all four indexed fields** — instance tabs: `{ type, domain, scope, hashtag: null, order, addedAt }`, hashtag tabs: `{ type, domain: null, scope: null, hashtag, order, addedAt }`.
|
||||
- `POST /admin/reader/api/tabs/remove` — Remove tab. Body: `{ type, domain?, scope?, hashtag? }`. After removal, re-compacts order numbers to avoid gaps.
|
||||
- `PATCH /admin/reader/api/tabs/reorder` — Reorder tabs. Body: `{ tabIds: [id1, id2, ...] }` — array of MongoDB `_id` strings in desired order. Sets `order = index` for each.
|
||||
- `GET /admin/reader/api/tabs` — List all tabs sorted by `order` ascending.
|
||||
- **`validateHashtag()` helper** (new, alongside `validateInstance()`): (1) Strip leading `#` characters, (2) Reject if empty after stripping, (3) Validate against `/^[\w]+$/` (alphanumeric + underscore only — matching Mastodon's hashtag rules), (4) Enforce max length of 100 chars. Call this in the add-tab endpoint for hashtag tabs AND in the hashtag explore endpoint (Task 6).
|
||||
- All POST/PATCH endpoints require CSRF token validation via `validateToken(request)`.
|
||||
- All domain inputs validated via `validateInstance()` (imported from explore.js).
|
||||
- All tab routes registered in the `routes` getter (not `routesPublic`) to ensure IndieAuth authentication protects them.
|
||||
- Reuse the existing CSRF and validation patterns from `decks.js`.
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] `GET /admin/reader/api/tabs` returns all tabs sorted by order
|
||||
- [ ] `POST /admin/reader/api/tabs` with `{ type: "instance", domain: "mastodon.social", scope: "local" }` creates a tab
|
||||
- [ ] `POST /admin/reader/api/tabs` with `{ type: "hashtag", hashtag: "indieweb" }` creates a tab
|
||||
- [ ] Duplicate tabs rejected with 409
|
||||
- [ ] `POST /admin/reader/api/tabs/remove` removes tab and re-compacts order
|
||||
- [ ] `PATCH /admin/reader/api/tabs/reorder` updates order for all specified tabs
|
||||
- [ ] `validateHashtag()` helper rejects empty, non-alphanumeric, and >100 char hashtags
|
||||
- [ ] Hashtag tabs insert with explicit `domain: null, scope: null`; instance tabs insert with explicit `hashtag: null`
|
||||
- [ ] CSRF validation on all mutating endpoints
|
||||
- [ ] SSRF validation on domain inputs
|
||||
- [ ] Old deck routes removed from `index.js`
|
||||
- [ ] `lib/controllers/decks.js` deleted
|
||||
|
||||
**Verify:**
|
||||
|
||||
- `grep -r "ap_decks\|decks.js" index.js` returns nothing
|
||||
- `grep "tabs.js\|api/tabs" index.js` shows new routes
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Search Tab — Form with Hashtag Field
|
||||
|
||||
**Objective:** Update the Search tab's search form to add an optional hashtag field. When a hashtag is entered, the API call switches from `/timelines/public` to `/timelines/tag/{hashtag}`. Update both `exploreController()` (initial page load) and `exploreApiController()` (AJAX infinite scroll) to handle the `hashtag` query param.
|
||||
|
||||
**Dependencies:** Task 1
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `views/activitypub-explore.njk` — Add hashtag input field to search form (within the Search tab content section only; leave the tab nav bar unchanged — Task 4 replaces it entirely). Pass hashtag value to infinite scroll data attributes.
|
||||
- Modify: `lib/controllers/explore.js` — Add `hashtag` query param handling in BOTH `exploreController()` AND `exploreApiController()`; change API URL construction to use `/timelines/tag/{hashtag}` when hashtag is provided; update template variables (remove deck references). Validate hashtag via `validateHashtag()` from `tabs.js`.
|
||||
- Modify: `assets/reader-autocomplete.js` — Extend `apInstanceSearch` to handle hashtag field state
|
||||
- Modify: `locales/en.json` — Update explore locale strings (remove deck strings, add tab/hashtag strings)
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- The hashtag field is a plain text input next to the instance field. When filled, the explore API fetches `/api/v1/timelines/tag/{encodedHashtag}?local=true|false` instead of `/api/v1/timelines/public`
|
||||
- The `hashtag` parameter is stripped of leading `#` and URL-encoded
|
||||
- **Both `exploreController` and `exploreApiController` must handle the hashtag param** — without this, infinite scroll on search tab with hashtag would revert to the public timeline after the first page
|
||||
- The template's infinite scroll data attributes must pass the hashtag value so the AJAX endpoint receives it on subsequent pages
|
||||
- The Search tab is always the first tab and cannot be removed
|
||||
- **Do NOT modify the tab navigation bar** (lines 14-24 of template) — leave it as-is in this task; Task 4 replaces it entirely with the dynamic tab bar. Only modify the Search tab content section.
|
||||
- Keep `instanceSearchApiController`, `instanceCheckApiController`, `popularAccountsApiController` unchanged in explore.js
|
||||
- Remove `decks`, `deckCount`, `isInDeck` from the controller's template variables
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] Search form has an optional hashtag text input field
|
||||
- [ ] When hashtag is provided, `exploreController` fetches from `/timelines/tag/{hashtag}` instead of `/timelines/public`
|
||||
- [ ] When hashtag is provided, `exploreApiController` also fetches from `/timelines/tag/{hashtag}` (infinite scroll stays in hashtag mode)
|
||||
- [ ] Hashtag is validated via `validateHashtag()`, URL-encoded, and stripped of leading `#`
|
||||
- [ ] Scope radio buttons still work with hashtag mode
|
||||
- [ ] Infinite scroll data attributes pass the hashtag value to the AJAX endpoint
|
||||
- [ ] i18n strings updated: deck strings removed, hashtag placeholder string added
|
||||
|
||||
**Verify:**
|
||||
|
||||
- Open explore page — Search tab renders with instance + hashtag fields
|
||||
- Submit with instance `mastodon.social` + hashtag `indieweb` — results load from tag timeline
|
||||
- Scroll down — infinite scroll continues fetching from tag timeline (not reverting to public)
|
||||
- Submit with instance only (no hashtag) — results load from public timeline (unchanged behavior)
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Tab Bar UI — Dynamic Tabs with Scope Badges, Reordering, Pin Button
|
||||
|
||||
**Objective:** Build the dynamic tab bar that shows Search + user-added Instance/Hashtag tabs. Each Instance tab shows domain + scope badge. Each Hashtag tab shows `#tag`. Tabs have close buttons and up/down reorder arrows. Replace the star/deck-toggle button with a "Pin as tab" button on search results. Add UI for creating hashtag tabs.
|
||||
|
||||
**Dependencies:** Task 2, Task 3
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `views/activitypub-explore.njk` — Replace static Search/Decks tab nav with dynamic tab bar; add "Pin as tab" button for search results; add "Add hashtag tab" UI
|
||||
- Create: `assets/reader-tabs.js` — Alpine.js component `apExploreTabs` for tab management (switching, adding, removing, reordering). **Guard init with DOM check:** `if (!document.querySelector('.ap-explore-tabs')) return;` — since the script loads on all reader pages via the shared layout, this prevents console errors on non-explore pages.
|
||||
- Modify: `assets/reader.css` — Remove `.ap-deck-*` styles, add `.ap-explore-tab-*` styles for dynamic tabs with badges, controls, and overflow handling
|
||||
- Modify: `views/layouts/ap-reader.njk` — Replace `reader-decks.js` script tag with `reader-tabs.js`. IMPORTANT: `reader-tabs.js` must load BEFORE the Alpine CDN script (same `defer` pattern as existing component scripts) since it registers Alpine data components via the `alpine:init` event.
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- Tab bar: horizontal scrollable row. Search tab first (no close button, no reorder). All other tabs (Instance + Hashtag) sorted by `order` field **regardless of type** — tabs are freely interleaved, not grouped by type.
|
||||
- Instance tab label: `{domain}` with a colored scope badge (local = blue, federated = purple) — reuse `.ap-deck-column__scope-badge` colors
|
||||
- Hashtag tab label: `#{hashtag}`
|
||||
- Each non-Search tab has: close button (×) and up/down arrows for reordering
|
||||
- "Pin as tab" button replaces the star/deck-toggle button in search results area
|
||||
- **"Add hashtag tab" UI:** A `+#` button at the end of the tab bar opens a small inline form (text input + confirm button) to add a hashtag tab. On submit, calls `POST /admin/reader/api/tabs` with `{ type: "hashtag", hashtag: value }`. The new tab appears in the tab bar with `#{hashtag}` label.
|
||||
- When a tab is clicked, the tab content area switches to show that tab's timeline (Alpine.js handles visibility)
|
||||
- The `apExploreTabs` Alpine component manages: active tab state, tab list from server, add/remove/reorder API calls
|
||||
- Tab data is loaded via `GET /admin/reader/api/tabs` on page init
|
||||
- **Reorder debouncing:** Debounce reorder API calls (500ms after last arrow click) so rapid clicks batch into a single request. This prevents race conditions from rapid successive clicks.
|
||||
- **Tab bar overflow:** When tabs overflow horizontally, show fade gradients at edges to indicate scrollable content. Tab labels use `text-overflow: ellipsis` with `max-width: 150px` to truncate long domain names. Reorder arrows only visible on hover (desktop) or on long-press (mobile) to save space.
|
||||
- **Accessibility (WAI-ARIA Tabs Pattern):** Tab bar uses `role="tablist"`, each tab uses `role="tab"` with `aria-selected`, `aria-controls` pointing to tab panel. Tab panels use `role="tabpanel"`. Arrow keys navigate between tabs.
|
||||
- **CSRF 403 handling:** When a tab API call returns 403, show a clear error: "Session expired — please refresh the page." This handles stale CSRF tokens on long-lived pages.
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] Tab bar shows Search tab + all user-created tabs from `ap_explore_tabs`
|
||||
- [ ] Instance tabs display domain + colored scope badge (local=blue, federated=purple)
|
||||
- [ ] Hashtag tabs display `#{hashtag}`
|
||||
- [ ] Clicking a tab switches the visible content panel
|
||||
- [ ] Close button (×) on non-Search tabs calls remove API and removes tab from bar
|
||||
- [ ] Up/down arrows on non-Search tabs call reorder API and move tab in bar (debounced 500ms)
|
||||
- [ ] "Pin as tab" button in search results adds instance+scope to tabs via add API
|
||||
- [ ] "Add hashtag tab" button (`+#`) opens inline form to add a hashtag tab
|
||||
- [ ] Tab bar overflow: fade gradients at edges, ellipsis on long labels
|
||||
- [ ] Tab bar follows WAI-ARIA Tabs Pattern (role=tablist, role=tab, role=tabpanel, aria-selected, aria-controls)
|
||||
- [ ] Tab bar is keyboard-navigable (arrow keys between tabs)
|
||||
- [ ] Alpine component guarded with DOM check for non-explore pages
|
||||
- [ ] Old `.ap-deck-*` CSS removed, new tab styles added
|
||||
- [ ] `reader-decks.js` script tag replaced with `reader-tabs.js` in layout
|
||||
|
||||
**Verify:**
|
||||
|
||||
- Open explore page — tab bar visible with Search tab
|
||||
- Browse an instance in Search tab — "Pin as tab" button appears
|
||||
- Click "Pin as tab" — new Instance tab appears in tab bar with scope badge
|
||||
- Click `+#` button — hashtag input form appears, enter "indieweb", confirm — hashtag tab appears as `#indieweb`
|
||||
- Click the Instance tab — content area switches (empty initially, loaded in Task 5)
|
||||
- Close button removes the tab
|
||||
- Up/down arrows reorder tabs (verify via page reload)
|
||||
- Open a non-explore reader page (e.g., timeline) — no console errors from reader-tabs.js
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Instance Tab Panel — Full-Width Timeline with Infinite Scroll
|
||||
|
||||
**Objective:** When an Instance tab is active, load and display the full-width timeline from that instance with infinite scroll. Reuses `mapMastodonStatusToItem()` and the `exploreApiController` pattern.
|
||||
|
||||
**Dependencies:** Task 4
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `views/activitypub-explore.njk` — Add instance tab panel template section (conditionally visible based on active tab)
|
||||
- Modify: `assets/reader-tabs.js` — Add timeline loading logic to `apExploreTabs` component for instance tab activation (fetch + render + infinite scroll)
|
||||
- Modify: `lib/controllers/explore.js` — Ensure `exploreApiController` works for tab-driven requests (may need to accept hashtag param for search tab hashtag mode — already handled in Task 3)
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- When an instance tab becomes active, if it hasn't loaded yet, fetch the first page from `GET /admin/reader/api/explore?instance={domain}&scope={scope}`
|
||||
- Infinite scroll uses IntersectionObserver on a sentinel element within the tab panel (same pattern as `apDeckColumn` in `reader-decks.js:100-118`)
|
||||
- **Tab content cache:** Cached in Alpine state — switching back to a tab shows previously loaded content without re-fetching. **Bounded to last 5 tabs** to prevent memory growth on mobile — when a 6th tab loads, the oldest cached tab's content is cleared (will re-fetch on next activation). Only the first page is cached; accumulated infinite scroll content is discarded on eviction.
|
||||
- Each tab panel shows loading spinner, error state, retry button (re-fetches without full page reload), and empty state (same states as the old deck column)
|
||||
- **AbortController:** Each tab's loading state includes an AbortController. When switching away from a loading tab, the in-flight client-side fetch is aborted. When switching back, if content wasn't loaded (cache miss), a fresh request starts. This prevents abandoned HTTP connections from piling up (especially important for hashtag tabs in Task 6).
|
||||
- Full-width layout — no cramped columns, content fills the available width
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] Clicking an Instance tab loads the first page of posts from the remote instance
|
||||
- [ ] Posts display in full-width layout using `ap-item-card.njk` partial
|
||||
- [ ] Infinite scroll loads more posts when scrolling near the bottom
|
||||
- [ ] Loading spinner shown during initial load
|
||||
- [ ] Error state with retry button shown on fetch failure (retry re-fetches without full page reload)
|
||||
- [ ] Empty state shown when no posts available
|
||||
- [ ] Switching away and back to a tab preserves already-loaded content (bounded to last 5 tabs)
|
||||
- [ ] Switching away from a loading tab aborts the in-flight fetch (AbortController)
|
||||
|
||||
**Verify:**
|
||||
|
||||
- Pin `mastodon.social` (local) as a tab
|
||||
- Click the tab — posts load in full-width layout
|
||||
- Scroll down — more posts load via infinite scroll
|
||||
- Switch to Search tab and back — previously loaded posts still visible
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Hashtag Tab Panel — Cross-Instance Aggregation
|
||||
|
||||
**Objective:** When a Hashtag tab is active, query the hashtag timeline from pinned instance tabs in parallel (capped at 10), merge results by date, and deduplicate by post URL. Uses per-instance cursor pagination for correct multi-source paging.
|
||||
|
||||
**Dependencies:** Task 2, Task 5
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `lib/controllers/hashtag-explore.js` — New API endpoint `hashtagExploreApiController` that takes a hashtag and per-instance cursor map, queries pinned instances in parallel, merges, deduplicates, and paginates
|
||||
- Modify: `index.js` — Register the new hashtag explore API route
|
||||
- Modify: `assets/reader-tabs.js` — Add hashtag tab loading logic (different API endpoint than instance tabs); manages per-instance cursor state client-side
|
||||
- Modify: `views/activitypub-explore.njk` — Add hashtag tab panel template section with source instances info line
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- `GET /admin/reader/api/explore/hashtag?hashtag={tag}&cursors={json}` — New endpoint
|
||||
- **`MAX_HASHTAG_INSTANCES = 10`**: Hard cap on the number of instances queried per hashtag request. Queries the first 10 instance tabs by `order`. If more exist, the response includes `{ instancesQueried: 10, instancesTotal: N }` so the UI can show "Searching 10 of N instances".
|
||||
- **Hashtag validation:** Validate hashtag via `validateHashtag()` from `tabs.js` before constructing remote API URLs. Reject invalid hashtags with 400.
|
||||
- Reads instance tabs from `ap_explore_tabs` where `type === "instance"`, capped at 10 by `order`, then queries each instance's `/api/v1/timelines/tag/{hashtag}?local={scope}&limit=20` in parallel using `Promise.allSettled()`
|
||||
- Results merged into a single array, sorted by `published` descending
|
||||
- Deduplication by `uid` (post URL) — first occurrence wins (most recent fetch)
|
||||
- **Per-instance cursor pagination:** The `cursors` query param is a JSON-encoded map of `{ domain: max_id }` pairs. On each request, each instance is queried with its own `max_id` from the cursor map. The response returns an updated cursor map reflecting the last item from each instance's results. The client stores this cursor map in Alpine state and sends it with the next "load more" request. This ensures correct pagination without missed or duplicate posts across instances with different timeline velocities.
|
||||
- **Processing pipeline order:** (1) Fetch from all instances in parallel, (2) Merge by published date, (3) Dedup by URL, (4) Slice to page_size (20), (5) THEN render HTML via `request.app.render()` only for the returned items. This prevents wasting CPU rendering items that will be discarded.
|
||||
- **Per-instance status in response metadata:** Response includes `sources` map: `{ "mastodon.social": "ok", "pixelfed.social": "error:404" }`. The hashtag tab panel shows a line like "Searching #indieweb across 3 instances: mastodon.social, fosstodon.org, ..." and "3 of 5 instances responded" when some fail. This makes the implicit coupling between instance tabs and hashtag tabs explicit.
|
||||
- Timeout per instance: 10s (same as existing `FETCH_TIMEOUT_MS`). Failed instances excluded from results but reported in `sources`.
|
||||
- If no instance tabs exist, returns empty results with a message "Pin some instances first"
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] `GET /admin/reader/api/explore/hashtag?hashtag=indieweb` returns posts from pinned instances (up to 10)
|
||||
- [ ] Hashtag validated via `validateHashtag()`, invalid hashtags return 400
|
||||
- [ ] Results sorted by published date descending
|
||||
- [ ] Duplicate posts (same URL from multiple instances) deduplicated
|
||||
- [ ] Per-instance status returned in response metadata (`sources` map)
|
||||
- [ ] Hashtag tab panel shows "Searching #tag across N instances: domain1, domain2, ..."
|
||||
- [ ] Infinite scroll works with per-instance cursor map pagination (no duplicates or gaps between pages)
|
||||
- [ ] Maximum 10 instances queried per request (cap enforced)
|
||||
- [ ] HTML rendering happens AFTER merge/dedup/paginate (not before)
|
||||
- [ ] Empty state shown when no instance tabs exist (message: "Pin some instances first")
|
||||
- [ ] Hashtag tab panel displays full-width timeline
|
||||
|
||||
**Verify:**
|
||||
|
||||
- Pin `mastodon.social` (local) and `fosstodon.org` (local) as instance tabs
|
||||
- Add a `#indieweb` hashtag tab
|
||||
- Click the hashtag tab — results from both instances appear, sorted by date
|
||||
- Source line shows "Searching #indieweb across 2 instances: mastodon.social, fosstodon.org"
|
||||
- No duplicate posts visible
|
||||
- Scroll down — infinite scroll loads more posts without duplicates (per-instance cursors work correctly)
|
||||
- Invalid hashtag (e.g., `../../path`) is rejected with 400
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Cleanup — Remove Old Deck Code, Update CSS, Update Locales
|
||||
|
||||
**Objective:** Remove all remaining references to the old deck system. Clean up CSS (remove `.ap-deck-*` classes), update locale strings, delete `assets/reader-decks.js` (note: `lib/controllers/decks.js` was already deleted in Task 2).
|
||||
|
||||
**Dependencies:** Task 4, Task 5, Task 6
|
||||
|
||||
**Files:**
|
||||
|
||||
- Delete: `assets/reader-decks.js` — Old Alpine deck components (fully replaced by `reader-tabs.js`)
|
||||
- Modify: `assets/reader.css` — Remove all `.ap-deck-*` and `.ap-explore-deck-toggle*` styles (lines 2063-2248)
|
||||
- Modify: `assets/reader-infinite-scroll.js` — Remove `apExploreScroll` component (replaced by tab-scoped scroll in `reader-tabs.js`); keep `apInfiniteScroll` unchanged
|
||||
- Modify: `locales/en.json` — Remove `explore.deck.*` and `explore.tabs.decks` strings; ensure new tab strings are present
|
||||
- Modify: `index.js` — Verify no remaining imports or references to `decks.js`
|
||||
|
||||
**Key Decisions / Notes:**
|
||||
|
||||
- `lib/controllers/decks.js` was already deleted in Task 2 — verify it's gone here
|
||||
- `reader-infinite-scroll.js` still contains `apInfiniteScroll` for the main reader timeline — only remove `apExploreScroll`
|
||||
- CSS cleanup: remove lines 2063-2248 from `reader.css` (deck toggle, deck grid, deck column, deck empty, deck responsive). Keep `.ap-tabs` styles (extended in Task 4).
|
||||
- Locale cleanup: remove `explore.deck.*` object entirely, remove `explore.tabs.decks` string
|
||||
- Verify `reader.css` does not exceed 300 lines per section after changes
|
||||
- The `ap_decks` collection is left in MongoDB (not explicitly dropped). Users can manually drop it via `mongosh` if desired: `db.ap_decks.drop()`
|
||||
|
||||
**Definition of Done:**
|
||||
|
||||
- [ ] `lib/controllers/decks.js` confirmed deleted (was done in Task 2)
|
||||
- [ ] `assets/reader-decks.js` deleted
|
||||
- [ ] No `.ap-deck-*` CSS classes remain in `reader.css`
|
||||
- [ ] `apExploreScroll` retained in `reader-infinite-scroll.js` (still used by Search tab's server-rendered infinite scroll)
|
||||
- [ ] `apInfiniteScroll` still works in `reader-infinite-scroll.js`
|
||||
- [ ] No `deck` or `ap_decks` references remain anywhere in codebase (except git history)
|
||||
- [ ] All locale strings clean — no orphaned deck strings
|
||||
|
||||
**Verify:**
|
||||
|
||||
- `grep -r "ap_decks\|apDeckColumn\|apDeckToggle\|reader-decks" --include="*.js" --include="*.njk" --include="*.json" lib/ views/ assets/ locales/ index.js` returns nothing (apExploreScroll intentionally retained for Search tab)
|
||||
- `grep -r "ap-deck-" assets/reader.css` returns nothing
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit tests:** No automated test suite exists for this plugin (manual testing only — see CLAUDE.md). However, each task will be verified by:
|
||||
1. Checking that the explore page renders correctly via Playwright
|
||||
2. Testing API endpoints with curl
|
||||
3. Verifying infinite scroll works
|
||||
- **Integration tests:** Test the full tab lifecycle: add instance tab → browse timeline → add hashtag tab → verify aggregation → reorder → remove
|
||||
- **Manual verification:**
|
||||
1. `playwright-cli open https://rmendes.net/activitypub/admin/reader/explore` — verify UI renders
|
||||
2. `curl` the tab API endpoints to verify CRUD operations
|
||||
3. Test with multiple instances to verify hashtag aggregation
|
||||
4. Test responsive layout on mobile widths
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
| --- | --- | --- | --- |
|
||||
| Hashtag aggregation slow with many instances | Med | Med | Hard cap at MAX_HASHTAG_INSTANCES = 10; `Promise.allSettled()` with per-instance 10s timeout; exclude failed instances; show partial results with source status |
|
||||
| Hashtag input injection / path traversal | Med | High | `validateHashtag()` enforces `/^[\w]+$/` regex, max 100 chars, strips leading `#`. Called in both tab CRUD and hashtag explore endpoint |
|
||||
| Mastodon API rate limiting on hashtag queries | Low | Med | Each tab loads independently on user click, not all at once on page load; 10s timeout per instance prevents hanging |
|
||||
| Tab reordering race condition (concurrent clicks) | Low | Low | Client-side debouncing (500ms) batches rapid arrow clicks into single API call; reorder endpoint accepts full ordered array |
|
||||
| MongoDB unique index bypass with null fields | Med | Med | All insertions explicitly set ALL four indexed fields (unused fields set to `null`); documented in Task 1 and Task 2 |
|
||||
| Abandoned HTTP connections on tab switch | Low | Med | AbortController aborts in-flight client fetch when switching away from a loading tab |
|
||||
| Old `ap_decks` data remains in MongoDB | Low | Low | Old collection is simply not registered; data stays in MongoDB but is unused. User can manually drop via `mongosh` if desired |
|
||||
| CSS file exceeds 300 line threshold after changes | Low | Med | Deck CSS removal (~185 lines) roughly offsets new tab CSS addition (~100 lines); net reduction in CSS |
|
||||
|
||||
## Open Questions
|
||||
|
||||
- None — all design decisions were made during brainstorming and refined by plan review findings.
|
||||
|
||||
### Deferred Ideas
|
||||
|
||||
- Drag-and-drop tab reordering (enhancement over up/down arrows)
|
||||
- Per-instance hashtag filter within instance tabs
|
||||
- Auto-refresh / live polling for active tabs
|
||||
- Tab color customization
|
||||
- Short-TTL caching (30-60s) for hashtag aggregation results to reduce re-querying on rapid scroll
|
||||
@@ -1,40 +0,0 @@
|
||||
# Reader Timeline Enhancements
|
||||
|
||||
## Features
|
||||
|
||||
### 1. New Posts Detection (Reader timeline)
|
||||
|
||||
30-second background poll checks for items newer than the top-most visible item's `published` date.
|
||||
|
||||
- **API**: `GET /admin/reader/api/timeline/count-new?after={isoDate}&tab={tab}` returns `{ count: N }`
|
||||
- **UI**: Sticky banner at top of timeline: "N new posts — Load"
|
||||
- Clicking loads new items via existing `api/timeline` with `after=` param, prepends to timeline
|
||||
- Banner disappears after loading; polling continues from newest item's date
|
||||
- Explore tabs excluded (external instance APIs don't support "since" queries efficiently)
|
||||
|
||||
### 2. Mark As Read on Scroll
|
||||
|
||||
IntersectionObserver watches each `.ap-card` at 50% threshold.
|
||||
|
||||
- When card is 50% visible, its `uid` is batched client-side
|
||||
- Every 5 seconds, batch flushes via `POST /admin/reader/api/timeline/mark-read` with `{ uids: [...] }`
|
||||
- Server sets `{ read: true }` on matching `ap_timeline` docs
|
||||
- **Visual**: `.ap-card--read` class applies `opacity: 0.7`, set immediately on observe
|
||||
- **Filter toggle**: "Show unread only" in tab bar adds `?unread=1` — server filters `{ read: { $ne: true } }`
|
||||
- `unreadCount` in template reflects actual unread items
|
||||
|
||||
### 3. Infinite Scroll + Load More
|
||||
|
||||
Already implemented via `apInfiniteScroll` and `apExploreScroll` Alpine components. No changes needed.
|
||||
|
||||
## Files to Modify
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `lib/controllers/api-timeline.js` | New `countNewController` and `markReadController` endpoints |
|
||||
| `lib/storage/timeline.js` | `countNewItems()` and `markItemsRead()` functions |
|
||||
| `lib/controllers/reader.js` | Pass `unread` filter param, compute `unreadCount` from DB |
|
||||
| `index.js` | Register new API routes |
|
||||
| `assets/reader-infinite-scroll.js` | New `apNewPostsBanner` Alpine component + read tracking observer |
|
||||
| `views/activitypub-reader.njk` | New posts banner markup, unread toggle, read class on cards |
|
||||
| `assets/reader.css` | `.ap-card--read`, banner styles, unread toggle styles |
|
||||
@@ -1,996 +0,0 @@
|
||||
# Reader Improvements Plan — Inspired by Elk & Phanpy
|
||||
|
||||
**Date:** 2026-03-03
|
||||
**Source:** `docs/research/2026-03-03-elk-phanpy-comparison.md`
|
||||
**Current version:** 2.4.5
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Prioritized improvements to the ActivityPub reader, organized into releases. Each release is a publishable npm version. Tasks within a release are ordered by dependency (later tasks may depend on earlier ones).
|
||||
|
||||
**Release 0 must ship first** — it unifies the reader and explore pipelines so that every subsequent release only needs to implement each feature once.
|
||||
|
||||
---
|
||||
|
||||
## Release 0: Unify Reader & Explore Pipeline (v2.5.0-rc.1)
|
||||
|
||||
**Impact:** Critical prerequisite — Without this, every improvement from Releases 1-8 must be implemented twice (once for inbox-sourced items, once for Mastodon API items), with different code in different files. This release eliminates that duplication.
|
||||
|
||||
### Problem Statement
|
||||
|
||||
The reader (followed accounts) and explore (public instance timelines) are the same feature with different data sources. But the code treats them as separate systems:
|
||||
|
||||
| Operation | Reader | Explore | Duplicated? |
|
||||
|-----------|--------|---------|-------------|
|
||||
| Item construction | `extractObjectData()` in `timeline-store.js` | `mapMastodonStatusToItem()` in `explore-utils.js` | Yes — same shape, different source |
|
||||
| Quote stripping | `reader.js:200-206` | `explore.js:102-108` AND `explore.js:193-199` | Yes — identical loop in 3 places |
|
||||
| Moderation filtering | `reader.js:84-146` | (missing) | Explore has none |
|
||||
| Interaction map | `reader.js:154-198` | `explore.js:134` (empty `{}`) | Different but same pattern |
|
||||
| Tab filtering | `reader.js:59-82` | N/A | Reader-only |
|
||||
| Mastodon API fetch | N/A | `explore.js:63-114` AND `explore.js:160-205` | Duplicated within explore itself |
|
||||
| Card HTML rendering | `api-timeline.js:148-170` | `explore.js:207-229` | Identical |
|
||||
| Infinite scroll JS | `apInfiniteScroll` (95 lines) | `apExploreScroll` (93 lines) | 80% identical |
|
||||
|
||||
Additionally, `reader.js` and `api-timeline.js` duplicate the same logic (moderation, interaction map, tab filtering, quote stripping) — the API endpoint is a copy-paste of the page controller.
|
||||
|
||||
### Task 0.1: Extract `postProcessItems()` shared utility
|
||||
|
||||
**File:** `lib/item-processing.js` (new)
|
||||
|
||||
Extract the shared post-processing that happens after items are loaded (from DB or API), regardless of source. This function takes raw items and returns processed items ready for rendering.
|
||||
|
||||
```js
|
||||
/**
|
||||
* Post-process timeline items for rendering.
|
||||
* Used by both reader and explore controllers.
|
||||
*
|
||||
* @param {Array} items - Raw timeline items (from DB or Mastodon API mapping)
|
||||
* @param {object} options
|
||||
* @param {object} [options.moderation] - { mutedUrls, mutedKeywords, blockedUrls, filterMode }
|
||||
* @param {object} [options.interactionsCol] - MongoDB collection for interaction state lookup
|
||||
* @returns {{ items: Array, interactionMap: object }}
|
||||
*/
|
||||
export async function postProcessItems(items, options = {}) {
|
||||
// 1. Apply moderation filters (muted actors, keywords, blocked actors)
|
||||
if (options.moderation) {
|
||||
items = applyModerationFilters(items, options.moderation);
|
||||
}
|
||||
|
||||
// 2. Strip "RE:" paragraphs from items with quote embeds
|
||||
stripQuoteReferences(items);
|
||||
|
||||
// 3. Build interaction map (likes, boosts) — empty for explore
|
||||
const interactionMap = options.interactionsCol
|
||||
? await buildInteractionMap(items, options.interactionsCol)
|
||||
: {};
|
||||
|
||||
return { items, interactionMap };
|
||||
}
|
||||
```
|
||||
|
||||
This eliminates 4 copies of the quote-stripping loop, 2 copies of the moderation filter, and 2 copies of the interaction map builder.
|
||||
|
||||
### Task 0.2: Extract `applyModerationFilters()` into shared utility
|
||||
|
||||
**File:** `lib/item-processing.js`
|
||||
|
||||
Move the moderation filtering logic from `reader.js:84-146` (and its duplicate in `api-timeline.js:63-111`) into a single function:
|
||||
|
||||
```js
|
||||
export function applyModerationFilters(items, { mutedUrls, mutedKeywords, blockedUrls, filterMode }) {
|
||||
const blockedSet = new Set(blockedUrls);
|
||||
const mutedSet = new Set(mutedUrls);
|
||||
|
||||
if (blockedSet.size === 0 && mutedSet.size === 0 && mutedKeywords.length === 0) {
|
||||
return items;
|
||||
}
|
||||
|
||||
return items.filter((item) => {
|
||||
if (item.author?.url && blockedSet.has(item.author.url)) return false;
|
||||
// ... (existing logic, written once)
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### Task 0.3: Extract `buildInteractionMap()` into shared utility
|
||||
|
||||
**File:** `lib/item-processing.js`
|
||||
|
||||
Move the interaction map logic from `reader.js:154-198` (and `api-timeline.js:113-136`) into:
|
||||
|
||||
```js
|
||||
export async function buildInteractionMap(items, interactionsCol) {
|
||||
const lookupUrls = new Set();
|
||||
const objectUrlToUid = new Map();
|
||||
for (const item of items) { /* ... existing logic ... */ }
|
||||
// Returns { [uid]: { like: true, boost: true } }
|
||||
}
|
||||
```
|
||||
|
||||
### Task 0.4: Extract `renderItemCards()` shared HTML renderer
|
||||
|
||||
**File:** `lib/item-processing.js`
|
||||
|
||||
Move the server-side card rendering from `api-timeline.js:148-170` (and identical code in `explore.js:207-229`) into:
|
||||
|
||||
```js
|
||||
/**
|
||||
* Render items to HTML using ap-item-card.njk.
|
||||
* Used by both timeline API and explore API for infinite scroll.
|
||||
*/
|
||||
export async function renderItemCards(items, request, templateData) {
|
||||
const htmlParts = await Promise.all(
|
||||
items.map((item) => new Promise((resolve, reject) => {
|
||||
request.app.render(
|
||||
"partials/ap-item-card.njk",
|
||||
{ ...templateData, item },
|
||||
(err, html) => err ? reject(err) : resolve(html),
|
||||
);
|
||||
})),
|
||||
);
|
||||
return htmlParts.join("");
|
||||
}
|
||||
```
|
||||
|
||||
### Task 0.5: Deduplicate Mastodon API fetch in explore controller
|
||||
|
||||
**File:** `lib/controllers/explore.js`
|
||||
|
||||
`exploreController()` (page load) and `exploreApiController()` (AJAX scroll) have 95% identical fetch logic. Extract:
|
||||
|
||||
```js
|
||||
/**
|
||||
* Fetch statuses from a remote Mastodon-compatible instance.
|
||||
* @returns {{ items: Array, nextMaxId: string|null }}
|
||||
*/
|
||||
async function fetchMastodonTimeline(instance, { scope, hashtag, maxId, limit }) {
|
||||
const isLocal = scope === "local";
|
||||
let apiUrl;
|
||||
if (hashtag) {
|
||||
apiUrl = new URL(`https://${instance}/api/v1/timelines/tag/${encodeURIComponent(hashtag)}`);
|
||||
} else {
|
||||
apiUrl = new URL(`https://${instance}/api/v1/timelines/public`);
|
||||
}
|
||||
apiUrl.searchParams.set("local", isLocal ? "true" : "false");
|
||||
apiUrl.searchParams.set("limit", String(limit));
|
||||
if (maxId) apiUrl.searchParams.set("max_id", maxId);
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS);
|
||||
const fetchRes = await fetch(apiUrl.toString(), {
|
||||
headers: { Accept: "application/json" },
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
if (!fetchRes.ok) throw new Error(`Remote returned HTTP ${fetchRes.status}`);
|
||||
const statuses = await fetchRes.json();
|
||||
if (!Array.isArray(statuses)) throw new Error("Unexpected API response");
|
||||
|
||||
const items = statuses.map((s) => mapMastodonStatusToItem(s, instance));
|
||||
const nextMaxId = (statuses.length === limit && statuses.length > 0)
|
||||
? statuses[statuses.length - 1].id
|
||||
: null;
|
||||
|
||||
return { items, nextMaxId };
|
||||
}
|
||||
```
|
||||
|
||||
Both controllers call this instead of duplicating the fetch.
|
||||
|
||||
### Task 0.6: Simplify reader controller and API controller
|
||||
|
||||
**Files:** `lib/controllers/reader.js`, `lib/controllers/api-timeline.js`
|
||||
|
||||
Rewrite both to use `postProcessItems()`:
|
||||
|
||||
**reader.js** (before — 70 lines of processing):
|
||||
```js
|
||||
const result = await getTimelineItems(collections, options);
|
||||
let items = applyTabFilter(result.items, tab);
|
||||
|
||||
const moderation = await loadModerationData(modCollections);
|
||||
const { items: processed, interactionMap } = await postProcessItems(items, {
|
||||
moderation,
|
||||
interactionsCol: application?.collections?.get("ap_interactions"),
|
||||
});
|
||||
```
|
||||
|
||||
**api-timeline.js** (before — 100 lines of duplicated processing):
|
||||
```js
|
||||
const result = await getTimelineItems(collections, options);
|
||||
let items = applyTabFilter(result.items, tab);
|
||||
|
||||
const moderation = await loadModerationData(modCollections);
|
||||
const { items: processed, interactionMap } = await postProcessItems(items, {
|
||||
moderation,
|
||||
interactionsCol: application?.collections?.get("ap_interactions"),
|
||||
});
|
||||
const html = await renderItemCards(processed, request, { ...response.locals, mountPath, csrfToken, interactionMap });
|
||||
response.json({ html, before: result.before });
|
||||
```
|
||||
|
||||
### Task 0.7: Simplify explore controllers
|
||||
|
||||
**File:** `lib/controllers/explore.js`
|
||||
|
||||
Rewrite both `exploreController()` and `exploreApiController()` to use `fetchMastodonTimeline()`, `postProcessItems()`, and `renderItemCards()`:
|
||||
|
||||
```js
|
||||
export function exploreApiController(mountPath) {
|
||||
return async (request, response, next) => {
|
||||
const instance = validateInstance(request.query.instance);
|
||||
if (!instance) return response.status(400).json({ error: "Invalid instance" });
|
||||
|
||||
const { items, nextMaxId } = await fetchMastodonTimeline(instance, {
|
||||
scope: request.query.scope,
|
||||
hashtag: validateHashtag(request.query.hashtag),
|
||||
maxId: request.query.max_id,
|
||||
limit: MAX_RESULTS,
|
||||
});
|
||||
|
||||
const { items: processed, interactionMap } = await postProcessItems(items);
|
||||
const html = await renderItemCards(processed, request, {
|
||||
...response.locals, mountPath, csrfToken: getToken(request.session), interactionMap,
|
||||
});
|
||||
|
||||
response.json({ html, maxId: nextMaxId });
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Task 0.8: Extract `applyTabFilter()` shared utility
|
||||
|
||||
**File:** `lib/item-processing.js`
|
||||
|
||||
The tab filtering logic is duplicated between `reader.js:71-82` and `api-timeline.js:49-61`:
|
||||
|
||||
```js
|
||||
export function applyTabFilter(items, tab) {
|
||||
if (tab === "replies") return items.filter((item) => item.inReplyTo);
|
||||
if (tab === "media") return items.filter((item) =>
|
||||
item.photo?.length > 0 || item.video?.length > 0 || item.audio?.length > 0
|
||||
);
|
||||
return items;
|
||||
}
|
||||
```
|
||||
|
||||
### Task 0.9: Unify infinite scroll Alpine component
|
||||
|
||||
**File:** `assets/reader-infinite-scroll.js`
|
||||
|
||||
Replace `apExploreScroll` and `apInfiniteScroll` with a single parameterized `apInfiniteScroll` component:
|
||||
|
||||
```js
|
||||
Alpine.data("apInfiniteScroll", () => ({
|
||||
loading: false,
|
||||
done: false,
|
||||
cursor: null, // Generic cursor — was "maxId" for explore, "before" for reader
|
||||
apiUrl: "", // Set from data-api-url attribute
|
||||
cursorParam: "", // Set from data-cursor-param ("max_id" or "before")
|
||||
cursorField: "", // Response field name for next cursor ("maxId" or "before")
|
||||
extraParams: {}, // Additional query params (instance, scope, hashtag, tab, tag)
|
||||
observer: null,
|
||||
|
||||
init() {
|
||||
const el = this.$el;
|
||||
this.cursor = el.dataset.cursor || null;
|
||||
this.apiUrl = el.dataset.apiUrl || "";
|
||||
this.cursorParam = el.dataset.cursorParam || "before";
|
||||
this.cursorField = el.dataset.cursorField || "before";
|
||||
|
||||
// Parse extra params from data-extra-params JSON attribute
|
||||
try {
|
||||
this.extraParams = JSON.parse(el.dataset.extraParams || "{}");
|
||||
} catch { this.extraParams = {}; }
|
||||
|
||||
if (!this.cursor) { this.done = true; return; }
|
||||
|
||||
this.observer = new IntersectionObserver(
|
||||
(entries) => {
|
||||
for (const entry of entries) {
|
||||
if (entry.isIntersecting && !this.loading && !this.done) {
|
||||
this.loadMore();
|
||||
}
|
||||
}
|
||||
},
|
||||
{ rootMargin: "200px" },
|
||||
);
|
||||
|
||||
if (this.$refs.sentinel) this.observer.observe(this.$refs.sentinel);
|
||||
},
|
||||
|
||||
async loadMore() {
|
||||
if (this.loading || this.done || !this.cursor) return;
|
||||
this.loading = true;
|
||||
|
||||
const params = new URLSearchParams({
|
||||
[this.cursorParam]: this.cursor,
|
||||
...this.extraParams,
|
||||
});
|
||||
|
||||
try {
|
||||
const res = await fetch(`${this.apiUrl}?${params}`, {
|
||||
headers: { Accept: "application/json" },
|
||||
});
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const data = await res.json();
|
||||
|
||||
const timeline = this.$refs.timeline || this.$el.querySelector("[data-timeline]");
|
||||
if (data.html && timeline) {
|
||||
timeline.insertAdjacentHTML("beforeend", data.html);
|
||||
}
|
||||
|
||||
if (data[this.cursorField]) {
|
||||
this.cursor = data[this.cursorField];
|
||||
} else {
|
||||
this.done = true;
|
||||
if (this.observer) this.observer.disconnect();
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("[ap-infinite-scroll] load failed:", err.message);
|
||||
} finally {
|
||||
this.loading = false;
|
||||
}
|
||||
},
|
||||
|
||||
destroy() {
|
||||
if (this.observer) this.observer.disconnect();
|
||||
},
|
||||
}));
|
||||
```
|
||||
|
||||
Template usage for reader:
|
||||
```njk
|
||||
<div x-data="apInfiniteScroll"
|
||||
data-cursor="{{ before }}"
|
||||
data-api-url="{{ mountPath }}/admin/reader/api/timeline"
|
||||
data-cursor-param="before"
|
||||
data-cursor-field="before"
|
||||
data-extra-params='{{ { tab: tab } | dump }}'>
|
||||
```
|
||||
|
||||
Template usage for explore:
|
||||
```njk
|
||||
<div x-data="apInfiniteScroll"
|
||||
data-cursor="{{ maxId }}"
|
||||
data-api-url="{{ mountPath }}/admin/reader/api/explore"
|
||||
data-cursor-param="max_id"
|
||||
data-cursor-field="maxId"
|
||||
data-extra-params='{{ { instance: instance, scope: scope, hashtag: hashtag } | dump }}'>
|
||||
```
|
||||
|
||||
### Task 0.10: Update templates to use unified component
|
||||
|
||||
**Files:** `views/activitypub-reader.njk`, `views/activitypub-explore.njk`
|
||||
|
||||
Replace `x-data="apExploreScroll"` with `x-data="apInfiniteScroll"` using the parameterized data attributes. Remove the `apExploreScroll` component definition.
|
||||
|
||||
### Task 0.11: Verify no regressions
|
||||
|
||||
Manual testing:
|
||||
- Reader timeline loads, infinite scroll works, new posts banner works
|
||||
- Explore search tab loads, infinite scroll works
|
||||
- Explore pinned tabs load, load-more buttons work
|
||||
- Quote embeds render in both views
|
||||
- Moderation filtering still works in reader
|
||||
- Interaction state (likes/boosts) still shows in reader
|
||||
- Read tracking still works
|
||||
|
||||
### Files changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `lib/item-processing.js` | **New** — `postProcessItems()`, `applyModerationFilters()`, `buildInteractionMap()`, `renderItemCards()`, `applyTabFilter()`, `stripQuoteReferences()` |
|
||||
| `lib/controllers/reader.js` | Simplified — uses `postProcessItems()` |
|
||||
| `lib/controllers/api-timeline.js` | Simplified — uses `postProcessItems()` + `renderItemCards()` |
|
||||
| `lib/controllers/explore.js` | Simplified — uses `fetchMastodonTimeline()`, `postProcessItems()`, `renderItemCards()` |
|
||||
| `assets/reader-infinite-scroll.js` | Unified — single `apInfiniteScroll` component replaces two |
|
||||
| `views/activitypub-reader.njk` | Updated data attributes for unified scroll component |
|
||||
| `views/activitypub-explore.njk` | Updated data attributes for unified scroll component |
|
||||
|
||||
### Impact on subsequent releases
|
||||
|
||||
After Release 0, every improvement only needs to be added in ONE place:
|
||||
|
||||
| Enhancement | Before Release 0 | After Release 0 |
|
||||
|-------------|-------------------|-----------------|
|
||||
| Custom emoji | `timeline-store.js` + `explore-utils.js` + `reader.js` + `explore.js` | `item-processing.js` (single post-process step) |
|
||||
| Quote stripping | 4 locations | `item-processing.js` only |
|
||||
| Moderation | 2 locations | `item-processing.js` only |
|
||||
| New content transforms | Must add to both pipelines | Single pipeline |
|
||||
|
||||
---
|
||||
|
||||
## Release 1: Custom Emoji Rendering (v2.5.0)
|
||||
|
||||
**Impact:** High — Custom emoji is ubiquitous on the fediverse. Without it, display names show raw `:shortcode:` text and post content loses visual meaning.
|
||||
|
||||
### Task 1.1: Store emoji data from ActivityPub inbox
|
||||
|
||||
**File:** `lib/timeline-store.js`
|
||||
|
||||
`extractObjectData()` currently ignores emoji data. Fedify's `Note`/`Article` objects expose custom emoji via the `getTags()` call — emoji are `Emoji` instances (a subclass of `Flag`) in the tags array, alongside `Hashtag` and `Mention`.
|
||||
|
||||
**Changes:**
|
||||
- In the tag extraction loop (~line 190), check for Fedify `Emoji` instances
|
||||
- Each Emoji has: `name` (`:shortcode:` with colons), and an `icon` property (an `Image` with `url`)
|
||||
- Extract to an `emojis` array: `[{ shortcode: "blobcat", url: "https://..." }]`
|
||||
- Add `emojis` to the returned item object
|
||||
- Also extract emojis from the actor object in `extractActorInfo()` for display name emoji
|
||||
|
||||
**Stored data shape:**
|
||||
```js
|
||||
emojis: [
|
||||
{ shortcode: "blobcat", url: "https://cdn.example/emoji/blobcat.png" },
|
||||
{ shortcode: "verified", url: "https://cdn.example/emoji/verified.png" }
|
||||
]
|
||||
```
|
||||
|
||||
### Task 1.2: Store emoji data from Mastodon REST API (explore view)
|
||||
|
||||
**File:** `lib/controllers/explore-utils.js`
|
||||
|
||||
Mastodon's REST API v1 returns `status.emojis` as an array of `{ shortcode, url, static_url, visible_in_picker }` objects, and `status.account.emojis` for display name emoji.
|
||||
|
||||
**Changes:**
|
||||
- In `mapMastodonStatusToItem()`, extract `status.emojis` → `item.emojis`
|
||||
- Extract `account.emojis` → `item.author.emojis`
|
||||
- Normalize to same shape as Task 1.1: `[{ shortcode, url }]`
|
||||
|
||||
### Task 1.3: Create emoji replacement utility
|
||||
|
||||
**File:** `lib/emoji-utils.js` (new)
|
||||
|
||||
A small utility that replaces `:shortcode:` patterns with `<img>` tags. Used in both content HTML and display names.
|
||||
|
||||
```js
|
||||
export function replaceCustomEmoji(html, emojis) {
|
||||
if (!emojis?.length) return html;
|
||||
for (const emoji of emojis) {
|
||||
const pattern = new RegExp(`:${escapeRegex(emoji.shortcode)}:`, "g");
|
||||
html = html.replace(pattern,
|
||||
`<img src="${emoji.url}" alt=":${emoji.shortcode}:" ` +
|
||||
`title=":${emoji.shortcode}:" class="ap-custom-emoji" loading="lazy">`
|
||||
);
|
||||
}
|
||||
return html;
|
||||
}
|
||||
```
|
||||
|
||||
Must escape regex special characters in shortcodes. Must be called AFTER `sanitizeContent()` (which would strip the `<img>` tags if run after).
|
||||
|
||||
### Task 1.4: Apply emoji replacement in content pipeline
|
||||
|
||||
**File:** `lib/item-processing.js`
|
||||
|
||||
Add an `applyCustomEmoji(items)` step to `postProcessItems()`. Since both reader and explore flow through this single function (after Release 0), emoji replacement happens once for all items regardless of source.
|
||||
|
||||
```js
|
||||
// Inside postProcessItems(), after quote stripping:
|
||||
applyCustomEmoji(items);
|
||||
```
|
||||
|
||||
The function iterates items, calling `replaceCustomEmoji(item.content.html, item.emojis)` on each.
|
||||
|
||||
### Task 1.5: Apply emoji replacement in display names
|
||||
|
||||
**File:** `lib/item-processing.js`
|
||||
|
||||
Add emoji replacement for display names inside the same `applyCustomEmoji()` step:
|
||||
|
||||
```js
|
||||
if (item.author?.emojis?.length && item.author.name) {
|
||||
item.author.nameHtml = replaceCustomEmoji(
|
||||
sanitizeHtml(item.author.name, { allowedTags: [], allowedAttributes: {} }),
|
||||
item.author.emojis,
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
This adds `author.nameHtml` alongside existing `author.name`. Template renders `nameHtml | safe` when present, falls back to `name`.
|
||||
|
||||
### Task 1.6: Add emoji CSS
|
||||
|
||||
**File:** `assets/reader.css`
|
||||
|
||||
```css
|
||||
.ap-custom-emoji {
|
||||
height: 1.2em;
|
||||
width: auto;
|
||||
vertical-align: middle;
|
||||
display: inline;
|
||||
margin: 0 0.05em;
|
||||
}
|
||||
```
|
||||
|
||||
### Task 1.7: Update sanitize-html allowlist
|
||||
|
||||
**File:** `lib/timeline-store.js` (or wherever `sanitizeContent` config lives)
|
||||
|
||||
The `sanitize-html` configuration must allow `<img>` tags with class `ap-custom-emoji` through — but only for emoji images, not arbitrary remote images. Since emoji replacement happens AFTER sanitization, this isn't an issue: the emoji `<img>` tags are inserted post-sanitization and never pass through the sanitizer.
|
||||
|
||||
Verify this ordering is correct in both codepaths (inbox + explore).
|
||||
|
||||
### Task 1.8: Store emoji in MongoDB
|
||||
|
||||
**File:** `lib/storage/timeline.js`
|
||||
|
||||
Add `emojis` to the stored fields in `addTimelineItem()`. Also add `author.emojis` if storing per-author emoji data.
|
||||
|
||||
---
|
||||
|
||||
## Release 2: Relative Timestamps (v2.5.1)
|
||||
|
||||
**Impact:** High — Every fediverse client shows "2m ago" instead of "Feb 25, 2026, 4:46 PM". Relative timestamps are dramatically faster to scan when reading a timeline.
|
||||
|
||||
### Task 2.1: Create relative time Alpine directive
|
||||
|
||||
**File:** `assets/reader-relative-time.js` (new)
|
||||
|
||||
A small Alpine.js directive that:
|
||||
1. Reads `datetime` attribute from a `<time>` element
|
||||
2. Computes relative string ("just now", "2m", "1h", "3d", "Feb 25")
|
||||
3. Updates every 60 seconds for recent posts
|
||||
4. Shows absolute time on hover via `title` attribute
|
||||
|
||||
Format rules (matching Mastodon/Elk conventions):
|
||||
- < 1 minute: "just now"
|
||||
- < 60 minutes: "Xm" (e.g., "5m")
|
||||
- < 24 hours: "Xh" (e.g., "3h")
|
||||
- < 7 days: "Xd" (e.g., "2d")
|
||||
- Same year: "Mar 3" (month + day)
|
||||
- Different year: "Mar 3, 2025" (month + day + year)
|
||||
|
||||
No external dependency — pure JS using `Intl.RelativeTimeFormat` or simple math.
|
||||
|
||||
### Task 2.2: Apply directive in item card template
|
||||
|
||||
**File:** `views/partials/ap-item-card.njk`
|
||||
|
||||
Change the timestamp rendering from:
|
||||
```njk
|
||||
<time>{{ item.published | date("PPp") }}</time>
|
||||
```
|
||||
To:
|
||||
```njk
|
||||
<time datetime="{{ item.published }}"
|
||||
title="{{ item.published | date('PPp') }}"
|
||||
x-data x-relative-time>
|
||||
{{ item.published | date("PPp") }}
|
||||
</time>
|
||||
```
|
||||
|
||||
The server-rendered absolute time remains as fallback (no-JS, initial paint). Alpine enhances it to relative on hydration.
|
||||
|
||||
### Task 2.3: Apply to quote embeds and other timestamp locations
|
||||
|
||||
**Files:** `views/partials/ap-quote-embed.njk`, `views/activitypub-notifications.njk`, `views/activitypub-activities.njk`
|
||||
|
||||
Apply the same `x-relative-time` directive to all timestamp `<time>` elements across the reader UI.
|
||||
|
||||
### Task 2.4: Load the directive script
|
||||
|
||||
**File:** `views/layouts/ap-reader.njk` (or equivalent layout)
|
||||
|
||||
Add `<script src="{{ mountPath }}/assets/reader-relative-time.js"></script>` alongside existing reader scripts. Ensure it loads before Alpine initializes.
|
||||
|
||||
---
|
||||
|
||||
## Release 3: Enriched Media Data Model (v2.5.2)
|
||||
|
||||
**Impact:** High — This is the prerequisite for ALT badges, blurhash placeholders, and focus-point cropping. Currently photos are stored as bare URL strings, losing all metadata.
|
||||
|
||||
### Task 3.1: Enrich photo extraction from ActivityPub
|
||||
|
||||
**File:** `lib/timeline-store.js`
|
||||
|
||||
Change `extractObjectData()` photo extraction from:
|
||||
```js
|
||||
photo.push(att.url?.href || "");
|
||||
```
|
||||
To:
|
||||
```js
|
||||
photo.push({
|
||||
url: att.url?.href || "",
|
||||
alt: att.name || "", // Fedify: Image.name is alt text
|
||||
width: att.width || null, // Fedify: Image.width
|
||||
height: att.height || null, // Fedify: Image.height
|
||||
blurhash: "", // Not available from AP objects directly
|
||||
});
|
||||
```
|
||||
|
||||
Fedify's `Image` class (attachment type) exposes `name` (alt text), `width`, `height`, and `url`.
|
||||
|
||||
### Task 3.2: Enrich photo extraction from Mastodon API
|
||||
|
||||
**File:** `lib/controllers/explore-utils.js`
|
||||
|
||||
Mastodon API `media_attachments[]` objects have: `url`, `description` (alt text), `blurhash`, `meta.original.width`, `meta.original.height`, `meta.focus.x`, `meta.focus.y`.
|
||||
|
||||
Change from:
|
||||
```js
|
||||
photo.push(url);
|
||||
```
|
||||
To:
|
||||
```js
|
||||
photo.push({
|
||||
url,
|
||||
alt: att.description || "",
|
||||
width: att.meta?.original?.width || null,
|
||||
height: att.meta?.original?.height || null,
|
||||
blurhash: att.blurhash || "",
|
||||
focus: att.meta?.focus || null, // { x: -0.5..0.5, y: -0.5..0.5 }
|
||||
});
|
||||
```
|
||||
|
||||
### Task 3.3: Backward-compatible template rendering
|
||||
|
||||
**File:** `views/partials/ap-item-media.njk`
|
||||
|
||||
Templates currently do `item.photo[0]` expecting a string URL. Must handle both formats during migration:
|
||||
|
||||
```njk
|
||||
{# Support both old string format and new object format #}
|
||||
{% set photoUrl = photo.url if photo.url else photo %}
|
||||
{% set photoAlt = photo.alt if photo.alt else "" %}
|
||||
```
|
||||
|
||||
### Task 3.4: Update MongoDB storage
|
||||
|
||||
**File:** `lib/storage/timeline.js`
|
||||
|
||||
No schema change needed — MongoDB stores whatever shape we give it. But the `getTimelineItems()` function should normalize old string-format photos to objects for template consistency:
|
||||
|
||||
```js
|
||||
// Normalize photo format (backward compat with string-only entries)
|
||||
photo: (item.photo || []).map(p => typeof p === "string" ? { url: p, alt: "" } : p),
|
||||
```
|
||||
|
||||
### Task 3.5: Update quote embed photo handling
|
||||
|
||||
**File:** `lib/og-unfurl.js` (in `fetchAndStoreQuote`)
|
||||
|
||||
The quote enrichment stores `quoteData.photo?.slice(0, 1)`. Ensure it works with the new object format.
|
||||
|
||||
---
|
||||
|
||||
## Release 4: ALT Text Badges (v2.5.3)
|
||||
|
||||
**Impact:** High — Accessibility feature that both Elk and Phanpy display prominently. Depends on Release 3.
|
||||
|
||||
### Task 4.1: Add ALT badge to media template
|
||||
|
||||
**File:** `views/partials/ap-item-media.njk`
|
||||
|
||||
For each photo in the grid:
|
||||
```njk
|
||||
<div class="ap-media__item">
|
||||
<img src="{{ photoUrl }}" alt="{{ photoAlt }}" loading="lazy">
|
||||
{% if photoAlt %}
|
||||
<button class="ap-media__alt-badge"
|
||||
type="button"
|
||||
x-data="{ open: false }"
|
||||
@click="open = !open"
|
||||
:aria-expanded="open">
|
||||
ALT
|
||||
</button>
|
||||
<div class="ap-media__alt-text" x-show="open" x-cloak>
|
||||
{{ photoAlt }}
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
```
|
||||
|
||||
### Task 4.2: Style ALT badges
|
||||
|
||||
**File:** `assets/reader.css`
|
||||
|
||||
```css
|
||||
.ap-media__alt-badge {
|
||||
position: absolute;
|
||||
bottom: 0.5rem;
|
||||
left: 0.5rem;
|
||||
background: rgba(0, 0, 0, 0.7);
|
||||
color: white;
|
||||
font-size: 0.65rem;
|
||||
font-weight: 700;
|
||||
padding: 0.15rem 0.35rem;
|
||||
border-radius: var(--border-radius-small);
|
||||
border: none;
|
||||
cursor: pointer;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.03em;
|
||||
}
|
||||
|
||||
.ap-media__alt-text {
|
||||
position: absolute;
|
||||
bottom: 2.5rem;
|
||||
left: 0.5rem;
|
||||
right: 0.5rem;
|
||||
background: rgba(0, 0, 0, 0.85);
|
||||
color: white;
|
||||
font-size: var(--font-size-s);
|
||||
padding: 0.5rem;
|
||||
border-radius: var(--border-radius-small);
|
||||
max-height: 8rem;
|
||||
overflow-y: auto;
|
||||
}
|
||||
```
|
||||
|
||||
### Task 4.3: Ensure media item containers are positioned
|
||||
|
||||
**File:** `assets/reader.css`
|
||||
|
||||
Each `.ap-media__item` must be `position: relative` for the absolute-positioned badge.
|
||||
|
||||
---
|
||||
|
||||
## Release 5: Interaction Counts (v2.5.4)
|
||||
|
||||
**Impact:** Medium — Shows social proof (3 likes, 12 boosts). Data is already available from Mastodon API; needs extraction from ActivityPub.
|
||||
|
||||
### Task 5.1: Store interaction counts from ActivityPub
|
||||
|
||||
**File:** `lib/timeline-store.js`
|
||||
|
||||
Fedify's `Note`/`Article` objects expose:
|
||||
- `likes` — a `Collection` with `totalItems`
|
||||
- `shares` — a `Collection` with `totalItems`
|
||||
- `replies` — a `Collection` with `totalItems`
|
||||
|
||||
Extract in `extractObjectData()`:
|
||||
```js
|
||||
const counts = {
|
||||
replies: null,
|
||||
boosts: null,
|
||||
likes: null,
|
||||
};
|
||||
try {
|
||||
const replies = await object.getReplies?.({ documentLoader });
|
||||
counts.replies = replies?.totalItems ?? null;
|
||||
} catch { /* ignore */ }
|
||||
// Same for likes (object.getLikes) and shares (object.getShares)
|
||||
```
|
||||
|
||||
Add `counts` to the returned object.
|
||||
|
||||
### Task 5.2: Store interaction counts from Mastodon API
|
||||
|
||||
**File:** `lib/controllers/explore-utils.js`
|
||||
|
||||
Direct mapping — Mastodon API provides:
|
||||
```js
|
||||
counts: {
|
||||
replies: status.replies_count || 0,
|
||||
boosts: status.reblogs_count || 0,
|
||||
likes: status.favourites_count || 0,
|
||||
}
|
||||
```
|
||||
|
||||
### Task 5.3: Display counts in interaction buttons
|
||||
|
||||
**File:** `views/partials/ap-item-card.njk`
|
||||
|
||||
Add count display next to each interaction button:
|
||||
```njk
|
||||
<button class="ap-interactions__btn ap-interactions__btn--like" ...>
|
||||
<svg>...</svg>
|
||||
{% if item.counts and item.counts.likes %}
|
||||
<span class="ap-interactions__count">{{ item.counts.likes }}</span>
|
||||
{% endif %}
|
||||
</button>
|
||||
```
|
||||
|
||||
### Task 5.4: Style interaction counts
|
||||
|
||||
**File:** `assets/reader.css`
|
||||
|
||||
```css
|
||||
.ap-interactions__count {
|
||||
font-size: var(--font-size-xs);
|
||||
color: var(--color-on-offset);
|
||||
margin-left: 0.25rem;
|
||||
}
|
||||
```
|
||||
|
||||
### Task 5.5: Update counts on interaction (optimistic UI)
|
||||
|
||||
**File:** `assets/reader-infinite-scroll.js` or `ap-item-card.njk` Alpine component
|
||||
|
||||
When a user likes/boosts, increment the displayed count optimistically. Revert on error.
|
||||
|
||||
---
|
||||
|
||||
## Release 6: Skeleton Loaders (v2.5.5)
|
||||
|
||||
**Impact:** Medium — Replaces "Loading..." text with card-shaped animated placeholders. Pure CSS, no data changes.
|
||||
|
||||
### Task 6.1: Create skeleton card partial
|
||||
|
||||
**File:** `views/partials/ap-skeleton-card.njk` (new)
|
||||
|
||||
```njk
|
||||
<div class="ap-card ap-card--skeleton" aria-hidden="true">
|
||||
<div class="ap-card__header">
|
||||
<div class="ap-skeleton ap-skeleton--avatar"></div>
|
||||
<div class="ap-skeleton-lines">
|
||||
<div class="ap-skeleton ap-skeleton--name"></div>
|
||||
<div class="ap-skeleton ap-skeleton--handle"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="ap-card__body">
|
||||
<div class="ap-skeleton ap-skeleton--line"></div>
|
||||
<div class="ap-skeleton ap-skeleton--line ap-skeleton--line-short"></div>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
### Task 6.2: Add skeleton CSS
|
||||
|
||||
**File:** `assets/reader.css`
|
||||
|
||||
```css
|
||||
.ap-skeleton {
|
||||
background: linear-gradient(90deg,
|
||||
var(--color-offset) 25%,
|
||||
var(--color-background) 50%,
|
||||
var(--color-offset) 75%);
|
||||
background-size: 200% 100%;
|
||||
animation: ap-skeleton-shimmer 1.5s infinite;
|
||||
border-radius: var(--border-radius-small);
|
||||
}
|
||||
|
||||
@keyframes ap-skeleton-shimmer {
|
||||
0% { background-position: 200% 0; }
|
||||
100% { background-position: -200% 0; }
|
||||
}
|
||||
```
|
||||
|
||||
### Task 6.3: Replace "Loading..." with skeleton cards
|
||||
|
||||
**Files:** `views/activitypub-reader.njk`, `views/activitypub-explore.njk`
|
||||
|
||||
Where the load-more mechanism currently shows "Loading..." text, show 3 skeleton cards instead.
|
||||
|
||||
---
|
||||
|
||||
## Release 7: Content Enhancements (v2.6.0)
|
||||
|
||||
**Impact:** Medium — Polish features that improve content readability.
|
||||
|
||||
### Task 7.1: Long URL shortening in content
|
||||
|
||||
**File:** `lib/timeline-store.js` or new `lib/content-utils.js`
|
||||
|
||||
After sanitization, shorten displayed URLs longer than 30 characters in `<a>` tags:
|
||||
```
|
||||
https://very-long-domain.example.com/path/to/page → very-long-domain.example.com/pa…
|
||||
```
|
||||
|
||||
Keep the full URL in `href`, only truncate the visible text node. Use a regex or DOM-like approach on the sanitized HTML.
|
||||
|
||||
### Task 7.2: Hashtag stuffing collapse
|
||||
|
||||
**File:** `lib/content-utils.js` (new utility)
|
||||
|
||||
Detect paragraphs that are 80%+ hashtag links (3+ tags). Wrap them in a collapsible container:
|
||||
```html
|
||||
<details class="ap-hashtag-overflow">
|
||||
<summary>Show tags</summary>
|
||||
<p>#tag1 #tag2 #tag3 #tag4 #tag5</p>
|
||||
</details>
|
||||
```
|
||||
|
||||
### Task 7.3: Bot account indicator
|
||||
|
||||
**Files:** `lib/timeline-store.js`, `lib/controllers/explore-utils.js`, `views/partials/ap-item-card.njk`
|
||||
|
||||
- Extract `bot` flag from actor data (Fedify: actor type === "Service"; Mastodon API: `account.bot`)
|
||||
- Store as `author.bot: true/false`
|
||||
- Display a small bot icon next to the display name in the card header
|
||||
|
||||
### Task 7.4: Edit indicator
|
||||
|
||||
**Files:** `lib/timeline-store.js`, `lib/controllers/explore-utils.js`, `views/partials/ap-item-card.njk`
|
||||
|
||||
- Extract `editedAt` / `updated` from post data
|
||||
- Display a pencil icon or "(edited)" text next to the timestamp when `editedAt` exists and differs from `published`
|
||||
|
||||
---
|
||||
|
||||
## Release 8: Visual Polish (v2.6.1)
|
||||
|
||||
**Impact:** Low-medium — Focus-point cropping and blurhash placeholders for images.
|
||||
|
||||
### Task 8.1: Focus-point cropping
|
||||
|
||||
**File:** `views/partials/ap-item-media.njk`, `assets/reader.css`
|
||||
|
||||
Use the `focus.x` / `focus.y` data (range -1 to 1) to compute `object-position`:
|
||||
```css
|
||||
/* Convert from -1..1 to 0..100% */
|
||||
object-position: calc(50% + focus.x * 50%) calc(50% - focus.y * 50%);
|
||||
```
|
||||
|
||||
Apply as inline style on `<img>` elements when focus data is available.
|
||||
|
||||
### Task 8.2: Blurhash placeholders
|
||||
|
||||
**File:** `assets/reader-blurhash.js` (new), `views/partials/ap-item-media.njk`
|
||||
|
||||
- Store blurhash string in photo objects (done in Release 3)
|
||||
- On client side, decode blurhash to a tiny canvas and use as background-image
|
||||
- Uses the [blurhash](https://github.com/woltapp/blurhash) JS decoder (~1KB)
|
||||
- Falls back gracefully — just shows the loading background color if blurhash unavailable
|
||||
|
||||
---
|
||||
|
||||
## Summary: Release Roadmap
|
||||
|
||||
| Release | Version | Key Feature | Prereqs | Scope |
|
||||
|---------|---------|-------------|---------|-------|
|
||||
| **0** | **v2.5.0-rc.1** | **Unify reader/explore pipeline** | **None** | **11 tasks** |
|
||||
| 1 | v2.5.0 | Custom emoji rendering | Release 0 | 8 tasks |
|
||||
| 2 | v2.5.1 | Relative timestamps | Release 0 | 4 tasks |
|
||||
| 3 | v2.5.2 | Enriched media data model | Release 0 | 5 tasks |
|
||||
| 4 | v2.5.3 | ALT text badges | Release 3 | 3 tasks |
|
||||
| 5 | v2.5.4 | Interaction counts | Release 0 | 5 tasks |
|
||||
| 6 | v2.5.5 | Skeleton loaders | Release 0 | 3 tasks |
|
||||
| 7 | v2.6.0 | Content enhancements (URLs, hashtags, bot, edit) | Release 1 | 4 tasks |
|
||||
| 8 | v2.6.1 | Visual polish (focus crop, blurhash) | Release 3 | 2 tasks |
|
||||
|
||||
**Release 0 is mandatory first** — all other releases depend on the unified pipeline.
|
||||
|
||||
After Release 0, releases 1-6 are independent of each other. 4 depends on 3, 7 depends on 1, 8 depends on 3.
|
||||
|
||||
**Recommended order:** 0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8
|
||||
|
||||
**Total: 45 tasks across 9 releases.**
|
||||
|
||||
---
|
||||
|
||||
## Files Modified Per Release
|
||||
|
||||
| File | R0 | R1 | R2 | R3 | R4 | R5 | R6 | R7 | R8 |
|
||||
|------|----|----|----|----|----|----|----|----|-----|
|
||||
| `lib/item-processing.js` | **new** | x | | | | x | | x | |
|
||||
| `lib/timeline-store.js` | | x | | x | | x | | x | |
|
||||
| `lib/controllers/explore-utils.js` | | x | | x | | x | | x | |
|
||||
| `lib/controllers/reader.js` | x | | | | | | | | |
|
||||
| `lib/controllers/api-timeline.js` | x | | | | | | | | |
|
||||
| `lib/controllers/explore.js` | x | | | | | | | | |
|
||||
| `lib/controllers/post-detail.js` | | x | | | | | | | |
|
||||
| `lib/emoji-utils.js` | | **new** | | | | | | | |
|
||||
| `lib/content-utils.js` | | | | | | | | **new** | |
|
||||
| `lib/storage/timeline.js` | | x | | x | | | | | |
|
||||
| `lib/og-unfurl.js` | | | | x | | | | | |
|
||||
| `assets/reader-infinite-scroll.js` | x | | | | | | | | |
|
||||
| `assets/reader.css` | | x | | | x | x | x | | x |
|
||||
| `assets/reader-relative-time.js` | | | **new** | | | | | | |
|
||||
| `assets/reader-blurhash.js` | | | | | | | | | **new** |
|
||||
| `views/partials/ap-item-card.njk` | | x | x | | | x | | x | |
|
||||
| `views/partials/ap-item-media.njk` | | | | x | x | | | | x |
|
||||
| `views/partials/ap-skeleton-card.njk` | | | | | | | **new** | | |
|
||||
| `views/partials/ap-quote-embed.njk` | | | x | | | | | | |
|
||||
| `views/activitypub-reader.njk` | x | | | | | | x | | |
|
||||
| `views/activitypub-explore.njk` | x | | | | | | x | | |
|
||||
| `views/layouts/ap-reader.njk` | | | x | | | | | | x |
|
||||
| `views/activitypub-notifications.njk` | | | x | | | | | | |
|
||||
| `views/activitypub-activities.njk` | | | x | | | | | | |
|
||||
|
||||
---
|
||||
|
||||
## Not Planned (Rationale)
|
||||
|
||||
| Feature | Why Not |
|
||||
|---------|---------|
|
||||
| Virtual scrolling | Server-rendered HTML is already lightweight; DOM nodes are cheap vs React/Vue vDOM |
|
||||
| WebSocket streaming | Would require a persistent WS server; polling at 30s is adequate |
|
||||
| Profile hover cards | Significant JS investment for marginal UX gain; clicking through to profile works fine |
|
||||
| Mention hover cards | Same as above — high effort, low return for server-rendered approach |
|
||||
| Keyboard shortcuts | Low demand; screen reader users already have nav shortcuts |
|
||||
| Video autoplay on scroll | Most users prefer manual control; respects data/battery |
|
||||
| Separate CW toggles (text vs media) | Current combined toggle works; splitting adds UI complexity |
|
||||
@@ -1,826 +0,0 @@
|
||||
# ActivityPub High-Impact Gaps Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Close all high-impact ActivityPub coverage gaps identified in the coverage audit, bringing federation compliance from ~70% to ~85%.
|
||||
|
||||
**Architecture:** Five independent features, each following existing codebase patterns. Outbound Delete uses the `broadcastActorUpdate()` batch delivery pattern. Visibility addressing extends `jf2ToAS2Activity()`. Content warnings add `sensitive`/`summary` to outbound objects. Inbound polls parse Fedify `Question` objects. Inbound reports add a `Flag` inbox listener with a new collection.
|
||||
|
||||
**Tech Stack:** Fedify 2.0 (`@fedify/fedify/vocab`), Express 5, MongoDB, Nunjucks, Alpine.js
|
||||
|
||||
**Audit Correction:** The audit listed "Outbound Block" as not implemented. During plan research, `lib/controllers/moderation.js:148-182` was found to already send `Block` via `ctx.sendActivity()` on block and `Undo(Block)` on unblock. **Block is fully implemented — no work needed.**
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Outbound Delete Activity
|
||||
|
||||
When a post is deleted from Indiekit, remote servers currently keep showing it forever. This task adds a `broadcastDelete(postUrl)` method and an admin API route to send `Delete` activities to all followers.
|
||||
|
||||
**Files:**
|
||||
- Modify: `index.js` (add method + route + import)
|
||||
- No new files needed
|
||||
|
||||
### Step 1: Add `broadcastDelete()` method to `index.js`
|
||||
|
||||
Add this method after `broadcastActorUpdate()` (after line ~870). It follows the exact same pattern: create context, build activity, fetch followers, deduplicate by shared inbox, batch deliver.
|
||||
|
||||
```javascript
|
||||
/**
|
||||
* Send Delete activity to all followers for a removed post.
|
||||
* Mirrors broadcastActorUpdate() pattern: batch delivery with shared inbox dedup.
|
||||
* @param {string} postUrl - Full URL of the deleted post
|
||||
*/
|
||||
async broadcastDelete(postUrl) {
|
||||
if (!this._federation) return;
|
||||
|
||||
try {
|
||||
const { Delete } = await import("@fedify/fedify/vocab");
|
||||
const handle = this.options.actor.handle;
|
||||
const ctx = this._federation.createContext(
|
||||
new URL(this._publicationUrl),
|
||||
{ handle, publicationUrl: this._publicationUrl },
|
||||
);
|
||||
|
||||
const del = new Delete({
|
||||
actor: ctx.getActorUri(handle),
|
||||
object: new URL(postUrl),
|
||||
});
|
||||
|
||||
const followers = await this._collections.ap_followers
|
||||
.find({})
|
||||
.project({ actorUrl: 1, inbox: 1, sharedInbox: 1 })
|
||||
.toArray();
|
||||
|
||||
const inboxMap = new Map();
|
||||
for (const f of followers) {
|
||||
const key = f.sharedInbox || f.inbox;
|
||||
if (key && !inboxMap.has(key)) {
|
||||
inboxMap.set(key, f);
|
||||
}
|
||||
}
|
||||
|
||||
const uniqueRecipients = [...inboxMap.values()];
|
||||
const BATCH_SIZE = 25;
|
||||
const BATCH_DELAY_MS = 5000;
|
||||
let delivered = 0;
|
||||
let failed = 0;
|
||||
|
||||
console.info(
|
||||
`[ActivityPub] Broadcasting Delete for ${postUrl} to ${uniqueRecipients.length} ` +
|
||||
`unique inboxes (${followers.length} followers)`,
|
||||
);
|
||||
|
||||
for (let i = 0; i < uniqueRecipients.length; i += BATCH_SIZE) {
|
||||
const batch = uniqueRecipients.slice(i, i + BATCH_SIZE);
|
||||
const recipients = batch.map((f) => ({
|
||||
id: new URL(f.actorUrl),
|
||||
inboxId: new URL(f.inbox || f.sharedInbox),
|
||||
endpoints: f.sharedInbox
|
||||
? { sharedInbox: new URL(f.sharedInbox) }
|
||||
: undefined,
|
||||
}));
|
||||
|
||||
try {
|
||||
await ctx.sendActivity(
|
||||
{ identifier: handle },
|
||||
recipients,
|
||||
del,
|
||||
{ preferSharedInbox: true },
|
||||
);
|
||||
delivered += batch.length;
|
||||
} catch (error) {
|
||||
failed += batch.length;
|
||||
console.warn(
|
||||
`[ActivityPub] Delete batch ${Math.floor(i / BATCH_SIZE) + 1} failed: ${error.message}`,
|
||||
);
|
||||
}
|
||||
|
||||
if (i + BATCH_SIZE < uniqueRecipients.length) {
|
||||
await new Promise((resolve) => setTimeout(resolve, BATCH_DELAY_MS));
|
||||
}
|
||||
}
|
||||
|
||||
console.info(
|
||||
`[ActivityPub] Delete broadcast complete for ${postUrl}: ${delivered} delivered, ${failed} failed`,
|
||||
);
|
||||
|
||||
await logActivity(this._collections.ap_activities, {
|
||||
direction: "outbound",
|
||||
type: "Delete",
|
||||
actorUrl: this._publicationUrl,
|
||||
objectUrl: postUrl,
|
||||
summary: `Sent Delete for ${postUrl} to ${delivered} inboxes`,
|
||||
});
|
||||
} catch (error) {
|
||||
console.warn("[ActivityPub] broadcastDelete failed:", error.message);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Add admin API route for federation delete
|
||||
|
||||
In the `get routes()` getter (after line ~311), add:
|
||||
|
||||
```javascript
|
||||
router.post("/admin/federation/delete", deleteFederationController(mp, this));
|
||||
```
|
||||
|
||||
### Step 3: Create the controller function
|
||||
|
||||
Add a new export in `lib/controllers/messages.js` — or better, create it inline in `index.js` near the route registration. The simplest approach: add the controller as a local function before the class, or add it to an existing controller file.
|
||||
|
||||
Create a minimal controller. Add this import at the top of `index.js` alongside other controller imports:
|
||||
|
||||
```javascript
|
||||
import { deleteFederationController } from "./lib/controllers/federation-delete.js";
|
||||
```
|
||||
|
||||
Create `lib/controllers/federation-delete.js`:
|
||||
|
||||
```javascript
|
||||
/**
|
||||
* POST /admin/federation/delete — Send Delete activity to all followers.
|
||||
* Removes a post from the fediverse after local deletion.
|
||||
* @param {string} mountPath - Plugin mount path
|
||||
* @param {object} plugin - ActivityPub plugin instance
|
||||
*/
|
||||
import { validateToken } from "../csrf.js";
|
||||
|
||||
export function deleteFederationController(mountPath, plugin) {
|
||||
return async (request, response, next) => {
|
||||
try {
|
||||
if (!validateToken(request)) {
|
||||
return response.status(403).json({
|
||||
success: false,
|
||||
error: "Invalid CSRF token",
|
||||
});
|
||||
}
|
||||
|
||||
const { url } = request.body;
|
||||
if (!url) {
|
||||
return response.status(400).json({
|
||||
success: false,
|
||||
error: "Missing post URL",
|
||||
});
|
||||
}
|
||||
|
||||
await plugin.broadcastDelete(url);
|
||||
|
||||
if (request.headers.accept?.includes("application/json")) {
|
||||
return response.json({ success: true, url });
|
||||
}
|
||||
|
||||
const referrer = request.get("Referrer") || `${mountPath}/admin/activities`;
|
||||
return response.redirect(referrer);
|
||||
} catch (error) {
|
||||
next(error);
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Add i18n string
|
||||
|
||||
In `locales/en.json`, add under a new `"federation"` key:
|
||||
|
||||
```json
|
||||
"federation": {
|
||||
"deleteSuccess": "Delete activity sent to followers",
|
||||
"deleteButton": "Delete from fediverse"
|
||||
}
|
||||
```
|
||||
|
||||
### Step 5: Verify syntax
|
||||
|
||||
Run: `node -c index.js && node -c lib/controllers/federation-delete.js`
|
||||
Expected: No errors
|
||||
|
||||
### Step 6: Commit
|
||||
|
||||
```bash
|
||||
git add index.js lib/controllers/federation-delete.js locales/en.json
|
||||
git commit -m "feat: outbound Delete activity — broadcast to followers when posts are removed"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Visibility Addressing (Unlisted + Followers-Only)
|
||||
|
||||
Currently all syndicated posts use public addressing (`to: PUBLIC, cc: followers`). This task adds support for unlisted and followers-only visibility via a `defaultVisibility` config option and per-post `visibility` property.
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/jf2-to-as2.js:151-267` (addressing logic)
|
||||
- Modify: `index.js:423-505` (pass visibility to converter, add config option)
|
||||
|
||||
### Step 1: Update `jf2ToAS2Activity()` addressing
|
||||
|
||||
In `lib/jf2-to-as2.js`, modify the function signature and addressing block (lines 151, 179-194).
|
||||
|
||||
**Change function signature** (line 151):
|
||||
|
||||
```javascript
|
||||
export function jf2ToAS2Activity(properties, actorUrl, publicationUrl, options = {}) {
|
||||
```
|
||||
|
||||
No change needed — `options` already exists. We'll pass `visibility` through it.
|
||||
|
||||
**Replace lines 179-194** (the addressing block) with:
|
||||
|
||||
```javascript
|
||||
const noteOptions = {
|
||||
attributedTo: actorUri,
|
||||
};
|
||||
|
||||
// Determine visibility: per-post override > option default > "public"
|
||||
const visibility = properties.visibility || options.visibility || "public";
|
||||
|
||||
// Addressing based on visibility
|
||||
// - "public": to: PUBLIC, cc: followers (+ reply author)
|
||||
// - "unlisted": to: followers, cc: PUBLIC (+ reply author)
|
||||
// - "followers": to: followers (+ reply author), no PUBLIC
|
||||
// - "direct": handled separately (DMs)
|
||||
const PUBLIC = new URL("https://www.w3.org/ns/activitystreams#Public");
|
||||
const followersUri = new URL(followersUrl);
|
||||
|
||||
if (replyToActorUrl && properties["in-reply-to"]) {
|
||||
const replyAuthor = new URL(replyToActorUrl);
|
||||
if (visibility === "unlisted") {
|
||||
noteOptions.to = followersUri;
|
||||
noteOptions.ccs = [PUBLIC, replyAuthor];
|
||||
} else if (visibility === "followers") {
|
||||
noteOptions.tos = [followersUri, replyAuthor];
|
||||
} else {
|
||||
// public (default)
|
||||
noteOptions.to = PUBLIC;
|
||||
noteOptions.ccs = [followersUri, replyAuthor];
|
||||
}
|
||||
} else {
|
||||
if (visibility === "unlisted") {
|
||||
noteOptions.to = followersUri;
|
||||
noteOptions.cc = PUBLIC;
|
||||
} else if (visibility === "followers") {
|
||||
noteOptions.to = followersUri;
|
||||
} else {
|
||||
// public (default)
|
||||
noteOptions.to = PUBLIC;
|
||||
noteOptions.cc = followersUri;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Also update the plain JSON-LD function `jf2ToActivityStreams()` (lines 81-82) with the same pattern. Replace:
|
||||
|
||||
```javascript
|
||||
to: ["https://www.w3.org/ns/activitystreams#Public"],
|
||||
cc: [`${actorUrl.replace(/\/$/, "")}/followers`],
|
||||
```
|
||||
|
||||
With:
|
||||
|
||||
```javascript
|
||||
to: visibility === "unlisted"
|
||||
? [`${actorUrl.replace(/\/$/, "")}/followers`]
|
||||
: visibility === "followers"
|
||||
? [`${actorUrl.replace(/\/$/, "")}/followers`]
|
||||
: ["https://www.w3.org/ns/activitystreams#Public"],
|
||||
cc: visibility === "unlisted"
|
||||
? ["https://www.w3.org/ns/activitystreams#Public"]
|
||||
: visibility === "followers"
|
||||
? []
|
||||
: [`${actorUrl.replace(/\/$/, "")}/followers`],
|
||||
```
|
||||
|
||||
Note: `jf2ToActivityStreams` needs `visibility` passed in. Add a fourth parameter:
|
||||
|
||||
```javascript
|
||||
export function jf2ToActivityStreams(properties, actorUrl, publicationUrl, options = {}) {
|
||||
const visibility = properties.visibility || options.visibility || "public";
|
||||
```
|
||||
|
||||
### Step 2: Add `defaultVisibility` config option
|
||||
|
||||
In `index.js`, add to the defaults object (near the top, in constructor defaults):
|
||||
|
||||
```javascript
|
||||
defaultVisibility: "public", // "public" | "unlisted" | "followers"
|
||||
```
|
||||
|
||||
### Step 3: Pass visibility in syndicator
|
||||
|
||||
In `index.js` syndicator `syndicate()` method (around line 466), where `jf2ToAS2Activity()` is called, pass the visibility option:
|
||||
|
||||
```javascript
|
||||
const activity = jf2ToAS2Activity(properties, actorUrl, self._publicationUrl, {
|
||||
replyToActorUrl: originalAuthorUrl,
|
||||
replyToActorHandle: originalAuthorHandle,
|
||||
visibility: properties.visibility || self.options.defaultVisibility,
|
||||
});
|
||||
```
|
||||
|
||||
### Step 4: Verify syntax
|
||||
|
||||
Run: `node -c lib/jf2-to-as2.js && node -c index.js`
|
||||
Expected: No errors
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add lib/jf2-to-as2.js index.js
|
||||
git commit -m "feat: unlisted + followers-only visibility addressing"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Content Warning / Sensitive Flag (Outbound)
|
||||
|
||||
Inbound sensitive content is already parsed (`timeline-store.js:152-153`) and rendered with a toggle (`ap-item-card.njk:79-87`). This task adds `sensitive` and `summary` (CW text) to outbound activities.
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/jf2-to-as2.js:207-230` (add sensitive/summary to Note/Article options)
|
||||
|
||||
### Step 1: Add sensitive + summary to Fedify objects
|
||||
|
||||
In `lib/jf2-to-as2.js`, after the published date block (after line 207) and before the content block (line 209), add:
|
||||
|
||||
```javascript
|
||||
// Content warning / sensitive flag
|
||||
if (properties.sensitive) {
|
||||
noteOptions.sensitive = true;
|
||||
}
|
||||
if (properties["post-status"] === "sensitive") {
|
||||
noteOptions.sensitive = true;
|
||||
}
|
||||
// Summary doubles as CW text in Mastodon
|
||||
if (properties.summary && !isArticle) {
|
||||
noteOptions.summary = properties.summary;
|
||||
noteOptions.sensitive = true;
|
||||
}
|
||||
```
|
||||
|
||||
Note: For articles, summary is already handled at line 228-231. The `sensitive` flag should still be set:
|
||||
|
||||
After line 231, add:
|
||||
|
||||
```javascript
|
||||
if (properties.sensitive && isArticle) {
|
||||
noteOptions.sensitive = true;
|
||||
}
|
||||
```
|
||||
|
||||
Also add to the plain JSON-LD function `jf2ToActivityStreams()` — after line 112, add:
|
||||
|
||||
```javascript
|
||||
if (properties.sensitive || properties["post-status"] === "sensitive") {
|
||||
object.sensitive = true;
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Verify syntax
|
||||
|
||||
Run: `node -c lib/jf2-to-as2.js`
|
||||
Expected: No errors
|
||||
|
||||
### Step 3: Commit
|
||||
|
||||
```bash
|
||||
git add lib/jf2-to-as2.js
|
||||
git commit -m "feat: outbound content warning / sensitive flag support"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Question / Poll Support (Inbound)
|
||||
|
||||
Poll posts from Mastodon currently render without options because `extractObjectData()` doesn't handle `Question` objects. This task adds poll parsing and a template partial for rendering poll options.
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/timeline-store.js:122-137` (add Question type detection + option extraction)
|
||||
- Create: `views/partials/ap-poll-options.njk` (poll rendering partial)
|
||||
- Modify: `views/partials/ap-item-card.njk` (include poll partial)
|
||||
- Modify: `locales/en.json` (add poll i18n strings)
|
||||
|
||||
### Step 1: Add Question import to timeline-store.js
|
||||
|
||||
At the top of `lib/timeline-store.js`, find the import from `@fedify/fedify/vocab` and add `Question`:
|
||||
|
||||
```javascript
|
||||
import { Article, Question } from "@fedify/fedify/vocab";
|
||||
```
|
||||
|
||||
If `Article` is already imported individually, just add `Question` to the same import.
|
||||
|
||||
### Step 2: Add Question type detection in `extractObjectData()`
|
||||
|
||||
In `lib/timeline-store.js`, after the type detection block (lines 130-137), extend it:
|
||||
|
||||
```javascript
|
||||
// Determine type — use instanceof for Fedify vocab objects
|
||||
let type = "note";
|
||||
if (object instanceof Article) {
|
||||
type = "article";
|
||||
}
|
||||
if (object instanceof Question) {
|
||||
type = "question";
|
||||
}
|
||||
if (options.boostedBy) {
|
||||
type = "boost";
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Extract poll options
|
||||
|
||||
After the `sensitive` extraction (line 153), add poll option extraction:
|
||||
|
||||
```javascript
|
||||
// Poll options (Question type)
|
||||
let pollOptions = [];
|
||||
let votersCount = 0;
|
||||
let pollClosed = false;
|
||||
let pollEndTime = "";
|
||||
|
||||
if (object instanceof Question) {
|
||||
// Fedify reads both oneOf (single-choice) and anyOf (multi-choice)
|
||||
try {
|
||||
const exclusive = [];
|
||||
for await (const opt of object.getExclusiveOptions?.() || []) {
|
||||
exclusive.push({
|
||||
name: opt.name?.toString() || "",
|
||||
votes: typeof opt.replies?.totalItems === "number" ? opt.replies.totalItems : 0,
|
||||
});
|
||||
}
|
||||
const inclusive = [];
|
||||
for await (const opt of object.getInclusiveOptions?.() || []) {
|
||||
inclusive.push({
|
||||
name: opt.name?.toString() || "",
|
||||
votes: typeof opt.replies?.totalItems === "number" ? opt.replies.totalItems : 0,
|
||||
});
|
||||
}
|
||||
pollOptions = exclusive.length > 0 ? exclusive : inclusive;
|
||||
} catch {
|
||||
// Poll options couldn't be extracted — show as regular post
|
||||
}
|
||||
|
||||
votersCount = typeof object.votersCount === "number" ? object.votersCount : 0;
|
||||
pollEndTime = object.endTime ? String(object.endTime) : "";
|
||||
pollClosed = object.closed != null;
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Include poll data in returned object
|
||||
|
||||
In the return object of `extractObjectData()` (around lines 304-325), add the poll fields:
|
||||
|
||||
```javascript
|
||||
pollOptions,
|
||||
votersCount,
|
||||
pollClosed,
|
||||
pollEndTime,
|
||||
```
|
||||
|
||||
### Step 5: Create poll rendering partial
|
||||
|
||||
Create `views/partials/ap-poll-options.njk`:
|
||||
|
||||
```nunjucks
|
||||
{# Poll options partial — renders vote results for Question-type posts #}
|
||||
{% if item.pollOptions and item.pollOptions.length > 0 %}
|
||||
{% set totalVotes = 0 %}
|
||||
{% for opt in item.pollOptions %}
|
||||
{% set totalVotes = totalVotes + opt.votes %}
|
||||
{% endfor %}
|
||||
|
||||
<div class="ap-poll">
|
||||
{% for opt in item.pollOptions %}
|
||||
{% set pct = (totalVotes > 0) and ((opt.votes / totalVotes * 100) | round) or 0 %}
|
||||
<div class="ap-poll__option">
|
||||
<div class="ap-poll__bar" style="width: {{ pct }}%"></div>
|
||||
<span class="ap-poll__label">{{ opt.name }}</span>
|
||||
<span class="ap-poll__votes">{{ pct }}%</span>
|
||||
</div>
|
||||
{% endfor %}
|
||||
<div class="ap-poll__footer">
|
||||
{% if item.votersCount > 0 %}
|
||||
{{ item.votersCount }} {{ __("activitypub.poll.voters") }}
|
||||
{% elif totalVotes > 0 %}
|
||||
{{ totalVotes }} {{ __("activitypub.poll.votes") }}
|
||||
{% endif %}
|
||||
{% if item.pollClosed %}
|
||||
· {{ __("activitypub.poll.closed") }}
|
||||
{% elif item.pollEndTime %}
|
||||
· {{ __("activitypub.poll.endsAt") }} <time datetime="{{ item.pollEndTime }}">{{ item.pollEndTime | date("PPp") }}</time>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
{% endif %}
|
||||
```
|
||||
|
||||
### Step 6: Include partial in item card
|
||||
|
||||
In `views/partials/ap-item-card.njk`, after the content block (after the `</div>` that closes `.ap-item__content`) and before attachments/link preview, add:
|
||||
|
||||
```nunjucks
|
||||
{# Poll options #}
|
||||
{% if item.type == "question" or (item.pollOptions and item.pollOptions.length > 0) %}
|
||||
{% include "partials/ap-poll-options.njk" %}
|
||||
{% endif %}
|
||||
```
|
||||
|
||||
### Step 7: Add CSS for poll rendering
|
||||
|
||||
In `assets/reader.css`, add a new section:
|
||||
|
||||
```css
|
||||
/* ==========================================================================
|
||||
Poll / Question
|
||||
========================================================================== */
|
||||
|
||||
.ap-poll {
|
||||
margin-top: var(--space-s);
|
||||
}
|
||||
|
||||
.ap-poll__option {
|
||||
position: relative;
|
||||
padding: var(--space-xs) var(--space-s);
|
||||
margin-bottom: var(--space-xs);
|
||||
border-radius: var(--border-radius-small);
|
||||
background: var(--color-offset);
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.ap-poll__bar {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
bottom: 0;
|
||||
background: var(--color-primary);
|
||||
opacity: 0.15;
|
||||
border-radius: var(--border-radius-small);
|
||||
}
|
||||
|
||||
.ap-poll__label {
|
||||
position: relative;
|
||||
font-size: var(--font-size-s);
|
||||
color: var(--color-on-background);
|
||||
}
|
||||
|
||||
.ap-poll__votes {
|
||||
position: relative;
|
||||
float: right;
|
||||
font-size: var(--font-size-s);
|
||||
font-weight: 600;
|
||||
color: var(--color-on-offset);
|
||||
}
|
||||
|
||||
.ap-poll__footer {
|
||||
font-size: var(--font-size-xs);
|
||||
color: var(--color-on-offset);
|
||||
margin-top: var(--space-xs);
|
||||
}
|
||||
```
|
||||
|
||||
### Step 8: Add i18n strings
|
||||
|
||||
In `locales/en.json`, add:
|
||||
|
||||
```json
|
||||
"poll": {
|
||||
"voters": "voters",
|
||||
"votes": "votes",
|
||||
"closed": "Poll closed",
|
||||
"endsAt": "Ends"
|
||||
}
|
||||
```
|
||||
|
||||
### Step 9: Verify syntax
|
||||
|
||||
Run: `node -c lib/timeline-store.js`
|
||||
Expected: No errors
|
||||
|
||||
### Step 10: Commit
|
||||
|
||||
```bash
|
||||
git add lib/timeline-store.js views/partials/ap-poll-options.njk views/partials/ap-item-card.njk assets/reader.css locales/en.json
|
||||
git commit -m "feat: inbound poll/question support — parse and render vote options"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Flag Handler (Inbound Reports)
|
||||
|
||||
Other fediverse servers can send `Flag` activities to report abusive content or actors. Currently these are silently dropped. This task adds a `Flag` inbox listener, an `ap_reports` collection, admin notification, and a reports view in the moderation dashboard.
|
||||
|
||||
**Files:**
|
||||
- Modify: `lib/inbox-listeners.js` (add Flag handler)
|
||||
- Modify: `index.js` (register ap_reports collection + indexes)
|
||||
- Modify: `lib/storage/notifications.js:129` (add "report" to type counts)
|
||||
- Modify: `views/partials/ap-notification-card.njk` (add report notification type)
|
||||
- Modify: `views/activitypub-notifications.njk` (add Reports tab)
|
||||
- Modify: `lib/controllers/reader.js` (add "report" to validTabs)
|
||||
- Modify: `locales/en.json` (add report i18n strings)
|
||||
|
||||
### Step 1: Register `ap_reports` collection in `index.js`
|
||||
|
||||
In the collection registration block (around line 891), add:
|
||||
|
||||
```javascript
|
||||
Indiekit.addCollection("ap_reports");
|
||||
```
|
||||
|
||||
In the collection storage block (around line 910), add:
|
||||
|
||||
```javascript
|
||||
ap_reports: indiekitCollections.get("ap_reports"),
|
||||
```
|
||||
|
||||
In the indexes block (around line 1000), add:
|
||||
|
||||
```javascript
|
||||
// ap_reports indexes
|
||||
try {
|
||||
await this._collections.ap_reports.createIndex(
|
||||
{ createdAt: 1 },
|
||||
{ expireAfterSeconds: notifRetention || undefined },
|
||||
);
|
||||
await this._collections.ap_reports.createIndex({ reporterUrl: 1 });
|
||||
await this._collections.ap_reports.createIndex({ reportedUrl: 1 });
|
||||
} catch {
|
||||
// Indexes may already exist
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Add Flag inbox listener
|
||||
|
||||
In `lib/inbox-listeners.js`, add the `Flag` import to the destructure from `@fedify/fedify/vocab` (around line 6-24). Then add a new handler after the Block handler (after line ~744):
|
||||
|
||||
```javascript
|
||||
// ── Flag (Report) ──────────────────────────────────────────────
|
||||
.on(Flag, async (ctx, flag) => {
|
||||
try {
|
||||
const authLoader = getAuthLoader ? await getAuthLoader(ctx) : undefined;
|
||||
const actorObj = await flag.getActor({ documentLoader: authLoader }).catch(() => null);
|
||||
|
||||
const reporterUrl = actorObj?.id?.href || flag.actorId?.href || "";
|
||||
const reporterName = actorObj?.name?.toString() || actorObj?.preferredUsername?.toString() || reporterUrl;
|
||||
|
||||
// Extract reported objects — Flag can report actors or posts
|
||||
const reportedIds = flag.objectIds?.map((u) => u.href) || [];
|
||||
const reason = flag.content?.toString() || "";
|
||||
|
||||
if (reportedIds.length === 0 && !reason) {
|
||||
console.info("[ActivityPub] Ignoring empty Flag from", reporterUrl);
|
||||
return;
|
||||
}
|
||||
|
||||
// Store report
|
||||
if (collections.ap_reports) {
|
||||
await collections.ap_reports.insertOne({
|
||||
reporterUrl,
|
||||
reporterName,
|
||||
reportedUrls: reportedIds,
|
||||
reason,
|
||||
createdAt: new Date().toISOString(),
|
||||
read: false,
|
||||
});
|
||||
}
|
||||
|
||||
// Create notification
|
||||
if (collections.ap_notifications) {
|
||||
const { addNotification } = await import("./storage/notifications.js");
|
||||
await addNotification(collections, {
|
||||
uid: `flag:${reporterUrl}:${Date.now()}`,
|
||||
type: "report",
|
||||
actorUrl: reporterUrl,
|
||||
actorName: reporterName,
|
||||
actorPhoto: actorObj?.iconUrl?.href || actorObj?.icon?.url?.href || "",
|
||||
actorHandle: actorObj?.preferredUsername
|
||||
? `@${actorObj.preferredUsername}@${new URL(reporterUrl).hostname}`
|
||||
: reporterUrl,
|
||||
objectUrl: reportedIds[0] || "",
|
||||
summary: reason ? reason.slice(0, 200) : "Report received",
|
||||
published: new Date().toISOString(),
|
||||
});
|
||||
}
|
||||
|
||||
await logActivity(collections, {
|
||||
direction: "inbound",
|
||||
type: "Flag",
|
||||
actorUrl: reporterUrl,
|
||||
objectUrl: reportedIds[0] || "",
|
||||
summary: `Report from ${reporterName}: ${reason.slice(0, 100)}`,
|
||||
});
|
||||
|
||||
console.info(`[ActivityPub] Flag received from ${reporterName} — ${reportedIds.length} objects reported`);
|
||||
} catch (error) {
|
||||
console.warn("[ActivityPub] Flag handler error:", error.message);
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
### Step 3: Update notification type handling
|
||||
|
||||
In `lib/storage/notifications.js`, in `getNotificationCountsByType()` (around line 129), add `report: 0` to the counts object:
|
||||
|
||||
```javascript
|
||||
const counts = { all: 0, reply: 0, like: 0, boost: 0, follow: 0, dm: 0, report: 0 };
|
||||
```
|
||||
|
||||
And add the case to handle `_id === "report"`.
|
||||
|
||||
### Step 4: Update notification card template
|
||||
|
||||
In `views/partials/ap-notification-card.njk`, add the report type badge and action text:
|
||||
|
||||
Type badge (alongside other `elif` checks):
|
||||
```nunjucks
|
||||
{% elif item.type == "report" %}⚑
|
||||
```
|
||||
|
||||
Action text:
|
||||
```nunjucks
|
||||
{% elif item.type == "report" %}{{ __("activitypub.reports.sentReport") }}
|
||||
```
|
||||
|
||||
### Step 5: Add Reports tab to notifications
|
||||
|
||||
In `views/activitypub-notifications.njk`, add a Reports tab alongside the DMs tab:
|
||||
|
||||
```nunjucks
|
||||
<a href="?tab=report"
|
||||
class="ap-tab{% if activeTab == 'report' %} ap-tab--active{% endif %}">
|
||||
{{ __("activitypub.notifications.tabs.reports") }}
|
||||
{% if counts.report > 0 %}
|
||||
<span class="ap-tab__count">{{ counts.report }}</span>
|
||||
{% endif %}
|
||||
</a>
|
||||
```
|
||||
|
||||
### Step 6: Add "report" to valid tabs
|
||||
|
||||
In `lib/controllers/reader.js`, add `"report"` to the `validTabs` array:
|
||||
|
||||
```javascript
|
||||
const validTabs = ["all", "reply", "like", "boost", "follow", "dm", "report"];
|
||||
```
|
||||
|
||||
### Step 7: Add i18n strings
|
||||
|
||||
In `locales/en.json`, add:
|
||||
|
||||
```json
|
||||
"reports": {
|
||||
"sentReport": "filed a report",
|
||||
"title": "Reports"
|
||||
}
|
||||
```
|
||||
|
||||
And add to `notifications.tabs`:
|
||||
|
||||
```json
|
||||
"reports": "Reports"
|
||||
```
|
||||
|
||||
### Step 8: Verify syntax
|
||||
|
||||
Run: `node -c lib/inbox-listeners.js && node -c lib/storage/notifications.js && node -c index.js`
|
||||
Expected: No errors
|
||||
|
||||
### Step 9: Commit
|
||||
|
||||
```bash
|
||||
git add lib/inbox-listeners.js lib/storage/notifications.js views/partials/ap-notification-card.njk views/activitypub-notifications.njk lib/controllers/reader.js locales/en.json index.js
|
||||
git commit -m "feat: inbound Flag handler — receive and display abuse reports"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Verification Plan
|
||||
|
||||
After all tasks are implemented:
|
||||
|
||||
1. **Outbound Delete** — Create a test post, syndicate to fediverse. Delete from Indiekit. Call `POST /activitypub/admin/federation/delete` with the post URL. Check activity log shows outbound Delete. Verify from a Mastodon account that the post is removed.
|
||||
|
||||
2. **Visibility** — Set `defaultVisibility: "unlisted"` in config. Create a post. Check from Mastodon that the post appears in the home timeline of followers but NOT on the public/federated timeline. Reset to "public".
|
||||
|
||||
3. **Content Warning** — Create a post with `sensitive: true` and a `summary` field via Micropub. Verify from Mastodon that the post shows behind a CW toggle with the summary text.
|
||||
|
||||
4. **Polls** — From Mastodon, create a poll and post it. View the reader timeline. Verify poll options render with percentage bars and voter count.
|
||||
|
||||
5. **Reports** — From a Mastodon instance, report the test actor. Check that:
|
||||
- A notification appears in the Reports tab
|
||||
- The activity log shows an inbound Flag
|
||||
- The `ap_reports` collection has the report entry
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Task | Gap Closed | Priority | Files Changed |
|
||||
|------|-----------|----------|---------------|
|
||||
| 1 | Outbound Delete | P1 — High | index.js, new controller, en.json |
|
||||
| 2 | Unlisted + Followers-only | P1/P2 | jf2-to-as2.js, index.js |
|
||||
| 3 | Content Warning (outbound) | P2 | jf2-to-as2.js |
|
||||
| 4 | Question/Poll (inbound) | P2 | timeline-store.js, new partial, item-card, CSS, en.json |
|
||||
| 5 | Flag Handler (inbound) | P2 | inbox-listeners.js, notifications.js, templates, en.json, index.js |
|
||||
| — | Block (outbound) | **Already implemented** | No work needed |
|
||||
|
||||
**Estimated coverage after implementation:** ~85% of Fedify capabilities, ~98% of real-world fediverse traffic.
|
||||
@@ -1,440 +0,0 @@
|
||||
# Elk & Phanpy Deep Dive — Lessons for Our ActivityPub Reader
|
||||
|
||||
**Date:** 2026-03-03
|
||||
**Purpose:** Identify concrete improvements by comparing our reader with two best-in-class fediverse clients.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Comparison
|
||||
|
||||
| Aspect | Elk (Vue/Nuxt) | Phanpy (React/Vite) | Our Reader (Nunjucks/Alpine) |
|
||||
|--------|----------------|---------------------|------------------------------|
|
||||
| Rendering | Client-side SPA | Client-side SPA | Server-side HTML + Alpine sprinkles |
|
||||
| Content processing | AST parse → VNode tree | DOM manipulation pipeline | Server-side sanitize-html |
|
||||
| State management | Vue refs + composables | Valtio proxy state | Alpine.js `x-data` components |
|
||||
| Pagination | Virtual scroller + stream | IntersectionObserver + debounce | IntersectionObserver + cursor |
|
||||
| CSS | UnoCSS (Tailwind-like) | CSS Modules + custom properties | Indiekit theme custom properties |
|
||||
|
||||
**Key insight:** Both Elk and Phanpy are full SPAs with rich client-side rendering. Our server-rendered approach is fundamentally different — we can't replicate everything, but we can cherry-pick the most impactful patterns.
|
||||
|
||||
---
|
||||
|
||||
## 1. Content Rendering
|
||||
|
||||
### What Elk & Phanpy Do Better
|
||||
|
||||
**Elk's content pipeline:**
|
||||
1. Parse HTML into AST (ultrahtml)
|
||||
2. Sanitize with element whitelist
|
||||
3. Transform mentions → interactive hover cards
|
||||
4. Transform hashtags → hover cards with usage stats
|
||||
5. Transform emoji shortcodes → inline images with tooltips
|
||||
6. Transform code blocks (backtick syntax)
|
||||
7. Render as Vue VNodes
|
||||
|
||||
**Phanpy's content pipeline:**
|
||||
1. Parse HTML into DOM
|
||||
2. Shorten long URLs (>30 chars): `https://...example.com/long`
|
||||
3. Detect hashtag stuffing (3+ tags in paragraph) → collapse
|
||||
4. Replace custom emoji shortcodes with `<img>` elements
|
||||
5. Convert backtick code blocks to `<pre><code>`
|
||||
6. Add `is-quote` class to quote links in content
|
||||
7. Wrap bare text in `<span>` for Safari text-decoration fix
|
||||
|
||||
### What We're Missing
|
||||
|
||||
| Feature | Elk | Phanpy | Us | Priority |
|
||||
|---------|-----|--------|-----|----------|
|
||||
| Custom emoji rendering | ✅ `:emoji:` → `<img>` with tooltip | ✅ `:emoji:` → `<img>` | ❌ Raw shortcodes shown | **High** |
|
||||
| Long URL shortening | ❌ | ✅ Truncate >30 chars | ❌ Full URLs shown | Medium |
|
||||
| Hashtag stuffing collapse | ❌ | ✅ 3+ tags collapsed | ❌ All tags shown inline | Low |
|
||||
| Mention hover cards | ✅ Full profile card on hover | ❌ | ❌ Links only | Low (needs client JS) |
|
||||
| Code block rendering | ✅ Syntax highlighting | ✅ Backtick → `<pre>` | ❌ Pass-through only | Low |
|
||||
| Inline code | ✅ | ✅ Backtick → `<code>` | ❌ | Low |
|
||||
|
||||
### Recommended Action: Custom Emoji
|
||||
|
||||
Both clients treat this as essential. Implementation for server-rendered HTML:
|
||||
|
||||
In `sanitizeContent()` or a new `processEmoji()` step, replace `:shortcode:` with `<img>` tags using the emoji data from the Mastodon API status object (`status.emojis` array). Each emoji has `{ shortcode, url, static_url }`.
|
||||
|
||||
```js
|
||||
// In timeline-store.js or explore-utils.js
|
||||
function replaceCustomEmoji(html, emojis) {
|
||||
if (!emojis?.length) return html;
|
||||
for (const emoji of emojis) {
|
||||
const re = new RegExp(`:${emoji.shortcode}:`, 'g');
|
||||
html = html.replace(re,
|
||||
`<img src="${emoji.url}" alt=":${emoji.shortcode}:" title=":${emoji.shortcode}:" class="ap-custom-emoji" loading="lazy">`
|
||||
);
|
||||
}
|
||||
return html;
|
||||
}
|
||||
```
|
||||
|
||||
CSS: `.ap-custom-emoji { height: 1.2em; vertical-align: middle; display: inline; }`
|
||||
|
||||
---
|
||||
|
||||
## 2. Quote Posts
|
||||
|
||||
### How Elk Handles Quotes
|
||||
|
||||
- Dedicated `StatusQuote.vue` component
|
||||
- Handles **7 quote states**: pending, revoked, deleted, blocked_account, blocked_domain, muted_account, rejected, accepted
|
||||
- Only renders full quote embed for `accepted` state
|
||||
- Renders as a nested `StatusCard` inside a `<blockquote cite="">` element
|
||||
- Supports shallow quotes (fetch on render) and pre-embedded quotes
|
||||
- Nesting limit: shows full card for levels 0-2, then author-only for 3+
|
||||
|
||||
### How Phanpy Handles Quotes
|
||||
|
||||
- `QuoteStatus` / `ShallowQuote` components
|
||||
- Full quote chain unwrapping (follows `quotedStatusId` up to 30 levels!)
|
||||
- Handles unfulfilled states (deleted, blocked, muted) with icon + message + optional "Show anyway" button
|
||||
- Marks quote links in parent content with `is-quote` CSS class (to visually distinguish them)
|
||||
- Nesting limit: level 3+ shows `@author …` only
|
||||
- State tracked in Valtio: `states.statusQuotes[statusKey]`
|
||||
|
||||
### What We Should Adopt
|
||||
|
||||
| Feature | Status | Priority |
|
||||
|---------|--------|----------|
|
||||
| Basic quote embed (author, content, photo) | ✅ Done (v2.4.3) | — |
|
||||
| Strip RE: link when quote renders | ✅ Done (v2.4.2) | — |
|
||||
| Quote state handling (deleted, pending) | ❌ We show stale/broken embeds | Medium |
|
||||
| Mark quote links in content CSS | ❌ Quote link looks like any other link | **High** |
|
||||
| Quote nesting depth limit | ❌ No nesting at all yet | Low |
|
||||
|
||||
### Recommended Action: Quote Link Styling
|
||||
|
||||
When we strip the `RE: <link>` paragraph, the remaining content is clean. But if we DON'T strip it (e.g., quote not yet fetched), the link should look distinct. Phanpy adds `is-quote` class. We could do this in `sanitizeContent` or in the template.
|
||||
|
||||
---
|
||||
|
||||
## 3. Media Rendering
|
||||
|
||||
### Elk's Media System
|
||||
|
||||
- **Grid layouts**: 1 item = full width, 2 = 50/50, 3-4 = 2-column grid
|
||||
- **Focus point cropping**: Uses `meta.focus.x/y` for intelligent CSS `object-position`
|
||||
- **Blurhash placeholders**: Generates colored placeholder from blurhash until image loads
|
||||
- **Progressive loading**: Blurhash → low-res → full-res
|
||||
- **Lightbox**: Full-screen modal with arrow navigation, counter, alt text display
|
||||
- **Alt text badge**: "ALT" badge on images with descriptions, click to expand
|
||||
- **Aspect ratio clamping**: Between 0.8 and 6.0 to prevent extreme shapes
|
||||
- **Data saving mode**: Blur images until explicit click to load
|
||||
- **Video autoplay**: IntersectionObserver at 75% visibility, respects reduced-motion preference
|
||||
|
||||
### Phanpy's Media System
|
||||
|
||||
- **Grid**: `media-eq1` through `media-gt4` CSS classes
|
||||
- **QuickPinchZoom**: Mobile pinch-to-zoom on images
|
||||
- **Blurhash**: Average color extracted as background during load
|
||||
- **Focal point**: CSS custom property `--original-aspect-ratio`
|
||||
- **Media carousel**: Swipe navigation with snap scroll, RTL support
|
||||
- **ALT badges**: Indexed "ALT¹", "ALT²" for multiple media
|
||||
- **Audio/video**: Full HTML5 controls, no autoplay, preload metadata
|
||||
|
||||
### What We Have vs. What We're Missing
|
||||
|
||||
| Feature | Elk | Phanpy | Us | Priority |
|
||||
|---------|-----|--------|-----|----------|
|
||||
| Photo grid (1-4+) | ✅ 2-column adaptive | ✅ CSS class-based | ✅ Grid with +N badge | — |
|
||||
| Lightbox | ✅ Modal + carousel | ✅ Pinch zoom | ✅ Alpine.js overlay | — |
|
||||
| Blurhash placeholder | ✅ Canvas decode | ✅ OffscreenCanvas | ❌ No placeholder | Medium |
|
||||
| Focus point crop | ✅ object-position | ✅ CSS custom prop | ❌ Center crop only | Medium |
|
||||
| ALT text indicator | ✅ Badge + dropdown | ✅ Indexed badges | ❌ Not shown | **High** |
|
||||
| Video autoplay/pause | ✅ IntersectionObserver | ✅ Auto-pause on scroll | ❌ Manual only | Low |
|
||||
| Aspect ratio clamping | ✅ 0.8–6.0 range | ✅ Custom property | ❌ Max-height only | Low |
|
||||
|
||||
### Recommended Action: ALT Text Badges
|
||||
|
||||
Both clients prominently show ALT text availability. This is an accessibility feature and visual polish win.
|
||||
|
||||
```njk
|
||||
{# In ap-item-media.njk, on each image #}
|
||||
{% if photo.alt or photo.description %}
|
||||
<span class="ap-media__alt-badge" title="{{ photo.alt or photo.description }}">ALT</span>
|
||||
{% endif %}
|
||||
```
|
||||
|
||||
Note: Our current data model stores photos as URL strings, not objects with alt text. We'd need to change `extractObjectData()` to store `{ url, alt, blurhash, width, height }` objects.
|
||||
|
||||
---
|
||||
|
||||
## 4. Infinite Scroll / Pagination
|
||||
|
||||
### Elk's Approach
|
||||
|
||||
- **Virtual scroller** (optional): `vue-virtual-scroller` renders only visible items
|
||||
- **Stream integration**: WebSocket pushes new posts in real-time
|
||||
- **New posts banner**: Collected in `prevItems`, shown as "X new items" button
|
||||
- **Buffering**: Next page items held until buffer reaches 10, then batch-inserted
|
||||
- **End anchor**: Loads next page when within 2x viewport height of bottom
|
||||
|
||||
### Phanpy's Approach
|
||||
|
||||
- **IntersectionObserver** with rootMargin = 1.5x screen height
|
||||
- **Debounced loading** (1s) prevents rapid re-requests
|
||||
- **Skeleton loaders** during fetch
|
||||
- **"Show more..." button** as fallback inside observer target
|
||||
- **Auto-refresh**: Polls periodically if user is near top and window is visible
|
||||
|
||||
### Our Current Approach
|
||||
|
||||
- **IntersectionObserver** with rootMargin = 200px
|
||||
- **Cursor-based pagination** with `before` parameter
|
||||
- **New posts banner** polling every 30s
|
||||
- **No virtual scrolling** — all cards in DOM
|
||||
- **No skeleton loaders** — button text changes to "Loading..."
|
||||
|
||||
### What We're Missing
|
||||
|
||||
| Feature | Elk | Phanpy | Us | Priority |
|
||||
|---------|-----|--------|-----|----------|
|
||||
| IntersectionObserver auto-load | ✅ | ✅ 1.5x screen | ✅ 200px margin | — |
|
||||
| Manual "Load more" button | ✅ | ✅ | ✅ (just added for tabs) | — |
|
||||
| Skeleton loaders | ✅ | ✅ | ❌ Text "Loading..." only | Medium |
|
||||
| New posts banner | ✅ WebSocket stream | ✅ Polling | ✅ Polling 30s | — |
|
||||
| Virtual scrolling | ✅ Optional | ❌ | ❌ | Low (server-rendered) |
|
||||
| Debounced loading | ❌ | ✅ 1s debounce | ❌ | Low |
|
||||
|
||||
### Recommended: Larger IntersectionObserver Margin
|
||||
|
||||
Our 200px rootMargin means auto-load triggers late. Both clients use 1.5-2x viewport height. Easy fix:
|
||||
|
||||
```js
|
||||
{ rootMargin: `0px 0px ${window.innerHeight}px 0px` }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Content Warnings / Sensitive Content
|
||||
|
||||
### Elk's System
|
||||
|
||||
- Separate toggles for text spoiler vs. sensitive media
|
||||
- User preferences: `expandCWByDefault`, `expandMediaByDefault`
|
||||
- Content filter integration (server-side filters shown as CW)
|
||||
- Eye icon toggle button
|
||||
- Dotted border separator between CW text and hidden content
|
||||
|
||||
### Phanpy's System
|
||||
|
||||
- `states.spoilers[id]` and `states.spoilersMedia[id]` — separate state per post
|
||||
- User preferences: `readingExpandSpoilers`, `readingExpandMedia`
|
||||
- Filtered content: Shows filter reason with separate reveal button
|
||||
- Three sensitivity levels: show_all, hide_all, user-controlled
|
||||
|
||||
### Our System
|
||||
|
||||
- Single toggle for both text and media (combined)
|
||||
- CW button with spoiler text shown
|
||||
- No user preference for auto-expand
|
||||
- Works well but lacks granularity
|
||||
|
||||
### Gap Analysis
|
||||
|
||||
| Feature | Elk | Phanpy | Us | Priority |
|
||||
|---------|-----|--------|-----|----------|
|
||||
| CW text toggle | ✅ | ✅ | ✅ | — |
|
||||
| Separate media toggle | ✅ | ✅ | ❌ Combined | Low |
|
||||
| Auto-expand preference | ✅ | ✅ | ❌ | Low |
|
||||
| Blurred media preview | ✅ Blurhash | ❌ | ❌ | Medium |
|
||||
|
||||
---
|
||||
|
||||
## 6. Author Display
|
||||
|
||||
### Elk's Approach
|
||||
|
||||
- Display name with custom emoji
|
||||
- Handle with `@username@domain` format
|
||||
- Bot indicator icon
|
||||
- Lock (private account) indicator
|
||||
- **Hover card**: Full profile preview on mouseover (500ms delay) with bio, stats, follow button
|
||||
- Relative time ("2h ago") with absolute tooltip
|
||||
|
||||
### Phanpy's Approach
|
||||
|
||||
- Display name with custom emoji and bold
|
||||
- Username shown only if different from display name (smart dedup)
|
||||
- Bot accounts get squircle avatar shape
|
||||
- Role tags (moderator/admin badges)
|
||||
- **Relative time** with smart formatting
|
||||
- Punycode handling for international domains
|
||||
- RTL-safe username display with `bidi-isolate`
|
||||
|
||||
### Our Approach
|
||||
|
||||
- Display name (sanitized plain text)
|
||||
- Handle with `@username@domain`
|
||||
- Absolute timestamp only ("Feb 25, 2026, 4:46 PM")
|
||||
- No bot/lock indicators
|
||||
- No hover cards
|
||||
- No custom emoji in display names
|
||||
|
||||
### What We're Missing
|
||||
|
||||
| Feature | Elk | Phanpy | Us | Priority |
|
||||
|---------|-----|--------|-----|----------|
|
||||
| Custom emoji in names | ✅ | ✅ | ❌ Stripped to text | **High** (same fix as content emoji) |
|
||||
| Relative timestamps | ✅ "2h ago" | ✅ Smart format | ❌ Absolute only | **High** |
|
||||
| Bot/lock indicators | ✅ Icons | ✅ Squircle avatar | ❌ | Low |
|
||||
| Profile hover cards | ✅ Full card | ❌ | ❌ | Low (needs significant JS) |
|
||||
|
||||
### Recommended Action: Relative Timestamps
|
||||
|
||||
Both clients use relative time for in-feed cards. This is a major readability improvement. Since we server-render, we have two options:
|
||||
|
||||
**Option A: Server-side relative time** — Compute in controller, but goes stale.
|
||||
**Option B: Client-side via Alpine** — Use a small Alpine component that converts ISO dates to relative strings. This is what both Elk and Phanpy do (client-side).
|
||||
|
||||
```js
|
||||
// Small Alpine directive or component
|
||||
Alpine.directive('relative-time', (el) => {
|
||||
const iso = el.getAttribute('datetime');
|
||||
const update = () => { el.textContent = formatRelative(iso); };
|
||||
update();
|
||||
el._interval = setInterval(update, 60000);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Hashtag Rendering
|
||||
|
||||
### Elk
|
||||
|
||||
- Hashtags in content preserved as links
|
||||
- `TagHoverWrapper` shows usage stats on hover
|
||||
- Sanitizer allows `hashtag` CSS class through
|
||||
|
||||
### Phanpy
|
||||
|
||||
- Spanifies: `#<span>hashtag</span>` inside link
|
||||
- Detects **hashtag stuffing** (3+ tags in one paragraph) → collapses with tooltip
|
||||
- Separate hashtag tags section at bottom of post (from API `tags` array, deduped against content)
|
||||
|
||||
### Us
|
||||
|
||||
- Hashtags extracted to `category` array, rendered as linked tags below content
|
||||
- Content HTML hashtag links pass through sanitization
|
||||
- **Bug found:** Inside `-webkit-line-clamp` containers (quote embeds), the `#<span>tag</span>` structure breaks because `-webkit-box` makes spans block-level (fixed in v2.4.5)
|
||||
|
||||
### Recommended Action
|
||||
|
||||
Our hashtag rendering is adequate. The main improvement would be Phanpy's hashtag stuffing collapse — but it's low priority since our tag rendering already extracts tags to a footer section.
|
||||
|
||||
---
|
||||
|
||||
## 8. Interaction UI
|
||||
|
||||
### Elk
|
||||
|
||||
- **4 buttons**: Reply (blue), Boost (green), Quote (purple), Favorite (rose/yellow)
|
||||
- **Counts** shown per button (configurable to hide)
|
||||
- **Color-coded hover states**: Each button tints its area on hover
|
||||
- **Keyboard shortcuts**: r=reply, b=boost, f=favorite
|
||||
- **Bookmark** as 5th action
|
||||
|
||||
### Phanpy
|
||||
|
||||
- **4 buttons**: Reply, Boost, Like, Bookmark
|
||||
- **StatusButton component** with dual title (checked/unchecked)
|
||||
- **Shortened counts**: "123K" for large numbers
|
||||
- **Keyboard shortcuts**: r, b, f, m
|
||||
|
||||
### Us
|
||||
|
||||
- **5 buttons**: Reply (link), Boost (toggle), Like (toggle), View Original, Save (optional)
|
||||
- Optimistic UI with revert on error
|
||||
- CSRF-protected POSTs
|
||||
- No keyboard shortcuts
|
||||
- No counts shown
|
||||
|
||||
### Gap Analysis
|
||||
|
||||
| Feature | Elk | Phanpy | Us | Priority |
|
||||
|---------|-----|--------|-----|----------|
|
||||
| Like/Boost/Reply | ✅ | ✅ | ✅ | — |
|
||||
| Interaction counts | ✅ Per-button | ✅ Shortened | ❌ | Medium |
|
||||
| Keyboard shortcuts | ✅ | ✅ | ❌ | Low |
|
||||
| Color-coded buttons | ✅ | ✅ | Partial (active states) | Low |
|
||||
| Bookmark | ✅ | ✅ | ✅ (Save) | — |
|
||||
| Quote button | ✅ | ❌ | ❌ | Low |
|
||||
|
||||
---
|
||||
|
||||
## Priority Improvements — Ranked by Impact
|
||||
|
||||
### Tier 1: High Impact, Moderate Effort
|
||||
|
||||
1. **Custom emoji rendering** — Both clients treat this as essential. Affects display names AND post content. Single utility function applicable everywhere.
|
||||
|
||||
2. **Relative timestamps** — Both clients use this. Major readability improvement for timeline scanning. Small Alpine component.
|
||||
|
||||
3. **ALT text badges on media** — Both clients show this prominently. Accessibility win. Requires enriching photo data model from URL strings to objects.
|
||||
|
||||
4. **Quote link styling in content** — When `RE:` link isn't stripped (pending quote), distinguish it visually. CSS-only change.
|
||||
|
||||
### Tier 2: Medium Impact, Moderate Effort
|
||||
|
||||
5. **Skeleton loaders** for pagination — Replace "Loading..." text with card-shaped placeholder skeletons. CSS-only.
|
||||
|
||||
6. **Blurhash placeholders** for media — Show colored placeholder while images load. Requires storing blurhash data from API.
|
||||
|
||||
7. **Focus point cropping** — Use focal point data for smarter image crops. Requires storing focus data.
|
||||
|
||||
8. **Interaction counts** — Show like/boost/reply counts on buttons. Data already available from API.
|
||||
|
||||
### Tier 3: Lower Impact or High Effort
|
||||
|
||||
9. **Hashtag stuffing collapse** — Collapse posts that are mostly hashtags.
|
||||
10. **Long URL shortening** — Truncate displayed URLs in content.
|
||||
11. **Bot/lock indicators** — Show account type badges.
|
||||
12. **Keyboard shortcuts** — Navigation and interaction hotkeys.
|
||||
13. **Video autoplay/pause on scroll** — IntersectionObserver for video elements.
|
||||
14. **Quote state handling** (deleted, pending, blocked) — Show appropriate message instead of broken embed.
|
||||
15. **Profile hover cards** — Full profile preview on author hover (significant JS investment).
|
||||
|
||||
---
|
||||
|
||||
## Data Model Gaps
|
||||
|
||||
Our timeline items store minimal data compared to what Elk/Phanpy consume. Key missing fields:
|
||||
|
||||
| Field | Source | Used For |
|
||||
|-------|--------|----------|
|
||||
| `emojis[]` | `status.emojis` | Custom emoji rendering in content + names |
|
||||
| `media[].alt` | `attachment.description` | ALT text badges |
|
||||
| `media[].blurhash` | `attachment.blurhash` | Placeholder images |
|
||||
| `media[].focus` | `attachment.meta.focus` | Smart cropping |
|
||||
| `media[].width/height` | `attachment.meta.original` | Aspect ratio |
|
||||
| `repliesCount` | `status.replies_count` | Interaction counts |
|
||||
| `reblogsCount` | `status.reblogs_count` | Interaction counts |
|
||||
| `favouritesCount` | `status.favourites_count` | Interaction counts |
|
||||
| `account.bot` | `account.bot` | Bot indicator |
|
||||
| `account.emojis` | `account.emojis` | Custom emoji in display names |
|
||||
| `poll` | `status.poll` | Poll rendering |
|
||||
| `editedAt` | `status.edited_at` | Edit indicator |
|
||||
|
||||
For **inbox-received posts** (via ActivityPub), some of these map to Fedify object properties. For **explore view posts** (via Mastodon REST API), all fields are directly available in the status object.
|
||||
|
||||
---
|
||||
|
||||
## Architectural Constraints
|
||||
|
||||
Our server-rendered approach means we can't do everything Elk and Phanpy do:
|
||||
|
||||
1. **No reactive state** — We can't update a card's like count in real-time without a page refresh or AJAX call
|
||||
2. **No virtual scrolling** — All cards are in the DOM (but server-rendered HTML is lighter than React/Vue vDOM)
|
||||
3. **No hover cards** — Would require significant Alpine.js investment and API endpoints
|
||||
4. **No WebSocket streaming** — We poll instead (already have 30s new posts banner)
|
||||
|
||||
But we have advantages too:
|
||||
- **Faster initial load** — Server-rendered HTML is immediately visible
|
||||
- **Works without JS** — Basic reading works even if Alpine fails
|
||||
- **Simpler deployment** — No build step, no client bundle
|
||||
- **Lower maintenance** — No framework version churn
|
||||
100
index.js
100
index.js
@@ -2,6 +2,7 @@ import express from "express";
|
||||
|
||||
import { setupFederation, buildPersonActor } from "./lib/federation-setup.js";
|
||||
import { initRedisCache } from "./lib/redis-cache.js";
|
||||
import { lookupWithSecurity } from "./lib/lookup-helpers.js";
|
||||
import {
|
||||
createFedifyMiddleware,
|
||||
} from "./lib/federation-bridge.js";
|
||||
@@ -35,10 +36,16 @@ import {
|
||||
unmuteController,
|
||||
blockController,
|
||||
unblockController,
|
||||
blockServerController,
|
||||
unblockServerController,
|
||||
moderationController,
|
||||
filterModeController,
|
||||
} from "./lib/controllers/moderation.js";
|
||||
import { followersController } from "./lib/controllers/followers.js";
|
||||
import {
|
||||
approveFollowController,
|
||||
rejectFollowController,
|
||||
} from "./lib/controllers/follow-requests.js";
|
||||
import { followingController } from "./lib/controllers/following.js";
|
||||
import { activitiesController } from "./lib/controllers/activities.js";
|
||||
import {
|
||||
@@ -99,6 +106,9 @@ import { logActivity } from "./lib/activity-log.js";
|
||||
import { resolveAuthor } from "./lib/resolve-author.js";
|
||||
import { scheduleCleanup } from "./lib/timeline-cleanup.js";
|
||||
import { runSeparateMentionsMigration } from "./lib/migrations/separate-mentions.js";
|
||||
import { loadBlockedServersToRedis } from "./lib/storage/server-blocks.js";
|
||||
import { scheduleKeyRefresh } from "./lib/key-refresh.js";
|
||||
import { startInboxProcessor } from "./lib/inbox-queue.js";
|
||||
import { deleteFederationController } from "./lib/controllers/federation-delete.js";
|
||||
import {
|
||||
federationMgmtController,
|
||||
@@ -304,7 +314,11 @@ export default class ActivityPubEndpoint {
|
||||
router.post("/admin/reader/unmute", unmuteController(mp, this));
|
||||
router.post("/admin/reader/block", blockController(mp, this));
|
||||
router.post("/admin/reader/unblock", unblockController(mp, this));
|
||||
router.post("/admin/reader/block-server", blockServerController(mp));
|
||||
router.post("/admin/reader/unblock-server", unblockServerController(mp));
|
||||
router.get("/admin/followers", followersController(mp));
|
||||
router.post("/admin/followers/approve", approveFollowController(mp, this));
|
||||
router.post("/admin/followers/reject", rejectFollowController(mp, this));
|
||||
router.get("/admin/following", followingController(mp));
|
||||
router.get("/admin/activities", activitiesController(mp));
|
||||
router.get("/admin/featured", featuredGetController(mp));
|
||||
@@ -421,7 +435,7 @@ export default class ActivityPubEndpoint {
|
||||
"properties.url": requestUrl,
|
||||
});
|
||||
|
||||
if (!post) {
|
||||
if (!post || post.properties?.deleted) {
|
||||
return next();
|
||||
}
|
||||
|
||||
@@ -510,7 +524,7 @@ export default class ActivityPubEndpoint {
|
||||
let replyToActor = null;
|
||||
if (properties["in-reply-to"]) {
|
||||
try {
|
||||
const remoteObject = await ctx.lookupObject(
|
||||
const remoteObject = await lookupWithSecurity(ctx,
|
||||
new URL(properties["in-reply-to"]),
|
||||
);
|
||||
if (remoteObject && typeof remoteObject.getAttributedTo === "function") {
|
||||
@@ -542,7 +556,7 @@ export default class ActivityPubEndpoint {
|
||||
|
||||
for (const { handle } of mentionHandles) {
|
||||
try {
|
||||
const mentionedActor = await ctx.lookupObject(
|
||||
const mentionedActor = await lookupWithSecurity(ctx,
|
||||
new URL(`acct:${handle}`),
|
||||
);
|
||||
if (mentionedActor?.id) {
|
||||
@@ -718,7 +732,7 @@ export default class ActivityPubEndpoint {
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const remoteActor = await ctx.lookupObject(actorUrl, {
|
||||
const remoteActor = await lookupWithSecurity(ctx,actorUrl, {
|
||||
documentLoader,
|
||||
});
|
||||
if (!remoteActor) {
|
||||
@@ -819,7 +833,7 @@ export default class ActivityPubEndpoint {
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const remoteActor = await ctx.lookupObject(actorUrl, {
|
||||
const remoteActor = await lookupWithSecurity(ctx,actorUrl, {
|
||||
documentLoader,
|
||||
});
|
||||
if (!remoteActor) {
|
||||
@@ -1258,6 +1272,14 @@ export default class ActivityPubEndpoint {
|
||||
Indiekit.addCollection("ap_explore_tabs");
|
||||
// Reports collection
|
||||
Indiekit.addCollection("ap_reports");
|
||||
// Pending follow requests (manual approval)
|
||||
Indiekit.addCollection("ap_pending_follows");
|
||||
// Server-level blocks
|
||||
Indiekit.addCollection("ap_blocked_servers");
|
||||
// Key freshness tracking for proactive refresh
|
||||
Indiekit.addCollection("ap_key_freshness");
|
||||
// Async inbox processing queue
|
||||
Indiekit.addCollection("ap_inbox_queue");
|
||||
|
||||
// Store collection references (posts resolved lazily)
|
||||
const indiekitCollections = Indiekit.collections;
|
||||
@@ -1283,6 +1305,14 @@ export default class ActivityPubEndpoint {
|
||||
ap_explore_tabs: indiekitCollections.get("ap_explore_tabs"),
|
||||
// Reports collection
|
||||
ap_reports: indiekitCollections.get("ap_reports"),
|
||||
// Pending follow requests (manual approval)
|
||||
ap_pending_follows: indiekitCollections.get("ap_pending_follows"),
|
||||
// Server-level blocks
|
||||
ap_blocked_servers: indiekitCollections.get("ap_blocked_servers"),
|
||||
// Key freshness tracking
|
||||
ap_key_freshness: indiekitCollections.get("ap_key_freshness"),
|
||||
// Async inbox processing queue
|
||||
ap_inbox_queue: indiekitCollections.get("ap_inbox_queue"),
|
||||
get posts() {
|
||||
return indiekitCollections.get("posts");
|
||||
},
|
||||
@@ -1474,6 +1504,36 @@ export default class ActivityPubEndpoint {
|
||||
{ reportedUrls: 1 },
|
||||
{ background: true },
|
||||
);
|
||||
// Pending follow requests — unique on actorUrl
|
||||
this._collections.ap_pending_follows.createIndex(
|
||||
{ actorUrl: 1 },
|
||||
{ unique: true, background: true },
|
||||
);
|
||||
this._collections.ap_pending_follows.createIndex(
|
||||
{ requestedAt: -1 },
|
||||
{ background: true },
|
||||
);
|
||||
// Server-level blocks
|
||||
this._collections.ap_blocked_servers.createIndex(
|
||||
{ hostname: 1 },
|
||||
{ unique: true, background: true },
|
||||
);
|
||||
// Key freshness tracking
|
||||
this._collections.ap_key_freshness.createIndex(
|
||||
{ actorUrl: 1 },
|
||||
{ unique: true, background: true },
|
||||
);
|
||||
|
||||
// Inbox queue indexes
|
||||
this._collections.ap_inbox_queue.createIndex(
|
||||
{ status: 1, receivedAt: 1 },
|
||||
{ background: true },
|
||||
);
|
||||
// TTL: auto-prune completed items after 24h
|
||||
this._collections.ap_inbox_queue.createIndex(
|
||||
{ processedAt: 1 },
|
||||
{ expireAfterSeconds: 86_400, background: true },
|
||||
);
|
||||
} catch {
|
||||
// Index creation failed — collections not yet available.
|
||||
// Indexes already exist from previous startups; non-fatal.
|
||||
@@ -1518,7 +1578,7 @@ export default class ActivityPubEndpoint {
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const actor = await ctx.lookupObject(new URL(actorUrl), {
|
||||
const actor = await lookupWithSecurity(ctx,new URL(actorUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
if (!actor) return "";
|
||||
@@ -1569,6 +1629,34 @@ export default class ActivityPubEndpoint {
|
||||
if (this.options.timelineRetention > 0) {
|
||||
scheduleCleanup(this._collections, this.options.timelineRetention);
|
||||
}
|
||||
|
||||
// Load server blocks into Redis for fast inbox checks
|
||||
loadBlockedServersToRedis(this._collections).catch((error) => {
|
||||
console.warn("[ActivityPub] Failed to load blocked servers to Redis:", error.message);
|
||||
});
|
||||
|
||||
// Schedule proactive key refresh for stale follower keys (runs on startup + every 24h)
|
||||
const keyRefreshHandle = this.options.actor.handle;
|
||||
const keyRefreshFederation = this._federation;
|
||||
const keyRefreshPubUrl = this._publicationUrl;
|
||||
scheduleKeyRefresh(
|
||||
this._collections,
|
||||
() => keyRefreshFederation?.createContext(new URL(keyRefreshPubUrl), {
|
||||
handle: keyRefreshHandle,
|
||||
publicationUrl: keyRefreshPubUrl,
|
||||
}),
|
||||
keyRefreshHandle,
|
||||
);
|
||||
|
||||
// Start async inbox queue processor (processes one item every 3s)
|
||||
this._inboxProcessorInterval = startInboxProcessor(
|
||||
this._collections,
|
||||
() => this._federation?.createContext(new URL(this._publicationUrl), {
|
||||
handle: this.options.actor.handle,
|
||||
publicationUrl: this._publicationUrl,
|
||||
}),
|
||||
this.options.actor.handle,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { lookupWithSecurity } from "./lookup-helpers.js";
|
||||
|
||||
/**
|
||||
* Batch re-follow processor for imported accounts.
|
||||
*
|
||||
@@ -232,7 +234,7 @@ async function processOneFollow(options, entry) {
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const remoteActor = await ctx.lookupObject(entry.actorUrl, {
|
||||
const remoteActor = await lookupWithSecurity(ctx,entry.actorUrl, {
|
||||
documentLoader,
|
||||
});
|
||||
if (!remoteActor) {
|
||||
|
||||
@@ -5,36 +5,9 @@
|
||||
import { Create, Note, Mention } from "@fedify/fedify/vocab";
|
||||
import { getToken, validateToken } from "../csrf.js";
|
||||
import { sanitizeContent } from "../timeline-store.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
import { addNotification } from "../storage/notifications.js";
|
||||
|
||||
function createPublicationAwareDocumentLoader(documentLoader, publicationUrl) {
|
||||
if (typeof documentLoader !== "function") {
|
||||
return documentLoader;
|
||||
}
|
||||
|
||||
let publicationHost = "";
|
||||
try {
|
||||
publicationHost = new URL(publicationUrl).hostname;
|
||||
} catch {
|
||||
return documentLoader;
|
||||
}
|
||||
|
||||
return (url, options = {}) => {
|
||||
try {
|
||||
const parsed = new URL(
|
||||
typeof url === "string" ? url : (url?.href || String(url)),
|
||||
);
|
||||
if (parsed.hostname === publicationHost) {
|
||||
return documentLoader(url, { ...options, allowPrivateAddress: true });
|
||||
}
|
||||
} catch {
|
||||
// Fall through to default loader behavior.
|
||||
}
|
||||
|
||||
return documentLoader(url, options);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch syndication targets from the Micropub config endpoint.
|
||||
* @param {object} application - Indiekit application locals
|
||||
@@ -106,14 +79,10 @@ export function composeController(mountPath, plugin) {
|
||||
{ handle, publicationUrl: plugin._publicationUrl },
|
||||
);
|
||||
// Use authenticated document loader for Authorized Fetch
|
||||
const rawDocumentLoader = await ctx.getDocumentLoader({
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const documentLoader = createPublicationAwareDocumentLoader(
|
||||
rawDocumentLoader,
|
||||
plugin._publicationUrl,
|
||||
);
|
||||
const remoteObject = await ctx.lookupObject(new URL(replyTo), {
|
||||
const remoteObject = await lookupWithSecurity(ctx,new URL(replyTo), {
|
||||
documentLoader,
|
||||
});
|
||||
|
||||
@@ -156,19 +125,6 @@ export function composeController(mountPath, plugin) {
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch syndication targets for Micropub path
|
||||
const token = request.session?.access_token;
|
||||
const syndicationTargets = token
|
||||
? await getSyndicationTargets(application, token)
|
||||
: [];
|
||||
|
||||
// Default-check only AP (Fedify) and Bluesky targets
|
||||
// "@rick@rmendes.net" = AP Fedify, "@rmendes.net" = Bluesky
|
||||
for (const target of syndicationTargets) {
|
||||
const name = target.name || "";
|
||||
target.defaultChecked = name === "@rick@rmendes.net" || name === "@rmendes.net";
|
||||
}
|
||||
|
||||
// Check if this is a direct/private message reply by looking at notification metadata
|
||||
let isDirect = false;
|
||||
let senderActorUrl = "";
|
||||
@@ -186,6 +142,19 @@ export function composeController(mountPath, plugin) {
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch syndication targets for Micropub path
|
||||
const token = request.session?.access_token;
|
||||
const syndicationTargets = token
|
||||
? await getSyndicationTargets(application, token)
|
||||
: [];
|
||||
|
||||
// Default-check only AP (Fedify) and Bluesky targets
|
||||
// "@rick@rmendes.net" = AP Fedify, "@rmendes.net" = Bluesky
|
||||
for (const target of syndicationTargets) {
|
||||
const name = target.name || "";
|
||||
target.defaultChecked = name === "@rick@rmendes.net" || name === "@rmendes.net";
|
||||
}
|
||||
|
||||
const csrfToken = getToken(request.session);
|
||||
|
||||
response.render("activitypub-compose", {
|
||||
@@ -194,10 +163,10 @@ export function composeController(mountPath, plugin) {
|
||||
replyTo,
|
||||
replyContext,
|
||||
syndicationTargets,
|
||||
isDirect,
|
||||
senderActorUrl,
|
||||
csrfToken,
|
||||
mountPath,
|
||||
isDirect,
|
||||
senderActorUrl,
|
||||
});
|
||||
} catch (error) {
|
||||
next(error);
|
||||
@@ -206,7 +175,7 @@ export function composeController(mountPath, plugin) {
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /admin/reader/compose — Submit reply via Micropub.
|
||||
* POST /admin/reader/compose — Submit reply via Micropub (public) or native AP (direct).
|
||||
* @param {string} mountPath - Plugin mount path
|
||||
* @param {object} plugin - ActivityPub plugin instance
|
||||
*/
|
||||
@@ -270,7 +239,7 @@ export function submitComposeController(mountPath, plugin) {
|
||||
const documentLoader = await ctx.getDocumentLoader({ identifier: handle });
|
||||
let recipient;
|
||||
try {
|
||||
recipient = await ctx.lookupObject(new URL(senderActorUrl), { documentLoader });
|
||||
recipient = await lookupWithSecurity(ctx, new URL(senderActorUrl), { documentLoader });
|
||||
} catch (lookupError) {
|
||||
console.warn(`[ActivityPub] Actor lookup failed for ${senderActorUrl}:`, lookupError.message);
|
||||
}
|
||||
@@ -359,7 +328,7 @@ export function submitComposeController(mountPath, plugin) {
|
||||
}
|
||||
|
||||
if (cwEnabled && summary && summary.trim()) {
|
||||
micropubData.append("summary", summary.trim());
|
||||
micropubData.append("content-warning", summary.trim());
|
||||
micropubData.append("sensitive", "true");
|
||||
}
|
||||
|
||||
|
||||
@@ -3,8 +3,10 @@
|
||||
* the relationship between local content and the fediverse.
|
||||
*/
|
||||
|
||||
import Redis from "ioredis";
|
||||
import { getToken, validateToken } from "../csrf.js";
|
||||
import { jf2ToActivityStreams } from "../jf2-to-as2.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
|
||||
const PAGE_SIZE = 20;
|
||||
|
||||
@@ -37,10 +39,12 @@ export function federationMgmtController(mountPath, plugin) {
|
||||
const { application } = request.app.locals;
|
||||
const collections = application?.collections;
|
||||
|
||||
const redisUrl = plugin.options.redisUrl || "";
|
||||
|
||||
// Parallel: collection stats + posts + recent activities
|
||||
const [collectionStats, postsResult, recentActivities] =
|
||||
await Promise.all([
|
||||
getCollectionStats(collections),
|
||||
getCollectionStats(collections, { redisUrl }),
|
||||
getPaginatedPosts(collections, request.query.page),
|
||||
getRecentActivities(collections),
|
||||
]);
|
||||
@@ -219,7 +223,7 @@ export function lookupObjectController(mountPath, plugin) {
|
||||
identifier: handle,
|
||||
});
|
||||
|
||||
const object = await ctx.lookupObject(query, { documentLoader });
|
||||
const object = await lookupWithSecurity(ctx,query, { documentLoader });
|
||||
|
||||
if (!object) {
|
||||
return response
|
||||
@@ -239,11 +243,16 @@ export function lookupObjectController(mountPath, plugin) {
|
||||
|
||||
// --- Helpers ---
|
||||
|
||||
async function getCollectionStats(collections) {
|
||||
async function getCollectionStats(collections, { redisUrl = "" } = {}) {
|
||||
if (!collections) return [];
|
||||
|
||||
const stats = await Promise.all(
|
||||
AP_COLLECTIONS.map(async (name) => {
|
||||
// When Redis handles KV, count fedify::* keys from Redis instead
|
||||
if (name === "ap_kv" && redisUrl) {
|
||||
const count = await countRedisKvKeys(redisUrl);
|
||||
return { name: "ap_kv (redis)", count };
|
||||
}
|
||||
const col = collections.get(name);
|
||||
const count = col ? await col.countDocuments() : 0;
|
||||
return { name, count };
|
||||
@@ -253,6 +262,36 @@ async function getCollectionStats(collections) {
|
||||
return stats;
|
||||
}
|
||||
|
||||
/**
|
||||
* Count Fedify KV keys in Redis (prefix: "fedify::").
|
||||
* Uses SCAN to avoid blocking on large key spaces.
|
||||
*/
|
||||
async function countRedisKvKeys(redisUrl) {
|
||||
let client;
|
||||
try {
|
||||
client = new Redis(redisUrl, { lazyConnect: true, connectTimeout: 3000 });
|
||||
await client.connect();
|
||||
let count = 0;
|
||||
let cursor = "0";
|
||||
do {
|
||||
const [nextCursor, keys] = await client.scan(
|
||||
cursor,
|
||||
"MATCH",
|
||||
"fedify::*",
|
||||
"COUNT",
|
||||
500,
|
||||
);
|
||||
cursor = nextCursor;
|
||||
count += keys.length;
|
||||
} while (cursor !== "0");
|
||||
return count;
|
||||
} catch {
|
||||
return 0;
|
||||
} finally {
|
||||
client?.disconnect();
|
||||
}
|
||||
}
|
||||
|
||||
async function getPaginatedPosts(collections, pageParam) {
|
||||
const postsCol = collections?.get("posts");
|
||||
if (!postsCol) return { posts: [], cursor: null };
|
||||
|
||||
253
lib/controllers/follow-requests.js
Normal file
253
lib/controllers/follow-requests.js
Normal file
@@ -0,0 +1,253 @@
|
||||
/**
|
||||
* Follow request controllers — approve and reject pending follow requests
|
||||
* when manual follow approval is enabled.
|
||||
*/
|
||||
|
||||
import { validateToken } from "../csrf.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
import { logActivity } from "../activity-log.js";
|
||||
import { addNotification } from "../storage/notifications.js";
|
||||
import { extractActorInfo } from "../timeline-store.js";
|
||||
|
||||
/**
|
||||
* POST /admin/followers/approve — Accept a pending follow request.
|
||||
*/
|
||||
export function approveFollowController(mountPath, plugin) {
|
||||
return async (request, response, next) => {
|
||||
try {
|
||||
if (!validateToken(request)) {
|
||||
return response.status(403).json({
|
||||
success: false,
|
||||
error: "Invalid CSRF token",
|
||||
});
|
||||
}
|
||||
|
||||
const { actorUrl } = request.body;
|
||||
|
||||
if (!actorUrl) {
|
||||
return response.status(400).json({
|
||||
success: false,
|
||||
error: "Missing actor URL",
|
||||
});
|
||||
}
|
||||
|
||||
const { application } = request.app.locals;
|
||||
const pendingCol = application?.collections?.get("ap_pending_follows");
|
||||
const followersCol = application?.collections?.get("ap_followers");
|
||||
|
||||
if (!pendingCol || !followersCol) {
|
||||
return response.status(503).json({
|
||||
success: false,
|
||||
error: "Collections not available",
|
||||
});
|
||||
}
|
||||
|
||||
// Find the pending request
|
||||
const pending = await pendingCol.findOne({ actorUrl });
|
||||
if (!pending) {
|
||||
return response.status(404).json({
|
||||
success: false,
|
||||
error: "No pending follow request from this actor",
|
||||
});
|
||||
}
|
||||
|
||||
// Move to ap_followers
|
||||
await followersCol.updateOne(
|
||||
{ actorUrl },
|
||||
{
|
||||
$set: {
|
||||
actorUrl: pending.actorUrl,
|
||||
handle: pending.handle || "",
|
||||
name: pending.name || "",
|
||||
avatar: pending.avatar || "",
|
||||
inbox: pending.inbox || "",
|
||||
sharedInbox: pending.sharedInbox || "",
|
||||
followedAt: new Date().toISOString(),
|
||||
},
|
||||
},
|
||||
{ upsert: true },
|
||||
);
|
||||
|
||||
// Remove from pending
|
||||
await pendingCol.deleteOne({ actorUrl });
|
||||
|
||||
// Send Accept(Follow) via federation
|
||||
if (plugin._federation) {
|
||||
try {
|
||||
const { Accept, Follow } = await import("@fedify/fedify/vocab");
|
||||
const handle = plugin.options.actor.handle;
|
||||
const ctx = plugin._federation.createContext(
|
||||
new URL(plugin._publicationUrl),
|
||||
{ handle, publicationUrl: plugin._publicationUrl },
|
||||
);
|
||||
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
|
||||
// Resolve the remote actor for delivery
|
||||
const remoteActor = await lookupWithSecurity(ctx, new URL(actorUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
|
||||
if (remoteActor) {
|
||||
// Reconstruct the Follow using stored activity ID
|
||||
const followObj = new Follow({
|
||||
id: pending.followActivityId
|
||||
? new URL(pending.followActivityId)
|
||||
: undefined,
|
||||
actor: new URL(actorUrl),
|
||||
object: ctx.getActorUri(handle),
|
||||
});
|
||||
|
||||
await ctx.sendActivity(
|
||||
{ identifier: handle },
|
||||
remoteActor,
|
||||
new Accept({
|
||||
actor: ctx.getActorUri(handle),
|
||||
object: followObj,
|
||||
}),
|
||||
{ orderingKey: actorUrl },
|
||||
);
|
||||
}
|
||||
|
||||
const activitiesCol = application?.collections?.get("ap_activities");
|
||||
if (activitiesCol) {
|
||||
await logActivity(activitiesCol, {
|
||||
direction: "outbound",
|
||||
type: "Accept(Follow)",
|
||||
actorUrl: plugin._publicationUrl,
|
||||
objectUrl: actorUrl,
|
||||
actorName: pending.name || actorUrl,
|
||||
summary: `Approved follow request from ${pending.name || actorUrl}`,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
`[ActivityPub] Could not send Accept to ${actorUrl}: ${error.message}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
console.info(
|
||||
`[ActivityPub] Approved follow request from ${pending.name || actorUrl}`,
|
||||
);
|
||||
|
||||
// Redirect back to followers page
|
||||
return response.redirect(`${mountPath}/admin/followers`);
|
||||
} catch (error) {
|
||||
next(error);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /admin/followers/reject — Reject a pending follow request.
|
||||
*/
|
||||
export function rejectFollowController(mountPath, plugin) {
|
||||
return async (request, response, next) => {
|
||||
try {
|
||||
if (!validateToken(request)) {
|
||||
return response.status(403).json({
|
||||
success: false,
|
||||
error: "Invalid CSRF token",
|
||||
});
|
||||
}
|
||||
|
||||
const { actorUrl } = request.body;
|
||||
|
||||
if (!actorUrl) {
|
||||
return response.status(400).json({
|
||||
success: false,
|
||||
error: "Missing actor URL",
|
||||
});
|
||||
}
|
||||
|
||||
const { application } = request.app.locals;
|
||||
const pendingCol = application?.collections?.get("ap_pending_follows");
|
||||
|
||||
if (!pendingCol) {
|
||||
return response.status(503).json({
|
||||
success: false,
|
||||
error: "Collections not available",
|
||||
});
|
||||
}
|
||||
|
||||
// Find the pending request
|
||||
const pending = await pendingCol.findOne({ actorUrl });
|
||||
if (!pending) {
|
||||
return response.status(404).json({
|
||||
success: false,
|
||||
error: "No pending follow request from this actor",
|
||||
});
|
||||
}
|
||||
|
||||
// Remove from pending
|
||||
await pendingCol.deleteOne({ actorUrl });
|
||||
|
||||
// Send Reject(Follow) via federation
|
||||
if (plugin._federation) {
|
||||
try {
|
||||
const { Reject, Follow } = await import("@fedify/fedify/vocab");
|
||||
const handle = plugin.options.actor.handle;
|
||||
const ctx = plugin._federation.createContext(
|
||||
new URL(plugin._publicationUrl),
|
||||
{ handle, publicationUrl: plugin._publicationUrl },
|
||||
);
|
||||
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
|
||||
const remoteActor = await lookupWithSecurity(ctx, new URL(actorUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
|
||||
if (remoteActor) {
|
||||
const followObj = new Follow({
|
||||
id: pending.followActivityId
|
||||
? new URL(pending.followActivityId)
|
||||
: undefined,
|
||||
actor: new URL(actorUrl),
|
||||
object: ctx.getActorUri(handle),
|
||||
});
|
||||
|
||||
await ctx.sendActivity(
|
||||
{ identifier: handle },
|
||||
remoteActor,
|
||||
new Reject({
|
||||
actor: ctx.getActorUri(handle),
|
||||
object: followObj,
|
||||
}),
|
||||
{ orderingKey: actorUrl },
|
||||
);
|
||||
}
|
||||
|
||||
const activitiesCol = application?.collections?.get("ap_activities");
|
||||
if (activitiesCol) {
|
||||
await logActivity(activitiesCol, {
|
||||
direction: "outbound",
|
||||
type: "Reject(Follow)",
|
||||
actorUrl: plugin._publicationUrl,
|
||||
objectUrl: actorUrl,
|
||||
actorName: pending.name || actorUrl,
|
||||
summary: `Rejected follow request from ${pending.name || actorUrl}`,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
`[ActivityPub] Could not send Reject to ${actorUrl}: ${error.message}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
console.info(
|
||||
`[ActivityPub] Rejected follow request from ${pending.name || actorUrl}`,
|
||||
);
|
||||
|
||||
return response.redirect(`${mountPath}/admin/followers`);
|
||||
} catch (error) {
|
||||
next(error);
|
||||
}
|
||||
};
|
||||
}
|
||||
@@ -1,6 +1,9 @@
|
||||
/**
|
||||
* Followers list controller — paginated list of accounts following this actor.
|
||||
* Followers list controller — paginated list of accounts following this actor,
|
||||
* with pending follow requests tab when manual approval is enabled.
|
||||
*/
|
||||
import { getToken } from "../csrf.js";
|
||||
|
||||
const PAGE_SIZE = 20;
|
||||
|
||||
export function followersController(mountPath) {
|
||||
@@ -8,6 +11,9 @@ export function followersController(mountPath) {
|
||||
try {
|
||||
const { application } = request.app.locals;
|
||||
const collection = application?.collections?.get("ap_followers");
|
||||
const pendingCol = application?.collections?.get("ap_pending_follows");
|
||||
|
||||
const tab = request.query.tab || "followers";
|
||||
|
||||
if (!collection) {
|
||||
return response.render("activitypub-followers", {
|
||||
@@ -15,11 +21,50 @@ export function followersController(mountPath) {
|
||||
parent: { href: mountPath, text: response.locals.__("activitypub.title") },
|
||||
followers: [],
|
||||
followerCount: 0,
|
||||
pendingFollows: [],
|
||||
pendingCount: 0,
|
||||
tab,
|
||||
mountPath,
|
||||
csrfToken: getToken(request),
|
||||
});
|
||||
}
|
||||
|
||||
const page = Math.max(1, Number.parseInt(request.query.page, 10) || 1);
|
||||
|
||||
// Count pending follow requests
|
||||
const pendingCount = pendingCol
|
||||
? await pendingCol.countDocuments()
|
||||
: 0;
|
||||
|
||||
if (tab === "pending") {
|
||||
// Show pending follow requests
|
||||
const totalPages = Math.ceil(pendingCount / PAGE_SIZE);
|
||||
const pendingFollows = pendingCol
|
||||
? await pendingCol
|
||||
.find()
|
||||
.sort({ requestedAt: -1 })
|
||||
.skip((page - 1) * PAGE_SIZE)
|
||||
.limit(PAGE_SIZE)
|
||||
.toArray()
|
||||
: [];
|
||||
|
||||
const cursor = buildCursor(page, totalPages, mountPath + "/admin/followers?tab=pending");
|
||||
|
||||
return response.render("activitypub-followers", {
|
||||
title: response.locals.__("activitypub.followers"),
|
||||
parent: { href: mountPath, text: response.locals.__("activitypub.title") },
|
||||
followers: [],
|
||||
followerCount: await collection.countDocuments(),
|
||||
pendingFollows,
|
||||
pendingCount,
|
||||
tab,
|
||||
mountPath,
|
||||
cursor,
|
||||
csrfToken: getToken(request),
|
||||
});
|
||||
}
|
||||
|
||||
// Show accepted followers (default)
|
||||
const totalCount = await collection.countDocuments();
|
||||
const totalPages = Math.ceil(totalCount / PAGE_SIZE);
|
||||
|
||||
@@ -37,8 +82,12 @@ export function followersController(mountPath) {
|
||||
parent: { href: mountPath, text: response.locals.__("activitypub.title") },
|
||||
followers,
|
||||
followerCount: totalCount,
|
||||
pendingFollows: [],
|
||||
pendingCount,
|
||||
tab,
|
||||
mountPath,
|
||||
cursor,
|
||||
csrfToken: getToken(request),
|
||||
});
|
||||
} catch (error) {
|
||||
next(error);
|
||||
@@ -49,12 +98,14 @@ export function followersController(mountPath) {
|
||||
function buildCursor(page, totalPages, basePath) {
|
||||
if (totalPages <= 1) return null;
|
||||
|
||||
const separator = basePath.includes("?") ? "&" : "?";
|
||||
|
||||
return {
|
||||
previous: page > 1
|
||||
? { href: `${basePath}?page=${page - 1}` }
|
||||
? { href: `${basePath}${separator}page=${page - 1}` }
|
||||
: undefined,
|
||||
next: page < totalPages
|
||||
? { href: `${basePath}?page=${page + 1}` }
|
||||
? { href: `${basePath}${separator}page=${page + 1}` }
|
||||
: undefined,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -198,6 +198,7 @@ export function unboostController(mountPath, plugin) {
|
||||
// Send to followers
|
||||
await ctx.sendActivity({ identifier: handle }, "followers", undo, {
|
||||
preferSharedInbox: true,
|
||||
syncCollection: true,
|
||||
orderingKey: url,
|
||||
});
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
import { getToken, validateToken } from "../csrf.js";
|
||||
import { sanitizeContent } from "../timeline-store.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
import {
|
||||
getMessages,
|
||||
getConversationPartners,
|
||||
@@ -180,11 +181,11 @@ export function submitMessageController(mountPath, plugin) {
|
||||
try {
|
||||
const recipientInput = to.trim();
|
||||
if (recipientInput.startsWith("http")) {
|
||||
recipient = await ctx.lookupObject(recipientInput, { documentLoader });
|
||||
recipient = await lookupWithSecurity(ctx,recipientInput, { documentLoader });
|
||||
} else {
|
||||
// Handle @user@domain format
|
||||
const handle = recipientInput.replace(/^@/, "");
|
||||
recipient = await ctx.lookupObject(handle, { documentLoader });
|
||||
recipient = await lookupWithSecurity(ctx,handle, { documentLoader });
|
||||
}
|
||||
} catch {
|
||||
recipient = null;
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
*/
|
||||
|
||||
import { validateToken, getToken } from "../csrf.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
import {
|
||||
addMuted,
|
||||
removeMuted,
|
||||
@@ -13,6 +14,11 @@ import {
|
||||
getFilterMode,
|
||||
setFilterMode,
|
||||
} from "../storage/moderation.js";
|
||||
import {
|
||||
addBlockedServer,
|
||||
removeBlockedServer,
|
||||
getAllBlockedServers,
|
||||
} from "../storage/server-blocks.js";
|
||||
|
||||
/**
|
||||
* Helper to get moderation collections from request.
|
||||
@@ -22,6 +28,7 @@ function getModerationCollections(request) {
|
||||
return {
|
||||
ap_muted: application?.collections?.get("ap_muted"),
|
||||
ap_blocked: application?.collections?.get("ap_blocked"),
|
||||
ap_blocked_servers: application?.collections?.get("ap_blocked_servers"),
|
||||
ap_timeline: application?.collections?.get("ap_timeline"),
|
||||
ap_profile: application?.collections?.get("ap_profile"),
|
||||
};
|
||||
@@ -157,7 +164,7 @@ export function blockController(mountPath, plugin) {
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const remoteActor = await ctx.lookupObject(new URL(url), {
|
||||
const remoteActor = await lookupWithSecurity(ctx,new URL(url), {
|
||||
documentLoader,
|
||||
});
|
||||
|
||||
@@ -236,7 +243,7 @@ export function unblockController(mountPath, plugin) {
|
||||
const documentLoader = await ctx.getDocumentLoader({
|
||||
identifier: handle,
|
||||
});
|
||||
const remoteActor = await ctx.lookupObject(new URL(url), {
|
||||
const remoteActor = await lookupWithSecurity(ctx,new URL(url), {
|
||||
documentLoader,
|
||||
});
|
||||
|
||||
@@ -281,6 +288,77 @@ export function unblockController(mountPath, plugin) {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /admin/reader/block-server — Block a server by hostname.
|
||||
*/
|
||||
export function blockServerController(mountPath) {
|
||||
return async (request, response, next) => {
|
||||
try {
|
||||
if (!validateToken(request)) {
|
||||
return response.status(403).json({
|
||||
success: false,
|
||||
error: "Invalid CSRF token",
|
||||
});
|
||||
}
|
||||
|
||||
const { hostname, reason } = request.body;
|
||||
if (!hostname) {
|
||||
return response.status(400).json({
|
||||
success: false,
|
||||
error: "Missing hostname",
|
||||
});
|
||||
}
|
||||
|
||||
const collections = getModerationCollections(request);
|
||||
await addBlockedServer(collections, hostname, reason);
|
||||
|
||||
console.info(`[ActivityPub] Blocked server: ${hostname}`);
|
||||
return response.json({ success: true, type: "block-server", hostname });
|
||||
} catch (error) {
|
||||
console.error("[ActivityPub] Block server failed:", error.message);
|
||||
return response.status(500).json({
|
||||
success: false,
|
||||
error: "Operation failed. Please try again later.",
|
||||
});
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* POST /admin/reader/unblock-server — Unblock a server.
|
||||
*/
|
||||
export function unblockServerController(mountPath) {
|
||||
return async (request, response, next) => {
|
||||
try {
|
||||
if (!validateToken(request)) {
|
||||
return response.status(403).json({
|
||||
success: false,
|
||||
error: "Invalid CSRF token",
|
||||
});
|
||||
}
|
||||
|
||||
const { hostname } = request.body;
|
||||
if (!hostname) {
|
||||
return response.status(400).json({
|
||||
success: false,
|
||||
error: "Missing hostname",
|
||||
});
|
||||
}
|
||||
|
||||
const collections = getModerationCollections(request);
|
||||
await removeBlockedServer(collections, hostname);
|
||||
|
||||
console.info(`[ActivityPub] Unblocked server: ${hostname}`);
|
||||
return response.json({ success: true, type: "unblock-server", hostname });
|
||||
} catch (error) {
|
||||
return response.status(500).json({
|
||||
success: false,
|
||||
error: "Operation failed. Please try again later.",
|
||||
});
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* GET /admin/reader/moderation — View muted/blocked lists.
|
||||
*/
|
||||
@@ -290,9 +368,10 @@ export function moderationController(mountPath) {
|
||||
const collections = getModerationCollections(request);
|
||||
const csrfToken = getToken(request.session);
|
||||
|
||||
const [muted, blocked, filterMode] = await Promise.all([
|
||||
const [muted, blocked, blockedServers, filterMode] = await Promise.all([
|
||||
getAllMuted(collections),
|
||||
getAllBlocked(collections),
|
||||
getAllBlockedServers(collections),
|
||||
getFilterMode(collections),
|
||||
]);
|
||||
|
||||
@@ -304,6 +383,7 @@ export function moderationController(mountPath) {
|
||||
readerParent: { href: `${mountPath}/admin/reader`, text: response.locals.__("activitypub.reader.title") },
|
||||
muted,
|
||||
blocked,
|
||||
blockedServers,
|
||||
mutedActors,
|
||||
mutedKeywords,
|
||||
filterMode,
|
||||
|
||||
@@ -4,6 +4,7 @@ import { getToken } from "../csrf.js";
|
||||
import { extractObjectData, extractActorInfo } from "../timeline-store.js";
|
||||
import { getCached, setCache } from "../lookup-cache.js";
|
||||
import { fetchAndStoreQuote, stripQuoteReferenceHtml } from "../og-unfurl.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
|
||||
// Load parent posts (inReplyTo chain) up to maxDepth levels
|
||||
async function loadParentChain(ctx, documentLoader, timelineCol, parentUrl, maxDepth = 5) {
|
||||
@@ -28,7 +29,7 @@ async function loadParentChain(ctx, documentLoader, timelineCol, parentUrl, maxD
|
||||
|
||||
if (!object) {
|
||||
try {
|
||||
object = await ctx.lookupObject(new URL(currentUrl), {
|
||||
object = await lookupWithSecurity(ctx,new URL(currentUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
if (object) {
|
||||
@@ -61,25 +62,52 @@ async function loadParentChain(ctx, documentLoader, timelineCol, parentUrl, maxD
|
||||
return parents;
|
||||
}
|
||||
|
||||
// Load replies collection (best-effort)
|
||||
async function loadReplies(object, ctx, documentLoader, timelineCol, maxReplies = 10) {
|
||||
const replies = [];
|
||||
// Load local replies from ap_timeline (items where inReplyTo matches this post)
|
||||
async function loadLocalReplies(timelineCol, postUrl, postUid, maxReplies = 20) {
|
||||
if (!timelineCol) return [];
|
||||
|
||||
const matchUrls = [postUrl, postUid].filter(Boolean);
|
||||
if (matchUrls.length === 0) return [];
|
||||
|
||||
const localReplies = await timelineCol
|
||||
.find({ inReplyTo: { $in: matchUrls } })
|
||||
.sort({ published: 1 })
|
||||
.limit(maxReplies)
|
||||
.toArray();
|
||||
|
||||
return localReplies;
|
||||
}
|
||||
|
||||
// Load replies collection (best-effort) — merges local + remote
|
||||
async function loadReplies(object, ctx, documentLoader, timelineCol, maxReplies = 20) {
|
||||
const postUrl = object?.id?.href || object?.url?.href;
|
||||
|
||||
// Start with local replies already in our timeline (from organic inbox delivery
|
||||
// or reply chain fetching). These are fast and free — no network requests.
|
||||
const seenUrls = new Set();
|
||||
const replies = await loadLocalReplies(timelineCol, postUrl, postUrl, maxReplies);
|
||||
for (const r of replies) {
|
||||
if (r.uid) seenUrls.add(r.uid);
|
||||
if (r.url) seenUrls.add(r.url);
|
||||
}
|
||||
|
||||
// Supplement with remote replies collection (may contain items we don't have locally)
|
||||
if (object && replies.length < maxReplies) {
|
||||
try {
|
||||
const repliesCollection = await object.getReplies({ documentLoader });
|
||||
if (!repliesCollection) return replies;
|
||||
|
||||
if (repliesCollection) {
|
||||
let items = [];
|
||||
try {
|
||||
items = await repliesCollection.getItems({ documentLoader });
|
||||
} catch {
|
||||
return replies;
|
||||
// Remote fetch failed — continue with local replies only
|
||||
}
|
||||
|
||||
for (const replyItem of items.slice(0, maxReplies)) {
|
||||
for (const replyItem of items.slice(0, maxReplies - replies.length)) {
|
||||
try {
|
||||
const replyUrl = replyItem.id?.href || replyItem.url?.href;
|
||||
if (!replyUrl) continue;
|
||||
if (!replyUrl || seenUrls.has(replyUrl)) continue;
|
||||
seenUrls.add(replyUrl);
|
||||
|
||||
// Check timeline first
|
||||
let reply = timelineCol
|
||||
@@ -102,9 +130,18 @@ async function loadReplies(object, ctx, documentLoader, timelineCol, maxReplies
|
||||
continue; // Skip failed replies
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// getReplies() failed or not available
|
||||
}
|
||||
}
|
||||
|
||||
// Sort all replies chronologically
|
||||
replies.sort((a, b) => {
|
||||
const dateA = a.published || "";
|
||||
const dateB = b.published || "";
|
||||
return dateA < dateB ? -1 : dateA > dateB ? 1 : 0;
|
||||
});
|
||||
|
||||
return replies;
|
||||
}
|
||||
@@ -180,7 +217,7 @@ export function postDetailController(mountPath, plugin) {
|
||||
object = cached;
|
||||
} else {
|
||||
try {
|
||||
object = await ctx.lookupObject(new URL(objectUrl), {
|
||||
object = await lookupWithSecurity(ctx,new URL(objectUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
if (object) {
|
||||
@@ -326,7 +363,7 @@ export function postDetailController(mountPath, plugin) {
|
||||
);
|
||||
const qLoader = await qCtx.getDocumentLoader({ identifier: handle });
|
||||
|
||||
const quoteObject = await qCtx.lookupObject(new URL(timelineItem.quoteUrl), {
|
||||
const quoteObject = await lookupWithSecurity(qCtx,new URL(timelineItem.quoteUrl), {
|
||||
documentLoader: qLoader,
|
||||
});
|
||||
|
||||
@@ -336,7 +373,7 @@ export function postDetailController(mountPath, plugin) {
|
||||
// If author photo is empty, try fetching the actor directly
|
||||
if (!quoteData.author.photo && quoteData.author.url) {
|
||||
try {
|
||||
const actor = await qCtx.lookupObject(new URL(quoteData.author.url), { documentLoader: qLoader });
|
||||
const actor = await lookupWithSecurity(qCtx,new URL(quoteData.author.url), { documentLoader: qLoader });
|
||||
if (actor) {
|
||||
const actorInfo = await extractActorInfo(actor, { documentLoader: qLoader });
|
||||
if (actorInfo.photo) quoteData.author.photo = actorInfo.photo;
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
|
||||
import { getToken, validateToken } from "../csrf.js";
|
||||
import { sanitizeContent } from "../timeline-store.js";
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
|
||||
/**
|
||||
* GET /admin/reader/profile — Show remote actor profile.
|
||||
@@ -43,7 +44,7 @@ export function remoteProfileController(mountPath, plugin) {
|
||||
let actor;
|
||||
|
||||
try {
|
||||
actor = await ctx.lookupObject(new URL(actorUrl), { documentLoader });
|
||||
actor = await lookupWithSecurity(ctx,new URL(actorUrl), { documentLoader });
|
||||
} catch {
|
||||
return response.status(404).render("error", {
|
||||
title: "Error",
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
* Resolve controller — accepts any fediverse URL or handle, resolves it
|
||||
* via lookupObject(), and redirects to the appropriate internal view.
|
||||
*/
|
||||
import { lookupWithSecurity } from "../lookup-helpers.js";
|
||||
import {
|
||||
Article,
|
||||
Note,
|
||||
@@ -59,7 +60,7 @@ export function resolveController(mountPath, plugin) {
|
||||
let object;
|
||||
|
||||
try {
|
||||
object = await ctx.lookupObject(lookupInput, { documentLoader });
|
||||
object = await lookupWithSecurity(ctx,lookupInput, { documentLoader });
|
||||
} catch (error) {
|
||||
console.warn(
|
||||
`[resolve] lookupObject failed for "${query}":`,
|
||||
|
||||
@@ -92,6 +92,12 @@ async function sendFedifyResponse(res, response, request) {
|
||||
if (json.attachment && !Array.isArray(json.attachment)) {
|
||||
json.attachment = [json.attachment];
|
||||
}
|
||||
// WORKAROUND: Fedify serializes endpoints with "type": "as:Endpoints"
|
||||
// which is not a valid AS2 type. The endpoints object should be a plain
|
||||
// object with just sharedInbox/proxyUrl etc. Strip the invalid type.
|
||||
if (json.endpoints && json.endpoints.type) {
|
||||
delete json.endpoints.type;
|
||||
}
|
||||
const patched = JSON.stringify(json);
|
||||
res.setHeader("content-length", Buffer.byteLength(patched));
|
||||
res.end(patched);
|
||||
|
||||
@@ -40,6 +40,10 @@ import Redis from "ioredis";
|
||||
import { MongoKvStore } from "./kv-store.js";
|
||||
import { registerInboxListeners } from "./inbox-listeners.js";
|
||||
import { jf2ToAS2Activity, resolvePostUrl } from "./jf2-to-as2.js";
|
||||
import { cachedQuery } from "./redis-cache.js";
|
||||
import { onOutboxPermanentFailure } from "./outbox-failure.js";
|
||||
|
||||
const COLLECTION_CACHE_TTL = 300; // 5 minutes
|
||||
|
||||
/**
|
||||
* Create and configure a Fedify Federation instance.
|
||||
@@ -350,25 +354,15 @@ export function setupFederation(options) {
|
||||
});
|
||||
|
||||
// Handle permanent delivery failures (Fedify 2.0).
|
||||
// Fires when a remote inbox returns 404/410 — the server is gone.
|
||||
// Log it and let the admin see which followers are unreachable.
|
||||
// Fires when a remote inbox returns 404/410.
|
||||
// 410: immediate full cleanup. 404: strike system (3 strikes over 7 days).
|
||||
federation.setOutboxPermanentFailureHandler(async (_ctx, values) => {
|
||||
const { inbox, error, actorIds } = values;
|
||||
const inboxUrl = inbox?.href || String(inbox);
|
||||
const actors = actorIds?.map((id) => id?.href || String(id)) || [];
|
||||
console.warn(
|
||||
`[ActivityPub] Permanent delivery failure to ${inboxUrl}: ${error?.message || "unknown"}` +
|
||||
(actors.length ? ` (actors: ${actors.join(", ")})` : ""),
|
||||
await onOutboxPermanentFailure(
|
||||
values.statusCode,
|
||||
values.actorIds,
|
||||
values.inbox,
|
||||
collections,
|
||||
);
|
||||
collections.ap_activities.insertOne({
|
||||
direction: "outbound",
|
||||
type: "DeliveryFailed",
|
||||
actorUrl: publicationUrl,
|
||||
objectUrl: inboxUrl,
|
||||
summary: `Permanent delivery failure to ${inboxUrl}: ${error?.message || "unknown"}`,
|
||||
affectedActors: actors,
|
||||
receivedAt: new Date().toISOString(),
|
||||
}).catch(() => {});
|
||||
});
|
||||
|
||||
// Wrap with debug dashboard if enabled. The debugger proxies the
|
||||
@@ -415,10 +409,12 @@ function setupFollowers(federation, mountPath, handle, collections) {
|
||||
// as Recipient objects so sendActivity("followers") can deliver.
|
||||
// See: https://fedify.dev/manual/collections#one-shot-followers-collection-for-gathering-recipients
|
||||
if (cursor == null) {
|
||||
const docs = await collections.ap_followers
|
||||
const docs = await cachedQuery("col:followers:recipients", COLLECTION_CACHE_TTL, async () => {
|
||||
return await collections.ap_followers
|
||||
.find()
|
||||
.sort({ followedAt: -1 })
|
||||
.toArray();
|
||||
});
|
||||
return {
|
||||
items: docs.map((f) => ({
|
||||
id: new URL(f.actorUrl),
|
||||
@@ -433,13 +429,16 @@ function setupFollowers(federation, mountPath, handle, collections) {
|
||||
// Paginated collection: for remote browsing of /followers endpoint
|
||||
const pageSize = 20;
|
||||
const skip = Number.parseInt(cursor, 10);
|
||||
const docs = await collections.ap_followers
|
||||
const [docs, total] = await cachedQuery(`col:followers:page:${cursor}`, COLLECTION_CACHE_TTL, async () => {
|
||||
const d = await collections.ap_followers
|
||||
.find()
|
||||
.sort({ followedAt: -1 })
|
||||
.skip(skip)
|
||||
.limit(pageSize)
|
||||
.toArray();
|
||||
const total = await collections.ap_followers.countDocuments();
|
||||
const t = await collections.ap_followers.countDocuments();
|
||||
return [d, t];
|
||||
});
|
||||
|
||||
return {
|
||||
items: docs.map((f) => new URL(f.actorUrl)),
|
||||
@@ -450,7 +449,9 @@ function setupFollowers(federation, mountPath, handle, collections) {
|
||||
)
|
||||
.setCounter(async (ctx, identifier) => {
|
||||
if (identifier !== handle) return 0;
|
||||
return await cachedQuery("col:followers:count", COLLECTION_CACHE_TTL, async () => {
|
||||
return await collections.ap_followers.countDocuments();
|
||||
});
|
||||
})
|
||||
.setFirstCursor(async () => "0");
|
||||
}
|
||||
@@ -463,13 +464,16 @@ function setupFollowing(federation, mountPath, handle, collections) {
|
||||
if (identifier !== handle) return null;
|
||||
const pageSize = 20;
|
||||
const skip = cursor ? Number.parseInt(cursor, 10) : 0;
|
||||
const docs = await collections.ap_following
|
||||
const [docs, total] = await cachedQuery(`col:following:page:${cursor}`, COLLECTION_CACHE_TTL, async () => {
|
||||
const d = await collections.ap_following
|
||||
.find()
|
||||
.sort({ followedAt: -1 })
|
||||
.skip(skip)
|
||||
.limit(pageSize)
|
||||
.toArray();
|
||||
const total = await collections.ap_following.countDocuments();
|
||||
const t = await collections.ap_following.countDocuments();
|
||||
return [d, t];
|
||||
});
|
||||
|
||||
return {
|
||||
items: docs.map((f) => new URL(f.actorUrl)),
|
||||
@@ -480,7 +484,9 @@ function setupFollowing(federation, mountPath, handle, collections) {
|
||||
)
|
||||
.setCounter(async (ctx, identifier) => {
|
||||
if (identifier !== handle) return 0;
|
||||
return await cachedQuery("col:following:count", COLLECTION_CACHE_TTL, async () => {
|
||||
return await collections.ap_following.countDocuments();
|
||||
});
|
||||
})
|
||||
.setFirstCursor(async () => "0");
|
||||
}
|
||||
@@ -496,13 +502,16 @@ function setupLiked(federation, mountPath, handle, collections) {
|
||||
const pageSize = 20;
|
||||
const skip = cursor ? Number.parseInt(cursor, 10) : 0;
|
||||
const query = { "properties.post-type": "like" };
|
||||
const docs = await collections.posts
|
||||
const [docs, total] = await cachedQuery(`col:liked:page:${cursor}`, COLLECTION_CACHE_TTL, async () => {
|
||||
const d = await collections.posts
|
||||
.find(query)
|
||||
.sort({ "properties.published": -1 })
|
||||
.skip(skip)
|
||||
.limit(pageSize)
|
||||
.toArray();
|
||||
const total = await collections.posts.countDocuments(query);
|
||||
const t = await collections.posts.countDocuments(query);
|
||||
return [d, t];
|
||||
});
|
||||
|
||||
const items = docs
|
||||
.map((d) => {
|
||||
@@ -521,9 +530,11 @@ function setupLiked(federation, mountPath, handle, collections) {
|
||||
.setCounter(async (ctx, identifier) => {
|
||||
if (identifier !== handle) return 0;
|
||||
if (!collections.posts) return 0;
|
||||
return await cachedQuery("col:liked:count", COLLECTION_CACHE_TTL, async () => {
|
||||
return await collections.posts.countDocuments({
|
||||
"properties.post-type": "like",
|
||||
});
|
||||
});
|
||||
})
|
||||
.setFirstCursor(async () => "0");
|
||||
}
|
||||
@@ -612,6 +623,7 @@ function setupOutbox(federation, mountPath, handle, collections) {
|
||||
const federationVisibilityQuery = {
|
||||
"properties.post-status": { $ne: "draft" },
|
||||
"properties.visibility": { $ne: "unlisted" },
|
||||
"properties.deleted": { $exists: false },
|
||||
};
|
||||
const total = await postsCollection.countDocuments(
|
||||
federationVisibilityQuery,
|
||||
@@ -653,6 +665,7 @@ function setupOutbox(federation, mountPath, handle, collections) {
|
||||
return await postsCollection.countDocuments({
|
||||
"properties.post-status": { $ne: "draft" },
|
||||
"properties.visibility": { $ne: "unlisted" },
|
||||
"properties.deleted": { $exists: false },
|
||||
});
|
||||
})
|
||||
.setFirstCursor(async () => "0");
|
||||
@@ -671,6 +684,8 @@ function setupObjectDispatchers(federation, mountPath, handle, collections, publ
|
||||
if (!post) return null;
|
||||
if (post?.properties?.["post-status"] === "draft") return null;
|
||||
if (post?.properties?.visibility === "unlisted") return null;
|
||||
// Soft-deleted posts should not be dereferenceable
|
||||
if (post.properties?.deleted) return null;
|
||||
const actorUrl = ctx.getActorUri(handle).href;
|
||||
const activity = jf2ToAS2Activity(post.properties, actorUrl, publicationUrl);
|
||||
// Only Create activities wrap Note/Article objects
|
||||
|
||||
1102
lib/inbox-handlers.js
Normal file
1102
lib/inbox-handlers.js
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
99
lib/inbox-queue.js
Normal file
99
lib/inbox-queue.js
Normal file
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* MongoDB-backed inbox processing queue.
|
||||
* Runs a setInterval-based processor that dequeues and processes
|
||||
* one activity at a time from ap_inbox_queue.
|
||||
* @module inbox-queue
|
||||
*/
|
||||
|
||||
import { routeToHandler } from "./inbox-handlers.js";
|
||||
|
||||
/**
|
||||
* Process the next pending item from the inbox queue.
|
||||
* Uses findOneAndUpdate for atomic claim (prevents double-processing).
|
||||
*
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {object} ctx - Fedify context
|
||||
* @param {string} handle - Our actor handle
|
||||
*/
|
||||
async function processNextItem(collections, ctx, handle) {
|
||||
const { ap_inbox_queue } = collections;
|
||||
if (!ap_inbox_queue) return;
|
||||
|
||||
const item = await ap_inbox_queue.findOneAndUpdate(
|
||||
{ status: "pending" },
|
||||
{ $set: { status: "processing" } },
|
||||
{ sort: { receivedAt: 1 }, returnDocument: "after" },
|
||||
);
|
||||
if (!item) return;
|
||||
|
||||
try {
|
||||
await routeToHandler(item, collections, ctx, handle);
|
||||
await ap_inbox_queue.updateOne(
|
||||
{ _id: item._id },
|
||||
{ $set: { status: "completed", processedAt: new Date().toISOString() } },
|
||||
);
|
||||
} catch (error) {
|
||||
const attempts = (item.attempts || 0) + 1;
|
||||
await ap_inbox_queue.updateOne(
|
||||
{ _id: item._id },
|
||||
{
|
||||
$set: {
|
||||
status: attempts >= (item.maxAttempts || 3) ? "failed" : "pending",
|
||||
attempts,
|
||||
error: error.message,
|
||||
},
|
||||
},
|
||||
);
|
||||
console.error(`[inbox-queue] Failed processing ${item.activityType} from ${item.actorUrl}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Enqueue an activity for async processing.
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {object} params
|
||||
* @param {string} params.activityType - Activity type name
|
||||
* @param {string} params.actorUrl - Actor URL
|
||||
* @param {string} [params.objectUrl] - Object URL
|
||||
* @param {object} params.rawJson - Full activity JSON-LD
|
||||
*/
|
||||
export async function enqueueActivity(collections, { activityType, actorUrl, objectUrl, rawJson }) {
|
||||
const { ap_inbox_queue } = collections;
|
||||
if (!ap_inbox_queue) return;
|
||||
|
||||
await ap_inbox_queue.insertOne({
|
||||
activityType,
|
||||
actorUrl: actorUrl || "",
|
||||
objectUrl: objectUrl || "",
|
||||
rawJson,
|
||||
status: "pending",
|
||||
attempts: 0,
|
||||
maxAttempts: 3,
|
||||
receivedAt: new Date().toISOString(),
|
||||
processedAt: null,
|
||||
error: null,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Start the background inbox processor.
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {Function} getCtx - Function returning a Fedify context
|
||||
* @param {string} handle - Our actor handle
|
||||
* @returns {NodeJS.Timeout} Interval ID (for cleanup)
|
||||
*/
|
||||
export function startInboxProcessor(collections, getCtx, handle) {
|
||||
const intervalId = setInterval(async () => {
|
||||
try {
|
||||
const ctx = getCtx();
|
||||
if (ctx) {
|
||||
await processNextItem(collections, ctx, handle);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("[inbox-queue] Processor error:", error.message);
|
||||
}
|
||||
}, 3_000); // Every 3 seconds
|
||||
|
||||
console.info("[ActivityPub] Inbox queue processor started (3s interval)");
|
||||
return intervalId;
|
||||
}
|
||||
@@ -189,6 +189,12 @@ export function jf2ToActivityStreams(properties, actorUrl, publicationUrl, optio
|
||||
object.sensitive = true;
|
||||
}
|
||||
|
||||
// Content warning text for Mastodon CW display
|
||||
if (properties["content-warning"]) {
|
||||
object.summary = properties["content-warning"];
|
||||
object.sensitive = true;
|
||||
}
|
||||
|
||||
if (properties["in-reply-to"]) {
|
||||
object.inReplyTo = properties["in-reply-to"];
|
||||
}
|
||||
@@ -360,9 +366,9 @@ export function jf2ToAS2Activity(properties, actorUrl, publicationUrl, options =
|
||||
if (properties["post-status"] === "sensitive") {
|
||||
noteOptions.sensitive = true;
|
||||
}
|
||||
// Summary doubles as CW text in Mastodon (notes only — articles already use summary for description)
|
||||
if (properties.summary && !isArticle) {
|
||||
noteOptions.summary = properties.summary;
|
||||
// Content warning text for Mastodon CW display
|
||||
if (properties["content-warning"]) {
|
||||
noteOptions.summary = properties["content-warning"];
|
||||
noteOptions.sensitive = true;
|
||||
}
|
||||
|
||||
|
||||
138
lib/key-refresh.js
Normal file
138
lib/key-refresh.js
Normal file
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* Proactive key refresh for remote actors.
|
||||
* Periodically re-fetches actor documents for active followers
|
||||
* whose keys may have rotated, keeping Fedify's KV cache fresh.
|
||||
* @module key-refresh
|
||||
*/
|
||||
|
||||
import { lookupWithSecurity } from "./lookup-helpers.js";
|
||||
|
||||
/**
|
||||
* Update key freshness tracking after successfully processing
|
||||
* an activity from a remote actor.
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {string} actorUrl - Remote actor URL
|
||||
*/
|
||||
export async function touchKeyFreshness(collections, actorUrl) {
|
||||
if (!actorUrl || !collections.ap_key_freshness) return;
|
||||
try {
|
||||
await collections.ap_key_freshness.updateOne(
|
||||
{ actorUrl },
|
||||
{
|
||||
$set: { lastSeenAt: new Date().toISOString() },
|
||||
$setOnInsert: { lastRefreshedAt: new Date().toISOString() },
|
||||
},
|
||||
{ upsert: true },
|
||||
);
|
||||
} catch {
|
||||
// Non-critical
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Refresh stale keys for active followers.
|
||||
* Finds followers whose keys haven't been refreshed in 7+ days
|
||||
* and re-fetches their actor documents (up to 10 per cycle).
|
||||
*
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {object} ctx - Fedify context (for lookupObject)
|
||||
* @param {string} handle - Our actor handle
|
||||
*/
|
||||
export async function refreshStaleKeys(collections, ctx, handle) {
|
||||
if (!collections.ap_key_freshness || !collections.ap_followers) return;
|
||||
|
||||
const sevenDaysAgo = new Date(Date.now() - 7 * 86_400_000).toISOString();
|
||||
|
||||
// Find actors with stale keys who are still our followers
|
||||
const staleActors = await collections.ap_key_freshness
|
||||
.aggregate([
|
||||
{
|
||||
$match: {
|
||||
lastRefreshedAt: { $lt: sevenDaysAgo },
|
||||
},
|
||||
},
|
||||
{
|
||||
$lookup: {
|
||||
from: "ap_followers",
|
||||
localField: "actorUrl",
|
||||
foreignField: "actorUrl",
|
||||
as: "follower",
|
||||
},
|
||||
},
|
||||
{ $match: { "follower.0": { $exists: true } } },
|
||||
{ $limit: 10 },
|
||||
])
|
||||
.toArray();
|
||||
|
||||
if (staleActors.length === 0) return;
|
||||
|
||||
console.info(`[ActivityPub] Refreshing keys for ${staleActors.length} stale actors`);
|
||||
|
||||
const documentLoader = await ctx.getDocumentLoader({ identifier: handle });
|
||||
|
||||
for (const entry of staleActors) {
|
||||
try {
|
||||
const result = await lookupWithSecurity(ctx, new URL(entry.actorUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
|
||||
await collections.ap_key_freshness.updateOne(
|
||||
{ actorUrl: entry.actorUrl },
|
||||
{ $set: { lastRefreshedAt: new Date().toISOString() } },
|
||||
);
|
||||
|
||||
if (!result) {
|
||||
// Actor gone — log as stale
|
||||
await collections.ap_activities?.insertOne({
|
||||
direction: "system",
|
||||
type: "StaleActor",
|
||||
actorUrl: entry.actorUrl,
|
||||
summary: `Actor ${entry.actorUrl} could not be resolved during key refresh`,
|
||||
receivedAt: new Date().toISOString(),
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
const status = error?.cause?.status || error?.message || "unknown";
|
||||
if (status === 410 || String(status).includes("410")) {
|
||||
// 410 Gone — actor deleted
|
||||
await collections.ap_activities?.insertOne({
|
||||
direction: "system",
|
||||
type: "StaleActor",
|
||||
actorUrl: entry.actorUrl,
|
||||
summary: `Actor ${entry.actorUrl} returned 410 Gone during key refresh`,
|
||||
receivedAt: new Date().toISOString(),
|
||||
});
|
||||
}
|
||||
// Update lastRefreshedAt even on failure to avoid retrying every cycle
|
||||
await collections.ap_key_freshness.updateOne(
|
||||
{ actorUrl: entry.actorUrl },
|
||||
{ $set: { lastRefreshedAt: new Date().toISOString() } },
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Schedule key refresh job (runs on startup + every 24h).
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {Function} getCtx - Function returning a Fedify context
|
||||
* @param {string} handle - Our actor handle
|
||||
*/
|
||||
export function scheduleKeyRefresh(collections, getCtx, handle) {
|
||||
const run = async () => {
|
||||
try {
|
||||
const ctx = getCtx();
|
||||
if (ctx) {
|
||||
await refreshStaleKeys(collections, ctx, handle);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("[ActivityPub] Key refresh failed:", error.message);
|
||||
}
|
||||
};
|
||||
|
||||
// Run once on startup (delayed to let federation initialize)
|
||||
setTimeout(run, 30_000);
|
||||
|
||||
// Then every 24 hours
|
||||
setInterval(run, 86_400_000);
|
||||
}
|
||||
27
lib/lookup-helpers.js
Normal file
27
lib/lookup-helpers.js
Normal file
@@ -0,0 +1,27 @@
|
||||
/**
|
||||
* Centralized wrapper for ctx.lookupObject() with FEP-fe34 origin-based
|
||||
* security. All lookupObject calls MUST go through this helper so the
|
||||
* crossOrigin policy is applied consistently.
|
||||
*
|
||||
* @module lookup-helpers
|
||||
*/
|
||||
|
||||
/**
|
||||
* Look up a remote ActivityPub object with cross-origin security.
|
||||
*
|
||||
* FEP-fe34 prevents spoofed attribution attacks by verifying that a
|
||||
* fetched object's `id` matches the origin of the URL used to fetch it.
|
||||
* Using `crossOrigin: "ignore"` tells Fedify to silently discard objects
|
||||
* whose id doesn't match the fetch origin, rather than throwing.
|
||||
*
|
||||
* @param {object} ctx - Fedify Context
|
||||
* @param {string|URL} input - URL or handle to look up
|
||||
* @param {object} [options] - Additional options passed to lookupObject
|
||||
* @returns {Promise<object|null>} Resolved object or null
|
||||
*/
|
||||
export function lookupWithSecurity(ctx, input, options = {}) {
|
||||
return ctx.lookupObject(input, {
|
||||
crossOrigin: "ignore",
|
||||
...options,
|
||||
});
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
|
||||
import { unfurl } from "unfurl.js";
|
||||
import { extractObjectData } from "./timeline-store.js";
|
||||
import { lookupWithSecurity } from "./lookup-helpers.js";
|
||||
|
||||
const USER_AGENT =
|
||||
"Mozilla/5.0 (compatible; Indiekit/1.0; +https://getindiekit.com)";
|
||||
@@ -262,7 +263,7 @@ export async function fetchAndStorePreviews(collections, uid, html) {
|
||||
*/
|
||||
export async function fetchAndStoreQuote(collections, uid, quoteUrl, ctx, documentLoader) {
|
||||
try {
|
||||
const object = await ctx.lookupObject(new URL(quoteUrl), { documentLoader });
|
||||
const object = await lookupWithSecurity(ctx,new URL(quoteUrl), { documentLoader });
|
||||
if (!object) return;
|
||||
|
||||
const quoteData = await extractObjectData(object, { documentLoader });
|
||||
@@ -270,7 +271,7 @@ export async function fetchAndStoreQuote(collections, uid, quoteUrl, ctx, docume
|
||||
// If author photo is empty, try fetching the actor directly
|
||||
if (!quoteData.author.photo && quoteData.author.url) {
|
||||
try {
|
||||
const actor = await ctx.lookupObject(new URL(quoteData.author.url), { documentLoader });
|
||||
const actor = await lookupWithSecurity(ctx,new URL(quoteData.author.url), { documentLoader });
|
||||
if (actor) {
|
||||
const { extractActorInfo } = await import("./timeline-store.js");
|
||||
const actorInfo = await extractActorInfo(actor, { documentLoader });
|
||||
|
||||
139
lib/outbox-failure.js
Normal file
139
lib/outbox-failure.js
Normal file
@@ -0,0 +1,139 @@
|
||||
/**
|
||||
* Outbox permanent failure handling.
|
||||
* Cleans up dead followers when delivery permanently fails.
|
||||
*
|
||||
* - 410 Gone: Immediate full cleanup (actor is permanently gone)
|
||||
* - 404: Strike system — 3 failures over 7+ days triggers full cleanup
|
||||
*
|
||||
* @module outbox-failure
|
||||
*/
|
||||
|
||||
import { logActivity } from "./activity-log.js";
|
||||
|
||||
const STRIKE_THRESHOLD = 3;
|
||||
const STRIKE_WINDOW_MS = 7 * 24 * 60 * 60 * 1000; // 7 days
|
||||
|
||||
/**
|
||||
* Clean up all data associated with an actor.
|
||||
* Removes follower record, their timeline items, and their notifications.
|
||||
*
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {string} actorUrl - Actor URL to clean up
|
||||
* @param {string} reason - Reason for cleanup (for logging)
|
||||
*/
|
||||
async function cleanupActor(collections, actorUrl, reason) {
|
||||
const { ap_followers, ap_timeline, ap_notifications } = collections;
|
||||
|
||||
// Remove from followers
|
||||
const deleted = await ap_followers.deleteOne({ actorUrl });
|
||||
|
||||
// Remove their timeline items
|
||||
if (ap_timeline) {
|
||||
await ap_timeline.deleteMany({ "author.url": actorUrl });
|
||||
}
|
||||
|
||||
// Remove their notifications
|
||||
if (ap_notifications) {
|
||||
await ap_notifications.deleteMany({ actorUrl });
|
||||
}
|
||||
|
||||
if (deleted.deletedCount > 0) {
|
||||
console.info(`[outbox-failure] Cleaned up actor ${actorUrl}: ${reason}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle permanent outbox delivery failure.
|
||||
* Called by Fedify's setOutboxPermanentFailureHandler.
|
||||
*
|
||||
* @param {number} statusCode - HTTP status code (404, 410, etc.)
|
||||
* @param {readonly URL[]} actorIds - Array of actor ID URLs
|
||||
* @param {URL} inbox - The inbox URL that failed
|
||||
* @param {object} collections - MongoDB collections
|
||||
*/
|
||||
export async function onOutboxPermanentFailure(statusCode, actorIds, inbox, collections) {
|
||||
const inboxUrl = inbox?.href || String(inbox);
|
||||
|
||||
for (const actorId of actorIds) {
|
||||
const actorUrl = actorId?.href || String(actorId);
|
||||
|
||||
if (statusCode === 410) {
|
||||
// 410 Gone — immediate full cleanup
|
||||
await cleanupActor(collections, actorUrl, `410 Gone from ${inboxUrl}`);
|
||||
|
||||
await logActivity(collections.ap_activities, {
|
||||
direction: "outbound",
|
||||
type: "DeliveryFailed:410",
|
||||
actorUrl,
|
||||
objectUrl: inboxUrl,
|
||||
summary: `Permanent delivery failure (410 Gone) to ${inboxUrl} — actor cleaned up`,
|
||||
}, {});
|
||||
} else {
|
||||
// 404 or other — strike system
|
||||
const now = new Date();
|
||||
const result = await collections.ap_followers.findOneAndUpdate(
|
||||
{ actorUrl },
|
||||
{
|
||||
$inc: { deliveryFailures: 1 },
|
||||
$setOnInsert: { firstFailureAt: now.toISOString() },
|
||||
$set: { lastFailureAt: now.toISOString() },
|
||||
},
|
||||
{ returnDocument: "after" },
|
||||
);
|
||||
|
||||
if (!result) {
|
||||
// Not a follower — nothing to track or clean up
|
||||
continue;
|
||||
}
|
||||
|
||||
const failures = result.deliveryFailures || 1;
|
||||
const firstFailure = result.firstFailureAt
|
||||
? new Date(result.firstFailureAt)
|
||||
: now;
|
||||
const windowElapsed = now.getTime() - firstFailure.getTime() >= STRIKE_WINDOW_MS;
|
||||
|
||||
if (failures >= STRIKE_THRESHOLD && windowElapsed) {
|
||||
// Confirmed dead — full cleanup
|
||||
await cleanupActor(
|
||||
collections,
|
||||
actorUrl,
|
||||
`${failures} failures over ${Math.round((now.getTime() - firstFailure.getTime()) / 86400000)}d (HTTP ${statusCode})`,
|
||||
);
|
||||
|
||||
await logActivity(collections.ap_activities, {
|
||||
direction: "outbound",
|
||||
type: `DeliveryFailed:${statusCode}:cleanup`,
|
||||
actorUrl,
|
||||
objectUrl: inboxUrl,
|
||||
summary: `${failures} delivery failures over 7+ days — actor cleaned up`,
|
||||
}, {});
|
||||
} else {
|
||||
// Strike recorded, not yet confirmed dead
|
||||
await logActivity(collections.ap_activities, {
|
||||
direction: "outbound",
|
||||
type: `DeliveryFailed:${statusCode}:strike`,
|
||||
actorUrl,
|
||||
objectUrl: inboxUrl,
|
||||
summary: `Delivery strike ${failures}/${STRIKE_THRESHOLD} for ${actorUrl} (HTTP ${statusCode})`,
|
||||
}, {});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Reset delivery failure strikes for an actor.
|
||||
* Called when we receive an inbound activity from an actor,
|
||||
* proving they are alive despite previous delivery failures.
|
||||
*
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {string} actorUrl - Actor URL
|
||||
*/
|
||||
export async function resetDeliveryStrikes(collections, actorUrl) {
|
||||
if (!actorUrl) return;
|
||||
// Only update if the fields exist — avoid unnecessary writes
|
||||
await collections.ap_followers.updateOne(
|
||||
{ actorUrl, deliveryFailures: { $exists: true } },
|
||||
{ $unset: { deliveryFailures: "", firstFailureAt: "", lastFailureAt: "" } },
|
||||
);
|
||||
}
|
||||
@@ -96,3 +96,19 @@ export async function cacheExists(key) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Cache-aside wrapper for query functions.
|
||||
* Returns cached result if available, otherwise runs queryFn and caches result.
|
||||
* @param {string} key - Cache key (without prefix — cacheGet/cacheSet add it)
|
||||
* @param {number} ttlSeconds - TTL in seconds
|
||||
* @param {Function} queryFn - Async function to run on cache miss
|
||||
* @returns {Promise<unknown>}
|
||||
*/
|
||||
export async function cachedQuery(key, ttlSeconds, queryFn) {
|
||||
const cached = await cacheGet(key);
|
||||
if (cached !== null) return cached;
|
||||
const result = await queryFn();
|
||||
await cacheSet(key, result, ttlSeconds);
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -10,6 +10,8 @@
|
||||
* 3. Extract author URL from post URL pattern → lookupObject
|
||||
*/
|
||||
|
||||
import { lookupWithSecurity } from "./lookup-helpers.js";
|
||||
|
||||
/**
|
||||
* Extract a probable author URL from a post URL using common fediverse patterns.
|
||||
*
|
||||
@@ -51,47 +53,6 @@ export function extractAuthorUrl(postUrl) {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Wraps a Fedify document loader to allow private/loopback addresses for
|
||||
* requests targeting the publication's own hostname.
|
||||
*
|
||||
* Fedify blocks requests to private IP ranges by default. When the publication
|
||||
* is self-hosted (e.g. localhost or a private IP), author lookups for posts on
|
||||
* that same host fail with a private-address error. This wrapper opts in to
|
||||
* allowPrivateAddress only when the target URL is on the publication's own host.
|
||||
*
|
||||
* @param {Function} documentLoader - Fedify authenticated document loader
|
||||
* @param {string} publicationUrl - The publication's canonical URL (e.g. ctx.url.href)
|
||||
* @returns {Function} Wrapped document loader
|
||||
*/
|
||||
function createPublicationAwareDocumentLoader(documentLoader, publicationUrl) {
|
||||
if (typeof documentLoader !== "function") {
|
||||
return documentLoader;
|
||||
}
|
||||
|
||||
let publicationHost = "";
|
||||
try {
|
||||
publicationHost = new URL(publicationUrl).hostname;
|
||||
} catch {
|
||||
return documentLoader;
|
||||
}
|
||||
|
||||
return (url, options = {}) => {
|
||||
try {
|
||||
const parsed = new URL(
|
||||
typeof url === "string" ? url : (url?.href || String(url)),
|
||||
);
|
||||
if (parsed.hostname === publicationHost) {
|
||||
return documentLoader(url, { ...options, allowPrivateAddress: true });
|
||||
}
|
||||
} catch {
|
||||
// Fall through to default loader behavior.
|
||||
}
|
||||
|
||||
return documentLoader(url, options);
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the author Actor for a given post URL.
|
||||
*
|
||||
@@ -107,20 +68,13 @@ export async function resolveAuthor(
|
||||
documentLoader,
|
||||
collections,
|
||||
) {
|
||||
const publicationLoader = createPublicationAwareDocumentLoader(
|
||||
documentLoader,
|
||||
ctx?.url?.href || "",
|
||||
);
|
||||
|
||||
// Strategy 1: Look up remote post via Fedify (signed request)
|
||||
try {
|
||||
const remoteObject = await ctx.lookupObject(new URL(postUrl), {
|
||||
documentLoader: publicationLoader,
|
||||
const remoteObject = await lookupWithSecurity(ctx,new URL(postUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
if (remoteObject && typeof remoteObject.getAttributedTo === "function") {
|
||||
const author = await remoteObject.getAttributedTo({
|
||||
documentLoader: publicationLoader,
|
||||
});
|
||||
const author = await remoteObject.getAttributedTo({ documentLoader });
|
||||
const recipient = Array.isArray(author) ? author[0] : author;
|
||||
if (recipient) {
|
||||
console.info(
|
||||
@@ -160,8 +114,8 @@ export async function resolveAuthor(
|
||||
|
||||
if (authorUrl) {
|
||||
try {
|
||||
const actor = await ctx.lookupObject(new URL(authorUrl), {
|
||||
documentLoader: publicationLoader,
|
||||
const actor = await lookupWithSecurity(ctx,new URL(authorUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
if (actor) {
|
||||
console.info(
|
||||
@@ -182,8 +136,8 @@ export async function resolveAuthor(
|
||||
const extractedUrl = extractAuthorUrl(postUrl);
|
||||
if (extractedUrl) {
|
||||
try {
|
||||
const actor = await ctx.lookupObject(new URL(extractedUrl), {
|
||||
documentLoader: publicationLoader,
|
||||
const actor = await lookupWithSecurity(ctx,new URL(extractedUrl), {
|
||||
documentLoader,
|
||||
});
|
||||
if (actor) {
|
||||
console.info(
|
||||
|
||||
@@ -65,8 +65,11 @@ export async function getNotifications(collections, options = {}) {
|
||||
// Type filter
|
||||
if (options.type) {
|
||||
// "reply" tab shows replies only; mentions have their own "mention" tab
|
||||
// "follow" tab shows both follows and follow_requests
|
||||
if (options.type === "reply") {
|
||||
query.type = "reply";
|
||||
} else if (options.type === "follow") {
|
||||
query.type = { $in: ["follow", "follow_request"] };
|
||||
} else {
|
||||
query.type = options.type;
|
||||
}
|
||||
@@ -133,6 +136,8 @@ export async function getNotificationCountsByType(collections, unreadOnly = fals
|
||||
counts.reply += count;
|
||||
} else if (_id === "mention") {
|
||||
counts.mention += count;
|
||||
} else if (_id === "follow_request") {
|
||||
counts.follow += count;
|
||||
} else if (counts[_id] !== undefined) {
|
||||
counts[_id] = count;
|
||||
}
|
||||
|
||||
121
lib/storage/server-blocks.js
Normal file
121
lib/storage/server-blocks.js
Normal file
@@ -0,0 +1,121 @@
|
||||
/**
|
||||
* Server-level blocking storage operations.
|
||||
* Blocks entire instances by hostname, checked in inbox listeners
|
||||
* before any expensive work is done.
|
||||
* @module storage/server-blocks
|
||||
*/
|
||||
|
||||
import { getRedisClient } from "../redis-cache.js";
|
||||
|
||||
const REDIS_KEY = "indiekit:blocked_servers";
|
||||
|
||||
/**
|
||||
* Add a server block by hostname.
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {string} hostname - Hostname to block (lowercase, no protocol)
|
||||
* @param {string} [reason] - Optional admin note
|
||||
*/
|
||||
export async function addBlockedServer(collections, hostname, reason) {
|
||||
const { ap_blocked_servers } = collections;
|
||||
const normalized = hostname.toLowerCase().trim();
|
||||
|
||||
await ap_blocked_servers.updateOne(
|
||||
{ hostname: normalized },
|
||||
{
|
||||
$setOnInsert: {
|
||||
hostname: normalized,
|
||||
blockedAt: new Date().toISOString(),
|
||||
...(reason ? { reason } : {}),
|
||||
},
|
||||
},
|
||||
{ upsert: true },
|
||||
);
|
||||
|
||||
// Incremental Redis update
|
||||
const redis = getRedisClient();
|
||||
if (redis) {
|
||||
try {
|
||||
await redis.sadd(REDIS_KEY, normalized);
|
||||
} catch {
|
||||
// Non-critical
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a server block by hostname.
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @param {string} hostname - Hostname to unblock
|
||||
*/
|
||||
export async function removeBlockedServer(collections, hostname) {
|
||||
const { ap_blocked_servers } = collections;
|
||||
const normalized = hostname.toLowerCase().trim();
|
||||
|
||||
await ap_blocked_servers.deleteOne({ hostname: normalized });
|
||||
|
||||
const redis = getRedisClient();
|
||||
if (redis) {
|
||||
try {
|
||||
await redis.srem(REDIS_KEY, normalized);
|
||||
} catch {
|
||||
// Non-critical
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all blocked servers.
|
||||
* @param {object} collections - MongoDB collections
|
||||
* @returns {Promise<object[]>} Array of block entries
|
||||
*/
|
||||
export async function getAllBlockedServers(collections) {
|
||||
const { ap_blocked_servers } = collections;
|
||||
return await ap_blocked_servers.find({}).sort({ blockedAt: -1 }).toArray();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a server is blocked by actor URL.
|
||||
* Uses Redis Set (O(1)) with MongoDB fallback.
|
||||
* @param {string} actorUrl - Full actor URL
|
||||
* @param {object} collections - MongoDB collections (fallback only)
|
||||
* @returns {Promise<boolean>}
|
||||
*/
|
||||
export async function isServerBlocked(actorUrl, collections) {
|
||||
if (!actorUrl) return false;
|
||||
try {
|
||||
const hostname = new URL(actorUrl).hostname.toLowerCase();
|
||||
const redis = getRedisClient();
|
||||
if (redis) {
|
||||
return (await redis.sismember(REDIS_KEY, hostname)) === 1;
|
||||
}
|
||||
// Fallback: direct MongoDB check
|
||||
const { ap_blocked_servers } = collections;
|
||||
return !!(await ap_blocked_servers.findOne({ hostname }));
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load all blocked hostnames into Redis Set on startup.
|
||||
* Replaces existing set contents entirely.
|
||||
* @param {object} collections - MongoDB collections
|
||||
*/
|
||||
export async function loadBlockedServersToRedis(collections) {
|
||||
const redis = getRedisClient();
|
||||
if (!redis) return;
|
||||
|
||||
try {
|
||||
const { ap_blocked_servers } = collections;
|
||||
const docs = await ap_blocked_servers.find({}).toArray();
|
||||
const hostnames = docs.map((d) => d.hostname);
|
||||
|
||||
// Replace: delete existing set, then add all
|
||||
await redis.del(REDIS_KEY);
|
||||
if (hostnames.length > 0) {
|
||||
await redis.sadd(REDIS_KEY, ...hostnames);
|
||||
}
|
||||
} catch {
|
||||
// Non-critical — isServerBlocked falls back to MongoDB
|
||||
}
|
||||
}
|
||||
@@ -73,6 +73,18 @@ export async function getTimelineItems(collections, options = {}) {
|
||||
|
||||
const query = {};
|
||||
|
||||
// Exclude context-only items (ancestors fetched for thread reconstruction)
|
||||
// unless explicitly requested via options.includeContext
|
||||
if (!options.includeContext) {
|
||||
query.isContext = { $ne: true };
|
||||
}
|
||||
|
||||
// Exclude private/direct posts from the main timeline feed —
|
||||
// these belong in messages/notifications, not the public reader
|
||||
if (!options.includePrivate) {
|
||||
query.visibility = { $nin: ["private", "direct"] };
|
||||
}
|
||||
|
||||
// Type filter
|
||||
if (options.type) {
|
||||
query.type = options.type;
|
||||
@@ -252,7 +264,11 @@ export async function countNewItems(collections, after, options = {}) {
|
||||
const { ap_timeline } = collections;
|
||||
if (!after || Number.isNaN(new Date(after).getTime())) return 0;
|
||||
|
||||
const query = { published: { $gt: after } };
|
||||
const query = {
|
||||
published: { $gt: after },
|
||||
isContext: { $ne: true },
|
||||
visibility: { $nin: ["private", "direct"] },
|
||||
};
|
||||
if (options.type) query.type = options.type;
|
||||
if (options.excludeReplies) {
|
||||
query.$or = [
|
||||
@@ -289,5 +305,9 @@ export async function markItemsRead(collections, uids) {
|
||||
*/
|
||||
export async function countUnreadItems(collections) {
|
||||
const { ap_timeline } = collections;
|
||||
return await ap_timeline.countDocuments({ read: { $ne: true } });
|
||||
return await ap_timeline.countDocuments({
|
||||
read: { $ne: true },
|
||||
isContext: { $ne: true },
|
||||
visibility: { $nin: ["private", "direct"] },
|
||||
});
|
||||
}
|
||||
|
||||
@@ -33,6 +33,29 @@ export function sanitizeContent(html) {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace custom emoji :shortcode: placeholders with inline <img> tags.
|
||||
* Applied AFTER sanitization — the <img> tags are controlled output from
|
||||
* trusted emoji data, not user-supplied HTML.
|
||||
*
|
||||
* @param {string} html - Content HTML (already sanitized)
|
||||
* @param {Array<{shortcode: string, url: string}>} emojis - Custom emoji data
|
||||
* @returns {string} HTML with shortcodes replaced by <img> tags
|
||||
*/
|
||||
export function replaceCustomEmoji(html, emojis) {
|
||||
if (!emojis?.length || !html) return html;
|
||||
let result = html;
|
||||
for (const { shortcode, url } of emojis) {
|
||||
const escaped = shortcode.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
|
||||
const pattern = new RegExp(`:${escaped}:`, "g");
|
||||
result = result.replace(
|
||||
pattern,
|
||||
`<img class="ap-custom-emoji" src="${url}" alt=":${shortcode}:" title=":${shortcode}:" draggable="false">`,
|
||||
);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract actor information from Fedify Person/Application/Service object
|
||||
* @param {object} actor - Fedify actor object
|
||||
@@ -104,7 +127,10 @@ export async function extractActorInfo(actor, options = {}) {
|
||||
// Bot detection — Service and Application actors are automated accounts
|
||||
const bot = actor instanceof Service || actor instanceof Application;
|
||||
|
||||
return { name, url, photo, handle, emojis, bot };
|
||||
// Replace custom emoji shortcodes in display name with <img> tags
|
||||
const nameHtml = replaceCustomEmoji(name, emojis);
|
||||
|
||||
return { name, nameHtml, url, photo, handle, emojis, bot };
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -336,6 +362,10 @@ export async function extractObjectData(object, options = {}) {
|
||||
if (shares?.totalItems != null) counts.boosts = shares.totalItems;
|
||||
} catch { /* ignore */ }
|
||||
|
||||
// Replace custom emoji :shortcode: in content with inline <img> tags.
|
||||
// Applied after sanitization — these are trusted emoji from the post's tags.
|
||||
content.html = replaceCustomEmoji(content.html, emojis);
|
||||
|
||||
// Build base timeline item
|
||||
const item = {
|
||||
uid,
|
||||
|
||||
@@ -10,6 +10,13 @@
|
||||
"noActivity": "No activity yet. Once your actor is federated, interactions will appear here.",
|
||||
"noFollowers": "No followers yet.",
|
||||
"noFollowing": "Not following anyone yet.",
|
||||
"pendingFollows": "Pending",
|
||||
"noPendingFollows": "No pending follow requests.",
|
||||
"approve": "Approve",
|
||||
"reject": "Reject",
|
||||
"followApproved": "Follow request approved.",
|
||||
"followRejected": "Follow request rejected.",
|
||||
"followRequest": "requested to follow you",
|
||||
"followerCount": "%d follower",
|
||||
"followerCount_plural": "%d followers",
|
||||
"followingCount": "%d following",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@rmdes/indiekit-endpoint-activitypub",
|
||||
"version": "2.12.2",
|
||||
"version": "2.15.4",
|
||||
"description": "ActivityPub federation endpoint for Indiekit via Fedify. Adds full fediverse support: actor, inbox, outbox, followers, following, syndication, and Mastodon migration.",
|
||||
"keywords": [
|
||||
"indiekit",
|
||||
|
||||
@@ -6,6 +6,53 @@
|
||||
{% from "pagination/macro.njk" import pagination with context %}
|
||||
|
||||
{% block content %}
|
||||
{# Tab navigation — only show if there are pending requests #}
|
||||
{% if pendingCount > 0 %}
|
||||
{% set followersBase = mountPath + "/admin/followers" %}
|
||||
<nav class="ap-tabs">
|
||||
<a href="{{ followersBase }}" class="ap-tab{% if tab == 'followers' %} ap-tab--active{% endif %}">
|
||||
{{ __("activitypub.followers") }}
|
||||
{% if followerCount %}<span class="ap-tab__count">{{ followerCount }}</span>{% endif %}
|
||||
</a>
|
||||
<a href="{{ followersBase }}?tab=pending" class="ap-tab{% if tab == 'pending' %} ap-tab--active{% endif %}">
|
||||
{{ __("activitypub.pendingFollows") }}
|
||||
<span class="ap-tab__count">{{ pendingCount }}</span>
|
||||
</a>
|
||||
</nav>
|
||||
{% endif %}
|
||||
|
||||
{% if tab == "pending" %}
|
||||
{# Pending follow requests #}
|
||||
{% if pendingFollows.length > 0 %}
|
||||
{% for pending in pendingFollows %}
|
||||
<div class="ap-follow-request">
|
||||
{{ card({
|
||||
title: pending.name or pending.handle or pending.actorUrl,
|
||||
url: pending.actorUrl,
|
||||
photo: { url: pending.avatar, alt: pending.name } if pending.avatar,
|
||||
description: { text: "@" + pending.handle if pending.handle }
|
||||
}) }}
|
||||
<div class="ap-follow-request__actions">
|
||||
<form method="post" action="{{ mountPath }}/admin/followers/approve" class="ap-follow-request__form">
|
||||
<input type="hidden" name="_csrf" value="{{ csrfToken }}">
|
||||
<input type="hidden" name="actorUrl" value="{{ pending.actorUrl }}">
|
||||
<button type="submit" class="button">{{ __("activitypub.approve") }}</button>
|
||||
</form>
|
||||
<form method="post" action="{{ mountPath }}/admin/followers/reject" class="ap-follow-request__form">
|
||||
<input type="hidden" name="_csrf" value="{{ csrfToken }}">
|
||||
<input type="hidden" name="actorUrl" value="{{ pending.actorUrl }}">
|
||||
<button type="submit" class="button button--danger">{{ __("activitypub.reject") }}</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
|
||||
{{ pagination(cursor) if cursor }}
|
||||
{% else %}
|
||||
{{ prose({ text: __("activitypub.noPendingFollows") }) }}
|
||||
{% endif %}
|
||||
{% else %}
|
||||
{# Accepted followers #}
|
||||
{% if followers.length > 0 %}
|
||||
{% for follower in followers %}
|
||||
{{ card({
|
||||
@@ -21,4 +68,5 @@
|
||||
{% else %}
|
||||
{{ prose({ text: __("activitypub.noFollowers") }) }}
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
|
||||
@@ -26,6 +26,38 @@
|
||||
</div>
|
||||
</section>
|
||||
|
||||
{# Blocked servers #}
|
||||
<section class="ap-moderation__section">
|
||||
<h2>Blocked Servers</h2>
|
||||
<p class="ap-moderation__hint">Block entire instances by hostname. Activities from blocked servers are rejected before any processing.</p>
|
||||
{% if blockedServers and blockedServers.length > 0 %}
|
||||
<ul class="ap-moderation__list" x-ref="serverList">
|
||||
{% for entry in blockedServers %}
|
||||
<li class="ap-moderation__entry" data-hostname="{{ entry.hostname }}">
|
||||
<code>{{ entry.hostname }}</code>
|
||||
{% if entry.reason %}<span class="ap-moderation__reason">({{ entry.reason }})</span>{% endif %}
|
||||
<button class="ap-moderation__remove"
|
||||
@click="removeEntry($el, 'unblock-server', { hostname: $el.closest('li').dataset.hostname })">
|
||||
Unblock
|
||||
</button>
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% else %}
|
||||
<p class="ap-moderation__empty" x-ref="serverEmpty">No servers blocked.</p>
|
||||
{% endif %}
|
||||
|
||||
<form class="ap-moderation__add-form" @submit.prevent="addBlockedServer()">
|
||||
<input type="text" x-model="newServerHostname"
|
||||
placeholder="spam.instance.social"
|
||||
class="ap-moderation__input"
|
||||
x-ref="serverInput">
|
||||
<button type="submit" :disabled="submitting" class="ap-moderation__add-btn">
|
||||
Block Server
|
||||
</button>
|
||||
</form>
|
||||
</section>
|
||||
|
||||
{# Blocked actors #}
|
||||
<section class="ap-moderation__section">
|
||||
<h2>{{ __("activitypub.moderation.blockedTitle") }}</h2>
|
||||
@@ -108,6 +140,7 @@
|
||||
document.addEventListener('alpine:init', () => {
|
||||
Alpine.data('moderationPage', () => ({
|
||||
newKeyword: '',
|
||||
newServerHostname: '',
|
||||
submitting: false,
|
||||
error: '',
|
||||
|
||||
@@ -157,6 +190,50 @@
|
||||
this.submitting = false;
|
||||
},
|
||||
|
||||
async addBlockedServer() {
|
||||
const hostname = this.newServerHostname.trim();
|
||||
if (!hostname) return;
|
||||
this.submitting = true;
|
||||
this.error = '';
|
||||
try {
|
||||
const res = await fetch(this.mountPath + '/admin/reader/block-server', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'X-CSRF-Token': this.csrfToken,
|
||||
},
|
||||
body: JSON.stringify({ hostname }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (data.success) {
|
||||
const list = this.$refs.serverList;
|
||||
if (list) {
|
||||
const li = document.createElement('li');
|
||||
li.className = 'ap-moderation__entry';
|
||||
li.dataset.hostname = hostname;
|
||||
const code = document.createElement('code');
|
||||
code.textContent = hostname;
|
||||
const btn = document.createElement('button');
|
||||
btn.className = 'ap-moderation__remove';
|
||||
btn.textContent = 'Unblock';
|
||||
btn.addEventListener('click', () => {
|
||||
this.removeEntry(btn, 'unblock-server', { hostname });
|
||||
});
|
||||
li.append(code, btn);
|
||||
list.appendChild(li);
|
||||
}
|
||||
if (this.$refs.serverEmpty) this.$refs.serverEmpty.remove();
|
||||
this.newServerHostname = '';
|
||||
this.$refs.serverInput.focus();
|
||||
} else {
|
||||
this.error = data.error || 'Failed to block server';
|
||||
}
|
||||
} catch (e) {
|
||||
this.error = 'Request failed';
|
||||
}
|
||||
this.submitting = false;
|
||||
},
|
||||
|
||||
async removeEntry(el, action, payload) {
|
||||
const li = el.closest('li');
|
||||
if (!li) return;
|
||||
|
||||
@@ -65,6 +65,9 @@
|
||||
</time>
|
||||
{% if item.updated %}<span class="ap-card__edited" title="{{ item.updated | date('PPp') }}">✏️</span>{% endif %}
|
||||
</a>
|
||||
{% if item.visibility and item.visibility != "public" %}
|
||||
<span class="ap-card__visibility ap-card__visibility--{{ item.visibility }}" title="{% if item.visibility == 'unlisted' %}{{ __('activitypub.reader.compose.visibilityUnlisted') }}{% elif item.visibility == 'private' %}{{ __('activitypub.reader.compose.visibilityFollowers') }}{% elif item.visibility == 'direct' %}DM{% endif %}">{% if item.visibility == "unlisted" %}🔓{% elif item.visibility == "private" %}🔒{% elif item.visibility == "direct" %}✉️{% endif %}</span>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</header>
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
{% endif %}
|
||||
<span class="ap-notification__avatar ap-notification__avatar--default" aria-hidden="true">{{ item.actorName[0] | upper if item.actorName else "?" }}</span>
|
||||
<span class="ap-notification__type-badge">
|
||||
{% if item.type == "like" %}❤{% elif item.type == "boost" %}🔁{% elif item.type == "follow" %}👤{% elif item.type == "reply" %}💬{% elif item.type == "mention" %}{% if item.isDirect %}🔒{% else %}@{% endif %}{% elif item.type == "dm" %}✉{% elif item.type == "report" %}⚑{% endif %}
|
||||
{% if item.type == "like" %}❤{% elif item.type == "boost" %}🔁{% elif item.type == "follow" or item.type == "follow_request" %}👤{% elif item.type == "reply" %}💬{% elif item.type == "mention" %}{% if item.isDirect %}🔒{% else %}@{% endif %}{% elif item.type == "dm" %}✉{% elif item.type == "report" %}⚑{% endif %}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
@@ -32,6 +32,8 @@
|
||||
{{ __("activitypub.notifications.boostedPost") }}
|
||||
{% elif item.type == "follow" %}
|
||||
{{ __("activitypub.notifications.followedYou") }}
|
||||
{% elif item.type == "follow_request" %}
|
||||
{{ __("activitypub.followRequest") }}
|
||||
{% elif item.type == "reply" %}
|
||||
{{ __("activitypub.notifications.repliedTo") }}
|
||||
{% elif item.type == "mention" %}
|
||||
|
||||
Reference in New Issue
Block a user