diff --git a/docs/research/PERMISSIONED_DATA.md b/docs/research/PERMISSIONED_DATA.md new file mode 100644 index 0000000..6686ee6 --- /dev/null +++ b/docs/research/PERMISSIONED_DATA.md @@ -0,0 +1,689 @@ +# ATProto Permissioned Data (Research Notes) + +> STATUS: EXTERNAL RESEARCH. Last reviewed 2026-08-03. +> +> **This describes work happening in the upstream `bluesky-social/atproto` +> project, not in ATCR.** Nothing here is implemented in this repository, and +> nothing here is a commitment that ATCR will adopt it. The upstream spec is +> itself explicitly labeled "a proposal, not the final specification" and its +> reference implementation is an open, WIP pull request. Details, terminology, +> and wire formats are all expected to change. +> +> This document exists so we can track the design as it stabilizes and reason +> about what, if anything, it means for ATCR. + +## Sources + +| Source | What it is | Date | +|---|---|---| +| [Proposal 0016 Permissioned Data](https://github.com/bluesky-social/proposals/blob/main/0016-permissioned-data/README.md) | The spec of record. Supersedes the diary posts wherever they disagree. | current | +| [atproto#5187 `permissioned-data`](https://github.com/bluesky-social/atproto/pull/5187) | Reference implementation. Marked "⚠️ WIP ⚠️". 167 files, +12,178/−732. | updated 2026-07-31 | +| [Discourse discussion thread](https://discourse.atprotocol.community/t/permissioned-data-proposal-discussion/946) | Community forum discussion of the proposal. | ongoing | + +Design log by Daniel Holmgren, oldest first. Useful for *why*, unreliable for +*what*: see [What changed since the diaries](#what-changed-since-the-diaries). + +| Post | Date | +|---|---| +| [Diary 1: To Encrypt or Not to Encrypt](https://dholms.leaflet.pub/3meluqcwky22a) | 2026-02-11 | +| [Diary 2: Buckets](https://dholms.leaflet.pub/3mfrsbcn2gk2a) | 2026-02-26 | +| [Diary 3: Your Bucket, My Data](https://dholms.leaflet.pub/3mguviy6iks2a) | 2026-03-12 | +| [Interlude: Spaces](https://dholms.leaflet.pub/3mhbuoc64xk2a) | 2026-03-17 | +| [Diary 4: The Big Picture](https://dholms.leaflet.pub/3mhj6bcqats2o) | 2026-03-20 | +| [Diary 5: What's in a Name?](https://dholms.leaflet.pub/3mlegohgtps2k) | 2026-05-08 | +| [Modeling communities on permissioned data](https://dholms.leaflet.pub/3mndhk7ihsc2g) | 2026-06-02 | +| [Diary 6: Boring Auth](https://dholms.leaflet.pub/3mnkrxp7rt22i) | 2026-06-05 | +| [Diary 7: Off the Record](https://dholms.leaflet.pub/3mqtqvjidqs2p) | 2026-07-17 | + +Referenced external specs: [LtHash](https://eprint.iacr.org/2019/227), +[CAR](https://dasl.ing/car.html), [DRISL](https://dasl.ing/drisl.html), +[TLS 1.3 §3.4](https://www.rfc-editor.org/rfc/rfc8446), +[atproto permission spec](https://atproto.com/specs/permission). + +## Summary + +Atproto today is a protocol for **public broadcast**: signed, redistributable, +universally addressable records that anyone can crawl. Permissioned data adds a +second data protocol for records with an **access perimeter**, targeting +personal data (bookmarks, mutes, drafts), gated content (paid newsletters), +socially shared content (private posts, stories), and groups (private forums, +group chats). + +It deliberately keeps the same abstract shape as public broadcast: DID-based +authority, per-user repositories, lexicon-typed records, applications crawling +PDSes to build views. What changes is the repository format, the sync +mechanism, the addressing scheme, and the resolution path. + +The single most important framing: this provides **access control, not +confidentiality**. It is explicitly not end-to-end encrypted. Servers (both +PDSes and authorized applications) read plaintext, which is required for +search, indexing, notifications, aggregation, and moderation. E2EE is left as +an app-layer concern. + +| | Public broadcast | Permissioned data | +|---|---|---| +| Unit of data | Record in a repo | Record in a permissioned repo | +| Repo scope | One repo per user | One permissioned repo per (user, space) | +| Record authority | User DID | User DID | +| URI authority | User DID | Space authority DID | +| Commit | Merkle Search Tree root | LtHash set-hash digest | +| Signature | Rebroadcastable, archival | Deniable on rebroadcast | +| Addressing | `at://` URI | `at://` URI with `space` segment | +| Access | Public | Gated by space credential | +| Distribution | Relay firehose | Direct pull from each repo host | + +## Core model + +### Spaces + +A **space** is an authorization and sync boundary representing a shared social +context. It is *not* a storage location. Each participant stores their own +records for that space in a **permissioned repo** on their own PDS. A space is +the aggregation of those per-user repos across the network: an application +pulls each member's repo from its host, assembles the view, and applies access +control to its own users. + +A space is identified by a triple: + +1. **space authority** — a DID, the root of authority +2. **space type** — an NSID naming the modality +3. **space key** (`skey`) — a string distinguishing spaces of the same type + under the same authority (max 512 bytes, `rkey` syntax) + +The authority may be a user's own DID (personal data: bookmarks, mutes) or a +dedicated DID, which lets a shared space transfer between owners without +breaking backlinks. + +Spaces scale from one user's bookmarks to communities of millions. + +### Terminology + +| Term | Meaning | +|---|---| +| Space | Authorization + sync boundary, identified by `(authority, type, skey)` | +| Permissioned repo | One user's records within one space, on their repo host | +| Repo host | Service storing and serving users' permissioned repos | +| Space host | Service answering for a space as a whole (credentials, writer set, notification routing) | +| Space authority | The DID at the root of a space | +| Space credential | Token from the authority granting read access to a space | +| Delegation token | Token from a user's PDS, exchanged for a space credential | +| Client attestation | Token signed by an app's own key, proving app identity | +| Syncer | Application keeping its own copy of a space in sync | + +A PDS fills both the repo-host and space-host roles, but they are specified +separately because neither has to be a PDS. + +### Addressing + +Permissioned data reuses `at://` with a fixed literal `space` marker where a +collection NSID would sit: + +``` +Space: at://{spaceDid}/space/{spaceType}/{skey} +Record: at://{spaceDid}/space/{spaceType}/{skey}/{authorDid}/{collection}/{rkey} +``` + +Never ambiguous with a public URI: a collection NSID always contains at least +two dots, `space` contains none. The ordering is `SPACE/author/collection/rkey` +rather than author-first because user authority derives from space membership, +and because a space reference is then a clean prefix of a record URI. + +The POC modifies `packages/syntax/src/aturi.ts` and +`packages/syntax/src/aturi_validation.ts` accordingly. + +### Space type declarations + +A space type NSID resolves to a new kind of Lexicon definition, +`"type": "space"`, which must be the `main` def: + +```json +{ + "lexicon": 1, + "id": "com.atmoboards.forum", + "defs": { + "main": { + "type": "space", + "description": "A discussion forum", + "key": "any", + "name": "AtmoBoards Forum", + "name:lang": { "es": "Foro AtmoBoards", "ja": "AtmoBoards 掲示板" }, + "collections": ["com.atmoboards.thread", "com.atmoboards.reply"] + } + } +} +``` + +| Field | Required | Purpose | +|---|---|---| +| `type: "space"` | yes | Marks the declaration | +| `key` | yes | Recommended skey type | +| `name` (1–64) | yes | **User-facing**, shown on OAuth consent screens | +| `name:lang` | no | Localized names | +| `collections` | yes | Default collection set for a `space:` scope of this type | +| `description` | no | Developer-facing only | + +`collections` is a default, not a constraint: any collection may be written to +any space at the protocol level. + +### DID document entries + +A space authority is resolved through two optional DID doc entries: + +- verification method `#atproto_space` — key verifying the space's credentials +- service `#atproto_space_host` — the space host endpoint + +When absent they fall back to `#atproto` and `#atproto_pds` respectively, so an +authority hosted on an ordinary PDS needs no extra DID doc entries at all. + +## Access control + +Reading a space requires a **space credential** issued by the space authority. +The authority decides based on two independent axes, and **the protocol does +not define the decision procedure** — that is the concern of a space-management +implementation layered above it (see [simplespace](#simplespace)). + +### Three token classes + +All three share a wire shape and are implemented as one parameterized signer in +`packages/space/src/credential.ts`. + +| Token | `typ` | Signed by | Lifetime | `aud` | Reuse | +|---|---|---|---|---|---| +| Delegation token | `atproto-space-delegation+jwt` | user (`kid: #atproto`) | 60s | space host | single-use | +| Client attestation | `atproto-client-attestation+jwt` | app's client-auth key | 60s | space host | single-use | +| Space credential | `atproto-space-credential+jwt` | space authority | 2h | **none** | multi-use | + +**Delegation token.** Minted by the user's PDS via +`com.atproto.space.getDelegationToken`. Asserts *only* the user-to-app +delegation, and says nothing about whether the user is a member of the space. +Structurally close to a service auth token but a distinct credential class: no +`lxm` claim, bound to a target space through `sub`. + +**Client attestation.** Structurally a `private_key_jwt` client assertion, the +same shape a confidential client already presents to its authorization server, +but addressed to the space authority. `iss` and `sub` are both the `client_id`. +The authority verifies it by resolving `client_id` → `client-metadata.json` → +JWKS → the key named by `kid`. Required only when a space gates on app +identity, so public clients still work against `#open` spaces. + +**Space credential.** Deliberately has **no `aud`**: one credential is +presented to every repo host serving a repo in the space, and any host verifies +it against the authority's published key without contacting the authority. + +### Credential flow + +``` +┌──────┐ ┌────────────┐ ┌─────────────┐ ┌─────────────────┐ +│ User │ │ User's PDS │ │ Application │ │ Space Authority │ +└───┬──┘ └──────┬─────┘ └──────┬──────┘ └────────┬────────┘ + │ │ │ │ + ├── OAuth consent ───► │ │ + │ ├───── OAuth token ─────► │ + │ │ │ │ + │ ◄─ getDelegationToken ──┤ │ + │ ├── delegation token ───► │ + │ │ │ getSpaceCredential │ + │ │ ├─(token [+ attestation])─►│ + │ │ ◄──── space credential ────┤ +``` + +The two tokens are presented together but signed by different parties and +evaluated independently. An application serving many users of a space needs +only **one** credential, obtainable via any single user's session; when it +loses every OAuth session for that space it can no longer renew and loses +access. + +## Repo format + +### Commit digest: LtHash + +No Merkle Search Tree. Permissioned repos sync as complete units bounded by +space membership, so the MST's partial-proof machinery buys nothing. + +The digest is [LtHash](https://eprint.iacr.org/2019/227), a homomorphic +set hash built on a lattice problem (hence quantum-secure): + +- State is a fixed **2048-byte** buffer read as **1024 little-endian u16 lanes** +- Each record maps to the element `{collection}/{rkey}/{record_cid}` +- **Add**: expand the element to 2048 bytes with BLAKE3 in XOF mode, read as + 1024 LE u16 lanes, add lane-wise mod 2^16 +- **Remove**: identical, but subtract +- Empty repo state is all zeroes +- Commit carries only `sha256(state)`; the host keeps the full 2048-byte state + +Both operations commute, so the state depends only on the current record set, +never on write order. Adding or removing a record is one cheap operation rather +than a recomputation, so hosts maintain the state incrementally on each write. +Two repos holding the same records always produce the same digest. + +Implementation: `packages/space/src/lthash.ts` (~80 lines, `@noble/hashes`, +two views over one `ArrayBuffer` so `Uint16Array` arithmetic wraps mod 2^16 for +free). + +### Deniable commit signatures + +A signature over the content digest would be a **rebroadcastable proof of what +a user wrote in a private space**. So the signature covers only random +per-reader bytes, and the digest is bound to those bytes by a *symmetric* MAC. + +Both are domain-separated by a shared context string using the TLS 1.3 §3.4 +variable-length vector encoding: + +``` +ctx = "atproto-space-v1" // fixed protocol tag + || uint16be(len(space)) || space // space URI + || uint16be(len(author)) || author // author DID of the repo + || uint16be(len(rev)) || rev // commit revision (TID) + || uint16be(len(ikm)) || ikm // per-signature nonce +``` + +Producing a commit: + +1. Generate `ikm`, 32 fresh random bytes. **A new `ikm` per reader served.** +2. `sig = sign(ctx)` with the user's signing key. The signed message contains + space, author, rev, and ikm, but **not** the repository hash. +3. `mac = HMAC-SHA256(HKDF-SHA256(ikm, ctx), hash)` + +A reader verifies `sig` (authenticity), then recomputes `mac` (integrity), and +from there trusts `hash`. But because the MAC key derives from the *public* +`ikm`, anyone holding a leaked commit can compute a valid MAC for any `hash` at +all. A rebroadcast commit therefore proves only that the user signed a +`(space, author, rev, ikm)` context, never what they wrote. A leak becomes a +social and reputational matter rather than a cryptographically provable one. + +`com.atproto.space.defs#signedCommit`: + +| Field | Type | Description | +|---|---|---| +| `ver` | integer | Commit format version, currently `1` | +| `hash` | bytes | `sha256` of the LtHash state (32 bytes) | +| `ikm` | bytes | Per-signature nonce (32 random bytes) | +| `sig` | bytes | `sign(ctx)` by the user's signing key | +| `mac` | bytes | `HMAC-SHA256(HKDF-SHA256(ikm, ctx), hash)` | +| `rev` | string | Commit revision (TID), also bound into `ctx` | + +Byte-order gotcha the spec calls out explicitly: `ctx` length prefixes are +**big-endian** (TLS convention), LtHash lanes are **little-endian** (LtHash +reference construction). Each keeps its native order. + +Implementation: `packages/space/src/repo-commit.ts`; `hmacSha256` and +`hkdfSha256` were added to `@atproto/crypto`. + +### CAR serialization + +A permissioned repo serializes to a CAR file with **two roots, in order**: + +1. the **signed commit** block +2. the **index** — a DRISL (DAG-CBOR) map from `"{collection}/{rkey}"` to the + record CID, keys in lexicographic order + +Record blocks follow, in the same order as their index entries. Blobs are +excluded and fetched separately via `getBlob`. + +This layout lets a consumer stream-verify in three stages with no buffering: + +1. Verify the commit's signature and MAC. `hash` is now trusted. +2. Fold each index entry's `{collection}/{rkey}/{cid}` into a running set hash + as it is read, compare against `hash`. **This authenticates the entire index + without reading a single record.** +3. Verify each record block against its own CID as it streams past. + +Implementation: `packages/space/src/sync/provider.ts` (serialize), +`packages/space/src/sync/consumer.ts` (streaming verify). + +## Sync + +**There is no relay.** Permissioned repos are non-rebroadcastable by nature, so +there is no collated firehose. Applications pull directly from each repo host +and are responsible for keeping their own copy in sync. + +### Incremental sync + +A syncer keeps its own copy of a repo plus its own running set hash over that +copy. Agreement with the host's digest means exactly up to date; disagreement +means behind or corrupted. + +`com.atproto.space.listRepoOps(since)` returns operations after a revision. +Each entry is `{ rev, collection, rkey, cid, prev }`: + +- `cid` null → delete +- `prev` null → create +- entries sharing a `rev` → one atomic multi-record write + +Record values are **inlined by default**, so a syncer advances in a single call +with no per-write `getRecord` round-trip. Only the current value for a path is +inlined; superseded intermediates are omitted. `excludeValues` gives +metadata-only entries. + +If a response includes the last available operation, it must also carry the +member's current signed commit. The syncer compares it against its running set +hash: match means fully in sync and authenticated; mismatch means diverged, +fall back to recovery. + +Sync is **self-healing** because correctness rests on the set-hash comparison +rather than on receiving every operation. A missed op is detected on a later +sync. Total history disjunction is likewise detectable. + +The oplog is a **transport optimization, not a committed data structure**. Its +contents and history are not guaranteed; a host may compact or drop it, keeping +only a backfill window, and it **resets on account migration** since a new host +starts fresh. A syncer that cannot find its `since` falls back to full-state +recovery, which does not depend on the oplog. + +### Full-state recovery + +Fetch the whole repo as a serialized CAR from `com.atproto.space.getRepo`, +verify as described above, rebuild locally. A syncer replacing an existing copy +diffs and keeps only what it is missing. + +For *healing* a slightly diverged copy there is a lighter path: `getLatestCommit` +for the commit, `listRecords` with `excludeValues` for the paths → CIDs +structure, diff locally, then `getRecord` only the differences. Trades one +round-trip for a smaller total transfer. + +### Write notifications + +Best-effort, carrying no record data, stating only that a repo reached a new +revision. + +A syncer registers via `com.atproto.space.registerNotify` — against the space +host for the whole space (the normal case), or against a specific repo host for +individual repos. Authenticated with a space credential; the registration +expiry may outlive the credential's. + +When a member writes, their PDS sends `com.atproto.space.notifyWrite` to each +registered endpoint. Since a PDS generally does not know who is syncing the +space, the **space authority registers itself as a subscriber on each repo +host** and forwards notifications to syncers registered with it. Repo hosts +**auto-register** the authority's `#atproto_space_host` endpoint on the first +write into a shared space, so no out-of-band setup is needed. Personal-data +spaces, where the authority is the account's own DID, need none of this. + +Dropped notifications are recovered by a later write's notification or by a +periodic sweep: `listRepos` returns each repo's current `rev`, so a syncer +re-syncs only repos that have advanced. + +### The writer set + +To sync a space in full, or to start syncing one, an app needs the set of +accounts holding data in it. That **writer set** comes from +`com.atproto.space.listRepos` on the space authority, with each repo's current +`rev` and `hash`. + +Important scoping: + +- It enumerates accounts that have **written at least one record**, not accounts + merely *allowed* to write (which the authority may not track). +- **Readers are never enumerated at the protocol level.** Apps that want a + reader list must build one from records published in the space. +- It is what the authority *claims*, current only as far as notifications have + kept it. The repo host is source of truth; a syncer confirms each repo by + syncing it directly. + +## OAuth scopes + +``` +space:[?authority=][&skey=][&collection=...][&action=...][&manage=...] +``` + +| Parameter | Multiple | Default | Values | +|---|---|---|---| +| `spaceType` (positional, required) | no | — | space-type NSID, or `*` | +| `authority` | no | `self` | authority DID, `self`, or `*` | +| `skey` | no | `*` | space key (1–512 chars), or `*` | +| `collection` | yes | the type declaration's `collections` | collection NSID, or `*` | +| `action` | yes | `read`, `create`, `update`, `delete` | `read_self`, `read`, `create`, `update`, `delete` | +| `manage` | yes | *(none)* | `create`, `update`, `delete` | + +`authority`/`spaceType`/`skey` select **which spaces**; `action` (+ `collection`) +govern operations on **records**; `manage` governs operations on the **space +itself** and is empty by default, so an ordinary access grant confers no admin +capability. + +Points worth internalizing: + +- **`authority` defaults to `self`.** A bare `space:com.example.bookmarks` + covers only the user's own spaces. Reaching a forum anchored elsewhere + requires naming the authority or `authority=*`. +- **`read` is all-or-nothing at the space boundary.** There is no partial, + per-record, or per-author whole-space read grant. It confers both the read and + sync methods on the holder's own PDS *and* access to `getDelegationToken`. +- **`read_self`** confers the same methods but only for the holder's own repo, + and **not** `getDelegationToken`. This is the grant for a backup or export + tool that must not see other members' posts. `read` implies `read_self`. +- A **space credential** grants whole-space read/sync directly, so read and sync + methods accept either a covering OAuth scope or a credential. Write methods + accept **only** OAuth, since a write is attributed to the authoring user. +- **`collection` defaults dynamically.** It resolves from the space type + declaration *as it stands when the grant is evaluated*, not frozen at consent + time, following permission-set semantics. If the declaration later adds a + collection, existing bare grants widen. Apps that don't want that must + enumerate collections explicitly. +- When `spaceType` is `*` there is no declaration to draw from, so the default + collection set is empty and the grant confers no write targets. +- In a **permission set**, `spaceType` may **not** be a wildcard. A cross-type + grant is expressible only as a standalone requested scope. + +Consent screens show the declaration's `name` in place of the raw NSID, and a +specified `authority` as its bidirectionally-linked handle (or the raw DID if +none validates). Wildcards on both `authority` and `spaceType` must be presented +with a prominent warning. + +Examples: + +``` +space:com.example.bookmarks + the user's own bookmarks space, read + write to declared collections + +space:com.atmoboards.forum?authority=* + every forum the user is in, any authority; the typical forum-client grant + +space:com.atmoboards.forum?authority=*&action=read_self&collection=* + the user's own posts only, across all forums; backup/export tools + +space:com.atmoboards.forum?authority=*&action=read_self&manage=update&manage=delete + administer the user's forums without record-write access + +space:*?authority=did:plc:abc123 + every space under one authority, any type +``` + +## simplespace + +The protocol does not specify how spaces are created or how an authority +decides who may read them. `com.atproto.simplespace` is the space-management +implementation **every PDS MUST support** — not privileged, just guaranteed +present, so applications have a baseline available on every account. Spaces are +anchored on a user's own DID. + +Other space types may define their own management implementations and are full +protocol participants, but they run on bespoke space services rather than on +the PDS. + +Methods (all called with an OAuth credential carrying the relevant `manage` +scope): `createSpace`, `updateSpace`, `deleteSpace`, `addMember`, +`removeMember`, `listMembers`. + +### Configuration + +| Field | Values | Decides | +|---|---|---| +| `policy` | `member-list` (default) \| `public` \| `managing-app` | whether to authorize a **user** | +| `appAccess` | `#open` (default) \| `#allowList` | whether to authorize an **app** | +| `managingApp` | service identifier (DID + fragment) | request routing and the `managing-app` check target | + +Both must pass for a credential to be minted, and a valid delegation token is +required regardless. + +`#open` requires no client attestation, so public clients work. `#allowList` is +evaluated against the **attested** `client_id` (the `iss` of a verified client +attestation), making it enforceable rather than advisory. + +### The managing app + +Under `managing-app` policy, at mint time the authority calls +`com.atproto.simplespace.checkUserAccess` on the space's `managingApp` — served +by the app, not the PDS — with itself as `iss` and the app's service identifier +as `aud`, passing the space, the requesting user, and the attested `client_id`. + +This is the escape hatch for dynamic policy: follower-gating, paid-subscription +status, join approvals, anything the app tracks, without the app maintaining a +literal member list. Any account with space access can read the `managingApp` +from `getSpace`, so it also serves as a routing target for app-level requests +(join requests and similar) a generic PDS cannot handle. + +## XRPC surface + +All under `com.atproto.space`. Roles: **repo** methods concern one account's +repo (repo host), **host** methods concern a space as a whole (space host), +**pds** methods are the required PDS baseline, **syncer** methods are +notification delivery. One service is usually several roles at once. + +| Method | Role | Type | Auth | +|---|---|---|---| +| `getSpace` | host | query | space credential | +| `getSpaceCredential` | host | procedure | delegation token (+ attestation) | +| `listRepos` | host | query | space credential | +| `getRecord` | repo | query | OAuth / space credential | +| `listRecords` | repo | query | OAuth / space credential | +| `getBlob` | repo | query | OAuth / space credential | +| `getLatestCommit` | repo | query | OAuth / space credential | +| `getRepo` | repo | query | OAuth / space credential | +| `listRepoOps` | repo | query | OAuth / space credential | +| `getDelegationToken` | pds | query | OAuth | +| `createRecord` | pds | procedure | OAuth | +| `putRecord` | pds | procedure | OAuth | +| `deleteRecord` | pds | procedure | OAuth | +| `applyWrites` | pds | procedure | OAuth | +| `listSpaces` | pds | query | OAuth | +| `registerNotify` | repo/host | procedure | space credential | +| `notifyWrite` | syncer/host | procedure | service auth | +| `notifySpaceDeleted` | syncer/repo | procedure | service auth | + +## Space deletion + +An authority stops issuing credentials, stops answering for the space, deletes +its own repo in it, then notifies registered syncers and known repo hosts via +`notifySpaceDeleted` over the same best-effort path as write notifications. + +The two sides behave differently, deliberately: + +- A **syncer** should delete every copy it holds, pulled repos and derived state + alike, since it is no longer authorized to retain them. +- A **repo host** should *flag* the member's repo as belonging to a deleted + space rather than erase it. The data is the user's own, so the host decides + how to surface it (export, grace period) before garbage-collecting. + +## Cross-cutting concerns + +**Moderation.** The model is unchanged except that a moderation service cannot +observe a space it has not been admitted to; it is just another reader and needs +a space credential like anyone else. Labelers **should not** publish public +labels for permissioned records, since that leaks metadata about private data. +`com.atproto.label.subscribeLabels` is a poor fit; labels should instead be +records in a permissioned repo inside the same access boundary. The authority +gains one lever with no public analogue: it can decline to issue a credential +and thereby cut off read access entirely. + +**Scaling.** Similar to public broadcast (partitioned per user, no node holds a +whole space), with the absence of relays as the one real difference, placing +load directly on PDSes. Three mitigations: the sync protocol is lighter than +public sync with no MST structural nodes; load scales with the number of +*applications* syncing a space, not end users, since an app pulls each repo once +and fans out from its own copy; and the sync APIs are credentialed rather than +public, so the syncer set is closed, correlatable, and rate-limitable. + +**Account lifecycle.** Tied to the same DID and signing key as the public +identity, so migration, key rotation, deactivation, and deletion all work the +same way. The wrinkle is that a user has **many** permissioned repos rather than +one, so migration flows must enumerate and track all of them plus their blobs. +Notably, an application syncing *only* permissioned repos **still needs a +firehose subscription** to receive `#account` and `#identity` events. A +subscription carrying only those two event types is listed as future work. + +## Implementation status (atproto#5187) + +Present in the branch: + +| Area | Files | +|---|---| +| New `@atproto/space` package | `lthash.ts`, `repo-commit.ts`, `credential.ts`, `sync/provider.ts`, `sync/consumer.ts`, `types.ts`, `util.ts`, `error.ts` + tests for each | +| Lexicons | 17 under `com/atproto/space/`, 8 under `com/atproto/simplespace/` | +| PDS API | all `com.atproto.space.*` and `com.atproto.simplespace.*` handlers | +| PDS storage | `actor-store/space/{reader,transactor}.ts`, migration `002-space.ts` | +| PDS auth | `client-attestation-verifier.ts`, `auth-verifier.ts`, `auth-output.ts` | +| OAuth | `scopes/space-permission.ts`, scope-set + transition changes, provider lexicon manager | +| Consent UI | `oauth-provider-ui` consent form/view, `scope-description.tsx`, 6 locale files | +| Lexicon tooling | `"type": "space"` support across `lex-document`, `lex-schema`, `lex-builder`, `lex-installer` | +| Syntax | `at://` space-URI parsing and validation | +| Crypto | `hmacSha256`, `hkdfSha256` | +| Common | CAR read/write stream helpers | +| Dev env | `bin-multi-pds.ts`, `network-no-appview.ts`, `introspect.ts` | +| Tests | `spaces.test.ts`, `space-scope.test.ts`, `client-attestation.test.ts` | + +The PDS actor-store schema is a useful summary of what a repo host tracks: + +``` +space uri PK, isOwner, policy, managingApp, + appAccessType, appAllowed, createdAt, deletedAt +space_member (space, did) PK -- host-internal list +space_record (space, collection, rkey) PK, cid, value, repoRev +space_repo space PK, setHash (raw 2048B blob), rev +space_record_oplog (space, rev, idx) PK, action, collection, rkey, + cid, prev -- append-only +space_writer (space, did) PK, rev, hash -- the writer set +space_credential_recipient (space, serviceDid) PK, serviceEndpoint, + lastIssuedAt -- registerNotify +``` + +**Not present:** + +- **No syncer or appview implementation.** The consumer-side verification + library exists but nothing in-tree consumes it. Testing is currently PDS↔PDS + via `network-no-appview`, which is what the new dev-env harnesses are for. +- **No relay work**, and no `#account`/`#identity`-only subscription. +- No `@atproto/api` client surface and no Bluesky product surface. Nothing here + is wired to a shipping app yet. + +## What changed since the diaries + +Read the diary posts for reasoning, not for wire details. These were reversed: + +| Diary said | Proposal says | +|---|---| +| A separate `ats://` scheme (Diary 5, argued at length) | `at://` with a literal `space` segment | +| ECMH, elliptic-curve multiset hash (Diary 4) | LtHash, lattice-based | +| A protocol-level member list of `(DID, read\|write)` tuples as the "narrow waist" (Diary 4, 6) | The protocol carries **no** member list; membership is a simplespace implementation detail and the mint decision is undefined at protocol level | +| "Buckets" | "Spaces" (renamed in the Mar 17 interlude; "bucket" wrongly implied colocation). The Diary 2 concept is otherwise intact | +| Asymmetric signatures over repo state | Deniable HMAC-bound commits (Diary 7) | +| Space credentials "~2-4 hour expiration" | 2 hours by default | + +Stable across the whole series: + +- No E2EE at the protocol layer (Diary 1, never revisited) +- Partitioned rather than colocated storage — settled in Diary 3 on storage + burden, moderation liability, availability coupling, and trust grounds +- Pull-based sync, no relay +- **Write permissions are not protocol-enforced.** Anyone can write to their own + repo; readers validate against space rules, the same way apps validate replies + on public posts +- Per-modality spaces rather than one universal community container. The June 2 + post rejects universal spaces on two grounds: consent screens become + illegible ("your Open Communities" vs "your AtmoBoards forums"), and access + boundaries land in the wrong place (an events app would get chat, photos, and + forum posts too). A community is instead several typed spaces under one DID, + with governance provided by services layered on top rather than by the + protocol + +## Open questions + +Things the proposal leaves undefined or flags as future work: + +- The `#account`/`#identity`-only subscription that permissioned-only apps need +- How space *discovery* works for a user across authorities beyond `listSpaces` + on their own PDS +- Migration flows enumerating many permissioned repos and their blobs +- Whether anything outside `simplespace` materializes, and what a bespoke space + service looks like in practice +- Oplog retention windows and compaction policy (host-defined today) +- Non-normative status of the whole "Considerations" section diff --git a/lexicon-authority/wrangler.jsonc b/lexicon-authority/wrangler.jsonc index 54244f3..74a34a3 100644 --- a/lexicon-authority/wrangler.jsonc +++ b/lexicon-authority/wrangler.jsonc @@ -29,8 +29,9 @@ "enabled": true } - // Workers Caching is safe here — its cache key includes the query string, which is what - // XRPC dispatches on. Enable if you want edge caching in front of the script: - // - // "cache": { "enabled": true } + // There is no config key that puts the edge cache in front of a Worker on a Custom Domain: + // every request runs the script (verified — Cloudflare reports cacheStatus=none for 100% of + // traffic). Caching, if ever wanted, means calling the Cache API inside the handler. The + // cache key would include the query string, which is what XRPC dispatches on, so it is safe + // in principle — but at ~0.4 ms CPU per request it buys little. }