mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-26 12:14:17 +00:00
690 lines
33 KiB
Markdown
690 lines
33 KiB
Markdown
# ATProto Permissioned Data (Research Notes)
|
||
|
||
> STATUS: EXTERNAL RESEARCH. Last reviewed 2026-08-03.
|
||
>
|
||
> **This describes work happening in the upstream `bluesky-social/atproto`
|
||
> project, not in ATCR.** Nothing here is implemented in this repository, and
|
||
> nothing here is a commitment that ATCR will adopt it. The upstream spec is
|
||
> itself explicitly labeled "a proposal, not the final specification" and its
|
||
> reference implementation is an open, WIP pull request. Details, terminology,
|
||
> and wire formats are all expected to change.
|
||
>
|
||
> This document exists so we can track the design as it stabilizes and reason
|
||
> about what, if anything, it means for ATCR.
|
||
|
||
## Sources
|
||
|
||
| Source | What it is | Date |
|
||
|---|---|---|
|
||
| [Proposal 0016 Permissioned Data](https://github.com/bluesky-social/proposals/blob/main/0016-permissioned-data/README.md) | The spec of record. Supersedes the diary posts wherever they disagree. | current |
|
||
| [atproto#5187 `permissioned-data`](https://github.com/bluesky-social/atproto/pull/5187) | Reference implementation. Marked "⚠️ WIP ⚠️". 167 files, +12,178/−732. | updated 2026-07-31 |
|
||
| [Discourse discussion thread](https://discourse.atprotocol.community/t/permissioned-data-proposal-discussion/946) | Community forum discussion of the proposal. | ongoing |
|
||
|
||
Design log by Daniel Holmgren, oldest first. Useful for *why*, unreliable for
|
||
*what*: see [What changed since the diaries](#what-changed-since-the-diaries).
|
||
|
||
| Post | Date |
|
||
|---|---|
|
||
| [Diary 1: To Encrypt or Not to Encrypt](https://dholms.leaflet.pub/3meluqcwky22a) | 2026-02-11 |
|
||
| [Diary 2: Buckets](https://dholms.leaflet.pub/3mfrsbcn2gk2a) | 2026-02-26 |
|
||
| [Diary 3: Your Bucket, My Data](https://dholms.leaflet.pub/3mguviy6iks2a) | 2026-03-12 |
|
||
| [Interlude: Spaces](https://dholms.leaflet.pub/3mhbuoc64xk2a) | 2026-03-17 |
|
||
| [Diary 4: The Big Picture](https://dholms.leaflet.pub/3mhj6bcqats2o) | 2026-03-20 |
|
||
| [Diary 5: What's in a Name?](https://dholms.leaflet.pub/3mlegohgtps2k) | 2026-05-08 |
|
||
| [Modeling communities on permissioned data](https://dholms.leaflet.pub/3mndhk7ihsc2g) | 2026-06-02 |
|
||
| [Diary 6: Boring Auth](https://dholms.leaflet.pub/3mnkrxp7rt22i) | 2026-06-05 |
|
||
| [Diary 7: Off the Record](https://dholms.leaflet.pub/3mqtqvjidqs2p) | 2026-07-17 |
|
||
|
||
Referenced external specs: [LtHash](https://eprint.iacr.org/2019/227),
|
||
[CAR](https://dasl.ing/car.html), [DRISL](https://dasl.ing/drisl.html),
|
||
[TLS 1.3 §3.4](https://www.rfc-editor.org/rfc/rfc8446),
|
||
[atproto permission spec](https://atproto.com/specs/permission).
|
||
|
||
## Summary
|
||
|
||
Atproto today is a protocol for **public broadcast**: signed, redistributable,
|
||
universally addressable records that anyone can crawl. Permissioned data adds a
|
||
second data protocol for records with an **access perimeter**, targeting
|
||
personal data (bookmarks, mutes, drafts), gated content (paid newsletters),
|
||
socially shared content (private posts, stories), and groups (private forums,
|
||
group chats).
|
||
|
||
It deliberately keeps the same abstract shape as public broadcast: DID-based
|
||
authority, per-user repositories, lexicon-typed records, applications crawling
|
||
PDSes to build views. What changes is the repository format, the sync
|
||
mechanism, the addressing scheme, and the resolution path.
|
||
|
||
The single most important framing: this provides **access control, not
|
||
confidentiality**. It is explicitly not end-to-end encrypted. Servers (both
|
||
PDSes and authorized applications) read plaintext, which is required for
|
||
search, indexing, notifications, aggregation, and moderation. E2EE is left as
|
||
an app-layer concern.
|
||
|
||
| | Public broadcast | Permissioned data |
|
||
|---|---|---|
|
||
| Unit of data | Record in a repo | Record in a permissioned repo |
|
||
| Repo scope | One repo per user | One permissioned repo per (user, space) |
|
||
| Record authority | User DID | User DID |
|
||
| URI authority | User DID | Space authority DID |
|
||
| Commit | Merkle Search Tree root | LtHash set-hash digest |
|
||
| Signature | Rebroadcastable, archival | Deniable on rebroadcast |
|
||
| Addressing | `at://` URI | `at://` URI with `space` segment |
|
||
| Access | Public | Gated by space credential |
|
||
| Distribution | Relay firehose | Direct pull from each repo host |
|
||
|
||
## Core model
|
||
|
||
### Spaces
|
||
|
||
A **space** is an authorization and sync boundary representing a shared social
|
||
context. It is *not* a storage location. Each participant stores their own
|
||
records for that space in a **permissioned repo** on their own PDS. A space is
|
||
the aggregation of those per-user repos across the network: an application
|
||
pulls each member's repo from its host, assembles the view, and applies access
|
||
control to its own users.
|
||
|
||
A space is identified by a triple:
|
||
|
||
1. **space authority** — a DID, the root of authority
|
||
2. **space type** — an NSID naming the modality
|
||
3. **space key** (`skey`) — a string distinguishing spaces of the same type
|
||
under the same authority (max 512 bytes, `rkey` syntax)
|
||
|
||
The authority may be a user's own DID (personal data: bookmarks, mutes) or a
|
||
dedicated DID, which lets a shared space transfer between owners without
|
||
breaking backlinks.
|
||
|
||
Spaces scale from one user's bookmarks to communities of millions.
|
||
|
||
### Terminology
|
||
|
||
| Term | Meaning |
|
||
|---|---|
|
||
| Space | Authorization + sync boundary, identified by `(authority, type, skey)` |
|
||
| Permissioned repo | One user's records within one space, on their repo host |
|
||
| Repo host | Service storing and serving users' permissioned repos |
|
||
| Space host | Service answering for a space as a whole (credentials, writer set, notification routing) |
|
||
| Space authority | The DID at the root of a space |
|
||
| Space credential | Token from the authority granting read access to a space |
|
||
| Delegation token | Token from a user's PDS, exchanged for a space credential |
|
||
| Client attestation | Token signed by an app's own key, proving app identity |
|
||
| Syncer | Application keeping its own copy of a space in sync |
|
||
|
||
A PDS fills both the repo-host and space-host roles, but they are specified
|
||
separately because neither has to be a PDS.
|
||
|
||
### Addressing
|
||
|
||
Permissioned data reuses `at://` with a fixed literal `space` marker where a
|
||
collection NSID would sit:
|
||
|
||
```
|
||
Space: at://{spaceDid}/space/{spaceType}/{skey}
|
||
Record: at://{spaceDid}/space/{spaceType}/{skey}/{authorDid}/{collection}/{rkey}
|
||
```
|
||
|
||
Never ambiguous with a public URI: a collection NSID always contains at least
|
||
two dots, `space` contains none. The ordering is `SPACE/author/collection/rkey`
|
||
rather than author-first because user authority derives from space membership,
|
||
and because a space reference is then a clean prefix of a record URI.
|
||
|
||
The POC modifies `packages/syntax/src/aturi.ts` and
|
||
`packages/syntax/src/aturi_validation.ts` accordingly.
|
||
|
||
### Space type declarations
|
||
|
||
A space type NSID resolves to a new kind of Lexicon definition,
|
||
`"type": "space"`, which must be the `main` def:
|
||
|
||
```json
|
||
{
|
||
"lexicon": 1,
|
||
"id": "com.atmoboards.forum",
|
||
"defs": {
|
||
"main": {
|
||
"type": "space",
|
||
"description": "A discussion forum",
|
||
"key": "any",
|
||
"name": "AtmoBoards Forum",
|
||
"name:lang": { "es": "Foro AtmoBoards", "ja": "AtmoBoards 掲示板" },
|
||
"collections": ["com.atmoboards.thread", "com.atmoboards.reply"]
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
| Field | Required | Purpose |
|
||
|---|---|---|
|
||
| `type: "space"` | yes | Marks the declaration |
|
||
| `key` | yes | Recommended skey type |
|
||
| `name` (1–64) | yes | **User-facing**, shown on OAuth consent screens |
|
||
| `name:lang` | no | Localized names |
|
||
| `collections` | yes | Default collection set for a `space:` scope of this type |
|
||
| `description` | no | Developer-facing only |
|
||
|
||
`collections` is a default, not a constraint: any collection may be written to
|
||
any space at the protocol level.
|
||
|
||
### DID document entries
|
||
|
||
A space authority is resolved through two optional DID doc entries:
|
||
|
||
- verification method `#atproto_space` — key verifying the space's credentials
|
||
- service `#atproto_space_host` — the space host endpoint
|
||
|
||
When absent they fall back to `#atproto` and `#atproto_pds` respectively, so an
|
||
authority hosted on an ordinary PDS needs no extra DID doc entries at all.
|
||
|
||
## Access control
|
||
|
||
Reading a space requires a **space credential** issued by the space authority.
|
||
The authority decides based on two independent axes, and **the protocol does
|
||
not define the decision procedure** — that is the concern of a space-management
|
||
implementation layered above it (see [simplespace](#simplespace)).
|
||
|
||
### Three token classes
|
||
|
||
All three share a wire shape and are implemented as one parameterized signer in
|
||
`packages/space/src/credential.ts`.
|
||
|
||
| Token | `typ` | Signed by | Lifetime | `aud` | Reuse |
|
||
|---|---|---|---|---|---|
|
||
| Delegation token | `atproto-space-delegation+jwt` | user (`kid: #atproto`) | 60s | space host | single-use |
|
||
| Client attestation | `atproto-client-attestation+jwt` | app's client-auth key | 60s | space host | single-use |
|
||
| Space credential | `atproto-space-credential+jwt` | space authority | 2h | **none** | multi-use |
|
||
|
||
**Delegation token.** Minted by the user's PDS via
|
||
`com.atproto.space.getDelegationToken`. Asserts *only* the user-to-app
|
||
delegation, and says nothing about whether the user is a member of the space.
|
||
Structurally close to a service auth token but a distinct credential class: no
|
||
`lxm` claim, bound to a target space through `sub`.
|
||
|
||
**Client attestation.** Structurally a `private_key_jwt` client assertion, the
|
||
same shape a confidential client already presents to its authorization server,
|
||
but addressed to the space authority. `iss` and `sub` are both the `client_id`.
|
||
The authority verifies it by resolving `client_id` → `client-metadata.json` →
|
||
JWKS → the key named by `kid`. Required only when a space gates on app
|
||
identity, so public clients still work against `#open` spaces.
|
||
|
||
**Space credential.** Deliberately has **no `aud`**: one credential is
|
||
presented to every repo host serving a repo in the space, and any host verifies
|
||
it against the authority's published key without contacting the authority.
|
||
|
||
### Credential flow
|
||
|
||
```
|
||
┌──────┐ ┌────────────┐ ┌─────────────┐ ┌─────────────────┐
|
||
│ User │ │ User's PDS │ │ Application │ │ Space Authority │
|
||
└───┬──┘ └──────┬─────┘ └──────┬──────┘ └────────┬────────┘
|
||
│ │ │ │
|
||
├── OAuth consent ───► │ │
|
||
│ ├───── OAuth token ─────► │
|
||
│ │ │ │
|
||
│ ◄─ getDelegationToken ──┤ │
|
||
│ ├── delegation token ───► │
|
||
│ │ │ getSpaceCredential │
|
||
│ │ ├─(token [+ attestation])─►│
|
||
│ │ ◄──── space credential ────┤
|
||
```
|
||
|
||
The two tokens are presented together but signed by different parties and
|
||
evaluated independently. An application serving many users of a space needs
|
||
only **one** credential, obtainable via any single user's session; when it
|
||
loses every OAuth session for that space it can no longer renew and loses
|
||
access.
|
||
|
||
## Repo format
|
||
|
||
### Commit digest: LtHash
|
||
|
||
No Merkle Search Tree. Permissioned repos sync as complete units bounded by
|
||
space membership, so the MST's partial-proof machinery buys nothing.
|
||
|
||
The digest is [LtHash](https://eprint.iacr.org/2019/227), a homomorphic
|
||
set hash built on a lattice problem (hence quantum-secure):
|
||
|
||
- State is a fixed **2048-byte** buffer read as **1024 little-endian u16 lanes**
|
||
- Each record maps to the element `{collection}/{rkey}/{record_cid}`
|
||
- **Add**: expand the element to 2048 bytes with BLAKE3 in XOF mode, read as
|
||
1024 LE u16 lanes, add lane-wise mod 2^16
|
||
- **Remove**: identical, but subtract
|
||
- Empty repo state is all zeroes
|
||
- Commit carries only `sha256(state)`; the host keeps the full 2048-byte state
|
||
|
||
Both operations commute, so the state depends only on the current record set,
|
||
never on write order. Adding or removing a record is one cheap operation rather
|
||
than a recomputation, so hosts maintain the state incrementally on each write.
|
||
Two repos holding the same records always produce the same digest.
|
||
|
||
Implementation: `packages/space/src/lthash.ts` (~80 lines, `@noble/hashes`,
|
||
two views over one `ArrayBuffer` so `Uint16Array` arithmetic wraps mod 2^16 for
|
||
free).
|
||
|
||
### Deniable commit signatures
|
||
|
||
A signature over the content digest would be a **rebroadcastable proof of what
|
||
a user wrote in a private space**. So the signature covers only random
|
||
per-reader bytes, and the digest is bound to those bytes by a *symmetric* MAC.
|
||
|
||
Both are domain-separated by a shared context string using the TLS 1.3 §3.4
|
||
variable-length vector encoding:
|
||
|
||
```
|
||
ctx = "atproto-space-v1" // fixed protocol tag
|
||
|| uint16be(len(space)) || space // space URI
|
||
|| uint16be(len(author)) || author // author DID of the repo
|
||
|| uint16be(len(rev)) || rev // commit revision (TID)
|
||
|| uint16be(len(ikm)) || ikm // per-signature nonce
|
||
```
|
||
|
||
Producing a commit:
|
||
|
||
1. Generate `ikm`, 32 fresh random bytes. **A new `ikm` per reader served.**
|
||
2. `sig = sign(ctx)` with the user's signing key. The signed message contains
|
||
space, author, rev, and ikm, but **not** the repository hash.
|
||
3. `mac = HMAC-SHA256(HKDF-SHA256(ikm, ctx), hash)`
|
||
|
||
A reader verifies `sig` (authenticity), then recomputes `mac` (integrity), and
|
||
from there trusts `hash`. But because the MAC key derives from the *public*
|
||
`ikm`, anyone holding a leaked commit can compute a valid MAC for any `hash` at
|
||
all. A rebroadcast commit therefore proves only that the user signed a
|
||
`(space, author, rev, ikm)` context, never what they wrote. A leak becomes a
|
||
social and reputational matter rather than a cryptographically provable one.
|
||
|
||
`com.atproto.space.defs#signedCommit`:
|
||
|
||
| Field | Type | Description |
|
||
|---|---|---|
|
||
| `ver` | integer | Commit format version, currently `1` |
|
||
| `hash` | bytes | `sha256` of the LtHash state (32 bytes) |
|
||
| `ikm` | bytes | Per-signature nonce (32 random bytes) |
|
||
| `sig` | bytes | `sign(ctx)` by the user's signing key |
|
||
| `mac` | bytes | `HMAC-SHA256(HKDF-SHA256(ikm, ctx), hash)` |
|
||
| `rev` | string | Commit revision (TID), also bound into `ctx` |
|
||
|
||
Byte-order gotcha the spec calls out explicitly: `ctx` length prefixes are
|
||
**big-endian** (TLS convention), LtHash lanes are **little-endian** (LtHash
|
||
reference construction). Each keeps its native order.
|
||
|
||
Implementation: `packages/space/src/repo-commit.ts`; `hmacSha256` and
|
||
`hkdfSha256` were added to `@atproto/crypto`.
|
||
|
||
### CAR serialization
|
||
|
||
A permissioned repo serializes to a CAR file with **two roots, in order**:
|
||
|
||
1. the **signed commit** block
|
||
2. the **index** — a DRISL (DAG-CBOR) map from `"{collection}/{rkey}"` to the
|
||
record CID, keys in lexicographic order
|
||
|
||
Record blocks follow, in the same order as their index entries. Blobs are
|
||
excluded and fetched separately via `getBlob`.
|
||
|
||
This layout lets a consumer stream-verify in three stages with no buffering:
|
||
|
||
1. Verify the commit's signature and MAC. `hash` is now trusted.
|
||
2. Fold each index entry's `{collection}/{rkey}/{cid}` into a running set hash
|
||
as it is read, compare against `hash`. **This authenticates the entire index
|
||
without reading a single record.**
|
||
3. Verify each record block against its own CID as it streams past.
|
||
|
||
Implementation: `packages/space/src/sync/provider.ts` (serialize),
|
||
`packages/space/src/sync/consumer.ts` (streaming verify).
|
||
|
||
## Sync
|
||
|
||
**There is no relay.** Permissioned repos are non-rebroadcastable by nature, so
|
||
there is no collated firehose. Applications pull directly from each repo host
|
||
and are responsible for keeping their own copy in sync.
|
||
|
||
### Incremental sync
|
||
|
||
A syncer keeps its own copy of a repo plus its own running set hash over that
|
||
copy. Agreement with the host's digest means exactly up to date; disagreement
|
||
means behind or corrupted.
|
||
|
||
`com.atproto.space.listRepoOps(since)` returns operations after a revision.
|
||
Each entry is `{ rev, collection, rkey, cid, prev }`:
|
||
|
||
- `cid` null → delete
|
||
- `prev` null → create
|
||
- entries sharing a `rev` → one atomic multi-record write
|
||
|
||
Record values are **inlined by default**, so a syncer advances in a single call
|
||
with no per-write `getRecord` round-trip. Only the current value for a path is
|
||
inlined; superseded intermediates are omitted. `excludeValues` gives
|
||
metadata-only entries.
|
||
|
||
If a response includes the last available operation, it must also carry the
|
||
member's current signed commit. The syncer compares it against its running set
|
||
hash: match means fully in sync and authenticated; mismatch means diverged,
|
||
fall back to recovery.
|
||
|
||
Sync is **self-healing** because correctness rests on the set-hash comparison
|
||
rather than on receiving every operation. A missed op is detected on a later
|
||
sync. Total history disjunction is likewise detectable.
|
||
|
||
The oplog is a **transport optimization, not a committed data structure**. Its
|
||
contents and history are not guaranteed; a host may compact or drop it, keeping
|
||
only a backfill window, and it **resets on account migration** since a new host
|
||
starts fresh. A syncer that cannot find its `since` falls back to full-state
|
||
recovery, which does not depend on the oplog.
|
||
|
||
### Full-state recovery
|
||
|
||
Fetch the whole repo as a serialized CAR from `com.atproto.space.getRepo`,
|
||
verify as described above, rebuild locally. A syncer replacing an existing copy
|
||
diffs and keeps only what it is missing.
|
||
|
||
For *healing* a slightly diverged copy there is a lighter path: `getLatestCommit`
|
||
for the commit, `listRecords` with `excludeValues` for the paths → CIDs
|
||
structure, diff locally, then `getRecord` only the differences. Trades one
|
||
round-trip for a smaller total transfer.
|
||
|
||
### Write notifications
|
||
|
||
Best-effort, carrying no record data, stating only that a repo reached a new
|
||
revision.
|
||
|
||
A syncer registers via `com.atproto.space.registerNotify` — against the space
|
||
host for the whole space (the normal case), or against a specific repo host for
|
||
individual repos. Authenticated with a space credential; the registration
|
||
expiry may outlive the credential's.
|
||
|
||
When a member writes, their PDS sends `com.atproto.space.notifyWrite` to each
|
||
registered endpoint. Since a PDS generally does not know who is syncing the
|
||
space, the **space authority registers itself as a subscriber on each repo
|
||
host** and forwards notifications to syncers registered with it. Repo hosts
|
||
**auto-register** the authority's `#atproto_space_host` endpoint on the first
|
||
write into a shared space, so no out-of-band setup is needed. Personal-data
|
||
spaces, where the authority is the account's own DID, need none of this.
|
||
|
||
Dropped notifications are recovered by a later write's notification or by a
|
||
periodic sweep: `listRepos` returns each repo's current `rev`, so a syncer
|
||
re-syncs only repos that have advanced.
|
||
|
||
### The writer set
|
||
|
||
To sync a space in full, or to start syncing one, an app needs the set of
|
||
accounts holding data in it. That **writer set** comes from
|
||
`com.atproto.space.listRepos` on the space authority, with each repo's current
|
||
`rev` and `hash`.
|
||
|
||
Important scoping:
|
||
|
||
- It enumerates accounts that have **written at least one record**, not accounts
|
||
merely *allowed* to write (which the authority may not track).
|
||
- **Readers are never enumerated at the protocol level.** Apps that want a
|
||
reader list must build one from records published in the space.
|
||
- It is what the authority *claims*, current only as far as notifications have
|
||
kept it. The repo host is source of truth; a syncer confirms each repo by
|
||
syncing it directly.
|
||
|
||
## OAuth scopes
|
||
|
||
```
|
||
space:<spaceType>[?authority=<did>][&skey=<skey>][&collection=<nsid>...][&action=<action>...][&manage=<op>...]
|
||
```
|
||
|
||
| Parameter | Multiple | Default | Values |
|
||
|---|---|---|---|
|
||
| `spaceType` (positional, required) | no | — | space-type NSID, or `*` |
|
||
| `authority` | no | `self` | authority DID, `self`, or `*` |
|
||
| `skey` | no | `*` | space key (1–512 chars), or `*` |
|
||
| `collection` | yes | the type declaration's `collections` | collection NSID, or `*` |
|
||
| `action` | yes | `read`, `create`, `update`, `delete` | `read_self`, `read`, `create`, `update`, `delete` |
|
||
| `manage` | yes | *(none)* | `create`, `update`, `delete` |
|
||
|
||
`authority`/`spaceType`/`skey` select **which spaces**; `action` (+ `collection`)
|
||
govern operations on **records**; `manage` governs operations on the **space
|
||
itself** and is empty by default, so an ordinary access grant confers no admin
|
||
capability.
|
||
|
||
Points worth internalizing:
|
||
|
||
- **`authority` defaults to `self`.** A bare `space:com.example.bookmarks`
|
||
covers only the user's own spaces. Reaching a forum anchored elsewhere
|
||
requires naming the authority or `authority=*`.
|
||
- **`read` is all-or-nothing at the space boundary.** There is no partial,
|
||
per-record, or per-author whole-space read grant. It confers both the read and
|
||
sync methods on the holder's own PDS *and* access to `getDelegationToken`.
|
||
- **`read_self`** confers the same methods but only for the holder's own repo,
|
||
and **not** `getDelegationToken`. This is the grant for a backup or export
|
||
tool that must not see other members' posts. `read` implies `read_self`.
|
||
- A **space credential** grants whole-space read/sync directly, so read and sync
|
||
methods accept either a covering OAuth scope or a credential. Write methods
|
||
accept **only** OAuth, since a write is attributed to the authoring user.
|
||
- **`collection` defaults dynamically.** It resolves from the space type
|
||
declaration *as it stands when the grant is evaluated*, not frozen at consent
|
||
time, following permission-set semantics. If the declaration later adds a
|
||
collection, existing bare grants widen. Apps that don't want that must
|
||
enumerate collections explicitly.
|
||
- When `spaceType` is `*` there is no declaration to draw from, so the default
|
||
collection set is empty and the grant confers no write targets.
|
||
- In a **permission set**, `spaceType` may **not** be a wildcard. A cross-type
|
||
grant is expressible only as a standalone requested scope.
|
||
|
||
Consent screens show the declaration's `name` in place of the raw NSID, and a
|
||
specified `authority` as its bidirectionally-linked handle (or the raw DID if
|
||
none validates). Wildcards on both `authority` and `spaceType` must be presented
|
||
with a prominent warning.
|
||
|
||
Examples:
|
||
|
||
```
|
||
space:com.example.bookmarks
|
||
the user's own bookmarks space, read + write to declared collections
|
||
|
||
space:com.atmoboards.forum?authority=*
|
||
every forum the user is in, any authority; the typical forum-client grant
|
||
|
||
space:com.atmoboards.forum?authority=*&action=read_self&collection=*
|
||
the user's own posts only, across all forums; backup/export tools
|
||
|
||
space:com.atmoboards.forum?authority=*&action=read_self&manage=update&manage=delete
|
||
administer the user's forums without record-write access
|
||
|
||
space:*?authority=did:plc:abc123
|
||
every space under one authority, any type
|
||
```
|
||
|
||
## simplespace
|
||
|
||
The protocol does not specify how spaces are created or how an authority
|
||
decides who may read them. `com.atproto.simplespace` is the space-management
|
||
implementation **every PDS MUST support** — not privileged, just guaranteed
|
||
present, so applications have a baseline available on every account. Spaces are
|
||
anchored on a user's own DID.
|
||
|
||
Other space types may define their own management implementations and are full
|
||
protocol participants, but they run on bespoke space services rather than on
|
||
the PDS.
|
||
|
||
Methods (all called with an OAuth credential carrying the relevant `manage`
|
||
scope): `createSpace`, `updateSpace`, `deleteSpace`, `addMember`,
|
||
`removeMember`, `listMembers`.
|
||
|
||
### Configuration
|
||
|
||
| Field | Values | Decides |
|
||
|---|---|---|
|
||
| `policy` | `member-list` (default) \| `public` \| `managing-app` | whether to authorize a **user** |
|
||
| `appAccess` | `#open` (default) \| `#allowList` | whether to authorize an **app** |
|
||
| `managingApp` | service identifier (DID + fragment) | request routing and the `managing-app` check target |
|
||
|
||
Both must pass for a credential to be minted, and a valid delegation token is
|
||
required regardless.
|
||
|
||
`#open` requires no client attestation, so public clients work. `#allowList` is
|
||
evaluated against the **attested** `client_id` (the `iss` of a verified client
|
||
attestation), making it enforceable rather than advisory.
|
||
|
||
### The managing app
|
||
|
||
Under `managing-app` policy, at mint time the authority calls
|
||
`com.atproto.simplespace.checkUserAccess` on the space's `managingApp` — served
|
||
by the app, not the PDS — with itself as `iss` and the app's service identifier
|
||
as `aud`, passing the space, the requesting user, and the attested `client_id`.
|
||
|
||
This is the escape hatch for dynamic policy: follower-gating, paid-subscription
|
||
status, join approvals, anything the app tracks, without the app maintaining a
|
||
literal member list. Any account with space access can read the `managingApp`
|
||
from `getSpace`, so it also serves as a routing target for app-level requests
|
||
(join requests and similar) a generic PDS cannot handle.
|
||
|
||
## XRPC surface
|
||
|
||
All under `com.atproto.space`. Roles: **repo** methods concern one account's
|
||
repo (repo host), **host** methods concern a space as a whole (space host),
|
||
**pds** methods are the required PDS baseline, **syncer** methods are
|
||
notification delivery. One service is usually several roles at once.
|
||
|
||
| Method | Role | Type | Auth |
|
||
|---|---|---|---|
|
||
| `getSpace` | host | query | space credential |
|
||
| `getSpaceCredential` | host | procedure | delegation token (+ attestation) |
|
||
| `listRepos` | host | query | space credential |
|
||
| `getRecord` | repo | query | OAuth / space credential |
|
||
| `listRecords` | repo | query | OAuth / space credential |
|
||
| `getBlob` | repo | query | OAuth / space credential |
|
||
| `getLatestCommit` | repo | query | OAuth / space credential |
|
||
| `getRepo` | repo | query | OAuth / space credential |
|
||
| `listRepoOps` | repo | query | OAuth / space credential |
|
||
| `getDelegationToken` | pds | query | OAuth |
|
||
| `createRecord` | pds | procedure | OAuth |
|
||
| `putRecord` | pds | procedure | OAuth |
|
||
| `deleteRecord` | pds | procedure | OAuth |
|
||
| `applyWrites` | pds | procedure | OAuth |
|
||
| `listSpaces` | pds | query | OAuth |
|
||
| `registerNotify` | repo/host | procedure | space credential |
|
||
| `notifyWrite` | syncer/host | procedure | service auth |
|
||
| `notifySpaceDeleted` | syncer/repo | procedure | service auth |
|
||
|
||
## Space deletion
|
||
|
||
An authority stops issuing credentials, stops answering for the space, deletes
|
||
its own repo in it, then notifies registered syncers and known repo hosts via
|
||
`notifySpaceDeleted` over the same best-effort path as write notifications.
|
||
|
||
The two sides behave differently, deliberately:
|
||
|
||
- A **syncer** should delete every copy it holds, pulled repos and derived state
|
||
alike, since it is no longer authorized to retain them.
|
||
- A **repo host** should *flag* the member's repo as belonging to a deleted
|
||
space rather than erase it. The data is the user's own, so the host decides
|
||
how to surface it (export, grace period) before garbage-collecting.
|
||
|
||
## Cross-cutting concerns
|
||
|
||
**Moderation.** The model is unchanged except that a moderation service cannot
|
||
observe a space it has not been admitted to; it is just another reader and needs
|
||
a space credential like anyone else. Labelers **should not** publish public
|
||
labels for permissioned records, since that leaks metadata about private data.
|
||
`com.atproto.label.subscribeLabels` is a poor fit; labels should instead be
|
||
records in a permissioned repo inside the same access boundary. The authority
|
||
gains one lever with no public analogue: it can decline to issue a credential
|
||
and thereby cut off read access entirely.
|
||
|
||
**Scaling.** Similar to public broadcast (partitioned per user, no node holds a
|
||
whole space), with the absence of relays as the one real difference, placing
|
||
load directly on PDSes. Three mitigations: the sync protocol is lighter than
|
||
public sync with no MST structural nodes; load scales with the number of
|
||
*applications* syncing a space, not end users, since an app pulls each repo once
|
||
and fans out from its own copy; and the sync APIs are credentialed rather than
|
||
public, so the syncer set is closed, correlatable, and rate-limitable.
|
||
|
||
**Account lifecycle.** Tied to the same DID and signing key as the public
|
||
identity, so migration, key rotation, deactivation, and deletion all work the
|
||
same way. The wrinkle is that a user has **many** permissioned repos rather than
|
||
one, so migration flows must enumerate and track all of them plus their blobs.
|
||
Notably, an application syncing *only* permissioned repos **still needs a
|
||
firehose subscription** to receive `#account` and `#identity` events. A
|
||
subscription carrying only those two event types is listed as future work.
|
||
|
||
## Implementation status (atproto#5187)
|
||
|
||
Present in the branch:
|
||
|
||
| Area | Files |
|
||
|---|---|
|
||
| New `@atproto/space` package | `lthash.ts`, `repo-commit.ts`, `credential.ts`, `sync/provider.ts`, `sync/consumer.ts`, `types.ts`, `util.ts`, `error.ts` + tests for each |
|
||
| Lexicons | 17 under `com/atproto/space/`, 8 under `com/atproto/simplespace/` |
|
||
| PDS API | all `com.atproto.space.*` and `com.atproto.simplespace.*` handlers |
|
||
| PDS storage | `actor-store/space/{reader,transactor}.ts`, migration `002-space.ts` |
|
||
| PDS auth | `client-attestation-verifier.ts`, `auth-verifier.ts`, `auth-output.ts` |
|
||
| OAuth | `scopes/space-permission.ts`, scope-set + transition changes, provider lexicon manager |
|
||
| Consent UI | `oauth-provider-ui` consent form/view, `scope-description.tsx`, 6 locale files |
|
||
| Lexicon tooling | `"type": "space"` support across `lex-document`, `lex-schema`, `lex-builder`, `lex-installer` |
|
||
| Syntax | `at://` space-URI parsing and validation |
|
||
| Crypto | `hmacSha256`, `hkdfSha256` |
|
||
| Common | CAR read/write stream helpers |
|
||
| Dev env | `bin-multi-pds.ts`, `network-no-appview.ts`, `introspect.ts` |
|
||
| Tests | `spaces.test.ts`, `space-scope.test.ts`, `client-attestation.test.ts` |
|
||
|
||
The PDS actor-store schema is a useful summary of what a repo host tracks:
|
||
|
||
```
|
||
space uri PK, isOwner, policy, managingApp,
|
||
appAccessType, appAllowed, createdAt, deletedAt
|
||
space_member (space, did) PK -- host-internal list
|
||
space_record (space, collection, rkey) PK, cid, value, repoRev
|
||
space_repo space PK, setHash (raw 2048B blob), rev
|
||
space_record_oplog (space, rev, idx) PK, action, collection, rkey,
|
||
cid, prev -- append-only
|
||
space_writer (space, did) PK, rev, hash -- the writer set
|
||
space_credential_recipient (space, serviceDid) PK, serviceEndpoint,
|
||
lastIssuedAt -- registerNotify
|
||
```
|
||
|
||
**Not present:**
|
||
|
||
- **No syncer or appview implementation.** The consumer-side verification
|
||
library exists but nothing in-tree consumes it. Testing is currently PDS↔PDS
|
||
via `network-no-appview`, which is what the new dev-env harnesses are for.
|
||
- **No relay work**, and no `#account`/`#identity`-only subscription.
|
||
- No `@atproto/api` client surface and no Bluesky product surface. Nothing here
|
||
is wired to a shipping app yet.
|
||
|
||
## What changed since the diaries
|
||
|
||
Read the diary posts for reasoning, not for wire details. These were reversed:
|
||
|
||
| Diary said | Proposal says |
|
||
|---|---|
|
||
| A separate `ats://` scheme (Diary 5, argued at length) | `at://` with a literal `space` segment |
|
||
| ECMH, elliptic-curve multiset hash (Diary 4) | LtHash, lattice-based |
|
||
| A protocol-level member list of `(DID, read\|write)` tuples as the "narrow waist" (Diary 4, 6) | The protocol carries **no** member list; membership is a simplespace implementation detail and the mint decision is undefined at protocol level |
|
||
| "Buckets" | "Spaces" (renamed in the Mar 17 interlude; "bucket" wrongly implied colocation). The Diary 2 concept is otherwise intact |
|
||
| Asymmetric signatures over repo state | Deniable HMAC-bound commits (Diary 7) |
|
||
| Space credentials "~2-4 hour expiration" | 2 hours by default |
|
||
|
||
Stable across the whole series:
|
||
|
||
- No E2EE at the protocol layer (Diary 1, never revisited)
|
||
- Partitioned rather than colocated storage — settled in Diary 3 on storage
|
||
burden, moderation liability, availability coupling, and trust grounds
|
||
- Pull-based sync, no relay
|
||
- **Write permissions are not protocol-enforced.** Anyone can write to their own
|
||
repo; readers validate against space rules, the same way apps validate replies
|
||
on public posts
|
||
- Per-modality spaces rather than one universal community container. The June 2
|
||
post rejects universal spaces on two grounds: consent screens become
|
||
illegible ("your Open Communities" vs "your AtmoBoards forums"), and access
|
||
boundaries land in the wrong place (an events app would get chat, photos, and
|
||
forum posts too). A community is instead several typed spaces under one DID,
|
||
with governance provided by services layered on top rather than by the
|
||
protocol
|
||
|
||
## Open questions
|
||
|
||
Things the proposal leaves undefined or flags as future work:
|
||
|
||
- The `#account`/`#identity`-only subscription that permissioned-only apps need
|
||
- How space *discovery* works for a user across authorities beyond `listSpaces`
|
||
on their own PDS
|
||
- Migration flows enumerating many permissioned repos and their blobs
|
||
- Whether anything outside `simplespace` materializes, and what a bespoke space
|
||
service looks like in practice
|
||
- Oplog retention windows and compaction policy (host-defined today)
|
||
- Non-normative status of the whole "Considerations" section
|