Compare commits

...
Author SHA1 Message Date
nelind 75c341b94b chore: bump version to 0.6.7 2026-09-20 22:29:26 +02:00
nelind 0b29c40b27 docs: list jola.dev as a core team maintainer 2026-09-20 18:51:59 +02:00
Anastasia CalicoandTangled 34ece34296 feat: Implement s3_path in storage config. 2026-09-20 16:47:22 +00:00
Johanna LarssonandTangled 833356116a Set inputmode on inputs
This improves mobile behavior for the inputs without affecting other things.

Two inputs set to email, rest to URL pre recommendation in https://tangled.org/tranquil.farm/tranquil-pds/issues/140

Also disables autocapitalize, autocorrect, and spellcheck, while we're at it. All really annoying when typing in handles on a mobile phone.
2026-09-19 14:10:15 +00:00
Johanna LarssonandTangled 643e1bb902 Make ownership repair job report failures more loudly 2026-09-17 12:35:55 +00:00
Johanna LarssonandTangled 9f05ea5f31 Introduce job to repair missing blobs
My last PR introduced a migration to blobs, changing them from being unique per cid to being unique per cid+user. This is because two users can upload the same blob, get the same cid, and then previously in tranquil the second user would never actually get their ownership recorded, meaning listBlobs was missing the blob, and a migration _off_ of tranquil would leave them sans blob https://tangled.org/tranquil.farm/tranquil-pds/pulls/261/round/3

This PR attempts to add a job to repair existing instances where blobs have been deduplicated and not attributed to each owning user. It does this by pulling all records for each user, and then in batches crawling the records to find blob references. For each blob reference, it checks whether that user has its ownership recorded, and if not, repairs it by adding the entry.

To avoid re-running this potentially heavy job on every startup, I've used the config table to store a little record of whether it has been run.
2026-09-17 12:35:55 +00:00
TrezyandTangled 71cd282d1e feat: show repo actions narrowed out by client metadata on consent
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 0e40bdca19 fix: store filtered scopes on tokens so refresh can skip the client metadata check
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 156066fe1b fix: reject PAR requests missing the atproto scope
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 311530a9a9 test: give the mock client a scope so NotRegistered actually gets exercised
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 08cd3fa100 chore: remove unreachable scope format check in consent
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 12a8712eae fix: unrecognized scopes can no longer fail PAR
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
LunaandTangled cdd5fa70c9 fix: add MIME version to outgoing mail 2026-09-14 13:40:08 +00:00
xan.lolandTangled 218741050d add a period (spelling fix) 2026-09-13 22:47:12 +00:00
Johanna LarssonandTangled b3ff62c221 Simplify blob migration 2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 2088f59197 Stop deleting logo blob when switching
This also means we can clean up the last blob operations that were per cid rather than user+cid 🪓

We should be setting ourselves up to be able to go garbage collect blobs safely so deleting the logo blobs manually won't matter anyway.
2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 2fc5f2e308 Wrap in counter lock and stop deleting blobs by cid 2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 877b587481 Ensure listing missing blobs always checks per user blobs 2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 695a7d981c Allow blobs to be owned by multiple accounts
The current tranquil database design only allows each blob to be owned by one account. This means that if a second account also has that blob, tranquil skips associated the blob with the account. That works fine a lot of the time, since blobs are looked up by cid and the blob exists. However, it can lead to loss of data under certain scenarios.

One example is where I upload a blob, the blob already exists in my instance so insertion is skipped (postgres requires cid to be unique in blobs, fjall only allows one owner per blob cid), I then decide to migrate off tranquil, the blob does not come with me since it is not mine.

Another example is where an account is deleted. If a blob was uploaded for account a, then account b uploads the same blob tranquil skips storing it since it exists. Then I delete account a, now account b's blob is missing.

I accidentally stumbled upon this when I migrated my account to my own tranuil instance and list blobs now lists 2 fewer blobs than before, two images that had been uploaded by accounts already on the PDS.

ps I found record_blobs a bit confusing, at first it looked like a blob ownership table, but then it turns out to just be used for migrations!

This PR makes the blob primary key be cid+user for postgres, and updates the queries to account for there being multiple "blobs" with the same cid. For queries that just care about the blob existing, it doesn't matter "whose" blob it is, so limit 1.

Most of the work is on the metastore side. Adds ref_count to track how many are referencing the blob since we can't just check for other rows. Instead of storing blobs directly, we now store a per account cid, and the blob reference itself is shared and keyed by cid only. This means some of these operations now require updating two places, so they're done in `batch`es.

With the new layout get_blob_value becomes simpler, all blob data is a single "table" or whatever it's called, so we just grab it using cid instead of looking it up for the user.

Migrates blobs rather than maintaining two different versions of the tables, although it seems like that could be supported.

I removed a test that asserted the old behavior, and added a reasonable (?) set of new tests that assert the new behavior, including a parity test.
2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 04689cbe25 Improve logging and correct translation string 2026-09-11 19:54:37 +00:00
Johanna LarssonandTangled 3474ed588d Make legacy login alerts configurable
This adds security alerts section to the communication settings, with one new item: a toggle for enabling/disabling legacy login alerts. It's enabled by default. Disabling it means that you no longer get emails when logging in through a non-2FA enabled login flow, like createSession.

The reason I tackled this is that I have a bot account that automatically refreshes its session, using username and app password, and I'm close to having 100 of these emails in my inbox. I also tried to just disable TOTP but I wasn't allowed (I fixed that in a different PR).

English and Swedish translations were me, the rest were MyMemory. I imagine someone can improve on them after this is merged!
2026-09-11 19:54:37 +00:00
Johanna LarssonandTangled 09ba5e4521 Allow disabling TOTP even if last session was legacy
This removes the `require_legacy_session_mfa` call on the TOTP disable path. It looks at the last session, which isn't really relevant, and we're getting a fresh TOTP from the form anyway, which should be enough proof to allow disabling TOTP.

The way I bumped into this was having a "bot account" log in with app password on some app on a schdule, triggering lots of warning emails. So I tried to disable TOTP to stop the flood and wasn't allowed.
2026-09-05 18:16:56 +00:00
SteffoandTangled 6750aeccaf fix: Use "registered notification channel" where applicable 2026-09-03 13:37:09 +00:00
LewisandTangled 8c3386a3ab caddy: on-demand TLS endpoint
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-09-01 16:31:10 +00:00
TrezyandTangled eba8167da8 chore: clean up supersedence shtuff
Signed-off-by: Trezy <tre@trezy.com>
2026-08-29 20:11:10 +00:00
TrezyandTangled 2e92310518 fix: allow transition:generic to be used with granular scopes
Signed-off-by: Trezy <tre@trezy.com>
2026-08-29 20:11:10 +00:00
nelindandTangled 0e82a38add fix(api): dont do rotation key validation in signPlcOperation as it blocks migrations 2026-08-29 05:42:03 +00:00
TrezyandTangled 1866d2bf0e test: i dunno what youre talking about i would never leave prints in production code 🙃
Signed-off-by: Trezy <tre@trezy.com>
2026-08-28 20:28:54 +00:00
TrezyandTangled dc2f924130 fix: add transition: scopes to owner-level delegates
Signed-off-by: Trezy <tre@trezy.com>
2026-08-28 20:28:54 +00:00
TrezyandTangled 97224551bf test: verify taxonomy entries parse to the correct type
Signed-off-by: Trezy <tre@trezy.com>
2026-08-28 20:28:54 +00:00
TrezyandTangled 68ae485a52 fix: restore rpc: scopes for delegation
Signed-off-by: Trezy <tre@trezy.com>
2026-08-28 20:28:54 +00:00
Johanna LarssonandTangled 228c1bbbf5 Normalize the allow private IPs flag application 2026-08-28 12:36:47 +00:00
Johanna LarssonandTangled 093484388f Allow private IPs in dev
Running tranquil with `just run-dev` using `pds.test` seems to be broken due to a recent change that blocks requests to private ips. Inside the compose network `pds.test` resolves to the traefik container's private IP. So it can't make requests to stuff like `https://pds.test/oauth-client-metadata.json` or the local plc.

Introduces a new flag, default off, that allows connecting to private IPs, set to true for the dev compose.
2026-08-28 12:36:47 +00:00
Johanna LarssonandTangled 0f0c50f7d4 Loosen CORS header requirements
Bluesky started sending `x-bsky-is-beta-user` for some users on XRPC requests, but tranquil has a strict allowlist of CORS headers. The spec doesn't (?) specify any requirements around CORS headers, so we can avoid trouble when Bluesky make changes by just allowing all headers.

This PR replaces the allowlist with the request mirror behavior, where any headers sent on a CORS request are echoed back. This also matches the reference PDS.
2026-08-28 07:16:50 +00:00
Jack PlattenandTangled 26aa399cda Fix name of test 2026-08-27 20:07:59 +00:00
Jack PlattenandTangled f296bb68df fix: allow path-empty URIs and drop authority/path charset checks
RFC 3986 lets hier-part be path-empty, so "urn:" alone is a valid
URI; treat it as one.
2026-08-27 20:07:59 +00:00
Jack PlattenandTangled d979cb969a add underscore to scheme support.
adds tests for mbid directly, as well as underscore scheme.

Followup:
`scheme:` is not currently passing this function, but can wait for now
2026-08-27 20:07:59 +00:00
739db41130 fix: accept RFC 3986 scheme:opaque-part URIs without //
is_valid_uri required a literal "://", but the atproto uri string
format follows RFC 3986's generic URI grammar, which also allows
"scheme:opaque-part" forms with no authority (e.g. urn:isbn:...).
Records using such values were rejected once production lexicons
enable strict validation.

Reported as #130.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-27 20:07:59 +00:00
Johanna LarssonandTangled c0caa93228 Dev compose improvements
1. Set the max connections to 20, I frequently see

db-1            | 2026-08-23 08:57:02.784 UTC [12641] FATAL:  sorry, too many clients already

2. Add a wildcard route in Traefik to serve DID documents locally.

3. Expose the PLC port locally so I can point an app at `http://localhost:2582` and be able to do full OAuth flows.
2026-08-23 09:50:08 +00:00
blooym.devandTangled 3ade3d10c1 docs: clarify the postgres seq fix command
There was a syntax error in this command, so I updated the doc to clarify it and also show it needs an integer, not a string like it implied before.
2026-08-21 17:35:46 +00:00
Jack PlattenandTangled 0189aa9f96 Update config commit to create new round 2026-08-21 16:16:45 +00:00
Jack PlattenandTangled d495d7d729 Use crate::types::queuedcomms export
also generate example.toml
2026-08-21 16:16:45 +00:00
73cb89c9b7 resolve review feedback.
- eliminates panic opportunity on receiving email
- strict enum
- added unit test for ensuring that atmos headers don't leak onto
  directmx

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-21 16:16:45 +00:00
Jack PlattenandTangled ecb7934a20 fix: fix missing test failure 2026-08-21 16:16:45 +00:00
Jack PlattenandTangled 9edc7dcdd8 comms: add comail.at category support
Adds a defaulted to off option to add the `X-Atmos-Category` headers
to emails sent via smarthost, for proper categorization by comail.

Category breakdown is as follows:

verification: EmailVerification, ChannelVerification, ChannelVerified,
  MigrationVerification, LegacyLoginAlert, EmailUpdate, PlcOperation,
  AccountDeletion
password-reset: PasswordReset, PasskeyRecovery
mfa-otp: TwoFactorCode
bulk: Welcome
untagged: AdminEmail
2026-08-21 16:16:45 +00:00
Matan KushnerandTangled 479fa3ed22 fix: require DPoP for loopback clients 2026-08-21 11:41:29 +00:00
Louis EscherandTangled aa815931e0 Update lib.rs 2026-08-20 08:21:53 +00:00
Louis EscherandTangled 0ce725174d fix: DID length test, service test, cloning, dead code (should be it!) 2026-08-20 08:21:53 +00:00
Louis EscherandTangled dae3cc7e08 fix: aud fragment matching 2026-08-20 08:21:53 +00:00
Louis EscherandTangled b9e7955606 fix: pass exp to token creation 2026-08-20 08:21:53 +00:00
Louis EscherandTangled 32c58b1d0b fix: make thingy allow list 2026-08-20 08:21:53 +00:00
Louis EscherandTangled 1b5a2b319c fix: getServiceAuth aud parsing 2026-08-20 08:21:53 +00:00
LewisandTangled ed3d129594 just: clippy over all targets, lint the bsky-off build
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
LewisandTangled 8d0b6f8322 cache: DID, SSO, & OAuth client metadata caches onto shared cache
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
LewisandTangled 0fc577316e lexicon: schema docs & negative results via cluster cache
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
LewisandTangled 52d5236e89 plc: dedup fetch paths, cache TTL from config
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
LewisandTangled 0274f19d75 auth: EmailTokenPurpose from tranquil-types, shared cache key fns, MemoryCache in tests
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
LewisandTangled 135912194d types: HttpUrl newtypes, shared cache key/JSON helpers
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
LewisandTangled 0b8787d1de pds: compile bsky-specific proxy, CORS, & validation out under bsky features
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 17:15:23 +00:00
Lewis 18455f54f2 api: moorfc compliance endpoint toggle
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-08-16 19:58:24 +03:00
Louis EscherandTangled ce2f05b9d4 fix: make coverage triple state instead of boolean 2026-08-13 16:09:03 +00:00
Louis EscherandTangled c88f69f31d fix: Address PR review 2026-08-13 16:09:03 +00:00
Louis EscherandTangled b3c314ce66 fix: Address review comments 2026-08-13 16:09:03 +00:00
Louis EscherandTangled 434079a732 feat: compress large token scopes with brotli 2026-08-13 16:09:03 +00:00
Louis EscherandTangled a5a2f30bbe fix: Collapse action parameters for repo scopes
TODO: Still missing tests!
2026-08-13 16:09:03 +00:00
Jack PlattenandTangled dc2fbe6654 chore: use JSON array for healthcheck now that container is distroless 2026-08-13 05:35:23 +00:00
Johanna LarssonandTangled bc751b0ee2 Bring back thing that made yubikey work 2026-08-08 15:41:29 +00:00
Johanna LarssonandTangled 9e78206cf4 Switch back to SecurityKey, remove hint 2026-08-08 15:41:29 +00:00
Johanna LarssonandTangled 779dc1b985 Replace SecurityKey with Passkey 2026-08-08 15:41:29 +00:00
Edmund EdgarandTangled 1dc0c40206 fix: don't require a server-custodied rotation key
Remove the requirement that the user's did can be controlled by the
server rotation key and signing key. This was preventing users who
didn't want to trust the PDS with their rotation keys from activating
their accounts. Errors are removed but we issue debug messages.
2026-07-26 11:45:09 +03:00
nelind 72fa88d79a chore: bump version to 0.6.6 2026-07-26 00:25:56 +02:00
nelindandTangled 596b9b15fd feat(docs): explain our definition of "correct PDS implementation" 2026-07-25 18:31:06 +03:00
nelindandTangled 59934cc184 feat: add bsky and bsky-support cargo features to manage bsky specific code 2026-07-25 18:31:06 +03:00
nelindandTangled 34a47e6e5a chore: clean up Cargo.toml files with cargo-shear and remove default feature on lib crates 2026-07-25 18:31:06 +03:00
183 changed files with 7257 additions and 2490 deletions
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1",
"query": "SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1 LIMIT 1",
"describe": {
"columns": [
{
@@ -30,5 +30,5 @@
false
]
},
"hash": "dd1b61d6ec81fd891d4effd3b51e6c22308b878acdc5355dfcb04c5664c9463b"
"hash": "03f129e4984e1bed9e87294adc9caf1730906d889101b9039113ec8aa234618d"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as \"total!\" FROM blobs",
"query": "SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as \"total!\"\n FROM (SELECT DISTINCT cid, size_bytes FROM blobs) t",
"describe": {
"columns": [
{
@@ -16,5 +16,5 @@
null
]
},
"hash": "0890b2c7c921005f58ed0e57b6e062b2085ce804a4cccb27b4ae2ba6711f24c4"
"hash": "155efbae4cd55f73ec0709dda7b18a76e92065e6ae4a6081bd38a19821fbfcc3"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT cid, takedown_ref FROM blobs WHERE cid = $1",
"query": "SELECT cid, takedown_ref FROM blobs WHERE cid = $1 ORDER BY takedown_ref NULLS LAST LIMIT 1",
"describe": {
"columns": [
{
@@ -24,5 +24,5 @@
true
]
},
"hash": "62942bd21d545eb15bfea4f46378b6c2ebfe12b8bc9e27c63a6c0f77a9105303"
"hash": "5996484ff0f8dbc3b278cfd01b8375dbf7bf6da8d903145b12871dda6e1fd5d9"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key as \"storage_key!\" FROM blobs b\n WHERE created_by_user = $1\n AND NOT EXISTS (\n SELECT 1 FROM blobs o\n WHERE o.cid = b.cid AND o.created_by_user <> $1\n )",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "storage_key!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "8844d942ef2810afc386e5a9838624ee07a43c380d2df31efdba5cf299aab571"
}
@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT u.id as user_id, u.did\n FROM users u\n JOIN repos r ON r.user_id = u.id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "user_id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "did",
"type_info": "Text"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false
]
},
"hash": "8951136879711bca5b562c34f88e691a8ee16f370f6ef9b88ddb3873ddf2b45f"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (cid) DO NOTHING RETURNING cid",
"query": "INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (cid, created_by_user) DO NOTHING RETURNING cid",
"describe": {
"columns": [
{
@@ -22,5 +22,5 @@
false
]
},
"hash": "8afea2b745385348f4c78b51f74145d6718bfcf9a3a0c218109ec691aeb930ba"
"hash": "996e5513fb55670fe3304a6046381e377da6a187dfa3347bd285078a7b4410f2"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key FROM blobs WHERE cid = $1",
"query": "SELECT storage_key FROM blobs WHERE cid = $1 LIMIT 1",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
false
]
},
"hash": "6131bb5b39ca81bdbb193c0a9867bead8d9f3d793ad4eca97a79d166467a5052"
"hash": "9fb9e128076b20ff067d01955221488ce7e5b886dba0529fb073c3e0461fe030"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM blobs WHERE cid = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "d2990ce7f233d2489bb36a63920571c9f454a0605cc463829693d581bc0dce12"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)\n SELECT DISTINCT b.cid, b.mime_type, b.size_bytes, $1::uuid, b.storage_key\n FROM blobs b WHERE b.cid = $2\n ON CONFLICT (cid, created_by_user) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": []
},
"hash": "e248d71f595abf0207b01bc2f4e1f312d0c96b0f2f5131dfc13bfbb42a79d886"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key as \"storage_key!\" FROM blobs WHERE created_by_user = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "storage_key!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "f59010ecdd7f782489e0e03288a06dacd72b33d04c1e2b98475018ad25485852"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT rb.blob_cid, rb.record_uri\n FROM record_blobs rb\n LEFT JOIN blobs b ON rb.blob_cid = b.cid\n WHERE rb.repo_id = $1 AND b.cid IS NULL AND rb.blob_cid > $2\n ORDER BY rb.blob_cid\n LIMIT $3",
"query": "SELECT rb.blob_cid, rb.record_uri\n FROM record_blobs rb\n LEFT JOIN blobs b ON rb.blob_cid = b.cid AND b.created_by_user = $1\n WHERE rb.repo_id = $1 AND b.cid IS NULL AND rb.blob_cid > $2\n ORDER BY rb.blob_cid\n LIMIT $3",
"describe": {
"columns": [
{
@@ -26,5 +26,5 @@
false
]
},
"hash": "8e88ec169f0ef14c7207944cd4c2c0970e302c0457f9e317ec752dc13a5b1393"
"hash": "f8bb421e07e47f7b0a3b2789e368abfa6ad64152e8660e3efc7117b2d9320f22"
}
+55
View File
@@ -4,6 +4,7 @@
In order of importance:
- If your change involves how Tranquil implements atproto make sure its correct! See more below.
- **You must run your change! Every contribution that says "here's xyz. untested." does not help the project.**
- Relevant tests to your PR must pass. The whole suite doesn't have to be proven to have run, because there are a *ton* of tests and they're quite heavy, but hopefully there are existing tests for whatever you're PRing, and if there aren't, please add those too.
- Run cargo fmt :P
@@ -16,6 +17,60 @@ Things that would also be nice but aren't like, a pain in our side:
- Big changes should be stacked PRs that are broken up into digestible pieces. Those stacked PRs should hopefully be able to be merged individually if necessary.
### How we define a "correct" PDS implementation
The atproto specs are notoriously imprecise, ambiguious,
lacks specifications for large parts of the protocol and network (even including what implementing a PDS entails!)
and is generally none specific.
This is bad.
We won't waste time here describing all the ways in which that is problematic,
the important thing for Tranquil is that this means that "follows spec" is not sufficient to describe a "correct" PDS implementation.
Thus we need to come up with a description of "correct".
In order of importance the following rules describe what "correct" means for Tranquil:
- The specs take precedence.
If the spec *is* specific enough then follow it.
Even if the reference implementation doesn't.
- If the specs aren't sufficiently specific
rely on the reference implementation, potential supporting documents or discussions,
and/or community sentiment or common sense.
If the matter is still debated and/or PBCs opinion differs from community sentiment we generally side with the community.
- Examples here include what features and APIs to implement,
here we look at what the reference implementation implements
as well as https://github.com/bluesky-social/atproto/discussions/2350 as a supporting document.
Another example is whether `include` scopes are allowed to use a `*` `aud` parameter.
Discussion here has happened in https://github.com/bluesky-social/atproto/issues/4490.
PBC has voiced an opinion that this should be disallowed,
community sentiment seems to strongly lean to allowing it. Tranquil allows it.
- Please mark locations like this with a `// SPECAMB: ...` comment explaining the ambiguity
and what parts of the reference implementation and/or supporting documents have been used as reference.
- If the reference implementation has behaviour that is only ever relevant for the Bluesky application.
Implementions of such behaviour **must** be gated behind a `bsky-support` cargo feature of the implementing crate.
- Examples here include bluesky feedgen specific service proxying behaviour,
the `app.bsky.actor.getPreferences` and `app.bsky.actor.putPreferences` APIs,
and special handling of the `X-BSKY-TOPICS` HTTP header during service proxying.
- Please add a comment next to these implementations with an explanation of the behaviour.
- Most of these behaviours are required for proper functioning of the official Bluesky client, though not all.
If the behaviour isn't required for the official client consider not implementing it.
- One such behaviour that we have a *hard rule* to never implement is default proxying to a configured Bluesky appview
for `app.bsky.*` APIs and as fallback for `com.atproto.repo.getRecord`.
Many third-party Bluesky clients rely on this behaviour, the official client used to do the same but does not anymore.
Third-party clients breaking because they don't specify an `atproto-proxy` header is thus *not* a Tranquil bug but a bug in said clients.
- Bluesky is the only application that will ever recieve application specific behaviour like this.
It does so only because such a big section of atproto usage is Bluesky
and because Bluesky is the only application that can practically rely on application specific behaviour.
Application specific behaviour for other applications may still be added to Tranquil if such behaviour is a Tranquil feature,
for example for Tranquils rudimentary banned content moderation feature,
and not something said application relies on for proper functioning.
There is bound to be edge cases that these rules don't fully cover.
Here common sense, community sentiment, furthering the goals of atproto itself, and ultimately maintainer opinion take precedence over support for any individual application.
Even Bluesky.
The rules above are meant to capture Tranquils goals of being correct while being community oriented and avoiding as much "Bluesky-defaultism" as possible.
Tranquil is a community atproto PDS, *not* a company-led Bluesky (or other atproto app) PDS.
See also "Tranquil & the world" in docs/1_WELCOME_TO_TRANQUIL_PDS.md.
## Local Development
### Prerequisites
Generated
+71 -75
View File
@@ -105,6 +105,21 @@ version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "250f629c0161ad8107cf89319e990051fae62832fd343083bea452d93e2205fd"
[[package]]
name = "alloc-no-stdlib"
version = "2.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc7bb162ec39d46ab1ca8c77bf72e890535becd1751bb45f64c597edb4c8c6b3"
[[package]]
name = "alloc-stdlib"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0e76a019e91224d279006ff972f1e984179a6e9feb050adba6ce8274aef23195"
dependencies = [
"alloc-no-stdlib",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
@@ -1250,6 +1265,27 @@ dependencies = [
"cfg_aliases",
]
[[package]]
name = "brotli"
version = "8.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cc91aac060a7a1e25823bdccbfb6af1875b88f17c6daac97894eed8207166b3"
dependencies = [
"alloc-no-stdlib",
"alloc-stdlib",
"brotli-decompressor",
]
[[package]]
name = "brotli-decompressor"
version = "5.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a32acac15fe1967bc3986b2a6347dffc965602354ea6f450ad07e8bfd253583"
dependencies = [
"alloc-no-stdlib",
"alloc-stdlib",
]
[[package]]
name = "bs58"
version = "0.5.1"
@@ -2486,7 +2522,6 @@ checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
"zlib-rs",
]
[[package]]
@@ -7630,9 +7665,8 @@ dependencies = [
[[package]]
name = "tranquil-api"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"anyhow",
"axum",
"backon",
"base32",
@@ -7666,27 +7700,25 @@ dependencies = [
"tokio",
"tracing",
"tranquil-config",
"tranquil-db",
"tranquil-db-traits",
"tranquil-lexicon",
"tranquil-pds",
"tranquil-scopes",
"tranquil-signal",
"tranquil-types",
"urlencoding",
"uuid",
"webauthn-rs",
"zip",
]
[[package]]
name = "tranquil-auth"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"anyhow",
"base32",
"base64 0.22.1",
"bcrypt",
"brotli",
"chrono",
"hmac",
"k256",
@@ -7705,7 +7737,7 @@ dependencies = [
[[package]]
name = "tranquil-cache"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7720,7 +7752,7 @@ dependencies = [
[[package]]
name = "tranquil-comms"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7734,7 +7766,6 @@ dependencies = [
"rsa",
"secrecy",
"serde_json",
"sqlx",
"thiserror 2.0.18",
"tokio",
"tracing",
@@ -7746,15 +7777,16 @@ dependencies = [
[[package]]
name = "tranquil-config"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"confique",
"serde",
"tranquil-types",
]
[[package]]
name = "tranquil-crypto"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"aes-gcm",
"base64 0.22.1",
@@ -7770,7 +7802,7 @@ dependencies = [
[[package]]
name = "tranquil-db"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"chrono",
@@ -7787,7 +7819,7 @@ dependencies = [
[[package]]
name = "tranquil-db-traits"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7803,18 +7835,20 @@ dependencies = [
[[package]]
name = "tranquil-infra"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"bytes",
"futures",
"serde",
"serde_json",
"thiserror 2.0.18",
"tranquil-config",
"tranquil-types",
]
[[package]]
name = "tranquil-lexicon"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"chrono",
"futures",
@@ -7826,15 +7860,15 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"tracing",
"tranquil-infra",
"tranquil-types",
"unicode-segmentation",
"urlencoding",
"wiremock",
]
[[package]]
name = "tranquil-oauth"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"anyhow",
"axum",
@@ -7851,13 +7885,14 @@ dependencies = [
"sqlx",
"tokio",
"tracing",
"tranquil-infra",
"tranquil-types",
"uuid",
]
[[package]]
name = "tranquil-oauth-server"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"axum",
@@ -7882,6 +7917,7 @@ dependencies = [
"tranquil-config",
"tranquil-crypto",
"tranquil-db-traits",
"tranquil-infra",
"tranquil-pds",
"tranquil-scopes",
"tranquil-types",
@@ -7892,7 +7928,7 @@ dependencies = [
[[package]]
name = "tranquil-pds"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"aes-gcm",
"anyhow",
@@ -7904,7 +7940,6 @@ dependencies = [
"base32",
"base64 0.22.1",
"bcrypt",
"bs58",
"bytes",
"chrono",
"ciborium",
@@ -7920,7 +7955,6 @@ dependencies = [
"hmac",
"http 1.4.0",
"image",
"infer",
"ipld-core",
"iroh-car",
"jacquard-common",
@@ -7962,9 +7996,9 @@ dependencies = [
"tranquil-cache",
"tranquil-comms",
"tranquil-config",
"tranquil-crypto",
"tranquil-db",
"tranquil-db-traits",
"tranquil-infra",
"tranquil-lexicon",
"tranquil-oauth",
"tranquil-oauth-server",
@@ -7981,12 +8015,11 @@ dependencies = [
"webauthn-rs",
"webauthn-rs-proto",
"wiremock",
"zip",
]
[[package]]
name = "tranquil-repo"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"bytes",
"cid",
@@ -7998,7 +8031,7 @@ dependencies = [
[[package]]
name = "tranquil-ripple"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"backon",
@@ -8027,7 +8060,7 @@ dependencies = [
[[package]]
name = "tranquil-scopes"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"axum",
"futures",
@@ -8044,7 +8077,7 @@ dependencies = [
[[package]]
name = "tranquil-server"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"arc-swap",
"axum",
@@ -8081,7 +8114,7 @@ dependencies = [
[[package]]
name = "tranquil-signal"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"chrono",
@@ -8089,7 +8122,6 @@ dependencies = [
"futures",
"presage",
"rand 0.9.2",
"serde",
"serde_json",
"sqlx",
"tempfile",
@@ -8097,14 +8129,13 @@ dependencies = [
"tokio",
"tokio-util",
"tracing",
"tranquil-signal",
"url",
"uuid",
]
[[package]]
name = "tranquil-storage"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"aws-config",
@@ -8121,7 +8152,7 @@ dependencies = [
[[package]]
name = "tranquil-store"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"async-trait",
"bytes",
@@ -8161,7 +8192,6 @@ dependencies = [
"tranquil-db",
"tranquil-db-traits",
"tranquil-oauth",
"tranquil-repo",
"tranquil-store",
"tranquil-types",
"uuid",
@@ -8170,7 +8200,7 @@ dependencies = [
[[package]]
name = "tranquil-sync"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"anyhow",
"axum",
@@ -8192,17 +8222,21 @@ dependencies = [
[[package]]
name = "tranquil-types"
version = "0.6.5"
version = "0.6.7"
dependencies = [
"base64 0.22.1",
"chrono",
"cid",
"jacquard-common",
"rand 0.8.5",
"reqwest",
"serde",
"serde_json",
"sqlx",
"thiserror 2.0.18",
"tokio",
"tracing",
"url",
"uuid",
]
@@ -8255,12 +8289,6 @@ version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ea3136b675547379c4bd395ca6b938e5ad3c3d20fad76e7fe85f9e0d011419c"
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.19.0"
@@ -9462,20 +9490,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "zip"
version = "7.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c42e33efc22a0650c311c2ef19115ce232583abbe80850bc8b66509ebef02de0"
dependencies = [
"crc32fast",
"flate2",
"indexmap 2.13.0",
"memchr",
"typed-path",
"zopfli",
]
[[package]]
name = "zkcredential"
version = "0.1.0"
@@ -9524,30 +9538,12 @@ dependencies = [
"zkcredential",
]
[[package]]
name = "zlib-rs"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3be3d40e40a133f9c916ee3f9f4fa2d9d63435b5fbe1bfc6d9dae0aa0ada1513"
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[package]]
name = "zune-core"
version = "0.5.1"
+2 -3
View File
@@ -26,7 +26,7 @@ members = [
]
[workspace.package]
version = "0.6.5"
version = "0.6.7"
edition = "2024"
license = "AGPL-3.0-or-later"
@@ -47,7 +47,6 @@ tranquil-db = { path = "crates/tranquil-db" }
tranquil-ripple = { path = "crates/tranquil-ripple" }
tranquil-lexicon = { path = "crates/tranquil-lexicon" }
tranquil-pds = { path = "crates/tranquil-pds" }
tranquil-server = { path = "crates/tranquil-server" }
tranquil-sync = { path = "crates/tranquil-sync" }
tranquil-oauth-server = { path = "crates/tranquil-oauth-server" }
tranquil-api = { path = "crates/tranquil-api" }
@@ -138,11 +137,11 @@ tower-layer = "0.3"
tracing = "0.1"
tracing-subscriber = "0.3"
urlencoding = "2.1"
url = "2.5"
uuid = { version = "1.19", features = ["v4", "v5", "v7", "fast-rng", "serde"] }
webauthn-rs = { version = "0.5", features = ["danger-allow-state-serialisation", "danger-user-presence-only-security-keys", "conditional-ui"] }
webauthn-rs-proto = "0.5"
x509-parser = "0.18"
zip = { version = "7.0", default-features = false, features = ["deflate"] }
ciborium = "0.2"
ctor = "0.6"
+1
View File
@@ -79,6 +79,7 @@ We currently don't have a shared space to chat and organize Tranquil things, but
- [@oyster.cafe](https://tangled.org/did:plc:3fwecdnvtcscjnrx2p4n7alz)
- [@nel.pet](https://tangled.org/did:plc:h5wsnqetncv6lu2weom35lg2)
- [@jola.dev](https://tangled.org/did:plc:bvraa6gajy4tfr3eh2sisdkr)
### Amazing contributors
+2
View File
@@ -1,6 +1,7 @@
[server]
hostname = "pds.test"
allow_http_proxy = true
allow_private_fetch = true
invite_code_required = false
disable_rate_limiting = true
@@ -10,6 +11,7 @@ dir = "/app/frontend/public"
[database]
url = "postgres://postgres:postgres@db:5432/pds"
max_connections = 20
[storage]
path = "/var/lib/tranquil-pds/blobs"
+4 -4
View File
@@ -8,13 +8,10 @@ license.workspace = true
tranquil-pds = { workspace = true }
tranquil-types = { workspace = true }
tranquil-config = { workspace = true }
tranquil-db = { workspace = true }
tranquil-db-traits = { workspace = true }
tranquil-lexicon = { workspace = true, features = ["resolve"] }
tranquil-scopes = { workspace = true }
tranquil-signal = { workspace = true }
anyhow = { workspace = true }
axum = { workspace = true }
backon = { workspace = true }
base32 = { workspace = true }
@@ -50,4 +47,7 @@ tracing = { workspace = true }
urlencoding = { workspace = true }
uuid = { workspace = true }
webauthn-rs = { workspace = true }
zip = { workspace = true }
[features]
bsky = ["bsky-support"]
bsky-support = []
-3
View File
@@ -1,3 +0,0 @@
mod preferences;
pub use preferences::{get_preferences, put_preferences};
@@ -66,11 +66,11 @@ pub async fn update_account_handle(
{
return Err(ApiError::InvalidHandle(None));
}
let available_domains = tranquil_config::get().server.available_user_domain_list();
let handle = if !input_handle.contains('.') {
format!("{}.{}", input_handle, &available_domains[0])
} else {
let primary = tranquil_pds::handle::ServiceDomains::for_user_handles().primary();
let handle = if input_handle.contains('.') {
input_handle.to_string()
} else {
format!("{}.{}", input_handle, primary)
};
let old_handle = state.repos.user.get_handle_by_did(did).await.ok().flatten();
let user_id = state
-42
View File
@@ -1,10 +1,8 @@
use axum::{Json, extract::State};
use serde::{Deserialize, Serialize};
use tracing::{error, warn};
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{Admin, Auth};
use tranquil_pds::state::AppState;
use tranquil_types::CidLink;
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
@@ -183,46 +181,6 @@ pub async fn update_server_config(
}
if let Some(ref logo_cid) = req.logo_cid {
let old_logo_cid = state
.repos
.infra
.get_server_config("logo_cid")
.await
.ok()
.flatten();
let should_delete_old = match (&old_logo_cid, logo_cid.is_empty()) {
(Some(old), true) => Some(old.clone()),
(Some(old), false) if old != logo_cid => Some(old.clone()),
_ => None,
};
if let Some(old_cid_str) = should_delete_old {
match CidLink::new(old_cid_str) {
Ok(old_cid) => {
if let Ok(Some(storage_key)) = state
.repos
.infra
.get_blob_storage_key_by_cid(&old_cid)
.await
{
if let Err(e) = state.blob_store.delete(&storage_key).await {
error!("Failed to delete old logo blob from storage: {:?}", e);
}
if let Err(e) = state.repos.infra.delete_blob_by_cid(&old_cid).await {
error!("Failed to delete old logo blob record: {:?}", e);
}
}
}
Err(e) => {
warn!(
"Old logo CID in database is invalid, skipping cleanup: {:?}",
e
);
}
}
}
if logo_cid.is_empty() {
state
.repos
+14 -9
View File
@@ -12,8 +12,8 @@ use tranquil_pds::api::{
};
use tranquil_pds::auth::{Active, Auth};
use tranquil_pds::delegation::{
DelegationActionType, SCOPE_PRESETS, ValidatedDelegationScope, verify_can_add_controllers,
verify_can_control_accounts,
DelegationActionType, IdentityResolutionError, SCOPE_PRESETS, ValidatedDelegationScope,
verify_can_add_controllers, verify_can_control_accounts,
};
use tranquil_pds::rate_limit::{AccountCreationLimit, RateLimited};
use tranquil_pds::state::AppState;
@@ -65,16 +65,16 @@ pub async fn add_controller(
) -> Result<Json<SuccessResponse>, ApiError> {
let resolved = tranquil_pds::delegation::resolve_identity(&state, &input.controller_did)
.await
.map_err(|_| ApiError::ControllerNotFound)?;
.map_err(|e| match e {
IdentityResolutionError::PdsEndpoint(_) => ApiError::InvalidDelegation(
"Controller PDS endpoint isn't a usable https URL".into(),
),
IdentityResolutionError::DidResolution(_) => ApiError::ControllerNotFound,
})?;
if !resolved.is_local
&& let Some(ref pds_url) = resolved.pds_url
{
if !pds_url.starts_with("https://") {
return Err(ApiError::InvalidDelegation(
"Controller PDS must use HTTPS".into(),
));
}
match state
.cross_pds_oauth
.check_remote_is_delegated(pds_url, &input.controller_did)
@@ -477,7 +477,12 @@ pub async fn resolve_controller(
let resolved = tranquil_pds::delegation::resolve_identity(&state, &did)
.await
.map_err(|_| ApiError::ControllerNotFound)?;
.map_err(|e| match e {
IdentityResolutionError::PdsEndpoint(_) => ApiError::InvalidDelegation(
"Controller PDS endpoint isn't a usable https URL".into(),
),
IdentityResolutionError::DidResolution(_) => ApiError::ControllerNotFound,
})?;
Ok(Json(resolved))
}
+2 -7
View File
@@ -147,12 +147,7 @@ async fn try_reactivate_migration(
Json(CreateAccountOutput {
handle: handle.clone(),
did: did.clone(),
did_doc: state
.did_resolver
.fetch_did_document(did)
.await
.ok()
.map(|f| (*f).clone()),
did_doc: state.did_resolver.fetch_did_document(did).await.ok(),
access_jwt: access_meta.token,
refresh_jwt: refresh_meta.token,
verification_required,
@@ -568,7 +563,7 @@ pub async fn create_account(
Json(CreateAccountOutput {
handle: handle.clone(),
did,
did_doc: did_doc.map(|f| (*f).clone()),
did_doc,
access_jwt: session.access_jwt,
refresh_jwt: session.refresh_jwt,
verification_required: !is_migration,
+13 -26
View File
@@ -132,12 +132,9 @@ pub async fn well_known_did(State(state): State<AppState>, headers: HeaderMap) -
let host_header = get_header_str(&headers, http::header::HOST).unwrap_or(hostname);
let host_without_port = host_header.split(':').next().unwrap_or(host_header);
if host_without_port != hostname_without_port {
let is_subdomain = cfg
.server
.available_user_domain_list()
.into_iter()
.chain(std::iter::once(hostname_without_port.to_string()))
.any(|d| host_without_port.ends_with(&format!(".{}", d)));
let is_subdomain = tranquil_pds::handle::ServiceDomains::served()
.split_handle(host_without_port)
.is_some();
if is_subdomain {
return serve_handle_did_doc(&state, host_without_port, hostname).await;
}
@@ -582,26 +579,16 @@ pub async fn update_handle(
"Inappropriate language in handle".into(),
)));
}
let handle_domains = tranquil_config::get().server.user_handle_domain_list();
let matched_handle_domain = handle_domains
.iter()
.filter(|d| new_handle.ends_with(&format!(".{}", d)))
.max_by_key(|d| d.len())
.cloned();
let is_domain_itself = handle_domains.iter().any(|d| d == &new_handle);
let handle: Handle = if (!new_handle.contains('.') || matched_handle_domain.is_some())
&& !is_domain_itself
{
let (short_part, full_handle) = match &matched_handle_domain {
Some(domain) => {
let suffix = format!(".{}", domain);
let short = new_handle.strip_suffix(&suffix).unwrap_or(&new_handle);
(short.to_string(), new_handle.clone())
}
None => {
let primary = &handle_domains[0];
(new_handle.clone(), format!("{}.{}", new_handle, primary))
}
let handle_domains = tranquil_pds::handle::ServiceDomains::for_user_handles();
let split = handle_domains.split_handle(&new_handle);
let is_domain_itself = handle_domains.contains(&new_handle);
let handle: Handle = if (!new_handle.contains('.') || split.is_some()) && !is_domain_itself {
let (short_part, full_handle) = match split {
Some((_domain, short)) => (short.to_string(), new_handle.clone()),
None => (
new_handle.clone(),
format!("{}.{}", new_handle, handle_domains.primary()),
),
};
if full_handle == current_handle {
let handle: Handle = match full_handle.parse() {
+1 -16
View File
@@ -9,10 +9,7 @@ use tranquil_pds::api::ApiError;
use tranquil_pds::api::error::DbResultExt;
use tranquil_pds::auth::{Auth, Permissive};
use tranquil_pds::circuit_breaker::with_circuit_breaker;
use tranquil_pds::plc::{
PlcError, PlcService, create_update_op, missing_required_rotation_key, sign_operation,
signing_key_to_did_key,
};
use tranquil_pds::plc::{PlcError, PlcService, create_update_op, sign_operation};
use tranquil_pds::state::AppState;
#[derive(Debug, Deserialize)]
@@ -118,18 +115,6 @@ pub async fn sign_plc_operation(
}
})?;
let signing_did_key = signing_key_to_did_key(&signing_key);
if let Some(rotation_keys) = unsigned_op.get("rotationKeys").and_then(Value::as_array) {
let rotation_key_strs: Vec<&str> = rotation_keys.iter().filter_map(Value::as_str).collect();
if let Some(missing) = missing_required_rotation_key(
&rotation_key_strs,
&signing_did_key,
tranquil_config::get().secrets.plc_rotation_key.as_deref(),
) {
return Err(ApiError::InvalidRequest(missing.message().into()));
}
}
let signed_op = sign_operation(&unsigned_op, &signing_key).map_err(|e| {
error!("Failed to sign PLC operation: {:?}", e);
ApiError::InternalError(None)
+24 -13
View File
@@ -164,6 +164,13 @@ pub async fn resolve_signing_key(
}
}
#[cfg_attr(
not(feature = "bsky"),
expect(
unused_variables,
reason = "only the bsky block writes display_name into the default profile record"
)
)]
pub async fn sequence_new_account(
state: &AppState,
did: &Did,
@@ -205,20 +212,24 @@ pub async fn sequence_new_account(
{
tracing::warn!("Failed to sequence sync event for {}: {}", did, e);
}
let profile_record = serde_json::json!({
"$type": "app.bsky.actor.profile",
"displayName": display_name
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
state,
did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
// TODO: make this configurable and also deduplicate with tranquil-oauth-server/src/sso_endpoints.rs:1210
#[cfg(feature = "bsky")]
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
let profile_record = serde_json::json!({
"$type": "app.bsky.actor.profile",
"displayName": display_name
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
state,
did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
};
}
}
+42 -15
View File
@@ -1,5 +1,12 @@
pub mod actor;
// BSKY: Bluesky requires PDSs to implement its private preferences API
#[cfg(feature = "bsky-support")]
pub mod actor {
mod preferences;
pub use preferences::{get_preferences, put_preferences};
}
pub mod admin;
#[cfg(feature = "bsky")]
pub mod age_assurance;
pub mod common;
pub mod delegation;
@@ -21,7 +28,7 @@ pub fn api_routes() -> axum::Router<AppState> {
let blob_body_limit =
DefaultBodyLimit::max(tranquil_config::get().server.max_blob_size as usize);
axum::Router::new()
let router = axum::Router::new()
.route("/_health", get(server::health))
.route(
"/com.atproto.server.describeServer",
@@ -373,14 +380,6 @@ pub fn api_routes() -> axum::Router<AppState> {
post(admin::update_subject_status),
)
.route("/com.atproto.admin.sendEmail", post(admin::send_email))
.route(
"/app.bsky.actor.getPreferences",
get(actor::get_preferences),
)
.route(
"/app.bsky.actor.putPreferences",
post(actor::put_preferences),
)
.route(
"/com.atproto.temp.checkSignupQueue",
get(temp::check_signup_queue),
@@ -438,7 +437,21 @@ pub fn api_routes() -> axum::Router<AppState> {
.route(
"/_delegation.resolveController",
get(delegation::resolve_controller),
);
#[cfg(feature = "bsky-support")]
let router = router
.route(
"/app.bsky.actor.getPreferences",
get(actor::get_preferences),
)
.route(
"/app.bsky.actor.putPreferences",
post(actor::put_preferences),
);
#[cfg(feature = "bsky")]
let router = router
.route(
"/app.bsky.ageassurance.getState",
get(age_assurance::get_state),
@@ -446,15 +459,23 @@ pub fn api_routes() -> axum::Router<AppState> {
.route(
"/app.bsky.unspecced.getAgeAssuranceState",
get(age_assurance::get_age_assurance_state),
)
);
router
}
pub fn well_known_api_routes() -> axum::Router<AppState> {
use axum::routing::get;
axum::Router::new()
let routes = axum::Router::new()
.route("/did.json", get(identity::well_known_did))
.route("/atproto-did", get(identity::well_known_atproto_did))
.route("/atproto-did", get(identity::well_known_atproto_did));
if tranquil_config::get().server.enable_caddy_on_demand_tls {
routes.route("/caddy/ask", get(server::caddy_ask))
} else {
routes
}
}
pub fn webhook_routes() -> axum::Router<AppState> {
@@ -474,9 +495,15 @@ pub fn webhook_routes() -> axum::Router<AppState> {
pub fn misc_routes() -> axum::Router<AppState> {
use axum::routing::get;
axum::Router::new()
let router = axum::Router::new()
.route("/health", get(server::health))
.route("/robots.txt", get(server::robots_txt))
.route("/favicon.ico", get(server::get_logo))
.route("/u/{handle}/did.json", get(identity::user_did_doc))
.route("/u/{handle}/did.json", get(identity::user_did_doc));
if tranquil_config::get().server.rfc_moo_compliance {
router.route("/cow.txt", get(server::cow_txt))
} else {
router
}
}
+3 -2
View File
@@ -12,7 +12,7 @@ use tranquil_pds::api::ApiError;
use tranquil_pds::api::proxy_client::{is_ssrf_safe, proxy_client};
use tranquil_pds::auth::{AnyUser, Auth};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{Did, Nsid};
use tranquil_pds::types::{Did, DidRef, Nsid};
static CREATE_REPORT_NSID: LazyLock<Nsid> =
LazyLock::new(|| "com.atproto.moderation.createReport".parse().unwrap());
@@ -151,8 +151,9 @@ async fn proxy_to_report_service(
let service_token = match tranquil_pds::auth::create_service_token(
&auth_user.did,
service_did,
&DidRef::from(service_did),
Some(&CREATE_REPORT_NSID),
None,
&key_bytes,
) {
Ok(t) => t,
@@ -19,6 +19,7 @@ pub struct NotificationPrefsOutput {
pub telegram_verified: bool,
pub signal_username: Option<String>,
pub signal_verified: bool,
pub legacy_login_alerts: bool,
}
pub async fn get_notification_prefs(
@@ -32,6 +33,26 @@ pub async fn get_notification_prefs(
.await
.log_db_err("get notification prefs")?
.ok_or(ApiError::AccountNotFound)?;
let user_id = state
.repos
.user
.get_id_by_did(&auth.did)
.await
.log_db_err("get user by did")?
.ok_or(ApiError::AccountNotFound)?;
let legacy_login_alerts = state
.repos
.infra
.get_account_preferences(user_id)
.await
.log_db_err("get legacy login alert prefs")?
.iter()
.find(|(name, _)| name == "legacy_login_alerts")
.and_then(|(_, value)| value.as_bool())
.unwrap_or(true);
Ok(Json(NotificationPrefsOutput {
preferred_channel: prefs.preferred_channel,
email: prefs.email,
@@ -41,6 +62,7 @@ pub async fn get_notification_prefs(
telegram_verified: prefs.telegram_verified,
signal_username: prefs.signal_username,
signal_verified: prefs.signal_verified,
legacy_login_alerts,
}))
}
@@ -121,6 +143,7 @@ pub struct UpdateNotificationPrefsInput {
pub discord_username: Option<String>,
pub telegram_username: Option<String>,
pub signal_username: Option<String>,
pub legacy_login_alerts: Option<bool>,
}
#[derive(Serialize)]
@@ -435,6 +458,15 @@ pub async fn update_notification_prefs(
.await?;
}
if let Some(alerts) = input.legacy_login_alerts {
state
.repos
.infra
.upsert_account_preference(user_id, "legacy_login_alerts", json!(alerts))
.await
.log_db_err("update legacy login alert prefs")?;
}
Ok(Json(UpdateNotificationPrefsOutput {
success: true,
verification_required,
+9 -18
View File
@@ -148,7 +148,13 @@ pub async fn upload_blob(
size, cid_str
);
match state
if let Err(e) = state.blob_store.copy(&temp_key, &storage_key).await {
let _ = state.blob_store.delete(&temp_key).await;
error!("Failed to copy blob to final location: {:?}", e);
return Err(ApiError::InternalError(Some("Failed to store blob".into())));
}
if let Err(e) = state
.repos
.blob
.insert_blob(
@@ -160,24 +166,9 @@ pub async fn upload_blob(
)
.await
{
Ok(_) => {}
Err(e) => {
let _ = state.blob_store.delete(&temp_key).await;
error!("Failed to insert blob record: {:?}", e);
return Err(ApiError::InternalError(None));
}
};
if let Err(e) = state.blob_store.copy(&temp_key, &storage_key).await {
let _ = state.blob_store.delete(&temp_key).await;
if let Err(db_err) = state.repos.blob.delete_blob_by_cid(&cid_link).await {
error!(
"Failed to clean up orphaned blob record after copy failure: {:?}",
db_err
);
}
error!("Failed to copy blob to final location: {:?}", e);
return Err(ApiError::InternalError(Some("Failed to store blob".into())));
error!("Failed to insert blob record: {:?}", e);
return Err(ApiError::InternalError(None));
}
let _ = state.blob_store.delete(&temp_key).await;
@@ -1 +0,0 @@
pub use tranquil_pds::repo_ops::*;
@@ -10,7 +10,7 @@ use serde_json::Value;
use std::str::FromStr;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use tracing::{error, info, warn};
use tracing::{debug, error, info, warn};
use tranquil_pds::api::EmptyResponse;
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{Auth, NotTakendown, Permissive, require_legacy_session_mfa};
@@ -212,9 +212,10 @@ async fn assert_valid_did_document_for_service(
if let Some(ref expected_rotation_key) = server_rotation_key
&& !doc_rotation_keys.contains(&expected_rotation_key.as_str())
{
return Err(ApiError::InvalidRequest(
"Server rotation key not included in PLC DID data".into(),
));
debug!(
"DID {} rotation keys {:?} omit the PDS-managed server rotation key {}",
did, doc_rotation_keys, expected_rotation_key
);
}
let doc_signing_key = doc_data
@@ -251,13 +252,10 @@ async fn assert_valid_did_document_for_service(
}
if !doc_rotation_keys.contains(&expected_did_key.as_str()) {
warn!(
debug!(
"DID {} rotation keys {:?} omit the PDS-managed signing key {}",
did, doc_rotation_keys, expected_did_key
);
return Err(ApiError::InvalidRequest(
"PLC rotation keys omit the PDS-managed signing key required to sign operations for this identity".into(),
));
}
}
} else if let Some(host_and_path) = did.as_str().strip_prefix("did:web:") {
+43
View File
@@ -0,0 +1,43 @@
use axum::extract::{Query, State};
use axum::http::StatusCode;
use serde::de::Error as _;
use serde::{Deserialize, Deserializer};
use tracing::error;
use tranquil_pds::handle::ServiceDomains;
use tranquil_pds::state::AppState;
use tranquil_pds::types::Handle;
pub struct AskedDomain(Handle);
impl<'de> Deserialize<'de> for AskedDomain {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let raw = String::deserialize(deserializer)?;
let without_root_dot = raw.strip_suffix('.').unwrap_or(&raw);
Handle::new(without_root_dot)
.map(Self)
.map_err(D::Error::custom)
}
}
#[derive(Deserialize)]
pub struct CaddyAskQuery {
pub domain: AskedDomain,
}
pub async fn caddy_ask(
State(state): State<AppState>,
Query(ask): Query<CaddyAskQuery>,
) -> StatusCode {
let AskedDomain(handle) = ask.domain;
if ServiceDomains::served().contains(handle.as_str()) {
return StatusCode::OK;
}
match state.repos.user.get_by_handle(&handle).await {
Ok(Some(_)) => StatusCode::OK,
Ok(None) => StatusCode::NOT_FOUND,
Err(e) => {
error!("caddy ask couldn't look up handle {handle}: {e:?}");
StatusCode::INTERNAL_SERVER_ERROR
}
}
}
+57
View File
@@ -0,0 +1,57 @@
..........................
....*o|||||||8#@@@@@@@@@@@@@@@@@@@@@@@###&|o:_..
..*:o|||&8##@###8888888######@#@###########################|*...
.:o|||8#####8888|:::**. *&########################@@################&o_
.*o&8###@#8&o*_. :###@##############@########################@@##&o_
.*o8########& :##@#@##############@############################@###|_
.*o|8##########8o .#######################################################&o_
*&##|_ ..*&##8&o*|88888|_ _#######################################@##################|.
*#####& *&######&o_..*o|o:_ .&##o _###########################################################&_
_##8*##8 .|88|:::|#######8###8|*:_ .&#@@8 _##@@@########################################################&_
_#@8_##8_ *8#8|*_ _:|#####&&####8 .&##############################################################|
_#@8.|##8_ _::o###8&##8 .|##@############################8###########################@@#|_
*###o.|88o ..*&####|..##& _|##########################8|_ .|#############################8
*|###|_ ._&####8|*_ _*_ _::&8888888888888888|::*_ .|##@####@@@##################|
*&###|_ _:_ .&88###8|*_ ..... .|#####@@@##################8
.##@#& _##& .|##o _#@@#@#| .|#######&:_ _|###@####################8
.:8##8o _o:*&##| *##8_.&@@##@#| _::o8#8|::|#####|_ _|#################88###8
.&##&*_ *###o_###| .|##8*&##|*###o _###8####8|_ _:|###|_ .*o|||o:_ _:::&8888888888|_ _##8
.###|_. _###o *###|*&#######8 *##8 .##8_ _:|###|_ _|###|_ .&########o _##&
_|####8|&##8:_ _|#########88o .##8 *##& _|###o .|###o .#########| .o##o
o#8|*:#@@###o _:::*__*_ _##8_ _##8_ _&##|_ *##8_ *8#####8|_ .*oo:_ o##|
*###o.&#####& _oo* .8##& .8##8_ .|##& o##& _::::_.*o|8######|_ .##8.
_###&o&##8_:*_ .###& .###|_&#####| _##8 :###o *ooo&#########@#@#& ....:##&
.|8||###&. _**_ .###88##|*&###|*._&##& *|##8_ *o&####@@####@@######& .*o||||||&#######8_
*&###o _|88##8_ _:8######|*:###|_ _##################88|_ *&#################&
*#####o *&8o *##& _:::*_.&##|_ _#@##############8_ :##################8*
.###&##8_.|88o *&8o _@@& .###| .&####@#########|_ .####@###@@########8*
_##&.|###|_.... .|88o _##8* *&###|_ *###o _:&88######8|_ .*o|||##################o
_##8_ _|########|_ .*o8####&#@@#@##o *#@8 _*:*. .&###@###################|
.|##& _:::::&##& .*&##############@#8_ .###o _#@####################|_
.&##|_ .&##8_ *o&####################8_ *##& .&#####@@############8*
.|##8_.&###&####8_ _########################8**##& _##################|_
.&###&##888888##8_ .|88######@###########|*######o _|8###############|_
.&#####o *###|_ _::::::*o##8**o##8 .|###8o .&##@#############&*
.|####o _|###|_ _##8.*&##& _*_ .#################|
_*_ _|###|_.. .|#####8|_ *&#@#########8###&
*#######&|o:_... ..*:::*. ......._:o&8####888&o:#####8_
_###|&888#####@#####&|||o:_........................._:o||||8##@@@@####8|:*_ _:::*_
.|#@###o _:::o#@#888######@@@@@@@@@@@@@@@@@@@@@@@@#####888|::::::**_
_::*_ :##& *&8|_:::::::::::::::::::::::::**_
.###8||&##8o
_|888888|_
+10 -1
View File
@@ -29,6 +29,10 @@ pub async fn robots_txt() -> impl IntoResponse {
"# Hello!\n\n# Crawling the public API is allowed\nUser-agent: *\nAllow: /\n",
)
}
pub async fn cow_txt() -> &'static str {
include_str!("cow.txt")
}
pub fn is_self_hosted_did_web_enabled() -> bool {
tranquil_config::get().server.enable_pds_hosted_did_web
}
@@ -73,7 +77,12 @@ pub async fn describe_server(State(state): State<AppState>) -> Json<DescribeServ
let pds_hostname = &cfg.server.hostname;
Json(DescribeServerOutput {
available_user_domains: cfg.server.user_handle_domain_list(),
available_user_domains: match cfg.server.user_handle_domains.as_deref() {
Some(domains) if !domains.is_empty() => {
domains.iter().map(|d| d.as_str().to_owned()).collect()
}
_ => vec![cfg.server.hostname_without_port().to_owned()],
},
invite_code_required: cfg.server.invite_code_required,
did: format!("did:web:{}", pds_hostname),
links: DescribeServerLinks {
+3 -1
View File
@@ -1,5 +1,6 @@
pub mod account_status;
pub mod app_password;
pub mod caddy;
pub mod email;
pub mod invite;
pub mod logo;
@@ -22,13 +23,14 @@ pub use account_status::{
request_account_delete,
};
pub use app_password::{create_app_password, list_app_passwords, revoke_app_password};
pub use caddy::caddy_ask;
pub use email::{
authorize_email_update, check_channel_verified, check_email_in_use, check_email_update_status,
check_email_verified, confirm_email, request_email_update, update_email,
};
pub use invite::{create_invite_code, create_invite_codes, get_account_invite_codes};
pub use logo::get_logo;
pub use meta::{describe_server, health, robots_txt};
pub use meta::{cow_txt, describe_server, health, robots_txt};
pub use migration::{get_did_document, update_did_document};
pub use passkey_account::{
complete_passkey_setup, create_passkey_account, recover_passkey_account,
+15 -10
View File
@@ -11,7 +11,7 @@ use tracing::{error, info, warn};
use tranquil_pds::api::error::ApiError;
use tranquil_pds::auth::extractor::{Auth, Permissive};
use tranquil_pds::state::AppState;
use tranquil_pds::types::Did;
use tranquil_pds::types::DidRef;
use tranquil_types::Nsid;
static CREATE_ACCOUNT_NSID: LazyLock<Nsid> =
@@ -45,7 +45,7 @@ static PROTECTED_METHODS: LazyLock<HashSet<&'static str>> = LazyLock::new(|| {
#[derive(Deserialize)]
pub struct GetServiceAuthParams {
pub aud: Did,
pub aud: DidRef,
pub lxm: Option<Nsid>,
pub exp: Option<i64>,
}
@@ -169,14 +169,19 @@ pub async fn get_service_auth(
}
}
let service_token =
match tranquil_pds::auth::create_service_token(&auth.did, &params.aud, lxm, &key_bytes) {
Ok(t) => t,
Err(e) => {
error!("Failed to create service token: {:?}", e);
return ApiError::InternalError(None).into_response();
}
};
let service_token = match tranquil_pds::auth::create_service_token(
&auth.did,
&params.aud,
lxm,
params.exp,
&key_bytes,
) {
Ok(t) => t,
Err(e) => {
error!("Failed to create service token: {:?}", e);
return ApiError::InternalError(None).into_response();
}
};
(
StatusCode::OK,
Json(GetServiceAuthOutput {
+34 -20
View File
@@ -317,23 +317,37 @@ pub async fn create_session(
return Err(ApiError::InternalError(None));
}
if is_legacy_login && !used_totp_factor {
warn!(
did = %row.did,
ip = %client_ip,
"Legacy login on TOTP-enabled account - sending notification"
);
let hostname = &tranquil_config::get().server.hostname;
if let Err(e) = tranquil_pds::comms::comms_repo::enqueue_legacy_login(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
row.id,
hostname,
client_ip,
row.preferred_comms_channel,
)
.await
{
error!("Failed to queue legacy login notification: {:?}", e);
let alerts_enabled = match state.repos.infra.get_account_preferences(row.id).await {
Ok(prefs) => prefs
.iter()
.find(|(name, _)| name == "legacy_login_alerts")
.and_then(|(_, value)| value.as_bool())
.unwrap_or(true),
Err(e) => {
warn!("Failed to fetch legacy login alert preference: {:?}", e);
true
}
};
if alerts_enabled {
warn!(
did = %row.did,
ip = %client_ip,
"Legacy login on TOTP-enabled account - sending notification"
);
let hostname = &tranquil_config::get().server.hostname;
if let Err(e) = tranquil_pds::comms::comms_repo::enqueue_legacy_login(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
row.id,
hostname,
client_ip,
row.preferred_comms_channel,
)
.await
{
error!("Failed to queue legacy login notification: {:?}", e);
}
}
}
let handle = row.handle.clone();
@@ -351,7 +365,7 @@ pub async fn create_session(
refresh_jwt: refresh_meta.token,
handle,
did: row.did,
did_doc: did_doc.ok().map(|f| (*f).clone()),
did_doc: did_doc.ok(),
email: row.email,
email_confirmed: Some(row.channel_verification.email),
email_auth_factor: email_auth_factor_out,
@@ -444,7 +458,7 @@ pub async fn get_session(
status: account_state.status_for_session().map(String::from),
migrated_to_pds,
migrated_at,
did_doc: did_doc.ok().map(|f| (*f).clone()),
did_doc: did_doc.ok(),
}))
}
Ok(None) => Err(ApiError::AuthenticationFailed(None)),
@@ -800,7 +814,7 @@ async fn build_refresh_session_output(
preferred_locale: u.preferred_locale,
is_admin: u.is_admin,
active: account_state.is_active(),
did_doc: did_doc.ok().map(|f| (*f).clone()),
did_doc: did_doc.ok(),
status: account_state.status_for_session().map(String::from),
}))
}
+4 -6
View File
@@ -6,8 +6,8 @@ use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{
Active, Auth, decrypt_totp_secret, encrypt_totp_secret, generate_backup_codes,
generate_qr_png_base64, generate_totp_secret, generate_totp_uri, hash_backup_code,
is_backup_code_format, require_legacy_session_mfa, verify_backup_code, verify_password_mfa,
verify_totp_code, verify_totp_mfa,
is_backup_code_format, verify_backup_code, verify_password_mfa, verify_totp_code,
verify_totp_mfa,
};
use tranquil_pds::rate_limit::{TotpVerifyLimit, check_user_rate_limit_with_message};
use tranquil_pds::state::AppState;
@@ -163,11 +163,9 @@ pub async fn disable_totp(
auth: Auth<Active>,
Json(input): Json<DisableTotpInput>,
) -> Result<Json<EmptyResponse>, ApiError> {
let session_mfa = require_legacy_session_mfa(&state, &auth).await?;
let _rate_limit = check_user_rate_limit_with_message::<TotpVerifyLimit>(
&state,
session_mfa.did(),
auth.did.as_str(),
"Too many verification attempts. Please try again in a few minutes.",
)
.await?;
@@ -184,7 +182,7 @@ pub async fn disable_totp(
tranquil_pds::auth::legacy_2fa::clear_challenge(state.cache.as_ref(), &auth.did).await;
info!(did = %session_mfa.did(), "TOTP disabled (verified via {} and {})", password_mfa.method(), totp_mfa.method());
info!(did = %password_mfa.did(), "TOTP disabled (verified via {} and {})", password_mfa.method(), totp_mfa.method());
Ok(Json(EmptyResponse {}))
}
-31
View File
@@ -1,31 +0,0 @@
use axum::{
Json,
extract::State,
response::{IntoResponse, Response},
};
use serde::Deserialize;
use tranquil_pds::api::SuccessResponse;
use tranquil_pds::state::AppState;
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ConfirmChannelVerificationInput {
pub channel: tranquil_db_traits::CommsChannel,
pub identifier: String,
pub code: String,
}
pub async fn confirm_channel_verification(
State(state): State<AppState>,
Json(input): Json<ConfirmChannelVerificationInput>,
) -> Response {
let token_input = crate::server::VerifyTokenInput {
token: input.code,
identifier: input.identifier,
};
match crate::server::verify_token_internal(&state, token_input).await {
Ok(_output) => SuccessResponse::ok().into_response(),
Err(e) => e.into_response(),
}
}
+1
View File
@@ -24,3 +24,4 @@ subtle = { workspace = true }
totp-rs = { workspace = true }
urlencoding = { workspace = true }
uuid = { workspace = true }
brotli = "8.0.4"
+183
View File
@@ -0,0 +1,183 @@
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use brotli::{CompressorWriter, Decompressor};
use std::fmt;
use std::io::{Read, Write};
const COMPRESSED_PREFIX: &str = "$br$";
const QUALITY: u32 = 9;
const WINDOW_BITS: u32 = 16;
const BUFFER_SIZE: usize = 4096;
const MAX_SCOPE_LEN: u64 = 64 * 1024;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ScopeDecodeError {
Base64DecodeFailed,
DecompressFailed,
TooLarge,
}
impl fmt::Display for ScopeDecodeError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Base64DecodeFailed => write!(f, "Base64 decode of compressed scope failed"),
Self::DecompressFailed => write!(f, "Brotli decompression of scope failed"),
Self::TooLarge => write!(f, "Decompressed scope exceeds maximum length"),
}
}
}
impl std::error::Error for ScopeDecodeError {}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ScopeEncodeError {
TooLarge,
}
impl fmt::Display for ScopeEncodeError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::TooLarge => write!(f, "Scope exceeds maximum length"),
}
}
}
impl std::error::Error for ScopeEncodeError {}
fn brotli_compress(input: &str) -> Vec<u8> {
let mut writer = CompressorWriter::new(Vec::new(), BUFFER_SIZE, QUALITY, WINDOW_BITS);
writer
.write_all(input.as_bytes())
.expect("writing to a Vec cannot fail");
writer.into_inner()
}
fn brotli_decompress(input: &[u8]) -> Result<String, ScopeDecodeError> {
let mut output = String::new();
Decompressor::new(input, BUFFER_SIZE)
.take(MAX_SCOPE_LEN + 1)
.read_to_string(&mut output)
.map_err(|_| ScopeDecodeError::DecompressFailed)?;
if output.len() as u64 > MAX_SCOPE_LEN {
return Err(ScopeDecodeError::TooLarge);
}
Ok(output)
}
pub fn encode_scope(scope: &str) -> Result<String, ScopeEncodeError> {
if scope.len() as u64 > MAX_SCOPE_LEN {
return Err(ScopeEncodeError::TooLarge);
}
let tagged = format!(
"{COMPRESSED_PREFIX}{}",
URL_SAFE_NO_PAD.encode(brotli_compress(scope))
);
if tagged.len() < scope.len() || scope.starts_with(COMPRESSED_PREFIX) {
Ok(tagged)
} else {
Ok(scope.to_owned())
}
}
pub fn decode_scope(scope: &str) -> Result<String, ScopeDecodeError> {
let Some(encoded) = scope.strip_prefix(COMPRESSED_PREFIX) else {
return Ok(scope.to_owned());
};
let compressed = URL_SAFE_NO_PAD
.decode(encoded)
.map_err(|_| ScopeDecodeError::Base64DecodeFailed)?;
brotli_decompress(&compressed)
}
#[cfg(test)]
mod tests {
use super::*;
fn long_scope() -> String {
let mut scope = String::from("transition:generic transition:chat.bsky");
for collection in [
"social.colibri.message",
"social.colibri.community",
"social.colibri.reaction",
"social.colibri.member",
"social.colibri.channel.read",
] {
scope.push_str(&format!(" repo:{collection}?action=create&action=delete"));
}
scope
}
#[test]
fn long_scope_roundtrips_through_compression() {
let scope = long_scope();
let encoded = encode_scope(&scope).unwrap();
assert!(encoded.starts_with(COMPRESSED_PREFIX));
assert!(encoded.len() < scope.len());
assert_eq!(decode_scope(&encoded).unwrap(), scope);
}
#[test]
fn short_scope_stays_plaintext() {
let encoded = encode_scope("com.atproto.access").unwrap();
assert_eq!(encoded, "com.atproto.access");
assert_eq!(decode_scope(&encoded).unwrap(), "com.atproto.access");
}
#[test]
fn untagged_scope_passes_through() {
assert_eq!(
decode_scope("com.atproto.refresh").unwrap(),
"com.atproto.refresh"
);
assert_eq!(decode_scope("").unwrap(), "");
}
#[test]
fn malformed_compressed_scope_errors_instead_of_panicking() {
assert_eq!(
decode_scope("$br$not valid base64!"),
Err(ScopeDecodeError::Base64DecodeFailed)
);
assert_eq!(
decode_scope("$br$AAAAAAAAAAAAAAAA"),
Err(ScopeDecodeError::DecompressFailed)
);
}
#[test]
fn compression_bomb_is_rejected() {
let bomb = URL_SAFE_NO_PAD.encode(brotli_compress(&"a".repeat(MAX_SCOPE_LEN as usize * 2)));
assert_eq!(
decode_scope(&format!("{COMPRESSED_PREFIX}{bomb}")),
Err(ScopeDecodeError::TooLarge)
);
}
#[test]
fn plaintext_that_looks_compressed_roundtrips() {
let scope = "$br$repo:*";
let encoded = encode_scope(scope).unwrap();
assert!(encoded.starts_with(COMPRESSED_PREFIX));
assert_eq!(decode_scope(&encoded).unwrap(), scope);
}
#[test]
fn encode_rejects_oversized_scope() {
let oversized = "a".repeat(MAX_SCOPE_LEN as usize + 1);
assert_eq!(encode_scope(&oversized), Err(ScopeEncodeError::TooLarge));
assert!(encode_scope(&"a".repeat(MAX_SCOPE_LEN as usize)).is_ok());
}
}
+3
View File
@@ -1,3 +1,4 @@
mod compress;
mod token;
mod totp;
mod types;
@@ -12,6 +13,8 @@ pub use token::{
create_service_token_hs256,
};
pub use compress::{ScopeDecodeError, ScopeEncodeError, decode_scope, encode_scope};
pub use totp::{
TotpError, decrypt_totp_secret, encrypt_totp_secret, generate_backup_codes,
generate_qr_png_base64, generate_totp_secret, generate_totp_uri, hash_backup_code,
+15 -9
View File
@@ -1,14 +1,16 @@
use crate::compress::encode_scope;
use super::types::{
ActClaim, Claims, Header, SigningAlgorithm, TokenScope, TokenType, TokenWithMetadata,
};
use anyhow::Result;
use anyhow::{Context, Result};
use base64::Engine as _;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use chrono::{DateTime, Duration, Utc};
use hmac::{Hmac, Mac};
use k256::ecdsa::{Signature, SigningKey, signature::Signer};
use sha2::Sha256;
use tranquil_types::{Did, Jti, Nsid};
use tranquil_types::{Did, DidRef, Jti, Nsid};
type HmacSha256 = Hmac<Sha256>;
@@ -125,16 +127,20 @@ pub fn create_refresh_token_with_jti(
pub fn create_service_token(
did: &Did,
aud: &Did,
aud: &DidRef,
lxm: Option<&Nsid>,
exp: Option<i64>,
key_bytes: &[u8],
) -> Result<String> {
let signing_key = SigningKey::from_slice(key_bytes)?;
let expiration = Utc::now()
.checked_add_signed(Duration::seconds(60))
.expect("valid timestamp")
.timestamp();
let expiration = match exp {
Some(exp) => exp,
None => Utc::now()
.checked_add_signed(Duration::seconds(60))
.expect("valid timestamp")
.timestamp(),
};
let claims = Claims {
iss: did.clone(),
@@ -205,7 +211,7 @@ fn create_signed_token_pinned(
aud: format!("did:web:{}", aud_hostname),
exp: expiration,
iat: Utc::now().timestamp(),
scope: Some(scope.to_string()),
scope: Some(encode_scope(scope).context("Scope too large to encode")?),
lxm: None,
jti: jti.clone(),
act,
@@ -328,7 +334,7 @@ fn create_hs256_token_with_metadata(
),
exp: expiration,
iat: Utc::now().timestamp(),
scope: Some(scope.to_string()),
scope: Some(encode_scope(scope).context("Scope too large to encode")?),
lxm: None,
jti: jti.clone(),
act: None,
+14 -2
View File
@@ -1,3 +1,5 @@
use crate::compress::decode_scope;
use super::types::{
Claims, Header, SigningAlgorithm, TokenData, TokenDecodeError, TokenScope, TokenType,
TokenVerifyError, UnsafeClaims,
@@ -164,9 +166,15 @@ pub fn verify_token_es256k(
.decode(claims_b64)
.map_err(|_| TokenVerifyError::Invalid("Base64 decode of claims failed"))?;
let claims: Claims = serde_json::from_slice(&claims_bytes)
let mut claims: Claims = serde_json::from_slice(&claims_bytes)
.map_err(|_| TokenVerifyError::Invalid("JSON decode of claims failed"))?;
if let Some(scope) = &claims.scope {
claims.scope = Some(
decode_scope(scope).map_err(|_| TokenVerifyError::Invalid("Invalid token scope"))?,
);
}
let now = Utc::now().timestamp();
if claims.exp < now {
return Err(TokenVerifyError::Expired);
@@ -244,9 +252,13 @@ fn verify_token_hs256_internal(
.decode(claims_b64)
.context("Base64 decode of claims failed")?;
let claims: Claims =
let mut claims: Claims =
serde_json::from_slice(&claims_bytes).context("JSON decode of claims failed")?;
if let Some(scope) = &claims.scope {
claims.scope = Some(decode_scope(scope).context("Invalid scope claim encoding")?);
}
let now = Utc::now().timestamp();
if claims.exp < now {
return Err(anyhow!("Token expired"));
+1 -2
View File
@@ -5,12 +5,11 @@ edition.workspace = true
license.workspace = true
[features]
default = []
valkey = ["dep:redis"]
[dependencies]
tranquil-config = { workspace = true }
tranquil-infra = { workspace = true }
tranquil-infra = { workspace = true, features = ["cache-keys"] }
tranquil-ripple = { workspace = true }
async-trait = { workspace = true }
+4 -3
View File
@@ -1,4 +1,6 @@
pub use tranquil_infra::{Cache, CacheError, DistributedRateLimiter};
pub use tranquil_infra::{
Cache, CacheError, DistributedRateLimiter, cache_keys, cached_json, read_json, write_json,
};
use async_trait::async_trait;
use std::sync::Arc;
@@ -173,11 +175,10 @@ pub async fn create_cache(
) -> Result<(Arc<dyn Cache>, Arc<dyn DistributedRateLimiter>), CacheInitError> {
let cache_cfg = tranquil_config::try_get().map(|c| &c.cache);
let backend = cache_cfg.map(|c| c.backend.as_str()).unwrap_or("ripple");
let valkey_url = cache_cfg.and_then(|c| c.valkey_url.as_deref());
#[cfg(feature = "valkey")]
if backend == "valkey" {
if let Some(url) = valkey_url {
if let Some(url) = cache_cfg.and_then(|c| c.valkey_url.as_deref()) {
match ValkeyCache::new(url).await {
Ok(cache) => {
tracing::info!("using valkey cache at {url}");
-1
View File
@@ -19,7 +19,6 @@ reqwest = { workspace = true }
rsa = { workspace = true }
secrecy = { workspace = true }
serde_json = { workspace = true }
sqlx = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
+126 -9
View File
@@ -1,25 +1,42 @@
use lettre::Message;
use lettre::message::Mailbox;
use lettre::message::header::ContentType;
use lettre::message::header::{ContentType, MIME_VERSION_1_0};
use lettre::message::header::{Header, HeaderName, HeaderValue};
use uuid::Uuid;
use super::types::EmailDomain;
use crate::sender::SendError;
use crate::types::QueuedComms;
use crate::types::{CommsType, QueuedComms};
pub(super) fn build(from: &Mailbox, qc: &QueuedComms) -> Result<Message, SendError> {
pub(super) fn build(
from: &Mailbox,
qc: &QueuedComms,
apply_atmos_categories: bool,
) -> Result<Message, SendError> {
let to: Mailbox = qc
.recipient
.parse()
.map_err(|e: lettre::address::AddressError| SendError::InvalidRecipient(e.to_string()))?;
let subject = qc.subject.as_deref().unwrap_or("Notification");
let message_id = format!("<{}@{}>", Uuid::new_v4(), from.email.domain());
Message::builder()
let builder = Message::builder()
.from(from.clone())
.to(to)
.subject(subject)
.message_id(Some(message_id))
.header(ContentType::TEXT_PLAIN)
.header(MIME_VERSION_1_0)
.header(ContentType::TEXT_PLAIN);
let category = apply_atmos_categories
.then(|| atmos_category(qc.comms_type))
.flatten();
let builder = match category {
Some(category) => builder.header(category),
None => builder,
};
builder
.body(qc.body.clone())
.map_err(|e| SendError::MessageBuild(e.to_string()))
}
@@ -34,10 +51,57 @@ pub(super) fn recipient_domain(message: &Message) -> Result<EmailDomain, SendErr
.map_err(|e| SendError::InvalidRecipient(format!("invalid recipient domain: {e}")))
}
// for use with comail.at
#[derive(Debug, Clone, PartialEq, Eq, Copy)]
enum AtmosCategory {
PasswordReset,
MfaOtp,
Verification,
}
impl AtmosCategory {
fn as_str(self) -> &'static str {
match self {
Self::PasswordReset => "password-reset",
Self::MfaOtp => "mfa-otp",
Self::Verification => "verification",
}
}
}
impl Header for AtmosCategory {
fn name() -> HeaderName {
HeaderName::new_from_ascii_str("X-Atmos-Category")
}
fn parse(_s: &str) -> Result<Self, Box<dyn std::error::Error + Send + Sync>> {
//since we're never receiving email, we don't care about parsing
Err("X-Atmos-Category is write-only".into())
}
fn display(&self) -> HeaderValue {
HeaderValue::new(Self::name(), self.as_str().to_string())
}
}
fn atmos_category(comms_type: CommsType) -> Option<AtmosCategory> {
use CommsType::*;
match comms_type {
EmailVerification
| ChannelVerification
| ChannelVerified
| MigrationVerification
| LegacyLoginAlert
| EmailUpdate
| PlcOperation
| AccountDeletion
| Welcome => Some(AtmosCategory::Verification),
PasswordReset | PasskeyRecovery => Some(AtmosCategory::PasswordReset),
TwoFactorCode => Some(AtmosCategory::MfaOtp),
AdminEmail => None,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::{CommsChannel, CommsStatus, CommsType};
use crate::types::{CommsChannel, CommsStatus};
use chrono::Utc;
use uuid::Uuid;
@@ -71,6 +135,7 @@ mod tests {
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("Welcome"), "Hello world."),
false,
)
.unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
@@ -78,6 +143,7 @@ mod tests {
assert!(raw.contains("From: \"Test Sender\" <noreply@nel.pet>"));
assert!(raw.contains("To: user@nel.pet"));
assert!(raw.contains("Subject: Welcome"));
assert!(raw.contains("MIME-Version: 1.0"));
assert!(lower.contains("content-type: text/plain"));
assert!(raw.contains("Hello world."));
}
@@ -87,6 +153,7 @@ mod tests {
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("héllo wörld"), "Body"),
false,
)
.unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
@@ -99,6 +166,7 @@ mod tests {
let result = build(
&from_mailbox(),
&fixture("x@nel.pet\r\nBcc: evil@x", Some("s"), "b"),
false,
);
assert!(matches!(result, Err(SendError::InvalidRecipient(_))));
}
@@ -108,6 +176,7 @@ mod tests {
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("hi\r\nBcc: evil@nel.pet"), "body"),
false,
)
.expect("subject CRLF should be encoded, not rejected");
let raw = String::from_utf8(msg.formatted()).unwrap();
@@ -123,7 +192,12 @@ mod tests {
#[test]
fn message_id_uses_from_domain() {
let msg = build(&from_mailbox(), &fixture("user@nel.pet", Some("s"), "b")).unwrap();
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("s"), "b"),
false,
)
.unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
let line = raw
.lines()
@@ -137,15 +211,58 @@ mod tests {
#[test]
fn missing_subject_uses_default() {
let msg = build(&from_mailbox(), &fixture("user@nel.pet", None, "Body")).unwrap();
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", None, "Body"),
false,
)
.unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(raw.contains("Subject: Notification"));
}
#[test]
fn recipient_domain_extracted() {
let msg = build(&from_mailbox(), &fixture("user@Nel.PET", Some("s"), "b")).unwrap();
let msg = build(
&from_mailbox(),
&fixture("user@Nel.PET", Some("s"), "b"),
false,
)
.unwrap();
let d = recipient_domain(&msg).unwrap();
assert_eq!(d.as_str(), "nel.pet");
}
#[test]
fn atmos_category_header_present_when_enabled_and_mapped() {
let qc = QueuedComms {
comms_type: CommsType::PasswordReset,
..fixture("user@nel.pet", Some("s"), "b")
};
let msg = build(&from_mailbox(), &qc, true).unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(raw.contains("X-Atmos-Category: password-reset"));
}
#[test]
fn atmos_category_header_absent_when_disabled() {
let qc = QueuedComms {
comms_type: CommsType::PasswordReset,
..fixture("user@nel.pet", Some("s"), "b")
};
let msg = build(&from_mailbox(), &qc, false).unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(!raw.contains("X-Atmos-Category"));
}
#[test]
fn atmos_category_header_absent_when_unmapped() {
let qc = QueuedComms {
comms_type: CommsType::AdminEmail,
..fixture("user@nel.pet", Some("s"), "b")
};
let msg = build(&from_mailbox(), &qc, true).unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(!raw.contains("X-Atmos-Category"));
}
}
+55 -1
View File
@@ -124,6 +124,7 @@ fn build_smarthost(
Ok(SendMode::Smarthost {
transport: Box::new(builder.build()),
total_timeout,
apply_atmos_categories: cfg.email.smarthost.apply_atmos_categories,
})
}
@@ -176,6 +177,16 @@ fn build_dkim(cfg: &tranquil_config::DkimConfig) -> Result<Option<DkimSigner>, S
DkimSigner::load(selector, domain, path).map(Some)
}
fn wants_atmos_categories(mode: &SendMode) -> bool {
match mode {
SendMode::Smarthost {
apply_atmos_categories,
..
} => *apply_atmos_categories,
SendMode::DirectMx { .. } => false,
}
}
#[async_trait]
impl CommsSender for EmailSender {
fn channel(&self) -> CommsChannel {
@@ -183,7 +194,8 @@ impl CommsSender for EmailSender {
}
async fn send(&self, notification: &QueuedComms) -> Result<(), SendError> {
let mut message = message::build(&self.from, notification)?;
let mut message =
message::build(&self.from, notification, wants_atmos_categories(&self.mode))?;
if let Some(signer) = &self.dkim {
signer.sign(&mut message);
}
@@ -196,3 +208,45 @@ impl CommsSender for EmailSender {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use lettre::Tokio1Executor;
use std::time::Duration;
fn dummy_smarthost(apply_atmos_categories: bool) -> SendMode {
let transport =
AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous("localhost").build();
SendMode::Smarthost {
transport: Box::new(transport),
total_timeout: Duration::from_secs(10),
apply_atmos_categories,
}
}
fn dummy_direct_mx() -> SendMode {
SendMode::DirectMx {
resolver: Arc::new(TokioAsyncResolver::tokio(
ResolverConfig::default(),
ResolverOpts::default(),
)),
helo: HeloName::parse("mta.nel.pet").unwrap(),
command_timeout: Duration::from_secs(5),
total_timeout: Duration::from_secs(10),
require_tls: false,
inflight: Arc::new(Semaphore::new(1)),
}
}
#[tokio::test]
async fn smarthost_reflects_its_own_flag() {
assert!(wants_atmos_categories(&dummy_smarthost(true)));
assert!(!wants_atmos_categories(&dummy_smarthost(false)));
}
#[test]
fn direct_mx_never_wants_atmos_categories() {
assert!(!wants_atmos_categories(&dummy_direct_mx()));
}
}
+11 -2
View File
@@ -19,6 +19,7 @@ pub enum SendMode {
Smarthost {
transport: Box<AsyncSmtpTransport<Tokio1Executor>>,
total_timeout: Duration,
apply_atmos_categories: bool,
},
DirectMx {
resolver: Arc<TokioAsyncResolver>,
@@ -33,8 +34,15 @@ pub enum SendMode {
impl std::fmt::Debug for SendMode {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Smarthost { total_timeout, .. } => {
write!(f, "SendMode::Smarthost(total_timeout={total_timeout:?})")
Self::Smarthost {
total_timeout,
apply_atmos_categories,
..
} => {
write!(
f,
"SendMode::Smarthost(total_timeout={total_timeout:?}, apply_atmos_categories={apply_atmos_categories:?})"
)
}
Self::DirectMx {
helo, require_tls, ..
@@ -52,6 +60,7 @@ pub async fn dispatch(mode: &SendMode, message: Message) -> Result<(), SendError
SendMode::Smarthost {
transport,
total_timeout,
..
} => with_total_timeout(*total_timeout, run_send(transport, message)).await,
SendMode::DirectMx {
resolver,
@@ -53,6 +53,7 @@ fn build_smarthost_sender_with_total_timeout(
SendMode::Smarthost {
transport: Box::new(transport),
total_timeout,
apply_atmos_categories: false,
},
None,
)
+2 -1
View File
@@ -5,5 +5,6 @@ edition.workspace = true
license.workspace = true
[dependencies]
confique = { workspace = true }
serde = { workspace = true }
tranquil-types = { workspace = true }
confique = { workspace = true }
+40 -22
View File
@@ -2,6 +2,7 @@ use confique::Config;
use std::fmt;
use std::path::PathBuf;
use std::sync::OnceLock;
use tranquil_types::Domain;
static CONFIG: OnceLock<TranquilConfig> = OnceLock::new();
@@ -30,7 +31,6 @@ impl fmt::Display for ConfigError {
}
impl std::error::Error for ConfigError {}
/// Initialize the global configuration. Must be called once at startup before
/// any other code accesses the configuration. Panics if called more than once.
pub fn init(config: TranquilConfig) {
@@ -224,6 +224,12 @@ impl TranquilConfig {
}
}
if let Err(e) = Domain::new(self.server.hostname_without_port()) {
errors.push(format!(
"server.hostname (PDS_HOSTNAME) must be a plain domain, {e}"
));
}
// -- email -----------------------------------------------------------
self.email
.validate(self.server.hostname_without_port(), &mut errors);
@@ -428,7 +434,7 @@ pub struct ServerConfig {
pub hostname: String,
/// Address to bind the HTTP server to.
#[config(env = "SERVER_HOST", default = "127.0.0.1")]
#[config(env = "SERVER_HOST", default = "[::1]")]
pub host: String,
/// Port to bind the HTTP server to.
@@ -438,13 +444,25 @@ pub struct ServerConfig {
/// List of domains for user handles.
/// Defaults to the PDS hostname when not set.
#[config(env = "PDS_USER_HANDLE_DOMAINS", parse_env = split_comma_list)]
pub user_handle_domains: Option<Vec<String>>,
pub user_handle_domains: Option<Vec<Domain>>,
/// Enable PDS-hosted did:web identities. Hosting did:web requires a
/// long-term commitment to serve DID documents; opt-in only.
#[config(env = "ENABLE_PDS_HOSTED_DID_WEB", default = false)]
pub enable_pds_hosted_did_web: bool,
/// The caddy on-demand TLS requires we serve
/// the endpoint `/.well-known/caddy/ask`.
/// It will be used so that caddy can create TLS
/// certs for us on the fly
/// and we don't have to do annoying wildcard certs.
#[config(env = "ENABLE_CADDY_ON_DEMAND_TLS", default = true)]
pub enable_caddy_on_demand_tls: bool,
/// iykyk!
#[config(env = "RFC_MOO_COMPLIANCE", default = false)]
pub rfc_moo_compliance: bool,
/// When set to true, skip age-assurance birthday prompt for all accounts.
#[config(env = "PDS_AGE_ASSURANCE_OVERRIDE", default = false)]
pub age_assurance_override: bool,
@@ -461,6 +479,10 @@ pub struct ServerConfig {
#[config(env = "DISABLE_RATE_LIMITING", default = false)]
pub disable_rate_limiting: bool,
/// Allow outbound fetches to private network addresses. Useful for local development using docker compose.
#[config(env = "ALLOW_PRIVATE_FETCH", default = false)]
pub allow_private_fetch: bool,
/// Skip the verified-comms-channel gate for login and record writes.
/// Please keep this off unless you're an invite-only PDS!
#[config(env = "DISABLE_ACCOUNT_VERIFICATION_GATE", default = false)]
@@ -565,20 +587,6 @@ impl ServerConfig {
pub fn banned_word_list(&self) -> Vec<String> {
self.banned_words.clone().unwrap_or_default()
}
/// Returns the user handle domains, falling back to `[hostname_without_port]`.
pub fn user_handle_domain_list(&self) -> Vec<String> {
self.user_handle_domains
.as_deref()
.filter(|v| !v.is_empty())
.map(|v| v.to_vec())
.unwrap_or_else(|| vec![self.hostname_without_port().to_string()])
}
/// Alias for `user_handle_domain_list` (for callers that were using the now-removed `available_user_domains` field).
pub fn available_user_domain_list(&self) -> Vec<String> {
self.user_handle_domain_list()
}
}
#[derive(Debug, Config)]
@@ -762,6 +770,10 @@ pub struct StorageConfig {
#[config(env = "S3_ENDPOINT")]
pub s3_endpoint: Option<String>,
/// Path on the storage for the S3 blob backend.
#[config(env = "S3_PATH", default = "")]
pub s3_path: String,
/// Repository backend: `postgres` by default, or `tranquil-store`, our embedded db.
/// tranquil-store is EXPERIMENTAL!!!! RISK OF TOTAL DATA LOSS.
#[config(env = "REPO_BACKEND", default = "postgres")]
@@ -831,7 +843,7 @@ pub struct PlcConfig {
#[config(env = "PLC_CONNECT_TIMEOUT_SECS", default = 5)]
pub connect_timeout_secs: u64,
/// Seconds to cache DID documents in memory.
/// Seconds to cache DID documents.
#[config(env = "DID_CACHE_TTL_SECS", default = 300)]
pub did_cache_ttl_secs: u64,
}
@@ -1120,6 +1132,10 @@ pub struct SmarthostConfig {
/// stuck relay cannot stall the comms queue.
#[config(env = "MAIL_SMARTHOST_TOTAL_TIMEOUT_SECS", default = 60)]
pub total_timeout_secs: u64,
/// Apply Atmos/Comail.at categories for headers to be categorized appropriately.
#[config(env = "MAIL_APPLY_ATMOS_CATEGORIES", default = false)]
pub apply_atmos_categories: bool,
}
#[derive(Debug, Config)]
@@ -1472,12 +1488,13 @@ pub struct ImportConfig {
/// trimming whitespace and dropping empty entries.
///
/// Signature matches confique's `parse_env` expectation: `fn(&str) -> Result<T, E>`.
fn split_comma_list(value: &str) -> Result<Vec<String>, std::convert::Infallible> {
Ok(value
fn split_comma_list<T: std::str::FromStr>(value: &str) -> Result<Vec<T>, T::Err> {
value
.split(',')
.map(|item| item.trim().to_string())
.map(str::trim)
.filter(|item| !item.is_empty())
.collect())
.map(T::from_str)
.collect()
}
#[derive(Debug, Config)]
@@ -1981,6 +1998,7 @@ port = 587
pool_size: 4,
command_timeout_secs: 30,
total_timeout_secs: 60,
apply_atmos_categories: false,
},
direct_mx: DirectMxConfig {
command_timeout_secs: 30,
+2 -2
View File
@@ -70,12 +70,12 @@ pub trait BlobRepository: Send + Sync {
takedown_ref: Option<&str>,
) -> Result<bool, DbError>;
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<bool, DbError>;
async fn delete_blobs_by_user(&self, user_id: Uuid) -> Result<u64, DbError>;
async fn get_blob_storage_keys_by_user(&self, user_id: Uuid) -> Result<Vec<String>, DbError>;
async fn ensure_blob_ownership(&self, user_id: Uuid, cid: &CidLink) -> Result<bool, DbError>;
async fn insert_record_blobs(
&self,
repo_id: Uuid,
+1 -5
View File
@@ -1,7 +1,7 @@
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use tranquil_types::{CidLink, Did, Handle, InviteCode};
use tranquil_types::{Did, Handle, InviteCode};
use uuid::Uuid;
use crate::DbError;
@@ -417,10 +417,6 @@ pub trait InfraRepository: Send + Sync {
async fn delete_server_config(&self, key: &str) -> Result<(), DbError>;
async fn get_blob_storage_key_by_cid(&self, cid: &CidLink) -> Result<Option<String>, DbError>;
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<(), DbError>;
async fn get_admin_account_info_by_did(
&self,
did: &Did,
+2 -2
View File
@@ -36,8 +36,8 @@ pub use repo::{
AccountStatus, ApplyCommitError, ApplyCommitInput, ApplyCommitResult, CommitEventData,
EventBlockInline, EventBlocks, FullRecordInfo, ImportBlock, ImportRecord, ImportRepoError,
PruneCount, RecordDelete, RecordInfo, RecordUpsert, RecordWithTakedown, RepoAccountInfo,
RepoEventNotifier, RepoEventReceiver, RepoEventType, RepoInfo, RepoListItem, RepoRepository,
RepoSeqEvent, RepoWithoutRev, SequencedEvent, UserNeedingRecordBlobsBackfill,
RepoEventNotifier, RepoEventReceiver, RepoEventType, RepoIdentity, RepoInfo, RepoListItem,
RepoRepository, RepoSeqEvent, RepoWithoutRev, SequencedEvent, UserNeedingRecordBlobsBackfill,
UserWithoutBlocks,
};
pub use scope::{DbScope, InvalidScopeError};
+8
View File
@@ -171,6 +171,12 @@ pub struct UserNeedingRecordBlobsBackfill {
pub did: Did,
}
#[derive(Debug, Clone)]
pub struct RepoIdentity {
pub user_id: Uuid,
pub did: Did,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RepoSeqEvent {
pub seq: SequenceNumber,
@@ -545,6 +551,8 @@ pub trait RepoRepository: Send + Sync {
limit: i64,
) -> Result<Vec<UserNeedingRecordBlobsBackfill>, DbError>;
async fn get_all_repo_identities(&self) -> Result<Vec<RepoIdentity>, DbError>;
async fn insert_record_blobs(
&self,
repo_id: Uuid,
-2
View File
@@ -5,9 +5,7 @@ edition.workspace = true
license.workspace = true
[features]
default = ["postgres"]
postgres = []
sqlite = []
[dependencies]
tranquil-db-traits = { workspace = true }
+35 -20
View File
@@ -33,7 +33,7 @@ impl BlobRepository for PostgresBlobRepository {
let result = sqlx::query_scalar!(
r#"INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (cid) DO NOTHING RETURNING cid"#,
ON CONFLICT (cid, created_by_user) DO NOTHING RETURNING cid"#,
cid.as_str(),
mime_type,
size_bytes,
@@ -49,7 +49,7 @@ impl BlobRepository for PostgresBlobRepository {
async fn get_blob_metadata(&self, cid: &CidLink) -> Result<Option<BlobMetadata>, DbError> {
let result = sqlx::query!(
"SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1",
"SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1 LIMIT 1",
cid.as_str()
)
.fetch_optional(&self.pool)
@@ -68,7 +68,7 @@ impl BlobRepository for PostgresBlobRepository {
cid: &CidLink,
) -> Result<Option<BlobWithTakedown>, DbError> {
let result = sqlx::query!(
"SELECT cid, takedown_ref FROM blobs WHERE cid = $1",
"SELECT cid, takedown_ref FROM blobs WHERE cid = $1 ORDER BY takedown_ref NULLS LAST LIMIT 1",
cid.as_str()
)
.fetch_optional(&self.pool)
@@ -86,11 +86,13 @@ impl BlobRepository for PostgresBlobRepository {
}
async fn get_blob_storage_key(&self, cid: &CidLink) -> Result<Option<String>, DbError> {
let result =
sqlx::query_scalar!("SELECT storage_key FROM blobs WHERE cid = $1", cid.as_str())
.fetch_optional(&self.pool)
.await
.map_err(map_sqlx_error)?;
let result = sqlx::query_scalar!(
"SELECT storage_key FROM blobs WHERE cid = $1 LIMIT 1",
cid.as_str()
)
.fetch_optional(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result)
}
@@ -147,7 +149,8 @@ impl BlobRepository for PostgresBlobRepository {
async fn sum_blob_storage(&self) -> Result<i64, DbError> {
let result = sqlx::query_scalar!(
r#"SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as "total!" FROM blobs"#
r#"SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as "total!"
FROM (SELECT DISTINCT cid, size_bytes FROM blobs) t"#
)
.fetch_one(&self.pool)
.await
@@ -173,15 +176,6 @@ impl BlobRepository for PostgresBlobRepository {
Ok(result.rows_affected() > 0)
}
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<bool, DbError> {
let result = sqlx::query!("DELETE FROM blobs WHERE cid = $1", cid.as_str())
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result.rows_affected() > 0)
}
async fn delete_blobs_by_user(&self, user_id: Uuid) -> Result<u64, DbError> {
let result = sqlx::query!("DELETE FROM blobs WHERE created_by_user = $1", user_id)
.execute(&self.pool)
@@ -193,7 +187,12 @@ impl BlobRepository for PostgresBlobRepository {
async fn get_blob_storage_keys_by_user(&self, user_id: Uuid) -> Result<Vec<String>, DbError> {
let results = sqlx::query_scalar!(
r#"SELECT storage_key as "storage_key!" FROM blobs WHERE created_by_user = $1"#,
r#"SELECT storage_key as "storage_key!" FROM blobs b
WHERE created_by_user = $1
AND NOT EXISTS (
SELECT 1 FROM blobs o
WHERE o.cid = b.cid AND o.created_by_user <> $1
)"#,
user_id
)
.fetch_all(&self.pool)
@@ -203,6 +202,22 @@ impl BlobRepository for PostgresBlobRepository {
Ok(results)
}
async fn ensure_blob_ownership(&self, user_id: Uuid, cid: &CidLink) -> Result<bool, DbError> {
let result = sqlx::query!(
r#"INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)
SELECT DISTINCT b.cid, b.mime_type, b.size_bytes, $1::uuid, b.storage_key
FROM blobs b WHERE b.cid = $2
ON CONFLICT (cid, created_by_user) DO NOTHING"#,
user_id,
cid.as_str()
)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result.rows_affected() > 0)
}
async fn insert_record_blobs(
&self,
repo_id: Uuid,
@@ -238,7 +253,7 @@ impl BlobRepository for PostgresBlobRepository {
let results = sqlx::query!(
r#"SELECT rb.blob_cid, rb.record_uri
FROM record_blobs rb
LEFT JOIN blobs b ON rb.blob_cid = b.cid
LEFT JOIN blobs b ON rb.blob_cid = b.cid AND b.created_by_user = $1
WHERE rb.repo_id = $1 AND b.cid IS NULL AND rb.blob_cid > $2
ORDER BY rb.blob_cid
LIMIT $3"#,
+1 -20
View File
@@ -7,7 +7,7 @@ use tranquil_db_traits::{
InviteCodeSortOrder, InviteCodeState, InviteCodeUse, NotificationHistoryRow, PlcTokenInfo,
QueuedComms, ReservedSigningKey, ReservedSigningKeyFull, ValidatedInviteCode,
};
use tranquil_types::{CidLink, Did, InviteCode};
use tranquil_types::{Did, InviteCode};
use uuid::Uuid;
use super::col;
@@ -1010,25 +1010,6 @@ impl InfraRepository for PostgresInfraRepository {
Ok(())
}
async fn get_blob_storage_key_by_cid(&self, cid: &CidLink) -> Result<Option<String>, DbError> {
let result =
sqlx::query_scalar!("SELECT storage_key FROM blobs WHERE cid = $1", cid.as_str())
.fetch_optional(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result)
}
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<(), DbError> {
sqlx::query!("DELETE FROM blobs WHERE cid = $1", cid.as_str())
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(())
}
async fn get_admin_account_info_by_did(
&self,
did: &Did,
+25 -2
View File
@@ -4,8 +4,9 @@ use sqlx::PgPool;
use tranquil_db_traits::{
AccountStatus, CommitEventData, DbError, EventBlockInline, EventBlocks, FullRecordInfo,
ImportBlock, ImportRecord, ImportRepoError, PruneCount, RecordInfo, RecordWithTakedown,
RepoAccountInfo, RepoEventType, RepoInfo, RepoListItem, RepoRepository, RepoWithoutRev,
SequenceNumber, SequencedEvent, UserNeedingRecordBlobsBackfill, UserWithoutBlocks,
RepoAccountInfo, RepoEventType, RepoIdentity, RepoInfo, RepoListItem, RepoRepository,
RepoWithoutRev, SequenceNumber, SequencedEvent, UserNeedingRecordBlobsBackfill,
UserWithoutBlocks,
};
use tranquil_types::{AtUri, CidLink, Did, Handle, Nsid, Rkey, Tid};
use uuid::Uuid;
@@ -1650,6 +1651,28 @@ impl RepoRepository for PostgresRepoRepository {
.collect()
}
async fn get_all_repo_identities(&self) -> Result<Vec<RepoIdentity>, DbError> {
let rows = sqlx::query!(
r#"
SELECT u.id as user_id, u.did
FROM users u
JOIN repos r ON r.user_id = u.id
"#
)
.fetch_all(&self.pool)
.await
.map_err(map_sqlx_error)?;
rows.into_iter()
.map(|r| {
Ok(RepoIdentity {
user_id: r.user_id,
did: column(r.did, col::USERS_DID)?,
})
})
.collect()
}
async fn insert_record_blobs(
&self,
repo_id: Uuid,
+7 -1
View File
@@ -4,10 +4,16 @@ version.workspace = true
edition.workspace = true
license.workspace = true
[features]
testing = []
cache-keys = ["dep:tranquil-types"]
[dependencies]
tranquil-config = { workspace = true }
tranquil-types = { workspace = true, optional = true }
async-trait = { workspace = true }
bytes = { workspace = true }
futures = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
+103
View File
@@ -0,0 +1,103 @@
use tranquil_types::{
CidLink, ClientId, CrossPdsState, Did, EmailTokenPurpose, Handle, Jti, JwksUri, Nsid, PdsUrl,
SsoIssuer, SsoJwksUri,
};
pub fn session_key(did: &Did, jti: &Jti) -> String {
format!("auth:session:{}:{}", did, jti)
}
pub fn signing_key_key(did: &Did) -> String {
format!("auth:key:{}", did)
}
pub fn user_status_key(did: &Did) -> String {
format!("auth:status:{}", did)
}
pub fn handle_key(handle: &Handle) -> String {
format!("handle:{}", handle)
}
pub fn reauth_key(did: &Did) -> String {
format!("reauth:{}", did)
}
pub fn plc_doc_key(did: &Did) -> String {
format!("plc:doc:{}", did)
}
pub fn plc_data_key(did: &Did) -> String {
format!("plc:data:{}", did)
}
pub fn did_web_doc_key(did: &Did) -> String {
format!("did:web:doc:{}", did)
}
pub fn email_update_key(did: &Did) -> String {
format!("email_update:{}", did)
}
pub fn email_token_key(did: &Did, purpose: EmailTokenPurpose) -> String {
format!("email_token:{}:{}", purpose, did)
}
pub fn legacy_2fa_challenge_key(did: &Did) -> String {
format!("legacy_2fa:{}", did)
}
pub fn legacy_2fa_cooldown_key(did: &Did) -> String {
format!("legacy_2fa_cooldown:{}", did)
}
pub fn scope_ref_key(cid: &CidLink) -> String {
format!("scope_ref:{}", cid)
}
pub fn auto_verify_sent_key(did: &Did) -> String {
format!("auto_verify_sent:{}", did)
}
pub fn permission_set_key(nsid: &Nsid, aud: Option<&str>) -> String {
match aud {
Some(a) => format!("permset:{}:{}", nsid, a),
None => format!("permset:{}", nsid),
}
}
pub fn oauth_client_meta_key(client_id: &ClientId) -> String {
format!("oauth:client_meta:{}", client_id)
}
pub fn oauth_client_jwks_key(jwks_uri: &JwksUri) -> String {
format!("oauth:jwks:{}", jwks_uri.canonical())
}
pub fn oauth_client_jwks_cooldown_key(jwks_uri: &JwksUri) -> String {
format!("oauth:jwks_cooldown:{}", jwks_uri.canonical())
}
pub fn sso_jwks_key(jwks_uri: &SsoJwksUri) -> String {
format!("sso:jwks:{}", jwks_uri.canonical())
}
pub fn oidc_discovery_key(issuer: &SsoIssuer) -> String {
format!("oidc:discovery:{}", issuer.canonical())
}
pub fn cross_pds_state_key(state: &CrossPdsState) -> String {
format!("cross_pds_state:{}", state)
}
pub fn cross_pds_oauth_meta_key(pds_url: &PdsUrl) -> String {
format!("cross_pds_oauth_meta:v2:{}", pds_url.canonical())
}
pub fn lexicon_doc_key(nsid: &Nsid) -> String {
format!("lexicon:doc:{}", nsid)
}
pub fn lexicon_negative_key(nsid: &Nsid) -> String {
format!("lexicon:neg:{}", nsid)
}
+45
View File
@@ -1,6 +1,15 @@
#[cfg(feature = "cache-keys")]
pub mod cache_keys;
#[cfg(feature = "testing")]
mod memory_cache;
#[cfg(feature = "testing")]
pub use memory_cache::MemoryCache;
use async_trait::async_trait;
use bytes::Bytes;
use futures::Stream;
use std::future::Future;
use std::pin::Pin;
use std::time::Duration;
@@ -57,6 +66,42 @@ pub trait Cache: Send + Sync {
}
}
pub async fn read_json<T: serde::de::DeserializeOwned>(cache: &dyn Cache, key: &str) -> Option<T> {
let json = cache.get(key).await?;
serde_json::from_str(&json).ok()
}
pub async fn write_json<T: serde::Serialize>(
cache: &dyn Cache,
key: &str,
value: &T,
ttl: Duration,
) {
if let Ok(json) = serde_json::to_string(value) {
let _ = cache.set(key, &json, ttl).await;
}
}
pub async fn cached_json<T, E, Fut>(
cache: &dyn Cache,
key: &str,
ttl: Duration,
fetch: impl FnOnce() -> Fut,
) -> Result<T, E>
where
T: serde::Serialize + serde::de::DeserializeOwned,
Fut: Future<Output = Result<T, E>>,
{
match read_json(cache, key).await {
Some(value) => Ok(value),
None => {
let value = fetch().await?;
write_json(cache, key, &value, ttl).await;
Ok(value)
}
}
}
#[async_trait]
pub trait DistributedRateLimiter: Send + Sync {
async fn check_rate_limit(&self, key: &str, limit: u32, window_ms: u64) -> bool;
+74
View File
@@ -0,0 +1,74 @@
use crate::{Cache, CacheError};
use async_trait::async_trait;
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::{Duration, Instant};
struct Entry {
value: Vec<u8>,
expires_at: Instant,
}
#[derive(Default)]
pub struct MemoryCache {
entries: Mutex<HashMap<String, Entry>>,
}
impl MemoryCache {
pub fn new() -> Self {
Self::default()
}
fn read(&self, key: &str) -> Option<Vec<u8>> {
let now = Instant::now();
let mut entries = self.entries.lock().unwrap_or_else(|e| e.into_inner());
match entries.get(key) {
Some(entry) if entry.expires_at > now => Some(entry.value.clone()),
Some(_) => {
entries.remove(key);
None
}
None => None,
}
}
fn write(&self, key: &str, value: Vec<u8>, ttl: Duration) {
let entry = Entry {
value,
expires_at: Instant::now() + ttl,
};
self.entries
.lock()
.unwrap_or_else(|e| e.into_inner())
.insert(key.to_string(), entry);
}
}
#[async_trait]
impl Cache for MemoryCache {
async fn get(&self, key: &str) -> Option<String> {
self.read(key).and_then(|v| String::from_utf8(v).ok())
}
async fn set(&self, key: &str, value: &str, ttl: Duration) -> Result<(), CacheError> {
self.write(key, value.as_bytes().to_vec(), ttl);
Ok(())
}
async fn delete(&self, key: &str) -> Result<(), CacheError> {
self.entries
.lock()
.unwrap_or_else(|e| e.into_inner())
.remove(key);
Ok(())
}
async fn get_bytes(&self, key: &str) -> Option<Vec<u8>> {
self.read(key)
}
async fn set_bytes(&self, key: &str, value: &[u8], ttl: Duration) -> Result<(), CacheError> {
self.write(key, value.to_vec(), ttl);
Ok(())
}
}
+4 -4
View File
@@ -5,11 +5,11 @@ edition.workspace = true
license.workspace = true
[features]
default = []
resolve = ["dep:reqwest", "dep:hickory-resolver", "dep:tokio", "dep:parking_lot", "dep:tracing", "dep:urlencoding"]
resolve = ["dep:reqwest", "dep:hickory-resolver", "dep:tokio", "dep:parking_lot", "dep:tracing", "dep:tranquil-infra"]
[dependencies]
tranquil-types = { path = "../tranquil-types", default-features = false }
tranquil-types = { workspace = true }
tranquil-infra = { workspace = true, optional = true, features = ["cache-keys"] }
serde = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
@@ -20,9 +20,9 @@ hickory-resolver = { workspace = true, optional = true }
tokio = { workspace = true, optional = true }
parking_lot = { workspace = true, optional = true }
tracing = { workspace = true, optional = true }
urlencoding = { workspace = true, optional = true }
[dev-dependencies]
wiremock = { workspace = true }
tokio = { workspace = true }
futures = { workspace = true }
tranquil-infra = { workspace = true, features = ["testing", "cache-keys"] }
+217 -31
View File
@@ -6,9 +6,11 @@ use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, Instant};
use tokio::sync::Notify;
use tranquil_infra::cache_keys::{lexicon_doc_key, lexicon_negative_key};
use tranquil_infra::{Cache, read_json, write_json};
use tranquil_types::Nsid;
const NEGATIVE_CACHE_TTL: Duration = Duration::from_secs(24 * 60 * 60);
const NEGATIVE_CACHE_TTL: Duration = Duration::from_secs(60 * 60);
const POSITIVE_CACHE_TTL: Duration = Duration::from_secs(24 * 60 * 60);
const REFRESH_FAILURE_BACKOFF: Duration = Duration::from_secs(60);
const MAX_DYNAMIC_SCHEMAS: usize = 1024;
@@ -17,6 +19,13 @@ struct NegativeEntry {
expires_at: Instant,
}
fn negative_ttl_for(error: &ResolveError) -> Duration {
match error.is_definitive() {
true => NEGATIVE_CACHE_TTL,
false => REFRESH_FAILURE_BACKOFF,
}
}
struct PositiveEntry {
doc: Arc<LexiconDoc>,
expires_at: Instant,
@@ -44,6 +53,7 @@ pub struct DynamicRegistry {
negative_cache: RwLock<HashMap<Nsid, NegativeEntry>>,
in_flight: RwLock<HashMap<Nsid, Arc<Notify>>>,
network_disabled: AtomicBool,
shared: RwLock<Option<Arc<dyn Cache>>>,
}
struct InFlightGuard<'a> {
@@ -70,9 +80,18 @@ impl DynamicRegistry {
negative_cache: RwLock::new(HashMap::new()),
in_flight: RwLock::new(HashMap::new()),
network_disabled: AtomicBool::new(false),
shared: RwLock::new(None),
}
}
pub fn set_shared_cache(&self, cache: Arc<dyn Cache>) {
*self.shared.write() = Some(cache);
}
fn shared_cache(&self) -> Option<Arc<dyn Cache>> {
self.shared.read().clone()
}
pub fn from_env() -> Self {
let registry = Self::new();
let disabled =
@@ -105,13 +124,17 @@ impl DynamicRegistry {
}
pub fn is_negative_cached(&self, nsid: &Nsid) -> bool {
let cache = self.negative_cache.read();
cache
.get(nsid)
.is_some_and(|entry| entry.expires_at > Instant::now())
self.negative_remaining(nsid).is_some()
}
fn insert_negative(&self, nsid: &Nsid) {
fn negative_remaining(&self, nsid: &Nsid) -> Option<Duration> {
self.negative_cache
.read()
.get(nsid)
.and_then(|entry| entry.expires_at.checked_duration_since(Instant::now()))
}
fn insert_negative(&self, nsid: &Nsid, ttl: Duration) {
let mut cache = self.negative_cache.write();
if cache.len() >= MAX_DYNAMIC_SCHEMAS {
let now = Instant::now();
@@ -120,7 +143,7 @@ impl DynamicRegistry {
cache.insert(
nsid.clone(),
NegativeEntry {
expires_at: Instant::now() + NEGATIVE_CACHE_TTL,
expires_at: Instant::now() + ttl,
},
);
}
@@ -159,6 +182,44 @@ impl DynamicRegistry {
arc
}
async fn shared_get(&self, nsid: &Nsid) -> Option<Arc<LexiconDoc>> {
let cache = self.shared_cache()?;
let doc = read_json::<LexiconDoc>(cache.as_ref(), &lexicon_doc_key(nsid)).await?;
Some(self.insert_schema(doc))
}
async fn shared_put(&self, doc: &LexiconDoc) {
let Some(cache) = self.shared_cache() else {
return;
};
write_json(
cache.as_ref(),
&lexicon_doc_key(&doc.id),
doc,
POSITIVE_CACHE_TTL,
)
.await;
let _ = cache.delete(&lexicon_negative_key(&doc.id)).await;
}
async fn shared_is_negative(&self, nsid: &Nsid) -> bool {
match self.shared_cache() {
Some(cache) => cache.get(&lexicon_negative_key(nsid)).await.is_some(),
None => false,
}
}
async fn shared_put_negative(&self, nsid: &Nsid, error: &ResolveError) {
if !error.is_definitive() {
return;
}
if let Some(cache) = self.shared_cache() {
let _ = cache
.set(&lexicon_negative_key(nsid), "1", NEGATIVE_CACHE_TTL)
.await;
}
}
fn bump_expiry(&self, nsid: &Nsid, duration: Duration) {
let mut store = self.store.write();
if let Some(entry) = store.schemas.get_mut(nsid) {
@@ -203,15 +264,23 @@ impl DynamicRegistry {
match self.acquire_leadership(nsid) {
Some(_guard) => match resolver(nsid.clone()).await {
Ok(doc) => Ok(self.insert_schema(doc)),
Ok(doc) => {
self.shared_put(&doc).await;
Ok(self.insert_schema(doc))
}
Err(e) => {
let (doc, source) = match self.shared_get(nsid).await {
Some(doc) => (doc, "shared"),
None => (stale, "local"),
};
self.bump_expiry(nsid, REFRESH_FAILURE_BACKOFF);
tracing::warn!(
nsid = %nsid,
error = %e,
"lexicon refresh failed, serving stale cached entry"
source,
"lexicon refresh failed, serving cached entry"
);
Ok(stale)
Ok(doc)
}
},
None => {
@@ -230,34 +299,59 @@ impl DynamicRegistry {
F: FnOnce(Nsid) -> Fut,
Fut: std::future::Future<Output = Result<LexiconDoc, ResolveError>>,
{
if self.network_disabled.load(Ordering::Relaxed) {
return Err(ResolveError::NetworkDisabled);
if let Some(doc) = self.shared_get(nsid).await {
return Ok(doc);
}
if self.is_negative_cached(nsid) {
if let Some(remaining) = self.negative_remaining(nsid) {
return Err(ResolveError::NegativelyCached {
nsid: nsid.clone(),
ttl_secs: NEGATIVE_CACHE_TTL.as_secs(),
ttl_secs: remaining.as_secs(),
});
}
if self.shared_is_negative(nsid).await {
// Cache reports 0 remaining TTL for shared negative hit,
// so we mirror for the backoff rather than a full `NEGATIVE_CACHE_TTL`.
self.insert_negative(nsid, REFRESH_FAILURE_BACKOFF);
return Err(ResolveError::NegativelyCached {
nsid: nsid.clone(),
ttl_secs: REFRESH_FAILURE_BACKOFF.as_secs(),
});
}
if self.network_disabled.load(Ordering::Relaxed) {
return Err(ResolveError::NetworkDisabled);
}
match self.acquire_leadership(nsid) {
Some(_guard) => match resolver(nsid.clone()).await {
Ok(doc) => Ok(self.insert_schema(doc)),
Ok(doc) => {
self.shared_put(&doc).await;
Ok(self.insert_schema(doc))
}
Err(e) => {
self.insert_negative(nsid);
tracing::debug!(nsid = %nsid, error = %e, "caching negative resolution result");
let ttl = negative_ttl_for(&e);
self.insert_negative(nsid, ttl);
self.shared_put_negative(nsid, &e).await;
tracing::debug!(
nsid = %nsid,
error = %e,
ttl_secs = ttl.as_secs(),
"caching negative resolution result"
);
Err(e)
}
},
None => {
self.wait_for_leader(nsid).await;
match self.get_cached(nsid) {
Some(doc) => Ok(doc),
None if self.is_negative_cached(nsid) => Err(ResolveError::NegativelyCached {
match (self.get_cached(nsid), self.negative_remaining(nsid)) {
(Some(doc), _) => Ok(doc),
(None, Some(remaining)) => Err(ResolveError::NegativelyCached {
nsid: nsid.clone(),
ttl_secs: NEGATIVE_CACHE_TTL.as_secs(),
ttl_secs: remaining.as_secs(),
}),
None => Err(ResolveError::LeaderAborted { nsid: nsid.clone() }),
(None, None) => Err(ResolveError::LeaderAborted { nsid: nsid.clone() }),
}
}
}
@@ -316,6 +410,7 @@ impl Default for DynamicRegistry {
#[cfg(test)]
mod tests {
use super::*;
use tranquil_infra::MemoryCache;
fn nsid(s: &str) -> Nsid {
s.parse().unwrap()
@@ -324,19 +419,19 @@ mod tests {
#[test]
fn test_negative_cache() {
let registry = DynamicRegistry::new();
assert!(!registry.is_negative_cached(&nsid("com.example.test")));
assert!(!registry.is_negative_cached(&nsid("pet.nel.negative")));
registry.insert_negative(&nsid("com.example.test"));
assert!(registry.is_negative_cached(&nsid("com.example.test")));
registry.insert_negative(&nsid("pet.nel.negative"), NEGATIVE_CACHE_TTL);
assert!(registry.is_negative_cached(&nsid("pet.nel.negative")));
}
#[tokio::test]
async fn test_negative_cache_returns_appropriate_error_variant() {
let registry = DynamicRegistry::new();
registry.insert_negative(&nsid("com.example.cached"));
registry.insert_negative(&nsid("pet.nel.cached"), NEGATIVE_CACHE_TTL);
let err = registry
.resolve_and_cache(&nsid("com.example.cached"))
.resolve_and_cache(&nsid("pet.nel.cached"))
.await
.unwrap_err();
@@ -383,17 +478,17 @@ mod tests {
fn test_negative_cache_cleared_on_insert() {
let registry = DynamicRegistry::new();
registry.insert_negative(&nsid("com.example.test"));
assert!(registry.is_negative_cached(&nsid("com.example.test")));
registry.insert_negative(&nsid("pet.nel.cleared"), NEGATIVE_CACHE_TTL);
assert!(registry.is_negative_cached(&nsid("pet.nel.cleared")));
let doc = LexiconDoc {
lexicon: 1,
id: nsid("com.example.test"),
id: nsid("pet.nel.cleared"),
defs: HashMap::new(),
};
registry.insert_schema(doc);
assert!(!registry.is_negative_cached(&nsid("com.example.test")));
assert!(!registry.is_negative_cached(&nsid("pet.nel.cleared")));
}
#[test]
@@ -692,4 +787,95 @@ mod tests {
"evicted Arc should be freed when no external references remain"
);
}
#[tokio::test]
async fn test_shared_positive_hit_skips_resolver() {
let registry = DynamicRegistry::new();
let cache = Arc::new(MemoryCache::new());
registry.set_shared_cache(cache.clone());
let doc = LexiconDoc {
lexicon: 1,
id: nsid("pet.nel.sharedDoc"),
defs: HashMap::new(),
};
cache
.set(
&lexicon_doc_key(&nsid("pet.nel.sharedDoc")),
&serde_json::to_string(&doc).unwrap(),
POSITIVE_CACHE_TTL,
)
.await
.unwrap();
let resolved = registry
.resolve_and_cache_with(&nsid("pet.nel.sharedDoc"), |_| async move {
panic!("resolver mustn't run on a shared positive hit")
})
.await
.unwrap();
assert_eq!(resolved.id, "pet.nel.sharedDoc");
assert!(registry.get_cached(&nsid("pet.nel.sharedDoc")).is_some());
}
#[tokio::test]
async fn test_definitive_failure_writes_shared_negative_and_peers_mirror_it() {
let cache = Arc::new(MemoryCache::new());
let registry = DynamicRegistry::new();
registry.set_shared_cache(cache.clone());
let _ = registry
.resolve_and_cache_with(&nsid("pet.nel.gone"), |n| async move {
Err::<LexiconDoc, _>(ResolveError::SchemaNotFound {
nsid: n,
url: "https://oyster.cafe".to_string(),
})
})
.await;
assert!(
cache
.get(&lexicon_negative_key(&nsid("pet.nel.gone")))
.await
.is_some(),
"definitive failure must write the shared negative key"
);
let _ = registry
.resolve_and_cache_with(&nsid("pet.nel.transient"), |n| async move {
Err::<LexiconDoc, _>(ResolveError::DnsLookup {
domain: n.into_inner(),
reason: "simulated".to_string(),
})
})
.await;
assert!(
cache
.get(&lexicon_negative_key(&nsid("pet.nel.transient")))
.await
.is_none(),
"transient failure must stay out of the shared negative key"
);
let peer = DynamicRegistry::new();
peer.set_shared_cache(cache);
let err = peer
.resolve_and_cache_with(&nsid("pet.nel.gone"), |_| async move {
panic!("resolver mustn't run on a shared negative hit")
})
.await
.unwrap_err();
match err {
ResolveError::NegativelyCached { ttl_secs, .. } => assert!(
ttl_secs <= REFRESH_FAILURE_BACKOFF.as_secs(),
"local mirror must use the backoff TTL, got {}s",
ttl_secs
),
other => panic!("expected NegativelyCached, got: {}", other),
}
assert!(
peer.negative_remaining(&nsid("pet.nel.gone"))
.expect("local mirror exists")
<= REFRESH_FAILURE_BACKOFF
);
}
}
+47 -8
View File
@@ -34,15 +34,28 @@ pub fn is_valid_datetime(s: &str) -> bool {
chrono::DateTime::parse_from_rfc3339(s).is_ok()
}
/// Checks the scheme only, not the character set or structure of what
/// follows. The aim is to accept at least all valid URIs; we can always
/// tighten this later. It does not parse the authority, because at-uris
/// put colons in the authority (at://did:plc:abc123/collection/rkey) and
/// any 3986 authority parser reads that as a non-numeric port and rejects
/// it.
pub fn is_valid_uri(s: &str) -> bool {
s.split_once("://").is_some_and(|(scheme, rest)| {
!scheme.is_empty()
&& scheme
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '+' || c == '.' || c == '-')
&& scheme.starts_with(|c: char| c.is_ascii_alphabetic())
&& !rest.is_empty()
})
let Some((scheme, rest)) = s.split_once(':') else {
return false;
};
let valid_scheme = !scheme.is_empty()
&& scheme.starts_with(|c: char| c.is_ascii_alphabetic())
&& scheme
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '+' || c == '.' || c == '-' || c == '_');
if !valid_scheme {
return false;
}
match rest.strip_prefix("//") {
Some(authority_and_path) => !authority_and_path.is_empty(),
None => true,
}
}
pub fn is_valid_cid(s: &str) -> bool {
@@ -151,6 +164,32 @@ mod tests {
assert!(!is_valid_uri("https://"));
}
#[test]
fn test_valid_uris_without_authority() {
// RFC 3986 hier-part doesn't require "//": scheme ":" opaque-part is also a URI.
assert!(is_valid_uri("urn:isbn:9780141439518"));
assert!(is_valid_uri("mailto:user@example.com"));
assert!(is_valid_uri("mbid:70766a5a-3f95-4b19-96c8-a2c9c4a5e6e5")); //authority-less / path-rootless
assert!(is_valid_uri(
"has_an_underscore:70766a5a-3f95-4b19-96c8-a2c9c4a5e6e5"
));
assert!(is_valid_uri("urn:"));
}
#[test]
fn test_invalid_uri_without_scheme() {
assert!(!is_valid_uri(":no-scheme"));
}
#[test]
fn test_valid_uris_dont_reject_at_uri_authority_colons() {
// at-uri authorities contain colons (did:plc:...); is_valid_uri must not
// reject them the way a strict RFC 3986 authority parser would.
assert!(is_valid_uri(
"at://did:plc:cwdkf4xxjpznceembuuspt3d/sh.tangled.repo.pull/3mtjn7zouwn22"
));
}
#[test]
fn test_valid_cids() {
assert!(is_valid_cid("bafyreiabcdef123456"));
+5
View File
@@ -125,6 +125,11 @@ impl LexiconRegistry {
pub fn is_negative_cached(&self, nsid: &Nsid) -> bool {
self.dynamic.is_negative_cached(nsid)
}
#[cfg(feature = "resolve")]
pub fn set_shared_cache(&self, cache: Arc<dyn tranquil_infra::Cache>) {
self.dynamic.set_shared_cache(cache);
}
}
pub struct ResolvedRef {
+95 -82
View File
@@ -4,7 +4,10 @@ use hickory_resolver::config::{ResolverConfig, ResolverOpts};
use reqwest::Client;
use std::sync::OnceLock;
use std::time::Duration;
use tranquil_types::{Did, Nsid};
use tranquil_types::did_doc::extract_pds_endpoint;
use tranquil_types::{
Did, Nsid, SchemaHostUrl, UrlKind, dns_guard, redirect_policy, url_kind, url_reach_permits,
};
static RESOLVER_CLIENT: OnceLock<Client> = OnceLock::new();
@@ -17,7 +20,8 @@ fn client() -> &'static Client {
.connect_timeout(Duration::from_secs(5))
.pool_max_idle_per_host(4)
.pool_idle_timeout(Duration::from_secs(60))
.redirect(reqwest::redirect::Policy::limited(3))
.redirect(redirect_policy(url_kind::SchemaHost::REACH_POLICY))
.dns_resolver(dns_guard(url_kind::SchemaHost::REACH_POLICY))
.build()
.expect("failed to build lexicon resolver HTTP client")
})
@@ -63,6 +67,8 @@ pub enum ResolveError {
NoPdsEndpoint { did: Did },
#[error("schema fetch failed from {url}: {reason}")]
SchemaFetch { url: String, reason: String },
#[error("no schema record for {nsid} at {url}")]
SchemaNotFound { nsid: Nsid, url: String },
#[error("schema deserialization failed: {0}")]
InvalidSchema(String),
#[error("schema resolution recently failed for {nsid}, cached for {ttl_secs}s")]
@@ -73,6 +79,23 @@ pub enum ResolveError {
LeaderAborted { nsid: Nsid },
}
impl ResolveError {
pub fn is_definitive(&self) -> bool {
match self {
Self::NoDid { .. }
| Self::NoPdsEndpoint { .. }
| Self::InvalidSchema(_)
| Self::SchemaNotFound { .. } => true,
Self::DnsLookup { .. }
| Self::DidResolution { .. }
| Self::SchemaFetch { .. }
| Self::NegativelyCached { .. }
| Self::NetworkDisabled
| Self::LeaderAborted { .. } => false,
}
}
}
pub fn nsid_to_authority(nsid: &Nsid) -> String {
let mut segments: Vec<&str> = nsid.split('.').collect();
segments.pop();
@@ -123,7 +146,7 @@ pub async fn resolve_did_from_dns(authority: &str) -> Result<Did, ResolveError>
pub async fn resolve_pds_endpoint(
did: &Did,
plc_directory_url: Option<&str>,
) -> Result<String, ResolveError> {
) -> Result<SchemaHostUrl, ResolveError> {
let plc_base = plc_directory_url.unwrap_or(DEFAULT_PLC_DIRECTORY);
let url = match did
@@ -131,7 +154,20 @@ pub async fn resolve_pds_endpoint(
.and_then(|(_, rest)| rest.split_once(':'))
{
Some(("plc", _)) => format!("{}/{}", plc_base.trim_end_matches('/'), did),
Some(("web", domain)) => format!("https://{}/.well-known/did.json", domain),
Some(("web", domain)) => {
let url = format!("https://{}/.well-known/did.json", domain);
let permitted = reqwest::Url::parse(&url)
.is_ok_and(|u| url_reach_permits(&u, url_kind::SchemaHost::REACH_POLICY));
match permitted {
true => url,
false => {
return Err(ResolveError::DidResolution {
did: did.clone(),
reason: "did:web host is outside the allowed host reach".to_string(),
});
}
}
}
_ => {
return Err(ResolveError::DidResolution {
did: did.clone(),
@@ -162,39 +198,29 @@ pub async fn resolve_pds_endpoint(
reason: e.to_string(),
})?;
extract_pds_endpoint(&doc).ok_or_else(|| ResolveError::NoPdsEndpoint { did: did.clone() })
extract_pds_endpoint(&doc).map_err(|_| ResolveError::NoPdsEndpoint { did: did.clone() })
}
fn extract_pds_endpoint(doc: &serde_json::Value) -> Option<String> {
doc.get("service")
.and_then(|s| s.as_array())
.and_then(|services| {
services.iter().find_map(|svc| {
let is_pds = svc
.get("type")
.and_then(|t| t.as_str())
.is_some_and(|t| t == "AtprotoPersonalDataServer");
is_pds
.then(|| svc.get("serviceEndpoint").and_then(|ep| ep.as_str()))?
.map(|s| s.to_string())
})
})
fn is_record_absent(xrpc_error: &str, xrpc_message: &str) -> bool {
xrpc_error == "RecordNotFound"
|| xrpc_error == "InvalidRequest" && xrpc_message.starts_with("Could not locate record")
}
pub async fn fetch_schema_from_pds(
pds_endpoint: &str,
pds_endpoint: &SchemaHostUrl,
did: &Did,
nsid: &Nsid,
) -> Result<LexiconDoc, ResolveError> {
let url = format!(
"{}/xrpc/com.atproto.repo.getRecord?repo={}&collection=com.atproto.lexicon.schema&rkey={}",
pds_endpoint.trim_end_matches('/'),
urlencoding::encode(did.as_str()),
urlencoding::encode(nsid.as_str())
);
let mut request_url = pds_endpoint.endpoint("xrpc/com.atproto.repo.getRecord");
request_url
.query_pairs_mut()
.append_pair("repo", did.as_str())
.append_pair("collection", "com.atproto.lexicon.schema")
.append_pair("rkey", nsid.as_str());
let url = request_url.to_string();
let resp = client()
.get(&url)
.get(request_url)
.send()
.await
.map_err(|e| ResolveError::SchemaFetch {
@@ -204,10 +230,27 @@ pub async fn fetch_schema_from_pds(
let status = resp.status();
if !status.is_success() {
return Err(ResolveError::SchemaFetch {
url,
reason: format!("HTTP {}", status),
});
let body = read_body_limited(resp, MAX_RESPONSE_BYTES)
.await
.ok()
.and_then(|bytes| serde_json::from_slice::<serde_json::Value>(&bytes).ok())
.unwrap_or(serde_json::Value::Null);
let field = |name: &str| {
body.get(name)
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string()
};
return match is_record_absent(&field("error"), &field("message")) {
true => Err(ResolveError::SchemaNotFound {
nsid: nsid.clone(),
url,
}),
false => Err(ResolveError::SchemaFetch {
url,
reason: format!("HTTP {}", status),
}),
};
}
let body = read_body_limited(resp, MAX_RESPONSE_BYTES)
@@ -292,6 +335,27 @@ mod tests {
s.parse().unwrap()
}
#[test]
fn is_record_absent_recognizes_only_the_reference_pds_absence_shapes() {
assert!(is_record_absent(
"RecordNotFound",
"Could not locate record: at://did:plc:nel/com.atproto.lexicon.schema/x"
));
assert!(is_record_absent("RecordNotFound", ""));
assert!(is_record_absent(
"InvalidRequest",
"Could not locate record"
));
assert!(!is_record_absent(
"InvalidRequest",
"Error: rkey must be a valid record key"
));
assert!(!is_record_absent("InvalidRequest", ""));
assert!(!is_record_absent("InternalServerError", ""));
assert!(!is_record_absent("RateLimitExceeded", ""));
assert!(!is_record_absent("", ""));
}
#[test]
fn test_nsid_to_authority() {
assert_eq!(
@@ -316,57 +380,6 @@ mod tests {
);
}
#[test]
fn test_extract_pds_endpoint_valid() {
let doc = serde_json::json!({
"service": [{
"type": "AtprotoPersonalDataServer",
"serviceEndpoint": "https://pds.example.com"
}]
});
assert_eq!(
extract_pds_endpoint(&doc),
Some("https://pds.example.com".to_string())
);
}
#[test]
fn test_extract_pds_endpoint_multiple_services() {
let doc = serde_json::json!({
"service": [
{
"type": "AtprotoLabeler",
"serviceEndpoint": "https://labeler.example.com"
},
{
"type": "AtprotoPersonalDataServer",
"serviceEndpoint": "https://pds.example.com"
}
]
});
assert_eq!(
extract_pds_endpoint(&doc),
Some("https://pds.example.com".to_string())
);
}
#[test]
fn test_extract_pds_endpoint_missing() {
let doc = serde_json::json!({
"service": [{
"type": "AtprotoLabeler",
"serviceEndpoint": "https://labeler.example.com"
}]
});
assert_eq!(extract_pds_endpoint(&doc), None);
}
#[test]
fn test_extract_pds_endpoint_no_services() {
let doc = serde_json::json!({});
assert_eq!(extract_pds_endpoint(&doc), None);
}
#[test]
fn test_validate_fetched_schema_ok() {
let doc = LexiconDoc {
+15 -15
View File
@@ -1,8 +1,8 @@
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use tranquil_types::Nsid;
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
pub struct LexiconDoc {
pub lexicon: u32,
pub id: Nsid,
@@ -10,7 +10,7 @@ pub struct LexiconDoc {
pub defs: HashMap<String, LexDef>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "type")]
pub enum LexDef {
#[serde(rename = "record")]
@@ -35,14 +35,14 @@ pub enum LexDef {
PermissionSet {},
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
pub struct LexRecord {
#[serde(default)]
pub key: Option<String>,
pub record: LexObject,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
pub struct LexObject {
#[serde(default)]
pub required: Vec<String>,
@@ -52,7 +52,7 @@ pub struct LexObject {
pub properties: HashMap<String, LexProperty>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "type")]
pub enum LexProperty {
#[serde(rename = "string")]
@@ -79,7 +79,7 @@ pub enum LexProperty {
Object(LexObject),
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct LexString {
#[serde(default)]
@@ -102,7 +102,7 @@ pub struct LexString {
pub default: Option<String>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
pub struct LexInteger {
#[serde(default)]
pub minimum: Option<i64>,
@@ -116,7 +116,7 @@ pub struct LexInteger {
pub const_value: Option<i64>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct LexBytes {
#[serde(default)]
@@ -125,7 +125,7 @@ pub struct LexBytes {
pub min_length: Option<u64>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct LexBlob {
#[serde(default)]
@@ -134,7 +134,7 @@ pub struct LexBlob {
pub max_size: Option<u64>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct LexArray {
pub items: Box<LexProperty>,
@@ -144,7 +144,7 @@ pub struct LexArray {
pub max_length: Option<u64>,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
pub struct LexUnion {
#[serde(default)]
pub refs: Vec<String>,
@@ -152,14 +152,14 @@ pub struct LexUnion {
pub closed: bool,
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct LexRef {
#[serde(rename = "ref")]
pub reference: String,
}
#[derive(Debug, Clone, Deserialize)]
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum StringFormat {
#[serde(rename = "did")]
Did,
@@ -204,6 +204,6 @@ pub fn parse_ref(reference: &str) -> ParsedRef<'_> {
}
}
#[derive(Debug, Deserialize)]
#[derive(Debug, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct LexStringDef {}
@@ -74,7 +74,7 @@ async fn test_resolve_pds_endpoint_from_plc() {
let endpoint = resolve_pds_endpoint(&did.parse().unwrap(), Some(&plc_server.uri()))
.await
.unwrap();
assert_eq!(endpoint, "https://pds.example.com");
assert_eq!(endpoint.as_str(), "https://pds.example.com");
}
#[tokio::test]
@@ -130,14 +130,17 @@ async fn test_resolve_pds_endpoint_multiple_services_picks_pds() {
"id": did,
"service": [
{
"id": "#atproto_labeler",
"type": "AtprotoLabeler",
"serviceEndpoint": "https://labeler.example.com"
},
{
"id": "#bsky_notif",
"type": "BskyNotificationService",
"serviceEndpoint": "https://notify.example.com"
},
{
"id": "#atproto_pds",
"type": "AtprotoPersonalDataServer",
"serviceEndpoint": "https://pds.example.com"
}
@@ -149,7 +152,7 @@ async fn test_resolve_pds_endpoint_multiple_services_picks_pds() {
let endpoint = resolve_pds_endpoint(&did.parse().unwrap(), Some(&plc_server.uri()))
.await
.unwrap();
assert_eq!(endpoint, "https://pds.example.com");
assert_eq!(endpoint.as_str(), "https://pds.example.com");
}
#[tokio::test]
@@ -168,7 +171,7 @@ async fn test_fetch_schema_from_pds_success() {
.await;
let doc = fetch_schema_from_pds(
&pds_server.uri(),
&pds_server.uri().parse().unwrap(),
&did.parse().unwrap(),
&nsid.parse().unwrap(),
)
@@ -195,7 +198,7 @@ async fn test_fetch_schema_missing_value_field() {
.await;
let result = fetch_schema_from_pds(
&pds_server.uri(),
&pds_server.uri().parse().unwrap(),
&did.parse().unwrap(),
&nsid.parse().unwrap(),
)
@@ -222,7 +225,7 @@ async fn test_fetch_schema_invalid_lexicon_json() {
.await;
let result = fetch_schema_from_pds(
&pds_server.uri(),
&pds_server.uri().parse().unwrap(),
&did.parse().unwrap(),
&nsid.parse().unwrap(),
)
@@ -352,7 +355,7 @@ async fn test_pds_trailing_slash_handled() {
let pds_url_with_slash = format!("{}/", pds_server.uri());
let doc = fetch_schema_from_pds(
&pds_url_with_slash,
&pds_url_with_slash.parse().unwrap(),
&did.parse().unwrap(),
&nsid.parse().unwrap(),
)
@@ -377,7 +380,7 @@ async fn test_fetch_schema_error_status_gives_meaningful_error() {
.await;
let result = fetch_schema_from_pds(
&pds_server.uri(),
&pds_server.uri().parse().unwrap(),
&did.parse().unwrap(),
&nsid.parse().unwrap(),
)
+4
View File
@@ -37,3 +37,7 @@ webauthn-rs = { workspace = true }
[dev-dependencies]
async-trait = { workspace = true }
tranquil-infra = { workspace = true, features = ["testing"] }
[features]
bsky = []
@@ -1,4 +1,5 @@
use super::*;
use tranquil_scopes::{ParsedScope, parse_scope};
use tranquil_types::Nsid;
#[derive(Debug, Serialize)]
@@ -10,6 +11,9 @@ pub struct ScopeInfo {
pub display_name: String,
pub granted: Option<bool>,
pub restricted: bool,
pub superseded: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub effective_scope: Option<String>,
}
#[derive(Debug, Serialize)]
@@ -25,6 +29,7 @@ pub struct PermissionSetInfo {
pub expanded: Vec<ScopeInfo>,
pub granted: Option<bool>,
pub restricted: bool,
pub superseded: bool,
}
#[derive(Debug, Serialize)]
@@ -38,6 +43,13 @@ pub struct FailedSetInfo {
pub reason: tranquil_scopes::ResolveFailure,
}
#[derive(Debug, Serialize)]
pub struct RejectedScopeInfo {
// The scope exactly as the client requested it, which may be invalid or malformed.
pub scope: String,
pub reason: tranquil_scopes::ScopeRejection,
}
#[derive(Debug, Serialize)]
pub struct ConsentResponse {
pub request_uri: String,
@@ -47,7 +59,9 @@ pub struct ConsentResponse {
pub logo_uri: Option<String>,
pub scopes: Vec<ScopeInfo>,
pub permission_sets: Vec<PermissionSetInfo>,
pub transition_supersedes: bool,
pub failed_sets: Vec<FailedSetInfo>,
pub rejected_scopes: Vec<RejectedScopeInfo>,
pub show_consent: bool,
pub did: Did,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -118,7 +132,7 @@ pub async fn consent_get(
};
let did = flow_with_user.did().clone();
let client_cache = ClientMetadataCache::new(3600);
let client_cache = &state.client_metadata_cache;
let client_metadata = client_cache
.get(&request_data.parameters.client_id)
.await
@@ -150,9 +164,13 @@ pub async fn consent_get(
Some(grant) => scope_resolution::Authority::Delegated(&grant.granted_scopes),
None => scope_resolution::Authority::FullSelf,
};
let effective =
scope_resolution::resolve_effective_scopes(&*state.cache, requested_scope_str, authority)
.await;
let effective = scope_resolution::resolve_effective_scopes(
&*state.cache,
requested_scope_str,
authority,
client_metadata.as_ref().and_then(|m| m.scope.as_deref()),
)
.await;
let requested_scopes: Vec<&str> = effective.permitted.split_whitespace().collect();
let preferences = state
.repos
@@ -164,16 +182,7 @@ pub async fn consent_get(
.iter()
.map(|p| (p.scope.as_str(), p.granted))
.collect();
let presented_item_strings: Vec<String> = effective
.outcome
.passthrough
.iter()
.cloned()
.chain(effective.outcome.sets.iter().map(|g| match &g.aud {
Some(a) => format!("include:{}?aud={}", g.nsid, a),
None => format!("include:{}", g.nsid),
}))
.collect();
let presented_item_strings = effective.outcome.unexpanded_scopes();
let show_consent = should_show_consent(
state.repos.oauth.as_ref(),
&did,
@@ -183,23 +192,35 @@ pub async fn consent_get(
.await
.unwrap_or(true);
let has_granular_scopes = requested_scopes.iter().any(|s| is_granular_scope(s));
let has_transition_generic = requested_scopes
.iter()
.any(|s| matches!(parse_scope(s), ParsedScope::TransitionGeneric));
let grant_scope_str: Option<&str> =
delegation_grant.as_ref().map(|g| g.granted_scopes.as_str());
let is_restricted = |scope: &str| -> bool {
grant_scope_str.is_some_and(|g| !tranquil_pds::delegation::grant_covers(g, scope))
let coverage_of = |scope: &str| -> tranquil_pds::delegation::GrantCoverage {
match grant_scope_str {
Some(g) => tranquil_pds::delegation::grant_coverage(g, scope),
None => tranquil_pds::delegation::GrantCoverage::Full,
}
};
let make_scope_info = |scope: &str| -> ScopeInfo {
let (restricted, effective_scope) = match coverage_of(scope) {
tranquil_pds::delegation::GrantCoverage::Full => (false, None),
tranquil_pds::delegation::GrantCoverage::Narrowed(narrowed) => (false, Some(narrowed)),
tranquil_pds::delegation::GrantCoverage::Withheld => (true, None),
};
let described = effective_scope.as_deref().unwrap_or(scope);
let (category, required, description, display_name) =
if let Some(def) = tranquil_pds::oauth::scopes::SCOPE_DEFINITIONS.get(scope) {
let desc = if scope == "atproto" && has_granular_scopes {
if let Some(def) = tranquil_pds::oauth::scopes::SCOPE_DEFINITIONS.get(described) {
let desc = if described == "atproto" && has_granular_scopes {
"AT Protocol baseline scope (permissions determined by selected options below)"
.to_string()
} else {
def.description.to_string()
};
let name = if scope == "atproto" && has_granular_scopes {
let name = if described == "atproto" && has_granular_scopes {
"AT Protocol Access".to_string()
} else {
def.display_name.to_string()
@@ -210,22 +231,24 @@ pub async fn consent_get(
desc,
name,
)
} else if scope.starts_with("ref:") {
} else if described.starts_with("ref:") {
(
"Reference".to_string(),
false,
"Referenced scope".to_string(),
scope.to_string(),
described.to_string(),
)
} else {
(
"Other".to_string(),
false,
format!("Access to {}", scope),
scope.to_string(),
format!("Access to {}", described),
described.to_string(),
)
};
let granted = pref_map.get(scope).copied();
let superseded = has_transition_generic
&& tranquil_scopes::superseded_by_transition_generic(&parse_scope(scope));
ScopeInfo {
scope: scope.to_string(),
category,
@@ -233,7 +256,9 @@ pub async fn consent_get(
description,
display_name,
granted,
restricted: is_restricted(scope),
restricted,
superseded,
effective_scope,
}
};
@@ -249,12 +274,10 @@ pub async fn consent_get(
.sets
.iter()
.map(|g| {
let include_scope = match &g.aud {
Some(a) => format!("include:{}?aud={}", g.nsid, a),
None => format!("include:{}", g.nsid),
};
let include_scope = g.include_token();
let expanded: Vec<ScopeInfo> = g.expanded.iter().map(|s| make_scope_info(s)).collect();
let restricted = !expanded.is_empty() && expanded.iter().all(|s| s.restricted);
let superseded = !expanded.is_empty() && expanded.iter().all(|s| s.superseded);
PermissionSetInfo {
nsid: g.nsid.clone(),
aud: g.aud.clone(),
@@ -264,6 +287,7 @@ pub async fn consent_get(
include_scope,
expanded,
restricted,
superseded,
}
})
.collect();
@@ -279,6 +303,16 @@ pub async fn consent_get(
})
.collect();
let rejected_scopes: Vec<RejectedScopeInfo> = effective
.outcome
.rejected
.iter()
.map(|r| RejectedScopeInfo {
scope: r.scope.clone(),
reason: r.reason,
})
.collect();
let account_handle = state
.repos
.user
@@ -320,6 +354,9 @@ pub async fn consent_get(
(None, None, None, None)
};
let transition_supersedes =
scopes.iter().any(|s| s.superseded) || permission_sets.iter().any(|s| s.superseded);
Json(ConsentResponse {
request_uri: query.request_uri.clone(),
client_id: request_data.parameters.client_id.clone(),
@@ -328,7 +365,9 @@ pub async fn consent_get(
logo_uri: client_metadata.as_ref().and_then(|m| m.logo_uri.clone()),
scopes,
permission_sets,
transition_supersedes,
failed_sets,
rejected_scopes,
show_consent,
did: did.clone(),
handle: account_handle,
@@ -420,9 +459,19 @@ pub async fn consent_post(
Some(grant) => scope_resolution::Authority::Delegated(&grant.granted_scopes),
None => scope_resolution::Authority::FullSelf,
};
let effective =
scope_resolution::resolve_effective_scopes(&*state.cache, original_scope_str, authority)
.await;
let client_scope = state
.client_metadata_cache
.get(&request_data.parameters.client_id)
.await
.ok()
.and_then(|m| m.scope);
let effective = scope_resolution::resolve_effective_scopes(
&*state.cache,
original_scope_str,
authority,
client_scope.as_deref(),
)
.await;
let include_token = |nsid: &str, aud: &Option<String>| -> String {
match aud {
Some(a) => format!("include:{}?aud={}", nsid, a),
@@ -449,19 +498,7 @@ pub async fn consent_post(
),
);
}
let presented_items: Vec<String> = effective
.outcome
.passthrough
.iter()
.cloned()
.chain(
effective
.outcome
.sets
.iter()
.map(|g| include_token(&g.nsid, &g.aud)),
)
.collect();
let presented_items = effective.outcome.unexpanded_scopes();
let atproto_was_requested = presented_items.iter().any(|s| s == "atproto");
if atproto_was_requested && !form.approved_scopes.contains(&"atproto".to_string()) {
return json_error(
@@ -480,14 +517,6 @@ pub async fn consent_post(
);
}
let approved_scope_str = final_approved.join(" ");
let has_valid_scope = final_approved.iter().all(|s| is_valid_scope(s));
if !has_valid_scope {
return json_error(
StatusCode::BAD_REQUEST,
"invalid_request",
"Invalid scope format",
);
}
if form.remember {
let preferences: Vec<ScopePreference> = presented_items
.iter()
@@ -80,7 +80,7 @@ pub async fn authorize_get(
"Authorization request has expired. Please start a new request.",
);
}
let client_cache = ClientMetadataCache::new(3600);
let client_cache = &state.client_metadata_cache;
let client_name = client_cache
.get(&request_data.parameters.client_id)
.await
@@ -14,8 +14,7 @@ use tranquil_db_traits::{ScopePreference, WebauthnChallengeType};
use tranquil_pds::auth::{BareLoginIdentifier, NormalizedLoginIdentifier};
use tranquil_pds::comms::comms_repo::enqueue_2fa_code;
use tranquil_pds::oauth::{
AuthFlow, ClientMetadataCache, DeviceData, DeviceId, OAuthError, Prompt, SessionId,
db::should_show_consent,
AuthFlow, DeviceData, DeviceId, OAuthError, Prompt, SessionId, db::should_show_consent,
};
use tranquil_pds::rate_limit::{
OAuthAuthorizeLimit, OAuthRateLimited, OAuthRegisterCompleteLimit, TotpVerifyLimit,
@@ -78,15 +77,6 @@ fn is_granular_scope(s: &str) -> bool {
|| s.starts_with("identity:")
}
fn is_valid_scope(s: &str) -> bool {
s == "atproto"
|| s == "transition:generic"
|| s == "transition:chat.bsky"
|| s == "transition:email"
|| is_granular_scope(s)
|| s.starts_with("include:")
}
fn extract_device_cookie(headers: &HeaderMap) -> Option<tranquil_types::DeviceId> {
headers
.get("cookie")
@@ -1,8 +1,10 @@
use tranquil_db_traits::DbScope;
use tranquil_pds::cache::Cache;
use tranquil_pds::delegation::intersect_scopes;
use tranquil_pds::delegation::{GrantCoverage, grant_coverage, intersect_scopes};
use tranquil_pds::oauth::permission_set_resolver::expand_scopes;
use tranquil_scopes::ExpansionOutcome;
use tranquil_scopes::{
ExpansionOutcome, ParsedScope, RejectedScope, RepoScope, ScopeRejection, parse_scope,
};
pub enum Authority<'a> {
FullSelf,
@@ -20,8 +22,12 @@ pub async fn resolve_effective_scopes(
cache: &dyn Cache,
requested: &str,
authority: Authority<'_>,
client_scope: Option<&str>,
) -> EffectiveScopes {
let outcome = expand_scopes(cache, requested).await;
let mut outcome = expand_scopes(cache, requested).await;
if let Some(registered) = client_scope.map(str::trim).filter(|s| !s.is_empty()) {
reject_unregistered(&mut outcome, registered);
}
let expanded = outcome.to_scope_string();
let permitted = match authority {
Authority::FullSelf => expanded,
@@ -30,39 +36,69 @@ pub async fn resolve_effective_scopes(
EffectiveScopes { permitted, outcome }
}
fn reject_unregistered(outcome: &mut ExpansionOutcome, registered: &str) {
let mut rejected = Vec::new();
let mut keep = |scope: String| match grant_coverage(registered, &scope) {
GrantCoverage::Full => Some(scope),
GrantCoverage::Narrowed(narrowed) => {
rejected.extend(narrowed_out(&scope, &narrowed).map(|scope| RejectedScope {
scope,
reason: ScopeRejection::NotRegistered,
}));
Some(narrowed)
}
GrantCoverage::Withheld => {
rejected.push(RejectedScope {
scope,
reason: ScopeRejection::NotRegistered,
});
None
}
};
outcome.passthrough = std::mem::take(&mut outcome.passthrough)
.into_iter()
.filter_map(&mut keep)
.collect();
outcome.sets = std::mem::take(&mut outcome.sets)
.into_iter()
.filter(|group| keep(group.include_token()).is_some())
.collect();
outcome.rejected.extend(rejected);
}
/// The repo actions dropped when `requested` was narrowed to `narrowed`, as a scope of their own.
/// Only repo scopes are ever narrowed; anything else yields `None`.
fn narrowed_out(requested: &str, narrowed: &str) -> Option<String> {
let (ParsedScope::Repo(requested), ParsedScope::Repo(narrowed)) =
(parse_scope(requested), parse_scope(narrowed))
else {
return None;
};
let actions: std::collections::HashSet<_> = requested
.actions
.difference(&narrowed.actions)
.copied()
.collect();
(!actions.is_empty()).then(|| {
RepoScope {
collection: requested.collection,
actions,
}
.to_scope_string()
})
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use std::sync::Mutex;
use std::time::Duration;
use tranquil_pds::cache::{Cache, CacheError};
use tranquil_infra::MemoryCache;
use tranquil_pds::cache::Cache;
#[derive(Default)]
struct MapCache(Mutex<HashMap<String, String>>);
#[async_trait::async_trait]
impl Cache for MapCache {
async fn get(&self, k: &str) -> Option<String> {
self.0.lock().unwrap().get(k).cloned()
}
async fn set(&self, k: &str, v: &str, _t: Duration) -> Result<(), CacheError> {
self.0.lock().unwrap().insert(k.into(), v.into());
Ok(())
}
async fn delete(&self, k: &str) -> Result<(), CacheError> {
self.0.lock().unwrap().remove(k);
Ok(())
}
async fn get_bytes(&self, _k: &str) -> Option<Vec<u8>> {
None
}
async fn set_bytes(&self, _k: &str, _v: &[u8], _t: Duration) -> Result<(), CacheError> {
Ok(())
}
}
fn cache_with(nsid: &str, scopes: &str) -> MapCache {
let c = MapCache::default();
async fn cache_with(nsid: &str, scopes: &str) -> MemoryCache {
let c = MemoryCache::new();
let key = tranquil_pds::cache_keys::permission_set_key(
&tranquil_types::Nsid::new(nsid).unwrap(),
None,
@@ -74,7 +110,7 @@ mod tests {
"refreshed_at": chrono::Utc::now().timestamp(),
})
.to_string();
c.0.lock().unwrap().insert(key, json);
let _ = c.set(&key, &json, Duration::from_secs(3600)).await;
c
}
@@ -83,11 +119,13 @@ mod tests {
let c = cache_with(
"io.atcr.authFullApp",
"repo:io.atcr.manifest?action=create identity:*",
);
)
.await;
let eff = resolve_effective_scopes(
&c,
"atproto include:io.atcr.authFullApp",
Authority::FullSelf,
None,
)
.await;
assert!(eff.permitted.contains("atproto"));
@@ -104,12 +142,14 @@ mod tests {
let c = cache_with(
"io.atcr.authFullApp",
"repo:io.atcr.manifest?action=create identity:*",
);
)
.await;
let granted = DbScope::new("atproto repo:* blob:*/* account:*?action=manage").unwrap();
let eff = resolve_effective_scopes(
&c,
"atproto include:io.atcr.authFullApp",
Authority::Delegated(&granted),
None,
)
.await;
assert!(eff.permitted.contains("atproto"));
@@ -119,4 +159,106 @@ mod tests {
);
assert!(!eff.permitted.contains("identity"));
}
#[tokio::test]
async fn unrecognized_scopes_never_reach_permitted() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(&c, "atproto chat", Authority::FullSelf, None).await;
assert!(eff.permitted.split_whitespace().any(|s| s == "atproto"));
assert!(
!eff.permitted.split_whitespace().any(|s| s == "chat"),
"permitted was {:?}",
eff.permitted
);
assert_eq!(eff.outcome.rejected.len(), 1);
assert_eq!(eff.outcome.rejected[0].reason, ScopeRejection::Unrecognized);
}
#[tokio::test]
async fn scopes_absent_from_client_metadata_are_rejected() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(
&c,
"atproto identity:*",
Authority::FullSelf,
Some("atproto"),
)
.await;
assert!(!eff.permitted.split_whitespace().any(|s| s == "identity:*"));
assert_eq!(eff.outcome.rejected.len(), 1);
assert_eq!(eff.outcome.rejected[0].scope, "identity:*");
assert_eq!(
eff.outcome.rejected[0].reason,
ScopeRejection::NotRegistered
);
}
#[tokio::test]
async fn wildcard_client_registration_covers_narrower_request() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(
&c,
"atproto repo:app.bsky.feed.post?action=create",
Authority::FullSelf,
Some("atproto repo:*"),
)
.await;
assert!(eff.outcome.rejected.is_empty());
assert!(
eff.permitted
.contains("repo:app.bsky.feed.post?action=create")
);
}
#[tokio::test]
async fn absent_client_metadata_scope_constrains_nothing() {
let c = MemoryCache::new();
let eff =
resolve_effective_scopes(&c, "atproto identity:*", Authority::FullSelf, None).await;
assert!(eff.outcome.rejected.is_empty());
assert!(eff.permitted.contains("identity:*"));
}
#[tokio::test]
async fn set_expanded_scopes_bypass_the_client_registration_check() {
let c = cache_with("io.atcr.authFullApp", "identity:*").await;
let eff = resolve_effective_scopes(
&c,
"atproto include:io.atcr.authFullApp",
Authority::FullSelf,
Some("atproto include:io.atcr.authFullApp"),
)
.await;
assert!(
eff.outcome.rejected.is_empty(),
"a permission set legitimately expands to scopes the client never registered"
);
assert!(eff.permitted.contains("identity:*"));
}
#[tokio::test]
async fn actions_narrowed_out_by_client_metadata_are_reported_as_rejected() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(
&c,
"atproto repo:app.bsky.feed.post?action=create&action=delete",
Authority::FullSelf,
Some("atproto repo:*?action=create"),
)
.await;
assert!(
eff.permitted
.split_whitespace()
.any(|s| s == "repo:app.bsky.feed.post?action=create"),
"permitted was {:?}",
eff.permitted
);
assert_eq!(
eff.outcome.rejected,
vec![RejectedScope {
scope: "repo:app.bsky.feed.post?action=delete".to_string(),
reason: ScopeRejection::NotRegistered,
}]
);
}
}
@@ -13,7 +13,8 @@ use tranquil_pds::rate_limit::{LoginLimit, OAuthRateLimited, TotpVerifyLimit};
use tranquil_pds::state::AppState;
use tranquil_pds::types::PlainPassword;
use tranquil_pds::util::ClientIp;
use tranquil_types::did_doc::{extract_handle, extract_pds_endpoint};
use tranquil_types::did_doc::{PdsEndpointError, extract_handle, extract_pds_endpoint};
use tranquil_types::url_kind;
use tranquil_types::{Did, RequestId};
#[allow(clippy::result_large_err)]
@@ -231,11 +232,17 @@ pub async fn delegation_auth(
}
};
let pds_url = match extract_pds_endpoint(&did_doc) {
Some(url) => url,
None => {
let pds_url = match extract_pds_endpoint::<url_kind::Pds>(&did_doc) {
Ok(url) => url,
Err(PdsEndpointError::Missing) => {
return DelegationAuthResponse::err("Controller has no PDS endpoint");
}
Err(PdsEndpointError::Invalid(e)) => {
tracing::warn!(controller = %controller_did, error = %e, "Controller PDS endpoint rejected");
return DelegationAuthResponse::err(
"Controller PDS endpoint isn't a usable https URL",
);
}
};
let hostname = &tranquil_config::get().server.hostname;
@@ -447,7 +454,7 @@ pub async fn delegation_auth_token(
#[derive(Debug, Deserialize)]
pub struct CrossPdsCallbackParams {
pub code: tranquil_types::AuthorizationCode,
pub state: String,
pub state: tranquil_types::CrossPdsState,
pub iss: Option<String>,
}
@@ -474,7 +481,7 @@ pub async fn delegation_callback(
if let Some(ref expected_issuer) = auth_state.expected_issuer {
match &params.iss {
Some(iss) if iss != expected_issuer => {
Some(iss) if iss.as_str() != expected_issuer.as_str() => {
tracing::error!(
"Cross-PDS issuer mismatch: expected {}, got {}",
expected_issuer,
@@ -3,9 +3,8 @@ use axum::{Json, extract::State, http::HeaderMap};
use chrono::{Duration, Utc};
use serde::{Deserialize, Serialize};
use tranquil_pds::oauth::{
AuthorizationRequestParameters, ClientAuth, ClientMetadataCache, CodeChallengeMethod,
OAuthError, Prompt, RequestData, RequestId, ResponseMode, ResponseType,
scopes::{ParsedScope, parse_scope},
AuthorizationRequestParameters, ClientAuth, CodeChallengeMethod, OAuthError, Prompt,
RequestData, RequestId, ResponseMode, ResponseType,
};
use tranquil_pds::rate_limit::{OAuthParLimit, OAuthRateLimited};
use tranquil_pds::state::AppState;
@@ -80,11 +79,11 @@ pub async fn pushed_authorization_request(
.ok_or_else(|| OAuthError::InvalidRequest("code_challenge is required".to_string()))?;
let code_challenge_method =
parse_code_challenge_method(request.code_challenge_method.as_deref())?;
let client_cache = ClientMetadataCache::new(3600);
let client_cache = &state.client_metadata_cache;
let client_metadata = client_cache.get(&request.client_id).await?;
client_cache.validate_redirect_uri(&client_metadata, &request.redirect_uri)?;
let client_auth = determine_client_auth(&request)?;
let validated_scope = validate_scope(&request.scope, &client_metadata)?;
let validated_scope = normalize_scope(&request.scope)?;
let request_id = RequestId::generate();
let expires_at = Utc::now() + Duration::seconds(PAR_EXPIRY_SECONDS);
let response_mode = parse_response_mode(request.response_mode.as_deref())?;
@@ -165,10 +164,7 @@ fn determine_client_auth(request: &ParRequest) -> Result<ClientAuth, OAuthError>
Ok(ClientAuth::None)
}
fn validate_scope(
requested_scope: &Option<String>,
client_metadata: &tranquil_pds::oauth::ClientMetadata,
) -> Result<Option<String>, OAuthError> {
fn normalize_scope(requested_scope: &Option<String>) -> Result<Option<String>, OAuthError> {
let scope_str = match requested_scope {
Some(s) if !s.is_empty() => s,
_ => return Ok(Some("atproto".to_string())),
@@ -177,80 +173,14 @@ fn validate_scope(
if requested_scopes.is_empty() {
return Ok(Some("atproto".to_string()));
}
if let Some(unknown) = requested_scopes
.iter()
.find(|s| matches!(parse_scope(s), ParsedScope::Unknown(_)))
{
return Err(OAuthError::InvalidScope(format!(
"Unsupported scope: {}",
unknown
)));
}
let has_transition = requested_scopes.iter().any(|s| {
matches!(
parse_scope(s),
ParsedScope::TransitionGeneric
| ParsedScope::TransitionChat
| ParsedScope::TransitionEmail
)
});
let has_granular = requested_scopes.iter().any(|s| {
matches!(
parse_scope(s),
ParsedScope::Repo(_)
| ParsedScope::Blob(_)
| ParsedScope::Rpc(_)
| ParsedScope::Account(_)
| ParsedScope::Identity(_)
| ParsedScope::Include(_)
)
});
if has_transition && has_granular {
if !requested_scopes.contains(&"atproto") {
return Err(OAuthError::InvalidScope(
"Cannot mix transition scopes with granular scopes. Use either transition:* scopes OR granular scopes (repo:*, blob:*, rpc:*, account:*, include:*), not both.".to_string()
"The atproto scope is required".to_string(),
));
}
if let Some(client_scope) = &client_metadata.scope {
let client_scopes: Vec<&str> = client_scope.split_whitespace().collect();
if let Some(unregistered) = requested_scopes
.iter()
.find(|scope| !client_scopes.iter().any(|cs| scope_matches(cs, scope)))
{
return Err(OAuthError::InvalidScope(format!(
"Scope '{}' not registered for this client",
unregistered
)));
}
}
Ok(Some(requested_scopes.join(" ")))
}
fn scope_matches(client_scope: &str, requested_scope: &str) -> bool {
if client_scope == requested_scope {
return true;
}
fn get_resource_type(scope: &str) -> &str {
let base = scope.split('?').next().unwrap_or(scope);
base.split(':').next().unwrap_or(base)
}
let client_type = get_resource_type(client_scope);
let requested_type = get_resource_type(requested_scope);
if client_type == requested_type {
let client_base = client_scope.split('?').next().unwrap_or(client_scope);
if client_base.contains('*') {
return true;
}
}
false
}
fn parse_response_type(value: &str) -> Result<ResponseType, OAuthError> {
match value {
"code" => Ok(ResponseType::Code),
@@ -300,3 +230,45 @@ fn parse_prompt(value: Option<&str>) -> Result<Option<Prompt>, OAuthError> {
))),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn normalized(scope: Option<&str>) -> Result<Option<String>, OAuthError> {
normalize_scope(&scope.map(str::to_string))
}
#[test]
fn absent_or_blank_scope_defaults_to_atproto() {
assert_eq!(normalized(None).unwrap().as_deref(), Some("atproto"));
assert_eq!(normalized(Some("")).unwrap().as_deref(), Some("atproto"));
assert_eq!(normalized(Some(" ")).unwrap().as_deref(), Some("atproto"));
}
#[test]
fn scope_without_atproto_is_invalid() {
assert!(matches!(
normalized(Some("repo:*?action=create blob:*/*")),
Err(OAuthError::InvalidScope(_))
));
}
#[test]
fn atproto_need_not_come_first() {
assert_eq!(
normalized(Some("repo:*?action=create atproto"))
.unwrap()
.as_deref(),
Some("repo:*?action=create atproto")
);
}
#[test]
fn unrecognized_scopes_still_pass_par() {
assert_eq!(
normalized(Some("atproto chat")).unwrap().as_deref(),
Some("atproto chat")
);
}
}
@@ -8,8 +8,7 @@ use chrono::{Duration, Utc};
use tranquil_db_traits::RefreshTokenLookup;
use tranquil_pds::config::AuthConfig;
use tranquil_pds::oauth::{
AuthFlow, ClientAuth, ClientMetadataCache, DPoPVerifier, OAuthError, RefreshToken, TokenData,
TokenId,
AuthFlow, ClientAuth, DPoPVerifier, OAuthError, RefreshToken, TokenData, TokenId,
db::{enforce_token_limit_for_user, lookup_refresh_token},
verify_client_auth,
};
@@ -63,7 +62,7 @@ pub async fn handle_authorization_code_grant(
return Err(OAuthError::InvalidGrant("client_id mismatch".to_string()));
}
let did = authorized.did.clone();
let client_metadata_cache = ClientMetadataCache::new(3600);
let client_metadata_cache = &state.client_metadata_cache;
let client_metadata = client_metadata_cache.get(&authorized.client_id).await?;
let client_auth = match &request.client_auth {
RequestClientAuth::PrivateKeyJwt {
@@ -85,7 +84,7 @@ pub async fn handle_authorization_code_grant(
},
RequestClientAuth::None { .. } => ClientAuth::None,
};
verify_client_auth(&client_metadata_cache, &client_metadata, &client_auth).await?;
verify_client_auth(client_metadata_cache, &client_metadata, &client_auth).await?;
verify_pkce(&authorized.parameters.code_challenge, &code_verifier)?;
if let Some(req_redirect_uri) = &redirect_uri
&& req_redirect_uri != &authorized.parameters.redirect_uri
@@ -158,6 +157,7 @@ pub async fn handle_authorization_code_grant(
&*state.cache,
requested_for_resolve,
authority,
client_metadata.scope.as_deref(),
)
.await;
if !effective.outcome.failures.is_empty() {
@@ -202,7 +202,10 @@ pub async fn handle_authorization_code_grant(
details: None,
code: None,
current_refresh_token: Some(refresh_token.clone()),
scope: requested_scope.clone(),
// Filtered but unexpanded: a remembered consent skips the consent screen, so the raw
// request can still hold scopes the client no longer registers. Sets stay as `include:`
// tokens so refresh re-resolves them.
scope: Some(effective.outcome.unexpanded_scopes().join(" ")),
controller_did: controller_did.clone(),
};
state
@@ -275,10 +278,13 @@ async fn recompute_resolved_scope(
Some(g) => crate::endpoints::authorize::scope_resolution::Authority::Delegated(g),
None => crate::endpoints::authorize::scope_resolution::Authority::FullSelf,
};
// No client metadata check here: `token_data.scope` was already filtered against it when
// the token was issued, so there is nothing for a re-check to remove.
let effective = crate::endpoints::authorize::scope_resolution::resolve_effective_scopes(
&*state.cache,
requested,
authority,
None,
)
.await;
if !effective.outcome.failures.is_empty() {
@@ -43,7 +43,8 @@ pub fn create_access_token_with_delegation(
let issuer = format!("https://{}", pds_hostname);
let now = Utc::now().timestamp();
let exp = now + ACCESS_TOKEN_EXPIRY_SECONDS;
let actual_scope = scope.unwrap_or("atproto");
let actual_scope = tranquil_pds::auth::encode_scope(scope.unwrap_or("atproto"))
.map_err(|_| OAuthError::InvalidScope("Scope too large".to_string()))?;
let mut payload = json!({
"iss": issuer,
"sub": sub.as_str(),
@@ -789,13 +789,16 @@ pub async fn check_handle_available(
}
};
let available_domains = tranquil_config::get().server.available_user_domain_list();
if let Some(ref d) = query.domain
&& !available_domains.iter().any(|ad| ad == d)
let available_domains = tranquil_pds::handle::ServiceDomains::for_user_handles();
if let Some(d) = &query.domain
&& !available_domains.contains(d.as_str())
{
return Err(ApiError::InvalidRequest("Unknown user domain".into()));
}
let domain = query.domain.as_deref().unwrap_or(&available_domains[0]);
let domain = query
.domain
.as_deref()
.unwrap_or_else(|| available_domains.primary().as_str());
let full_handle = format!("{}.{}", validated, domain);
let handle: tranquil_pds::types::Handle = match full_handle.parse() {
Ok(h) => h,
@@ -882,34 +885,33 @@ pub async fn complete_registration(
let cfg = tranquil_config::get();
let hostname = &cfg.server.hostname;
let available_domains = cfg.server.available_user_domain_list();
let available_domains = tranquil_pds::handle::ServiceDomains::for_user_handles();
let matched_domain = available_domains
.iter()
.filter(|d| input.handle.ends_with(&format!(".{}", d)))
.max_by_key(|d| d.len());
let split = available_domains.split_handle(&input.handle);
let handle: tranquil_pds::types::Handle =
if !input.handle.contains('.') || matched_domain.is_some() {
let handle_to_validate = match matched_domain {
Some(domain) => input
.handle
.strip_suffix(&format!(".{}", domain))
.unwrap_or(&input.handle),
None => &input.handle,
};
match tranquil_pds::api::validation::validate_short_handle(handle_to_validate) {
Ok(h) => format!("{}.{}", h, matched_domain.unwrap_or(&available_domains[0]))
.parse()
.map_err(|_| ApiError::InvalidHandle(None))?,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
} else {
match tranquil_pds::api::validation::validate_full_domain_handle(&input.handle) {
Ok(h) => h,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
let handle: tranquil_pds::types::Handle = if !input.handle.contains('.') || split.is_some() {
let handle_to_validate = match split {
Some((_domain, short)) => short,
None => input.handle.as_str(),
};
match tranquil_pds::api::validation::validate_short_handle(handle_to_validate) {
Ok(h) => format!(
"{}.{}",
h,
split
.map(|(d, _)| d)
.unwrap_or_else(|| available_domains.primary())
)
.parse()
.map_err(|_| ApiError::InvalidHandle(None))?,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
} else {
match tranquil_pds::api::validation::validate_full_domain_handle(&input.handle) {
Ok(h) => h,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
};
let verification_channel = input
.verification_channel
@@ -1209,20 +1211,23 @@ pub async fn complete_registration(
tracing::warn!("Failed to sequence account event for {}: {}", did, e);
}
let profile_record = json!({
"$type": "app.bsky.actor.profile",
"displayName": handle.as_str()
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
&state,
&did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
#[cfg(feature = "bsky")]
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
let profile_record = json!({
"$type": "app.bsky.actor.profile",
"displayName": handle.as_str()
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
&state,
&did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
};
}
let app_password = generate_app_password();
+1
View File
@@ -6,6 +6,7 @@ license.workspace = true
[dependencies]
tranquil-types = { workspace = true }
tranquil-infra = { workspace = true, features = ["cache-keys"] }
anyhow = { workspace = true }
sqlx = { workspace = true }
+108 -90
View File
@@ -1,12 +1,19 @@
use reqwest::Client;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;
use std::time::Duration;
use crate::OAuthError;
use crate::types::ClientAuth;
use tranquil_types::ClientId;
use tranquil_infra::cache_keys::{
oauth_client_jwks_cooldown_key, oauth_client_jwks_key, oauth_client_meta_key,
};
use tranquil_infra::{Cache, cached_json, write_json};
use tranquil_types::{
ClientId, JwksUri, ReachPolicy, dns_guard, redirect_policy, url_reach_permits,
};
const JWKS_REFRESH_COOLDOWN: Duration = Duration::from_secs(60);
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ClientMetadata {
@@ -30,8 +37,12 @@ pub struct ClientMetadata {
pub dpop_bound_access_tokens: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub jwks: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
pub jwks_uri: Option<String>,
#[serde(
default,
skip_serializing_if = "Option::is_none",
deserialize_with = "tranquil_types::http_url::deserialize_optional"
)]
pub jwks_uri: Option<JwksUri>,
#[serde(skip_serializing_if = "Option::is_none")]
pub application_type: Option<String>,
}
@@ -58,33 +69,24 @@ impl Default for ClientMetadata {
#[derive(Clone)]
pub struct ClientMetadataCache {
cache: Arc<RwLock<HashMap<String, CachedMetadata>>>,
jwks_cache: Arc<RwLock<HashMap<String, CachedJwks>>>,
cache: Arc<dyn Cache>,
http_client: Client,
cache_ttl_secs: u64,
}
struct CachedMetadata {
metadata: ClientMetadata,
cached_at: std::time::Instant,
}
struct CachedJwks {
jwks: serde_json::Value,
cached_at: std::time::Instant,
cache_ttl: Duration,
fetch_policy: ReachPolicy,
}
impl ClientMetadataCache {
pub fn new(cache_ttl_secs: u64) -> Self {
pub fn new(cache: Arc<dyn Cache>, cache_ttl: Duration, fetch_policy: ReachPolicy) -> Self {
Self {
cache: Arc::new(RwLock::new(HashMap::new())),
jwks_cache: Arc::new(RwLock::new(HashMap::new())),
cache,
http_client: {
let builder = Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.pool_max_idle_per_host(10)
.pool_idle_timeout(std::time::Duration::from_secs(90))
.redirect(redirect_policy(fetch_policy))
.dns_resolver(dns_guard(fetch_policy))
.user_agent(concat!(
"Tranquil-PDS/",
env!("CARGO_PKG_VERSION"),
@@ -92,9 +94,12 @@ impl ClientMetadataCache {
));
#[cfg(feature = "native-tls-roots")]
let builder = builder.danger_accept_invalid_certs(true);
builder.build().unwrap_or_else(|_| Client::new())
builder
.build()
.expect("failed to build client metadata HTTP client")
},
cache_ttl_secs,
cache_ttl,
fetch_policy,
}
}
@@ -139,7 +144,7 @@ impl ClientMetadataCache {
response_types: vec!["code".into()],
scope,
token_endpoint_auth_method: Some("none".into()),
dpop_bound_access_tokens: Some(false),
dpop_bound_access_tokens: Some(true),
jwks: None,
jwks_uri: None,
application_type: Some("native".into()),
@@ -150,26 +155,13 @@ impl ClientMetadataCache {
if Self::is_loopback_client(client_id) {
return Self::build_loopback_metadata(client_id);
}
{
let cache = self.cache.read().await;
if let Some(cached) = cache.get(client_id.as_str())
&& cached.cached_at.elapsed().as_secs() < self.cache_ttl_secs
{
return Ok(cached.metadata.clone());
}
}
let metadata = self.fetch_metadata(client_id).await?;
{
let mut cache = self.cache.write().await;
cache.insert(
client_id.to_string(),
CachedMetadata {
metadata: metadata.clone(),
cached_at: std::time::Instant::now(),
},
);
}
Ok(metadata)
cached_json(
self.cache.as_ref(),
&oauth_client_meta_key(client_id),
self.cache_ttl,
|| self.fetch_metadata(client_id),
)
.await
}
pub async fn get_jwks(
@@ -181,43 +173,57 @@ impl ClientMetadataCache {
}
let jwks_uri = metadata.jwks_uri.as_ref().ok_or_else(|| {
OAuthError::InvalidClient(
"Client using private_key_jwt must have jwks or jwks_uri".to_string(),
"Client using private_key_jwt must have jwks or a usable jwks_uri".to_string(),
)
})?;
{
let cache = self.jwks_cache.read().await;
if let Some(cached) = cache.get(jwks_uri)
&& cached.cached_at.elapsed().as_secs() < self.cache_ttl_secs
{
return Ok(cached.jwks.clone());
cached_json(
self.cache.as_ref(),
&oauth_client_jwks_key(jwks_uri),
self.cache_ttl,
|| self.fetch_jwks(jwks_uri),
)
.await
}
async fn refresh_jwks(
&self,
metadata: &ClientMetadata,
) -> Result<Option<serde_json::Value>, OAuthError> {
match (&metadata.jwks, &metadata.jwks_uri) {
(None, Some(jwks_uri)) => {
let cooldown_key = oauth_client_jwks_cooldown_key(jwks_uri);
if self.cache.get(&cooldown_key).await.is_some() {
return Ok(None);
}
let _ = self
.cache
.set(&cooldown_key, "1", JWKS_REFRESH_COOLDOWN)
.await;
self.fetch_and_store_jwks(jwks_uri).await.map(Some)
}
_ => Ok(None),
}
}
async fn fetch_and_store_jwks(
&self,
jwks_uri: &JwksUri,
) -> Result<serde_json::Value, OAuthError> {
let jwks = self.fetch_jwks(jwks_uri).await?;
{
let mut cache = self.jwks_cache.write().await;
cache.insert(
jwks_uri.clone(),
CachedJwks {
jwks: jwks.clone(),
cached_at: std::time::Instant::now(),
},
);
}
write_json(
self.cache.as_ref(),
&oauth_client_jwks_key(jwks_uri),
&jwks,
self.cache_ttl,
)
.await;
Ok(jwks)
}
async fn fetch_jwks(&self, jwks_uri: &str) -> Result<serde_json::Value, OAuthError> {
if !jwks_uri.starts_with("https://")
&& (!jwks_uri.starts_with("http://")
|| (!jwks_uri.contains("localhost") && !jwks_uri.contains("127.0.0.1")))
{
return Err(OAuthError::InvalidClient(
"jwks_uri must use https (except for localhost)".to_string(),
));
}
async fn fetch_jwks(&self, jwks_uri: &JwksUri) -> Result<serde_json::Value, OAuthError> {
let response = self
.http_client
.get(jwks_uri)
.get(jwks_uri.as_str())
.header("Accept", "application/json")
.send()
.await
@@ -243,22 +249,16 @@ impl ClientMetadataCache {
}
async fn fetch_metadata(&self, client_id: &ClientId) -> Result<ClientMetadata, OAuthError> {
if !client_id.starts_with("http://") && !client_id.starts_with("https://") {
let url = reqwest::Url::parse(client_id)
.map_err(|_| OAuthError::InvalidClient("client_id must be a URL".to_string()))?;
if !url_reach_permits(&url, self.fetch_policy) {
return Err(OAuthError::InvalidClient(
"client_id must be a URL".to_string(),
));
}
if client_id.starts_with("http://")
&& !client_id.contains("localhost")
&& !client_id.contains("127.0.0.1")
{
return Err(OAuthError::InvalidClient(
"Non-localhost client_id must use https".to_string(),
"client_id must be an https URL inside the allowed host reach".to_string(),
));
}
let response = self
.http_client
.get(client_id.as_str())
.get(url)
.header("Accept", "application/json")
.send()
.await
@@ -514,7 +514,29 @@ async fn verify_private_key_jwt_async(
"client_assertion iat is in the future".to_string(),
));
}
let signing_input = format!("{}.{}", parts[0], parts[1]);
let signature_bytes = URL_SAFE_NO_PAD
.decode(parts[2])
.map_err(|_| OAuthError::InvalidClient("Invalid signature encoding".to_string()))?;
let jwks = cache.get_jwks(metadata).await?;
match verify_assertion_signature(&jwks, kid, alg, &signing_input, &signature_bytes) {
Ok(()) => Ok(()),
Err(cached_failure) => match cache.refresh_jwks(metadata).await {
Ok(Some(fresh)) => {
verify_assertion_signature(&fresh, kid, alg, &signing_input, &signature_bytes)
}
Ok(None) | Err(_) => Err(cached_failure),
},
}
}
fn verify_assertion_signature(
jwks: &serde_json::Value,
kid: Option<&str>,
alg: &str,
signing_input: &str,
signature: &[u8],
) -> Result<(), OAuthError> {
let keys = jwks
.get("keys")
.and_then(|k| k.as_array())
@@ -531,10 +553,6 @@ async fn verify_private_key_jwt_async(
"No matching key found in client JWKS".to_string(),
));
}
let signing_input = format!("{}.{}", parts[0], parts[1]);
let signature_bytes = URL_SAFE_NO_PAD
.decode(parts[2])
.map_err(|_| OAuthError::InvalidClient("Invalid signature encoding".to_string()))?;
matching_keys
.into_iter()
.filter(|key| {
@@ -544,12 +562,12 @@ async fn verify_private_key_jwt_async(
.find_map(|key| {
let kty = key.get("kty").and_then(|k| k.as_str()).unwrap_or("");
match (alg, kty) {
("ES256", "EC") => verify_es256(key, &signing_input, &signature_bytes).ok(),
("ES384", "EC") => verify_es384(key, &signing_input, &signature_bytes).ok(),
("ES256", "EC") => verify_es256(key, signing_input, signature).ok(),
("ES384", "EC") => verify_es384(key, signing_input, signature).ok(),
("RS256" | "RS384" | "RS512", "RSA") => {
verify_rsa(alg, key, &signing_input, &signature_bytes).ok()
verify_rsa(alg, key, signing_input, signature).ok()
}
("EdDSA", "OKP") => verify_eddsa(key, &signing_input, &signature_bytes).ok(),
("EdDSA", "OKP") => verify_eddsa(key, signing_input, signature).ok(),
_ => None,
}
})
+5 -5
View File
@@ -1,7 +1,7 @@
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use serde_json::Value as JsonValue;
use tranquil_types::{ClientId, Did};
use tranquil_types::{AuthServerEndpoint, ClientId, Did, Issuer};
pub use tranquil_types::{AuthorizationCode, DeviceId, RefreshToken, RequestId, TokenId};
@@ -195,9 +195,9 @@ pub struct ProtectedResourceMetadata {
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AuthorizationServerMetadata {
pub issuer: String,
pub authorization_endpoint: String,
pub token_endpoint: String,
pub issuer: Issuer,
pub authorization_endpoint: AuthServerEndpoint,
pub token_endpoint: AuthServerEndpoint,
pub jwks_uri: String,
pub registration_endpoint: Option<String>,
pub scopes_supported: Option<Vec<String>>,
@@ -206,7 +206,7 @@ pub struct AuthorizationServerMetadata {
pub grant_types_supported: Option<Vec<String>>,
pub token_endpoint_auth_methods_supported: Option<Vec<String>>,
pub code_challenge_methods_supported: Option<Vec<String>>,
pub pushed_authorization_request_endpoint: Option<String>,
pub pushed_authorization_request_endpoint: Option<AuthServerEndpoint>,
pub require_pushed_authorization_requests: Option<bool>,
pub dpop_signing_alg_values_supported: Option<Vec<String>>,
pub authorization_response_iss_parameter_supported: Option<bool>,
+12 -14
View File
@@ -7,7 +7,6 @@ license.workspace = true
[dependencies]
tranquil-types = { workspace = true }
tranquil-config = { workspace = true }
tranquil-crypto = { workspace = true }
tranquil-storage = { workspace = true }
tranquil-cache = { workspace = true }
tranquil-repo = { workspace = true }
@@ -16,7 +15,7 @@ tranquil-auth = { workspace = true }
tranquil-oauth = { workspace = true }
tranquil-comms = { workspace = true }
tranquil-signal = { workspace = true }
tranquil-db = { workspace = true }
tranquil-db = { workspace = true, features = ["postgres"] }
tranquil-db-traits = { workspace = true }
tranquil-store = { workspace = true }
tranquil-lexicon = { workspace = true, features = ["resolve"] }
@@ -29,13 +28,11 @@ axum = { workspace = true }
base32 = { workspace = true }
base64 = { workspace = true }
bcrypt = { workspace = true }
bs58 = { workspace = true }
bytes = { workspace = true }
chrono = { workspace = true }
cid = { workspace = true }
ed25519-dalek = { workspace = true }
futures = { workspace = true }
hex = { workspace = true }
futures-util = { workspace = true }
governor = { workspace = true }
hickory-resolver = { workspace = true }
@@ -43,7 +40,6 @@ hkdf = { workspace = true }
hmac = { workspace = true }
http = { workspace = true }
image = { workspace = true }
infer = { workspace = true }
ipld-core = { workspace = true }
iroh-car = { workspace = true }
jacquard-common = { workspace = true }
@@ -57,7 +53,6 @@ multihash = { workspace = true }
p256 = { workspace = true }
parking_lot = { workspace = true }
rand = { workspace = true }
redis = { workspace = true, optional = true }
regex = { workspace = true }
reqwest = { workspace = true }
serde = { workspace = true }
@@ -70,7 +65,6 @@ sqlx = { workspace = true }
subtle = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tokio-tungstenite = { workspace = true }
tokio-util = { workspace = true }
tower = { workspace = true }
tower-http = { workspace = true }
@@ -80,20 +74,19 @@ urlencoding = { workspace = true }
uuid = { workspace = true }
webauthn-rs = { workspace = true }
webauthn-rs-proto = { workspace = true }
zip = { workspace = true }
aws-config = { workspace = true, optional = true }
aws-sdk-s3 = { workspace = true, optional = true }
[features]
default = ["frontend", "s3", "valkey"]
bsky = ["bsky-support"]
bsky-support = []
external-infra = []
s3-storage = ["tranquil-storage/s3", "dep:aws-config", "dep:aws-sdk-s3"]
s3 = ["s3-storage"]
valkey = ["tranquil-cache/valkey", "dep:redis"]
postgres = ["tranquil-db/postgres"]
s3 = ["tranquil-storage/s3"]
valkey = ["tranquil-cache/valkey"]
frontend = []
native-tls-roots = ["tranquil-oauth/native-tls-roots"]
[dev-dependencies]
tranquil-infra = { workspace = true, features = ["testing"] }
tempfile = "3"
ciborium = { workspace = true }
ctor = { workspace = true }
@@ -105,3 +98,8 @@ tranquil-api = { workspace = true }
tranquil-oauth-server = { workspace = true }
tracing-subscriber = { workspace = true, features = ["env-filter"] }
wiremock = { workspace = true }
hex = { workspace = true }
tokio-tungstenite = { workspace = true }
aws-config = { workspace = true }
aws-sdk-s3 = { workspace = true }
redis = { workspace = true }
+1 -2
View File
@@ -763,8 +763,7 @@ impl From<crate::api::validation::HandleValidationError> for ApiError {
HandleValidationError::BannedWord => {
Self::InvalidHandle(Some("Inappropriate language in handle".to_string()))
}
HandleValidationError::UnusableHandleDomain
| HandleValidationError::NoHandleDomains => Self::InternalError(Some(e.to_string())),
HandleValidationError::UnusableHandleDomain => Self::InternalError(Some(e.to_string())),
_ => Self::InvalidHandle(Some(e.to_string())),
}
}
+20 -10
View File
@@ -5,7 +5,7 @@ use std::sync::LazyLock;
use crate::api::error::ApiError;
use crate::api::proxy_client::proxy_client;
use crate::state::AppState;
use crate::types::{Did, Nsid};
use crate::types::{Did, DidRef, Nsid};
use crate::util::get_header_str;
use axum::{
body::Bytes,
@@ -19,9 +19,7 @@ use tower::{Service, util::BoxCloneSyncService};
use tracing::{error, info, warn};
static PROTECTED_METHODS: LazyLock<HashSet<&'static str>> = LazyLock::new(|| {
[
"app.bsky.actor.getPreferences",
"app.bsky.actor.putPreferences",
let mut methods: HashSet<&str> = [
"com.atproto.admin.deleteAccount",
"com.atproto.admin.disableAccountInvites",
"com.atproto.admin.disableInviteCodes",
@@ -103,7 +101,13 @@ static PROTECTED_METHODS: LazyLock<HashSet<&'static str>> = LazyLock::new(|| {
"com.atproto.temp.dereferenceScope",
]
.into_iter()
.collect()
.collect();
// BSKY: the Bluesky preferences API must be implemented by PDSs
if cfg!(feature = "bsky-support") {
methods.insert("app.bsky.actor.getPreferences");
methods.insert("app.bsky.actor.putPreferences");
};
methods
});
fn is_protected_method(method: &str) -> bool {
@@ -111,6 +115,7 @@ fn is_protected_method(method: &str) -> bool {
}
/// Fetch the `feed` generator record from the AppView and return its `did`.
#[cfg(feature = "bsky-support")]
async fn resolve_feed_generator_did(appview_url: &str, query: Option<&str>) -> Option<Did> {
#[derive(serde::Deserialize)]
struct GetFeedQuery {
@@ -201,6 +206,7 @@ impl<S: Service<Request, Response = Response, Error = Infallible>> Service<Reque
}
// If the age assurance override is set and this is an age assurance call then we dont want to proxy even if the client requests it
#[cfg(feature = "bsky")]
if tranquil_config::get().server.age_assurance_override
&& (path.ends_with("app.bsky.ageassurance.getState")
|| path.ends_with("app.bsky.unspecced.getAgeAssuranceState"))
@@ -328,7 +334,8 @@ async fn proxy_handler(
},
};
// getFeed must be audienced to the feed generator, not the AppView.
// BSKY: getFeed must be audienced to the feed generator, not the AppView.
#[cfg(feature = "bsky-support")]
let (token_aud, token_lxm) = if method == "app.bsky.feed.getFeed" {
match resolve_feed_generator_did(&resolved.url, query.as_deref()).await {
Some(feed_did) => (
@@ -339,21 +346,24 @@ async fn proxy_handler(
),
None => {
warn!(
"getFeed proxy: could not resolve feed generator DID; refusing \
to mint an AppView-audienced token"
"getFeed proxy refuses to mint an AppView-audienced token \
because feed generator DID resolution failed"
);
return ApiError::InvalidRequest("Could not resolve feed".into())
return ApiError::InvalidRequest("Couldn't resolve feed".into())
.into_response();
}
}
} else {
(resolved.did.clone(), method_nsid.clone())
};
#[cfg(not(feature = "bsky-support"))]
let (token_aud, token_lxm) = (resolved.did.clone(), method_nsid.clone());
match crate::auth::create_service_token(
&auth_user.did,
&token_aud,
&DidRef::from(token_aud),
Some(&token_lxm),
None,
&key_bytes,
) {
Ok(new_token) => {
+11 -1
View File
@@ -147,7 +147,9 @@ impl std::fmt::Display for SsrfError {
}
impl std::error::Error for SsrfError {}
// TODO: update to match https://github.com/bluesky-social/atproto/blob/main/packages/pds/src/pipethrough.ts
// Currently spec says nothing at all!! about forwarding headers during proxying and https://github.com/bluesky-social/atproto/discussions/2350
#[cfg(feature = "bsky-support")]
pub static HEADERS_TO_FORWARD: LazyLock<[HeaderName; 4]> = LazyLock::new(|| {
[
HeaderName::from_static("accept-language"),
@@ -156,6 +158,14 @@ pub static HEADERS_TO_FORWARD: LazyLock<[HeaderName; 4]> = LazyLock::new(|| {
http::header::CONTENT_TYPE,
]
});
#[cfg(not(feature = "bsky-support"))]
pub static HEADERS_TO_FORWARD: LazyLock<[HeaderName; 3]> = LazyLock::new(|| {
[
HeaderName::from_static("accept-language"),
crate::util::HEADER_ATPROTO_ACCEPT_LABELERS,
http::header::CONTENT_TYPE,
]
});
pub static RESPONSE_HEADERS_TO_FORWARD: LazyLock<[HeaderName; 6]> = LazyLock::new(|| {
[
crate::util::HEADER_ATPROTO_REPO_REV,
+9 -22
View File
@@ -111,7 +111,6 @@ pub enum HandleValidationError {
InvalidSyntax,
DisallowedTld,
UnusableHandleDomain,
NoHandleDomains,
}
impl std::fmt::Display for HandleValidationError {
@@ -143,9 +142,6 @@ impl std::fmt::Display for HandleValidationError {
f,
"This server's handle domain has a reserved TLD, so no handle under it is a valid atproto handle"
),
Self::NoHandleDomains => {
write!(f, "No handle domains are configured on this server")
}
}
}
}
@@ -215,21 +211,14 @@ pub fn validate_short_handle(handle: &str) -> Result<String, HandleValidationErr
}
pub fn resolve_handle_input(input: &str) -> Result<Handle, HandleValidationError> {
let available_domains = tranquil_config::get().server.available_user_domain_list();
let matched_domain = available_domains
.iter()
.filter(|d| input.ends_with(&format!(".{}", d)))
.max_by_key(|d| d.len());
let domains = crate::handle::ServiceDomains::for_user_handles();
let split = domains.split_handle(input);
if !input.contains('.') || matched_domain.is_some() {
let handle_to_validate = match matched_domain {
Some(domain) => input.strip_suffix(&format!(".{}", domain)).unwrap_or(input),
None => input,
};
let validated = validate_short_handle(handle_to_validate)?;
let domain = matched_domain
.or_else(|| available_domains.first())
.ok_or(HandleValidationError::NoHandleDomains)?;
if !input.contains('.') || split.is_some() {
let (short, domain) = split
.map(|(domain, short)| (short, domain))
.unwrap_or((input, domains.primary()));
let validated = validate_short_handle(short)?;
let handle = Handle::new(format!("{}.{}", validated, domain))
.map_err(|_| HandleValidationError::InvalidSyntax)?;
match handle.has_disallowed_tld() {
@@ -246,11 +235,9 @@ pub fn domain_forms_valid_handles(domain: &str) -> bool {
}
pub fn warn_unusable_handle_domains() {
tranquil_config::get()
.server
.user_handle_domain_list()
crate::handle::ServiceDomains::for_user_handles()
.iter()
.filter(|domain| !domain_forms_valid_handles(domain))
.filter(|domain| !domain_forms_valid_handles(domain.as_str()))
.for_each(|domain| {
tracing::error!(
domain = %domain,
+14 -92
View File
@@ -2,32 +2,14 @@ use serde::{Deserialize, Serialize};
use std::time::Duration;
use crate::cache::Cache;
use crate::cache_keys::email_token_key;
use crate::types::Did;
use crate::util::{generate_token_code, normalize_token_code};
pub use tranquil_types::EmailTokenPurpose;
const TOKEN_TTL_SECS: u64 = 900;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EmailTokenPurpose {
UpdateEmail,
ConfirmEmail,
DeleteAccount,
ResetPassword,
PlcOperation,
}
impl EmailTokenPurpose {
fn as_str(&self) -> &'static str {
match self {
Self::UpdateEmail => "update_email",
Self::ConfirmEmail => "confirm_email",
Self::DeleteAccount => "delete_account",
Self::ResetPassword => "reset_password",
Self::PlcOperation => "plc_operation",
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
struct TokenData {
token: String,
@@ -42,10 +24,6 @@ pub enum TokenError {
ExpiredToken,
}
fn cache_key(did: &Did, purpose: EmailTokenPurpose) -> String {
format!("email_token:{}:{}", purpose.as_str(), did)
}
fn current_timestamp() -> u64 {
u64::try_from(chrono::Utc::now().timestamp()).unwrap_or(0)
}
@@ -69,7 +47,7 @@ pub async fn create_email_token(
cache
.set(
&cache_key(did, purpose),
&email_token_key(did, purpose),
&json,
Duration::from_secs(TOKEN_TTL_SECS),
)
@@ -89,7 +67,7 @@ pub async fn validate_email_token(
return Err(TokenError::CacheUnavailable);
}
let key = cache_key(did, purpose);
let key = email_token_key(did, purpose);
let json = cache.get(&key).await.ok_or(TokenError::InvalidToken)?;
let data: TokenData = serde_json::from_str(&json).map_err(|_| TokenError::InvalidToken)?;
@@ -112,7 +90,7 @@ pub async fn validate_email_token(
}
pub async fn delete_email_token(cache: &dyn Cache, did: &Did, purpose: EmailTokenPurpose) {
let _ = cache.delete(&cache_key(did, purpose)).await;
let _ = cache.delete(&email_token_key(did, purpose)).await;
}
fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
@@ -128,67 +106,11 @@ fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
#[cfg(test)]
mod tests {
use super::*;
use crate::cache::CacheError;
use async_trait::async_trait;
use std::collections::HashMap;
use std::sync::Mutex;
struct MockCache {
data: Mutex<HashMap<String, (String, u64)>>,
}
impl MockCache {
fn new() -> Self {
Self {
data: Mutex::new(HashMap::new()),
}
}
}
#[async_trait]
impl Cache for MockCache {
async fn get(&self, key: &str) -> Option<String> {
let data = self.data.lock().unwrap();
let now = current_timestamp();
data.get(key)
.filter(|(_, exp)| *exp > now)
.map(|(v, _)| v.clone())
}
async fn set(&self, key: &str, value: &str, ttl: Duration) -> Result<(), CacheError> {
let mut data = self.data.lock().unwrap();
let expires = current_timestamp() + ttl.as_secs();
data.insert(key.to_string(), (value.to_string(), expires));
Ok(())
}
async fn delete(&self, key: &str) -> Result<(), CacheError> {
let mut data = self.data.lock().unwrap();
data.remove(key);
Ok(())
}
async fn get_bytes(&self, _key: &str) -> Option<Vec<u8>> {
None
}
async fn set_bytes(
&self,
_key: &str,
_value: &[u8],
_ttl: Duration,
) -> Result<(), CacheError> {
Ok(())
}
fn is_available(&self) -> bool {
true
}
}
use tranquil_infra::MemoryCache;
#[tokio::test]
async fn test_create_and_validate_token() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
let token = create_email_token(&cache, &did, EmailTokenPurpose::UpdateEmail)
@@ -205,7 +127,7 @@ mod tests {
#[tokio::test]
async fn test_token_consumed_after_use() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
let token = create_email_token(&cache, &did, EmailTokenPurpose::UpdateEmail)
@@ -223,7 +145,7 @@ mod tests {
#[tokio::test]
async fn test_invalid_token_rejected() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
let _token = create_email_token(&cache, &did, EmailTokenPurpose::UpdateEmail)
@@ -237,7 +159,7 @@ mod tests {
#[tokio::test]
async fn test_wrong_purpose_rejected() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
let token = create_email_token(&cache, &did, EmailTokenPurpose::UpdateEmail)
@@ -252,7 +174,7 @@ mod tests {
#[tokio::test]
async fn test_token_format() {
// The emitted token is the display form: uppercase `XXXXX-XXXXX`.
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
(0..50).for_each(|_| {
let token = futures::executor::block_on(create_email_token(
@@ -269,7 +191,7 @@ mod tests {
#[tokio::test]
async fn test_case_insensitive_validation() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
let token = create_email_token(&cache, &did, EmailTokenPurpose::UpdateEmail)
@@ -284,7 +206,7 @@ mod tests {
#[tokio::test]
async fn test_hyphen_insensitive_validation() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:teq").expect("valid DID");
let token = create_email_token(&cache, &did, EmailTokenPurpose::UpdateEmail)
+28 -91
View File
@@ -3,6 +3,7 @@ use serde::{Deserialize, Serialize};
use std::time::Duration;
use crate::cache::Cache;
use crate::cache_keys::{legacy_2fa_challenge_key, legacy_2fa_cooldown_key};
use crate::types::Did;
use crate::util::{generate_token_code, normalize_token_code};
@@ -58,8 +59,8 @@ pub async fn create_challenge(
}
pub async fn clear_challenge(cache: &dyn Cache, did: &Did) {
let _ = cache.delete(&challenge_key(did)).await;
let _ = cache.delete(&cooldown_key(did)).await;
let _ = cache.delete(&legacy_2fa_challenge_key(did)).await;
let _ = cache.delete(&legacy_2fa_cooldown_key(did)).await;
}
async fn validate_challenge_internal(
@@ -71,7 +72,7 @@ async fn validate_challenge_internal(
return Err(ValidationError::CacheUnavailable);
}
let challenge_k = challenge_key(did);
let challenge_k = legacy_2fa_challenge_key(did);
let json = cache
.get(&challenge_k)
@@ -114,19 +115,11 @@ async fn validate_challenge_internal(
}
let _ = cache.delete(&challenge_k).await;
let _ = cache.delete(&cooldown_key(did)).await;
let _ = cache.delete(&legacy_2fa_cooldown_key(did)).await;
Ok(())
}
fn challenge_key(did: &Did) -> String {
format!("legacy_2fa:{}", did)
}
fn cooldown_key(did: &Did) -> String {
format!("legacy_2fa_cooldown:{}", did)
}
fn current_timestamp() -> u64 {
u64::try_from(Utc::now().timestamp()).unwrap_or(0)
}
@@ -226,7 +219,7 @@ async fn create_challenge_code(
return Err(ChallengeError::CacheUnavailable);
}
let cooldown = cooldown_key(did);
let cooldown = legacy_2fa_cooldown_key(did);
if cache.get(&cooldown).await.is_some() {
return Err(ChallengeError::RateLimited);
}
@@ -244,7 +237,7 @@ async fn create_challenge_code(
cache
.set(
&challenge_key(did),
&legacy_2fa_challenge_key(did),
&json,
Duration::from_secs(CHALLENGE_TTL_SECS),
)
@@ -280,67 +273,11 @@ impl From<ValidationError> for Legacy2faFlowError {
#[cfg(test)]
mod tests {
use super::*;
use crate::cache::CacheError;
use async_trait::async_trait;
use std::collections::HashMap;
use std::sync::Mutex;
struct MockCache {
data: Mutex<HashMap<String, (String, u64)>>,
}
impl MockCache {
fn new() -> Self {
Self {
data: Mutex::new(HashMap::new()),
}
}
}
#[async_trait]
impl Cache for MockCache {
async fn get(&self, key: &str) -> Option<String> {
let data = self.data.lock().unwrap();
let now = current_timestamp();
data.get(key)
.filter(|(_, exp)| *exp > now)
.map(|(v, _)| v.clone())
}
async fn set(&self, key: &str, value: &str, ttl: Duration) -> Result<(), CacheError> {
let mut data = self.data.lock().unwrap();
let expires = current_timestamp() + ttl.as_secs();
data.insert(key.to_string(), (value.to_string(), expires));
Ok(())
}
async fn delete(&self, key: &str) -> Result<(), CacheError> {
let mut data = self.data.lock().unwrap();
data.remove(key);
Ok(())
}
async fn get_bytes(&self, _key: &str) -> Option<Vec<u8>> {
None
}
async fn set_bytes(
&self,
_key: &str,
_value: &[u8],
_ttl: Duration,
) -> Result<(), CacheError> {
Ok(())
}
fn is_available(&self) -> bool {
true
}
}
use tranquil_infra::MemoryCache;
#[tokio::test]
async fn test_create_and_validate_challenge() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
@@ -352,7 +289,7 @@ mod tests {
#[tokio::test]
async fn test_challenge_code_format() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
@@ -364,7 +301,7 @@ mod tests {
#[tokio::test]
async fn test_case_insensitive_validation() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
@@ -375,7 +312,7 @@ mod tests {
#[tokio::test]
async fn test_hyphen_insensitive_validation() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
@@ -386,7 +323,7 @@ mod tests {
#[tokio::test]
async fn test_invalid_code_rejected() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let _code = create_challenge(&cache, &did).await.unwrap();
@@ -396,7 +333,7 @@ mod tests {
#[tokio::test]
async fn test_challenge_consumed_on_success() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
@@ -410,7 +347,7 @@ mod tests {
#[tokio::test]
async fn test_max_attempts_exceeded() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let _code = create_challenge(&cache, &did).await.unwrap();
@@ -425,7 +362,7 @@ mod tests {
#[tokio::test]
async fn test_rate_limiting() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let _first = create_challenge(&cache, &did).await.unwrap();
@@ -453,7 +390,7 @@ mod tests {
#[tokio::test]
async fn test_process_flow_not_required() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -470,7 +407,7 @@ mod tests {
#[tokio::test]
async fn test_process_flow_not_required_because_app_password() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: true,
@@ -487,7 +424,7 @@ mod tests {
#[tokio::test]
async fn test_process_flow_blocked() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -504,7 +441,7 @@ mod tests {
#[tokio::test]
async fn test_process_flow_challenge_sent_totp() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -521,7 +458,7 @@ mod tests {
#[tokio::test]
async fn test_process_flow_challenge_sent_email_2fa_enabled() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test2".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -538,7 +475,7 @@ mod tests {
#[tokio::test]
async fn test_process_flow_verified() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -557,7 +494,7 @@ mod tests {
#[tokio::test]
async fn test_attempts_persist_across_failures() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
@@ -590,7 +527,7 @@ mod tests {
#[tokio::test]
async fn test_totp_shaped_token_accepted_via_verifier() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:totp1".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -607,7 +544,7 @@ mod tests {
#[tokio::test]
async fn test_totp_shaped_token_rejected_does_not_touch_email_challenge() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:totp2".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -641,7 +578,7 @@ mod tests {
#[tokio::test]
async fn test_email_shaped_token_routes_to_email_path_when_totp_present() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:totp3".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -662,7 +599,7 @@ mod tests {
#[tokio::test]
async fn test_backup_code_shaped_token_routes_to_verifier() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:totp4".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
@@ -681,7 +618,7 @@ mod tests {
#[tokio::test]
async fn test_totp_shaped_token_ignored_when_no_totp() {
let cache = MockCache::new();
let cache = MemoryCache::new();
let did = Did::new("did:plc:totp5".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
+9 -8
View File
@@ -43,14 +43,15 @@ pub use scope_verified::{
pub use service::{ServiceTokenClaims, ServiceTokenError, ServiceTokenVerifier, is_service_token};
pub use tranquil_auth::{
ActClaim, Claims, Header, SigningAlgorithm, TokenData, TokenDecodeError, TokenScope, TokenType,
TokenVerifyError, TokenWithMetadata, TotpError, UnsafeClaims, create_access_token,
create_access_token_hs256, create_access_token_hs256_with_metadata,
create_access_token_with_delegation, create_access_token_with_jti,
create_access_token_with_metadata, create_access_token_with_scope_metadata,
create_refresh_token, create_refresh_token_hs256, create_refresh_token_hs256_with_metadata,
create_refresh_token_with_jti, create_refresh_token_with_metadata, create_service_token,
create_service_token_hs256, generate_backup_codes, generate_qr_png_base64,
ActClaim, Claims, Header, ScopeDecodeError, ScopeEncodeError, SigningAlgorithm, TokenData,
TokenDecodeError, TokenScope, TokenType, TokenVerifyError, TokenWithMetadata, TotpError,
UnsafeClaims, create_access_token, create_access_token_hs256,
create_access_token_hs256_with_metadata, create_access_token_with_delegation,
create_access_token_with_jti, create_access_token_with_metadata,
create_access_token_with_scope_metadata, create_refresh_token, create_refresh_token_hs256,
create_refresh_token_hs256_with_metadata, create_refresh_token_with_jti,
create_refresh_token_with_metadata, create_service_token, create_service_token_hs256,
decode_scope, encode_scope, generate_backup_codes, generate_qr_png_base64,
generate_totp_secret, generate_totp_uri, get_algorithm_from_token, get_did_from_token,
get_jti_from_token, hash_backup_code, is_backup_code_format, verify_access_token,
verify_access_token_hs256, verify_backup_code, verify_refresh_token,
+5
View File
@@ -67,6 +67,7 @@ impl WebAuthnConfig {
.get_or_insert_with(AuthenticatorSelectionCriteria::default);
sel.resident_key = Some(ResidentKeyRequirement::Required);
sel.require_resident_key = true;
ccr.public_key.hints = None;
(ccr, state)
})
.map_err(|e| WebauthnError::RegistrationFailed(e.to_string()))
@@ -88,6 +89,10 @@ impl WebAuthnConfig {
) -> Result<(RequestChallengeResponse, SecurityKeyAuthentication), WebauthnError> {
self.webauthn
.start_securitykey_authentication(&credentials)
.map(|(mut rcr, state)| {
rcr.public_key.hints = None;
(rcr, state)
})
.map_err(|e| WebauthnError::AuthenticationFailed(e.to_string()))
}
+3 -1
View File
@@ -1,4 +1,6 @@
pub use tranquil_cache::{Cache, CacheError, DistributedRateLimiter, NoOpCache, create_cache};
pub use tranquil_cache::{
Cache, CacheError, DistributedRateLimiter, NoOpCache, cached_json, create_cache,
};
#[cfg(feature = "valkey")]
pub use tranquil_cache::{RedisRateLimiter, ValkeyCache};
+1 -48
View File
@@ -1,48 +1 @@
use crate::types::{CidLink, Did, Handle, Jti};
pub fn session_key(did: &Did, jti: &Jti) -> String {
format!("auth:session:{}:{}", did, jti)
}
pub fn signing_key_key(did: &Did) -> String {
format!("auth:key:{}", did)
}
pub fn user_status_key(did: &Did) -> String {
format!("auth:status:{}", did)
}
pub fn handle_key(handle: &Handle) -> String {
format!("handle:{}", handle)
}
pub fn reauth_key(did: &Did) -> String {
format!("reauth:{}", did)
}
pub fn plc_doc_key(did: &Did) -> String {
format!("plc:doc:{}", did)
}
pub fn plc_data_key(did: &Did) -> String {
format!("plc:data:{}", did)
}
pub fn email_update_key(did: &Did) -> String {
format!("email_update:{}", did)
}
pub fn scope_ref_key(cid: &CidLink) -> String {
format!("scope_ref:{}", cid)
}
pub fn auto_verify_sent_key(did: &Did) -> String {
format!("auto_verify_sent:{}", did)
}
pub fn permission_set_key(nsid: &tranquil_types::Nsid, aud: Option<&str>) -> String {
match aud {
Some(a) => format!("permset:{}:{}", nsid, a),
None => format!("permset:{}", nsid),
}
}
pub use tranquil_cache::cache_keys::*;
+26 -18
View File
@@ -5,14 +5,25 @@ pub use roles::{
CanAddControllers, CanControlAccounts, verify_can_add_controllers, verify_can_control_accounts,
};
pub use scopes::{
EDITOR_FULL_SCOPES, InvalidDelegationScopeError, OWNER_FULL_SCOPES, SCOPE_PRESETS, ScopePreset,
ValidatedDelegationScope, grant_covers, intersect_scopes,
ADMIN_FULL_SCOPES, EDITOR_FULL_SCOPES, GrantCoverage, InvalidDelegationScopeError,
OWNER_FULL_SCOPES, SCOPE_PRESETS, ScopePreset, ValidatedDelegationScope, grant_coverage,
intersect_scopes,
};
pub use tranquil_db_traits::DelegationActionType;
use crate::did::DidResolutionError;
use crate::state::AppState;
use crate::types::{Did, Handle};
use tranquil_types::did_doc::{PdsEndpointError, extract_handle, extract_pds_endpoint};
use tranquil_types::{InvalidHttpUrl, PdsUrl};
#[derive(Debug, thiserror::Error)]
pub enum IdentityResolutionError {
#[error(transparent)]
DidResolution(#[from] DidResolutionError),
#[error("remote PDS endpoint is unusable: {0}")]
PdsEndpoint(InvalidHttpUrl),
}
#[derive(serde::Serialize)]
#[serde(rename_all = "camelCase")]
@@ -21,14 +32,14 @@ pub struct ResolvedIdentity {
#[serde(skip_serializing_if = "Option::is_none")]
pub handle: Option<Handle>,
#[serde(skip_serializing_if = "Option::is_none")]
pub pds_url: Option<String>,
pub pds_url: Option<PdsUrl>,
pub is_local: bool,
}
pub async fn resolve_identity(
state: &AppState,
did: &Did,
) -> Result<ResolvedIdentity, DidResolutionError> {
) -> Result<ResolvedIdentity, IdentityResolutionError> {
let is_local = state
.repos
.user
@@ -38,26 +49,23 @@ pub async fn resolve_identity(
.flatten()
.is_some();
let did_doc = state.did_resolver.resolve_did(did).await?;
let did_doc = state.did_resolver.fetch_did_document(did).await?;
let pds_url = did_doc.services.iter().find_map(|svc| {
if (svc.id == "#atproto_pds" || svc.id.ends_with("#atproto_pds"))
&& svc.service_type == "AtprotoPersonalDataServer"
{
Some(svc.service_endpoint.clone())
} else {
let pds_url = match (extract_pds_endpoint(&did_doc), is_local) {
(Ok(url), _) => Some(url),
(Err(PdsEndpointError::Missing), _) => None,
(Err(PdsEndpointError::Invalid(e)), true) => {
tracing::debug!(did = %did, error = %e, "local account has an unusable PDS endpoint");
None
}
});
let handle = did_doc
.also_known_as
.iter()
.find_map(|alias| alias.strip_prefix("at://"))
.and_then(|s| Handle::new(s).ok());
(Err(PdsEndpointError::Invalid(e)), false) => {
return Err(IdentityResolutionError::PdsEndpoint(e));
}
};
Ok(ResolvedIdentity {
did: did.clone(),
handle,
handle: extract_handle(&did_doc),
pds_url,
is_local,
})
+215 -49
View File
@@ -1,6 +1,6 @@
use std::collections::HashSet;
use std::collections::BTreeSet;
use tranquil_scopes::{covers, parse_scope};
use tranquil_scopes::{Coverage, ParsedScope, coverage, parse_scope};
pub use tranquil_db_traits::{
DbScope as ValidatedDelegationScope, InvalidScopeError as InvalidDelegationScopeError,
@@ -14,10 +14,15 @@ pub struct ScopePreset {
pub scopes: &'static str,
}
pub const OWNER_FULL_SCOPES: &str = "atproto repo:* blob:*/* identity:* account:*?action=manage";
pub const OWNER_FULL_SCOPES: &str = concat!(
"atproto repo:* blob:*/* rpc:* identity:* account:*?action=manage ",
"transition:generic transition:chat.bsky transition:email"
);
pub const ADMIN_FULL_SCOPES: &str = "atproto repo:* blob:*/* rpc:* account:*?action=manage";
pub const EDITOR_FULL_SCOPES: &str =
"atproto repo:*?action=create repo:*?action=update repo:*?action=delete blob:*/*";
"atproto repo:*?action=create repo:*?action=update repo:*?action=delete blob:*/* rpc:*";
pub const SCOPE_PRESETS: &[ScopePreset] = &[
ScopePreset {
@@ -30,7 +35,7 @@ pub const SCOPE_PRESETS: &[ScopePreset] = &[
name: "admin",
label: "Admin",
description: "Manage account settings, post content, upload media",
scopes: "atproto repo:* blob:*/* account:*?action=manage",
scopes: ADMIN_FULL_SCOPES,
},
ScopePreset {
name: "editor",
@@ -46,35 +51,51 @@ pub const SCOPE_PRESETS: &[ScopePreset] = &[
},
];
pub fn intersect_scopes(requested: &str, granted: &str) -> String {
let requested_set: HashSet<&str> = requested.split_whitespace().collect();
let granted_parsed: Vec<tranquil_scopes::ParsedScope> =
granted.split_whitespace().map(parse_scope).collect();
let mut scopes: Vec<&str> = requested_set
.iter()
.filter(|requested_scope| {
**requested_scope != "atproto" && any_granted_covers(requested_scope, &granted_parsed)
})
.copied()
.chain(requested_set.contains("atproto").then_some("atproto"))
.collect();
scopes.sort();
scopes.join(" ")
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum GrantCoverage {
Full,
Narrowed(String),
Withheld,
}
pub fn grant_covers(granted: &str, scope: &str) -> bool {
fn scope_coverage(granted: &[ParsedScope], scope: &str) -> GrantCoverage {
if scope == "atproto" {
return true;
return GrantCoverage::Full;
}
match coverage(granted, &parse_scope(scope)) {
Coverage::Full => GrantCoverage::Full,
Coverage::Narrowed(ParsedScope::Repo(repo)) => {
GrantCoverage::Narrowed(repo.to_scope_string())
}
Coverage::Narrowed(_) => GrantCoverage::Full,
Coverage::Withheld => GrantCoverage::Withheld,
}
let granted_parsed: Vec<tranquil_scopes::ParsedScope> =
granted.split_whitespace().map(parse_scope).collect();
any_granted_covers(scope, &granted_parsed)
}
fn any_granted_covers(requested: &str, granted: &[tranquil_scopes::ParsedScope]) -> bool {
let requested_parsed = parse_scope(requested);
granted.iter().any(|g| covers(g, &requested_parsed))
pub fn grant_coverage(granted: &str, scope: &str) -> GrantCoverage {
scope_coverage(&parse_grant(granted), scope)
}
fn parse_grant(granted: &str) -> Vec<ParsedScope> {
granted.split_whitespace().map(parse_scope).collect()
}
pub fn intersect_scopes(requested: &str, granted: &str) -> String {
let granted_parsed = parse_grant(granted);
let scopes: BTreeSet<String> = requested
.split_whitespace()
.filter_map(
|requested_scope| match scope_coverage(&granted_parsed, requested_scope) {
GrantCoverage::Full => Some(requested_scope.to_string()),
GrantCoverage::Narrowed(narrowed) => Some(narrowed),
GrantCoverage::Withheld => None,
},
)
.collect();
scopes.into_iter().collect::<Vec<String>>().join(" ")
}
#[cfg(test)]
@@ -220,12 +241,33 @@ mod tests {
}
#[test]
fn test_intersect_partial_action_grant_drops_actionless_request() {
fn test_intersect_partial_action_grant_narrows_actionless_request() {
let result = intersect_scopes(
"repo:app.bsky.feed.post",
"repo:*?action=create&action=delete",
);
assert_eq!(result, "");
assert_eq!(
result,
"repo:app.bsky.feed.post?action=create&action=delete"
);
}
#[test]
fn test_intersect_keeps_collapsed_request_under_split_action_grant() {
assert_eq!(
intersect_scopes(
"repo:io.atcr.manifest?action=create&action=delete",
EDITOR_FULL_SCOPES
),
"repo:io.atcr.manifest?action=create&action=delete"
);
assert_eq!(
intersect_scopes(
"repo:io.atcr.manifest?action=create&action=delete",
"repo:*?action=create"
),
"repo:io.atcr.manifest?action=create"
);
}
#[test]
@@ -262,33 +304,157 @@ mod tests {
}
#[test]
fn test_grant_covers_matches_intersection() {
fn test_grant_coverage_full_and_withheld() {
let granted = "atproto repo:* blob:*/* account:*?action=manage";
let intersected = intersect_scopes(
"repo:app.bsky.feed.post?action=create identity:* account:*?action=manage",
granted,
);
assert!(grant_covers(
granted,
"repo:app.bsky.feed.post?action=create"
));
assert!(grant_covers(granted, "account:*?action=manage"));
assert!(!grant_covers(granted, "identity:*"));
assert_eq!(grant_coverage(granted, "atproto"), GrantCoverage::Full);
assert_eq!(
grant_covers(granted, "identity:*"),
intersected.contains("identity")
grant_coverage(granted, "repo:app.bsky.feed.post?action=create"),
GrantCoverage::Full
);
assert_eq!(
grant_coverage(granted, "identity:*"),
GrantCoverage::Withheld
);
assert_eq!(grant_coverage("", "identity:*"), GrantCoverage::Withheld);
}
#[test]
fn test_grant_coverage_narrowed_when_grant_is_a_strict_action_subset() {
assert_eq!(
grant_coverage(
EDITOR_FULL_SCOPES,
"repo:io.atcr.manifest?action=create&action=delete"
),
GrantCoverage::Full
);
assert_eq!(
grant_coverage(
"atproto repo:*?action=create blob:*/*",
"repo:io.atcr.manifest?action=create&action=delete"
),
GrantCoverage::Narrowed("repo:io.atcr.manifest?action=create".to_string())
);
}
// Tracks all known scope prefixes
const GRANULAR_SCOPE_TAXONOMY: &[(&str, &str)] = &[
("repo", "repo:app.bsky.feed.post?action=create"),
("blob", "blob:image/png"),
("rpc", "rpc:app.bsky.actor.getProfile?aud=*"),
("account", "account:email?action=manage"),
("identity", "identity:handle"),
("transition:generic", "transition:generic"),
("transition:chat.bsky", "transition:chat.bsky"),
("transition:email", "transition:email"),
];
/// The taxonomy label a scope type must be represented by, or `None` for scope types
/// delegation never gates.
fn taxonomy_label(scope: &ParsedScope) -> Option<&'static str> {
match scope {
ParsedScope::Repo(_) => Some("repo"),
ParsedScope::Blob(_) => Some("blob"),
ParsedScope::Rpc(_) => Some("rpc"),
ParsedScope::Account(_) => Some("account"),
ParsedScope::Identity(_) => Some("identity"),
ParsedScope::TransitionGeneric => Some("transition:generic"),
ParsedScope::TransitionChat => Some("transition:chat.bsky"),
ParsedScope::TransitionEmail => Some("transition:email"),
ParsedScope::Atproto => None,
ParsedScope::Include(_) => None,
ParsedScope::Unknown(_) => None,
}
}
#[test]
fn test_taxonomy_entries_parse_to_the_scope_type_they_claim() {
GRANULAR_SCOPE_TAXONOMY.iter().for_each(|(label, scope)| {
assert_eq!(
taxonomy_label(&parse_scope(scope)),
Some(*label),
"taxonomy entry `{}` does not parse to a `{}` scope, so the reachability \
test is not actually exercising that scope type",
scope,
label
);
});
}
fn coverage_matrix() -> String {
GRANULAR_SCOPE_TAXONOMY
.iter()
.map(|(label, scope)| {
let granting: Vec<&str> = SCOPE_PRESETS
.iter()
.filter(|p| grant_coverage(p.scopes, scope) != GrantCoverage::Withheld)
.map(|p| p.name)
.collect();
match granting.is_empty() {
true => format!(" {:<9} ({}) -> NONE", label, scope),
false => format!(" {:<9} ({}) -> {}", label, scope, granting.join(", ")),
}
})
.collect::<Vec<String>>()
.join("\n")
}
#[test]
fn test_every_granular_scope_type_is_reachable_through_some_preset() {
let unreachable: Vec<&str> = GRANULAR_SCOPE_TAXONOMY
.iter()
.filter(|(_, scope)| {
SCOPE_PRESETS
.iter()
.all(|p| grant_coverage(p.scopes, scope) == GrantCoverage::Withheld)
})
.map(|(label, _)| *label)
.collect();
assert!(
unreachable.is_empty(),
"no delegation preset confers any `{}` scope, so delegated accounts cannot use \
that capability at all.\ncoverage by preset:\n{}",
unreachable.join("`, `"),
coverage_matrix()
);
}
#[test]
fn test_grant_covers_atproto_always_true() {
assert!(grant_covers("", "atproto"));
assert!(grant_covers("repo:*", "atproto"));
fn test_forbidden_rpc_wildcard_is_not_a_usable_grant() {
// `rpc:*?aud=*` wildcards both lxm and aud, which the spec forbids, so it parses to
// Unknown and confers nothing. A preset reaching for it to mean "all rpc" would look
// right and silently grant nothing -- `rpc:*` is the form that works.
assert_eq!(
grant_coverage("atproto rpc:*?aud=*", "rpc:app.bsky.actor.getProfile?aud=*"),
GrantCoverage::Withheld
);
assert_eq!(
grant_coverage("atproto rpc:*", "rpc:app.bsky.actor.getProfile?aud=*"),
GrantCoverage::Full
);
}
#[test]
fn test_grant_covers_empty_grant_covers_nothing_else() {
assert!(!grant_covers("", "repo:app.bsky.feed.post?action=create"));
assert!(!grant_covers("", "identity:*"));
fn test_forbidden_rpc_wildcard_request_stays_denied() {
assert_eq!(
grant_coverage("atproto rpc:*", "rpc:*?aud=*"),
GrantCoverage::Withheld
);
}
#[test]
fn test_grant_may_mix_transition_and_granular_scopes() {
assert!(ValidatedDelegationScope::new(OWNER_FULL_SCOPES).is_ok());
assert_eq!(
intersect_scopes("atproto transition:generic", OWNER_FULL_SCOPES),
"atproto transition:generic"
);
assert_eq!(
intersect_scopes(
"atproto repo:app.bsky.feed.post?action=create",
OWNER_FULL_SCOPES
),
"atproto repo:app.bsky.feed.post?action=create"
);
}
}
+74 -199
View File
@@ -1,10 +1,9 @@
use crate::cache::Cache;
use crate::types::Did;
use reqwest::Client;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::sync::RwLock;
use std::time::Duration;
use tracing::{debug, info, warn};
#[derive(Debug, thiserror::Error)]
@@ -13,6 +12,8 @@ pub enum DidResolutionError {
UnsupportedDidMethod(String),
#[error("Invalid did:web format")]
InvalidDidWeb,
#[error("did:web host {0} is outside the allowed host reach")]
DidWebHostRejected(String),
#[error("HTTP request failed: {0}")]
HttpFailed(String),
#[error("Invalid DID document: {0}")]
@@ -53,43 +54,50 @@ pub struct DidService {
pub struct ResolvedService {
pub url: String,
pub did: Did,
pub service_id: String,
}
type TimedCache<T> = RwLock<HashMap<Box<str>, (Instant, Arc<T>)>>;
pub struct DidResolver {
did_doc_cache: TimedCache<serde_json::Value>,
parsed_did_doc_cache: TimedCache<DidDocument>,
service_cache: TimedCache<ResolvedService>,
cache: Arc<dyn Cache>,
client: Client,
cache_ttl: Duration,
plc_directory_url: String,
fetch_policy: tranquil_types::ReachPolicy,
}
impl DidResolver {
pub fn new() -> Self {
pub fn new(cache: Arc<dyn Cache>) -> Self {
let cfg = tranquil_config::get();
let cache_ttl_secs = cfg.plc.did_cache_ttl_secs;
let plc_directory_url = cfg.plc.directory_url.clone();
let fetch_policy =
tranquil_types::ReachPolicy::from_private_fetch(cfg.server.allow_private_fetch);
let client = Client::builder()
.timeout(Duration::from_secs(10))
.connect_timeout(Duration::from_secs(5))
.pool_max_idle_per_host(10)
.redirect(tranquil_types::redirect_policy(fetch_policy))
.dns_resolver(tranquil_types::dns_guard(fetch_policy))
.build()
.unwrap_or_else(|_| Client::new());
.expect("failed to build DID resolver HTTP client");
info!("DID resolver initialized");
Self {
did_doc_cache: RwLock::new(HashMap::new()),
parsed_did_doc_cache: RwLock::new(HashMap::new()),
service_cache: RwLock::new(HashMap::new()),
cache,
client,
cache_ttl: Duration::from_secs(cache_ttl_secs),
plc_directory_url,
cache_ttl: Duration::from_secs(cfg.plc.did_cache_ttl_secs),
plc_directory_url: cfg.plc.directory_url.clone(),
fetch_policy,
}
}
fn doc_cache_key(did: &Did) -> Result<String, DidResolutionError> {
match (did.is_plc(), did.is_web()) {
(true, _) => Ok(crate::cache_keys::plc_doc_key(did)),
(_, true) => Ok(crate::cache_keys::did_web_doc_key(did)),
_ => {
warn!("Unsupported DID method: {}", did);
Err(DidResolutionError::UnsupportedDidMethod(did.to_string()))
}
}
}
@@ -97,182 +105,59 @@ impl DidResolver {
&self,
did: &Did,
service_id: &str,
) -> Result<Arc<ResolvedService>, ServiceResolutionError> {
{
let cache = self.service_cache.read().await;
if let Some(cached) = cache.get(&*format!("{did}#{service_id}"))
&& cached.0.elapsed() < self.cache_ttl
{
return Ok(cached.1.clone());
}
}
) -> Result<ResolvedService, ServiceResolutionError> {
let did_doc = self.resolve_did(did).await?;
let Some(service) = did_doc
let suffix = format!("#{service_id}");
did_doc
.services
.iter()
.find(|s| s.id.ends_with(&format!("#{service_id}")))
else {
return Err(ServiceResolutionError::ServiceIdNotFound(service_id.into()));
};
let resolved = Arc::new(ResolvedService {
url: service.service_endpoint.clone(),
did: did.clone(),
service_id: service_id.into(),
});
{
let mut cache = self.service_cache.write().await;
cache.insert(
format!("{did}#{service_id}").into(),
(Instant::now(), resolved.clone()),
);
}
Ok(resolved)
.find(|s| s.id.ends_with(&suffix))
.map(|service| ResolvedService {
url: service.service_endpoint.clone(),
did: did.clone(),
})
.ok_or_else(|| ServiceResolutionError::ServiceIdNotFound(service_id.into()))
}
pub async fn resolve_did(&self, did: &Did) -> Result<Arc<DidDocument>, DidResolutionError> {
{
let cache = self.parsed_did_doc_cache.read().await;
if let Some(cached) = cache.get(did.as_str())
&& cached.0.elapsed() < self.cache_ttl
{
return Ok(cached.1.clone());
}
}
let resolved = Arc::new(self.resolve_did_uncached(did).await?);
{
let mut cache = self.parsed_did_doc_cache.write().await;
cache.insert(did.as_str().into(), (Instant::now(), resolved.clone()));
}
Ok(resolved)
pub async fn resolve_did(&self, did: &Did) -> Result<DidDocument, DidResolutionError> {
self.cached_did_document(did).await
}
pub async fn refresh_did(&self, did: &Did) -> Result<Arc<DidDocument>, DidResolutionError> {
{
let mut cache = self.parsed_did_doc_cache.write().await;
cache.remove(did.as_str());
let mut cache = self.service_cache.write().await;
cache.retain(|k, _| !k.starts_with(did.as_str()));
}
pub async fn refresh_did(&self, did: &Did) -> Result<DidDocument, DidResolutionError> {
let _ = self.cache.delete(&Self::doc_cache_key(did)?).await;
self.resolve_did(did).await
}
async fn resolve_did_uncached(&self, did: &Did) -> Result<DidDocument, DidResolutionError> {
if did.is_web() {
self.resolve_did_web(did).await
} else if did.is_plc() {
self.resolve_did_plc(did).await
} else {
warn!("Unsupported DID method: {}", did);
Err(DidResolutionError::UnsupportedDidMethod(did.to_string()))
}
}
async fn resolve_did_web(&self, did: &Did) -> Result<DidDocument, DidResolutionError> {
let url = build_did_web_url(did)?;
debug!("Resolving did:web {} via {}", did, url);
let resp = self
.client
.get(&url)
.send()
.await
.map_err(|e| DidResolutionError::HttpFailed(e.to_string()))?;
if !resp.status().is_success() {
return Err(DidResolutionError::HttpFailed(format!(
"HTTP {}",
resp.status()
)));
}
resp.json::<DidDocument>()
.await
.map_err(|e| DidResolutionError::InvalidDocument(e.to_string()))
}
async fn resolve_did_plc(&self, did: &Did) -> Result<DidDocument, DidResolutionError> {
let url = format!(
"{}/{}",
self.plc_directory_url,
urlencoding::encode(did.as_str())
);
debug!("Resolving did:plc {} via {}", did, url);
let resp = self
.client
.get(&url)
.send()
.await
.map_err(|e| DidResolutionError::HttpFailed(e.to_string()))?;
if resp.status() == reqwest::StatusCode::NOT_FOUND {
return Err(DidResolutionError::NotFound);
}
if !resp.status().is_success() {
return Err(DidResolutionError::HttpFailed(format!(
"HTTP {}",
resp.status()
)));
}
resp.json::<DidDocument>()
.await
.map_err(|e| DidResolutionError::InvalidDocument(e.to_string()))
}
pub async fn fetch_did_document(
&self,
did: &Did,
) -> Result<Arc<serde_json::Value>, DidResolutionError> {
{
let cache = self.did_doc_cache.read().await;
if let Some(cached) = cache.get(did.as_str())
&& cached.0.elapsed() < self.cache_ttl
{
return Ok(cached.1.clone());
}
}
let resolved = Arc::new(self.fetch_did_document_uncached(did).await?);
{
let mut cache = self.did_doc_cache.write().await;
cache.insert(did.as_str().into(), (Instant::now(), resolved.clone()));
}
Ok(resolved)
) -> Result<serde_json::Value, DidResolutionError> {
self.cached_did_document(did).await
}
// TODO: make cached version
async fn fetch_did_document_uncached(
async fn cached_did_document<T: serde::de::DeserializeOwned>(
&self,
did: &Did,
) -> Result<serde_json::Value, DidResolutionError> {
if did.is_web() {
self.fetch_did_document_web(did).await
} else if did.is_plc() {
self.fetch_did_document_plc(did).await
} else {
warn!("Unsupported DID method: {}", did);
Err(DidResolutionError::UnsupportedDidMethod(did.to_string()))
}
) -> Result<T, DidResolutionError> {
let cache_key = Self::doc_cache_key(did)?;
let doc =
crate::cache::cached_json(self.cache.as_ref(), &cache_key, self.cache_ttl, || async {
match did.is_plc() {
true => self.fetch_did_document_plc(did).await,
false => self.fetch_did_document_web(did).await,
}
})
.await?;
serde_json::from_value(doc).map_err(|e| DidResolutionError::InvalidDocument(e.to_string()))
}
async fn fetch_did_document_web(
&self,
did: &Did,
) -> Result<serde_json::Value, DidResolutionError> {
let url = build_did_web_url(did)?;
let url = build_did_web_url(did, self.fetch_policy)?;
debug!("Resolving did:web {} via {}", did, url);
let resp = self
.client
@@ -303,6 +188,8 @@ impl DidResolver {
urlencoding::encode(did.as_str())
);
debug!("Resolving did:plc {} via {}", did, url);
let resp = self
.client
.get(&url)
@@ -325,24 +212,12 @@ impl DidResolver {
.await
.map_err(|e| DidResolutionError::InvalidDocument(e.to_string()))
}
pub async fn invalidate_cache(&self, did: &Did) {
let mut doc_cache = self.parsed_did_doc_cache.write().await;
doc_cache.remove(did.as_str());
}
}
impl Default for DidResolver {
fn default() -> Self {
Self::new()
}
}
pub fn create_did_resolver() -> Arc<DidResolver> {
Arc::new(DidResolver::new())
}
fn build_did_web_url(did: &Did) -> Result<String, DidResolutionError> {
fn build_did_web_url(
did: &Did,
policy: tranquil_types::ReachPolicy,
) -> Result<String, DidResolutionError> {
let host = did
.strip_prefix("did:web:")
.ok_or(DidResolutionError::InvalidDidWeb)?;
@@ -372,18 +247,18 @@ fn build_did_web_url(did: &Did) -> Result<String, DidResolutionError> {
}
};
let scheme =
if host.starts_with("localhost") || host.starts_with("127.0.0.1") || host.contains(':') {
"http"
} else {
"https"
};
let url = if path.is_empty() {
format!("{}://{}/.well-known/did.json", scheme, host)
let https = if path.is_empty() {
format!("https://{}/.well-known/did.json", host)
} else {
format!("{}://{}{}/did.json", scheme, host, path)
format!("https://{}{}/did.json", host, path)
};
Ok(url)
let mut url = reqwest::Url::parse(&https).map_err(|_| DidResolutionError::InvalidDidWeb)?;
if tranquil_types::url_reach(&url) == Some(tranquil_types::HostReach::Loopback) {
let _ = url.set_scheme("http");
}
match tranquil_types::url_reach_permits(&url, policy) {
true => Ok(url.to_string()),
false => Err(DidResolutionError::DidWebHostRejected(host)),
}
}
+133 -16
View File
@@ -3,8 +3,16 @@ pub mod reserved;
use crate::types::{Did, Handle};
use hickory_resolver::TokioAsyncResolver;
use hickory_resolver::config::{ResolverConfig, ResolverOpts};
use std::sync::LazyLock;
use thiserror::Error;
pub use tranquil_types::Domain;
static HOSTNAME_DOMAIN: LazyLock<Domain> = LazyLock::new(|| {
Domain::new(tranquil_config::get().server.hostname_without_port())
.expect("server.hostname is validated at config load")
});
#[derive(Error, Debug)]
pub enum HandleResolutionError {
#[error("DNS lookup failed: {0}")]
@@ -85,28 +93,137 @@ pub async fn verify_handle_ownership(
}
}
pub fn is_service_domain_handle(handle: &str, hostname: &str) -> bool {
if !handle.contains('.') {
return true;
#[derive(Clone, Copy)]
pub struct ServiceDomains<'a> {
user_domains: &'a [Domain],
hostname: &'a Domain,
serve_hostname: bool,
}
impl ServiceDomains<'static> {
pub fn for_user_handles() -> Self {
Self::from_config(false)
}
pub fn served() -> Self {
Self::from_config(true)
}
fn from_config(serve_hostname: bool) -> Self {
let server = &tranquil_config::get().server;
Self {
user_domains: server.user_handle_domains.as_deref().unwrap_or_default(),
hostname: &HOSTNAME_DOMAIN,
serve_hostname,
}
}
}
impl<'a> ServiceDomains<'a> {
pub fn iter(&self) -> impl Iterator<Item = &'a Domain> {
let hostname = (self.serve_hostname || self.user_domains.is_empty())
.then_some(self.hostname)
.filter(|h| !self.user_domains.contains(h));
self.user_domains.iter().chain(hostname)
}
pub fn primary(&self) -> &'a Domain {
self.user_domains.first().unwrap_or(self.hostname)
}
pub fn contains(&self, name: &str) -> bool {
self.iter().any(|d| d.eq_name(name))
}
pub fn split_handle<'h>(&self, handle: &'h str) -> Option<(&'a Domain, &'h str)> {
self.iter()
.filter_map(|d| d.strip_from(handle).map(|short| (d, short)))
.max_by_key(|(d, _)| d.as_str().len())
}
let service_domains = tranquil_config::try_get()
.map(|c| c.server.user_handle_domain_list())
.unwrap_or_else(|| vec![hostname.to_string()]);
service_domains
.iter()
.any(|domain| handle.ends_with(&format!(".{}", domain)) || handle == domain)
}
#[cfg(test)]
mod tests {
use super::*;
use super::{Domain, ServiceDomains};
use std::sync::LazyLock;
static HOST: LazyLock<Domain> = LazyLock::new(|| "pds.oyster.cafe".parse().unwrap());
fn domains(user_domains: &[Domain], serve_hostname: bool) -> ServiceDomains<'_> {
ServiceDomains {
user_domains,
hostname: &HOST,
serve_hostname,
}
}
fn owned(list: &[&str]) -> Vec<Domain> {
list.iter().map(|d| d.parse().unwrap()).collect()
}
#[test]
fn test_is_service_domain_handle() {
assert!(is_service_domain_handle("nel.oyster.cafe", "oyster.cafe"));
assert!(is_service_domain_handle("oyster.cafe", "oyster.cafe"));
assert!(is_service_domain_handle("myhandle", "oyster.cafe"));
assert!(!is_service_domain_handle("lyna.nel.pet", "oyster.cafe"));
assert!(!is_service_domain_handle("myhandle.xyz", "oyster.cafe"));
fn thostname_until_domains_are_configured() {
assert!(domains(&[], false).contains("pds.oyster.cafe"));
assert_eq!(domains(&[], false).primary(), "pds.oyster.cafe");
let configured = owned(&["oyster.cafe"]);
assert!(!domains(&configured, false).contains("pds.oyster.cafe"));
assert!(domains(&configured, false).contains("oyster.cafe"));
}
#[test]
fn served_set_covers_hostname_and_handle_domains() {
let configured = owned(&["oyster.cafe"]);
assert!(domains(&configured, true).contains("pds.oyster.cafe"));
assert!(domains(&configured, true).contains("oyster.cafe"));
}
#[test]
fn hostname_in_list_is_yielded_once() {
let configured = owned(&["pds.oyster.cafe", "oyster.cafe"]);
let served: Vec<&str> = domains(&configured, true)
.iter()
.map(Domain::as_str)
.collect();
assert_eq!(served, ["pds.oyster.cafe", "oyster.cafe"]);
let configured = owned(&["PDS.Oyster.Cafe"]);
let served: Vec<&str> = domains(&configured, true)
.iter()
.map(Domain::as_str)
.collect();
assert_eq!(served, ["pds.oyster.cafe"]);
}
#[test]
fn matching_case_insensitive() {
let configured = owned(&["oyster.cafe"]);
assert!(domains(&configured, false).contains("Oyster.Cafe"));
let (domain, short) = domains(&configured, false)
.split_handle("NEL.OYSTER.CAFE")
.unwrap();
assert_eq!(domain, "oyster.cafe");
assert_eq!(short, "NEL");
}
#[test]
fn longest_matching_domain_wins() {
let configured = owned(&["oyster.cafe", "pets.oyster.cafe"]);
let (domain, short) = domains(&configured, false)
.split_handle("nel.pets.oyster.cafe")
.unwrap();
assert_eq!(domain, "pets.oyster.cafe");
assert_eq!(short, "nel");
}
#[test]
fn split_handle_requires_a_dot() {
let configured = owned(&["oyster.cafe"]);
assert_eq!(
domains(&configured, false).split_handle("oyster.cafe"),
None
);
assert_eq!(
domains(&configured, false).split_handle("notoyster.cafe"),
None
);
}
}

Some files were not shown because too many files have changed in this diff Show More