Compare commits

..
Author SHA1 Message Date
nelind bfadc936c0 fix(api): dont verify signature or DID during importRepo 2026-06-11 23:23:37 +02:00
nelind 80afd764d3 fix(nix): provide a jemalloc build in the dev shell 2026-06-10 19:56:40 +02:00
LewisandTangled 5bbe2146ff server: serve xrpc over http/3
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-10 13:18:49 +03:00
LewisandTangled b009ccdaf2 repo: the pg side of MST structural repair
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-08 16:58:51 +03:00
nelind 39f74b5adf feat(nix): pratice what i preach. expose lib.mkPackages 2026-06-07 22:14:27 +02:00
aveandTangled a231d7da29 feat(auth): accept totp/backup codes in legacy login createSession 2026-06-06 10:42:24 +03:00
aveandTangled 63d84d38fb refactor(auth): unify short-code generation onto util::generate_token_code
Collapse the three ad-hoc short-code generators into one canonical generator
plus a shared normalizer:

- util::generate_token_code now emits the uppercase base32 XXXXX-XXXXX display
  form; new util::normalize_token_code canonicalizes user input (uppercase,
  strip hyphen/whitespace).
- email_token and legacy_2fa now generate via util, store the normalized form,
  and compare normalized input. Their private generate_short_token/generate_code
  (and BASE32_CHARS/CODE_LENGTH) are removed.
- PLC (request/sign) and password reset inline util::generate_token_code,
  persist the normalized form, email the display form, and normalize input
  before lookup. The generate_plc_token/generate_reset_code wrappers are removed.

Behavior changes: legacy login-2FA codes go from 8-digit numeric to XXXXX-XXXXX;
PLC and password-reset codes go from lowercase to uppercase. All four code types
are now accepted case-/hyphen-insensitively. OAuth web-login 2FA, account
deletion, and the long verification_token blobs are intentionally untouched.

Tests: add util normalize tests + email_token/legacy_2fa case/hyphen tests;
update integration tests to expect the canonical stored form and the new
emailed format.
2026-06-06 09:36:08 +03:00
aveandTangled fe9b88141c chore(auth): align at+jwt/refresh+jwt expiry with reference PDS 2026-06-06 01:43:46 +03:00
aveandTangled 72f5dce32b add ave.zone to contributors list in README 2026-06-06 01:42:54 +03:00
nelind cd7e01100e chore(auth): also mention that atproto spec requiers typ be "JWT" for inter-service tokens 2026-06-05 12:49:25 +02:00
aveandTangled 7c248be153 fix(auth): emit uppercase "JWT" typ in service-auth header
RFC 7519 §5.1 recommends the uppercase "JWT" typ for compatibility with
legacy implementations, and it matches the reference @atproto/pds. Parsing
already lowercases, so existing lowercase "jwt" tokens still verify.
2026-06-05 13:34:43 +03:00
aveandTangled 91999819c6 fix(proxy): limit audience of getFeed service-auth to the feed generator 2026-06-04 22:39:14 +03:00
nelindandTangled ffce1d5d05 feat(docs): add some general PDS debugging documentation 2026-06-04 13:45:08 +03:00
TylerandTangled 8e6ace2fe2 fix: derive lexicon DNS authority from all-but-last NSID segment
Permission-set expansion resolved the lexicon's DNS authority using a
fixed `parts[..2]`, which only works for three-segment NSIDs. For a
four-segment NSID such as community.lexicon.bookmarks.authManageBookmarks
this dropped a segment and queried _lexicon.lexicon.community instead of
_lexicon.bookmarks.lexicon.community, failing with "DNS resolution
failed: ... no record found".

The authority is every NSID segment except the last (the name),
reversed. Use parts[..parts.len() - 1] to match the spec and the
existing extract_namespace_authority helper, and update the DNS
authority test with three/four-segment and bookmarks regression cases.
2026-06-04 01:28:13 +03:00
LewisandTangled 3018a20843 fix(plc): allow arbitrary services to sign
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-03 11:23:51 +03:00
LewisandTangled 37fc06fb39 fix(store): unblock eventlog sync&freeze when writer dies
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
LewisandTangled 728a8c4d3b test(store): cross-store, firehose, read-validation coverage w/ faults
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
LewisandTangled 3d49e99cc3 test(store): generic consistency checker, gauntlet fault/read
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
LewisandTangled 7e823673ca test(store): untested metastore, eventlog, & archival stuff
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
Lewis 320933598c fix(auth): error num 401 for oauth
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:38:45 +03:00
Lewis 500dc2e0e6 test(store): gauntlet sweep configs for time-travel & fsync repro
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
Lewis a220611a8b test(store): D gauntlet faults, crash-loss oracley, recoverable scenarios
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
Lewis ca7a4b4b73 fix(store): recover eventlog lastseq from tail not sidecar
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
Lewis 8ff02610e4 feat(store): inline-commit mode, committed-extent recovery, failsafe verify&rollback
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
LewisandTangled 22f82489d5 test(pds): e2e & durability coverage for MST self-heal
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled cee483e358 feat(pds): selfhealing repo writing by detecting corruption & retrying
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled 7f8e858137 test(store): gauntlet MST-repairable & misdirected-write scenario
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled 44d73dac58 feat(store): rebuild & rewrite missing/corrupt MST blocks
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled b8cae15c12 feat(store): detect foreign&corrupt blocks on read & preserve blocks thru recovery
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled 9b58961bba feat(repo): missing $type? invent one
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 13:02:11 +03:00
Lewis 31ee12ecd3 fix(store): torn hint-file tail should be recoverable on reopen
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 11:37:21 +03:00
nelind ea106d5246 chore(nix): update frontend pnpm hash 2026-05-31 00:51:27 +02:00
Lewis 4015217a2e feat(store): Clock trait for DST
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 23:46:22 +03:00
Lewis 7e4c9e1dab chore: bump to 0.6.4
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 21:23:02 +03:00
LewisandTangled e9dc57d6f4 fix(firehose): lost events if seq commits out of order
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 21:10:38 +03:00
Lewis 4e2525b245 fix(migrate): oauth account:* spec, p256 support
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 21:07:16 +03:00
nelind 38508c2c6e feat(nix): base the data storage path defaults on the dataDir option 2026-05-30 00:37:55 +02:00
Lewis 7a54ccf6a3 fix(tranquil-config): reject unknown keys in TlsConfig, bump rust 1.96
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-29 09:39:14 +03:00
nelind e85394c314 build: bump version to 0.6.3 2026-05-27 19:28:10 +02:00
isabelandTangled 411c6be108 feat(nix): validate config at build time 2026-05-27 20:14:06 +03:00
isabelandTangled 28aa7ab7fc feat!(tranquil-config): error on unknown keys
an attempt to make it so that fixes like
fc6063dba8 never have to occur again
2026-05-27 20:13:58 +03:00
nelindandTangled 348b69d76b feat(nix): ensure that tranquil-store path is always writable by the service user just like file blob storage 2026-05-27 20:13:07 +03:00
nelindandTangled 4b23ca2c36 fix(docs): some wording on nix config and CI stuffs 2026-05-27 20:13:07 +03:00
LewisandTangled a7052e878c feat: tranquil's own TLS handling
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-27 13:54:33 +03:00
nelind fc6063dba8 fix(nix): override the right frontend package setting in the flake nixos module 2026-05-27 01:04:31 +02:00
LewisandTangled 44cb016762 fix(docs): install docs better
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-26 09:49:59 +03:00
LewisandTangled e6eee18ace fix(docs): migration oauth update
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-25 19:02:13 +03:00
Lewis 96c8375706 fix(test): oauth token eviction should be agnostic
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-25 16:17:10 +03:00
Lewis 86c5995568 fix(config): signal gate is useless since needs admin work to activate anyway
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-24 22:50:14 +03:00
LewisandTangled 036c317fd6 fix(migrate): oauth refreshing, embedded db first invite code
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-24 19:28:57 +03:00
LewisandTangled 4d2c7d4723 feat(auth): verification-gate override, inbound-migration bypass, store deleter improvement
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-23 23:02:43 +03:00
isabelandTangled f24a9f8bc0 build(nix): frontend pin pnpm
some future protections against this acursed toolchain breaking and
people who do .follows on their nixpkgs inputs
2026-05-22 23:24:45 +03:00
Lewis f6ef6ecbd9 feat(docs): first version of 'when PRing' section of contrib
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-21 22:46:59 +03:00
karitham.devandTangled a2567bdb1a fix(locales): french handle should be "identifier" not "username"
My bad translation on that earlier, made it consistent with the rest of the file and usage!
2026-05-21 10:39:29 +03:00
Lewis 56120d252d fix(locale): a little more drift
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-21 09:14:06 +03:00
TrezyandTangled 210b0f463c feat: add About page
Signed-off-by: Trezy <tre@trezy.com>
2026-05-21 09:06:42 +03:00
LewisandTangled c80f504dc0 fix(locale): a couple of drift keys missing in other langs
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-21 09:06:10 +03:00
karitham.devandTangled a3bd7c59ad feat(locales): add french locale, fix missing error keys, remove dead keys
Adds french locale registration and translation file. Fixes 4 missing error keys that were referenced in code but absent from all locale files. Removes 2 unreferenced keys from `register.validation` that duplicated `registerPasskey.errors`.

I *think* that's all that's needed for french locale? Couldn't find anything else required to make it work!
2026-05-20 22:20:20 +03:00
nelind 9ebde27540 fix(docs): markdown is tricky ... 2026-05-19 21:30:35 +02:00
nelindandTangled 28a7834304 docs: nel interjection: be right, be good, be fun 2026-05-19 22:18:38 +03:00
Lewis 191da5b311 chore: slight doc tweak
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-19 21:14:28 +03:00
Lewis a5b4ba7d65 feat(docs): init better docs
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-19 20:45:50 +03:00
TrezyandTangled 90dabd8840 docs: start a contributing file to track how to work with the repo
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 19eaccea74 chore: add dev startup to justfile
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled f620a6bc43 chore: add mailpit for local dev
Captures outbound emails, greatly simplifying local dev.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 04f370aaa1 chore: add frontend service to docker compose
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 3fd8f7ebbf chore: add app service with cargo-watch
Spins up a container with Tranquil running, but automatically uses incremental rebuilds for local dev.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 0afcb2ee28 chore: add plc directory for local dev
Prevents us from polluting the real PLC directory.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 8bd556f65b chore: add db health check to docker compose
Safer wafe to depend on database startup.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled cc92594506 chore: add traefik for local dev certs and routing
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 76f22b801b chore: allow native certs during local dev
Allows Traefik certs to be trusted by Tranquil.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 021b7dbec4 chore: add profiles to docker compose
Allows us to spin up groups from the compose file

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:57 +03:00
nelind eb034cb8b3 fix(ci): dont pass cachix cache name as a secret so it doesnt get redacted from CI logs 2026-05-17 17:07:34 +02:00
245 changed files with 14292 additions and 2524 deletions
+20
View File
@@ -106,6 +106,16 @@ slow-timeout = { period = "300s", terminate-after = 8 }
filter = "binary(compaction_restart) | binary(mst_refcount_integrity) | binary(gc_compaction_restart)"
slow-timeout = { period = "120s", terminate-after = 4 }
[[profile.default.overrides]]
filter = "test(/retention_time_travel_survives_many_seeds/)"
slow-timeout = { period = "300s", terminate-after = 24 }
test-group = "io-heavy-sim"
[[profile.default.overrides]]
filter = "binary(fd_lifecycle)"
slow-timeout = { period = "300s", terminate-after = 4 }
test-group = "io-heavy-sim"
[[profile.ci.overrides]]
filter = "test(/import_with_verification/) | test(/plc_migration/)"
test-group = "serial-env-tests"
@@ -137,3 +147,13 @@ test-group = "heavy-load-tests"
[[profile.ci.overrides]]
filter = "binary(repo_lifecycle)"
test-group = "heavy-load-tests"
[[profile.ci.overrides]]
filter = "test(/retention_time_travel_survives_many_seeds/)"
slow-timeout = { period = "300s", terminate-after = 24 }
test-group = "io-heavy-sim"
[[profile.ci.overrides]]
filter = "binary(fd_lifecycle)"
slow-timeout = { period = "300s", terminate-after = 4 }
test-group = "io-heavy-sim"
+12
View File
@@ -0,0 +1,12 @@
target/
.git/
.jj/
**/node_modules/
frontend/dist/
frontend/coverage/
frontend/.pnpm-store/
.direnv/
result
.env
*.output
reference-pds-bsky/
+3
View File
@@ -4,3 +4,6 @@ target/
result
frontend/node_modules/
frontend/dist/
frontend/coverage/
frontend/.pnpm-store
frontend/.npmrc
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT nextval('firehose_seq') as \"seq!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "09142b5a6b91fe4fd795fb6c85af5461e2621f6c3545b6d26e4618d813904b75"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE repo_seq r\n SET seq = $1::bigint + (o.ord - 1)\n FROM unnest($2::bigint[]) WITH ORDINALITY AS o(id, ord)\n WHERE r.id = o.id",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8Array"
]
},
"nullable": []
},
"hash": "0b895d105363a60040323c5c47bc897905af3da980d0dc1b3b60bbdcdd90ea7a"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM repo_seq\n WHERE did = $1\n AND id <> (SELECT id FROM repo_seq WHERE did = $1 ORDER BY id DESC LIMIT 1)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "122d79e006c0e11214e3cf11cfb98bdff6fbf156893b482572532ebcd8979e61"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin\n FROM users WHERE did = $1",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration\n FROM users WHERE handle = $1",
"describe": {
"columns": [
{
@@ -42,6 +42,11 @@
"ordinal": 7,
"name": "is_admin",
"type_info": "Bool"
},
{
"ordinal": 8,
"name": "inbound_migration",
"type_info": "Bool"
}
],
"parameters": {
@@ -57,8 +62,9 @@
false,
true,
true,
false,
false
]
},
"hash": "6b51995c40519a63f85c70f29ca8bd6ec1963c8562d78215d980785dc46a6384"
"hash": "18bbda5582db1b32d02ab8a3eee970c9508b9bd67239c2f936639a9f863b30ff"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE users SET deactivated_at = NULL WHERE did = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "23201d4e26bc650939e30f69fb0bca00d351d057098afebc1017f70a84b4bd22"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', false, 'deleted')\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', false, 'deleted')\n RETURNING id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "id",
"type_info": "Int8"
}
],
@@ -18,5 +18,5 @@
false
]
},
"hash": "a0ea3630adb9e411d94d88084df6b05d7fc3ba8c306146fc932b736fe342eceb"
"hash": "2c3c0740793f29b9bd9e0f81c2372d427dee091e20da8f71ae1e5f2769aa985d"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_try_advisory_xact_lock($1) as \"locked!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "locked!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "3cc7247d2cd7e85a6f21d783dd956f8fecc6f63c994014bb4e91504d498b5fd5"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, handle)\n VALUES ($1, 'identity', $2)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "4f3ef746cb5fc0916d3df5f92494c538497d3b9c1cfcc6e44c9b80f2193094f6"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, handle)\n VALUES ($1, 'identity', $2)\n RETURNING seq\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "50b2178cd174af1dc132a22048e52a6af492f4d8025ec512dd9c0c9ba62eb202"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE users SET deactivated_at = NULL, inbound_migration = FALSE WHERE did = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "51e029e572777e6a103fd7fd5550494de9d4cac7e3ff84e27ddec1a6aaefc047"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id as \"id!\" FROM repo_seq WHERE seq IS NULL ORDER BY id LIMIT $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "714a1c9a73e2fed8182d4ab2fe4a2c33bc10a52e27ed95d3a6641477ae42395e"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq = $1",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -90,7 +90,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -108,5 +108,5 @@
true
]
},
"hash": "4702a94384ce938d3f5384bc2106a48500f8749134e6bfd1c619a9151f5d3772"
"hash": "7cdf3b8136782a5b2e4aff91907a680566b4610e6cbf9d9396d31f9bbd99c3b8"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, rev)\n VALUES ($1, 'commit', $2, $3::TEXT, $4, $5, $6, $7, $8)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, rev)\n VALUES ($1, 'commit', $2, $3::TEXT, $4, $5, $6, $7, $8)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -21,9 +15,7 @@
"Text"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "a325862f897484bb1a8863fd7a1bc3fd17ab20b6e30498f9900daf80609c413d"
"hash": "800b93a38eef70bafdf0a1b221994f3c19ab562f84d18265376296a592b4c767"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -23,9 +17,7 @@
"Text"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "97dd3cfe72b74b4dac686c030a07f6eb56dabb2df72815f3248760b4e3f7950e"
"hash": "82276a34f473ff8b9790b78915ae3f72873d4b748cdb47bd2b81edc2f7a6e5a8"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM repo_seq WHERE did = $1 AND seq != $2",
"query": "DELETE FROM repo_seq WHERE did = $1 AND id <> $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "b6d6548acb89d6384cd226f6ed0d66de27fde3af24b4a7a3fce7e098812e38a5"
"hash": "856e458c1c78f030545dc147f76d1b91fa4cc6e178eb51133c52358855ba3f02"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', $2, $3)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', $2, $3)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -16,9 +10,7 @@
"Text"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "623042c0b901e17ff37c5bcc1a4eefe56fc8b996ca8326e75796897d13c9e9ed"
"hash": "8ccd42b5b1b75f45e094889ee26af8517de28ca687efa81fd9c4fce598338fe9"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -90,7 +90,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -108,5 +108,5 @@
true
]
},
"hash": "c47746c256fd8c9d386551f3bd270e9a96fecdb713642d07d60fb4aeb02c8426"
"hash": "957bc81c63510ab502d2f8cea75dafcbdf2c8ebb448c86990d5e404ee5280438"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, rev, block_cids, block_data)\n VALUES ($1, 'sync', $2, $3, $4, $5)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, rev, block_cids, block_data)\n VALUES ($1, 'sync', $2, $3, $4, $5)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -18,9 +12,7 @@
"ByteaArray"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "5996774d097a484f81353d49e4a99b07c30a003377c924add576bfb48ddde29b"
"hash": "9ba7869f29286cfd6b811ffa1e26bda1d637b98b44be66cdc265b1af4cec2d4f"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_advisory_xact_lock($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "pg_advisory_xact_lock",
"type_info": "Void"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "a06e1d9f6f95e4c4c2b98310ebddcc9d963cc033582bf2e945e8bf3a301b4247"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT setval('firehose_seq', $1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "setval",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "bd0407818c8bdae1f1024777951b08cb9c386fe751722e1caf6ad243e271697a"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "NOTIFY repo_updates",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "d4eacb9bca838dd18dcfd8ca39b83ef50968bfaaabc213536c049cc222ffe9c8"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -91,7 +91,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -109,5 +109,5 @@
true
]
},
"hash": "89c8ce80d8c52b4668c34f63e48b51cacc7e8ff300958a192378f1d72de04f9a"
"hash": "da2770e44c0ec21756ce4b6b0c57abff4f38cd8c680c5ced7d3b35f3c1f58da7"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1 AND seq < $2\n ORDER BY seq ASC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -91,7 +91,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -109,5 +109,5 @@
true
]
},
"hash": "0349b2af4d21105029ecc2bf81ad7be9987ec17ac088730596849d7a45558e57"
"hash": "dfe011b2d2bafbbbbea508a4977fc34bb1be4cf2612296df23763e6e5841c33e"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1 AND seq < $2\n ORDER BY seq ASC",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -91,7 +91,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -109,5 +109,5 @@
true
]
},
"hash": "ad784aff9bb5c1f19f90a3b1c3b3694a3e1aa889b0fe3e55cfebb7af8c620256"
"hash": "f16d41b41a5388997e6ed408da55056173340ed12b60b1a956de87f73f9d0824"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin\n FROM users WHERE handle = $1",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration\n FROM users WHERE did = $1",
"describe": {
"columns": [
{
@@ -42,6 +42,11 @@
"ordinal": 7,
"name": "is_admin",
"type_info": "Bool"
},
{
"ordinal": 8,
"name": "inbound_migration",
"type_info": "Bool"
}
],
"parameters": {
@@ -57,8 +62,9 @@
false,
true,
true,
false,
false
]
},
"hash": "584bceda60d8b6a02e7dc44d833e3fba13151f36ba9f75c64e33d6cb534cc939"
"hash": "f1c4ec28b02d09ffce35aa8249c1747a70c12a3ecfc1ff6ca1847840f770db2f"
}
@@ -18,4 +18,4 @@ steps:
- name: Build and push aarch64
command: |
SUBS="--option extra-substituters https://tranquil.cachix.org --option extra-trusted-public-keys tranquil.cachix.org-1:PoO+mGL6a6LcJiPakMDHN4E218/ei/7v2sxeDtNkSRg="
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds-aarch64 $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push "$CACHIX_CACHE_NAME"
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds-aarch64 $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push tranquil
+3 -3
View File
@@ -18,11 +18,11 @@ steps:
- name: Build and push x86_64
command: |
SUBS="--option extra-substituters https://tranquil.cachix.org --option extra-trusted-public-keys tranquil.cachix.org-1:PoO+mGL6a6LcJiPakMDHN4E218/ei/7v2sxeDtNkSRg="
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push "$CACHIX_CACHE_NAME"
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-frontend $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push "$CACHIX_CACHE_NAME"
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push tranquil
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-frontend $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push tranquil
- name: Build and push devShell
command: |
SUBS="--option extra-substituters https://tranquil.cachix.org --option extra-trusted-public-keys tranquil.cachix.org-1:PoO+mGL6a6LcJiPakMDHN4E218/ei/7v2sxeDtNkSRg="
nix develop $SUBS --profile dev-profile -c true
cachix push "$CACHIX_CACHE_NAME" dev-profile
cachix push tranquil dev-profile
+135
View File
@@ -0,0 +1,135 @@
# Contributing to Tranquil PDS
## When PRing
In order of importance:
- **You must run your change! Every contribution that says "here's xyz. untested." does not help the project.**
- Relevant tests to your PR must pass. The whole suite doesn't have to be proven to have run, because there are a *ton* of tests and they're quite heavy, but hopefully there are existing tests for whatever you're PRing, and if there aren't, please add those too.
- Run cargo fmt :P
> 🦪 Lewis
>
> Good CI fixes some of these. We should really get around to that.
Things that would also be nice but aren't like, a pain in our side:
- Big changes should be stacked PRs that are broken up into digestible pieces. Those stacked PRs should hopefully be able to be merged individually if necessary.
## Local Development
### Prerequisites
- [Docker](https://docs.docker.com/get-docker/) and Docker Compose
- Add `pds.test` to your hosts file (one-time setup):
```
127.0.0.1 pds.test
```
- **macOS / Linux:** `/etc/hosts`
- **Windows:** `C:\Windows\System32\drivers\etc\hosts`
### Starting the dev environment
```bash
just run-dev
```
This starts the following services via `docker-compose`:
- **Traefik** — HTTPS reverse proxy at `https://pds.test`
- **Backend** — Rust server with `cargo-watch` (auto-rebuilds on file changes)
- **Frontend** — Vite dev server with hot module replacement
- **Postgres** — Database on port 5432
- **PLC Directory** — Local [did-method-plc](https://github.com/did-method-plc/did-method-plc) server for DID registration
- **Mailpit** — Local email server with web UI at [http://localhost:8025](http://localhost:8025)
Once all services are running, open **https://pds.test** in your browser.
### Trusting the self-signed certificate
Traefik generates a self-signed TLS certificate. Your browser will show a security warning on first visit. You can either click through it, or add the certificate to your system trust store for a seamless experience:
**macOS:**
```bash
# Extract the cert from traefik and add it to the system keychain
echo | openssl s_client -connect localhost:443 -servername pds.test 2>/dev/null | openssl x509 > /tmp/pds-test.pem
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /tmp/pds-test.pem
```
**Linux (Debian/Ubuntu):**
```bash
echo | openssl s_client -connect localhost:443 -servername pds.test 2>/dev/null | openssl x509 | sudo tee /usr/local/share/ca-certificates/pds-test.crt
sudo update-ca-certificates
```
**Linux (Fedora/RHEL):**
```bash
echo | openssl s_client -connect localhost:443 -servername pds.test 2>/dev/null | openssl x509 | sudo tee /etc/pki/ca-trust/source/anchors/pds-test.pem
sudo update-ca-trust
```
**Windows (PowerShell as Administrator):**
```powershell
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cert.Import([System.Text.Encoding]::UTF8.GetBytes((echo | openssl s_client -connect localhost:443 -servername pds.test 2>$null | openssl x509)))
$store = New-Object System.Security.Cryptography.X509Certificates.X509Store("Root", "LocalMachine")
$store.Open("ReadWrite")
$store.Add($cert)
$store.Close()
```
Restart your browser after adding the certificate.
### Stopping the dev environment
```bash
# Stop containers (preserves database + build cache)
docker compose --profile dev down
# Stop and wipe all data (fresh start)
docker compose --profile dev down -v
```
### Direct database access
Postgres is exposed on port 5432:
```bash
psql postgres://postgres:postgres@localhost:5432/pds
```
### How it works
- **Source code** is bind-mounted into the containers so that changes made on the host will be immediately reflected in the application
- **Backend** uses `cargo-watch` to recompile and restart when Rust files change
- **Frontend** uses Vite's HMR for instant browser updates when frontend files change
- **Build cache** (`target/` directory and cargo registry) are stored in Docker volumes, so incremental compilation persists across container restarts
- **Traefik** routes `/`, `/xrpc`, `/oauth`, `/.well-known`, `/u`, and `/health` to the backend; everything else goes to the Vite dev server
- **Mailpit** captures all outgoing email — open [http://localhost:8025](http://localhost:8025) to view verification emails during registration
- **PLC Directory** runs locally so DID registration doesn't hit the real `plc.directory`
### Running the backend natively
If you prefer running the Rust backend outside Docker (faster incremental builds on host), you need:
- Rust toolchain (see `rust-toolchain.toml`)
- `protoc` (`brew install protobuf` on macOS)
- PostgreSQL (start with `docker compose up db`)
Then run:
```bash
cargo run -p tranquil-server -- --config config.toml
```
And start the frontend separately:
```bash
cd frontend && pnpm install && pnpm dev
```
Generated
+179 -32
View File
@@ -210,7 +210,7 @@ version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5493c3bedbacf7fd7382c6346bbd66687d12bbaad3a89a2d2c303ee6cf20b048"
dependencies = [
"asn1-rs-derive",
"asn1-rs-derive 0.5.1",
"asn1-rs-impl",
"displaydoc",
"nom 7.1.3",
@@ -220,6 +220,22 @@ dependencies = [
"time",
]
[[package]]
name = "asn1-rs"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8"
dependencies = [
"asn1-rs-derive 0.6.0",
"asn1-rs-impl",
"displaydoc",
"nom 7.1.3",
"num-traits",
"rusticata-macros",
"thiserror 2.0.18",
"time",
]
[[package]]
name = "asn1-rs-derive"
version = "0.5.1"
@@ -232,6 +248,18 @@ dependencies = [
"synstructure",
]
[[package]]
name = "asn1-rs-derive"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
"synstructure",
]
[[package]]
name = "asn1-rs-impl"
version = "0.2.0"
@@ -1046,7 +1074,7 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
dependencies = [
"bit-vec",
"bit-vec 0.8.0",
]
[[package]]
@@ -1055,6 +1083,15 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
[[package]]
name = "bit-vec"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b71798fca2c1fe1086445a7258a4bc81e6e49dcd24c8d0dd9a1e57395b603f51"
dependencies = [
"serde",
]
[[package]]
name = "bitflags"
version = "2.11.0"
@@ -1970,7 +2007,21 @@ version = "9.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cd0a5c643689626bec213c4d8bd4d96acc8ffdb4ad4bb6bc16abf27d5f4b553"
dependencies = [
"asn1-rs",
"asn1-rs 0.6.2",
"displaydoc",
"nom 7.1.3",
"num-bigint",
"num-traits",
"rusticata-macros",
]
[[package]]
name = "der-parser"
version = "10.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6"
dependencies = [
"asn1-rs 0.7.2",
"displaydoc",
"nom 7.1.3",
"num-bigint",
@@ -2834,6 +2885,34 @@ dependencies = [
"tracing",
]
[[package]]
name = "h3"
version = "0.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10872b55cfb02a821b69dc7cf8dc6a71d6af25eb9a79662bec4a9d016056b3be"
dependencies = [
"bytes",
"fastrand",
"futures-util",
"http 1.4.0",
"pin-project-lite",
"tokio",
]
[[package]]
name = "h3-quinn"
version = "0.0.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b2e732c8d91a74731663ac8479ab505042fbf547b9a207213ab7fbcbfc4f8b4"
dependencies = [
"bytes",
"futures",
"h3",
"quinn",
"tokio",
"tokio-util",
]
[[package]]
name = "half"
version = "2.7.1"
@@ -4614,7 +4693,16 @@ version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8d8034d9489cdaf79228eb9f6a3b8d7bb32ba00d6645ebd48eef4077ceb5bd9"
dependencies = [
"asn1-rs",
"asn1-rs 0.6.2",
]
[[package]]
name = "oid-registry"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7"
dependencies = [
"asn1-rs 0.7.2",
]
[[package]]
@@ -5184,7 +5272,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37566cb3fdacef14c0737f9546df7cfeadbfbc9fef10991038bf5015d0c80532"
dependencies = [
"bit-set",
"bit-vec",
"bit-vec 0.8.0",
"bitflags",
"num-traits",
"rand 0.9.2",
@@ -5427,6 +5515,7 @@ checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20"
dependencies = [
"bytes",
"cfg_aliases",
"futures-io",
"pin-project-lite",
"quinn-proto",
"quinn-udp",
@@ -5630,6 +5719,20 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "rcgen"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57f6d249aad744e274e682777a50283a225a32705394ee6d5fcc01efa25e4055"
dependencies = [
"pem",
"ring",
"rustls-pki-types",
"time",
"x509-parser 0.18.1",
"yasna",
]
[[package]]
name = "redis"
version = "1.1.0"
@@ -7527,7 +7630,7 @@ dependencies = [
[[package]]
name = "tranquil-api"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"anyhow",
"axum",
@@ -7578,7 +7681,7 @@ dependencies = [
[[package]]
name = "tranquil-auth"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"anyhow",
"base32",
@@ -7601,7 +7704,7 @@ dependencies = [
[[package]]
name = "tranquil-cache"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7615,7 +7718,7 @@ dependencies = [
[[package]]
name = "tranquil-comms"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7641,7 +7744,7 @@ dependencies = [
[[package]]
name = "tranquil-config"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"confique",
"serde",
@@ -7649,7 +7752,7 @@ dependencies = [
[[package]]
name = "tranquil-crypto"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"aes-gcm",
"base64 0.22.1",
@@ -7665,7 +7768,7 @@ dependencies = [
[[package]]
name = "tranquil-db"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"chrono",
@@ -7682,7 +7785,7 @@ dependencies = [
[[package]]
name = "tranquil-db-traits"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7698,7 +7801,7 @@ dependencies = [
[[package]]
name = "tranquil-infra"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"bytes",
@@ -7709,7 +7812,7 @@ dependencies = [
[[package]]
name = "tranquil-lexicon"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"chrono",
"futures",
@@ -7728,7 +7831,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"anyhow",
"axum",
@@ -7751,7 +7854,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth-server"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"axum",
"base64 0.22.1",
@@ -7784,7 +7887,7 @@ dependencies = [
[[package]]
name = "tranquil-pds"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"aes-gcm",
"anyhow",
@@ -7837,6 +7940,7 @@ dependencies = [
"sha2",
"sqlx",
"subtle",
"tempfile",
"testcontainers",
"testcontainers-modules",
"thiserror 2.0.18",
@@ -7877,7 +7981,7 @@ dependencies = [
[[package]]
name = "tranquil-repo"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"bytes",
"cid",
@@ -7889,7 +7993,7 @@ dependencies = [
[[package]]
name = "tranquil-ripple"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"backon",
@@ -7914,7 +8018,7 @@ dependencies = [
[[package]]
name = "tranquil-scopes"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"axum",
"futures",
@@ -7930,15 +8034,30 @@ dependencies = [
[[package]]
name = "tranquil-server"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"arc-swap",
"axum",
"bytes",
"clap",
"dotenvy",
"ed25519-dalek",
"futures-util",
"h3",
"h3-quinn",
"hex",
"http 1.4.0",
"hyper 1.8.1",
"hyper-util",
"quinn",
"rcgen",
"rustls 0.23.37",
"rustls-pemfile",
"thiserror 2.0.18",
"tokio",
"tokio-rustls 0.26.4",
"tokio-util",
"tower",
"tracing",
"tracing-subscriber",
"tranquil-api",
@@ -7951,7 +8070,7 @@ dependencies = [
[[package]]
name = "tranquil-signal"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"chrono",
@@ -7974,7 +8093,7 @@ dependencies = [
[[package]]
name = "tranquil-storage"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"aws-config",
@@ -7991,7 +8110,7 @@ dependencies = [
[[package]]
name = "tranquil-store"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"async-trait",
"bytes",
@@ -8040,7 +8159,7 @@ dependencies = [
[[package]]
name = "tranquil-sync"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"anyhow",
"axum",
@@ -8062,7 +8181,7 @@ dependencies = [
[[package]]
name = "tranquil-types"
version = "0.6.2"
version = "0.6.5"
dependencies = [
"chrono",
"cid",
@@ -8575,7 +8694,7 @@ checksum = "15784340a24c170ce60567282fb956a0938742dbfbf9eff5df793a686a009b8b"
dependencies = [
"base64 0.21.7",
"base64urlsafedata",
"der-parser",
"der-parser 9.0.0",
"hex",
"nom 7.1.3",
"openssl",
@@ -8591,7 +8710,7 @@ dependencies = [
"uuid",
"webauthn-attestation-ca",
"webauthn-rs-proto",
"x509-parser",
"x509-parser 0.16.0",
]
[[package]]
@@ -9145,17 +9264,35 @@ version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fcbc162f30700d6f3f82a24bf7cc62ffe7caea42c0b2cba8bf7f3ae50cf51f69"
dependencies = [
"asn1-rs",
"asn1-rs 0.6.2",
"data-encoding",
"der-parser",
"der-parser 9.0.0",
"lazy_static",
"nom 7.1.3",
"oid-registry",
"oid-registry 0.7.1",
"rusticata-macros",
"thiserror 1.0.69",
"time",
]
[[package]]
name = "x509-parser"
version = "0.18.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202"
dependencies = [
"asn1-rs 0.7.2",
"data-encoding",
"der-parser 10.0.0",
"lazy_static",
"nom 7.1.3",
"oid-registry 0.8.1",
"ring",
"rusticata-macros",
"thiserror 2.0.18",
"time",
]
[[package]]
name = "xattr"
version = "1.6.1"
@@ -9184,6 +9321,16 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfe53a6657fd280eaa890a3bc59152892ffa3e30101319d168b781ed6529b049"
[[package]]
name = "yasna"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5f6765e852b9b4dc8e2a76843e4d64d1cea8e79bcde0b6901aea8e7c7f08282"
dependencies = [
"bit-vec 0.9.1",
"time",
]
[[package]]
name = "yoke"
version = "0.8.1"
+12 -2
View File
@@ -26,7 +26,7 @@ members = [
]
[workspace.package]
version = "0.6.2"
version = "0.6.5"
edition = "2024"
license = "AGPL-3.0-or-later"
@@ -59,6 +59,7 @@ presage = { git = "https://github.com/whisperfish/presage", rev = "fe3ed54c4844a
unicode-segmentation = "1"
aes-gcm = "0.10"
arc-swap = "1"
backon = "1"
bincode = { version = "2", features = ["serde"] }
anyhow = "1.0"
@@ -81,11 +82,15 @@ foca = { version = "1", features = ["bincode-codec", "tracing"] }
futures = "0.3"
futures-util = "0.3"
governor = "0.10"
h3 = "0.0.8"
h3-quinn = "0.0.10"
hex = "0.4"
hickory-resolver = { version = "0.24", features = ["tokio-runtime"] }
hkdf = "0.12"
hmac = "0.12"
http = "1.4"
hyper = { version = "1", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "server-auto", "server-graceful", "service", "tokio"] }
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "gif", "webp"] }
qrcodegen = "1.8"
infer = "0.19"
@@ -103,10 +108,14 @@ parking_lot = "0.12"
multihash = "0.19"
p256 = { version = "0.13", features = ["ecdsa"] }
p384 = { version = "0.13", features = ["ecdsa"] }
quinn = { version = "0.11", default-features = false, features = ["runtime-tokio", "rustls-ring", "log"] }
rand = "0.8"
rcgen = { version = "0.14", default-features = false, features = ["ring", "pem"] }
redis = { version = "1.0", features = ["tokio-comp", "connection-manager"] }
regex = "1"
rsa = "0.9"
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
rustls-pemfile = "2"
secrecy = { version = "0.10", features = ["serde"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots", "http2", "charset", "macos-system-configuration"] }
serde = { version = "1.0", features = ["derive"] }
@@ -119,8 +128,9 @@ sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "postgres", "uuid"
subtle = "2.5"
thiserror = "2.0"
tokio = { version = "1.48", features = ["macros", "rt-multi-thread", "time", "signal", "process", "io-util", "fs"] }
tokio-util = "0.7.18"
tokio-util = { version = "0.7.18", features = ["rt"] }
tokio-tungstenite = { version = "0.28", features = ["rustls-tls-webpki-roots"] }
tokio-rustls = { version = "0.26", default-features = false, features = ["ring", "tls12", "logging"] }
totp-rs = { version = "5", features = ["qr"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["fs", "cors"] }
+2 -2
View File
@@ -6,7 +6,7 @@ RUN pnpm install --frozen-lockfile
COPY frontend/ ./
RUN pnpm build
FROM rust:1.92-alpine AS builder
FROM rust:1.96-alpine AS builder
RUN apk add --no-cache ca-certificates musl-dev pkgconfig openssl-dev openssl-libs-static mold clang protoc
ENV RUSTFLAGS="-C linker=clang -C link-arg=-fuse-ld=mold"
WORKDIR /app
@@ -50,7 +50,7 @@ RUN apk add --no-cache ca-certificates
COPY --from=builder /tmp/tranquil-pds /usr/local/bin/tranquil-pds
COPY --from=frontend /app/dist /var/lib/tranquil-pds/frontend
WORKDIR /app
ENV SERVER_HOST=0.0.0.0
ENV SERVER_HOST=[::]
ENV SERVER_PORT=3000
EXPOSE 3000
CMD ["tranquil-pds"]
+20
View File
@@ -0,0 +1,20 @@
FROM rust:1.96-alpine
RUN apk add --no-cache \
ca-certificates \
musl-dev \
pkgconfig \
openssl-dev \
openssl-libs-static \
mold \
clang \
protoc
RUN cargo install cargo-watch
ENV RUSTFLAGS="-C linker=clang -C link-arg=-fuse-ld=mold"
ENV SQLX_OFFLINE=true
WORKDIR /app
CMD ["cargo", "watch", "-x", "run -p tranquil-server -- --config /app/config.dev.toml"]
+14 -8
View File
@@ -6,7 +6,7 @@ A Personal Data Server for the AT Protocol.
We came together to make this PDS to enable and empower our users to better host their data on this shared protocol. All of our decisions as a project are guided by their usefulness to the community: PDS hosters and end-users both.
Comparatively: Bluesky the company created a "reference PDS" that we can self-host quite easily, and that's great, but Bluesky has an incentive to make software for themselvess first & foremost, then secondly their software can be useful for us self-hosters. In contrast, Tranquil is not from a company, and will never be.
Comparatively: Bluesky the company created a "reference PDS" that we can self-host quite easily, and that's great, but Bluesky has an incentive to make software for themselves first & foremost, then secondly their software can be useful for us self-hosters. In contrast, Tranquil is not from a company, and will never be.
## What's different about Tranquil PDS
@@ -20,7 +20,7 @@ It is a superset of the reference PDS, including:
- account delegation: letting others manage an account with configurable permission levels
- a built-in web UI for account management, repo browsing, and admin
Unlike the ref PDS, Tranquil itself is compiled to a single binary with no nodeJS runtime. However, at time of writing, Tranquil requires postgres running separately.
Unlike the ref PDS, Tranquil is a single binary with no nodejs runtime. That said, at time of writing, Tranquil does require postgres running separately.
## Quick Start
@@ -46,24 +46,28 @@ just test
just lint
```
Nix users can enter a devshell with `nix develop`, or `direnv allow` to auto-enter via the bundled `.envrc`. Pre-built artifacts (including the devshell) are available from our [binary cache](docs/install-nix.md#binary-cache).
Nix users can enter a devshell with `nix develop`, or `direnv allow` to auto-enter via the bundled `.envrc`. Pre-built artifacts including the devshell are available from our [binary cache](docs/2_INSTALL_NIX.md#binary-cache).
## Production Deployment
### Quick Deploy (Docker/Podman Compose)
Edit `config.toml` with your values. Generate secrets with `openssl rand -base64 48`.
`docker-compose.prod.yaml` pulls the prebuilt image `atcr.io/tranquil.farm/tranquil-pds:latest`. Sign in to the registry first with `podman login atcr.io`. The Containers guide covers building from source.
```bash
cp example.toml config.toml
```
Edit `config.toml` with your values and generate secrets with `openssl rand -base64 48`. Set the postgres password to match `docker-compose.prod.yaml`. nginx needs a TLS certificate before it starts, so follow the wildcard cert steps in the [Containers guide](docs/2_INSTALL_CONTAINERS.md).
```bash
podman-compose -f docker-compose.prod.yaml up -d
```
### Installation Guides
- [Nix](docs/install-nix.md)
- [Containers](docs/install-containers.md)
- [Kubernetes](docs/install-kubernetes.md)
- [Nix](docs/2_INSTALL_NIX.md)
- [Containers](docs/2_INSTALL_CONTAINERS.md)
## Community
@@ -76,7 +80,7 @@ We currently don't have a shared space to chat and organize Tranquil things, but
- [@oyster.cafe](https://tangled.org/did:plc:3fwecdnvtcscjnrx2p4n7alz)
- [@nel.pet](https://tangled.org/did:plc:h5wsnqetncv6lu2weom35lg2)
### Amazing contributers
### Amazing contributors
- [@isabelroses.com](https://tangled.org/did:plc:qxichs7jsycphrsmbujwqbfb)
- [@quilling.dev](https://tangled.org/did:plc:jrtgsidnmxaen4offglr5lsh)
@@ -87,6 +91,8 @@ We currently don't have a shared space to chat and organize Tranquil things, but
- [@a.starrysky.fyi](https://tangled.org/did:plc:uuyqs6y3pwtbteet4swt5i5y)
- [@sans-self.org](https://tangled.org/did:plc:wydyrngmxbcsqdvhmd7whmye)
- [@tachyonism.tngl.sh](https://tangled.org/did:plc:w6qiwij62bmdugsd3gemhpy2)
- [@trezy.codes](https://tangled.org/did:plc:4jrld6fwpnwqehtce56qshzv)
- [@ave.zone](https://tangled.org/did:plc:mchrltkrhuzpxleiwpmmvpar)
- Could be your name here too!
### Tranquil PDS instances in the wild!
+30
View File
@@ -0,0 +1,30 @@
[server]
hostname = "pds.test"
allow_http_proxy = true
invite_code_required = false
disable_rate_limiting = true
[frontend]
enabled = true
dir = "/app/frontend/public"
[database]
url = "postgres://postgres:postgres@db:5432/pds"
[storage]
path = "/var/lib/tranquil-pds/blobs"
[plc]
directory_url = "http://plc:2582"
[email]
from_address = "noreply@pds.test"
from_name = "Tranquil PDS (Dev)"
[email.smarthost]
host = "mailpit"
port = 1025
tls = "none"
[secrets]
allow_insecure = true
+1 -3
View File
@@ -9,7 +9,6 @@ use tranquil_pds::state::AppState;
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct SignalStatusOutput {
pub enabled: bool,
pub linked: bool,
}
@@ -23,13 +22,12 @@ pub async fn get_signal_status(
State(state): State<AppState>,
_auth: Auth<Admin>,
) -> Result<Json<SignalStatusOutput>, ApiError> {
let enabled = tranquil_config::get().signal.enabled;
let linked = match &state.signal_sender {
Some(slot) => slot.is_linked().await,
None => false,
};
Ok(Json(SignalStatusOutput { enabled, linked }))
Ok(Json(SignalStatusOutput { linked }))
}
pub async fn link_signal_device(
@@ -510,6 +510,7 @@ pub async fn create_account(
telegram_username: comms.telegram,
signal_username: comms.signal,
deactivated_at,
inbound_migration: is_migration || is_did_web_byod,
encrypted_key_bytes: repo.encrypted_key_bytes,
encryption_version: tranquil_pds::config::ENCRYPTION_VERSION,
reserved_key_id,
+1 -1
View File
@@ -3,5 +3,5 @@ mod sign;
mod submit;
pub use request::request_plc_operation_signature;
pub use sign::{ServiceInput, SignPlcOperationInput, SignPlcOperationOutput, sign_plc_operation};
pub use sign::{SignPlcOperationInput, SignPlcOperationOutput, sign_plc_operation};
pub use submit::{SubmitPlcOperationInput, submit_plc_operation};
@@ -6,10 +6,6 @@ use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{Auth, Permissive};
use tranquil_pds::state::AppState;
fn generate_plc_token() -> String {
tranquil_pds::util::generate_token_code()
}
pub async fn request_plc_operation_signature(
State(state): State<AppState>,
auth: Auth<Permissive>,
@@ -28,12 +24,13 @@ pub async fn request_plc_operation_signature(
.ok_or(ApiError::AccountNotFound)?;
let _ = state.repos.infra.delete_plc_tokens_for_user(user_id).await;
let plc_token = generate_plc_token();
let display_token = tranquil_pds::util::generate_token_code();
let stored_token = tranquil_pds::util::normalize_token_code(&display_token);
let expires_at = Utc::now() + Duration::minutes(10);
state
.repos
.infra
.insert_plc_token(user_id, &plc_token, expires_at)
.insert_plc_token(user_id, &stored_token, expires_at)
.await
.log_db_err("creating PLC token")?;
@@ -42,7 +39,7 @@ pub async fn request_plc_operation_signature(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
user_id,
&plc_token,
&display_token,
hostname,
)
.await
+5 -24
View File
@@ -9,7 +9,7 @@ use tranquil_pds::api::ApiError;
use tranquil_pds::api::error::DbResultExt;
use tranquil_pds::auth::{Auth, Permissive};
use tranquil_pds::circuit_breaker::with_circuit_breaker;
use tranquil_pds::plc::{PlcError, PlcService, ServiceType, create_update_op, sign_operation};
use tranquil_pds::plc::{PlcError, PlcService, create_update_op, sign_operation};
use tranquil_pds::state::AppState;
#[derive(Debug, Deserialize)]
@@ -19,14 +19,7 @@ pub struct SignPlcOperationInput {
pub rotation_keys: Option<Vec<String>>,
pub also_known_as: Option<Vec<String>>,
pub verification_methods: Option<HashMap<String, String>>,
pub services: Option<HashMap<String, ServiceInput>>,
}
#[derive(Debug, Deserialize, Clone)]
pub struct ServiceInput {
#[serde(rename = "type")]
pub service_type: ServiceType,
pub endpoint: String,
pub services: Option<HashMap<String, PlcService>>,
}
#[derive(Debug, Serialize)]
@@ -50,9 +43,10 @@ pub async fn sign_plc_operation(
"PLC operations are only valid for did:plc identities".into(),
));
}
let token = input.token.as_ref().ok_or_else(|| {
let raw_token = input.token.as_ref().ok_or_else(|| {
ApiError::InvalidRequest("Email confirmation token required to sign PLC operations".into())
})?;
let token = &tranquil_pds::util::normalize_token_code(raw_token);
let user_id = state
.repos
@@ -106,25 +100,12 @@ pub async fn sign_plc_operation(
if last_op.is_tombstone() {
return Err(ApiError::from(PlcError::Tombstoned));
}
let services = input.services.map(|s| {
s.into_iter()
.map(|(k, v)| {
(
k,
PlcService {
service_type: v.service_type,
endpoint: v.endpoint,
},
)
})
.collect()
});
let unsigned_op = create_update_op(
&last_op,
input.rotation_keys,
input.verification_methods,
input.also_known_as,
services,
input.services,
)
.map_err(|e| match e {
PlcError::Tombstoned => ApiError::InvalidRequest("Cannot update tombstoned DID".into()),
@@ -129,20 +129,13 @@ pub async fn submit_plc_operation(
.await
.map_err(ApiError::from)?;
match state
if let Err(e) = state
.repos
.repo
.insert_identity_event(did, Some(&user.handle))
.await
{
Ok(seq) => {
if let Err(e) = state.repos.repo.notify_update(seq).await {
warn!("Failed to notify identity event: {:?}", e);
}
}
Err(e) => {
warn!("Failed to sequence identity event: {:?}", e);
}
warn!("Failed to sequence identity event: {:?}", e);
}
let _ = state
.cache
+12 -2
View File
@@ -15,8 +15,12 @@ pub mod temp;
use tranquil_pds::state::AppState;
pub fn api_routes() -> axum::Router<AppState> {
use axum::extract::DefaultBodyLimit;
use axum::routing::{get, post};
let blob_body_limit =
DefaultBodyLimit::max(tranquil_config::get().server.max_blob_size as usize);
axum::Router::new()
.route("/_health", get(server::health))
.route(
@@ -68,7 +72,10 @@ pub fn api_routes() -> axum::Router<AppState> {
.route("/com.atproto.repo.deleteRecord", post(repo::delete_record))
.route("/com.atproto.repo.listRecords", get(repo::list_records))
.route("/com.atproto.repo.describeRepo", get(repo::describe_repo))
.route("/com.atproto.repo.uploadBlob", post(repo::upload_blob))
.route(
"/com.atproto.repo.uploadBlob",
post(repo::upload_blob).layer(blob_body_limit),
)
.route("/com.atproto.repo.applyWrites", post(repo::apply_writes))
.route(
"/com.atproto.server.checkAccountStatus",
@@ -247,7 +254,10 @@ pub fn api_routes() -> axum::Router<AppState> {
"/_identity.verifyHandleOwnership",
post(identity::verify_handle_ownership),
)
.route("/com.atproto.repo.importRepo", post(repo::import_repo))
.route(
"/com.atproto.repo.importRepo",
post(repo::import_repo).layer(blob_body_limit),
)
.route(
"/com.atproto.admin.deleteAccount",
post(admin::delete_account),
+8 -78
View File
@@ -77,27 +77,6 @@ pub async fn import_repo(
blocks.len(),
root
);
let Some(root_block) = blocks.get(&root) else {
return Err(ApiError::InvalidRequest(
"Root block not found in CAR file".into(),
));
};
let commit_did: Did = match jacquard_repo::commit::Commit::from_cbor(root_block) {
Ok(commit) => commit
.did()
.as_str()
.parse()
.map_err(|_| ApiError::InvalidRequest("Commit contains invalid DID".into()))?,
Err(e) => {
return Err(ApiError::InvalidRequest(format!("Invalid commit: {}", e)));
}
};
if commit_did != *did {
return Err(ApiError::InvalidRepo(format!(
"CAR file is for DID {} but you are authenticated as {}",
commit_did, did
)));
}
let skip_verification = std::env::var("SKIP_IMPORT_VERIFICATION")
.ok()
.map(|v| v == "true" || v == "1")
@@ -106,13 +85,15 @@ pub async fn import_repo(
.map(|c| c.import.skip_verification)
.unwrap_or(false)
});
let is_migration = user.deactivated_at.is_some();
let is_migration = user.inbound_migration && user.deactivated_at.is_some();
if skip_verification {
warn!("Skipping all CAR verification for import (SKIP_IMPORT_VERIFICATION=true)");
} else if is_migration {
debug!("Verifying CAR file structure for migration (skipping signature verification)");
warn!("Skipping all CAR verification for repo import (SKIP_IMPORT_VERIFICATION=true)");
} else {
debug!(
"Verifying CAR file structure for repo import (skipping signature and DID verification)"
);
let verifier = CarVerifier::new();
match verifier.verify_car_structure_only(did, &root, &blocks) {
match verifier.verify_car_structure_only(&root, &blocks) {
Ok(verified) => {
debug!(
"CAR structure verification successful: rev={}, data_cid={}",
@@ -142,56 +123,6 @@ pub async fn import_repo(
)));
}
}
} else {
debug!("Verifying CAR file signature and structure for DID {}", did);
let verifier = CarVerifier::new();
match verifier.verify_car(did, &root, &blocks).await {
Ok(verified) => {
debug!(
"CAR verification successful: rev={}, data_cid={}",
verified.rev, verified.data_cid
);
}
Err(tranquil_pds::sync::verify::VerifyError::DidMismatch {
commit_did,
expected_did,
}) => {
return Err(ApiError::InvalidRepo(format!(
"CAR file is for DID {} but you are authenticated as {}",
commit_did, expected_did
)));
}
Err(tranquil_pds::sync::verify::VerifyError::InvalidSignature) => {
return Err(ApiError::InvalidRequest(
"CAR file commit signature verification failed".into(),
));
}
Err(tranquil_pds::sync::verify::VerifyError::DidResolutionFailed(msg)) => {
warn!("DID resolution failed during import verification: {}", msg);
return Err(ApiError::InvalidRequest(format!(
"Failed to verify DID: {}",
msg
)));
}
Err(tranquil_pds::sync::verify::VerifyError::NoSigningKey) => {
return Err(ApiError::InvalidRequest(
"DID document does not contain a signing key".into(),
));
}
Err(tranquil_pds::sync::verify::VerifyError::MstValidationFailed(msg)) => {
return Err(ApiError::InvalidRequest(format!(
"MST validation failed: {}",
msg
)));
}
Err(e) => {
error!("CAR verification error: {:?}", e);
return Err(ApiError::InvalidRequest(format!(
"CAR verification failed: {}",
e
)));
}
}
}
let max_blocks = tranquil_config::get().import.max_blocks as usize;
let _write_lock = state.repo_write_locks.lock(user_id).await;
@@ -417,7 +348,6 @@ async fn sequence_import_event(
rev: None,
};
let seq = state.repos.repo.insert_commit_event(&data).await?;
state.repos.repo.notify_update(seq).await?;
state.repos.repo.insert_commit_event(&data).await?;
Ok(())
}
+61 -45
View File
@@ -1,6 +1,6 @@
use super::validation::validate_record_with_status;
use super::validation_mode::{ValidationMode, deserialize_validation_mode};
use crate::repo::record::write::CommitInfo;
use crate::repo::record::write::{CommitInfo, ensure_record_type};
use axum::{Json, extract::State};
use jacquard_repo::{mst::Mst, storage::BlockStore};
use serde::{Deserialize, Serialize};
@@ -14,8 +14,8 @@ use tranquil_pds::auth::{
};
use tranquil_pds::repo::TrackingBlockStore;
use tranquil_pds::repo_ops::{
FinalizeParams, RecordOp, begin_repo_write, extract_backlinks, extract_blob_cids,
finalize_repo_write,
CommitResult, FinalizeParams, RecordOp, begin_repo_write, extract_backlinks, extract_blob_cids,
finalize_repo_write, with_repair_retry,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AtIdentifier, AtUri, Did, Nsid, Rkey};
@@ -54,6 +54,8 @@ async fn process_single_write(
rkey,
value,
} => {
let value = ensure_record_type(value, collection);
let value = &*value;
let validation_status = if validate.should_skip() {
None
} else {
@@ -72,7 +74,7 @@ async fn process_single_write(
if mst
.get(&key)
.await
.map_err(|e| ApiError::InternalError(Some(format!("Failed to read MST: {e}"))))?
.map_err(|e| ApiError::from_mst_error("read MST for applyWrites create", &e))?
.is_some()
{
return Err(ApiError::InvalidRequest(format!(
@@ -90,7 +92,7 @@ async fn process_single_write(
let new_mst = mst
.add(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to add to MST".into())))?;
.map_err(|e| ApiError::from_mst_error("add record to MST", &e))?;
let uri = AtUri::from_parts(did, collection, &rkey);
backlinks_to_add.extend(extract_backlinks(&uri, value));
results.push(WriteResult::CreateResult {
@@ -117,6 +119,8 @@ async fn process_single_write(
rkey,
value,
} => {
let value = ensure_record_type(value, collection);
let value = &*value;
let validation_status = if validate.should_skip() {
None
} else {
@@ -134,9 +138,7 @@ async fn process_single_write(
let prev_record_cid = mst
.get(&key)
.await
.map_err(|e| {
ApiError::InternalError(Some(format!("Failed to read prev record: {}", e)))
})?
.map_err(|e| ApiError::from_mst_error("read update target from MST", &e))?
.ok_or_else(|| {
ApiError::InvalidRequest("Update target record does not exist".into())
})?;
@@ -151,7 +153,7 @@ async fn process_single_write(
let new_mst = mst
.update(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to update MST".into())))?;
.map_err(|e| ApiError::from_mst_error("update record in MST", &e))?;
let uri = AtUri::from_parts(did, collection, rkey);
backlinks_to_remove.push(uri.clone());
backlinks_to_add.extend(extract_backlinks(&uri, value));
@@ -180,16 +182,14 @@ async fn process_single_write(
let prev_record_cid = mst
.get(&key)
.await
.map_err(|e| {
ApiError::InternalError(Some(format!("Failed to read prev record: {}", e)))
})?
.map_err(|e| ApiError::from_mst_error("read delete target from MST", &e))?
.ok_or_else(|| {
ApiError::InvalidRequest("Delete target record does not exist".into())
})?;
let new_mst = mst
.delete(&key)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to delete from MST".into())))?;
.map_err(|e| ApiError::from_mst_error("delete record from MST", &e))?;
backlinks_to_remove.push(AtUri::from_parts(did, collection, rkey));
results.push(WriteResult::DeleteResult {});
ops.push(RecordOp::Delete {
@@ -232,6 +232,45 @@ async fn process_writes(
.await
}
async fn execute_apply_writes(
state: &AppState,
user_id: uuid::Uuid,
did: &Did,
input: &ApplyWritesInput,
controller_did: Option<&Did>,
write_summary: Option<serde_json::Value>,
) -> Result<(CommitResult, Vec<WriteResult>), ApiError> {
let (ctx, mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let WriteAccumulator {
mst: final_mst,
results,
ops,
all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
} = process_writes(&input.writes, mst, did, input.validate, &ctx.tracking_store).await?;
let commit_result = finalize_repo_write(
state,
ctx,
final_mst,
FinalizeParams {
did,
user_id,
controller_did,
delegation_detail: write_summary,
ops,
blob_cids: &all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
},
)
.await?;
Ok((commit_result, results))
}
#[derive(Deserialize)]
#[serde(tag = "$type")]
pub enum WriteOp {
@@ -346,24 +385,6 @@ pub async fn apply_writes(
.log_db_err("fetching user for batch write")?
.ok_or(ApiError::InternalError(Some("User not found".into())))?;
let (ctx, mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let WriteAccumulator {
mst: final_mst,
results,
ops,
all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
} = process_writes(
&input.writes,
mst,
&did,
input.validate,
&ctx.tracking_store,
)
.await?;
let write_summary: Option<serde_json::Value> = controller_did.as_ref().map(|_| {
let writes: Vec<serde_json::Value> = input
.writes
@@ -397,21 +418,16 @@ pub async fn apply_writes(
})
});
let commit_result = finalize_repo_write(
&state,
ctx,
final_mst,
FinalizeParams {
did: &did,
let (commit_result, results) = with_repair_retry(&state, user_id, || {
execute_apply_writes(
&state,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: write_summary,
ops,
blob_cids: &all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
},
)
&did,
&input,
controller_did.as_ref(),
write_summary.clone(),
)
})
.await?;
Ok(Json(ApplyWritesOutput {
+40 -21
View File
@@ -4,13 +4,15 @@ use cid::Cid;
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::str::FromStr;
use tracing::error;
use tranquil_pds::api::error::ApiError;
use tranquil_pds::auth::{Active, Auth, VerifyScope};
use tranquil_pds::cid_types::RecordCid;
use tranquil_pds::repo_ops::{FinalizeParams, RecordOp, begin_repo_write, finalize_repo_write};
use tranquil_pds::repo_ops::{
FinalizeParams, RecordOp, begin_repo_write, finalize_repo_write, with_repair_retry,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AtIdentifier, AtUri, Nsid, Rkey};
use tranquil_pds::types::{AtIdentifier, AtUri, Did, Nsid, Rkey};
use uuid::Uuid;
#[derive(Deserialize)]
pub struct DeleteRecordInput {
@@ -41,13 +43,30 @@ pub async fn delete_record(
let user_id = repo_auth.user_id;
let controller_did = repo_auth.controller_did;
let (ctx, mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let out = with_repair_retry(&state, user_id, || {
delete_record_inner(&state, &did, user_id, controller_did.as_ref(), &input)
})
.await?;
Ok(Json(out))
}
async fn delete_record_inner(
state: &AppState,
did: &Did,
user_id: Uuid,
controller_did: Option<&Did>,
input: &DeleteRecordInput,
) -> Result<DeleteRecordOutput, ApiError> {
let (ctx, mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let key = format!("{}/{}", input.collection, input.rkey);
if let Some(swap_record_str) = &input.swap_record {
let expected_cid = Cid::from_str(swap_record_str).ok();
let actual_cid = mst.get(&key).await.ok().flatten();
let actual_cid = mst
.get(&key)
.await
.map_err(|e| ApiError::from_mst_error("read swap target from MST", &e))?;
if expected_cid != actual_cid {
return Err(ApiError::InvalidSwap(Some(
"Record has been modified or does not exist".into(),
@@ -55,18 +74,18 @@ pub async fn delete_record(
}
}
let prev_record_cid = mst.get(&key).await.map_err(|e| {
error!("Failed to read prev record from MST: {}", e);
ApiError::InternalError(Some("Failed to read MST".into()))
})?;
let prev_record_cid = mst
.get(&key)
.await
.map_err(|e| ApiError::from_mst_error("read prev record from MST", &e))?;
let Some(prev_record_cid) = prev_record_cid else {
return Ok(Json(DeleteRecordOutput { commit: None }));
return Ok(DeleteRecordOutput { commit: None });
};
let new_mst = mst.delete(&key).await.map_err(|e| {
error!("Failed to delete from MST: {}", e);
ApiError::InternalError(Some("Failed to delete from MST".into()))
})?;
let new_mst = mst
.delete(&key)
.await
.map_err(|e| ApiError::from_mst_error("delete record from MST", &e))?;
let op = RecordOp::Delete {
collection: input.collection.clone(),
@@ -74,17 +93,17 @@ pub async fn delete_record(
prev: RecordCid::from(prev_record_cid),
};
let deleted_uri = AtUri::from_parts(&did, &input.collection, &input.rkey);
let deleted_uri = AtUri::from_parts(did, &input.collection, &input.rkey);
let commit_result = finalize_repo_write(
&state,
state,
ctx,
new_mst,
FinalizeParams {
did: &did,
did,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: controller_did.as_ref().map(|_| {
controller_did,
delegation_detail: controller_did.map(|_| {
json!({
"action": "delete",
"collection": input.collection,
@@ -99,10 +118,10 @@ pub async fn delete_record(
)
.await?;
Ok(Json(DeleteRecordOutput {
Ok(DeleteRecordOutput {
commit: Some(CommitInfo {
cid: commit_result.commit_cid.to_string(),
rev: commit_result.rev,
}),
}))
})
}
+100 -45
View File
@@ -5,8 +5,8 @@ use cid::Cid;
use jacquard_repo::storage::BlockStore;
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::borrow::Cow;
use std::str::FromStr;
use tracing::error;
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{
Active, Auth, AuthSource, RepoScopeAction, ScopeVerified, VerifyScope, require_not_migrated,
@@ -14,7 +14,7 @@ use tranquil_pds::auth::{
};
use tranquil_pds::repo_ops::{
FinalizeParams, RecordOp, begin_repo_write, extract_backlinks, extract_blob_cids,
finalize_repo_write,
finalize_repo_write, with_repair_retry,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AtIdentifier, AtUri, Did, Nsid, Rkey};
@@ -62,6 +62,29 @@ pub async fn prepare_repo_write<A: RepoScopeAction>(
})
}
pub(crate) fn ensure_record_type<'a>(
record: &'a serde_json::Value,
collection: &Nsid,
) -> Cow<'a, serde_json::Value> {
let serde_json::Value::Object(map) = record else {
return Cow::Borrowed(record);
};
let needs_fill = match map.get("$type") {
None | Some(serde_json::Value::Null) => true,
Some(serde_json::Value::String(existing)) => existing.is_empty(),
Some(_) => false,
};
if !needs_fill {
return Cow::Borrowed(record);
}
let mut map = map.clone();
map.insert(
"$type".to_string(),
serde_json::Value::String(collection.to_string()),
);
Cow::Owned(serde_json::Value::Object(map))
}
#[derive(Deserialize)]
#[allow(dead_code)]
pub struct CreateRecordInput {
@@ -95,15 +118,32 @@ pub struct CreateRecordOutput {
pub async fn create_record(
State(state): State<AppState>,
auth: Auth<Active>,
Json(input): Json<CreateRecordInput>,
Json(mut input): Json<CreateRecordInput>,
) -> Result<Json<CreateRecordOutput>, ApiError> {
if let Cow::Owned(record) = ensure_record_type(&input.record, &input.collection) {
input.record = record;
}
let scope_proof = auth.verify_repo_create(&input.collection)?;
let repo_auth = prepare_repo_write(&state, &scope_proof, &input.repo).await?;
let did = repo_auth.did;
let user_id = repo_auth.user_id;
let controller_did = repo_auth.controller_did;
let (ctx, mut mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let out = with_repair_retry(&state, user_id, || {
create_record_inner(&state, &did, user_id, controller_did.as_ref(), &input)
})
.await?;
Ok(Json(out))
}
async fn create_record_inner(
state: &AppState,
did: &Did,
user_id: Uuid,
controller_did: Option<&Did>,
input: &CreateRecordInput,
) -> Result<CreateRecordOutput, ApiError> {
let (ctx, mut mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let validation_status = if input.validate.should_skip() {
None
@@ -119,12 +159,12 @@ pub async fn create_record(
)
};
let rkey = input.rkey.unwrap_or_else(Rkey::generate);
let rkey = input.rkey.clone().unwrap_or_else(Rkey::generate);
let mut ops: Vec<RecordOp> = Vec::new();
let mut conflict_uris_to_cleanup: Vec<AtUri> = Vec::new();
if !input.validate.should_skip() {
let record_uri = AtUri::from_parts(&did, &input.collection, &rkey);
let record_uri = AtUri::from_parts(did, &input.collection, &rkey);
let backlinks = extract_backlinks(&record_uri, &input.record);
if !backlinks.is_empty() {
@@ -149,24 +189,18 @@ pub async fn create_record(
Ok(Some(cid)) => cid,
Ok(None) => continue,
Err(e) => {
error!(
"Failed to read conflict record from MST {}: {:?}",
conflict_uri, e
);
return Err(ApiError::InternalError(Some(
"Failed to read conflicting record from MST".into(),
)));
return Err(ApiError::from_mst_error(
&format!("read conflict record from MST {conflict_uri}"),
&e,
));
}
};
mst = mst.delete(&conflict_key).await.map_err(|e| {
error!(
"Failed to delete conflict from MST {}: {:?}",
conflict_uri, e
);
ApiError::InternalError(Some(
"Failed to delete conflicting record from MST".into(),
))
ApiError::from_mst_error(
&format!("delete conflict from MST {conflict_uri}"),
&e,
)
})?;
ops.push(RecordOp::Delete {
@@ -183,7 +217,7 @@ pub async fn create_record(
if mst
.get(&key)
.await
.map_err(|e| ApiError::InternalError(Some(format!("Failed to read MST: {e}"))))?
.map_err(|e| ApiError::from_mst_error("read MST for create existence check", &e))?
.is_some()
{
return Err(ApiError::InvalidRequest(format!(
@@ -202,7 +236,7 @@ pub async fn create_record(
mst = mst
.add(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to add to MST".into())))?;
.map_err(|e| ApiError::from_mst_error("add record to MST", &e))?;
ops.push(RecordOp::Create {
collection: input.collection.clone(),
@@ -212,18 +246,18 @@ pub async fn create_record(
let blob_cids = extract_blob_cids(&input.record);
let created_uri = AtUri::from_parts(&did, &input.collection, &rkey);
let created_uri = AtUri::from_parts(did, &input.collection, &rkey);
let backlinks_to_add = extract_backlinks(&created_uri, &input.record);
let commit_result = finalize_repo_write(
&state,
state,
ctx,
mst,
FinalizeParams {
did: &did,
did,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: controller_did.as_ref().map(|_| {
controller_did,
delegation_detail: controller_did.map(|_| {
json!({
"action": "create",
"collection": input.collection,
@@ -238,7 +272,7 @@ pub async fn create_record(
)
.await?;
Ok(Json(CreateRecordOutput {
Ok(CreateRecordOutput {
uri: created_uri,
cid: record_cid.to_string(),
commit: CommitInfo {
@@ -246,7 +280,7 @@ pub async fn create_record(
rev: commit_result.rev,
},
validation_status,
}))
})
}
#[derive(Deserialize)]
@@ -278,15 +312,32 @@ pub struct PutRecordOutput {
pub async fn put_record(
State(state): State<AppState>,
auth: Auth<Active>,
Json(input): Json<PutRecordInput>,
Json(mut input): Json<PutRecordInput>,
) -> Result<Json<PutRecordOutput>, ApiError> {
if let Cow::Owned(record) = ensure_record_type(&input.record, &input.collection) {
input.record = record;
}
let upsert_proof = auth.verify_repo_upsert(&input.collection)?;
let repo_auth = prepare_repo_write(&state, &upsert_proof, &input.repo).await?;
let did = repo_auth.did;
let user_id = repo_auth.user_id;
let controller_did = repo_auth.controller_did;
let (ctx, mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let out = with_repair_retry(&state, user_id, || {
put_record_inner(&state, &did, user_id, controller_did.as_ref(), &input)
})
.await?;
Ok(Json(out))
}
async fn put_record_inner(
state: &AppState,
did: &Did,
user_id: Uuid,
controller_did: Option<&Did>,
input: &PutRecordInput,
) -> Result<PutRecordOutput, ApiError> {
let (ctx, mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let validation_status = if input.validate.should_skip() {
None
@@ -304,9 +355,13 @@ pub async fn put_record(
let key = format!("{}/{}", input.collection, input.rkey);
let read_cid = |r: Result<Option<Cid>, jacquard_repo::error::RepoError>| {
r.map_err(|e| ApiError::from_mst_error("read MST for put", &e))
};
if let Some(swap_record_str) = &input.swap_record {
let expected_cid = Cid::from_str(swap_record_str).ok();
let actual_cid = mst.get(&key).await.ok().flatten();
let actual_cid = read_cid(mst.get(&key).await)?;
if expected_cid != actual_cid {
return Err(ApiError::InvalidSwap(Some(
"Record has been modified or does not exist".into(),
@@ -314,7 +369,7 @@ pub async fn put_record(
}
}
let existing_cid = mst.get(&key).await.ok().flatten();
let existing_cid = read_cid(mst.get(&key).await)?;
let record_ipld = tranquil_pds::util::json_to_ipld(&input.record);
let record_bytes = serde_ipld_dagcbor::to_vec(&record_ipld)
.map_err(|_| ApiError::InvalidRecord("Failed to serialize record".into()))?;
@@ -325,21 +380,21 @@ pub async fn put_record(
.map_err(|_| ApiError::InternalError(Some("Failed to save record block".into())))?;
if existing_cid == Some(record_cid) {
return Ok(Json(PutRecordOutput {
uri: AtUri::from_parts(&did, &input.collection, &input.rkey),
return Ok(PutRecordOutput {
uri: AtUri::from_parts(did, &input.collection, &input.rkey),
cid: record_cid.to_string(),
commit: None,
validation_status,
}));
});
}
let record_uri = AtUri::from_parts(&did, &input.collection, &input.rkey);
let record_uri = AtUri::from_parts(did, &input.collection, &input.rkey);
let (new_mst, op, is_update, backlinks_to_remove) = match existing_cid {
Some(prev_cid) => {
let new_mst = mst
.update(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to update MST".into())))?;
.map_err(|e| ApiError::from_mst_error("update record in MST", &e))?;
let op = RecordOp::Update {
collection: input.collection.clone(),
rkey: input.rkey.clone(),
@@ -352,7 +407,7 @@ pub async fn put_record(
let new_mst = mst
.add(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to add to MST".into())))?;
.map_err(|e| ApiError::from_mst_error("add record to MST", &e))?;
let op = RecordOp::Create {
collection: input.collection.clone(),
rkey: input.rkey.clone(),
@@ -366,14 +421,14 @@ pub async fn put_record(
let backlinks_to_add = extract_backlinks(&record_uri, &input.record);
let commit_result = finalize_repo_write(
&state,
state,
ctx,
new_mst,
FinalizeParams {
did: &did,
did,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: controller_did.as_ref().map(|_| {
controller_did,
delegation_detail: controller_did.map(|_| {
json!({
"action": if is_update { "update" } else { "create" },
"collection": input.collection,
@@ -388,7 +443,7 @@ pub async fn put_record(
)
.await?;
Ok(Json(PutRecordOutput {
Ok(PutRecordOutput {
uri: record_uri,
cid: record_cid.to_string(),
commit: Some(CommitInfo {
@@ -396,5 +451,5 @@ pub async fn put_record(
rev: commit_result.rev,
}),
validation_status,
}))
})
}
@@ -665,27 +665,22 @@ pub async fn delete_account(
error!("DB error deleting account: {:?}", e);
ApiError::InternalError(None)
})?;
let account_seq = tranquil_pds::repo_ops::sequence_account_event(
if let Err(e) = tranquil_pds::repo_ops::sequence_account_event(
&state,
did,
tranquil_db_traits::AccountStatus::Deleted,
)
.await;
match account_seq {
Ok(seq) => {
if let Err(e) = state.repos.repo.delete_sequences_except(did, seq).await {
warn!(
"Failed to cleanup sequences for deleted account {}: {}",
did, e
);
}
}
Err(e) => {
warn!(
"Failed to sequence account deletion event for {}: {}",
did, e
);
}
.await
{
warn!(
"Failed to sequence account deletion event for {}: {}",
did, e
);
} else if let Err(e) = state.repos.repo.purge_did_events_keeping_latest(did).await {
warn!(
"Failed to cleanup sequences for deleted account {}: {}",
did, e
);
}
let _ = state
.cache
+6 -4
View File
@@ -5,7 +5,7 @@ use tranquil_pds::BUILD_VERSION;
use tranquil_pds::state::AppState;
use tranquil_pds::util::{discord_app_id, discord_bot_username, telegram_bot_username};
fn get_available_comms_channels() -> Vec<CommsChannel> {
async fn get_available_comms_channels(state: &AppState) -> Vec<CommsChannel> {
let cfg = tranquil_config::get();
let mut channels = vec![CommsChannel::Email];
if cfg.discord.bot_token.is_some() {
@@ -14,7 +14,9 @@ fn get_available_comms_channels() -> Vec<CommsChannel> {
if cfg.telegram.bot_token.is_some() {
channels.push(CommsChannel::Telegram);
}
if cfg.signal.enabled {
if let Some(slot) = &state.signal_sender
&& slot.is_linked().await
{
channels.push(CommsChannel::Signal);
}
channels
@@ -66,7 +68,7 @@ pub struct DescribeServerOutput {
pub telegram_bot_username: Option<String>,
}
pub async fn describe_server() -> Json<DescribeServerOutput> {
pub async fn describe_server(State(state): State<AppState>) -> Json<DescribeServerOutput> {
let cfg = tranquil_config::get();
let pds_hostname = &cfg.server.hostname;
@@ -82,7 +84,7 @@ pub async fn describe_server() -> Json<DescribeServerOutput> {
email: cfg.server.contact_email.clone(),
},
version: BUILD_VERSION,
available_comms_channels: get_available_comms_channels(),
available_comms_channels: get_available_comms_channels(&state).await,
self_hosted_did_web_enabled: is_self_hosted_did_web_enabled(),
discord_bot_username: discord_bot_username().map(String::from),
discord_app_id: discord_app_id().map(String::from),
+1
View File
@@ -51,6 +51,7 @@ pub use session::{
auto_resend_verification, confirm_signup, create_session, delete_session,
get_legacy_login_preference, get_session, list_sessions, refresh_session, resend_verification,
revoke_all_sessions, revoke_session, update_legacy_login_preference, update_locale,
verification_blocks_login,
};
pub use signing_key::reserve_signing_key;
pub use totp::{
+11 -8
View File
@@ -13,10 +13,6 @@ use tranquil_pds::state::AppState;
use tranquil_pds::types::PlainPassword;
use tranquil_pds::validation::validate_password;
fn generate_reset_code() -> String {
tranquil_pds::util::generate_token_code()
}
#[derive(Deserialize)]
pub struct RequestPasswordResetInput {
#[serde(alias = "identifier")]
@@ -70,12 +66,13 @@ pub async fn request_password_reset(
return Err(ApiError::InternalError(None));
}
};
let code = generate_reset_code();
let display_code = tranquil_pds::util::generate_token_code();
let stored_code = tranquil_pds::util::normalize_token_code(&display_code);
let expires_at = Utc::now() + Duration::minutes(10);
if let Err(e) = state
.repos
.user
.set_password_reset_code(user_id, &code, expires_at)
.set_password_reset_code(user_id, &stored_code, expires_at)
.await
{
error!("DB error setting reset code: {:?}", e);
@@ -86,7 +83,7 @@ pub async fn request_password_reset(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
user_id,
&code,
&display_code,
hostname,
)
.await
@@ -133,7 +130,13 @@ pub async fn reset_password(
if let Err(e) = validate_password(password) {
return Err(ApiError::InvalidRequest(e.to_string()));
}
let user = match state.repos.user.get_user_by_reset_code(token).await {
let normalized_token = tranquil_pds::util::normalize_token_code(token);
let user = match state
.repos
.user
.get_user_by_reset_code(&normalized_token)
.await
{
Ok(Some(u)) => u,
Ok(None) => {
return Err(ApiError::InvalidToken(None));
+32 -5
View File
@@ -8,18 +8,28 @@ use bcrypt::verify;
use serde::{Deserialize, Serialize};
use serde_json::json;
use tracing::{error, info, warn};
use tranquil_db_traits::{SessionId, TokenFamilyId};
use tranquil_db_traits::{ChannelVerificationStatus, SessionId, TokenFamilyId};
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::api::{EmptyResponse, PreferredLocaleOutput, SuccessResponse};
use tranquil_pds::auth::{
Active, Auth, NormalizedLoginIdentifier, Permissive, require_legacy_session_mfa,
require_reauth_window,
};
use tranquil_pds::rate_limit::{LoginLimit, RateLimited, RefreshSessionLimit};
use tranquil_pds::rate_limit::{
LoginLimit, RateLimited, RefreshSessionLimit, TotpVerifyLimit,
check_user_rate_limit_with_message,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AccountState, Did, Handle, PlainPassword};
use tranquil_types::TokenId;
pub fn verification_blocks_login(channel_verification: &ChannelVerificationStatus) -> bool {
!tranquil_config::get()
.server
.disable_account_verification_gate
&& !channel_verification.has_any_verified()
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct CreateSessionInput {
@@ -129,14 +139,13 @@ pub async fn create_session(
warn!("Login attempt for takendown account: {}", row.did);
return Err(ApiError::AccountTakedown);
}
let is_verified = row.channel_verification.has_any_verified();
let is_delegated = state
.repos
.delegation
.is_delegated_account(&row.did)
.await
.unwrap_or(false);
if !is_verified && !is_delegated {
if verification_blocks_login(&row.channel_verification) && !is_delegated {
warn!("Login attempt for unverified account: {}", row.did);
let resend_info = auto_resend_verification(&state, &row.did).await;
let handle = resend_info
@@ -162,17 +171,35 @@ pub async fn create_session(
let has_totp = row.totp_enabled;
let email_2fa_enabled = row.email_2fa_enabled;
let is_legacy_login = has_totp || email_2fa_enabled;
let used_totp_factor = tranquil_pds::auth::legacy_2fa::used_totp_factor(
has_totp,
input.auth_factor_token.as_deref(),
);
let twofa_ctx = tranquil_pds::auth::legacy_2fa::Legacy2faContext {
is_app_password: app_password_name.is_some(),
email_2fa_enabled,
has_totp,
allow_legacy_login: row.allow_legacy_login,
};
if let Some(token) = input.auth_factor_token.as_deref()
&& has_totp
&& tranquil_pds::auth::legacy_2fa::looks_like_totp_token(token)
{
check_user_rate_limit_with_message::<TotpVerifyLimit>(
&state,
row.did.as_str(),
"Too many verification attempts. Please try again in a few minutes.",
)
.await?;
}
match tranquil_pds::auth::legacy_2fa::process_legacy_2fa(
state.cache.as_ref(),
&row.did,
&twofa_ctx,
input.auth_factor_token.as_deref(),
async |t: &str| {
crate::server::totp::verify_totp_or_backup_for_user(&state, &row.did, t).await
},
)
.await
{
@@ -279,7 +306,7 @@ pub async fn create_session(
error!("Failed to insert session: {:?}", e);
return Err(ApiError::InternalError(None));
}
if is_legacy_login {
if is_legacy_login && !used_totp_factor {
warn!(
did = %row.did,
ip = %client_ip,
+5 -5
View File
@@ -35,7 +35,7 @@ pub fn create_access_token_with_scope_metadata(
scope,
TokenType::Access,
key_bytes,
Duration::minutes(15),
Duration::minutes(120),
hostname,
)
}
@@ -54,7 +54,7 @@ pub fn create_access_token_with_delegation(
scope,
TokenType::Access,
key_bytes,
Duration::minutes(15),
Duration::minutes(120),
act,
hostname,
)
@@ -69,7 +69,7 @@ pub fn create_refresh_token_with_metadata(
TokenScope::Refresh.as_str(),
TokenType::Refresh,
key_bytes,
Duration::days(14),
Duration::days(90),
None,
)
}
@@ -198,7 +198,7 @@ pub fn create_access_token_hs256_with_metadata(
TokenScope::Access.as_str(),
TokenType::Access,
secret,
Duration::minutes(15),
Duration::minutes(120),
)
}
@@ -211,7 +211,7 @@ pub fn create_refresh_token_hs256_with_metadata(
TokenScope::Refresh.as_str(),
TokenType::Refresh,
secret,
Duration::days(14),
Duration::days(90),
)
}
+14 -1
View File
@@ -15,7 +15,9 @@ impl TokenType {
match self {
Self::Access => "at+jwt",
Self::Refresh => "refresh+jwt",
Self::Service => "jwt",
// RFC 7519 §5.1 recommends the uppercase "JWT".
// and for atproto inter-service auth its a requirement.
Self::Service => "JWT",
}
}
}
@@ -290,6 +292,17 @@ mod tests {
assert!(TokenType::from_str("bearer").is_err());
}
#[test]
fn service_token_header_serializes_typ_as_uppercase_jwt() {
// RFC 7519 §5.1 recommends the JWT `typ` header value be uppercase "JWT".
let header = Header {
alg: SigningAlgorithm::ES256K,
typ: TokenType::Service,
};
let json = serde_json::to_string(&header).expect("serialize header");
assert!(json.contains(r#""typ":"JWT""#), "got {json}");
}
#[test]
fn signing_algorithm_case_insensitive() {
assert_eq!(
+37 -1
View File
@@ -1,5 +1,5 @@
pub const DEFAULT_LOCALE: &str = "en";
pub const VALID_LOCALES: &[&str] = &["en", "zh", "ja", "ko", "sv", "fi"];
pub const VALID_LOCALES: &[&str] = &["en", "zh", "ja", "ko", "sv", "fi", "fr"];
pub fn validate_locale(locale: &str) -> &str {
if VALID_LOCALES.contains(&locale) {
@@ -44,6 +44,7 @@ pub fn get_strings(locale: &str) -> &'static NotificationStrings {
"ko" => &STRINGS_KO,
"sv" => &STRINGS_SV,
"fi" => &STRINGS_FI,
"fr" => &STRINGS_FR,
_ => &STRINGS_EN,
}
}
@@ -216,6 +217,34 @@ static STRINGS_FI: NotificationStrings = NotificationStrings {
channel_verification_body: "Vahvistuskoodisi on:\n{code}\n\nTai vahvista suoraan:\n{verify_link}",
};
static STRINGS_FR: NotificationStrings = NotificationStrings {
welcome_subject: "Bienvenue sur {hostname}",
welcome_body: "Bienvenue sur {hostname} !\n\nVotre identifiant est : @{handle}\n\nMerci de nous avoir rejoint.",
password_reset_subject: "Réinitialisation du mot de passe - {hostname}",
password_reset_body: "Bonjour @{handle},\n\nVotre code de réinitialisation du mot de passe est : {code}\n\nCe code expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, veuillez ignorer ce message.",
email_update_subject: "Confirmer votre nouvelle adresse e-mail - {hostname}",
email_update_body: "Bonjour @{handle},\n\nVotre code de vérification est :\n{code}\n\nCopiez le code ci-dessus et saisissez-le ici :\n{verify_page}\n\nCe code expirera dans 10 minutes.\n\nOu si vous aimez vivre dangereusement :\n{verify_link}\n\nSi vous n'avez pas demandé cela, veuillez ignorer cet e-mail.",
short_token_body: "Bonjour @{handle},\n\nVotre code de vérification est :\n{code}\n\nCe code expirera dans 15 minutes.\n\nSi vous n'avez pas demandé cela, veuillez ignorer cet e-mail.",
account_deletion_subject: "Demande de suppression de compte - {hostname}",
account_deletion_body: "Bonjour @{handle},\n\nVotre code de confirmation de suppression de compte est : {code}\n\nCe code expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, sécurisez votre compte immédiatement.",
plc_operation_subject: "{hostname} - Jeton d'opération PLC",
plc_operation_body: "Bonjour @{handle},\n\nVous avez demandé à signer une opération PLC pour votre compte.\n\nVotre jeton de vérification est : {token}\n\nCe jeton expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, vous pouvez ignorer ce message en toute sécurité.",
two_factor_code_subject: "Vérification de connexion - {hostname}",
two_factor_code_body: "Bonjour @{handle},\n\nVotre code de vérification de connexion est : {code}\n\nCe code expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, sécurisez votre compte immédiatement.",
passkey_recovery_subject: "Récupération de compte - {hostname}",
passkey_recovery_body: "Bonjour @{handle},\n\nVous avez demandé la récupération de votre compte à clé d'accès uniquement.\n\nCliquez sur le lien ci-dessous pour définir un mot de passe temporaire et retrouver l'accès :\n{url}\n\nCe lien expirera dans 1 heure.\n\nSi vous n'avez pas demandé cela, veuillez ignorer ce message. Votre compte reste sécurisé.",
signup_verification_subject: "Vérifier votre compte - {hostname}",
signup_verification_body: "Bienvenue ! Votre code de vérification est :\n{code}\n\nCopiez le code ci-dessus et saisissez-le ici :\n{verify_page}\n\nCe code expirera dans 30 minutes.\n\nOu si vous aimez vivre dangereusement :\n{verify_link}\n\nSi vous n'avez pas créé de compte sur {hostname}, veuillez ignorer ce message.",
legacy_login_subject: "Alerte de sécurité : Connexion classique détectée - {hostname}",
legacy_login_body: "Bonjour @{handle},\n\nUne connexion à votre compte a été détectée via une application classique (comme Bluesky) qui ne prend pas en charge la vérification TOTP.\n\nDétails :\n- Date : {timestamp}\n- Adresse IP : {ip}\n\nVotre protection TOTP a été contournée pour cette connexion. La session dispose de permissions limitées pour les opérations sensibles.\n\nSi ce n'était pas vous :\n1. Changez votre mot de passe immédiatement\n2. Vérifiez vos sessions actives\n3. Envisagez de désactiver les connexions d'applications classiques dans vos paramètres de sécurité\n\nRestez vigilant,\n{hostname}",
migration_verification_subject: "Vérifier votre adresse e-mail - {hostname}",
migration_verification_body: "Bienvenue sur {hostname} !\n\nVotre compte a été migré avec succès. Pour finaliser la configuration, veuillez vérifier votre adresse e-mail.\n\nVotre code de vérification est :\n{code}\n\nCopiez le code ci-dessus et saisissez-le ici :\n{verify_page}\n\nCe code expirera dans 48 heures.\n\nOu si vous aimez vivre dangereusement :\n{verify_link}\n\nSi vous n'avez pas migré votre compte, veuillez ignorer cet e-mail.",
channel_verified_subject: "Canal de notification vérifié - {hostname}",
channel_verified_body: "Bonjour {handle},\n\n{channel} a été vérifié comme canal de notification pour votre compte sur {hostname}.",
channel_verification_subject: "Vérifier votre canal - {hostname}",
channel_verification_body: "Votre code de vérification est :\n{code}\n\nOu vérifiez directement :\n{verify_link}",
};
pub fn format_message(template: &str, vars: &[(&str, &str)]) -> String {
vars.iter()
.fold(template.to_string(), |result, (key, value)| {
@@ -233,6 +262,9 @@ mod tests {
assert_eq!(validate_locale("zh"), "zh");
assert_eq!(validate_locale("ja"), "ja");
assert_eq!(validate_locale("ko"), "ko");
assert_eq!(validate_locale("sv"), "sv");
assert_eq!(validate_locale("fi"), "fi");
assert_eq!(validate_locale("fr"), "fr");
assert_eq!(validate_locale("invalid"), DEFAULT_LOCALE);
assert_eq!(validate_locale(""), DEFAULT_LOCALE);
}
@@ -252,5 +284,9 @@ mod tests {
let zh = get_strings("zh");
assert!(zh.welcome_subject.contains("{hostname}"));
assert!(zh.welcome_body.contains("欢迎"));
let fr = get_strings("fr");
assert!(fr.welcome_subject.contains("{hostname}"));
assert!(fr.welcome_body.contains("Bienvenue"));
}
}
+239 -11
View File
@@ -104,6 +104,7 @@ pub fn load(config_path: Option<&PathBuf>) -> Result<TranquilConfig, confique::E
// Root configuration
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TranquilConfig {
#[config(nested)]
pub server: ServerConfig,
@@ -141,9 +142,6 @@ pub struct TranquilConfig {
#[config(nested)]
pub telegram: TelegramConfig,
#[config(nested)]
pub signal: SignalConfig,
#[config(nested)]
pub notifications: NotificationConfig,
@@ -258,6 +256,9 @@ impl TranquilConfig {
}
}
// -- tls --------------------------------------------------------------
self.server.tls.validate(&mut errors);
// -- SSO providers ----------------------------------------------------
self.validate_sso_provider("sso.github", &self.sso.github, &mut errors);
self.validate_sso_provider("sso.google", &self.sso.google, &mut errors);
@@ -417,6 +418,7 @@ impl TranquilConfig {
// ---------------------------------------------------------------------------
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ServerConfig {
/// Public hostname of the PDS, such as `pds.example.com`.
#[config(env = "PDS_HOSTNAME")]
@@ -456,6 +458,11 @@ pub struct ServerConfig {
#[config(env = "DISABLE_RATE_LIMITING", default = false)]
pub disable_rate_limiting: bool,
/// Skip the verified-comms-channel gate for login and record writes.
/// Please keep this off unless you're an invite-only PDS!
#[config(env = "DISABLE_ACCOUNT_VERIFICATION_GATE", default = false)]
pub disable_account_verification_gate: bool,
/// List of additional banned words for handle validation.
#[config(env = "PDS_BANNED_WORDS", parse_env = split_comma_list)]
pub banned_words: Option<Vec<String>>,
@@ -479,6 +486,65 @@ pub struct ServerConfig {
/// Maximum allowed number of preferences
#[config(env = "MAX_PREFERENCES_COUNT", default = 1000)]
pub max_preferences_count: usize,
/// If you're not altering TLS config, you don't have to worry about this.
/// This is the number of trusted reverse proxies in front of Tranquil.
/// We read the client IP used for rate limiting and device records this many hops
/// from the right of the X-Forwarded-For header.
/// When left unset, Tranquil will assume:
/// - 0, if the TLS termination is happening here on Tranquil via the TLS config
/// - 1, if the TLS termination *isn't* happening here.
#[config(env = "TRUSTED_PROXY_COUNT")]
pub trusted_proxy_count: Option<usize>,
#[config(nested)]
pub tls: TlsConfig,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TlsConfig {
/// The path to the TLS cert chain.
/// If you set both this and `key_path`, the server terminates TLS itself rather than expecting
/// a reverse proxy to do it. The certificate and key reload on SIGHUP.
#[config(env = "TLS_CERT_PATH")]
pub cert_path: Option<String>,
/// Path to the TLS private key.
#[config(env = "TLS_KEY_PATH")]
pub key_path: Option<String>,
/// Serve HTTP/3 over QUIC on the same UDP port as the TCP listener.
/// Requires cert_path and key_path.
#[config(env = "TLS_HTTP3", default = false)]
pub http3: bool,
}
impl TlsConfig {
/// The certificate and key paths when both are configured.
pub fn material(&self) -> Option<(&str, &str)> {
match (self.cert_path.as_deref(), self.key_path.as_deref()) {
(Some(cert), Some(key)) => Some((cert, key)),
_ => None,
}
}
pub fn validate(&self, errors: &mut Vec<String>) {
if self.cert_path.is_some() != self.key_path.is_some() {
errors.push(
"server.tls.cert_path (TLS_CERT_PATH) and server.tls.key_path (TLS_KEY_PATH) \
must both be set to enable app-level TLS, or both be unset"
.to_string(),
);
}
if self.http3 && self.material().is_none() {
errors.push(
"server.tls.http3 (TLS_HTTP3) requires server.tls.cert_path \
and erver.tls.key_path"
.to_string(),
);
}
}
}
impl ServerConfig {
@@ -513,6 +579,7 @@ impl ServerConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct FrontendConfig {
/// Whether to enable the built in serving of the frontend.
#[config(env = "FRONTEND_ENABLED", default = true)]
@@ -525,6 +592,7 @@ pub struct FrontendConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DatabaseConfig {
/// PostgreSQL connection URL.
#[config(env = "DATABASE_URL")]
@@ -544,6 +612,7 @@ pub struct DatabaseConfig {
}
#[derive(Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SecretsConfig {
/// Secret used for signing JWTs. Must be at least 32 characters in
/// production.
@@ -670,6 +739,7 @@ impl fmt::Display for RepoBackend {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct StorageConfig {
/// Storage backend: `filesystem` or `s3`.
#[config(env = "BLOB_STORAGE_BACKEND", default = "filesystem")]
@@ -702,6 +772,7 @@ impl StorageConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct CacheConfig {
/// Cache backend: `ripple` by default, or `valkey`.
#[config(env = "CACHE_BACKEND", default = "ripple")]
@@ -716,6 +787,7 @@ pub struct CacheConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct PlcConfig {
/// Base URL of the PLC directory.
#[config(env = "PLC_DIRECTORY_URL", default = "https://plc.directory")]
@@ -735,6 +807,7 @@ pub struct PlcConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct FirehoseConfig {
/// Size of the in-memory broadcast buffer for firehose events.
#[config(env = "FIREHOSE_BUFFER_SIZE", default = 10000)]
@@ -765,6 +838,7 @@ impl FirehoseConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct EmailConfig {
/// Sender email address. When unset, email sending is disabled.
#[config(env = "MAIL_FROM_ADDRESS")]
@@ -970,6 +1044,7 @@ fn is_valid_dkim_selector(s: &str) -> bool {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SmarthostConfig {
/// SMTP relay host. When set, mail is delivered through this host
/// instead of resolving recipient MX records directly.
@@ -1009,6 +1084,7 @@ pub struct SmarthostConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DirectMxConfig {
/// Per-command SMTP timeout in seconds.
#[config(env = "MAIL_COMMAND_TIMEOUT_SECS", default = 30)]
@@ -1033,6 +1109,7 @@ pub struct DirectMxConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DkimConfig {
/// DKIM selector. When unset, outgoing mail is not signed.
#[config(env = "MAIL_DKIM_SELECTOR")]
@@ -1049,6 +1126,7 @@ pub struct DkimConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DiscordConfig {
/// Discord bot token. When unset, Discord integration is disabled.
#[config(env = "DISCORD_BOT_TOKEN")]
@@ -1056,6 +1134,7 @@ pub struct DiscordConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TelegramConfig {
/// Telegram bot token. When unset, Telegram integration is disabled.
#[config(env = "TELEGRAM_BOT_TOKEN")]
@@ -1067,14 +1146,7 @@ pub struct TelegramConfig {
}
#[derive(Debug, Config)]
pub struct SignalConfig {
/// Protocol state is stored in postgres' signal_* tables.
/// Link a device via the admin API before enabling.
#[config(env = "SIGNAL_ENABLED", default = false)]
pub enabled: bool,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct NotificationConfig {
/// Polling interval in milliseconds for the comms queue.
#[config(env = "NOTIFICATION_POLL_INTERVAL_MS", default = 1000)]
@@ -1097,6 +1169,7 @@ pub trait SsoProviderIssuerConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoConfig {
#[config(nested)]
pub github: SsoGitHubConfig,
@@ -1118,6 +1191,7 @@ pub struct SsoConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoGitHubConfig {
#[config(env = "SSO_GITHUB_ENABLED", default = false)]
pub enabled: bool,
@@ -1151,6 +1225,7 @@ impl SsoProviderConfig for SsoGitHubConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoDiscordConfig {
#[config(env = "SSO_DISCORD_ENABLED", default = false)]
pub enabled: bool,
@@ -1184,6 +1259,7 @@ impl SsoProviderConfig for SsoDiscordConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoGoogleConfig {
#[config(env = "SSO_GOOGLE_ENABLED", default = false)]
pub enabled: bool,
@@ -1217,6 +1293,7 @@ impl SsoProviderConfig for SsoGoogleConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoGitLabConfig {
#[config(env = "SSO_GITLAB_ENABLED", default = false)]
pub enabled: bool,
@@ -1259,6 +1336,7 @@ impl SsoProviderIssuerConfig for SsoGitLabConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoOidcConfig {
#[config(env = "SSO_OIDC_ENABLED", default = false)]
pub enabled: bool,
@@ -1301,6 +1379,7 @@ impl SsoProviderIssuerConfig for SsoOidcConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoAppleConfig {
#[config(env = "SSO_APPLE_ENABLED", default = false)]
pub enabled: bool,
@@ -1319,6 +1398,7 @@ pub struct SsoAppleConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ModerationConfig {
/// External report-handling service URL.
#[config(env = "REPORT_SERVICE_URL")]
@@ -1330,6 +1410,7 @@ pub struct ModerationConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ImportConfig {
/// Whether the PDS accepts repo imports.
#[config(env = "ACCEPTING_REPO_IMPORTS", default = true)]
@@ -1361,6 +1442,7 @@ fn split_comma_list(value: &str) -> Result<Vec<String>, std::convert::Infallible
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct RippleCacheConfig {
/// Address to bind the Ripple gossip protocol listener.
#[config(env = "RIPPLE_BIND", default = "0.0.0.0:0")]
@@ -1384,6 +1466,7 @@ pub struct RippleCacheConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ScheduledConfig {
/// Interval in seconds between scheduled delete checks.
#[config(env = "SCHEDULED_DELETE_CHECK_INTERVAL_SECS", default = 3600)]
@@ -1431,6 +1514,7 @@ pub struct ScheduledConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TranquilStoreConfig {
/// Directory for tranquil-store data: the metastore, eventlog, and blockstore.
#[config(
@@ -1505,6 +1589,96 @@ mod tests {
.for_each(|(k, v)| unsafe { std::env::set_var(k, v) });
}
#[test]
fn load_rejects_unknown_top_level_key() {
let dir = std::env::temp_dir().join(format!(
"tranquil-config-unknown-toplevel-{}",
std::process::id()
));
std::fs::create_dir_all(&dir).expect("mkdir tempdir");
let path = dir.join("config.toml");
std::fs::write(
&path,
r#"
[server]
hostname = "test.local"
[totally_made_up]
foo = "bar"
"#,
)
.expect("write tempfile");
let result = TranquilConfig::builder().file(&path).load();
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_dir(&dir);
let err = format!("{:#}", result.expect_err("load must reject unknown key"));
assert!(
err.contains("totally_made_up"),
"expected totally_made_up in error, got {err:?}"
);
}
#[test]
fn load_rejects_unknown_nested_key() {
let dir = std::env::temp_dir().join(format!(
"tranquil-config-unknown-nested-{}",
std::process::id()
));
std::fs::create_dir_all(&dir).expect("mkdir tempdir");
let path = dir.join("config.toml");
std::fs::write(
&path,
r#"
[server]
hostname = "test.local"
not_a_real_field = "oops"
"#,
)
.expect("write tempfile");
let result = TranquilConfig::builder().file(&path).load();
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_dir(&dir);
let err = format!("{:#}", result.expect_err("load must reject unknown key"));
assert!(
err.contains("not_a_real_field"),
"expected not_a_real_field in error, got {err:?}"
);
}
#[test]
fn load_accepts_known_keys() {
let dir =
std::env::temp_dir().join(format!("tranquil-config-known-keys-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("mkdir tempdir");
let path = dir.join("config.toml");
std::fs::write(
&path,
r#"
[server]
hostname = "test.local"
port = 3000
[database]
url = "postgres://localhost/test"
[email.smarthost]
host = "smtp.example"
port = 587
"#,
)
.expect("write tempfile");
let result = TranquilConfig::builder().file(&path).load();
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_dir(&dir);
result.expect("known keys must load successfully");
}
#[test]
fn serial_validate_rejects_legacy_sendmail_path() {
seed_required_env();
@@ -1631,6 +1805,60 @@ mod tests {
);
}
#[test]
fn tls_validate_accepts_both_paths_unset() {
let mut errors = Vec::new();
TlsConfig {
cert_path: None,
key_path: None,
http3: false,
}
.validate(&mut errors);
assert!(errors.is_empty(), "expected no errors, got {errors:?}");
}
#[test]
fn tls_validate_accepts_both_paths_set() {
let mut errors = Vec::new();
TlsConfig {
cert_path: Some("/etc/tranquil/cert.pem".to_string()),
key_path: Some("/etc/tranquil/key.pem".to_string()),
http3: false,
}
.validate(&mut errors);
assert!(errors.is_empty(), "expected no errors, got {errors:?}");
}
#[test]
fn tls_validate_rejects_cert_without_key() {
let mut errors = Vec::new();
TlsConfig {
cert_path: Some("/etc/tranquil/cert.pem".to_string()),
key_path: None,
http3: false,
}
.validate(&mut errors);
assert!(
errors.iter().any(|e| e.contains("server.tls")),
"expected server.tls error, got {errors:?}"
);
}
#[test]
fn tls_validate_rejects_http3_without_material() {
let mut errors = Vec::new();
TlsConfig {
cert_path: None,
key_path: None,
http3: true,
}
.validate(&mut errors);
assert!(
errors.iter().any(|e| e.contains("http3")),
"expected http3 error, got {errors:?}"
);
}
#[derive(Default)]
struct EmailOverrides {
from_address: Option<&'static str>,
+15 -18
View File
@@ -319,7 +319,6 @@ pub struct ApplyCommitInput {
#[derive(Debug, Clone)]
pub struct ApplyCommitResult {
pub seq: i64,
pub is_account_active: bool,
}
@@ -445,19 +444,15 @@ pub trait RepoRepository: Send + Sync {
async fn count_user_blocks(&self, user_id: Uuid) -> Result<i64, DbError>;
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<SequenceNumber, DbError>;
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<(), DbError>;
async fn insert_identity_event(
&self,
did: &Did,
handle: Option<&Handle>,
) -> Result<SequenceNumber, DbError>;
) -> Result<(), DbError>;
async fn insert_account_event(
&self,
did: &Did,
status: AccountStatus,
) -> Result<SequenceNumber, DbError>;
async fn insert_account_event(&self, did: &Did, status: AccountStatus) -> Result<(), DbError>;
async fn insert_sync_event(
&self,
@@ -465,7 +460,7 @@ pub trait RepoRepository: Send + Sync {
commit_cid: &CidLink,
rev: Option<&str>,
commit_bytes: &[u8],
) -> Result<SequenceNumber, DbError>;
) -> Result<(), DbError>;
async fn insert_genesis_commit_event(
&self,
@@ -475,14 +470,18 @@ pub trait RepoRepository: Send + Sync {
rev: &str,
commit_bytes: &[u8],
mst_root_bytes: &[u8],
) -> Result<SequenceNumber, DbError>;
async fn delete_sequences_except(
&self,
did: &Did,
keep_seq: SequenceNumber,
) -> Result<(), DbError>;
async fn purge_did_events_keeping_latest(&self, did: &Did) -> Result<(), DbError>;
async fn assign_pending_sequences(&self) -> Result<u64, DbError> {
Ok(0)
}
async fn flush_pending_sequences(&self) -> Result<(), DbError> {
Ok(())
}
async fn prune_events_older_than(&self, cutoff: DateTime<Utc>) -> Result<PruneCount, DbError>;
async fn get_max_seq(&self) -> Result<SequenceNumber, DbError>;
@@ -526,8 +525,6 @@ pub trait RepoRepository: Send + Sync {
async fn get_repo_root_cid_by_user_id(&self, user_id: Uuid)
-> Result<Option<CidLink>, DbError>;
async fn notify_update(&self, seq: SequenceNumber) -> Result<(), DbError>;
async fn import_repo_data(
&self,
user_id: Uuid,
@@ -563,5 +560,5 @@ pub trait RepoEventNotifier: Send + Sync {
#[async_trait]
pub trait RepoEventReceiver: Send {
async fn recv(&mut self) -> Option<i64>;
async fn recv(&mut self) -> Option<()>;
}
+3 -1
View File
@@ -44,6 +44,7 @@ pub struct UserRow {
pub deactivated_at: Option<DateTime<Utc>>,
pub takedown_ref: Option<String>,
pub is_admin: bool,
pub inbound_migration: bool,
}
#[derive(Debug, Clone)]
@@ -550,7 +551,7 @@ pub trait UserRepository: Send + Sync {
limit: i64,
) -> Result<Vec<ScheduledDeletionAccount>, DbError>;
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<i64, DbError>;
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<(), DbError>;
async fn create_password_account(
&self,
@@ -995,6 +996,7 @@ pub struct CreatePasswordAccountInput {
pub telegram_username: Option<String>,
pub signal_username: Option<String>,
pub deactivated_at: Option<DateTime<Utc>>,
pub inbound_migration: bool,
pub encrypted_key_bytes: Vec<u8>,
pub encryption_version: i32,
pub reserved_key_id: Option<Uuid>,
@@ -35,9 +35,9 @@ pub struct PostgresRepoEventReceiver {
#[async_trait]
impl RepoEventReceiver for PostgresRepoEventReceiver {
async fn recv(&mut self) -> Option<i64> {
async fn recv(&mut self) -> Option<()> {
match self.listener.recv().await {
Ok(notification) => notification.payload().parse().ok(),
Ok(_) => Some(()),
Err(_) => None,
}
}
+145 -88
View File
@@ -111,6 +111,64 @@ fn map_sequenced_row(r: SequencedEventRow) -> Result<SequencedEvent, DbError> {
})
}
const SEQUENCER_LOCK_KEY: i64 = 0x0074_7261_6e73_6571;
const SEQUENCER_BATCH_SIZE: i64 = 1000;
async fn notify_repo_pending(pool: &PgPool) {
if let Err(e) = sqlx::query!("NOTIFY repo_updates").execute(pool).await {
tracing::warn!(error = %e, "failed to notify repo_updates listeners");
}
}
async fn assign_one_batch(
mut tx: sqlx::Transaction<'_, sqlx::Postgres>,
pool: &PgPool,
) -> Result<i64, DbError> {
let pending_ids: Vec<i64> = sqlx::query_scalar!(
r#"SELECT id as "id!" FROM repo_seq WHERE seq IS NULL ORDER BY id LIMIT $1"#,
SEQUENCER_BATCH_SIZE
)
.fetch_all(&mut *tx)
.await
.map_err(map_sqlx_error)?;
let count = pending_ids.len() as i64;
if count == 0 {
tx.commit().await.map_err(map_sqlx_error)?;
return Ok(0);
}
let start = sqlx::query_scalar!(r#"SELECT nextval('firehose_seq') as "seq!""#)
.fetch_one(&mut *tx)
.await
.map_err(map_sqlx_error)?;
if count > 1 {
sqlx::query!("SELECT setval('firehose_seq', $1)", start + count - 1)
.fetch_one(&mut *tx)
.await
.map_err(map_sqlx_error)?;
}
let assigned = sqlx::query!(
r#"UPDATE repo_seq r
SET seq = $1::bigint + (o.ord - 1)
FROM unnest($2::bigint[]) WITH ORDINALITY AS o(id, ord)
WHERE r.id = o.id"#,
start,
&pending_ids
)
.execute(&mut *tx)
.await
.map_err(map_sqlx_error)?
.rows_affected();
tx.commit().await.map_err(map_sqlx_error)?;
if assigned > 0 {
notify_repo_pending(pool).await;
}
Ok(count)
}
pub struct PostgresRepoRepository {
pool: PgPool,
}
@@ -716,13 +774,12 @@ impl RepoRepository for PostgresRepoRepository {
Ok(rows.into_iter().map(|(cid,)| cid).collect())
}
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<SequenceNumber, DbError> {
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<(), DbError> {
let (block_cids, block_data) = inline_to_paired_blocks(data.blocks.as_deref());
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
RETURNING seq
"#,
data.did.as_str(),
data.event_type.as_str(),
@@ -735,67 +792,54 @@ impl RepoRepository for PostgresRepoRepository {
data.prev_data_cid.as_ref().map(|c| c.as_str()),
data.rev
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_identity_event(
&self,
did: &Did,
handle: Option<&Handle>,
) -> Result<SequenceNumber, DbError> {
) -> Result<(), DbError> {
let handle_str = handle.map(|h| h.as_str());
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, handle)
VALUES ($1, 'identity', $2)
RETURNING seq
"#,
did.as_str(),
handle_str
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_account_event(
&self,
did: &Did,
status: AccountStatus,
) -> Result<SequenceNumber, DbError> {
async fn insert_account_event(&self, did: &Did, status: AccountStatus) -> Result<(), DbError> {
let active = status.is_active();
let status_str = status.for_firehose();
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, active, status)
VALUES ($1, 'account', $2, $3)
RETURNING seq
"#,
did.as_str(),
active,
status_str
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_sync_event(
@@ -804,18 +848,17 @@ impl RepoRepository for PostgresRepoRepository {
commit_cid: &CidLink,
rev: Option<&str>,
commit_bytes: &[u8],
) -> Result<SequenceNumber, DbError> {
) -> Result<(), DbError> {
let cid_bytes = commit_cid
.to_cid()
.map(|c| c.to_bytes())
.unwrap_or_default();
let block_cids: Vec<Vec<u8>> = vec![cid_bytes];
let block_data: Vec<Vec<u8>> = vec![commit_bytes.to_vec()];
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, rev, block_cids, block_data)
VALUES ($1, 'sync', $2, $3, $4, $5)
RETURNING seq
"#,
did.as_str(),
commit_cid.as_str(),
@@ -823,16 +866,12 @@ impl RepoRepository for PostgresRepoRepository {
&block_cids as &[Vec<u8>],
&block_data as &[Vec<u8>]
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_genesis_commit_event(
@@ -843,7 +882,7 @@ impl RepoRepository for PostgresRepoRepository {
rev: &str,
commit_bytes: &[u8],
mst_root_bytes: &[u8],
) -> Result<SequenceNumber, DbError> {
) -> Result<(), DbError> {
let ops = serde_json::json!([]);
let blobs: Vec<String> = vec![];
let commit_cid_bytes = commit_cid
@@ -858,11 +897,10 @@ impl RepoRepository for PostgresRepoRepository {
let block_data: Vec<Vec<u8>> = vec![commit_bytes.to_vec(), mst_root_bytes.to_vec()];
let prev_cid: Option<&str> = None;
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, rev)
VALUES ($1, 'commit', $2, $3::TEXT, $4, $5, $6, $7, $8)
RETURNING seq
"#,
did.as_str(),
commit_cid.as_str(),
@@ -873,27 +911,22 @@ impl RepoRepository for PostgresRepoRepository {
&block_data as &[Vec<u8>],
rev
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn delete_sequences_except(
&self,
did: &Did,
keep_seq: SequenceNumber,
) -> Result<(), DbError> {
async fn purge_did_events_keeping_latest(&self, did: &Did) -> Result<(), DbError> {
sqlx::query!(
"DELETE FROM repo_seq WHERE did = $1 AND seq != $2",
did.as_str(),
keep_seq.as_i64()
r#"
DELETE FROM repo_seq
WHERE did = $1
AND id <> (SELECT id FROM repo_seq WHERE did = $1 ORDER BY id DESC LIMIT 1)
"#,
did.as_str()
)
.execute(&self.pool)
.await
@@ -902,6 +935,42 @@ impl RepoRepository for PostgresRepoRepository {
Ok(())
}
async fn assign_pending_sequences(&self) -> Result<u64, DbError> {
let mut total: u64 = 0;
loop {
let mut tx = self.pool.begin().await.map_err(map_sqlx_error)?;
let acquired = sqlx::query_scalar!(
r#"SELECT pg_try_advisory_xact_lock($1) as "locked!""#,
SEQUENCER_LOCK_KEY
)
.fetch_one(&mut *tx)
.await
.map_err(map_sqlx_error)?;
if !acquired {
return Ok(total);
}
let count = assign_one_batch(tx, &self.pool).await?;
total += count as u64;
if count < SEQUENCER_BATCH_SIZE {
return Ok(total);
}
}
}
async fn flush_pending_sequences(&self) -> Result<(), DbError> {
loop {
let mut tx = self.pool.begin().await.map_err(map_sqlx_error)?;
sqlx::query!("SELECT pg_advisory_xact_lock($1)", SEQUENCER_LOCK_KEY)
.execute(&mut *tx)
.await
.map_err(map_sqlx_error)?;
let count = assign_one_batch(tx, &self.pool).await?;
if count < SEQUENCER_BATCH_SIZE {
return Ok(());
}
}
}
async fn prune_events_older_than(&self, cutoff: DateTime<Utc>) -> Result<PruneCount, DbError> {
let result = sqlx::query!("DELETE FROM repo_seq WHERE created_at < $1", cutoff)
.execute(&self.pool)
@@ -965,7 +1034,7 @@ impl RepoRepository for PostgresRepoRepository {
Some(lim) => {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1
@@ -982,7 +1051,7 @@ impl RepoRepository for PostgresRepoRepository {
None => {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1
@@ -1004,7 +1073,7 @@ impl RepoRepository for PostgresRepoRepository {
) -> Result<Vec<SequencedEvent>, DbError> {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1 AND seq < $2
@@ -1024,7 +1093,7 @@ impl RepoRepository for PostgresRepoRepository {
) -> Result<Option<SequencedEvent>, DbError> {
let row = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq = $1"#,
@@ -1043,7 +1112,7 @@ impl RepoRepository for PostgresRepoRepository {
) -> Result<Vec<SequencedEvent>, DbError> {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1
@@ -1104,14 +1173,6 @@ impl RepoRepository for PostgresRepoRepository {
Ok(cid.map(CidLink::from))
}
async fn notify_update(&self, seq: SequenceNumber) -> Result<(), DbError> {
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq.as_i64()))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(())
}
async fn import_repo_data(
&self,
user_id: Uuid,
@@ -1402,28 +1463,27 @@ impl RepoRepository for PostgresRepoRepository {
let event = input.commit_event;
let (event_block_cids, event_block_data) = inline_into_paired_blocks(event.blocks);
let seq: i64 = sqlx::query_scalar(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
RETURNING seq
"#,
event.did.as_str(),
event.event_type.as_str(),
event.commit_cid.as_ref().map(|c| c.as_str()),
event.prev_cid.as_ref().map(|c| c.as_str()),
event.ops,
event.blobs.as_deref(),
&event_block_cids as &[Vec<u8>],
&event_block_data as &[Vec<u8>],
event.prev_data_cid.as_ref().map(|c| c.as_str()),
event.rev
)
.bind(event.did.as_str())
.bind(event.event_type.as_str())
.bind(event.commit_cid.as_ref().map(|c| c.as_str()))
.bind(event.prev_cid.as_ref().map(|c| c.as_str()))
.bind(&event.ops)
.bind(&event.blobs)
.bind(&event_block_cids)
.bind(&event_block_data)
.bind(event.prev_data_cid.as_ref().map(|c| c.as_str()))
.bind(&event.rev)
.fetch_one(&mut *tx)
.execute(&mut *tx)
.await
.map_err(|e| ApplyCommitError::Database(e.to_string()))?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
sqlx::query!("NOTIFY repo_updates")
.execute(&mut *tx)
.await
.map_err(|e| ApplyCommitError::Database(e.to_string()))?;
@@ -1432,10 +1492,7 @@ impl RepoRepository for PostgresRepoRepository {
.await
.map_err(|e| ApplyCommitError::Database(e.to_string()))?;
Ok(tranquil_db_traits::ApplyCommitResult {
seq,
is_account_active,
})
Ok(tranquil_db_traits::ApplyCommitResult { is_account_active })
}
async fn get_users_without_blocks(&self) -> Result<Vec<UserWithoutBlocks>, DbError> {
+15 -12
View File
@@ -47,7 +47,7 @@ pub(crate) fn map_sqlx_error(e: sqlx::Error) -> DbError {
impl UserRepository for PostgresUserRepository {
async fn get_by_did(&self, did: &Did) -> Result<Option<UserRow>, DbError> {
let row = sqlx::query!(
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration
FROM users WHERE did = $1"#,
did.as_str()
)
@@ -64,12 +64,13 @@ impl UserRepository for PostgresUserRepository {
deactivated_at: r.deactivated_at,
takedown_ref: r.takedown_ref,
is_admin: r.is_admin,
inbound_migration: r.inbound_migration,
}))
}
async fn get_by_handle(&self, handle: &Handle) -> Result<Option<UserRow>, DbError> {
let row = sqlx::query!(
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration
FROM users WHERE handle = $1"#,
handle.as_str()
)
@@ -86,6 +87,7 @@ impl UserRepository for PostgresUserRepository {
deactivated_at: r.deactivated_at,
takedown_ref: r.takedown_ref,
is_admin: r.is_admin,
inbound_migration: r.inbound_migration,
}))
}
@@ -1863,7 +1865,7 @@ impl UserRepository for PostgresUserRepository {
async fn activate_account(&self, did: &Did) -> Result<bool, DbError> {
let result = sqlx::query!(
"UPDATE users SET deactivated_at = NULL WHERE did = $1",
"UPDATE users SET deactivated_at = NULL, inbound_migration = FALSE WHERE did = $1",
did.as_str()
)
.execute(&self.pool)
@@ -2289,7 +2291,7 @@ impl UserRepository for PostgresUserRepository {
.collect())
}
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<i64, DbError> {
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<(), DbError> {
let mut tx = self.pool.begin().await.map_err(map_sqlx_error)?;
sqlx::query!("DELETE FROM blobs WHERE created_by_user = $1", user_id)
@@ -2368,11 +2370,11 @@ impl UserRepository for PostgresUserRepository {
.await
.map_err(map_sqlx_error)?;
let account_seq: i64 = sqlx::query_scalar!(
let event_id: i64 = sqlx::query_scalar!(
r#"
INSERT INTO repo_seq (did, event_type, active, status)
VALUES ($1, 'account', false, 'deleted')
RETURNING seq
RETURNING id
"#,
did.as_str()
)
@@ -2381,9 +2383,9 @@ impl UserRepository for PostgresUserRepository {
.map_err(map_sqlx_error)?;
sqlx::query!(
"DELETE FROM repo_seq WHERE did = $1 AND seq != $2",
"DELETE FROM repo_seq WHERE did = $1 AND id <> $2",
did.as_str(),
account_seq
event_id
)
.execute(&mut *tx)
.await
@@ -2391,12 +2393,12 @@ impl UserRepository for PostgresUserRepository {
tx.commit().await.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", account_seq))
sqlx::query!("NOTIFY repo_updates")
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(account_seq)
Ok(())
}
async fn create_password_account(
@@ -2426,8 +2428,8 @@ impl UserRepository for PostgresUserRepository {
handle, email, did, password_hash,
preferred_comms_channel,
discord_username, telegram_username, signal_username,
is_admin, deactivated_at, email_verified
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, FALSE) RETURNING id"#,
is_admin, deactivated_at, inbound_migration, email_verified
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, FALSE) RETURNING id"#,
)
.bind(input.handle.as_str())
.bind(&input.email)
@@ -2439,6 +2441,7 @@ impl UserRepository for PostgresUserRepository {
.bind(&input.signal_username)
.bind(is_first_user)
.bind(input.deactivated_at)
.bind(input.inbound_migration)
.fetch_one(&mut *tx)
.await;
@@ -310,6 +310,7 @@ pub async fn authorize_post(
State(state): State<AppState>,
_rate_limit: OAuthRateLimited<OAuthAuthorizeLimit>,
headers: HeaderMap,
client_ip: ClientIp,
Json(form): Json<AuthorizeSubmit>,
) -> Response {
let json_response = wants_json(&headers);
@@ -488,8 +489,7 @@ pub async fn authorize_post(
if !password_valid {
return show_login_error("Invalid identifier or password.", json_response);
}
let is_verified = user.channel_verification.has_any_verified();
if !is_verified {
if tranquil_api::server::verification_blocks_login(&user.channel_verification) {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &user.did).await;
let handle = resend_info
.as_ref()
@@ -617,7 +617,7 @@ pub async fn authorize_post(
let device_data = DeviceData {
session_id: SessionId::generate(),
user_agent: extract_user_agent(&headers),
ip_address: extract_client_ip(&headers, None),
ip_address: client_ip.into_string(),
last_seen_at: Utc::now(),
};
if state
@@ -854,8 +854,7 @@ pub async fn authorize_select(
);
}
};
let is_verified = user.channel_verification.has_any_verified();
if !is_verified {
if tranquil_api::server::verification_blocks_login(&user.channel_verification) {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &did).await;
return (
StatusCode::FORBIDDEN,
@@ -23,7 +23,7 @@ use tranquil_pds::rate_limit::{
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{Did, Handle, PlainPassword};
use tranquil_pds::util::extract_client_ip;
use tranquil_pds::util::ClientIp;
use tranquil_types::{AuthorizationCode, ClientId, DeviceId as DeviceIdType, RequestId};
use urlencoding::encode as url_encode;
@@ -289,9 +289,7 @@ async fn passkey_start_named(
.into_response();
}
let is_verified = user.channel_verification.has_any_verified();
if !is_verified {
if tranquil_api::server::verification_blocks_login(&user.channel_verification) {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &user.did).await;
return (
StatusCode::FORBIDDEN,
@@ -160,8 +160,10 @@ pub async fn register_complete(
.into_response();
}
let is_verified = match state.repos.user.get_session_info_by_did(&did).await {
Ok(Some(info)) => info.channel_verification.has_any_verified(),
let login_blocked = match state.repos.user.get_session_info_by_did(&did).await {
Ok(Some(info)) => {
tranquil_api::server::verification_blocks_login(&info.channel_verification)
}
Ok(None) => {
return (
StatusCode::FORBIDDEN,
@@ -189,7 +191,7 @@ pub async fn register_complete(
}
};
if !is_verified {
if login_blocked {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &did).await;
return (
StatusCode::FORBIDDEN,
@@ -300,6 +302,7 @@ pub async fn register_complete(
pub async fn establish_session(
State(state): State<AppState>,
headers: HeaderMap,
client_ip: ClientIp,
auth: tranquil_pds::auth::Auth<tranquil_pds::auth::Active>,
) -> Response {
let did = &auth.did;
@@ -317,7 +320,7 @@ pub async fn establish_session(
let device_data = DeviceData {
session_id: SessionId::generate(),
user_agent: extract_user_agent(&headers),
ip_address: extract_client_ip(&headers, None),
ip_address: client_ip.into_string(),
last_seen_at: Utc::now(),
};
@@ -75,6 +75,7 @@ pub async fn authorize_2fa_post(
State(state): State<AppState>,
_rate_limit: OAuthRateLimited<OAuthAuthorizeLimit>,
headers: HeaderMap,
client_ip: ClientIp,
Json(form): Json<Authorize2faSubmit>,
) -> Response {
let json_error = |status: StatusCode, error: &str, description: &str| -> Response {
@@ -251,7 +252,7 @@ pub async fn authorize_2fa_post(
let device_data = DeviceData {
session_id: SessionId::generate(),
user_agent: extract_user_agent(&headers),
ip_address: extract_client_ip(&headers, None),
ip_address: client_ip.into_string(),
last_seen_at: Utc::now(),
};
if state
@@ -12,7 +12,7 @@ use tranquil_pds::oauth::client::{build_client_metadata, delegation_oauth_urls};
use tranquil_pds::rate_limit::{LoginLimit, OAuthRateLimited, TotpVerifyLimit};
use tranquil_pds::state::AppState;
use tranquil_pds::types::PlainPassword;
use tranquil_pds::util::extract_client_ip;
use tranquil_pds::util::ClientIp;
use tranquil_types::did_doc::{extract_handle, extract_pds_endpoint};
use tranquil_types::{Did, RequestId};
@@ -402,6 +402,7 @@ pub struct DelegationTokenAuthSubmit {
pub async fn delegation_auth_token(
State(state): State<AppState>,
headers: HeaderMap,
client_ip: ClientIp,
auth: Auth<Active>,
Json(form): Json<DelegationTokenAuthSubmit>,
) -> Response {
@@ -428,7 +429,7 @@ pub async fn delegation_auth_token(
return resp;
}
let ip = extract_client_ip(&headers, None);
let ip = client_ip.into_string();
let user_agent = tranquil_pds::util::extract_user_agent(&headers);
finalize_delegation_auth(
@@ -402,13 +402,15 @@ async fn handle_sso_login(
}
};
let is_verified = match state
let login_blocked = match state
.repos
.user
.get_session_info_by_did(&identity.did)
.await
{
Ok(Some(info)) => info.channel_verification.has_any_verified(),
Ok(Some(info)) => {
tranquil_api::server::verification_blocks_login(&info.channel_verification)
}
Ok(None) => {
tracing::error!("User not found for SSO login: {}", identity.did);
return redirect_to_error("Account not found");
@@ -419,7 +421,7 @@ async fn handle_sso_login(
}
};
if !is_verified {
if login_blocked {
tracing::warn!(
did = %identity.did,
provider = %provider.as_str(),
+3
View File
@@ -23,3 +23,6 @@ sha2 = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
uuid = { workspace = true }
[features]
native-tls-roots = []
+15 -12
View File
@@ -78,18 +78,21 @@ impl ClientMetadataCache {
Self {
cache: Arc::new(RwLock::new(HashMap::new())),
jwks_cache: Arc::new(RwLock::new(HashMap::new())),
http_client: Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.pool_max_idle_per_host(10)
.pool_idle_timeout(std::time::Duration::from_secs(90))
.user_agent(concat!(
"Tranquil-PDS/",
env!("CARGO_PKG_VERSION"),
" (ATProto; +https://tangled.org/tranquil.farm/tranquil-pds)"
))
.build()
.unwrap_or_else(|_| Client::new()),
http_client: {
let builder = Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.pool_max_idle_per_host(10)
.pool_idle_timeout(std::time::Duration::from_secs(90))
.user_agent(concat!(
"Tranquil-PDS/",
env!("CARGO_PKG_VERSION"),
" (ATProto; +https://tangled.org/tranquil.farm/tranquil-pds)"
));
#[cfg(feature = "native-tls-roots")]
let builder = builder.danger_accept_invalid_certs(true);
builder.build().unwrap_or_else(|_| Client::new())
},
cache_ttl_secs,
}
}
+2
View File
@@ -91,8 +91,10 @@ s3-storage = ["tranquil-storage/s3", "dep:aws-config", "dep:aws-sdk-s3"]
s3 = ["s3-storage"]
valkey = ["tranquil-cache/valkey", "dep:redis"]
frontend = []
native-tls-roots = ["tranquil-oauth/native-tls-roots"]
[dev-dependencies]
tempfile = "3"
ciborium = { workspace = true }
ctor = { workspace = true }
testcontainers = { workspace = true }
+30 -2
View File
@@ -15,6 +15,7 @@ struct ErrorBody<'a> {
#[derive(Debug)]
pub enum ApiError {
InternalError(Option<String>),
RepoCorruption,
AuthenticationRequired,
AuthenticationFailed(Option<String>),
InvalidRequest(String),
@@ -121,10 +122,34 @@ pub enum ApiError {
},
}
const MST_NODE_MISSING_MARKER: &str = "MST node not found";
impl ApiError {
pub fn is_repo_corruption(&self) -> bool {
matches!(self, Self::RepoCorruption)
}
pub fn detail_is_repo_corruption(detail: &str) -> bool {
detail.contains(tranquil_store::blockstore::BLOCK_CORRUPTION_MARKER)
|| detail.contains(MST_NODE_MISSING_MARKER)
}
pub fn from_mst_error(context: &str, e: &jacquard_repo::error::RepoError) -> Self {
let detail = format!("{e:#}");
if Self::detail_is_repo_corruption(&detail) {
tracing::warn!("{context}: repairable MST damage: {detail}");
Self::RepoCorruption
} else {
tracing::error!("{context}: {detail}");
Self::InternalError(None)
}
}
fn status_code(&self) -> StatusCode {
match self {
Self::InternalError(_) | Self::DatabaseError => StatusCode::INTERNAL_SERVER_ERROR,
Self::InternalError(_) | Self::RepoCorruption | Self::DatabaseError => {
StatusCode::INTERNAL_SERVER_ERROR
}
Self::UpstreamFailure | Self::UpstreamUnavailable(_) | Self::UpstreamErrorMsg(_) => {
StatusCode::BAD_GATEWAY
}
@@ -223,7 +248,9 @@ impl ApiError {
}
fn error_name(&self) -> Cow<'static, str> {
match self {
Self::InternalError(_) | Self::DatabaseError => Cow::Borrowed("InternalServerError"),
Self::InternalError(_) | Self::RepoCorruption | Self::DatabaseError => {
Cow::Borrowed("InternalServerError")
}
Self::UpstreamFailure | Self::UpstreamUnavailable(_) | Self::UpstreamErrorMsg(_) => {
Cow::Borrowed("UpstreamError")
}
@@ -332,6 +359,7 @@ impl ApiError {
Self::InternalError(msg) => msg
.clone()
.unwrap_or_else(|| "Internal Server Error".into()),
Self::RepoCorruption => "Internal Server Error".into(),
Self::AuthenticationFailed(msg) => msg
.clone()
.unwrap_or_else(|| "Authentication failed".into()),
+48 -2
View File
@@ -109,6 +109,35 @@ fn is_protected_method(method: &str) -> bool {
PROTECTED_METHODS.contains(method)
}
/// Fetch the `feed` generator record from the AppView and return its `did`.
async fn resolve_feed_generator_did(appview_url: &str, query: Option<&str>) -> Option<String> {
#[derive(serde::Deserialize)]
struct GetFeedQuery {
feed: String,
}
let feed = serde_urlencoded::from_str::<GetFeedQuery>(query?)
.ok()?
.feed;
let at_uri = crate::types::AtUri::new(feed).ok()?;
let repo = at_uri.did()?;
let collection = at_uri.collection()?;
let rkey = at_uri.rkey()?;
let resp = proxy_client()
.get(format!("{appview_url}/xrpc/com.atproto.repo.getRecord"))
.query(&[("repo", repo), ("collection", collection), ("rkey", rkey)])
.send()
.await
.ok()?;
if !resp.status().is_success() {
warn!(status = %resp.status(), "getFeed proxy: getRecord for feed generator failed");
return None;
}
let body: serde_json::Value = resp.json().await.ok()?;
body.get("value")?.get("did")?.as_str().map(str::to_string)
}
pub struct XrpcProxyLayer {
state: AppState,
}
@@ -287,10 +316,27 @@ async fn proxy_handler(
},
};
// getFeed must be audienced to the feed generator, not the AppView.
let (token_aud, token_lxm) = if method == "app.bsky.feed.getFeed" {
match resolve_feed_generator_did(&resolved.url, query.as_deref()).await {
Some(feed_did) => (feed_did, "app.bsky.feed.getFeedSkeleton"),
None => {
warn!(
"getFeed proxy: could not resolve feed generator DID; refusing \
to mint an AppView-audienced token"
);
return ApiError::InvalidRequest("Could not resolve feed".into())
.into_response();
}
}
} else {
(resolved.did.clone(), method)
};
match crate::auth::create_service_token(
&auth_user.did,
&resolved.did,
Some(method),
&token_aud,
Some(token_lxm),
&key_bytes,
) {
Ok(new_token) => {
@@ -21,6 +21,13 @@ pub async fn require_verified_or_delegated<'a>(
state: &AppState,
user: &'a AuthenticatedUser,
) -> Result<AccountVerified<'a>, ApiError> {
if tranquil_config::get()
.server
.disable_account_verification_gate
{
return Ok(AccountVerified { user });
}
let is_verified = state
.repos
.user
+31 -27
View File
@@ -1,11 +1,10 @@
use rand::Rng;
use serde::{Deserialize, Serialize};
use std::time::Duration;
use crate::cache::Cache;
use crate::util::{generate_token_code, normalize_token_code};
const TOKEN_TTL_SECS: u64 = 900;
const BASE32_CHARS: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EmailTokenPurpose {
@@ -46,14 +45,6 @@ fn cache_key(did: &str, purpose: EmailTokenPurpose) -> String {
format!("email_token:{}:{}", purpose.as_str(), did)
}
fn generate_short_token() -> String {
let mut rng = rand::thread_rng();
let token: String = (0..10)
.map(|_| BASE32_CHARS[rng.gen_range(0..BASE32_CHARS.len())] as char)
.collect();
format!("{}-{}", &token[0..5], &token[5..10])
}
fn current_timestamp() -> u64 {
u64::try_from(chrono::Utc::now().timestamp()).unwrap_or(0)
}
@@ -67,9 +58,9 @@ pub async fn create_email_token(
return Err(TokenError::CacheUnavailable);
}
let token = generate_short_token();
let token = generate_token_code();
let data = TokenData {
token: token.clone(),
token: normalize_token_code(&token),
created_at: current_timestamp(),
};
@@ -108,10 +99,9 @@ pub async fn validate_email_token(
return Err(TokenError::ExpiredToken);
}
let normalized_input = token.to_uppercase().replace('-', "");
let normalized_stored = data.token.to_uppercase().replace('-', "");
let normalized_input = normalize_token_code(token);
if !constant_time_eq(normalized_input.as_bytes(), normalized_stored.as_bytes()) {
if !constant_time_eq(normalized_input.as_bytes(), data.token.as_bytes()) {
return Err(TokenError::InvalidToken);
}
@@ -260,20 +250,19 @@ mod tests {
#[tokio::test]
async fn test_token_format() {
(0..100).for_each(|_| {
let token = generate_short_token();
// The emitted token is the display form: uppercase `XXXXX-XXXXX`.
let cache = MockCache::new();
let did = "did:plc:test123";
(0..50).for_each(|_| {
let token = futures::executor::block_on(create_email_token(
&cache,
did,
EmailTokenPurpose::UpdateEmail,
))
.unwrap();
assert_eq!(token.len(), 11);
assert_eq!(&token[5..6], "-");
assert!(
token[0..5]
.chars()
.all(|c| BASE32_CHARS.contains(&(c as u8)))
);
assert!(
token[6..11]
.chars()
.all(|c| BASE32_CHARS.contains(&(c as u8)))
);
assert_eq!(token, token.to_uppercase());
});
}
@@ -292,6 +281,21 @@ mod tests {
assert!(result.is_ok());
}
#[tokio::test]
async fn test_hyphen_insensitive_validation() {
let cache = MockCache::new();
let did = "did:plc:test123";
let token = create_email_token(&cache, did, EmailTokenPurpose::UpdateEmail)
.await
.unwrap();
let no_hyphen = token.replace('-', "");
let result =
validate_email_token(&cache, did, EmailTokenPurpose::UpdateEmail, &no_hyphen).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_noop_cache_returns_unavailable() {
let cache = crate::cache::NoOpCache;
+227 -31
View File
@@ -1,15 +1,14 @@
use chrono::Utc;
use rand::Rng;
use serde::{Deserialize, Serialize};
use std::time::Duration;
use crate::cache::Cache;
use crate::types::Did;
use crate::util::{generate_token_code, normalize_token_code};
const CHALLENGE_TTL_SECS: u64 = 300;
const MIN_REMAINING_TTL_SECS: u64 = 10;
const MAX_ATTEMPTS: u8 = 5;
const CODE_LENGTH: usize = 8;
const COOLDOWN_SECS: u64 = 60;
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -94,7 +93,8 @@ async fn validate_challenge_internal(
return Err(ValidationError::ChallengeExpired);
}
if !constant_time_eq(code.as_bytes(), data.code.as_bytes()) {
let normalized_input = normalize_token_code(code);
if !constant_time_eq(normalized_input.as_bytes(), data.code.as_bytes()) {
let updated = ChallengeData {
code: data.code,
attempts: data.attempts + 1,
@@ -127,17 +127,19 @@ fn cooldown_key(did: &str) -> String {
format!("legacy_2fa_cooldown:{}", did)
}
fn generate_code() -> String {
let mut rng = rand::thread_rng();
(0..CODE_LENGTH)
.map(|_| rng.gen_range(0..10).to_string())
.collect()
}
fn current_timestamp() -> u64 {
u64::try_from(Utc::now().timestamp()).unwrap_or(0)
}
pub fn looks_like_totp_token(code: &str) -> bool {
let c = code.trim();
(c.len() == 6 && c.bytes().all(|b| b.is_ascii_digit())) || crate::auth::is_backup_code_format(c)
}
pub fn used_totp_factor(has_totp: bool, auth_factor_token: Option<&str>) -> bool {
has_totp && auth_factor_token.is_some_and(looks_like_totp_token)
}
fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
if a.len() != b.len() {
return false;
@@ -148,6 +150,7 @@ fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
== 0
}
#[derive(Debug)]
pub enum Legacy2faOutcome {
NotRequired,
Blocked,
@@ -177,6 +180,7 @@ pub async fn process_legacy_2fa(
did: &Did,
ctx: &Legacy2faContext,
auth_factor_token: Option<&str>,
verify_totp: impl AsyncFnOnce(&str) -> bool,
) -> Result<Legacy2faOutcome, Legacy2faFlowError> {
if !ctx.requires_2fa() {
return Ok(Legacy2faOutcome::NotRequired);
@@ -192,8 +196,16 @@ pub async fn process_legacy_2fa(
Ok(Legacy2faOutcome::ChallengeSent(code))
}
Some(token) => {
validate_challenge(cache, did, token).await?;
Ok(Legacy2faOutcome::Verified)
if ctx.has_totp && looks_like_totp_token(token) {
if verify_totp(token).await {
Ok(Legacy2faOutcome::Verified)
} else {
Err(Legacy2faFlowError::Validation(ValidationError::InvalidCode))
}
} else {
validate_challenge(cache, did, token).await?;
Ok(Legacy2faOutcome::Verified)
}
}
}
}
@@ -219,11 +231,11 @@ async fn create_challenge_code(
return Err(ChallengeError::RateLimited);
}
let code = generate_code();
let display = generate_token_code();
let now = current_timestamp();
let data = ChallengeData {
code: code.clone(),
code: normalize_token_code(&display),
attempts: 0,
created_at: now,
};
@@ -244,7 +256,7 @@ async fn create_challenge_code(
.await
.map_err(|_| ChallengeError::CacheError)?;
Ok(ChallengeCode(code))
Ok(ChallengeCode(display))
}
#[derive(Debug)]
@@ -332,12 +344,46 @@ mod tests {
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
assert_eq!(code.as_str().len(), CODE_LENGTH);
assert_eq!(code.as_str().len(), 11);
let result = validate_challenge(&cache, &did, code.as_str()).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_challenge_code_format() {
let cache = MockCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
let code = code.as_str();
assert_eq!(code.len(), 11);
assert_eq!(&code[5..6], "-");
assert_eq!(code, code.to_uppercase());
}
#[tokio::test]
async fn test_case_insensitive_validation() {
let cache = MockCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
let lowercase = code.as_str().to_lowercase();
let result = validate_challenge(&cache, &did, &lowercase).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_hyphen_insensitive_validation() {
let cache = MockCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
let no_hyphen = code.as_str().replace('-', "");
let result = validate_challenge(&cache, &did, &no_hyphen).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_invalid_code_rejected() {
let cache = MockCache::new();
@@ -396,15 +442,6 @@ mod tests {
assert_eq!(result.unwrap_err(), ChallengeError::CacheUnavailable);
}
#[tokio::test]
async fn test_code_generation_is_numeric() {
(0..100).for_each(|_| {
let code = generate_code();
assert!(code.chars().all(|c| c.is_ascii_digit()));
assert_eq!(code.len(), CODE_LENGTH);
});
}
#[tokio::test]
async fn test_constant_time_eq() {
assert!(constant_time_eq(b"12345678", b"12345678"));
@@ -425,7 +462,9 @@ mod tests {
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::NotRequired));
}
@@ -440,7 +479,9 @@ mod tests {
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::NotRequired));
}
@@ -455,7 +496,9 @@ mod tests {
allow_legacy_login: false,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Blocked));
}
@@ -470,7 +513,9 @@ mod tests {
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::ChallengeSent(_)));
}
@@ -485,7 +530,9 @@ mod tests {
allow_legacy_login: false,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::ChallengeSent(_)));
}
@@ -502,7 +549,7 @@ mod tests {
let code = create_challenge(&cache, &did).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some(code.as_str()))
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some(code.as_str()), reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
@@ -532,4 +579,153 @@ mod tests {
let result = validate_challenge(&cache, &did, "12345678").await;
assert_eq!(result.unwrap_err(), ValidationError::CacheUnavailable);
}
async fn reject_totp(_code: &str) -> bool {
false
}
async fn accept_totp(_code: &str) -> bool {
true
}
#[tokio::test]
async fn test_totp_shaped_token_accepted_via_verifier() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp1".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: false,
has_totp: true,
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some("123456"), accept_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_totp_shaped_token_rejected_does_not_touch_email_challenge() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp2".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: true,
has_totp: true,
allow_legacy_login: true,
};
// An email challenge exists for this user.
let email_code = create_challenge(&cache, &did).await.unwrap();
// Five wrong TOTP-shaped attempts. If these incremented the email attempt
// counter, the email challenge would be exhausted (MAX_ATTEMPTS = 5).
for _ in 0..5 {
let err = process_legacy_2fa(&cache, &did, &ctx, Some("000000"), reject_totp)
.await
.unwrap_err();
assert!(matches!(
err,
Legacy2faFlowError::Validation(ValidationError::InvalidCode)
));
}
// The email challenge is still valid and consumable.
let outcome =
process_legacy_2fa(&cache, &did, &ctx, Some(email_code.as_str()), reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_email_shaped_token_routes_to_email_path_when_totp_present() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp3".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: true,
has_totp: true,
allow_legacy_login: true,
};
let email_code = create_challenge(&cache, &did).await.unwrap();
// reject_totp would fail if this routed to the verifier; it must route to email.
let outcome =
process_legacy_2fa(&cache, &did, &ctx, Some(email_code.as_str()), reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_backup_code_shaped_token_routes_to_verifier() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp4".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: false,
has_totp: true,
allow_legacy_login: true,
};
// No email challenge created. If this routed to email it would be
// ChallengeNotFound; Verified proves it went to the verifier.
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some("ABCD2345"), accept_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_totp_shaped_token_ignored_when_no_totp() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp5".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: true,
has_totp: false,
allow_legacy_login: false,
};
// has_totp = false -> 6-digit token routes to email path; no challenge -> NotFound.
let err = process_legacy_2fa(&cache, &did, &ctx, Some("123456"), reject_totp)
.await
.unwrap_err();
assert!(matches!(
err,
Legacy2faFlowError::Validation(ValidationError::ChallengeNotFound)
));
}
#[tokio::test]
async fn test_looks_like_totp_token() {
// 6-digit TOTP codes
assert!(looks_like_totp_token("123456"));
assert!(looks_like_totp_token(" 000000 "));
// backup-code format (8 chars, backup alphabet)
assert!(looks_like_totp_token("ABCD2345"));
// email challenge codes normalize to 10 alphanumeric chars -> not TOTP-shaped
assert!(!looks_like_totp_token("ABCDEFGHIJ"));
assert!(!looks_like_totp_token("ABCDE-FGHIJ"));
// wrong lengths / non-digits
assert!(!looks_like_totp_token("12345"));
assert!(!looks_like_totp_token("1234567"));
assert!(!looks_like_totp_token("12345A"));
assert!(!looks_like_totp_token(""));
}
#[test]
fn test_used_totp_factor() {
// strong MFA factors completed the login -> true
assert!(used_totp_factor(true, Some("123456")));
assert!(used_totp_factor(true, Some("ABCD2345")));
// email-shaped code, or no token, or no TOTP on the account -> false
assert!(!used_totp_factor(true, Some("ABCDEFGHIJ")));
assert!(!used_totp_factor(true, None));
assert!(!used_totp_factor(false, Some("123456")));
assert!(!used_totp_factor(true, Some("")));
}
}
+1 -1
View File
@@ -522,7 +522,7 @@ async fn validate_bearer_token_with_options_internal(
auth_source: AuthSource::OAuth,
});
} else {
return Err(TokenValidationError::TokenExpired);
return Err(TokenValidationError::OAuthTokenExpired);
}
}
+3 -3
View File
@@ -51,6 +51,8 @@ pub const BUILD_VERSION: &str = concat!(
#[cfg(not(debug_assertions))]
pub const BUILD_VERSION: &str = env!("CARGO_PKG_VERSION");
pub const GENERAL_BODY_LIMIT: usize = 16 * 1024 * 1024;
pub struct ExternalRoutes {
pub xrpc: Router<AppState>,
pub oauth: Router<AppState>,
@@ -97,9 +99,7 @@ pub fn app_with_routes(state: AppState, external: ExternalRoutes) -> Router {
.nest("/.well-known", well_known_router)
.route("/metrics", get(metrics::metrics_handler))
.merge(external.extra)
.layer(DefaultBodyLimit::max(
tranquil_config::get().server.max_blob_size as usize,
))
.layer(DefaultBodyLimit::max(GENERAL_BODY_LIMIT))
.layer(axum::middleware::map_response(rewrite_extractor_errors))
.layer(middleware::from_fn(metrics::metrics_middleware))
.layer(
+135 -9
View File
@@ -39,27 +39,77 @@ pub enum PlcOpType {
Tombstone,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[derive(Debug, Error)]
#[error("service type must not be empty")]
pub struct EmptyServiceType;
mod custom_service_type {
use super::EmptyServiceType;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CustomServiceType(String);
impl CustomServiceType {
pub(super) fn new(name: String) -> Result<Self, EmptyServiceType> {
match name.as_str() {
"" => Err(EmptyServiceType),
_ => Ok(Self(name)),
}
}
pub fn as_str(&self) -> &str {
&self.0
}
}
}
pub use custom_service_type::CustomServiceType;
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ServiceType {
#[serde(rename = "AtprotoPersonalDataServer")]
Pds,
#[serde(rename = "AtprotoAppView")]
AppView,
#[serde(rename = "AtprotoLabeler")]
Labeler,
Other(CustomServiceType),
}
impl ServiceType {
pub fn as_str(self) -> &'static str {
pub fn as_str(&self) -> &str {
match self {
Self::Pds => "AtprotoPersonalDataServer",
Self::AppView => "AtprotoAppView",
Self::Labeler => "AtprotoLabeler",
Self::Other(name) => name.as_str(),
}
}
}
pub fn is_pds(self) -> bool {
matches!(self, Self::Pds)
impl TryFrom<String> for ServiceType {
type Error = EmptyServiceType;
fn try_from(name: String) -> Result<Self, Self::Error> {
match name.as_str() {
"AtprotoPersonalDataServer" => Ok(Self::Pds),
"AtprotoLabeler" => Ok(Self::Labeler),
_ => CustomServiceType::new(name).map(Self::Other),
}
}
}
impl Serialize for ServiceType {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
serializer.serialize_str(self.as_str())
}
}
impl<'de> Deserialize<'de> for ServiceType {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let name = String::deserialize(deserializer)?;
Self::try_from(name).map_err(serde::de::Error::custom)
}
}
@@ -625,4 +675,80 @@ mod tests {
let signed = sign_operation(&op, &key).unwrap();
assert!(signed.get("sig").is_some());
}
#[test]
fn test_service_type_known_round_trip() {
let cases = [
(ServiceType::Pds, "\"AtprotoPersonalDataServer\""),
(ServiceType::Labeler, "\"AtprotoLabeler\""),
];
cases.iter().for_each(|(variant, encoded)| {
assert_eq!(serde_json::to_string(variant).unwrap(), *encoded);
assert_eq!(
serde_json::from_str::<ServiceType>(encoded).unwrap(),
*variant
);
});
}
#[test]
fn test_service_type_custom_round_trips() {
let parsed: ServiceType = serde_json::from_str("\"ConchFeedGenerator\"").unwrap();
assert_eq!(
parsed,
ServiceType::try_from("ConchFeedGenerator".to_string()).unwrap()
);
assert_eq!(
serde_json::to_string(&parsed).unwrap(),
"\"ConchFeedGenerator\""
);
}
#[test]
fn test_service_type_custom_normalizes_known_names() {
assert_eq!(
ServiceType::try_from("AtprotoPersonalDataServer".to_string()).unwrap(),
ServiceType::Pds
);
assert_eq!(
ServiceType::try_from("AtprotoLabeler".to_string()).unwrap(),
ServiceType::Labeler
);
}
#[test]
fn test_appview_is_not_a_named_type() {
assert_eq!(
ServiceType::try_from("AtprotoAppView".to_string()).unwrap(),
ServiceType::Other(CustomServiceType::new("AtprotoAppView".to_string()).unwrap())
);
}
#[test]
fn test_service_type_rejects_empty() {
assert!(ServiceType::try_from(String::new()).is_err());
assert!(serde_json::from_str::<ServiceType>("\"\"").is_err());
}
#[test]
fn test_plc_operation_with_custom_service_round_trips() {
let op_json = json!({
"type": "plc_operation",
"rotationKeys": ["did:key:zScallop"],
"verificationMethods": { "atproto": "did:key:zUni" },
"alsoKnownAs": ["at://whelk.nel.pet"],
"services": {
"atproto_pds": { "type": "AtprotoPersonalDataServer", "endpoint": "https://nel.pet" },
"custom_feedgen": { "type": "ConchFeedGenerator", "endpoint": "https://feed.nel.pet" }
},
"prev": null
});
let op: PlcOperation = serde_json::from_value(op_json.clone()).unwrap();
assert_eq!(
op.services["custom_feedgen"].service_type,
ServiceType::try_from("ConchFeedGenerator".to_string()).unwrap()
);
assert_eq!(op.services["atproto_pds"].service_type, ServiceType::Pds);
assert_eq!(serde_json::to_value(&op).unwrap(), op_json);
}
}
@@ -9,7 +9,7 @@ use axum::{
use crate::api::error::ApiError;
use crate::oauth::OAuthError;
use crate::state::{AppState, RateLimitKind};
use crate::util::extract_client_ip;
use crate::util::client_ip_from_parts;
pub trait RateLimitPolicy: Send + Sync + 'static {
const KIND: RateLimitKind;
@@ -173,7 +173,7 @@ impl<P: RateLimitPolicy, R: RateLimitRejection> FromRequestParts<AppState>
parts: &mut Parts,
state: &AppState,
) -> Result<Self, Self::Rejection> {
let client_ip = extract_client_ip(&parts.headers, None);
let client_ip = client_ip_from_parts(parts);
if !state.check_rate_limit(P::KIND, &client_ip).await {
tracing::warn!(
+36 -3
View File
@@ -7,12 +7,13 @@ use cid::Cid;
use jacquard_repo::error::RepoError;
use jacquard_repo::repo::CommitData;
use jacquard_repo::storage::BlockStore;
use tranquil_store::blockstore::TranquilBlockStore;
use tranquil_store::blockstore::{RepairOutcome, TranquilBlockStore};
use tranquil_store::{RealIO, SystemClock};
#[derive(Clone)]
pub enum AnyBlockStore {
Postgres(PostgresBlockStore),
TranquilStore(TranquilBlockStore),
TranquilStore(TranquilBlockStore<RealIO, SystemClock>),
}
impl AnyBlockStore {
@@ -23,7 +24,7 @@ impl AnyBlockStore {
}
}
pub fn as_tranquil_store(&self) -> Option<&TranquilBlockStore> {
pub fn as_tranquil_store(&self) -> Option<&TranquilBlockStore<RealIO, SystemClock>> {
match self {
Self::TranquilStore(s) => Some(s),
Self::Postgres(_) => None,
@@ -36,6 +37,38 @@ impl AnyBlockStore {
Self::TranquilStore(s) => s.decrement_refs(cids).await,
}
}
pub async fn repair_structure(
&self,
entries: &[(String, Cid)],
expected_root: Cid,
) -> Result<RepairOutcome, RepoError> {
match self {
Self::Postgres(s) => {
let nodes =
tranquil_store::blockstore::rebuild_mst_nodes(entries, expected_root).await?;
let nodes_total = nodes.len();
let cids: Vec<Cid> = nodes.iter().map(|(cid, _)| *cid).collect();
let present = s.get_many(&cids).await?;
let missing: Vec<(Cid, Bytes)> = nodes
.into_iter()
.zip(present)
.filter_map(|((cid, bytes), found)| found.is_none().then_some((cid, bytes)))
.collect();
let nodes_repaired = missing.len() as u64;
if !missing.is_empty() {
s.put_many(missing).await?;
}
Ok(RepairOutcome {
nodes_total,
nodes_repaired,
})
}
Self::TranquilStore(s) => {
tranquil_store::blockstore::rebuild_and_repair_mst(s, entries, expected_root).await
}
}
}
}
impl BlockStore for AnyBlockStore {
+359 -10
View File
@@ -16,10 +16,10 @@ use k256::ecdsa::SigningKey;
use serde_json::{Value, json};
use std::collections::{BTreeMap, HashMap, HashSet};
use std::str::FromStr;
use std::sync::Arc;
use std::sync::{Arc, LazyLock};
use std::time::{Duration, Instant};
use tokio::sync::OwnedMutexGuard;
use tracing::{error, warn};
use tranquil_db_traits::SequenceNumber;
use uuid::Uuid;
#[derive(Debug)]
@@ -231,16 +231,251 @@ pub async fn begin_repo_write(
Ok((ctx, mst))
}
pub async fn repair_repo_structure(
state: &AppState,
user_id: Uuid,
) -> Result<tranquil_store::blockstore::RepairOutcome, ApiError> {
let _write_lock = state.repo_write_locks.lock(user_id).await;
let root_cid_str = state
.repos
.repo
.get_repo_root_cid_by_user_id(user_id)
.await
.map_err(|e| {
error!("repair: DB error fetching repo root: {}", e);
ApiError::InternalError(None)
})?
.ok_or_else(|| ApiError::InternalError(Some("Repo root not found".into())))?;
let current_root_cid = Cid::from_str(root_cid_str.as_str())
.map_err(|_| ApiError::InternalError(Some("Invalid repo root CID".into())))?;
let commit_bytes = state
.block_store
.get(&current_root_cid)
.await
.map_err(|e| {
error!("repair: failed to load commit block: {}", e);
ApiError::InternalError(None)
})?
.ok_or_else(|| ApiError::InternalError(Some("Commit block not found".into())))?;
let commit = Commit::from_cbor(&commit_bytes).map_err(|e| {
error!("repair: failed to parse commit: {}", e);
ApiError::InternalError(None)
})?;
let data_root = commit.data;
let repo_rev = commit.rev().to_string();
let records = state
.repos
.repo
.get_all_records(user_id)
.await
.map_err(|e| {
error!("repair: get_all_records failed: {}", e);
ApiError::InternalError(None)
})?;
let entries: Vec<(String, Cid)> = records
.into_iter()
.filter_map(|r| {
Cid::from_str(r.record_cid.as_str())
.ok()
.map(|cid| (format!("{}/{}", r.collection, r.rkey), cid))
})
.collect();
warn!(
user_id = %user_id,
records = entries.len(),
"repair: rebuilding full MST from record set"
);
let outcome = state
.block_store
.repair_structure(&entries, data_root)
.await
.map_err(|e| {
error!("repair: structural repair failed: {}", e);
ApiError::InternalError(Some("Structural repair failed".into()))
})?;
if outcome.nodes_repaired > 0 {
let block_cids =
crate::scheduled::collect_current_repo_blocks(&state.block_store, &current_root_cid)
.await
.map_err(|e| {
error!("repair: re-walk for user_blocks backfill failed: {}", e);
ApiError::InternalError(None)
})?;
let cids = block_cids
.iter()
.map(|bytes| Cid::try_from(bytes.as_slice()))
.collect::<Result<Vec<Cid>, _>>()
.map_err(|e| {
error!("repair: unparseable CID in repaired DAG walk: {e}");
ApiError::InternalError(None)
})?;
let present = state.block_store.get_many(&cids).await.map_err(|e| {
error!("repair: presence check during user_blocks backfill failed: {e}");
ApiError::InternalError(None)
})?;
let missing: Vec<Cid> = cids
.iter()
.zip(present)
.filter_map(|(cid, found)| found.is_none().then_some(*cid))
.collect();
if !missing.is_empty() {
error!(
user_id = %user_id,
missing = missing.len(),
sample = ?missing.iter().take(5).map(|c| c.to_string()).collect::<Vec<_>>(),
"repair: unrecoverable leaf data loss after structural repair"
);
return Err(ApiError::InternalError(Some(format!(
"unrecoverable leaf data loss: {} record block(s) missing after structural repair",
missing.len()
))));
}
state
.repos
.repo
.insert_user_blocks(user_id, &block_cids, &repo_rev)
.await
.map_err(|e| {
error!("repair: user_blocks backfill failed: {}", e);
ApiError::InternalError(None)
})?;
warn!(
user_id = %user_id,
blocks = block_cids.len(),
"repair: backfilled user_blocks from repaired DAG"
);
}
Ok(outcome)
}
pub async fn with_repair_retry<T, F, Fut>(
state: &AppState,
user_id: Uuid,
mut attempt: F,
) -> Result<T, ApiError>
where
F: FnMut() -> Fut,
Fut: std::future::Future<Output = Result<T, ApiError>>,
{
match attempt().await {
Err(e) if e.is_repo_corruption() => {
warn!(
"structural MST damage during repo write for user {user_id}, repairing and retrying"
);
match repair_repo_structure(state, user_id).await {
Ok(outcome) if outcome.nodes_repaired > 0 => attempt().await,
Ok(_) => {
warn!(
user_id = %user_id,
"structural repair rewrote no nodes; damage is not in the MST structure, returning original error without retry"
);
Err(e)
}
Err(repair_err) => {
error!(user_id = %user_id, "structural repair failed: {repair_err:?}");
Err(e)
}
}
}
other => other,
}
}
const REPAIR_COOLDOWN: Duration = Duration::from_secs(60);
const REPAIR_NOOP_COOLDOWN: Duration = Duration::from_secs(600);
struct RepairSlot {
in_flight: bool,
next_allowed: Instant,
}
struct RepairGuard {
slots: parking_lot::Mutex<HashMap<Uuid, RepairSlot>>,
}
impl RepairGuard {
fn try_claim(&self, user_id: Uuid, now: Instant) -> bool {
let mut slots = self.slots.lock();
let slot = slots.entry(user_id).or_insert(RepairSlot {
in_flight: false,
next_allowed: now,
});
if slot.in_flight || now < slot.next_allowed {
return false;
}
slot.in_flight = true;
true
}
fn release(&self, user_id: Uuid, now: Instant, cooldown: Duration) {
if let Some(slot) = self.slots.lock().get_mut(&user_id) {
slot.in_flight = false;
slot.next_allowed = now + cooldown;
}
}
}
static REPAIR_GUARD: LazyLock<RepairGuard> = LazyLock::new(|| RepairGuard {
slots: parking_lot::Mutex::new(HashMap::new()),
});
struct RepairLease {
user_id: Uuid,
cooldown: Duration,
}
impl Drop for RepairLease {
fn drop(&mut self) {
REPAIR_GUARD.release(self.user_id, Instant::now(), self.cooldown);
}
}
pub fn schedule_repo_repair(state: &AppState, user_id: Uuid) {
if !REPAIR_GUARD.try_claim(user_id, Instant::now()) {
return;
}
let state = state.clone();
tokio::spawn(async move {
let mut lease = RepairLease {
user_id,
cooldown: REPAIR_COOLDOWN,
};
match repair_repo_structure(&state, user_id).await {
Ok(outcome) => {
if outcome.nodes_repaired == 0 {
lease.cooldown = REPAIR_NOOP_COOLDOWN;
}
warn!(
user_id = %user_id,
nodes_repaired = outcome.nodes_repaired,
nodes_total = outcome.nodes_total,
"background MST repair complete"
);
}
Err(e) => error!(user_id = %user_id, "background MST repair failed: {e:?}"),
}
});
}
pub async fn finalize_repo_write(
state: &AppState,
ctx: RepoWriteContext,
mst: Mst<TrackingBlockStore>,
params: FinalizeParams<'_>,
) -> Result<CommitResult, ApiError> {
let new_mst_root = mst.persist().await.map_err(|e| {
error!("MST persist failed: {}", e);
ApiError::InternalError(None)
})?;
let new_mst_root = mst
.persist()
.await
.map_err(|e| ApiError::from_mst_error("MST persist", &e))?;
let written_bytes = ctx.tracking_store.take_written_blocks();
let new_tree_cids: Vec<Cid> = written_bytes.keys().copied().collect();
@@ -776,7 +1011,7 @@ pub async fn sequence_identity_event(
state: &AppState,
did: &Did,
handle: Option<&Handle>,
) -> Result<SequenceNumber, CommitError> {
) -> Result<(), CommitError> {
state
.repos
.repo
@@ -788,7 +1023,7 @@ pub async fn sequence_account_event(
state: &AppState,
did: &Did,
status: tranquil_db_traits::AccountStatus,
) -> Result<SequenceNumber, CommitError> {
) -> Result<(), CommitError> {
state
.repos
.repo
@@ -801,7 +1036,7 @@ pub async fn sequence_sync_event(
did: &Did,
commit_cid: &str,
rev: Option<&str>,
) -> Result<SequenceNumber, CommitError> {
) -> Result<(), CommitError> {
let cid_link: crate::types::CidLink = commit_cid
.parse()
.map_err(|_| CommitError::InvalidCid(commit_cid.to_string()))?;
@@ -829,7 +1064,7 @@ pub async fn sequence_genesis_commit(
commit_cid: &Cid,
mst_root_cid: &Cid,
rev: &str,
) -> Result<SequenceNumber, CommitError> {
) -> Result<(), CommitError> {
let commit_cid_link = crate::types::CidLink::from(commit_cid);
let mst_root_cid_link = crate::types::CidLink::from(mst_root_cid);
let commit_bytes = state
@@ -862,3 +1097,117 @@ pub async fn sequence_genesis_commit(
.await
.map_err(|e| CommitError::DatabaseError(format!("genesis commit event: {}", e)))
}
#[cfg(test)]
mod repair_guard_tests {
use super::*;
fn guard() -> RepairGuard {
RepairGuard {
slots: parking_lot::Mutex::new(HashMap::new()),
}
}
#[test]
fn claim_dedups_in_flight_then_respects_cooldown() {
let g = guard();
let user = Uuid::from_u128(1);
let t0 = Instant::now();
assert!(g.try_claim(user, t0), "first claim must succeed");
assert!(
!g.try_claim(user, t0),
"second claim while a repair is in flight must be rejected"
);
g.release(user, t0, REPAIR_COOLDOWN);
assert!(
!g.try_claim(user, t0 + Duration::from_secs(1)),
"claim within the cooldown window must be rejected"
);
assert!(
g.try_claim(user, t0 + REPAIR_COOLDOWN + Duration::from_millis(1)),
"claim after the cooldown window must succeed"
);
}
#[test]
fn distinct_users_do_not_block_each_other() {
let g = guard();
let t0 = Instant::now();
assert!(g.try_claim(Uuid::from_u128(1), t0));
assert!(g.try_claim(Uuid::from_u128(2), t0));
}
#[test]
fn concurrent_claims_for_one_user_admit_exactly_one() {
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, Barrier};
let g = Arc::new(guard());
let user = Uuid::from_u128(42);
let now = Instant::now();
let winners = Arc::new(AtomicUsize::new(0));
let gate = Arc::new(Barrier::new(32));
let handles: Vec<_> = (0..32)
.map(|_| {
let g = Arc::clone(&g);
let winners = Arc::clone(&winners);
let gate = Arc::clone(&gate);
std::thread::spawn(move || {
gate.wait();
if g.try_claim(user, now) {
winners.fetch_add(1, Ordering::Relaxed);
}
})
})
.collect();
handles
.into_iter()
.for_each(|h| h.join().expect("worker thread panicked"));
assert_eq!(
winners.load(Ordering::Relaxed),
1,
"exactly one concurrent claim must win the dedup race"
);
}
#[test]
fn release_re_enables_claim_after_cooldown_for_recurring_corruption() {
let g = guard();
let user = Uuid::from_u128(7);
let t0 = Instant::now();
assert!(g.try_claim(user, t0));
g.release(user, t0, REPAIR_COOLDOWN);
let after_cooldown = t0 + REPAIR_COOLDOWN + Duration::from_millis(1);
assert!(
g.try_claim(user, after_cooldown),
"a fresh corruption after the cooldown must be repairable again"
);
assert!(
!g.try_claim(user, after_cooldown),
"the re-claimed repair must again dedup while in flight"
);
}
#[test]
fn noop_repair_uses_longer_cooldown() {
let g = guard();
let user = Uuid::from_u128(9);
let t0 = Instant::now();
assert!(g.try_claim(user, t0));
g.release(user, t0, REPAIR_NOOP_COOLDOWN);
assert!(
!g.try_claim(user, t0 + REPAIR_COOLDOWN + Duration::from_millis(1)),
"after a no-op repair the standard cooldown must not re-admit a claim"
);
assert!(
g.try_claim(user, t0 + REPAIR_NOOP_COOLDOWN + Duration::from_millis(1)),
"after the longer no-op cooldown a fresh claim must be admitted"
);
}
}
+59 -16
View File
@@ -574,7 +574,10 @@ impl CompactionBlocklist {
}
fn run_compaction_pass(
store: &tranquil_store::blockstore::TranquilBlockStore,
store: &tranquil_store::blockstore::TranquilBlockStore<
tranquil_store::RealIO,
tranquil_store::SystemClock,
>,
liveness_threshold: f64,
grace_period_ms: u64,
blocklist: &parking_lot::Mutex<CompactionBlocklist>,
@@ -711,7 +714,7 @@ async fn delete_account_data(
warn!(storage_key = %key, error = %e, "Failed to delete blob from storage (continuing anyway)");
});
let _account_seq = user_repo
user_repo
.delete_account_with_firehose(user_id, did)
.await
.context("Failed to delete account")?;
@@ -727,10 +730,53 @@ async fn delete_account_data(
const CAR_BLOCK_BATCH_SIZE: usize = 500;
#[derive(Debug)]
pub enum RepoCarError {
MissingBlocks(Vec<Cid>),
Source(anyhow::Error),
}
impl RepoCarError {
pub fn is_repairable(&self) -> bool {
match self {
Self::MissingBlocks(_) => true,
Self::Source(e) => {
crate::api::error::ApiError::detail_is_repo_corruption(&format!("{e:#}"))
}
}
}
}
impl std::fmt::Display for RepoCarError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::MissingBlocks(cids) => write!(
f,
"repo CAR is incomplete: {} block(s) referenced by the MST are missing from storage. First 5: {}",
cids.len(),
cids.iter()
.take(5)
.map(|c| c.to_string())
.collect::<Vec<_>>()
.join(", ")
),
Self::Source(e) => write!(f, "{e:#}"),
}
}
}
impl std::error::Error for RepoCarError {}
impl From<anyhow::Error> for RepoCarError {
fn from(e: anyhow::Error) -> Self {
Self::Source(e)
}
}
pub async fn generate_repo_car(
block_store: &AnyBlockStore,
head_cid: &Cid,
) -> anyhow::Result<Vec<u8>> {
) -> Result<Vec<u8>, RepoCarError> {
let block_cids_bytes = collect_current_repo_blocks(block_store, head_cid).await?;
let block_cids: Vec<Cid> = block_cids_bytes
.iter()
@@ -757,16 +803,7 @@ pub async fn generate_repo_car(
.filter_map(|(cid, block_opt)| block_opt.is_none().then_some(*cid))
.collect();
if !missing.is_empty() {
anyhow::bail!(
"repo CAR is incomplete: {} block(s) referenced by the MST are missing from storage. First 5: {}",
missing.len(),
missing
.iter()
.take(5)
.map(|c| c.to_string())
.collect::<Vec<_>>()
.join(", ")
);
return Err(RepoCarError::MissingBlocks(missing));
}
chunk
@@ -800,7 +837,7 @@ pub async fn generate_repo_car_from_user_blocks(
block_store: &AnyBlockStore,
user_id: uuid::Uuid,
_head_cid: &Cid,
) -> anyhow::Result<Vec<u8>> {
) -> Result<Vec<u8>, RepoCarError> {
use std::str::FromStr;
let repo_root_cid_str: String = repo_repo
@@ -836,7 +873,10 @@ fn cid_to_bytes(cid: &Cid) -> anyhow::Result<CidBytes> {
}
fn walk_repo_dag_sync(
store: &tranquil_store::blockstore::TranquilBlockStore,
store: &tranquil_store::blockstore::TranquilBlockStore<
tranquil_store::RealIO,
tranquil_store::SystemClock,
>,
head_cid: &Cid,
reachable: &mut std::collections::HashSet<CidBytes>,
phantom_files: &mut std::collections::HashSet<tranquil_store::blockstore::DataFileId>,
@@ -960,7 +1000,10 @@ fn paginate_repos(
}
pub fn run_reachability_walk(
store: &tranquil_store::blockstore::TranquilBlockStore,
store: &tranquil_store::blockstore::TranquilBlockStore<
tranquil_store::RealIO,
tranquil_store::SystemClock,
>,
repo_repo: &dyn RepoRepository,
) -> anyhow::Result<ReachabilityResult> {
let rt = tokio::runtime::Handle::current();
+30 -23
View File
@@ -214,10 +214,10 @@ impl AppState {
pub async fn new(shutdown: CancellationToken) -> Result<Self, Box<dyn Error>> {
let cfg = tranquil_config::get();
match cfg.storage.repo_backend() {
let mut state = match cfg.storage.repo_backend() {
tranquil_config::RepoBackend::TranquilStore => {
tracing::info!("tranquil-store repo backend active. EXPERIMENTAL!");
Ok(Self::from_store(shutdown).await)
Self::from_store(shutdown).await
}
tranquil_config::RepoBackend::Postgres => {
let database_url = &cfg.database.url;
@@ -247,28 +247,21 @@ impl AppState {
.await
.map_err(|e| format!("Failed to run migrations: {}", e))?;
let bootstrap_invite_code = match (
cfg.server.invite_code_required,
sqlx::query_scalar!("SELECT COUNT(*) FROM users")
.fetch_one(&db)
.await,
) {
(true, Ok(Some(0))) => {
let code = crate::util::gen_invite_code();
tracing::info!(
"No users exist and invite codes are required. Bootstrap invite code: {}",
code
);
Some(code)
}
_ => None,
};
let mut state = Self::from_db(db, shutdown).await;
state.bootstrap_invite_code = bootstrap_invite_code;
Ok(state)
Self::from_db(db, shutdown).await
}
};
if cfg.server.invite_code_required && state.repos.user.count_users().await.unwrap_or(1) == 0
{
let code = crate::util::gen_invite_code();
tracing::info!(
"No users exist and invite codes are required. Bootstrap invite code: {}",
code
);
state.bootstrap_invite_code = Some(code);
}
Ok(state)
}
pub async fn from_db(db: PgPool, shutdown: CancellationToken) -> Self {
@@ -478,7 +471,10 @@ impl AppState {
}
struct TranquilStoreWiring {
blockstore: tranquil_store::blockstore::TranquilBlockStore,
blockstore: tranquil_store::blockstore::TranquilBlockStore<
tranquil_store::RealIO,
tranquil_store::SystemClock,
>,
signal_provider: Arc<dyn tranquil_signal::SignalStoreProvider>,
repos: PostgresRepositories,
segments_dir: PathBuf,
@@ -610,6 +606,17 @@ fn wire_tranquil_store(
}
}
if std::env::var("TRANQUIL_PURGE_ORPHAN_REPOS").is_ok_and(|v| v == "1") {
match metastore
.repo_ops()
.purge_orphan_repos(metastore.database())
{
Ok(0) => tracing::info!("orphan repo purge: no orphans found"),
Ok(n) => tracing::info!(purged = n, "orphan repo purge: removed orphan repo_meta"),
Err(e) => tracing::error!(error = %e, "orphan repo purge failed"),
}
}
let notifier = bridge.notifier();
let signal_db = metastore.database().clone();
let signal_ks = metastore.signal_keyspace();
+13 -15
View File
@@ -92,30 +92,20 @@ impl CarVerifier {
pub fn verify_car_structure_only(
&self,
expected_did: &Did,
root_cid: &Cid,
blocks: &HashMap<Cid, Bytes>,
) -> Result<VerifiedCar, VerifyError> {
) -> Result<StructureVerifiedCar, VerifyError> {
let root_block = blocks
.get(root_cid)
.ok_or_else(|| VerifyError::BlockNotFound(root_cid.to_string()))?;
let commit =
Commit::from_cbor(root_block).map_err(|e| VerifyError::InvalidCommit(e.to_string()))?;
let commit_did = commit.did().as_str();
if commit_did != expected_did.as_str() {
return Err(VerifyError::DidMismatch {
commit_did: commit_did.to_string(),
expected_did: expected_did.to_string(),
});
}
let commit_did = commit.did().to_string().into();
let data_cid = commit.data();
self.verify_mst_structure(data_cid, blocks)?;
debug!(
"MST structure verified for DID {} (signature verification skipped for migration)",
expected_did
);
Ok(VerifiedCar {
did: expected_did.clone(),
debug!("MST structure verified for commit: {:?}", commit);
Ok(StructureVerifiedCar {
did: commit_did,
rev: commit.rev().to_string(),
data_cid: *data_cid,
prev: commit.prev().cloned(),
@@ -289,6 +279,14 @@ impl CarVerifier {
}
}
#[derive(Debug, Clone)]
pub struct StructureVerifiedCar {
pub did: Did,
pub rev: String,
pub data_cid: Cid,
pub prev: Option<Cid>,
}
#[derive(Debug, Clone)]
pub struct VerifiedCar {
pub did: Did,
+261 -7
View File
@@ -7,6 +7,7 @@ use rand::Rng;
use serde_json::Value as JsonValue;
use std::collections::BTreeMap;
use std::net::SocketAddr;
use std::num::NonZeroUsize;
use std::str::FromStr;
use std::sync::OnceLock;
@@ -32,7 +33,20 @@ pub fn generate_token_code() -> String {
.map(|_| chars[rng.gen_range(0..chars.len())])
.collect()
};
format!("{}-{}", gen_segment(&mut rng), gen_segment(&mut rng))
// Human-entered short codes are displayed in uppercase; base32 digits are
// unaffected by the conversion.
format!("{}-{}", gen_segment(&mut rng), gen_segment(&mut rng)).to_uppercase()
}
/// Normalize a user-entered short code so that codes are accepted
/// case-insensitively and regardless of the separating hyphen or surrounding
/// whitespace.
pub fn normalize_token_code(input: &str) -> String {
input
.chars()
.filter(|c| !c.is_whitespace() && *c != '-')
.collect::<String>()
.to_uppercase()
}
pub fn parse_repeated_query_param(query: Option<&str>, key: &str) -> Vec<String> {
@@ -96,22 +110,99 @@ pub fn generate_random_token() -> String {
URL_SAFE_NO_PAD.encode(bytes)
}
pub fn extract_client_ip(headers: &HeaderMap, addr: Option<SocketAddr>) -> String {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum ForwardedTrust {
Peer,
Proxies(NonZeroUsize),
}
fn resolve_trust(configured: Option<usize>, terminates_tls: bool) -> ForwardedTrust {
let count = configured.unwrap_or(if terminates_tls { 0 } else { 1 });
match NonZeroUsize::new(count) {
Some(proxies) => ForwardedTrust::Proxies(proxies),
None => ForwardedTrust::Peer,
}
}
pub(crate) fn forwarded_trust() -> ForwardedTrust {
match tranquil_config::try_get() {
Some(cfg) => resolve_trust(
cfg.server.trusted_proxy_count,
cfg.server.tls.material().is_some(),
),
None => ForwardedTrust::Peer,
}
}
fn forwarded_client_ip(headers: &HeaderMap, trusted: NonZeroUsize) -> Option<String> {
if let Some(forwarded) = headers.get("x-forwarded-for")
&& let Ok(value) = forwarded.to_str()
&& let Some(first_ip) = value.split(',').next()
{
return first_ip.trim().to_string();
let hops: Vec<&str> = value
.split(',')
.map(str::trim)
.filter(|s| !s.is_empty())
.collect();
if let Some(client) = hops
.len()
.checked_sub(trusted.get())
.and_then(|idx| hops.get(idx))
{
return Some((*client).to_string());
}
}
if let Some(real_ip) = headers.get("x-real-ip")
if trusted.get() == 1
&& let Some(real_ip) = headers.get("x-real-ip")
&& let Ok(value) = real_ip.to_str()
&& !value.trim().is_empty()
{
return value.trim().to_string();
return Some(value.trim().to_string());
}
None
}
pub(crate) fn extract_client_ip(
headers: &HeaderMap,
addr: Option<SocketAddr>,
trust: ForwardedTrust,
) -> String {
if let ForwardedTrust::Proxies(trusted) = trust
&& let Some(client) = forwarded_client_ip(headers, trusted)
{
return client;
}
addr.map(|a| a.ip().to_string())
.unwrap_or_else(|| "unknown".to_string())
}
pub(crate) fn client_ip_from_parts(parts: &axum::http::request::Parts) -> String {
let addr = parts
.extensions
.get::<axum::extract::ConnectInfo<SocketAddr>>()
.map(|connect_info| connect_info.0);
extract_client_ip(&parts.headers, addr, forwarded_trust())
}
#[derive(Debug, Clone)]
pub struct ClientIp(String);
impl ClientIp {
pub fn into_string(self) -> String {
self.0
}
}
impl<S: Send + Sync> axum::extract::FromRequestParts<S> for ClientIp {
type Rejection = std::convert::Infallible;
async fn from_request_parts(
parts: &mut axum::http::request::Parts,
_state: &S,
) -> Result<Self, Self::Rejection> {
Ok(ClientIp(client_ip_from_parts(parts)))
}
}
pub fn set_discord_bot_username(username: String) {
DISCORD_BOT_USERNAME.set(username).ok();
}
@@ -227,6 +318,135 @@ pub fn is_self_hosted_did_web_enabled() -> bool {
#[cfg(test)]
mod tests {
use super::*;
use axum::extract::{ConnectInfo, FromRequestParts};
fn proxies(count: usize) -> ForwardedTrust {
ForwardedTrust::Proxies(NonZeroUsize::new(count).unwrap())
}
#[test]
fn resolve_trust_override_wins_over_tls() {
assert_eq!(resolve_trust(Some(1), true), proxies(1));
assert_eq!(resolve_trust(Some(3), false), proxies(3));
assert_eq!(resolve_trust(Some(0), true), ForwardedTrust::Peer);
assert_eq!(resolve_trust(Some(0), false), ForwardedTrust::Peer);
}
#[test]
fn resolve_trust_infers_from_tls_when_unset() {
assert_eq!(resolve_trust(None, true), ForwardedTrust::Peer);
assert_eq!(resolve_trust(None, false), proxies(1));
}
fn parts_with(
header: Option<(&str, &str)>,
peer: Option<SocketAddr>,
) -> axum::http::request::Parts {
let mut builder = axum::http::Request::builder();
if let Some((name, value)) = header {
builder = builder.header(name, value);
}
let mut parts = builder.body(()).unwrap().into_parts().0;
if let Some(addr) = peer {
parts.extensions.insert(ConnectInfo(addr));
}
parts
}
#[tokio::test]
async fn client_ip_falls_back_to_peer_socket() {
let peer: SocketAddr = "203.0.113.7:51000".parse().unwrap();
let mut parts = parts_with(None, Some(peer));
let ip = ClientIp::from_request_parts(&mut parts, &()).await.unwrap();
assert_eq!(ip.into_string(), "203.0.113.7");
}
#[tokio::test]
async fn client_ip_ignores_forwarded_when_config_absent() {
let peer: SocketAddr = "203.0.113.7:51000".parse().unwrap();
let mut parts = parts_with(
Some(("x-forwarded-for", "198.51.100.4, 10.0.0.1")),
Some(peer),
);
let ip = ClientIp::from_request_parts(&mut parts, &()).await.unwrap();
assert_eq!(ip.into_string(), "203.0.113.7");
}
#[tokio::test]
async fn client_ip_unknown_without_headers_or_peer() {
let mut parts = parts_with(None, None);
let ip = ClientIp::from_request_parts(&mut parts, &()).await.unwrap();
assert_eq!(ip.into_string(), "unknown");
}
#[tokio::test]
async fn client_ip_renders_ipv6_peer_without_brackets() {
let peer: SocketAddr = "[2001:db8::beef]:51000".parse().unwrap();
let mut parts = parts_with(None, Some(peer));
let ip = ClientIp::from_request_parts(&mut parts, &()).await.unwrap();
assert_eq!(ip.into_string(), "2001:db8::beef");
}
#[test]
fn extract_client_ip_single_proxy_takes_rightmost_forwarded_hop() {
let mut headers = HeaderMap::new();
headers.insert(
"x-forwarded-for",
"9.9.9.9, 198.51.100.4, 10.0.0.1".parse().unwrap(),
);
let peer: SocketAddr = "203.0.113.7:51000".parse().unwrap();
assert_eq!(
extract_client_ip(&headers, Some(peer), proxies(1)),
"10.0.0.1"
);
}
#[test]
fn extract_client_ip_two_proxies_skips_inner_hop() {
let mut headers = HeaderMap::new();
headers.insert(
"x-forwarded-for",
"9.9.9.9, 198.51.100.4, 10.0.0.1".parse().unwrap(),
);
let peer: SocketAddr = "203.0.113.7:51000".parse().unwrap();
assert_eq!(
extract_client_ip(&headers, Some(peer), proxies(2)),
"198.51.100.4"
);
}
#[test]
fn extract_client_ip_more_trusted_proxies_than_hops_uses_peer() {
let mut headers = HeaderMap::new();
headers.insert("x-forwarded-for", "10.0.0.1".parse().unwrap());
let peer: SocketAddr = "203.0.113.7:51000".parse().unwrap();
assert_eq!(
extract_client_ip(&headers, Some(peer), proxies(2)),
"203.0.113.7"
);
}
#[test]
fn extract_client_ip_ignores_forwarded_headers_for_direct_peer() {
let mut headers = HeaderMap::new();
headers.insert("x-forwarded-for", "9.9.9.9".parse().unwrap());
headers.insert("x-real-ip", "9.9.9.9".parse().unwrap());
let peer: SocketAddr = "203.0.113.7:51000".parse().unwrap();
assert_eq!(
extract_client_ip(&headers, Some(peer), ForwardedTrust::Peer),
"203.0.113.7"
);
}
#[test]
fn extract_client_ip_direct_peer_without_socket_is_unknown() {
let mut headers = HeaderMap::new();
headers.insert("x-forwarded-for", "9.9.9.9".parse().unwrap());
assert_eq!(
extract_client_ip(&headers, None, ForwardedTrust::Peer),
"unknown"
);
}
#[test]
fn test_parse_repeated_query_param_repeated() {
@@ -284,10 +504,44 @@ mod tests {
assert!(
code.chars()
.filter(|&c| c != '-')
.all(|c| BASE32_ALPHABET.contains(c))
.all(|c| BASE32_ALPHABET.to_uppercase().contains(c))
);
}
#[test]
fn test_generate_token_code_is_uppercase() {
(0..100).for_each(|_| {
let code = generate_token_code();
assert_eq!(code, code.to_uppercase(), "code must be uppercase: {code}");
});
}
#[test]
fn test_normalize_token_code_strips_hyphen_and_uppercases() {
assert_eq!(normalize_token_code("k7m2p-q9rst"), "K7M2PQ9RST");
assert_eq!(normalize_token_code("K7M2P-Q9RST"), "K7M2PQ9RST");
}
#[test]
fn test_normalize_token_code_strips_whitespace() {
assert_eq!(normalize_token_code(" k7m2p-q9rst \n"), "K7M2PQ9RST");
}
#[test]
fn test_normalize_token_code_is_idempotent() {
let once = normalize_token_code("k7m2p-q9rst");
assert_eq!(normalize_token_code(&once), once);
}
#[test]
fn test_generated_code_round_trips_through_normalize() {
let code = generate_token_code();
// A user re-typing the displayed code lowercased and without the hyphen
// must normalize to the same canonical form as the code itself.
let retyped = code.to_lowercase().replace('-', "");
assert_eq!(normalize_token_code(&code), normalize_token_code(&retyped));
}
#[test]
fn test_json_to_ipld_cid_link() {
let json = serde_json::json!({
@@ -0,0 +1,98 @@
use std::sync::Arc;
use cid::Cid;
use jacquard_repo::mst::Mst;
use jacquard_repo::storage::BlockStore;
use tranquil_pds::repo::AnyBlockStore;
use tranquil_pds::scheduled::{RepoCarError, generate_repo_car};
use tranquil_store::blockstore::{BlockStoreConfig, GroupCommitConfig, TranquilBlockStore};
const RECORD_COUNT: usize = 200;
fn open_store(dir: &std::path::Path) -> AnyBlockStore {
let cfg = BlockStoreConfig {
data_dir: dir.join("data"),
index_dir: dir.join("index"),
max_file_size: 64 * 1024,
group_commit: GroupCommitConfig::default(),
shard_count: 1,
};
AnyBlockStore::TranquilStore(TranquilBlockStore::open(cfg).expect("open block store"))
}
async fn build_tree(any: &AnyBlockStore) -> Cid {
let mut mst = Mst::new(Arc::new(any.clone()));
for i in 0..RECORD_COUNT {
let key = format!("app.bsky.feed.post/{i:0>6}");
let cid = any
.put(format!("record body {i}").as_bytes())
.await
.expect("put record");
mst.add_mut(&key, cid).await.expect("mst add");
}
mst.persist().await.expect("persist mst")
}
fn shred_data_files(data_dir: &std::path::Path) {
let mut shredded = false;
for entry in std::fs::read_dir(data_dir).expect("read data dir") {
let path = entry.expect("dir entry").path();
if path.extension().and_then(|e| e.to_str()) != Some("tqb") {
continue;
}
let mut bytes = std::fs::read(&path).expect("read data file");
let mut off = 5usize;
while off + 48 < bytes.len() {
bytes[off..off + 48].iter_mut().for_each(|b| *b = 0xFF);
off += 192;
shredded = true;
}
std::fs::write(&path, &bytes).expect("write corrupted data file");
}
assert!(shredded, "no .tqb data file was corrupted");
}
#[tokio::test]
async fn car_export_error_is_classified_as_repo_corruption() {
let dir = tempfile::tempdir().expect("tempdir");
let any = open_store(dir.path());
let root = build_tree(&any).await;
generate_repo_car(&any, &root)
.await
.expect("pristine CAR must generate");
shred_data_files(&dir.path().join("data"));
let err = generate_repo_car(&any, &root)
.await
.expect_err("corrupt CAR export must error");
assert!(
err.is_repairable(),
"{err} should classify as repairable so the sync path can self-heal"
);
}
#[tokio::test]
async fn car_export_missing_block_is_repairable() {
let dir = tempfile::tempdir().expect("tempdir");
let source = open_store(dir.path());
let root = build_tree(&source).await;
let pristine = open_store(&dir.path().join("pristine"));
let head_block = source
.get(&root)
.await
.expect("read root")
.expect("root present");
pristine.put(&head_block).await.expect("seed root only");
let err = generate_repo_car(&pristine, &root)
.await
.expect_err("CAR export over a store missing MST children must error");
assert!(
matches!(err, RepoCarError::MissingBlocks(ref cids) if !cids.is_empty()),
"{err} should surface as MissingBlocks when referenced blocks are absent"
);
assert!(err.is_repairable());
}
+50
View File
@@ -30,6 +30,7 @@ static TEST_TEMP_DIR: OnceLock<PathBuf> = OnceLock::new();
static CLUSTER: OnceLock<Vec<ServerInstance>> = OnceLock::new();
static TEST_REPOS: OnceLock<Arc<tranquil_db::PostgresRepositories>> = OnceLock::new();
static TEST_BLOCK_STORE: OnceLock<tranquil_pds::repo::AnyBlockStore> = OnceLock::new();
static TEST_APP_STATE: OnceLock<AppState> = OnceLock::new();
#[allow(dead_code)]
pub fn is_store_backend() -> bool {
@@ -130,6 +131,13 @@ pub fn pds_endpoint() -> String {
format!("https://{}", pds_hostname())
}
#[allow(dead_code)]
pub fn store_data_dir() -> Option<PathBuf> {
std::env::var("TRANQUIL_STORE_DATA_DIR")
.ok()
.map(PathBuf::from)
}
pub async fn base_url() -> &'static str {
SERVER_URL.get_or_init(|| {
let (tx, rx) = std::sync::mpsc::channel();
@@ -579,6 +587,7 @@ async fn spawn_server(config: ServerConfig) -> ServerInstance {
if let Some((cache, distributed_rate_limiter)) = config.cache {
state = state.with_cache(cache, distributed_rate_limiter);
}
TEST_APP_STATE.set(state.clone()).ok();
tranquil_sync::listener::start_sequencer_listener(state.clone()).await;
let app = tranquil_pds::app_with_routes(
state,
@@ -921,6 +930,47 @@ pub async fn get_test_block_store() -> &'static tranquil_pds::repo::AnyBlockStor
.expect("TEST_BLOCK_STORE not initialized")
}
#[allow(dead_code)]
pub async fn get_test_app_state() -> &'static AppState {
base_url().await;
TEST_APP_STATE
.get()
.expect("TEST_APP_STATE not initialized")
}
#[allow(dead_code)]
pub async fn flushed_max_seq(
repos: &tranquil_db::PostgresRepositories,
) -> tranquil_db_traits::SequenceNumber {
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
repos.repo.get_max_seq().await.expect("get_max_seq")
}
#[allow(dead_code)]
pub async fn sequenced_event_for_did(
repos: &tranquil_db::PostgresRepositories,
baseline: tranquil_db_traits::SequenceNumber,
did: &tranquil_types::Did,
) -> tranquil_db_traits::SequencedEvent {
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
repos
.repo
.get_events_since_seq(baseline, None)
.await
.expect("get_events_since_seq")
.into_iter()
.rfind(|event| &event.did == did)
.unwrap_or_else(|| panic!("event for did {did} not found after flush"))
}
fn extract_verification_code(body_text: &str) -> String {
let lines: Vec<&str> = body_text.lines().collect();
lines
+185
View File
@@ -0,0 +1,185 @@
mod common;
use base64::Engine;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
use std::sync::{Arc, Mutex};
use wiremock::matchers::{method, path};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
fn decode_jwt_claims(jwt: &str) -> Value {
let payload = jwt
.split('.')
.nth(1)
.expect("malformed jwt: no claims segment");
let bytes = URL_SAFE_NO_PAD
.decode(payload)
.expect("malformed jwt: claims not base64url");
serde_json::from_slice(&bytes).expect("malformed jwt: claims not json")
}
struct CaptureAuth(Arc<Mutex<Option<String>>>);
impl Respond for CaptureAuth {
fn respond(&self, req: &Request) -> ResponseTemplate {
let auth = req
.headers
.get("authorization")
.and_then(|v| v.to_str().ok())
.map(str::to_string);
*self.0.lock().unwrap() = auth;
ResponseTemplate::new(200).set_body_json(json!({ "feed": [] }))
}
}
/// getFeed's service-auth token must be audienced to the feed generator, not the AppView.
#[tokio::test]
async fn get_feed_service_auth_is_audienced_to_feed_generator() {
let client = client();
let (token, _did) = create_account_and_login(&client).await;
// One mock server doubles as the AppView: did:web doc, getRecord, and getFeed.
let appview = MockServer::start().await;
let appview_uri = appview.uri();
let host = appview_uri
.strip_prefix("http://")
.expect("mock uri should be http");
// Literal-colon host so did:web resolves over http to the local mock.
let appview_did = format!("did:web:{host}");
Mock::given(method("GET"))
.and(path("/.well-known/did.json"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"id": appview_did,
"service": [{
"id": "#bsky_appview",
"type": "BskyAppView",
"serviceEndpoint": appview_uri,
}]
})))
.mount(&appview)
.await;
let feed_did = "did:web:feedgen.example.com";
let feed_uri = "at://did:plc:feedcreator00000000000000/app.bsky.feed.generator/myfeed";
// The feed generator record resolves to its service DID.
Mock::given(method("GET"))
.and(path("/xrpc/com.atproto.repo.getRecord"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"uri": feed_uri,
"value": { "$type": "app.bsky.feed.generator", "did": feed_did },
})))
.mount(&appview)
.await;
// Capture the Authorization header the AppView is handed for getFeed.
let captured: Arc<Mutex<Option<String>>> = Arc::new(Mutex::new(None));
Mock::given(method("GET"))
.and(path("/xrpc/app.bsky.feed.getFeed"))
.respond_with(CaptureAuth(captured.clone()))
.mount(&appview)
.await;
// Send the params a real client sends, so feed extraction must ignore extras.
let res = client
.get(format!("{}/xrpc/app.bsky.feed.getFeed", base_url().await))
.query(&[("feed", feed_uri), ("limit", "30"), ("cursor", "abc123")])
.header("authorization", format!("Bearer {}", token))
.header("atproto-proxy", format!("{}#bsky_appview", appview_did))
.send()
.await
.expect("getFeed proxy request failed");
assert_eq!(
res.status(),
StatusCode::OK,
"getFeed proxy should succeed: {:?}",
res.text().await
);
let auth = captured
.lock()
.unwrap()
.clone()
.expect("AppView received no Authorization header");
let jwt = auth
.strip_prefix("Bearer ")
.expect("forwarded auth should be a bearer token");
let claims = decode_jwt_claims(jwt);
assert_eq!(
claims["aud"].as_str(),
Some(feed_did),
"service-auth token must be audienced to the feed generator, got {:?}",
claims["aud"]
);
assert_eq!(
claims["lxm"].as_str(),
Some("app.bsky.feed.getFeedSkeleton"),
"service-auth token lxm must be getFeedSkeleton, got {:?}",
claims["lxm"]
);
}
/// An unresolvable feed generator must be refused, not forwarded with an AppView aud.
#[tokio::test]
async fn get_feed_refuses_when_feed_generator_unresolvable() {
let client = client();
let (token, _did) = create_account_and_login(&client).await;
let appview = MockServer::start().await;
let appview_uri = appview.uri();
let host = appview_uri
.strip_prefix("http://")
.expect("mock uri should be http");
let appview_did = format!("did:web:{host}");
Mock::given(method("GET"))
.and(path("/.well-known/did.json"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"id": appview_did,
"service": [{
"id": "#bsky_appview",
"type": "BskyAppView",
"serviceEndpoint": appview_uri,
}]
})))
.mount(&appview)
.await;
// getRecord fails, so the feed generator DID can't be resolved.
Mock::given(method("GET"))
.and(path("/xrpc/com.atproto.repo.getRecord"))
.respond_with(ResponseTemplate::new(404).set_body_json(json!({
"error": "RecordNotFound",
})))
.mount(&appview)
.await;
// getFeed must never be reached with an AppView-audienced token.
Mock::given(method("GET"))
.and(path("/xrpc/app.bsky.feed.getFeed"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({ "feed": [] })))
.expect(0)
.mount(&appview)
.await;
let feed_uri = "at://did:plc:feedcreator00000000000000/app.bsky.feed.generator/myfeed";
let res = client
.get(format!("{}/xrpc/app.bsky.feed.getFeed", base_url().await))
.query(&[("feed", feed_uri)])
.header("authorization", format!("Bearer {token}"))
.header("atproto-proxy", format!("{appview_did}#bsky_appview"))
.send()
.await
.expect("getFeed proxy request failed");
assert_eq!(
res.status(),
StatusCode::BAD_REQUEST,
"unresolvable feed should be rejected, got {}",
res.status()
);
}
@@ -66,6 +66,11 @@ async fn commit_events_carry_inline_blocks() {
let repos = get_test_repos().await;
let typed_did = tranquil_types::Did::new(did.clone()).unwrap();
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
let events = repos
.repo
.get_events_since_seq(SequenceNumber::ZERO, None)
@@ -142,18 +147,14 @@ async fn sync_event_carries_inline_commit_block() {
let cid_link: CidLink = (&commit_cid).into();
let rev = "3kabcdefghij2";
let seq = repos
let baseline = repos.repo.get_max_seq().await.expect("get_max_seq");
repos
.repo
.insert_sync_event(&did, &cid_link, Some(rev), &commit_bytes)
.await
.expect("insert_sync_event");
let event = repos
.repo
.get_event_by_seq(seq)
.await
.expect("get_event_by_seq")
.expect("event present");
let event = sequenced_event_for_did(repos, baseline, &did).await;
assert_eq!(event.event_type, RepoEventType::Sync);
let blocks = event
@@ -193,7 +194,8 @@ async fn genesis_commit_event_carries_inline_blocks() {
let mst_link: CidLink = (&mst_root_cid).into();
let rev = "3kabcdefghij3";
let seq = repos
let baseline = repos.repo.get_max_seq().await.expect("get_max_seq");
repos
.repo
.insert_genesis_commit_event(
&did,
@@ -206,12 +208,7 @@ async fn genesis_commit_event_carries_inline_blocks() {
.await
.expect("insert_genesis_commit_event");
let event = repos
.repo
.get_event_by_seq(seq)
.await
.expect("get_event_by_seq")
.expect("event present");
let event = sequenced_event_for_did(repos, baseline, &did).await;
assert_eq!(event.event_type, RepoEventType::Commit);
let blocks = event
@@ -329,6 +326,11 @@ async fn import_event_carries_inline_commit_block() {
let repos = get_test_repos().await;
let typed_did = tranquil_types::Did::new(did.clone()).unwrap();
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
let events = repos
.repo
.get_events_since_seq(SequenceNumber::ZERO, None)
@@ -801,7 +801,7 @@ async fn test_firehose_outdated_cursor_info() {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
let repos = get_test_repos().await;
let max_seq = repos.repo.get_max_seq().await.unwrap().as_i64();
let max_seq = flushed_max_seq(repos).await.as_i64();
let outdated_cursor = (max_seq - 100).max(1);
let url = format!(
"ws://127.0.0.1:{}/xrpc/com.atproto.sync.subscribeRepos?cursor={}",
@@ -5,7 +5,12 @@ use common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
fn run_compaction(store: &tranquil_store::blockstore::TranquilBlockStore) {
fn run_compaction(
store: &tranquil_store::blockstore::TranquilBlockStore<
tranquil_store::RealIO,
tranquil_store::SystemClock,
>,
) {
let liveness = store.compaction_liveness(0).unwrap();
liveness
.iter()
@@ -73,7 +73,7 @@ fn write_varint(buf: &mut Vec<u8>, mut value: u64) {
}
#[tokio::test]
async fn test_import_rejects_car_for_different_user() {
async fn test_import_doesnt_reject_car_for_different_user() {
let client = client();
let (token_a, _did_a) = create_account_and_login(&client).await;
let (_token_b, did_b) = create_account_and_login(&client).await;
@@ -99,15 +99,9 @@ async fn test_import_rejects_car_for_different_user() {
.send()
.await
.expect("Import failed");
assert_eq!(import_res.status(), StatusCode::FORBIDDEN);
assert_eq!(import_res.status(), StatusCode::OK);
let body: serde_json::Value = import_res.json().await.unwrap();
assert!(
body["error"] == "InvalidRepo"
|| body["error"] == "InvalidRequest"
|| body["error"] == "DidMismatch",
"Expected InvalidRepo, DidMismatch, or InvalidRequest error, got: {:?}",
body
);
assert!(body.is_object() && body.as_object().unwrap().is_empty());
}
#[tokio::test]
+10 -12
View File
@@ -40,19 +40,17 @@ async fn get_2fa_code_from_queue(did: &str) -> Option<String> {
.await
.ok()?;
const ALPHABET: &str = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";
comms.first().and_then(|c| {
c.body
.lines()
.find(|line: &&str| line.chars().all(|c: char| c.is_ascii_digit()) && line.len() == 8)
.map(|s: &str| s.to_string())
.or_else(|| {
c.body
.split_whitespace()
.find(|word: &&str| {
word.chars().all(|c: char| c.is_ascii_digit()) && word.len() == 8
})
.map(|s: &str| s.to_string())
})
c.body.split_whitespace().find_map(|word: &str| {
let candidate = word.trim_matches(|ch: char| !ch.is_ascii_alphanumeric() && ch != '-');
let normalized = candidate.replace('-', "");
if normalized.len() == 10 && normalized.chars().all(|ch| ALPHABET.contains(ch)) {
Some(candidate.to_string())
} else {
None
}
})
})
}
@@ -773,3 +773,166 @@ async fn test_list_records_comprehensive() {
.expect("Failed with nonexistent repo");
assert_eq!(not_found_res.status(), StatusCode::BAD_REQUEST);
}
#[tokio::test]
async fn test_missing_type_is_filled_from_collection() {
let client = client();
let (did, jwt) = setup_new_user("missing-type").await;
let now = Utc::now().to_rfc3339();
let create_res = client
.post(format!(
"{}/xrpc/com.atproto.repo.createRecord",
base_url().await
))
.bearer_auth(&jwt)
.json(&json!({
"repo": did,
"collection": "app.bsky.feed.post",
"record": { "text": "no type set", "createdAt": now }
}))
.send()
.await
.expect("Failed to create record without $type");
assert_eq!(
create_res.status(),
StatusCode::OK,
"createRecord should fill missing $type from collection"
);
let create_body: Value = create_res.json().await.unwrap();
let create_rkey = create_body["uri"]
.as_str()
.unwrap()
.rsplit('/')
.next()
.unwrap()
.to_string();
let get_created = client
.get(format!(
"{}/xrpc/com.atproto.repo.getRecord",
base_url().await
))
.query(&[
("repo", did.as_str()),
("collection", "app.bsky.feed.post"),
("rkey", &create_rkey),
])
.send()
.await
.expect("Failed to get created record");
let created_body: Value = get_created.json().await.unwrap();
assert_eq!(created_body["value"]["$type"], "app.bsky.feed.post");
let put_res = client
.post(format!(
"{}/xrpc/com.atproto.repo.putRecord",
base_url().await
))
.bearer_auth(&jwt)
.json(&json!({
"repo": did,
"collection": "app.bsky.actor.profile",
"rkey": "self",
"record": { "displayName": "No Type" }
}))
.send()
.await
.expect("Failed to put record without $type");
assert_eq!(
put_res.status(),
StatusCode::OK,
"putRecord should fill missing $type from collection"
);
let get_put = client
.get(format!(
"{}/xrpc/com.atproto.repo.getRecord",
base_url().await
))
.query(&[
("repo", did.as_str()),
("collection", "app.bsky.actor.profile"),
("rkey", "self"),
])
.send()
.await
.expect("Failed to get put record");
let put_body: Value = get_put.json().await.unwrap();
assert_eq!(put_body["value"]["$type"], "app.bsky.actor.profile");
let apply_res = client
.post(format!(
"{}/xrpc/com.atproto.repo.applyWrites",
base_url().await
))
.bearer_auth(&jwt)
.json(&json!({
"repo": did,
"writes": [
{ "$type": "com.atproto.repo.applyWrites#create", "collection": "app.bsky.feed.post", "rkey": "batch-no-type", "value": { "text": "batch no type", "createdAt": now } }
]
}))
.send()
.await
.expect("Failed to apply writes without $type");
assert_eq!(
apply_res.status(),
StatusCode::OK,
"applyWrites should fill missing $type from collection"
);
let get_batch = client
.get(format!(
"{}/xrpc/com.atproto.repo.getRecord",
base_url().await
))
.query(&[
("repo", did.as_str()),
("collection", "app.bsky.feed.post"),
("rkey", "batch-no-type"),
])
.send()
.await
.expect("Failed to get batch record");
let batch_body: Value = get_batch.json().await.unwrap();
assert_eq!(batch_body["value"]["$type"], "app.bsky.feed.post");
let mismatch_res = client
.post(format!(
"{}/xrpc/com.atproto.repo.createRecord",
base_url().await
))
.bearer_auth(&jwt)
.json(&json!({
"repo": did,
"collection": "app.bsky.feed.post",
"record": { "$type": "app.bsky.feed.like", "text": "wrong type", "createdAt": now }
}))
.send()
.await
.expect("Failed to send mismatch request");
assert_eq!(
mismatch_res.status(),
StatusCode::BAD_REQUEST,
"explicit mismatched $type should still be rejected"
);
let non_string_type_res = client
.post(format!(
"{}/xrpc/com.atproto.repo.createRecord",
base_url().await
))
.bearer_auth(&jwt)
.json(&json!({
"repo": did,
"collection": "app.bsky.feed.post",
"record": { "$type": 123, "text": "non-string type", "createdAt": now }
}))
.send()
.await
.expect("Failed to send non-string type request");
assert_eq!(
non_string_type_res.status(),
StatusCode::BAD_REQUEST,
"present non-string $type should be rejected, not overwritten"
);
}
@@ -223,7 +223,7 @@ async fn websocket_firehose_frames_pass_inductive_forward_and_inverse() {
let (token, did) = create_account_and_login(&client).await;
let repos = get_test_repos().await;
let cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let cursor = flushed_max_seq(repos).await.as_i64();
let consumer = FirehoseConsumer::connect_with_cursor(app_port(), cursor).await;
tokio::time::sleep(Duration::from_millis(100)).await;
@@ -298,7 +298,7 @@ async fn websocket_firehose_car_root_matches_commit_cid() {
let (token, did) = create_account_and_login(&client).await;
let repos = get_test_repos().await;
let cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let cursor = flushed_max_seq(repos).await.as_i64();
let consumer = FirehoseConsumer::connect_with_cursor(app_port(), cursor).await;
tokio::time::sleep(Duration::from_millis(100)).await;
@@ -340,7 +340,7 @@ async fn websocket_firehose_resumption_from_cursor_yields_valid_frames() {
create_record(&client, &token, &did, &rkey_for(i), "pre").await;
}
let resume_cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let resume_cursor = flushed_max_seq(repos).await.as_i64();
for i in 5..12 {
create_record(&client, &token, &did, &rkey_for(i), "post").await;
@@ -371,7 +371,7 @@ async fn websocket_firehose_ops_include_prev_field_for_update_delete() {
let client = client();
let (token, did) = create_account_and_login(&client).await;
let repos = get_test_repos().await;
let cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let cursor = flushed_max_seq(repos).await.as_i64();
let consumer = FirehoseConsumer::connect_with_cursor(app_port(), cursor).await;
tokio::time::sleep(Duration::from_millis(100)).await;
@@ -417,7 +417,7 @@ async fn websocket_firehose_rebuild_new_mst_from_car_matches_commit_data() {
let client = client();
let (token, did) = create_account_and_login(&client).await;
let repos = get_test_repos().await;
let cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let cursor = flushed_max_seq(repos).await.as_i64();
let consumer = FirehoseConsumer::connect_with_cursor(app_port(), cursor).await;
tokio::time::sleep(Duration::from_millis(100)).await;
@@ -247,6 +247,11 @@ fn rkey_for(prefix: &str, i: usize) -> String {
async fn our_commit_events(did: &str) -> Vec<SequencedEvent> {
let repos = get_test_repos().await;
let typed_did = Did::new(did.to_string()).unwrap();
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
let events = repos
.repo
.get_events_since_seq(SequenceNumber::ZERO, None)
+7 -2
View File
@@ -216,6 +216,11 @@ async fn imported_repo_emits_commit_event_with_valid_car() {
let repos = get_test_repos().await;
let typed_did = Did::new(did.clone()).unwrap();
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
let events = repos
.repo
.get_events_since_seq(SequenceNumber::ZERO, None)
@@ -253,7 +258,7 @@ async fn firehose_commit_block_bytes_roundtrip_to_same_cid() {
let (token, did) = create_account_and_login(&client).await;
let repos = get_test_repos().await;
let cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let cursor = flushed_max_seq(repos).await.as_i64();
let consumer = FirehoseConsumer::connect_with_cursor(app_port(), cursor).await;
tokio::time::sleep(Duration::from_millis(100)).await;
@@ -284,7 +289,7 @@ async fn firehose_commit_car_contains_new_record_bytes_for_every_create() {
let (token, did) = create_account_and_login(&client).await;
let repos = get_test_repos().await;
let cursor = repos.repo.get_max_seq().await.unwrap().as_i64();
let cursor = flushed_max_seq(repos).await.as_i64();
let consumer = FirehoseConsumer::connect_with_cursor(app_port(), cursor).await;
tokio::time::sleep(Duration::from_millis(100)).await;
@@ -267,6 +267,11 @@ async fn fuzz_run_with_seed(seed: u64, steps: usize) -> Vec<String> {
let repos = get_test_repos().await;
let typed_did = Did::new(did.clone()).unwrap();
repos
.repo
.flush_pending_sequences()
.await
.expect("flush_pending_sequences");
let events = repos
.repo
.get_events_since_seq(SequenceNumber::ZERO, None)

Some files were not shown because too many files have changed in this diff Show More