Compare commits

...
Author SHA1 Message Date
nelind bfadc936c0 fix(api): dont verify signature or DID during importRepo 2026-06-11 23:23:37 +02:00
nelind 80afd764d3 fix(nix): provide a jemalloc build in the dev shell 2026-06-10 19:56:40 +02:00
LewisandTangled 5bbe2146ff server: serve xrpc over http/3
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-10 13:18:49 +03:00
LewisandTangled b009ccdaf2 repo: the pg side of MST structural repair
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-08 16:58:51 +03:00
nelind 39f74b5adf feat(nix): pratice what i preach. expose lib.mkPackages 2026-06-07 22:14:27 +02:00
aveandTangled a231d7da29 feat(auth): accept totp/backup codes in legacy login createSession 2026-06-06 10:42:24 +03:00
aveandTangled 63d84d38fb refactor(auth): unify short-code generation onto util::generate_token_code
Collapse the three ad-hoc short-code generators into one canonical generator
plus a shared normalizer:

- util::generate_token_code now emits the uppercase base32 XXXXX-XXXXX display
  form; new util::normalize_token_code canonicalizes user input (uppercase,
  strip hyphen/whitespace).
- email_token and legacy_2fa now generate via util, store the normalized form,
  and compare normalized input. Their private generate_short_token/generate_code
  (and BASE32_CHARS/CODE_LENGTH) are removed.
- PLC (request/sign) and password reset inline util::generate_token_code,
  persist the normalized form, email the display form, and normalize input
  before lookup. The generate_plc_token/generate_reset_code wrappers are removed.

Behavior changes: legacy login-2FA codes go from 8-digit numeric to XXXXX-XXXXX;
PLC and password-reset codes go from lowercase to uppercase. All four code types
are now accepted case-/hyphen-insensitively. OAuth web-login 2FA, account
deletion, and the long verification_token blobs are intentionally untouched.

Tests: add util normalize tests + email_token/legacy_2fa case/hyphen tests;
update integration tests to expect the canonical stored form and the new
emailed format.
2026-06-06 09:36:08 +03:00
aveandTangled fe9b88141c chore(auth): align at+jwt/refresh+jwt expiry with reference PDS 2026-06-06 01:43:46 +03:00
aveandTangled 72f5dce32b add ave.zone to contributors list in README 2026-06-06 01:42:54 +03:00
nelind cd7e01100e chore(auth): also mention that atproto spec requiers typ be "JWT" for inter-service tokens 2026-06-05 12:49:25 +02:00
aveandTangled 7c248be153 fix(auth): emit uppercase "JWT" typ in service-auth header
RFC 7519 §5.1 recommends the uppercase "JWT" typ for compatibility with
legacy implementations, and it matches the reference @atproto/pds. Parsing
already lowercases, so existing lowercase "jwt" tokens still verify.
2026-06-05 13:34:43 +03:00
aveandTangled 91999819c6 fix(proxy): limit audience of getFeed service-auth to the feed generator 2026-06-04 22:39:14 +03:00
nelindandTangled ffce1d5d05 feat(docs): add some general PDS debugging documentation 2026-06-04 13:45:08 +03:00
TylerandTangled 8e6ace2fe2 fix: derive lexicon DNS authority from all-but-last NSID segment
Permission-set expansion resolved the lexicon's DNS authority using a
fixed `parts[..2]`, which only works for three-segment NSIDs. For a
four-segment NSID such as community.lexicon.bookmarks.authManageBookmarks
this dropped a segment and queried _lexicon.lexicon.community instead of
_lexicon.bookmarks.lexicon.community, failing with "DNS resolution
failed: ... no record found".

The authority is every NSID segment except the last (the name),
reversed. Use parts[..parts.len() - 1] to match the spec and the
existing extract_namespace_authority helper, and update the DNS
authority test with three/four-segment and bookmarks regression cases.
2026-06-04 01:28:13 +03:00
LewisandTangled 3018a20843 fix(plc): allow arbitrary services to sign
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-03 11:23:51 +03:00
LewisandTangled 37fc06fb39 fix(store): unblock eventlog sync&freeze when writer dies
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
LewisandTangled 728a8c4d3b test(store): cross-store, firehose, read-validation coverage w/ faults
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
LewisandTangled 3d49e99cc3 test(store): generic consistency checker, gauntlet fault/read
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
LewisandTangled 7e823673ca test(store): untested metastore, eventlog, & archival stuff
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-02 17:29:21 +03:00
Lewis 320933598c fix(auth): error num 401 for oauth
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:38:45 +03:00
Lewis 500dc2e0e6 test(store): gauntlet sweep configs for time-travel & fsync repro
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
Lewis a220611a8b test(store): D gauntlet faults, crash-loss oracley, recoverable scenarios
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
Lewis ca7a4b4b73 fix(store): recover eventlog lastseq from tail not sidecar
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
Lewis 8ff02610e4 feat(store): inline-commit mode, committed-extent recovery, failsafe verify&rollback
Lewis: May this revision serve well! <lu5a@proton.me>
2026-06-01 19:01:04 +03:00
LewisandTangled 22f82489d5 test(pds): e2e & durability coverage for MST self-heal
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled cee483e358 feat(pds): selfhealing repo writing by detecting corruption & retrying
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled 7f8e858137 test(store): gauntlet MST-repairable & misdirected-write scenario
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled 44d73dac58 feat(store): rebuild & rewrite missing/corrupt MST blocks
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled b8cae15c12 feat(store): detect foreign&corrupt blocks on read & preserve blocks thru recovery
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 21:11:36 +03:00
LewisandTangled 9b58961bba feat(repo): missing $type? invent one
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 13:02:11 +03:00
Lewis 31ee12ecd3 fix(store): torn hint-file tail should be recoverable on reopen
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-31 11:37:21 +03:00
nelind ea106d5246 chore(nix): update frontend pnpm hash 2026-05-31 00:51:27 +02:00
Lewis 4015217a2e feat(store): Clock trait for DST
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 23:46:22 +03:00
Lewis 7e4c9e1dab chore: bump to 0.6.4
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 21:23:02 +03:00
LewisandTangled e9dc57d6f4 fix(firehose): lost events if seq commits out of order
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 21:10:38 +03:00
Lewis 4e2525b245 fix(migrate): oauth account:* spec, p256 support
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-30 21:07:16 +03:00
nelind 38508c2c6e feat(nix): base the data storage path defaults on the dataDir option 2026-05-30 00:37:55 +02:00
Lewis 7a54ccf6a3 fix(tranquil-config): reject unknown keys in TlsConfig, bump rust 1.96
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-29 09:39:14 +03:00
nelind e85394c314 build: bump version to 0.6.3 2026-05-27 19:28:10 +02:00
isabelandTangled 411c6be108 feat(nix): validate config at build time 2026-05-27 20:14:06 +03:00
isabelandTangled 28aa7ab7fc feat!(tranquil-config): error on unknown keys
an attempt to make it so that fixes like
fc6063dba8 never have to occur again
2026-05-27 20:13:58 +03:00
nelindandTangled 348b69d76b feat(nix): ensure that tranquil-store path is always writable by the service user just like file blob storage 2026-05-27 20:13:07 +03:00
nelindandTangled 4b23ca2c36 fix(docs): some wording on nix config and CI stuffs 2026-05-27 20:13:07 +03:00
LewisandTangled a7052e878c feat: tranquil's own TLS handling
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-27 13:54:33 +03:00
nelind fc6063dba8 fix(nix): override the right frontend package setting in the flake nixos module 2026-05-27 01:04:31 +02:00
LewisandTangled 44cb016762 fix(docs): install docs better
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-26 09:49:59 +03:00
LewisandTangled e6eee18ace fix(docs): migration oauth update
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-25 19:02:13 +03:00
Lewis 96c8375706 fix(test): oauth token eviction should be agnostic
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-25 16:17:10 +03:00
Lewis 86c5995568 fix(config): signal gate is useless since needs admin work to activate anyway
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-24 22:50:14 +03:00
LewisandTangled 036c317fd6 fix(migrate): oauth refreshing, embedded db first invite code
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-24 19:28:57 +03:00
LewisandTangled 4d2c7d4723 feat(auth): verification-gate override, inbound-migration bypass, store deleter improvement
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-23 23:02:43 +03:00
isabelandTangled f24a9f8bc0 build(nix): frontend pin pnpm
some future protections against this acursed toolchain breaking and
people who do .follows on their nixpkgs inputs
2026-05-22 23:24:45 +03:00
Lewis f6ef6ecbd9 feat(docs): first version of 'when PRing' section of contrib
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-21 22:46:59 +03:00
karitham.devandTangled a2567bdb1a fix(locales): french handle should be "identifier" not "username"
My bad translation on that earlier, made it consistent with the rest of the file and usage!
2026-05-21 10:39:29 +03:00
Lewis 56120d252d fix(locale): a little more drift
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-21 09:14:06 +03:00
TrezyandTangled 210b0f463c feat: add About page
Signed-off-by: Trezy <tre@trezy.com>
2026-05-21 09:06:42 +03:00
LewisandTangled c80f504dc0 fix(locale): a couple of drift keys missing in other langs
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-21 09:06:10 +03:00
karitham.devandTangled a3bd7c59ad feat(locales): add french locale, fix missing error keys, remove dead keys
Adds french locale registration and translation file. Fixes 4 missing error keys that were referenced in code but absent from all locale files. Removes 2 unreferenced keys from `register.validation` that duplicated `registerPasskey.errors`.

I *think* that's all that's needed for french locale? Couldn't find anything else required to make it work!
2026-05-20 22:20:20 +03:00
nelind 9ebde27540 fix(docs): markdown is tricky ... 2026-05-19 21:30:35 +02:00
nelindandTangled 28a7834304 docs: nel interjection: be right, be good, be fun 2026-05-19 22:18:38 +03:00
Lewis 191da5b311 chore: slight doc tweak
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-19 21:14:28 +03:00
Lewis a5b4ba7d65 feat(docs): init better docs
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-19 20:45:50 +03:00
TrezyandTangled 90dabd8840 docs: start a contributing file to track how to work with the repo
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 19eaccea74 chore: add dev startup to justfile
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled f620a6bc43 chore: add mailpit for local dev
Captures outbound emails, greatly simplifying local dev.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 04f370aaa1 chore: add frontend service to docker compose
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 3fd8f7ebbf chore: add app service with cargo-watch
Spins up a container with Tranquil running, but automatically uses incremental rebuilds for local dev.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 0afcb2ee28 chore: add plc directory for local dev
Prevents us from polluting the real PLC directory.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 8bd556f65b chore: add db health check to docker compose
Safer wafe to depend on database startup.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled cc92594506 chore: add traefik for local dev certs and routing
Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 76f22b801b chore: allow native certs during local dev
Allows Traefik certs to be trusted by Tranquil.

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:58 +03:00
TrezyandTangled 021b7dbec4 chore: add profiles to docker compose
Allows us to spin up groups from the compose file

Signed-off-by: Trezy <tre@trezy.com>
2026-05-19 19:46:57 +03:00
nelind eb034cb8b3 fix(ci): dont pass cachix cache name as a secret so it doesnt get redacted from CI logs 2026-05-17 17:07:34 +02:00
Lewis bdaf510898 build: bump workspace to 0.6.2
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-17 12:32:05 +03:00
Lewis deb2502112 feat(pds): phantom-file self-heal goes in scheduled compaction + reachability walk
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-17 12:32:05 +03:00
Lewis 1815ddba9f feat(gauntlet): index-backed/hint-backed/readable invariants, ExternalCorruption scenario
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-17 12:32:05 +03:00
Lewis a7517ed5c9 feat(store): consistency check & repair for orphan hints etc
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-17 11:50:32 +03:00
Lewis d07d702dd4 feat(store): try to self-heal phantom index entries on compaction
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-17 11:50:19 +03:00
isabelandTangled 1901b0a630 fix(nix): build against pnpm 11
side note: you may also notice that I have removed the `github:` url for
nixpkgs this is because it save a massive 15mb, is faster and more
reliable than github
See also:
-
<https://nix.dev/manual/nix/stable/protocols/tarball-fetcher#lockable-http-tarball-protocol>
-
<http://web.archive.org/web/20250806225139/https://nix.dev/manual/nix/2.28/protocols/tarball-fetcher#lockable-http-tarball-protocol>
2026-05-14 21:37:28 +03:00
serenityandTangled 58f8d327c1 feat(docs): clarify nix documentation 2026-05-14 16:52:10 +03:00
LewisandTangled a13343e1de fix(oauth): gc tokens in pg in the right order, more exposure of dpop err
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-14 16:51:53 +03:00
Bas van den WollenbergandTangled 60e10af4aa fix(Dockerfile): pnpm v11 build errors
- pnpm doesn't implicitly ignore post-install scripts anymore which means trying to build the docker image now fails. i've explicitly allowed them but idk could also ignore them maybe not sure what they actually do lol

- added `confirmModulesPurge: false` but not sure what it actually does as it's not documented? but either way it's also required for the `pnpm build` step to succeed non-interactively it seems idk weird stuff

- removed msmtp/sendmail while i was at it just like https://tangled.org/tranquil.farm/tranquil-pds/commit/f176f55862bc4c41472f8697cc2ca069c24d5e37
2026-05-14 09:56:07 +03:00
isabelandTangled f176f55862 chore(nix): remove sendmail 2026-05-02 23:32:57 +03:00
LewisandTangled fac9520a16 feat(tranquil-server): email config, tests, fmt
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-02 22:28:59 +03:00
LewisandTangled eee6fb9ff4 feat(comms): EmailSender, permanent/transient routing
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-02 22:28:59 +03:00
LewisandTangled 2462d0ab3b feat(tranquil-comms): smtp and dkim signing
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-02 22:28:59 +03:00
LewisandTangled 85f87f7b28 feat(tranquil-comms): message construction and mx resolution
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-02 22:28:59 +03:00
LewisandTangled b1d86caa78 feat(tranquil-comms): prework for email
Lewis: May this revision serve well! <lu5a@proton.me>
2026-05-02 22:28:59 +03:00
Lewis 9b2cfb3a7e fix(tranquil-store): durable-tail recovery + sync semantics
Lewis: May this revision serve well! <lu5a@proton.me>
2026-04-30 11:11:28 +03:00
Lewis efd499bb26 fix(tranquil-store): barrier durability + torn-header recovery
Lewis: May this revision serve well! <lu5a@proton.me>
2026-04-29 15:35:42 +03:00
nelindandTangled d4dfe838eb fix(ci): use kaniko to build 2026-04-28 23:06:36 +03:00
LewisandTangled af3821514f test(tranquil-pds): same-rkey batch coverage and inductive inverse for in-batch dups
Lewis: May this revision serve well! <lu5a@proton.me>
2026-04-28 22:05:03 +03:00
281 changed files with 20653 additions and 2939 deletions
+28
View File
@@ -68,6 +68,10 @@ test-group = "serial-env-tests"
filter = "package(tranquil-signal)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "package(tranquil-config)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "binary(whole_story)"
test-group = "heavy-load-tests"
@@ -102,6 +106,16 @@ slow-timeout = { period = "300s", terminate-after = 8 }
filter = "binary(compaction_restart) | binary(mst_refcount_integrity) | binary(gc_compaction_restart)"
slow-timeout = { period = "120s", terminate-after = 4 }
[[profile.default.overrides]]
filter = "test(/retention_time_travel_survives_many_seeds/)"
slow-timeout = { period = "300s", terminate-after = 24 }
test-group = "io-heavy-sim"
[[profile.default.overrides]]
filter = "binary(fd_lifecycle)"
slow-timeout = { period = "300s", terminate-after = 4 }
test-group = "io-heavy-sim"
[[profile.ci.overrides]]
filter = "test(/import_with_verification/) | test(/plc_migration/)"
test-group = "serial-env-tests"
@@ -118,6 +132,10 @@ test-group = "serial-env-tests"
filter = "package(tranquil-signal)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "package(tranquil-config)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "binary(whole_story)"
test-group = "heavy-load-tests"
@@ -129,3 +147,13 @@ test-group = "heavy-load-tests"
[[profile.ci.overrides]]
filter = "binary(repo_lifecycle)"
test-group = "heavy-load-tests"
[[profile.ci.overrides]]
filter = "test(/retention_time_travel_survives_many_seeds/)"
slow-timeout = { period = "300s", terminate-after = 24 }
test-group = "io-heavy-sim"
[[profile.ci.overrides]]
filter = "binary(fd_lifecycle)"
slow-timeout = { period = "300s", terminate-after = 4 }
test-group = "io-heavy-sim"
+12
View File
@@ -0,0 +1,12 @@
target/
.git/
.jj/
**/node_modules/
frontend/dist/
frontend/coverage/
frontend/.pnpm-store/
.direnv/
result
.env
*.output
reference-pds-bsky/
+3
View File
@@ -4,3 +4,6 @@ target/
result
frontend/node_modules/
frontend/dist/
frontend/coverage/
frontend/.pnpm-store
frontend/.npmrc
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "SELECT nextval('firehose_seq') as \"seq!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq!",
"type_info": "Int8"
}
],
"parameters": {
"Left": []
},
"nullable": [
null
]
},
"hash": "09142b5a6b91fe4fd795fb6c85af5461e2621f6c3545b6d26e4618d813904b75"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE repo_seq r\n SET seq = $1::bigint + (o.ord - 1)\n FROM unnest($2::bigint[]) WITH ORDINALITY AS o(id, ord)\n WHERE r.id = o.id",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Int8Array"
]
},
"nullable": []
},
"hash": "0b895d105363a60040323c5c47bc897905af3da980d0dc1b3b60bbdcdd90ea7a"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM repo_seq\n WHERE did = $1\n AND id <> (SELECT id FROM repo_seq WHERE did = $1 ORDER BY id DESC LIMIT 1)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "122d79e006c0e11214e3cf11cfb98bdff6fbf156893b482572532ebcd8979e61"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin\n FROM users WHERE did = $1",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration\n FROM users WHERE handle = $1",
"describe": {
"columns": [
{
@@ -42,6 +42,11 @@
"ordinal": 7,
"name": "is_admin",
"type_info": "Bool"
},
{
"ordinal": 8,
"name": "inbound_migration",
"type_info": "Bool"
}
],
"parameters": {
@@ -57,8 +62,9 @@
false,
true,
true,
false,
false
]
},
"hash": "6b51995c40519a63f85c70f29ca8bd6ec1963c8562d78215d980785dc46a6384"
"hash": "18bbda5582db1b32d02ab8a3eee970c9508b9bd67239c2f936639a9f863b30ff"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE users SET deactivated_at = NULL WHERE did = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "23201d4e26bc650939e30f69fb0bca00d351d057098afebc1017f70a84b4bd22"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', false, 'deleted')\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', false, 'deleted')\n RETURNING id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "id",
"type_info": "Int8"
}
],
@@ -18,5 +18,5 @@
false
]
},
"hash": "a0ea3630adb9e411d94d88084df6b05d7fc3ba8c306146fc932b736fe342eceb"
"hash": "2c3c0740793f29b9bd9e0f81c2372d427dee091e20da8f71ae1e5f2769aa985d"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_try_advisory_xact_lock($1) as \"locked!\"",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "locked!",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "3cc7247d2cd7e85a6f21d783dd956f8fecc6f63c994014bb4e91504d498b5fd5"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, handle)\n VALUES ($1, 'identity', $2)\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": []
},
"hash": "4f3ef746cb5fc0916d3df5f92494c538497d3b9c1cfcc6e44c9b80f2193094f6"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, handle)\n VALUES ($1, 'identity', $2)\n RETURNING seq\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "50b2178cd174af1dc132a22048e52a6af492f4d8025ec512dd9c0c9ba62eb202"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE users SET deactivated_at = NULL, inbound_migration = FALSE WHERE did = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "51e029e572777e6a103fd7fd5550494de9d4cac7e3ff84e27ddec1a6aaefc047"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE comms_queue\n SET status = 'failed'::comms_status,\n attempts = max_attempts,\n last_error = $2,\n updated_at = NOW()\n WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": []
},
"hash": "5bee4ed5296667e4ca7e1a97aec28d30a470b8aee7b378ec9ca4e34de4faf349"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id as \"id!\" FROM repo_seq WHERE seq IS NULL ORDER BY id LIMIT $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id!",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
false
]
},
"hash": "714a1c9a73e2fed8182d4ab2fe4a2c33bc10a52e27ed95d3a6641477ae42395e"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq = $1",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -90,7 +90,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -108,5 +108,5 @@
true
]
},
"hash": "4702a94384ce938d3f5384bc2106a48500f8749134e6bfd1c619a9151f5d3772"
"hash": "7cdf3b8136782a5b2e4aff91907a680566b4610e6cbf9d9396d31f9bbd99c3b8"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, rev)\n VALUES ($1, 'commit', $2, $3::TEXT, $4, $5, $6, $7, $8)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, rev)\n VALUES ($1, 'commit', $2, $3::TEXT, $4, $5, $6, $7, $8)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -21,9 +15,7 @@
"Text"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "a325862f897484bb1a8863fd7a1bc3fd17ab20b6e30498f9900daf80609c413d"
"hash": "800b93a38eef70bafdf0a1b221994f3c19ab562f84d18265376296a592b4c767"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)\n VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -23,9 +17,7 @@
"Text"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "97dd3cfe72b74b4dac686c030a07f6eb56dabb2df72815f3248760b4e3f7950e"
"hash": "82276a34f473ff8b9790b78915ae3f72873d4b748cdb47bd2b81edc2f7a6e5a8"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM repo_seq WHERE did = $1 AND seq != $2",
"query": "DELETE FROM repo_seq WHERE did = $1 AND id <> $2",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "b6d6548acb89d6384cd226f6ed0d66de27fde3af24b4a7a3fce7e098812e38a5"
"hash": "856e458c1c78f030545dc147f76d1b91fa4cc6e178eb51133c52358855ba3f02"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE comms_queue\n SET status = 'processing', updated_at = NOW()\n WHERE id IN (\n SELECT id FROM comms_queue\n WHERE status = 'pending'\n AND scheduled_for <= $1\n AND attempts < max_attempts\n ORDER BY scheduled_for ASC\n LIMIT $2\n FOR UPDATE SKIP LOCKED\n )\n RETURNING\n id, user_id,\n channel as \"channel: CommsChannel\",\n comms_type as \"comms_type: CommsType\",\n status as \"status: CommsStatus\",\n recipient, subject, body, metadata,\n attempts, max_attempts, last_error,\n created_at, updated_at, scheduled_for, processed_at",
"query": "UPDATE comms_queue\n SET status = 'processing', updated_at = NOW()\n WHERE id IN (\n SELECT id FROM comms_queue\n WHERE attempts < max_attempts\n AND scheduled_for <= $1\n AND (\n status = 'pending'\n OR (status = 'processing'\n AND updated_at < $1 - INTERVAL '10 minutes')\n )\n ORDER BY scheduled_for ASC\n LIMIT $2\n FOR UPDATE SKIP LOCKED\n )\n RETURNING\n id, user_id,\n channel as \"channel: CommsChannel\",\n comms_type as \"comms_type: CommsType\",\n status as \"status: CommsStatus\",\n recipient, subject, body, metadata,\n attempts, max_attempts, last_error,\n created_at, updated_at, scheduled_for, processed_at",
"describe": {
"columns": [
{
@@ -154,5 +154,5 @@
true
]
},
"hash": "8047fda41bd94f819213decb8b3e0aba49a8dbdb10217eefd77e3567f8c9694a"
"hash": "890aa92acdcb0fe2a3bf04d87e1f16a801d271da7cedc32fc42c2ef5b100faae"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', $2, $3)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, active, status)\n VALUES ($1, 'account', $2, $3)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -16,9 +10,7 @@
"Text"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "623042c0b901e17ff37c5bcc1a4eefe56fc8b996ca8326e75796897d13c9e9ed"
"hash": "8ccd42b5b1b75f45e094889ee26af8517de28ca687efa81fd9c4fce598338fe9"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "\n DELETE FROM oauth_token\n WHERE id IN (\n SELECT id FROM oauth_token\n WHERE did = $1\n ORDER BY updated_at ASC\n OFFSET $2\n )\n ",
"query": "\n DELETE FROM oauth_token\n WHERE id IN (\n SELECT id FROM oauth_token\n WHERE did = $1\n ORDER BY created_at DESC\n OFFSET $2\n )\n ",
"describe": {
"columns": [],
"parameters": {
@@ -11,5 +11,5 @@
},
"nullable": []
},
"hash": "56cd24903171eddc2ededd9079ffe10937c34e99b0305f25c980ca754da44625"
"hash": "8f4357f7a18ddcf6b686a4555f244d37c35917364b8f917ca6ee2d4030ace742"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -90,7 +90,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -108,5 +108,5 @@
true
]
},
"hash": "c47746c256fd8c9d386551f3bd270e9a96fecdb713642d07d60fb4aeb02c8426"
"hash": "957bc81c63510ab502d2f8cea75dafcbdf2c8ebb448c86990d5e404ee5280438"
}
@@ -1,14 +1,8 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, rev, block_cids, block_data)\n VALUES ($1, 'sync', $2, $3, $4, $5)\n RETURNING seq\n ",
"query": "\n INSERT INTO repo_seq (did, event_type, commit_cid, rev, block_cids, block_data)\n VALUES ($1, 'sync', $2, $3, $4, $5)\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"type_info": "Int8"
}
],
"columns": [],
"parameters": {
"Left": [
"Text",
@@ -18,9 +12,7 @@
"ByteaArray"
]
},
"nullable": [
false
]
"nullable": []
},
"hash": "5996774d097a484f81353d49e4a99b07c30a003377c924add576bfb48ddde29b"
"hash": "9ba7869f29286cfd6b811ffa1e26bda1d637b98b44be66cdc265b1af4cec2d4f"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT pg_advisory_xact_lock($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "pg_advisory_xact_lock",
"type_info": "Void"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "a06e1d9f6f95e4c4c2b98310ebddcc9d963cc033582bf2e945e8bf3a301b4247"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT setval('firehose_seq', $1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "setval",
"type_info": "Int8"
}
],
"parameters": {
"Left": [
"Int8"
]
},
"nullable": [
null
]
},
"hash": "bd0407818c8bdae1f1024777951b08cb9c386fe751722e1caf6ad243e271697a"
}
@@ -0,0 +1,12 @@
{
"db_name": "PostgreSQL",
"query": "NOTIFY repo_updates",
"describe": {
"columns": [],
"parameters": {
"Left": []
},
"nullable": []
},
"hash": "d4eacb9bca838dd18dcfd8ca39b83ef50968bfaaabc213536c049cc222ffe9c8"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -91,7 +91,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -109,5 +109,5 @@
true
]
},
"hash": "89c8ce80d8c52b4668c34f63e48b51cacc7e8ff300958a192378f1d72de04f9a"
"hash": "da2770e44c0ec21756ce4b6b0c57abff4f38cd8c680c5ced7d3b35f3c1f58da7"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1 AND seq < $2\n ORDER BY seq ASC",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -91,7 +91,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -109,5 +109,5 @@
true
]
},
"hash": "0349b2af4d21105029ecc2bf81ad7be9987ec17ac088730596849d7a45558e57"
"hash": "dfe011b2d2bafbbbbea508a4977fc34bb1be4cf2612296df23763e6e5841c33e"
}
@@ -1,11 +1,11 @@
{
"db_name": "PostgreSQL",
"query": "SELECT seq, did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1 AND seq < $2\n ORDER BY seq ASC",
"query": "SELECT seq as \"seq!\", did, created_at, event_type as \"event_type: RepoEventType\", commit_cid, prev_cid, prev_data_cid,\n ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev\n FROM repo_seq\n WHERE seq > $1\n ORDER BY seq ASC\n LIMIT $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "seq",
"name": "seq!",
"type_info": "Int8"
},
{
@@ -91,7 +91,7 @@
]
},
"nullable": [
false,
true,
false,
false,
false,
@@ -109,5 +109,5 @@
true
]
},
"hash": "ad784aff9bb5c1f19f90a3b1c3b3694a3e1aa889b0fe3e55cfebb7af8c620256"
"hash": "f16d41b41a5388997e6ed408da55056173340ed12b60b1a956de87f73f9d0824"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin\n FROM users WHERE handle = $1",
"query": "SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration\n FROM users WHERE did = $1",
"describe": {
"columns": [
{
@@ -42,6 +42,11 @@
"ordinal": 7,
"name": "is_admin",
"type_info": "Bool"
},
{
"ordinal": 8,
"name": "inbound_migration",
"type_info": "Bool"
}
],
"parameters": {
@@ -57,8 +62,9 @@
false,
true,
true,
false,
false
]
},
"hash": "584bceda60d8b6a02e7dc44d833e3fba13151f36ba9f75c64e33d6cb534cc939"
"hash": "f1c4ec28b02d09ffce35aa8249c1747a70c12a3ecfc1ff6ca1847840f770db2f"
}
@@ -18,4 +18,4 @@ steps:
- name: Build and push aarch64
command: |
SUBS="--option extra-substituters https://tranquil.cachix.org --option extra-trusted-public-keys tranquil.cachix.org-1:PoO+mGL6a6LcJiPakMDHN4E218/ei/7v2sxeDtNkSRg="
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds-aarch64 $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push "$CACHIX_CACHE_NAME"
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds-aarch64 $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push tranquil
+3 -3
View File
@@ -18,11 +18,11 @@ steps:
- name: Build and push x86_64
command: |
SUBS="--option extra-substituters https://tranquil.cachix.org --option extra-trusted-public-keys tranquil.cachix.org-1:PoO+mGL6a6LcJiPakMDHN4E218/ei/7v2sxeDtNkSRg="
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push "$CACHIX_CACHE_NAME"
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-frontend $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push "$CACHIX_CACHE_NAME"
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-pds $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push tranquil
nix-store -qR --include-outputs $(nix-store -qd $(nix build .#packages.x86_64-linux.tranquil-frontend $SUBS --print-out-paths --no-link)) | grep -v '\.drv$' | cachix push tranquil
- name: Build and push devShell
command: |
SUBS="--option extra-substituters https://tranquil.cachix.org --option extra-trusted-public-keys tranquil.cachix.org-1:PoO+mGL6a6LcJiPakMDHN4E218/ei/7v2sxeDtNkSRg="
nix develop $SUBS --profile dev-profile -c true
cachix push "$CACHIX_CACHE_NAME" dev-profile
cachix push tranquil dev-profile
+24 -12
View File
@@ -1,24 +1,36 @@
when:
- event: []
branch: []
- event: [ "manual" ]
- event: [ "push" ]
branch: [ "main" ]
engine: nixery
dependencies:
nixpkgs:
- podman
- kaniko
environment:
DOCKER_CONFIG: "/kaniko/.docker"
steps:
- name: Create podman config
- name: Configure Kaniko
command: |
mkdir -p ~/.config/containers
echo "unqualified-search-registries = [\"docker.io\"]" >> ~/.config/containers/registries.conf
mkdir -p /kaniko/.docker/
echo "{
\"auths\": {
\"https://atcr.io/v1\":{
\"auth\": \"$ATCR_CREDENTIALS\"
}
}
}" > /kaniko/.docker/config.json
- name: Build image
command: |
podman build . -t tranquil-pds:latest -t "tranquil-pds:$TANGLED_COMMIT_SHA"
- name: Publish image
command: |
podman push --creds "$ATCR_USERNAME:$ATCR_PASSWORD" tranquil-pds:latest "atcr.io/tranquil.farm/tranquil-pds:latest"
podman push --creds "$ATCR_USERNAME:$ATCR_PASSWORD" "tranquil-pds:$TANGLED_COMMIT_SHA" "atcr.io/tranquil.farm/tranquil-pds:$TANGLED_COMMIT_SHA"
executor \
--context=$(pwd) \
--ignore-path=$(pwd) \
--dockerfile=$(pwd)/Dockerfile \
--destination="atcr.io/tranquil.farm/tranquil-pds:latest" \
--destination="atcr.io/tranquil.farm/tranquil-pds:$TANGLED_COMMIT_SHA" \
--push-retry=3 \
--skip-push-permission-check
+135
View File
@@ -0,0 +1,135 @@
# Contributing to Tranquil PDS
## When PRing
In order of importance:
- **You must run your change! Every contribution that says "here's xyz. untested." does not help the project.**
- Relevant tests to your PR must pass. The whole suite doesn't have to be proven to have run, because there are a *ton* of tests and they're quite heavy, but hopefully there are existing tests for whatever you're PRing, and if there aren't, please add those too.
- Run cargo fmt :P
> 🦪 Lewis
>
> Good CI fixes some of these. We should really get around to that.
Things that would also be nice but aren't like, a pain in our side:
- Big changes should be stacked PRs that are broken up into digestible pieces. Those stacked PRs should hopefully be able to be merged individually if necessary.
## Local Development
### Prerequisites
- [Docker](https://docs.docker.com/get-docker/) and Docker Compose
- Add `pds.test` to your hosts file (one-time setup):
```
127.0.0.1 pds.test
```
- **macOS / Linux:** `/etc/hosts`
- **Windows:** `C:\Windows\System32\drivers\etc\hosts`
### Starting the dev environment
```bash
just run-dev
```
This starts the following services via `docker-compose`:
- **Traefik** — HTTPS reverse proxy at `https://pds.test`
- **Backend** — Rust server with `cargo-watch` (auto-rebuilds on file changes)
- **Frontend** — Vite dev server with hot module replacement
- **Postgres** — Database on port 5432
- **PLC Directory** — Local [did-method-plc](https://github.com/did-method-plc/did-method-plc) server for DID registration
- **Mailpit** — Local email server with web UI at [http://localhost:8025](http://localhost:8025)
Once all services are running, open **https://pds.test** in your browser.
### Trusting the self-signed certificate
Traefik generates a self-signed TLS certificate. Your browser will show a security warning on first visit. You can either click through it, or add the certificate to your system trust store for a seamless experience:
**macOS:**
```bash
# Extract the cert from traefik and add it to the system keychain
echo | openssl s_client -connect localhost:443 -servername pds.test 2>/dev/null | openssl x509 > /tmp/pds-test.pem
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /tmp/pds-test.pem
```
**Linux (Debian/Ubuntu):**
```bash
echo | openssl s_client -connect localhost:443 -servername pds.test 2>/dev/null | openssl x509 | sudo tee /usr/local/share/ca-certificates/pds-test.crt
sudo update-ca-certificates
```
**Linux (Fedora/RHEL):**
```bash
echo | openssl s_client -connect localhost:443 -servername pds.test 2>/dev/null | openssl x509 | sudo tee /etc/pki/ca-trust/source/anchors/pds-test.pem
sudo update-ca-trust
```
**Windows (PowerShell as Administrator):**
```powershell
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cert.Import([System.Text.Encoding]::UTF8.GetBytes((echo | openssl s_client -connect localhost:443 -servername pds.test 2>$null | openssl x509)))
$store = New-Object System.Security.Cryptography.X509Certificates.X509Store("Root", "LocalMachine")
$store.Open("ReadWrite")
$store.Add($cert)
$store.Close()
```
Restart your browser after adding the certificate.
### Stopping the dev environment
```bash
# Stop containers (preserves database + build cache)
docker compose --profile dev down
# Stop and wipe all data (fresh start)
docker compose --profile dev down -v
```
### Direct database access
Postgres is exposed on port 5432:
```bash
psql postgres://postgres:postgres@localhost:5432/pds
```
### How it works
- **Source code** is bind-mounted into the containers so that changes made on the host will be immediately reflected in the application
- **Backend** uses `cargo-watch` to recompile and restart when Rust files change
- **Frontend** uses Vite's HMR for instant browser updates when frontend files change
- **Build cache** (`target/` directory and cargo registry) are stored in Docker volumes, so incremental compilation persists across container restarts
- **Traefik** routes `/`, `/xrpc`, `/oauth`, `/.well-known`, `/u`, and `/health` to the backend; everything else goes to the Vite dev server
- **Mailpit** captures all outgoing email — open [http://localhost:8025](http://localhost:8025) to view verification emails during registration
- **PLC Directory** runs locally so DID registration doesn't hit the real `plc.directory`
### Running the backend natively
If you prefer running the Rust backend outside Docker (faster incremental builds on host), you need:
- Rust toolchain (see `rust-toolchain.toml`)
- `protoc` (`brew install protobuf` on macOS)
- PostgreSQL (start with `docker compose up db`)
Then run:
```bash
cargo run -p tranquil-server -- --config config.toml
```
And start the frontend separately:
```bash
cd frontend && pnpm install && pnpm dev
```
Generated
+268 -40
View File
@@ -9,7 +9,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "087113bd50d9adce24850eed5d0476c7d199d532fce8fab5173650331e09033a"
dependencies = [
"abnf-core",
"nom",
"nom 7.1.3",
]
[[package]]
@@ -18,7 +18,7 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c44e09c43ae1c368fb91a03a566472d0087c26cf7e1b9e8e289c14ede681dd7d"
dependencies = [
"nom",
"nom 7.1.3",
]
[[package]]
@@ -210,16 +210,32 @@ version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5493c3bedbacf7fd7382c6346bbd66687d12bbaad3a89a2d2c303ee6cf20b048"
dependencies = [
"asn1-rs-derive",
"asn1-rs-derive 0.5.1",
"asn1-rs-impl",
"displaydoc",
"nom",
"nom 7.1.3",
"num-traits",
"rusticata-macros",
"thiserror 1.0.69",
"time",
]
[[package]]
name = "asn1-rs"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7f43a50ac4fdca5df8e885c21b835997f0a1cdee65494a6847694a98652d9d8"
dependencies = [
"asn1-rs-derive 0.6.0",
"asn1-rs-impl",
"displaydoc",
"nom 7.1.3",
"num-traits",
"rusticata-macros",
"thiserror 2.0.18",
"time",
]
[[package]]
name = "asn1-rs-derive"
version = "0.5.1"
@@ -232,6 +248,18 @@ dependencies = [
"synstructure",
]
[[package]]
name = "asn1-rs-derive"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3109e49b1e4909e9db6515a30c633684d68cdeaa252f215214cb4fa1a5bfee2c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.117",
"synstructure",
]
[[package]]
name = "asn1-rs-impl"
version = "0.2.0"
@@ -1046,7 +1074,7 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08807e080ed7f9d5433fa9b275196cfc35414f66a0c79d864dc51a0d825231a3"
dependencies = [
"bit-vec",
"bit-vec 0.8.0",
]
[[package]]
@@ -1055,6 +1083,15 @@ version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7"
[[package]]
name = "bit-vec"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b71798fca2c1fe1086445a7258a4bc81e6e49dcd24c8d0dd9a1e57395b603f51"
dependencies = [
"serde",
]
[[package]]
name = "bitflags"
version = "2.11.0"
@@ -1970,9 +2007,23 @@ version = "9.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cd0a5c643689626bec213c4d8bd4d96acc8ffdb4ad4bb6bc16abf27d5f4b553"
dependencies = [
"asn1-rs",
"asn1-rs 0.6.2",
"displaydoc",
"nom",
"nom 7.1.3",
"num-bigint",
"num-traits",
"rusticata-macros",
]
[[package]]
name = "der-parser"
version = "10.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07da5016415d5a3c4dd39b11ed26f915f52fc4e0dc197d87908bc916e51bc1a6"
dependencies = [
"asn1-rs 0.7.2",
"displaydoc",
"nom 7.1.3",
"num-bigint",
"num-traits",
"rusticata-macros",
@@ -2216,6 +2267,22 @@ dependencies = [
"zeroize",
]
[[package]]
name = "email-encoding"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9298e6504d9b9e780ed3f7dfd43a61be8cd0e09eb07f7706a945b0072b6670b6"
dependencies = [
"base64 0.22.1",
"memchr",
]
[[package]]
name = "email_address"
version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e079f19b08ca6239f47f8ba8509c11cf3ea30095831f7fed61441475edd8c449"
[[package]]
name = "embedded-io"
version = "0.4.0"
@@ -2818,6 +2885,34 @@ dependencies = [
"tracing",
]
[[package]]
name = "h3"
version = "0.0.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10872b55cfb02a821b69dc7cf8dc6a71d6af25eb9a79662bec4a9d016056b3be"
dependencies = [
"bytes",
"fastrand",
"futures-util",
"http 1.4.0",
"pin-project-lite",
"tokio",
]
[[package]]
name = "h3-quinn"
version = "0.0.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b2e732c8d91a74731663ac8479ab505042fbf547b9a207213ab7fbcbfc4f8b4"
dependencies = [
"bytes",
"futures",
"h3",
"quinn",
"tokio",
"tokio-util",
]
[[package]]
name = "half"
version = "2.7.1"
@@ -3780,6 +3875,37 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "lettre"
version = "0.11.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dabda5859ee7c06b995b9d1165aa52c39110e079ef609db97178d86aeb051fa7"
dependencies = [
"async-trait",
"base64 0.22.1",
"ed25519-dalek",
"email-encoding",
"email_address",
"fastrand",
"futures-io",
"futures-util",
"httpdate",
"idna",
"mime",
"nom 8.0.0",
"percent-encoding",
"quoted_printable",
"rsa",
"rustls 0.23.37",
"sha2",
"socket2 0.6.3",
"tokio",
"tokio-rustls 0.26.4",
"tracing",
"url",
"webpki-roots 1.0.6",
]
[[package]]
name = "libc"
version = "0.2.183"
@@ -4409,6 +4535,15 @@ dependencies = [
"minimal-lexical",
]
[[package]]
name = "nom"
version = "8.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df9761775871bdef83bee530e60050f7e54b1105350d6884eb0fb4f46c2f9405"
dependencies = [
"memchr",
]
[[package]]
name = "nonzero_ext"
version = "0.3.0"
@@ -4558,7 +4693,16 @@ version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a8d8034d9489cdaf79228eb9f6a3b8d7bb32ba00d6645ebd48eef4077ceb5bd9"
dependencies = [
"asn1-rs",
"asn1-rs 0.6.2",
]
[[package]]
name = "oid-registry"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12f40cff3dde1b6087cc5d5f5d4d65712f34016a03ed60e9c08dcc392736b5b7"
dependencies = [
"asn1-rs 0.7.2",
]
[[package]]
@@ -4839,7 +4983,7 @@ checksum = "9114f9c1683dd09c5f4fa024c89fdad783eaae21d3d52dd23ddaaffa29ffb168"
dependencies = [
"either",
"fnv",
"nom",
"nom 7.1.3",
"once_cell",
"postcard",
"quick-xml",
@@ -5128,7 +5272,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37566cb3fdacef14c0737f9546df7cfeadbfbc9fef10991038bf5015d0c80532"
dependencies = [
"bit-set",
"bit-vec",
"bit-vec 0.8.0",
"bitflags",
"num-traits",
"rand 0.9.2",
@@ -5371,6 +5515,7 @@ checksum = "b9e20a958963c291dc322d98411f541009df2ced7b5a4f2bd52337638cfccf20"
dependencies = [
"bytes",
"cfg_aliases",
"futures-io",
"pin-project-lite",
"quinn-proto",
"quinn-udp",
@@ -5427,6 +5572,12 @@ dependencies = [
"proc-macro2",
]
[[package]]
name = "quoted_printable"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "478e0585659a122aa407eb7e3c0e1fa51b1d8a870038bd29f0cf4a8551eea972"
[[package]]
name = "r-efi"
version = "5.3.0"
@@ -5568,6 +5719,20 @@ dependencies = [
"crossbeam-utils",
]
[[package]]
name = "rcgen"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "57f6d249aad744e274e682777a50283a225a32705394ee6d5fcc01efa25e4055"
dependencies = [
"pem",
"ring",
"rustls-pki-types",
"time",
"x509-parser 0.18.1",
"yasna",
]
[[package]]
name = "redis"
version = "1.1.0"
@@ -5833,7 +5998,7 @@ version = "4.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf0c4a6ece9950b9abdb62b1cfcf2a68b3b67a10ba445b3bb85be2a293d0632"
dependencies = [
"nom",
"nom 7.1.3",
]
[[package]]
@@ -6050,6 +6215,16 @@ dependencies = [
"zeroize",
]
[[package]]
name = "secrecy"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e891af845473308773346dc847b2c23ee78fe442e0472ac50e22a18a93d3ae5a"
dependencies = [
"serde",
"zeroize",
]
[[package]]
name = "security-framework"
version = "3.7.0"
@@ -7455,7 +7630,7 @@ dependencies = [
[[package]]
name = "tranquil-api"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"anyhow",
"axum",
@@ -7506,7 +7681,7 @@ dependencies = [
[[package]]
name = "tranquil-auth"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"anyhow",
"base32",
@@ -7529,7 +7704,7 @@ dependencies = [
[[package]]
name = "tranquil-cache"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7543,11 +7718,19 @@ dependencies = [
[[package]]
name = "tranquil-comms"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"base64 0.22.1",
"chrono",
"ed25519-dalek",
"futures",
"hickory-resolver",
"lettre",
"rand 0.8.5",
"reqwest",
"rsa",
"secrecy",
"serde_json",
"sqlx",
"thiserror 2.0.18",
@@ -7561,7 +7744,7 @@ dependencies = [
[[package]]
name = "tranquil-config"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"confique",
"serde",
@@ -7569,7 +7752,7 @@ dependencies = [
[[package]]
name = "tranquil-crypto"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"aes-gcm",
"base64 0.22.1",
@@ -7585,7 +7768,7 @@ dependencies = [
[[package]]
name = "tranquil-db"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"chrono",
@@ -7602,7 +7785,7 @@ dependencies = [
[[package]]
name = "tranquil-db-traits"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7618,7 +7801,7 @@ dependencies = [
[[package]]
name = "tranquil-infra"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"bytes",
@@ -7629,7 +7812,7 @@ dependencies = [
[[package]]
name = "tranquil-lexicon"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"chrono",
"futures",
@@ -7648,7 +7831,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"anyhow",
"axum",
@@ -7671,7 +7854,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth-server"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"axum",
"base64 0.22.1",
@@ -7704,7 +7887,7 @@ dependencies = [
[[package]]
name = "tranquil-pds"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"aes-gcm",
"anyhow",
@@ -7744,6 +7927,7 @@ dependencies = [
"multibase",
"multihash",
"p256 0.13.2",
"parking_lot",
"rand 0.8.5",
"redis",
"regex",
@@ -7756,6 +7940,7 @@ dependencies = [
"sha2",
"sqlx",
"subtle",
"tempfile",
"testcontainers",
"testcontainers-modules",
"thiserror 2.0.18",
@@ -7796,7 +7981,7 @@ dependencies = [
[[package]]
name = "tranquil-repo"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"bytes",
"cid",
@@ -7808,7 +7993,7 @@ dependencies = [
[[package]]
name = "tranquil-ripple"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"backon",
@@ -7833,7 +8018,7 @@ dependencies = [
[[package]]
name = "tranquil-scopes"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"axum",
"futures",
@@ -7849,15 +8034,30 @@ dependencies = [
[[package]]
name = "tranquil-server"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"arc-swap",
"axum",
"bytes",
"clap",
"dotenvy",
"ed25519-dalek",
"futures-util",
"h3",
"h3-quinn",
"hex",
"http 1.4.0",
"hyper 1.8.1",
"hyper-util",
"quinn",
"rcgen",
"rustls 0.23.37",
"rustls-pemfile",
"thiserror 2.0.18",
"tokio",
"tokio-rustls 0.26.4",
"tokio-util",
"tower",
"tracing",
"tracing-subscriber",
"tranquil-api",
@@ -7870,7 +8070,7 @@ dependencies = [
[[package]]
name = "tranquil-signal"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"chrono",
@@ -7893,7 +8093,7 @@ dependencies = [
[[package]]
name = "tranquil-storage"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"aws-config",
@@ -7910,7 +8110,7 @@ dependencies = [
[[package]]
name = "tranquil-store"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"async-trait",
"bytes",
@@ -7959,7 +8159,7 @@ dependencies = [
[[package]]
name = "tranquil-sync"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"anyhow",
"axum",
@@ -7981,7 +8181,7 @@ dependencies = [
[[package]]
name = "tranquil-types"
version = "0.5.7"
version = "0.6.5"
dependencies = [
"chrono",
"cid",
@@ -8494,9 +8694,9 @@ checksum = "15784340a24c170ce60567282fb956a0938742dbfbf9eff5df793a686a009b8b"
dependencies = [
"base64 0.21.7",
"base64urlsafedata",
"der-parser",
"der-parser 9.0.0",
"hex",
"nom",
"nom 7.1.3",
"openssl",
"openssl-sys",
"rand 0.9.2",
@@ -8510,7 +8710,7 @@ dependencies = [
"uuid",
"webauthn-attestation-ca",
"webauthn-rs-proto",
"x509-parser",
"x509-parser 0.16.0",
]
[[package]]
@@ -9064,17 +9264,35 @@ version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fcbc162f30700d6f3f82a24bf7cc62ffe7caea42c0b2cba8bf7f3ae50cf51f69"
dependencies = [
"asn1-rs",
"asn1-rs 0.6.2",
"data-encoding",
"der-parser",
"der-parser 9.0.0",
"lazy_static",
"nom",
"oid-registry",
"nom 7.1.3",
"oid-registry 0.7.1",
"rusticata-macros",
"thiserror 1.0.69",
"time",
]
[[package]]
name = "x509-parser"
version = "0.18.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d43b0f71ce057da06bc0851b23ee24f3f86190b07203dd8f567d0b706a185202"
dependencies = [
"asn1-rs 0.7.2",
"data-encoding",
"der-parser 10.0.0",
"lazy_static",
"nom 7.1.3",
"oid-registry 0.8.1",
"ring",
"rusticata-macros",
"thiserror 2.0.18",
"time",
]
[[package]]
name = "xattr"
version = "1.6.1"
@@ -9103,6 +9321,16 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfe53a6657fd280eaa890a3bc59152892ffa3e30101319d168b781ed6529b049"
[[package]]
name = "yasna"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5f6765e852b9b4dc8e2a76843e4d64d1cea8e79bcde0b6901aea8e7c7f08282"
dependencies = [
"bit-vec 0.9.1",
"time",
]
[[package]]
name = "yoke"
version = "0.8.1"
+15 -2
View File
@@ -26,7 +26,7 @@ members = [
]
[workspace.package]
version = "0.5.7"
version = "0.6.5"
edition = "2024"
license = "AGPL-3.0-or-later"
@@ -59,6 +59,7 @@ presage = { git = "https://github.com/whisperfish/presage", rev = "fe3ed54c4844a
unicode-segmentation = "1"
aes-gcm = "0.10"
arc-swap = "1"
backon = "1"
bincode = { version = "2", features = ["serde"] }
anyhow = "1.0"
@@ -81,11 +82,15 @@ foca = { version = "1", features = ["bincode-codec", "tracing"] }
futures = "0.3"
futures-util = "0.3"
governor = "0.10"
h3 = "0.0.8"
h3-quinn = "0.0.10"
hex = "0.4"
hickory-resolver = { version = "0.24", features = ["tokio-runtime"] }
hkdf = "0.12"
hmac = "0.12"
http = "1.4"
hyper = { version = "1", features = ["server", "http1", "http2"] }
hyper-util = { version = "0.1", features = ["server", "server-auto", "server-graceful", "service", "tokio"] }
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "gif", "webp"] }
qrcodegen = "1.8"
infer = "0.19"
@@ -93,6 +98,7 @@ ipld-core = "0.4"
iroh-car = "0.5"
jacquard-common = { version = "0.9", features = ["crypto-k256"] }
jacquard-repo = "0.9"
lettre = { version = "0.11", default-features = false, features = ["builder", "smtp-transport", "tokio1", "tokio1-rustls-tls", "pool", "dkim", "tracing"] }
jsonwebtoken = { version = "10.2", features = ["rust_crypto"] }
k256 = { version = "0.13", features = ["ecdsa", "pem", "pkcs8"] }
metrics = "0.24"
@@ -102,9 +108,15 @@ parking_lot = "0.12"
multihash = "0.19"
p256 = { version = "0.13", features = ["ecdsa"] }
p384 = { version = "0.13", features = ["ecdsa"] }
quinn = { version = "0.11", default-features = false, features = ["runtime-tokio", "rustls-ring", "log"] }
rand = "0.8"
rcgen = { version = "0.14", default-features = false, features = ["ring", "pem"] }
redis = { version = "1.0", features = ["tokio-comp", "connection-manager"] }
regex = "1"
rsa = "0.9"
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12", "logging"] }
rustls-pemfile = "2"
secrecy = { version = "0.10", features = ["serde"] }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls-webpki-roots", "http2", "charset", "macos-system-configuration"] }
serde = { version = "1.0", features = ["derive"] }
serde_bytes = "0.11"
@@ -116,8 +128,9 @@ sqlx = { version = "0.8", features = ["runtime-tokio-rustls", "postgres", "uuid"
subtle = "2.5"
thiserror = "2.0"
tokio = { version = "1.48", features = ["macros", "rt-multi-thread", "time", "signal", "process", "io-util", "fs"] }
tokio-util = "0.7.18"
tokio-util = { version = "0.7.18", features = ["rt"] }
tokio-tungstenite = { version = "0.28", features = ["rustls-tls-webpki-roots"] }
tokio-rustls = { version = "0.26", default-features = false, features = ["ring", "tls12", "logging"] }
totp-rs = { version = "5", features = ["qr"] }
tower = "0.5"
tower-http = { version = "0.6", features = ["fs", "cors"] }
+5 -6
View File
@@ -1,12 +1,12 @@
FROM node:24-alpine AS frontend
RUN corepack enable && corepack prepare pnpm@latest --activate
WORKDIR /app
COPY frontend/package.json frontend/pnpm-lock.yaml ./
COPY frontend/package.json frontend/pnpm-lock.yaml frontend/pnpm-workspace.yaml ./
RUN pnpm install --frozen-lockfile
COPY frontend/ ./
RUN pnpm build
FROM rust:1.92-alpine AS builder
FROM rust:1.96-alpine AS builder
RUN apk add --no-cache ca-certificates musl-dev pkgconfig openssl-dev openssl-libs-static mold clang protoc
ENV RUSTFLAGS="-C linker=clang -C link-arg=-fuse-ld=mold"
WORKDIR /app
@@ -35,7 +35,7 @@ COPY crates/tranquil-oauth-server ./crates/tranquil-oauth-server
COPY crates/tranquil-store ./crates/tranquil-store
COPY crates/tranquil-signal ./crates/tranquil-signal
COPY crates/tranquil-server ./crates/tranquil-server
COPY migrations ./crates/tranquil-pds/migrations
COPY migrations ./migrations
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/app/target \
if [ "$SLIM" = "true" ]; then \
@@ -46,12 +46,11 @@ RUN --mount=type=cache,target=/usr/local/cargo/registry \
cp target/release/tranquil-server /tmp/tranquil-pds
FROM alpine:3.23
RUN apk add --no-cache msmtp ca-certificates \
&& ln -sf /usr/bin/msmtp /usr/sbin/sendmail
RUN apk add --no-cache ca-certificates
COPY --from=builder /tmp/tranquil-pds /usr/local/bin/tranquil-pds
COPY --from=frontend /app/dist /var/lib/tranquil-pds/frontend
WORKDIR /app
ENV SERVER_HOST=0.0.0.0
ENV SERVER_HOST=[::]
ENV SERVER_PORT=3000
EXPOSE 3000
CMD ["tranquil-pds"]
+20
View File
@@ -0,0 +1,20 @@
FROM rust:1.96-alpine
RUN apk add --no-cache \
ca-certificates \
musl-dev \
pkgconfig \
openssl-dev \
openssl-libs-static \
mold \
clang \
protoc
RUN cargo install cargo-watch
ENV RUSTFLAGS="-C linker=clang -C link-arg=-fuse-ld=mold"
ENV SQLX_OFFLINE=true
WORKDIR /app
CMD ["cargo", "watch", "-x", "run -p tranquil-server -- --config /app/config.dev.toml"]
+14 -8
View File
@@ -6,7 +6,7 @@ A Personal Data Server for the AT Protocol.
We came together to make this PDS to enable and empower our users to better host their data on this shared protocol. All of our decisions as a project are guided by their usefulness to the community: PDS hosters and end-users both.
Comparatively: Bluesky the company created a "reference PDS" that we can self-host quite easily, and that's great, but Bluesky has an incentive to make software for themselvess first & foremost, then secondly their software can be useful for us self-hosters. In contrast, Tranquil is not from a company, and will never be.
Comparatively: Bluesky the company created a "reference PDS" that we can self-host quite easily, and that's great, but Bluesky has an incentive to make software for themselves first & foremost, then secondly their software can be useful for us self-hosters. In contrast, Tranquil is not from a company, and will never be.
## What's different about Tranquil PDS
@@ -20,7 +20,7 @@ It is a superset of the reference PDS, including:
- account delegation: letting others manage an account with configurable permission levels
- a built-in web UI for account management, repo browsing, and admin
Unlike the ref PDS, Tranquil itself is compiled to a single binary with no nodeJS runtime. However, at time of writing, Tranquil requires postgres running separately.
Unlike the ref PDS, Tranquil is a single binary with no nodejs runtime. That said, at time of writing, Tranquil does require postgres running separately.
## Quick Start
@@ -46,24 +46,28 @@ just test
just lint
```
Nix users can enter a devshell with `nix develop`, or `direnv allow` to auto-enter via the bundled `.envrc`. Pre-built artifacts (including the devshell) are available from our [binary cache](docs/install-nix.md#binary-cache).
Nix users can enter a devshell with `nix develop`, or `direnv allow` to auto-enter via the bundled `.envrc`. Pre-built artifacts including the devshell are available from our [binary cache](docs/2_INSTALL_NIX.md#binary-cache).
## Production Deployment
### Quick Deploy (Docker/Podman Compose)
Edit `config.toml` with your values. Generate secrets with `openssl rand -base64 48`.
`docker-compose.prod.yaml` pulls the prebuilt image `atcr.io/tranquil.farm/tranquil-pds:latest`. Sign in to the registry first with `podman login atcr.io`. The Containers guide covers building from source.
```bash
cp example.toml config.toml
```
Edit `config.toml` with your values and generate secrets with `openssl rand -base64 48`. Set the postgres password to match `docker-compose.prod.yaml`. nginx needs a TLS certificate before it starts, so follow the wildcard cert steps in the [Containers guide](docs/2_INSTALL_CONTAINERS.md).
```bash
podman-compose -f docker-compose.prod.yaml up -d
```
### Installation Guides
- [Nix](docs/install-nix.md)
- [Containers](docs/install-containers.md)
- [Kubernetes](docs/install-kubernetes.md)
- [Nix](docs/2_INSTALL_NIX.md)
- [Containers](docs/2_INSTALL_CONTAINERS.md)
## Community
@@ -76,7 +80,7 @@ We currently don't have a shared space to chat and organize Tranquil things, but
- [@oyster.cafe](https://tangled.org/did:plc:3fwecdnvtcscjnrx2p4n7alz)
- [@nel.pet](https://tangled.org/did:plc:h5wsnqetncv6lu2weom35lg2)
### Amazing contributers
### Amazing contributors
- [@isabelroses.com](https://tangled.org/did:plc:qxichs7jsycphrsmbujwqbfb)
- [@quilling.dev](https://tangled.org/did:plc:jrtgsidnmxaen4offglr5lsh)
@@ -87,6 +91,8 @@ We currently don't have a shared space to chat and organize Tranquil things, but
- [@a.starrysky.fyi](https://tangled.org/did:plc:uuyqs6y3pwtbteet4swt5i5y)
- [@sans-self.org](https://tangled.org/did:plc:wydyrngmxbcsqdvhmd7whmye)
- [@tachyonism.tngl.sh](https://tangled.org/did:plc:w6qiwij62bmdugsd3gemhpy2)
- [@trezy.codes](https://tangled.org/did:plc:4jrld6fwpnwqehtce56qshzv)
- [@ave.zone](https://tangled.org/did:plc:mchrltkrhuzpxleiwpmmvpar)
- Could be your name here too!
### Tranquil PDS instances in the wild!
+30
View File
@@ -0,0 +1,30 @@
[server]
hostname = "pds.test"
allow_http_proxy = true
invite_code_required = false
disable_rate_limiting = true
[frontend]
enabled = true
dir = "/app/frontend/public"
[database]
url = "postgres://postgres:postgres@db:5432/pds"
[storage]
path = "/var/lib/tranquil-pds/blobs"
[plc]
directory_url = "http://plc:2582"
[email]
from_address = "noreply@pds.test"
from_name = "Tranquil PDS (Dev)"
[email.smarthost]
host = "mailpit"
port = 1025
tls = "none"
[secrets]
allow_insecure = true
+1 -3
View File
@@ -9,7 +9,6 @@ use tranquil_pds::state::AppState;
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct SignalStatusOutput {
pub enabled: bool,
pub linked: bool,
}
@@ -23,13 +22,12 @@ pub async fn get_signal_status(
State(state): State<AppState>,
_auth: Auth<Admin>,
) -> Result<Json<SignalStatusOutput>, ApiError> {
let enabled = tranquil_config::get().signal.enabled;
let linked = match &state.signal_sender {
Some(slot) => slot.is_linked().await,
None => false,
};
Ok(Json(SignalStatusOutput { enabled, linked }))
Ok(Json(SignalStatusOutput { linked }))
}
pub async fn link_signal_device(
@@ -510,6 +510,7 @@ pub async fn create_account(
telegram_username: comms.telegram,
signal_username: comms.signal,
deactivated_at,
inbound_migration: is_migration || is_did_web_byod,
encrypted_key_bytes: repo.encrypted_key_bytes,
encryption_version: tranquil_pds::config::ENCRYPTION_VERSION,
reserved_key_id,
+1 -1
View File
@@ -3,5 +3,5 @@ mod sign;
mod submit;
pub use request::request_plc_operation_signature;
pub use sign::{ServiceInput, SignPlcOperationInput, SignPlcOperationOutput, sign_plc_operation};
pub use sign::{SignPlcOperationInput, SignPlcOperationOutput, sign_plc_operation};
pub use submit::{SubmitPlcOperationInput, submit_plc_operation};
@@ -6,10 +6,6 @@ use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{Auth, Permissive};
use tranquil_pds::state::AppState;
fn generate_plc_token() -> String {
tranquil_pds::util::generate_token_code()
}
pub async fn request_plc_operation_signature(
State(state): State<AppState>,
auth: Auth<Permissive>,
@@ -28,12 +24,13 @@ pub async fn request_plc_operation_signature(
.ok_or(ApiError::AccountNotFound)?;
let _ = state.repos.infra.delete_plc_tokens_for_user(user_id).await;
let plc_token = generate_plc_token();
let display_token = tranquil_pds::util::generate_token_code();
let stored_token = tranquil_pds::util::normalize_token_code(&display_token);
let expires_at = Utc::now() + Duration::minutes(10);
state
.repos
.infra
.insert_plc_token(user_id, &plc_token, expires_at)
.insert_plc_token(user_id, &stored_token, expires_at)
.await
.log_db_err("creating PLC token")?;
@@ -42,7 +39,7 @@ pub async fn request_plc_operation_signature(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
user_id,
&plc_token,
&display_token,
hostname,
)
.await
+5 -24
View File
@@ -9,7 +9,7 @@ use tranquil_pds::api::ApiError;
use tranquil_pds::api::error::DbResultExt;
use tranquil_pds::auth::{Auth, Permissive};
use tranquil_pds::circuit_breaker::with_circuit_breaker;
use tranquil_pds::plc::{PlcError, PlcService, ServiceType, create_update_op, sign_operation};
use tranquil_pds::plc::{PlcError, PlcService, create_update_op, sign_operation};
use tranquil_pds::state::AppState;
#[derive(Debug, Deserialize)]
@@ -19,14 +19,7 @@ pub struct SignPlcOperationInput {
pub rotation_keys: Option<Vec<String>>,
pub also_known_as: Option<Vec<String>>,
pub verification_methods: Option<HashMap<String, String>>,
pub services: Option<HashMap<String, ServiceInput>>,
}
#[derive(Debug, Deserialize, Clone)]
pub struct ServiceInput {
#[serde(rename = "type")]
pub service_type: ServiceType,
pub endpoint: String,
pub services: Option<HashMap<String, PlcService>>,
}
#[derive(Debug, Serialize)]
@@ -50,9 +43,10 @@ pub async fn sign_plc_operation(
"PLC operations are only valid for did:plc identities".into(),
));
}
let token = input.token.as_ref().ok_or_else(|| {
let raw_token = input.token.as_ref().ok_or_else(|| {
ApiError::InvalidRequest("Email confirmation token required to sign PLC operations".into())
})?;
let token = &tranquil_pds::util::normalize_token_code(raw_token);
let user_id = state
.repos
@@ -106,25 +100,12 @@ pub async fn sign_plc_operation(
if last_op.is_tombstone() {
return Err(ApiError::from(PlcError::Tombstoned));
}
let services = input.services.map(|s| {
s.into_iter()
.map(|(k, v)| {
(
k,
PlcService {
service_type: v.service_type,
endpoint: v.endpoint,
},
)
})
.collect()
});
let unsigned_op = create_update_op(
&last_op,
input.rotation_keys,
input.verification_methods,
input.also_known_as,
services,
input.services,
)
.map_err(|e| match e {
PlcError::Tombstoned => ApiError::InvalidRequest("Cannot update tombstoned DID".into()),
@@ -129,20 +129,13 @@ pub async fn submit_plc_operation(
.await
.map_err(ApiError::from)?;
match state
if let Err(e) = state
.repos
.repo
.insert_identity_event(did, Some(&user.handle))
.await
{
Ok(seq) => {
if let Err(e) = state.repos.repo.notify_update(seq).await {
warn!("Failed to notify identity event: {:?}", e);
}
}
Err(e) => {
warn!("Failed to sequence identity event: {:?}", e);
}
warn!("Failed to sequence identity event: {:?}", e);
}
let _ = state
.cache
+12 -2
View File
@@ -15,8 +15,12 @@ pub mod temp;
use tranquil_pds::state::AppState;
pub fn api_routes() -> axum::Router<AppState> {
use axum::extract::DefaultBodyLimit;
use axum::routing::{get, post};
let blob_body_limit =
DefaultBodyLimit::max(tranquil_config::get().server.max_blob_size as usize);
axum::Router::new()
.route("/_health", get(server::health))
.route(
@@ -68,7 +72,10 @@ pub fn api_routes() -> axum::Router<AppState> {
.route("/com.atproto.repo.deleteRecord", post(repo::delete_record))
.route("/com.atproto.repo.listRecords", get(repo::list_records))
.route("/com.atproto.repo.describeRepo", get(repo::describe_repo))
.route("/com.atproto.repo.uploadBlob", post(repo::upload_blob))
.route(
"/com.atproto.repo.uploadBlob",
post(repo::upload_blob).layer(blob_body_limit),
)
.route("/com.atproto.repo.applyWrites", post(repo::apply_writes))
.route(
"/com.atproto.server.checkAccountStatus",
@@ -247,7 +254,10 @@ pub fn api_routes() -> axum::Router<AppState> {
"/_identity.verifyHandleOwnership",
post(identity::verify_handle_ownership),
)
.route("/com.atproto.repo.importRepo", post(repo::import_repo))
.route(
"/com.atproto.repo.importRepo",
post(repo::import_repo).layer(blob_body_limit),
)
.route(
"/com.atproto.admin.deleteAccount",
post(admin::delete_account),
+8 -78
View File
@@ -77,27 +77,6 @@ pub async fn import_repo(
blocks.len(),
root
);
let Some(root_block) = blocks.get(&root) else {
return Err(ApiError::InvalidRequest(
"Root block not found in CAR file".into(),
));
};
let commit_did: Did = match jacquard_repo::commit::Commit::from_cbor(root_block) {
Ok(commit) => commit
.did()
.as_str()
.parse()
.map_err(|_| ApiError::InvalidRequest("Commit contains invalid DID".into()))?,
Err(e) => {
return Err(ApiError::InvalidRequest(format!("Invalid commit: {}", e)));
}
};
if commit_did != *did {
return Err(ApiError::InvalidRepo(format!(
"CAR file is for DID {} but you are authenticated as {}",
commit_did, did
)));
}
let skip_verification = std::env::var("SKIP_IMPORT_VERIFICATION")
.ok()
.map(|v| v == "true" || v == "1")
@@ -106,13 +85,15 @@ pub async fn import_repo(
.map(|c| c.import.skip_verification)
.unwrap_or(false)
});
let is_migration = user.deactivated_at.is_some();
let is_migration = user.inbound_migration && user.deactivated_at.is_some();
if skip_verification {
warn!("Skipping all CAR verification for import (SKIP_IMPORT_VERIFICATION=true)");
} else if is_migration {
debug!("Verifying CAR file structure for migration (skipping signature verification)");
warn!("Skipping all CAR verification for repo import (SKIP_IMPORT_VERIFICATION=true)");
} else {
debug!(
"Verifying CAR file structure for repo import (skipping signature and DID verification)"
);
let verifier = CarVerifier::new();
match verifier.verify_car_structure_only(did, &root, &blocks) {
match verifier.verify_car_structure_only(&root, &blocks) {
Ok(verified) => {
debug!(
"CAR structure verification successful: rev={}, data_cid={}",
@@ -142,56 +123,6 @@ pub async fn import_repo(
)));
}
}
} else {
debug!("Verifying CAR file signature and structure for DID {}", did);
let verifier = CarVerifier::new();
match verifier.verify_car(did, &root, &blocks).await {
Ok(verified) => {
debug!(
"CAR verification successful: rev={}, data_cid={}",
verified.rev, verified.data_cid
);
}
Err(tranquil_pds::sync::verify::VerifyError::DidMismatch {
commit_did,
expected_did,
}) => {
return Err(ApiError::InvalidRepo(format!(
"CAR file is for DID {} but you are authenticated as {}",
commit_did, expected_did
)));
}
Err(tranquil_pds::sync::verify::VerifyError::InvalidSignature) => {
return Err(ApiError::InvalidRequest(
"CAR file commit signature verification failed".into(),
));
}
Err(tranquil_pds::sync::verify::VerifyError::DidResolutionFailed(msg)) => {
warn!("DID resolution failed during import verification: {}", msg);
return Err(ApiError::InvalidRequest(format!(
"Failed to verify DID: {}",
msg
)));
}
Err(tranquil_pds::sync::verify::VerifyError::NoSigningKey) => {
return Err(ApiError::InvalidRequest(
"DID document does not contain a signing key".into(),
));
}
Err(tranquil_pds::sync::verify::VerifyError::MstValidationFailed(msg)) => {
return Err(ApiError::InvalidRequest(format!(
"MST validation failed: {}",
msg
)));
}
Err(e) => {
error!("CAR verification error: {:?}", e);
return Err(ApiError::InvalidRequest(format!(
"CAR verification failed: {}",
e
)));
}
}
}
let max_blocks = tranquil_config::get().import.max_blocks as usize;
let _write_lock = state.repo_write_locks.lock(user_id).await;
@@ -417,7 +348,6 @@ async fn sequence_import_event(
rev: None,
};
let seq = state.repos.repo.insert_commit_event(&data).await?;
state.repos.repo.notify_update(seq).await?;
state.repos.repo.insert_commit_event(&data).await?;
Ok(())
}
+61 -45
View File
@@ -1,6 +1,6 @@
use super::validation::validate_record_with_status;
use super::validation_mode::{ValidationMode, deserialize_validation_mode};
use crate::repo::record::write::CommitInfo;
use crate::repo::record::write::{CommitInfo, ensure_record_type};
use axum::{Json, extract::State};
use jacquard_repo::{mst::Mst, storage::BlockStore};
use serde::{Deserialize, Serialize};
@@ -14,8 +14,8 @@ use tranquil_pds::auth::{
};
use tranquil_pds::repo::TrackingBlockStore;
use tranquil_pds::repo_ops::{
FinalizeParams, RecordOp, begin_repo_write, extract_backlinks, extract_blob_cids,
finalize_repo_write,
CommitResult, FinalizeParams, RecordOp, begin_repo_write, extract_backlinks, extract_blob_cids,
finalize_repo_write, with_repair_retry,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AtIdentifier, AtUri, Did, Nsid, Rkey};
@@ -54,6 +54,8 @@ async fn process_single_write(
rkey,
value,
} => {
let value = ensure_record_type(value, collection);
let value = &*value;
let validation_status = if validate.should_skip() {
None
} else {
@@ -72,7 +74,7 @@ async fn process_single_write(
if mst
.get(&key)
.await
.map_err(|e| ApiError::InternalError(Some(format!("Failed to read MST: {e}"))))?
.map_err(|e| ApiError::from_mst_error("read MST for applyWrites create", &e))?
.is_some()
{
return Err(ApiError::InvalidRequest(format!(
@@ -90,7 +92,7 @@ async fn process_single_write(
let new_mst = mst
.add(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to add to MST".into())))?;
.map_err(|e| ApiError::from_mst_error("add record to MST", &e))?;
let uri = AtUri::from_parts(did, collection, &rkey);
backlinks_to_add.extend(extract_backlinks(&uri, value));
results.push(WriteResult::CreateResult {
@@ -117,6 +119,8 @@ async fn process_single_write(
rkey,
value,
} => {
let value = ensure_record_type(value, collection);
let value = &*value;
let validation_status = if validate.should_skip() {
None
} else {
@@ -134,9 +138,7 @@ async fn process_single_write(
let prev_record_cid = mst
.get(&key)
.await
.map_err(|e| {
ApiError::InternalError(Some(format!("Failed to read prev record: {}", e)))
})?
.map_err(|e| ApiError::from_mst_error("read update target from MST", &e))?
.ok_or_else(|| {
ApiError::InvalidRequest("Update target record does not exist".into())
})?;
@@ -151,7 +153,7 @@ async fn process_single_write(
let new_mst = mst
.update(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to update MST".into())))?;
.map_err(|e| ApiError::from_mst_error("update record in MST", &e))?;
let uri = AtUri::from_parts(did, collection, rkey);
backlinks_to_remove.push(uri.clone());
backlinks_to_add.extend(extract_backlinks(&uri, value));
@@ -180,16 +182,14 @@ async fn process_single_write(
let prev_record_cid = mst
.get(&key)
.await
.map_err(|e| {
ApiError::InternalError(Some(format!("Failed to read prev record: {}", e)))
})?
.map_err(|e| ApiError::from_mst_error("read delete target from MST", &e))?
.ok_or_else(|| {
ApiError::InvalidRequest("Delete target record does not exist".into())
})?;
let new_mst = mst
.delete(&key)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to delete from MST".into())))?;
.map_err(|e| ApiError::from_mst_error("delete record from MST", &e))?;
backlinks_to_remove.push(AtUri::from_parts(did, collection, rkey));
results.push(WriteResult::DeleteResult {});
ops.push(RecordOp::Delete {
@@ -232,6 +232,45 @@ async fn process_writes(
.await
}
async fn execute_apply_writes(
state: &AppState,
user_id: uuid::Uuid,
did: &Did,
input: &ApplyWritesInput,
controller_did: Option<&Did>,
write_summary: Option<serde_json::Value>,
) -> Result<(CommitResult, Vec<WriteResult>), ApiError> {
let (ctx, mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let WriteAccumulator {
mst: final_mst,
results,
ops,
all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
} = process_writes(&input.writes, mst, did, input.validate, &ctx.tracking_store).await?;
let commit_result = finalize_repo_write(
state,
ctx,
final_mst,
FinalizeParams {
did,
user_id,
controller_did,
delegation_detail: write_summary,
ops,
blob_cids: &all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
},
)
.await?;
Ok((commit_result, results))
}
#[derive(Deserialize)]
#[serde(tag = "$type")]
pub enum WriteOp {
@@ -346,24 +385,6 @@ pub async fn apply_writes(
.log_db_err("fetching user for batch write")?
.ok_or(ApiError::InternalError(Some("User not found".into())))?;
let (ctx, mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let WriteAccumulator {
mst: final_mst,
results,
ops,
all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
} = process_writes(
&input.writes,
mst,
&did,
input.validate,
&ctx.tracking_store,
)
.await?;
let write_summary: Option<serde_json::Value> = controller_did.as_ref().map(|_| {
let writes: Vec<serde_json::Value> = input
.writes
@@ -397,21 +418,16 @@ pub async fn apply_writes(
})
});
let commit_result = finalize_repo_write(
&state,
ctx,
final_mst,
FinalizeParams {
did: &did,
let (commit_result, results) = with_repair_retry(&state, user_id, || {
execute_apply_writes(
&state,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: write_summary,
ops,
blob_cids: &all_blob_cids,
backlinks_to_add,
backlinks_to_remove,
},
)
&did,
&input,
controller_did.as_ref(),
write_summary.clone(),
)
})
.await?;
Ok(Json(ApplyWritesOutput {
+40 -21
View File
@@ -4,13 +4,15 @@ use cid::Cid;
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::str::FromStr;
use tracing::error;
use tranquil_pds::api::error::ApiError;
use tranquil_pds::auth::{Active, Auth, VerifyScope};
use tranquil_pds::cid_types::RecordCid;
use tranquil_pds::repo_ops::{FinalizeParams, RecordOp, begin_repo_write, finalize_repo_write};
use tranquil_pds::repo_ops::{
FinalizeParams, RecordOp, begin_repo_write, finalize_repo_write, with_repair_retry,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AtIdentifier, AtUri, Nsid, Rkey};
use tranquil_pds::types::{AtIdentifier, AtUri, Did, Nsid, Rkey};
use uuid::Uuid;
#[derive(Deserialize)]
pub struct DeleteRecordInput {
@@ -41,13 +43,30 @@ pub async fn delete_record(
let user_id = repo_auth.user_id;
let controller_did = repo_auth.controller_did;
let (ctx, mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let out = with_repair_retry(&state, user_id, || {
delete_record_inner(&state, &did, user_id, controller_did.as_ref(), &input)
})
.await?;
Ok(Json(out))
}
async fn delete_record_inner(
state: &AppState,
did: &Did,
user_id: Uuid,
controller_did: Option<&Did>,
input: &DeleteRecordInput,
) -> Result<DeleteRecordOutput, ApiError> {
let (ctx, mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let key = format!("{}/{}", input.collection, input.rkey);
if let Some(swap_record_str) = &input.swap_record {
let expected_cid = Cid::from_str(swap_record_str).ok();
let actual_cid = mst.get(&key).await.ok().flatten();
let actual_cid = mst
.get(&key)
.await
.map_err(|e| ApiError::from_mst_error("read swap target from MST", &e))?;
if expected_cid != actual_cid {
return Err(ApiError::InvalidSwap(Some(
"Record has been modified or does not exist".into(),
@@ -55,18 +74,18 @@ pub async fn delete_record(
}
}
let prev_record_cid = mst.get(&key).await.map_err(|e| {
error!("Failed to read prev record from MST: {}", e);
ApiError::InternalError(Some("Failed to read MST".into()))
})?;
let prev_record_cid = mst
.get(&key)
.await
.map_err(|e| ApiError::from_mst_error("read prev record from MST", &e))?;
let Some(prev_record_cid) = prev_record_cid else {
return Ok(Json(DeleteRecordOutput { commit: None }));
return Ok(DeleteRecordOutput { commit: None });
};
let new_mst = mst.delete(&key).await.map_err(|e| {
error!("Failed to delete from MST: {}", e);
ApiError::InternalError(Some("Failed to delete from MST".into()))
})?;
let new_mst = mst
.delete(&key)
.await
.map_err(|e| ApiError::from_mst_error("delete record from MST", &e))?;
let op = RecordOp::Delete {
collection: input.collection.clone(),
@@ -74,17 +93,17 @@ pub async fn delete_record(
prev: RecordCid::from(prev_record_cid),
};
let deleted_uri = AtUri::from_parts(&did, &input.collection, &input.rkey);
let deleted_uri = AtUri::from_parts(did, &input.collection, &input.rkey);
let commit_result = finalize_repo_write(
&state,
state,
ctx,
new_mst,
FinalizeParams {
did: &did,
did,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: controller_did.as_ref().map(|_| {
controller_did,
delegation_detail: controller_did.map(|_| {
json!({
"action": "delete",
"collection": input.collection,
@@ -99,10 +118,10 @@ pub async fn delete_record(
)
.await?;
Ok(Json(DeleteRecordOutput {
Ok(DeleteRecordOutput {
commit: Some(CommitInfo {
cid: commit_result.commit_cid.to_string(),
rev: commit_result.rev,
}),
}))
})
}
+100 -45
View File
@@ -5,8 +5,8 @@ use cid::Cid;
use jacquard_repo::storage::BlockStore;
use serde::{Deserialize, Serialize};
use serde_json::json;
use std::borrow::Cow;
use std::str::FromStr;
use tracing::error;
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{
Active, Auth, AuthSource, RepoScopeAction, ScopeVerified, VerifyScope, require_not_migrated,
@@ -14,7 +14,7 @@ use tranquil_pds::auth::{
};
use tranquil_pds::repo_ops::{
FinalizeParams, RecordOp, begin_repo_write, extract_backlinks, extract_blob_cids,
finalize_repo_write,
finalize_repo_write, with_repair_retry,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AtIdentifier, AtUri, Did, Nsid, Rkey};
@@ -62,6 +62,29 @@ pub async fn prepare_repo_write<A: RepoScopeAction>(
})
}
pub(crate) fn ensure_record_type<'a>(
record: &'a serde_json::Value,
collection: &Nsid,
) -> Cow<'a, serde_json::Value> {
let serde_json::Value::Object(map) = record else {
return Cow::Borrowed(record);
};
let needs_fill = match map.get("$type") {
None | Some(serde_json::Value::Null) => true,
Some(serde_json::Value::String(existing)) => existing.is_empty(),
Some(_) => false,
};
if !needs_fill {
return Cow::Borrowed(record);
}
let mut map = map.clone();
map.insert(
"$type".to_string(),
serde_json::Value::String(collection.to_string()),
);
Cow::Owned(serde_json::Value::Object(map))
}
#[derive(Deserialize)]
#[allow(dead_code)]
pub struct CreateRecordInput {
@@ -95,15 +118,32 @@ pub struct CreateRecordOutput {
pub async fn create_record(
State(state): State<AppState>,
auth: Auth<Active>,
Json(input): Json<CreateRecordInput>,
Json(mut input): Json<CreateRecordInput>,
) -> Result<Json<CreateRecordOutput>, ApiError> {
if let Cow::Owned(record) = ensure_record_type(&input.record, &input.collection) {
input.record = record;
}
let scope_proof = auth.verify_repo_create(&input.collection)?;
let repo_auth = prepare_repo_write(&state, &scope_proof, &input.repo).await?;
let did = repo_auth.did;
let user_id = repo_auth.user_id;
let controller_did = repo_auth.controller_did;
let (ctx, mut mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let out = with_repair_retry(&state, user_id, || {
create_record_inner(&state, &did, user_id, controller_did.as_ref(), &input)
})
.await?;
Ok(Json(out))
}
async fn create_record_inner(
state: &AppState,
did: &Did,
user_id: Uuid,
controller_did: Option<&Did>,
input: &CreateRecordInput,
) -> Result<CreateRecordOutput, ApiError> {
let (ctx, mut mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let validation_status = if input.validate.should_skip() {
None
@@ -119,12 +159,12 @@ pub async fn create_record(
)
};
let rkey = input.rkey.unwrap_or_else(Rkey::generate);
let rkey = input.rkey.clone().unwrap_or_else(Rkey::generate);
let mut ops: Vec<RecordOp> = Vec::new();
let mut conflict_uris_to_cleanup: Vec<AtUri> = Vec::new();
if !input.validate.should_skip() {
let record_uri = AtUri::from_parts(&did, &input.collection, &rkey);
let record_uri = AtUri::from_parts(did, &input.collection, &rkey);
let backlinks = extract_backlinks(&record_uri, &input.record);
if !backlinks.is_empty() {
@@ -149,24 +189,18 @@ pub async fn create_record(
Ok(Some(cid)) => cid,
Ok(None) => continue,
Err(e) => {
error!(
"Failed to read conflict record from MST {}: {:?}",
conflict_uri, e
);
return Err(ApiError::InternalError(Some(
"Failed to read conflicting record from MST".into(),
)));
return Err(ApiError::from_mst_error(
&format!("read conflict record from MST {conflict_uri}"),
&e,
));
}
};
mst = mst.delete(&conflict_key).await.map_err(|e| {
error!(
"Failed to delete conflict from MST {}: {:?}",
conflict_uri, e
);
ApiError::InternalError(Some(
"Failed to delete conflicting record from MST".into(),
))
ApiError::from_mst_error(
&format!("delete conflict from MST {conflict_uri}"),
&e,
)
})?;
ops.push(RecordOp::Delete {
@@ -183,7 +217,7 @@ pub async fn create_record(
if mst
.get(&key)
.await
.map_err(|e| ApiError::InternalError(Some(format!("Failed to read MST: {e}"))))?
.map_err(|e| ApiError::from_mst_error("read MST for create existence check", &e))?
.is_some()
{
return Err(ApiError::InvalidRequest(format!(
@@ -202,7 +236,7 @@ pub async fn create_record(
mst = mst
.add(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to add to MST".into())))?;
.map_err(|e| ApiError::from_mst_error("add record to MST", &e))?;
ops.push(RecordOp::Create {
collection: input.collection.clone(),
@@ -212,18 +246,18 @@ pub async fn create_record(
let blob_cids = extract_blob_cids(&input.record);
let created_uri = AtUri::from_parts(&did, &input.collection, &rkey);
let created_uri = AtUri::from_parts(did, &input.collection, &rkey);
let backlinks_to_add = extract_backlinks(&created_uri, &input.record);
let commit_result = finalize_repo_write(
&state,
state,
ctx,
mst,
FinalizeParams {
did: &did,
did,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: controller_did.as_ref().map(|_| {
controller_did,
delegation_detail: controller_did.map(|_| {
json!({
"action": "create",
"collection": input.collection,
@@ -238,7 +272,7 @@ pub async fn create_record(
)
.await?;
Ok(Json(CreateRecordOutput {
Ok(CreateRecordOutput {
uri: created_uri,
cid: record_cid.to_string(),
commit: CommitInfo {
@@ -246,7 +280,7 @@ pub async fn create_record(
rev: commit_result.rev,
},
validation_status,
}))
})
}
#[derive(Deserialize)]
@@ -278,15 +312,32 @@ pub struct PutRecordOutput {
pub async fn put_record(
State(state): State<AppState>,
auth: Auth<Active>,
Json(input): Json<PutRecordInput>,
Json(mut input): Json<PutRecordInput>,
) -> Result<Json<PutRecordOutput>, ApiError> {
if let Cow::Owned(record) = ensure_record_type(&input.record, &input.collection) {
input.record = record;
}
let upsert_proof = auth.verify_repo_upsert(&input.collection)?;
let repo_auth = prepare_repo_write(&state, &upsert_proof, &input.repo).await?;
let did = repo_auth.did;
let user_id = repo_auth.user_id;
let controller_did = repo_auth.controller_did;
let (ctx, mst) = begin_repo_write(&state, user_id, input.swap_commit.as_deref()).await?;
let out = with_repair_retry(&state, user_id, || {
put_record_inner(&state, &did, user_id, controller_did.as_ref(), &input)
})
.await?;
Ok(Json(out))
}
async fn put_record_inner(
state: &AppState,
did: &Did,
user_id: Uuid,
controller_did: Option<&Did>,
input: &PutRecordInput,
) -> Result<PutRecordOutput, ApiError> {
let (ctx, mst) = begin_repo_write(state, user_id, input.swap_commit.as_deref()).await?;
let validation_status = if input.validate.should_skip() {
None
@@ -304,9 +355,13 @@ pub async fn put_record(
let key = format!("{}/{}", input.collection, input.rkey);
let read_cid = |r: Result<Option<Cid>, jacquard_repo::error::RepoError>| {
r.map_err(|e| ApiError::from_mst_error("read MST for put", &e))
};
if let Some(swap_record_str) = &input.swap_record {
let expected_cid = Cid::from_str(swap_record_str).ok();
let actual_cid = mst.get(&key).await.ok().flatten();
let actual_cid = read_cid(mst.get(&key).await)?;
if expected_cid != actual_cid {
return Err(ApiError::InvalidSwap(Some(
"Record has been modified or does not exist".into(),
@@ -314,7 +369,7 @@ pub async fn put_record(
}
}
let existing_cid = mst.get(&key).await.ok().flatten();
let existing_cid = read_cid(mst.get(&key).await)?;
let record_ipld = tranquil_pds::util::json_to_ipld(&input.record);
let record_bytes = serde_ipld_dagcbor::to_vec(&record_ipld)
.map_err(|_| ApiError::InvalidRecord("Failed to serialize record".into()))?;
@@ -325,21 +380,21 @@ pub async fn put_record(
.map_err(|_| ApiError::InternalError(Some("Failed to save record block".into())))?;
if existing_cid == Some(record_cid) {
return Ok(Json(PutRecordOutput {
uri: AtUri::from_parts(&did, &input.collection, &input.rkey),
return Ok(PutRecordOutput {
uri: AtUri::from_parts(did, &input.collection, &input.rkey),
cid: record_cid.to_string(),
commit: None,
validation_status,
}));
});
}
let record_uri = AtUri::from_parts(&did, &input.collection, &input.rkey);
let record_uri = AtUri::from_parts(did, &input.collection, &input.rkey);
let (new_mst, op, is_update, backlinks_to_remove) = match existing_cid {
Some(prev_cid) => {
let new_mst = mst
.update(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to update MST".into())))?;
.map_err(|e| ApiError::from_mst_error("update record in MST", &e))?;
let op = RecordOp::Update {
collection: input.collection.clone(),
rkey: input.rkey.clone(),
@@ -352,7 +407,7 @@ pub async fn put_record(
let new_mst = mst
.add(&key, record_cid)
.await
.map_err(|_| ApiError::InternalError(Some("Failed to add to MST".into())))?;
.map_err(|e| ApiError::from_mst_error("add record to MST", &e))?;
let op = RecordOp::Create {
collection: input.collection.clone(),
rkey: input.rkey.clone(),
@@ -366,14 +421,14 @@ pub async fn put_record(
let backlinks_to_add = extract_backlinks(&record_uri, &input.record);
let commit_result = finalize_repo_write(
&state,
state,
ctx,
new_mst,
FinalizeParams {
did: &did,
did,
user_id,
controller_did: controller_did.as_ref(),
delegation_detail: controller_did.as_ref().map(|_| {
controller_did,
delegation_detail: controller_did.map(|_| {
json!({
"action": if is_update { "update" } else { "create" },
"collection": input.collection,
@@ -388,7 +443,7 @@ pub async fn put_record(
)
.await?;
Ok(Json(PutRecordOutput {
Ok(PutRecordOutput {
uri: record_uri,
cid: record_cid.to_string(),
commit: Some(CommitInfo {
@@ -396,5 +451,5 @@ pub async fn put_record(
rev: commit_result.rev,
}),
validation_status,
}))
})
}
@@ -665,27 +665,22 @@ pub async fn delete_account(
error!("DB error deleting account: {:?}", e);
ApiError::InternalError(None)
})?;
let account_seq = tranquil_pds::repo_ops::sequence_account_event(
if let Err(e) = tranquil_pds::repo_ops::sequence_account_event(
&state,
did,
tranquil_db_traits::AccountStatus::Deleted,
)
.await;
match account_seq {
Ok(seq) => {
if let Err(e) = state.repos.repo.delete_sequences_except(did, seq).await {
warn!(
"Failed to cleanup sequences for deleted account {}: {}",
did, e
);
}
}
Err(e) => {
warn!(
"Failed to sequence account deletion event for {}: {}",
did, e
);
}
.await
{
warn!(
"Failed to sequence account deletion event for {}: {}",
did, e
);
} else if let Err(e) = state.repos.repo.purge_did_events_keeping_latest(did).await {
warn!(
"Failed to cleanup sequences for deleted account {}: {}",
did, e
);
}
let _ = state
.cache
+6 -4
View File
@@ -5,7 +5,7 @@ use tranquil_pds::BUILD_VERSION;
use tranquil_pds::state::AppState;
use tranquil_pds::util::{discord_app_id, discord_bot_username, telegram_bot_username};
fn get_available_comms_channels() -> Vec<CommsChannel> {
async fn get_available_comms_channels(state: &AppState) -> Vec<CommsChannel> {
let cfg = tranquil_config::get();
let mut channels = vec![CommsChannel::Email];
if cfg.discord.bot_token.is_some() {
@@ -14,7 +14,9 @@ fn get_available_comms_channels() -> Vec<CommsChannel> {
if cfg.telegram.bot_token.is_some() {
channels.push(CommsChannel::Telegram);
}
if cfg.signal.enabled {
if let Some(slot) = &state.signal_sender
&& slot.is_linked().await
{
channels.push(CommsChannel::Signal);
}
channels
@@ -66,7 +68,7 @@ pub struct DescribeServerOutput {
pub telegram_bot_username: Option<String>,
}
pub async fn describe_server() -> Json<DescribeServerOutput> {
pub async fn describe_server(State(state): State<AppState>) -> Json<DescribeServerOutput> {
let cfg = tranquil_config::get();
let pds_hostname = &cfg.server.hostname;
@@ -82,7 +84,7 @@ pub async fn describe_server() -> Json<DescribeServerOutput> {
email: cfg.server.contact_email.clone(),
},
version: BUILD_VERSION,
available_comms_channels: get_available_comms_channels(),
available_comms_channels: get_available_comms_channels(&state).await,
self_hosted_did_web_enabled: is_self_hosted_did_web_enabled(),
discord_bot_username: discord_bot_username().map(String::from),
discord_app_id: discord_app_id().map(String::from),
+1
View File
@@ -51,6 +51,7 @@ pub use session::{
auto_resend_verification, confirm_signup, create_session, delete_session,
get_legacy_login_preference, get_session, list_sessions, refresh_session, resend_verification,
revoke_all_sessions, revoke_session, update_legacy_login_preference, update_locale,
verification_blocks_login,
};
pub use signing_key::reserve_signing_key;
pub use totp::{
+11 -8
View File
@@ -13,10 +13,6 @@ use tranquil_pds::state::AppState;
use tranquil_pds::types::PlainPassword;
use tranquil_pds::validation::validate_password;
fn generate_reset_code() -> String {
tranquil_pds::util::generate_token_code()
}
#[derive(Deserialize)]
pub struct RequestPasswordResetInput {
#[serde(alias = "identifier")]
@@ -70,12 +66,13 @@ pub async fn request_password_reset(
return Err(ApiError::InternalError(None));
}
};
let code = generate_reset_code();
let display_code = tranquil_pds::util::generate_token_code();
let stored_code = tranquil_pds::util::normalize_token_code(&display_code);
let expires_at = Utc::now() + Duration::minutes(10);
if let Err(e) = state
.repos
.user
.set_password_reset_code(user_id, &code, expires_at)
.set_password_reset_code(user_id, &stored_code, expires_at)
.await
{
error!("DB error setting reset code: {:?}", e);
@@ -86,7 +83,7 @@ pub async fn request_password_reset(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
user_id,
&code,
&display_code,
hostname,
)
.await
@@ -133,7 +130,13 @@ pub async fn reset_password(
if let Err(e) = validate_password(password) {
return Err(ApiError::InvalidRequest(e.to_string()));
}
let user = match state.repos.user.get_user_by_reset_code(token).await {
let normalized_token = tranquil_pds::util::normalize_token_code(token);
let user = match state
.repos
.user
.get_user_by_reset_code(&normalized_token)
.await
{
Ok(Some(u)) => u,
Ok(None) => {
return Err(ApiError::InvalidToken(None));
+32 -5
View File
@@ -8,18 +8,28 @@ use bcrypt::verify;
use serde::{Deserialize, Serialize};
use serde_json::json;
use tracing::{error, info, warn};
use tranquil_db_traits::{SessionId, TokenFamilyId};
use tranquil_db_traits::{ChannelVerificationStatus, SessionId, TokenFamilyId};
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::api::{EmptyResponse, PreferredLocaleOutput, SuccessResponse};
use tranquil_pds::auth::{
Active, Auth, NormalizedLoginIdentifier, Permissive, require_legacy_session_mfa,
require_reauth_window,
};
use tranquil_pds::rate_limit::{LoginLimit, RateLimited, RefreshSessionLimit};
use tranquil_pds::rate_limit::{
LoginLimit, RateLimited, RefreshSessionLimit, TotpVerifyLimit,
check_user_rate_limit_with_message,
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{AccountState, Did, Handle, PlainPassword};
use tranquil_types::TokenId;
pub fn verification_blocks_login(channel_verification: &ChannelVerificationStatus) -> bool {
!tranquil_config::get()
.server
.disable_account_verification_gate
&& !channel_verification.has_any_verified()
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct CreateSessionInput {
@@ -129,14 +139,13 @@ pub async fn create_session(
warn!("Login attempt for takendown account: {}", row.did);
return Err(ApiError::AccountTakedown);
}
let is_verified = row.channel_verification.has_any_verified();
let is_delegated = state
.repos
.delegation
.is_delegated_account(&row.did)
.await
.unwrap_or(false);
if !is_verified && !is_delegated {
if verification_blocks_login(&row.channel_verification) && !is_delegated {
warn!("Login attempt for unverified account: {}", row.did);
let resend_info = auto_resend_verification(&state, &row.did).await;
let handle = resend_info
@@ -162,17 +171,35 @@ pub async fn create_session(
let has_totp = row.totp_enabled;
let email_2fa_enabled = row.email_2fa_enabled;
let is_legacy_login = has_totp || email_2fa_enabled;
let used_totp_factor = tranquil_pds::auth::legacy_2fa::used_totp_factor(
has_totp,
input.auth_factor_token.as_deref(),
);
let twofa_ctx = tranquil_pds::auth::legacy_2fa::Legacy2faContext {
is_app_password: app_password_name.is_some(),
email_2fa_enabled,
has_totp,
allow_legacy_login: row.allow_legacy_login,
};
if let Some(token) = input.auth_factor_token.as_deref()
&& has_totp
&& tranquil_pds::auth::legacy_2fa::looks_like_totp_token(token)
{
check_user_rate_limit_with_message::<TotpVerifyLimit>(
&state,
row.did.as_str(),
"Too many verification attempts. Please try again in a few minutes.",
)
.await?;
}
match tranquil_pds::auth::legacy_2fa::process_legacy_2fa(
state.cache.as_ref(),
&row.did,
&twofa_ctx,
input.auth_factor_token.as_deref(),
async |t: &str| {
crate::server::totp::verify_totp_or_backup_for_user(&state, &row.did, t).await
},
)
.await
{
@@ -279,7 +306,7 @@ pub async fn create_session(
error!("Failed to insert session: {:?}", e);
return Err(ApiError::InternalError(None));
}
if is_legacy_login {
if is_legacy_login && !used_totp_factor {
warn!(
did = %row.did,
ip = %client_ip,
+5 -5
View File
@@ -35,7 +35,7 @@ pub fn create_access_token_with_scope_metadata(
scope,
TokenType::Access,
key_bytes,
Duration::minutes(15),
Duration::minutes(120),
hostname,
)
}
@@ -54,7 +54,7 @@ pub fn create_access_token_with_delegation(
scope,
TokenType::Access,
key_bytes,
Duration::minutes(15),
Duration::minutes(120),
act,
hostname,
)
@@ -69,7 +69,7 @@ pub fn create_refresh_token_with_metadata(
TokenScope::Refresh.as_str(),
TokenType::Refresh,
key_bytes,
Duration::days(14),
Duration::days(90),
None,
)
}
@@ -198,7 +198,7 @@ pub fn create_access_token_hs256_with_metadata(
TokenScope::Access.as_str(),
TokenType::Access,
secret,
Duration::minutes(15),
Duration::minutes(120),
)
}
@@ -211,7 +211,7 @@ pub fn create_refresh_token_hs256_with_metadata(
TokenScope::Refresh.as_str(),
TokenType::Refresh,
secret,
Duration::days(14),
Duration::days(90),
)
}
+14 -1
View File
@@ -15,7 +15,9 @@ impl TokenType {
match self {
Self::Access => "at+jwt",
Self::Refresh => "refresh+jwt",
Self::Service => "jwt",
// RFC 7519 §5.1 recommends the uppercase "JWT".
// and for atproto inter-service auth its a requirement.
Self::Service => "JWT",
}
}
}
@@ -290,6 +292,17 @@ mod tests {
assert!(TokenType::from_str("bearer").is_err());
}
#[test]
fn service_token_header_serializes_typ_as_uppercase_jwt() {
// RFC 7519 §5.1 recommends the JWT `typ` header value be uppercase "JWT".
let header = Header {
alg: SigningAlgorithm::ES256K,
typ: TokenType::Service,
};
let json = serde_json::to_string(&header).expect("serialize header");
assert!(json.contains(r#""typ":"JWT""#), "got {json}");
}
#[test]
fn signing_algorithm_case_insensitive() {
assert_eq!(
+11
View File
@@ -10,7 +10,14 @@ tranquil-signal = { workspace = true }
async-trait = { workspace = true }
base64 = { workspace = true }
ed25519-dalek = { workspace = true }
futures = { workspace = true }
hickory-resolver = { workspace = true }
lettre = { workspace = true }
rand = { workspace = true }
reqwest = { workspace = true }
rsa = { workspace = true }
secrecy = { workspace = true }
serde_json = { workspace = true }
sqlx = { workspace = true }
thiserror = { workspace = true }
@@ -18,3 +25,7 @@ tokio = { workspace = true }
tracing = { workspace = true }
tranquil-db-traits = { workspace = true }
uuid = { workspace = true }
[dev-dependencies]
chrono = { workspace = true }
tokio = { workspace = true, features = ["macros", "rt-multi-thread", "time", "io-util", "net"] }
+227
View File
@@ -0,0 +1,227 @@
use std::fs;
use base64::Engine;
use base64::engine::general_purpose::STANDARD as BASE64_STANDARD;
use ed25519_dalek::pkcs8::DecodePrivateKey as _;
use lettre::Message;
use lettre::message::dkim::{
DkimCanonicalization, DkimCanonicalizationType, DkimConfig as LettreDkimConfig,
DkimSigningAlgorithm, DkimSigningKey,
};
use lettre::message::header::HeaderName;
use rsa::pkcs1::EncodeRsaPrivateKey;
use rsa::pkcs8::LineEnding;
use super::types::{DkimKeyPath, DkimSelector, EmailDomain};
use crate::sender::SendError;
const SIGNED_HEADERS: &[&str] = &[
"From",
"Sender",
"Reply-To",
"To",
"Cc",
"Subject",
"Date",
"In-Reply-To",
"References",
"MIME-Version",
"Content-Type",
"Content-Transfer-Encoding",
];
pub struct DkimSigner {
config: LettreDkimConfig,
}
impl DkimSigner {
pub fn load(
selector: DkimSelector,
domain: EmailDomain,
path: DkimKeyPath,
) -> Result<Self, SendError> {
let pem = fs::read_to_string(path.as_path()).map_err(|e| {
SendError::DkimSign(format!("read DKIM key {}: {e}", path.as_path().display()))
})?;
Self::from_pem(selector, domain, &pem)
}
pub fn from_pem(
selector: DkimSelector,
domain: EmailDomain,
pem: &str,
) -> Result<Self, SendError> {
let key = parse_key(pem)?;
let canonicalization = DkimCanonicalization {
header: DkimCanonicalizationType::Relaxed,
body: DkimCanonicalizationType::Relaxed,
};
let headers = SIGNED_HEADERS
.iter()
.copied()
.map(HeaderName::new_from_ascii_str)
.collect();
let config = LettreDkimConfig::new(
selector.into_inner(),
domain.into_inner(),
key,
headers,
canonicalization,
);
Ok(Self { config })
}
pub fn sign(&self, message: &mut Message) {
message.sign(&self.config);
}
}
impl std::fmt::Debug for DkimSigner {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("DkimSigner")
}
}
fn parse_key(input: &str) -> Result<DkimSigningKey, SendError> {
let trimmed = input.trim_start();
match trimmed {
s if s.starts_with("-----BEGIN RSA PRIVATE KEY-----") => {
DkimSigningKey::new(input, DkimSigningAlgorithm::Rsa)
.map_err(|e| SendError::DkimSign(format!("RSA PKCS#1 PEM rejected: {e}")))
}
s if s.starts_with("-----BEGIN PRIVATE KEY-----") => parse_pkcs8(input),
s if s.starts_with("-----BEGIN") => Err(SendError::DkimSign(
"unrecognized PEM type; expected an RSA or Ed25519 private key".to_string(),
)),
_ => DkimSigningKey::new(input.trim(), DkimSigningAlgorithm::Ed25519).map_err(|e| {
SendError::DkimSign(format!(
"expected base64-encoded 32-byte Ed25519 seed or a PEM-wrapped key: {e}"
))
}),
}
}
fn parse_pkcs8(pem: &str) -> Result<DkimSigningKey, SendError> {
let ed25519_err = match ed25519_dalek::SigningKey::from_pkcs8_pem(pem) {
Ok(key) => {
let seed = BASE64_STANDARD.encode(key.to_bytes());
return DkimSigningKey::new(&seed, DkimSigningAlgorithm::Ed25519)
.map_err(|e| SendError::DkimSign(format!("re-import Ed25519 seed: {e}")));
}
Err(e) => e,
};
let rsa_err = match rsa::RsaPrivateKey::from_pkcs8_pem(pem) {
Ok(key) => {
let pkcs1 = key
.to_pkcs1_pem(LineEnding::LF)
.map_err(|e| SendError::DkimSign(format!("re-encode RSA PKCS#8 as PKCS#1: {e}")))?;
return DkimSigningKey::new(pkcs1.as_str(), DkimSigningAlgorithm::Rsa)
.map_err(|e| SendError::DkimSign(format!("re-import RSA PKCS#1: {e}")));
}
Err(e) => e,
};
Err(SendError::DkimSign(format!(
"PKCS#8 PEM rejected by both parsers; ed25519: {ed25519_err}; rsa: {rsa_err}"
)))
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::pkcs8::EncodePrivateKey as _;
use lettre::message::Mailbox;
use lettre::message::header::ContentType;
use rsa::pkcs1::DecodeRsaPrivateKey as _;
const ED25519_RAW_SEED_B64: &str = "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=";
const RSA_PKCS1_PEM: &str = include_str!("test_fixtures/rsa2048-priv-pkcs1.pem");
fn ed25519_pkcs8_pem() -> String {
let key = ed25519_dalek::SigningKey::from_bytes(&[7u8; 32]);
key.to_pkcs8_pem(LineEnding::LF).unwrap().to_string()
}
fn rsa_pkcs8_pem() -> String {
let key = rsa::RsaPrivateKey::from_pkcs1_pem(RSA_PKCS1_PEM).unwrap();
key.to_pkcs8_pem(LineEnding::LF).unwrap().to_string()
}
fn signer(pem: &str) -> DkimSigner {
DkimSigner::from_pem(
DkimSelector::parse("default").unwrap(),
EmailDomain::parse("nel.pet").unwrap(),
pem,
)
.expect("key should load")
}
fn signed_headers(signer: &DkimSigner) -> String {
let from: Mailbox = "sender@nel.pet".parse().unwrap();
let to: Mailbox = "recipient@nel.pet".parse().unwrap();
let mut message = Message::builder()
.from(from)
.to(to)
.subject("Roundtrip")
.header(ContentType::TEXT_PLAIN)
.body("Body".to_string())
.unwrap();
signer.sign(&mut message);
String::from_utf8(message.formatted()).unwrap()
}
#[test]
fn rejects_garbage() {
assert!(matches!(
parse_key("not a key"),
Err(SendError::DkimSign(_))
));
}
#[test]
fn rejects_unknown_pem_type() {
let pem = "-----BEGIN OPENSSH PRIVATE KEY-----\nx\n-----END OPENSSH PRIVATE KEY-----\n";
match parse_key(pem) {
Err(SendError::DkimSign(msg)) => assert!(msg.contains("unrecognized"), "msg: {msg}"),
other => panic!("expected unrecognized PEM error, got {other:?}"),
}
}
#[test]
fn ed25519_raw_seed_signs() {
let raw = signed_headers(&signer(ED25519_RAW_SEED_B64));
assert_signed_with(&raw, "a=ed25519-sha256");
}
#[test]
fn ed25519_pkcs8_pem_signs() {
let raw = signed_headers(&signer(&ed25519_pkcs8_pem()));
assert_signed_with(&raw, "a=ed25519-sha256");
}
#[test]
fn rsa_pkcs1_pem_signs() {
let raw = signed_headers(&signer(RSA_PKCS1_PEM));
assert_signed_with(&raw, "a=rsa-sha256");
}
#[test]
fn rsa_pkcs8_pem_signs() {
let raw = signed_headers(&signer(&rsa_pkcs8_pem()));
assert_signed_with(&raw, "a=rsa-sha256");
}
fn assert_signed_with(raw: &str, algorithm: &str) {
assert!(
raw.contains("DKIM-Signature:"),
"no signature header: {raw}"
);
assert!(raw.contains(algorithm), "missing {algorithm}: {raw}");
assert!(
raw.contains("c=relaxed/relaxed"),
"expected relaxed/relaxed canonicalization: {raw}"
);
}
}
+151
View File
@@ -0,0 +1,151 @@
use lettre::Message;
use lettre::message::Mailbox;
use lettre::message::header::ContentType;
use uuid::Uuid;
use super::types::EmailDomain;
use crate::sender::SendError;
use crate::types::QueuedComms;
pub(super) fn build(from: &Mailbox, qc: &QueuedComms) -> Result<Message, SendError> {
let to: Mailbox = qc
.recipient
.parse()
.map_err(|e: lettre::address::AddressError| SendError::InvalidRecipient(e.to_string()))?;
let subject = qc.subject.as_deref().unwrap_or("Notification");
let message_id = format!("<{}@{}>", Uuid::new_v4(), from.email.domain());
Message::builder()
.from(from.clone())
.to(to)
.subject(subject)
.message_id(Some(message_id))
.header(ContentType::TEXT_PLAIN)
.body(qc.body.clone())
.map_err(|e| SendError::MessageBuild(e.to_string()))
}
pub(super) fn recipient_domain(message: &Message) -> Result<EmailDomain, SendError> {
let envelope = message.envelope();
let first = envelope
.to()
.first()
.ok_or_else(|| SendError::MessageBuild("envelope has no recipients".to_string()))?;
EmailDomain::parse(first.domain())
.map_err(|e| SendError::InvalidRecipient(format!("invalid recipient domain: {e}")))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::types::{CommsChannel, CommsStatus, CommsType};
use chrono::Utc;
use uuid::Uuid;
fn from_mailbox() -> Mailbox {
"Test Sender <noreply@nel.pet>".parse().unwrap()
}
fn fixture(recipient: &str, subject: Option<&str>, body: &str) -> QueuedComms {
QueuedComms {
id: Uuid::new_v4(),
user_id: None,
channel: CommsChannel::Email,
comms_type: CommsType::Welcome,
status: CommsStatus::Pending,
recipient: recipient.to_string(),
subject: subject.map(String::from),
body: body.to_string(),
metadata: None,
attempts: 0,
max_attempts: 3,
last_error: None,
created_at: Utc::now(),
updated_at: Utc::now(),
scheduled_for: Utc::now(),
processed_at: None,
}
}
#[test]
fn build_basic_message() {
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("Welcome"), "Hello world."),
)
.unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
let lower = raw.to_lowercase();
assert!(raw.contains("From: \"Test Sender\" <noreply@nel.pet>"));
assert!(raw.contains("To: user@nel.pet"));
assert!(raw.contains("Subject: Welcome"));
assert!(lower.contains("content-type: text/plain"));
assert!(raw.contains("Hello world."));
}
#[test]
fn utf8_subject_is_encoded() {
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("héllo wörld"), "Body"),
)
.unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(raw.contains("=?utf-8?"));
assert!(!raw.contains("héllo"));
}
#[test]
fn header_injection_rejected() {
let result = build(
&from_mailbox(),
&fixture("x@nel.pet\r\nBcc: evil@x", Some("s"), "b"),
);
assert!(matches!(result, Err(SendError::InvalidRecipient(_))));
}
#[test]
fn subject_crlf_does_not_inject_headers() {
let msg = build(
&from_mailbox(),
&fixture("user@nel.pet", Some("hi\r\nBcc: evil@nel.pet"), "body"),
)
.expect("subject CRLF should be encoded, not rejected");
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(
!raw.contains("Bcc:"),
"CRLF in subject must not produce a Bcc header: {raw}"
);
assert!(
raw.contains("Subject: ="),
"subject with non-printable chars should be RFC 2047 encoded: {raw}"
);
}
#[test]
fn message_id_uses_from_domain() {
let msg = build(&from_mailbox(), &fixture("user@nel.pet", Some("s"), "b")).unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
let line = raw
.lines()
.find(|l| l.starts_with("Message-ID:") || l.starts_with("Message-Id:"))
.expect("message-id header present");
assert!(
line.contains("@nel.pet>"),
"message-id should use From domain: {line}"
);
}
#[test]
fn missing_subject_uses_default() {
let msg = build(&from_mailbox(), &fixture("user@nel.pet", None, "Body")).unwrap();
let raw = String::from_utf8(msg.formatted()).unwrap();
assert!(raw.contains("Subject: Notification"));
}
#[test]
fn recipient_domain_extracted() {
let msg = build(&from_mailbox(), &fixture("user@Nel.PET", Some("s"), "b")).unwrap();
let d = recipient_domain(&msg).unwrap();
assert_eq!(d.as_str(), "nel.pet");
}
}
+194
View File
@@ -0,0 +1,194 @@
pub mod dkim;
pub mod message;
mod mx;
pub mod transport;
pub mod types;
use std::sync::Arc;
use std::time::Duration;
use async_trait::async_trait;
use hickory_resolver::TokioAsyncResolver;
use lettre::message::Mailbox;
use lettre::transport::smtp::AsyncSmtpTransport;
use lettre::transport::smtp::PoolConfig;
use lettre::transport::smtp::authentication::Credentials;
use lettre::transport::smtp::extension::ClientId;
use tokio::sync::Semaphore;
use tracing::{info, warn};
pub use self::dkim::DkimSigner;
pub use self::transport::SendMode;
use self::types::{
DkimKeyPath, DkimSelector, EmailDomain, HeloName, SmtpHost, SmtpPassword, SmtpPort,
SmtpUsername, TlsMode,
};
use crate::sender::{CommsSender, SendError};
use crate::types::{CommsChannel, QueuedComms};
pub struct EmailSender {
from: Mailbox,
mode: SendMode,
dkim: Option<DkimSigner>,
}
impl EmailSender {
pub fn new(from: Mailbox, mode: SendMode, dkim: Option<DkimSigner>) -> Self {
Self { from, mode, dkim }
}
pub fn from_config(cfg: &tranquil_config::TranquilConfig) -> Result<Option<Self>, SendError> {
let Some(from_address) = cfg.email.from_address.as_deref().filter(|s| !s.is_empty()) else {
info!("Email sender disabled: MAIL_FROM_ADDRESS unset");
return Ok(None);
};
let from = build_from(&cfg.email.from_name, from_address)?;
let dkim = build_dkim(&cfg.email.dkim)?;
let mode = match cfg
.email
.smarthost
.host
.as_deref()
.filter(|h| !h.is_empty())
{
Some(host) => build_smarthost(cfg, host)?,
None => build_direct_mx(cfg)?,
};
info!(?mode, dkim = dkim.is_some(), "Email sender initialized");
Ok(Some(Self { from, mode, dkim }))
}
}
fn config_invalid(field: &str, error: impl std::fmt::Display) -> SendError {
SendError::ConfigInvalid(format!("{field}: {error}"))
}
fn build_from(from_name: &str, from_address: &str) -> Result<Mailbox, SendError> {
let raw = match from_name.is_empty() {
true => from_address.to_string(),
false => format!("\"{}\" <{}>", from_name.replace('"', "'"), from_address),
};
raw.parse::<Mailbox>()
.map_err(|e| config_invalid("MAIL_FROM_ADDRESS / MAIL_FROM_NAME", e))
}
fn build_smarthost(
cfg: &tranquil_config::TranquilConfig,
host_raw: &str,
) -> Result<SendMode, SendError> {
let host = SmtpHost::parse(host_raw).map_err(|e| config_invalid("MAIL_SMARTHOST_HOST", e))?;
let port = SmtpPort::parse(cfg.email.smarthost.port)
.map_err(|e| config_invalid("MAIL_SMARTHOST_PORT", e))?;
let tls = TlsMode::parse(&cfg.email.smarthost.tls)
.map_err(|e| config_invalid("MAIL_SMARTHOST_TLS", e))?;
let helo = resolve_helo(cfg)?;
let pool = PoolConfig::new()
.max_size(cfg.email.smarthost.pool_size)
.idle_timeout(Duration::from_secs(60));
let command_timeout = Duration::from_secs(cfg.email.smarthost.command_timeout_secs);
let total_timeout = Duration::from_secs(cfg.email.smarthost.total_timeout_secs);
let builder = match tls {
TlsMode::Implicit => AsyncSmtpTransport::<lettre::Tokio1Executor>::relay(host.as_str())
.map_err(|e| config_invalid("smarthost TLS setup", e))?,
TlsMode::Starttls => {
AsyncSmtpTransport::<lettre::Tokio1Executor>::starttls_relay(host.as_str())
.map_err(|e| config_invalid("smarthost TLS setup", e))?
}
TlsMode::None => {
AsyncSmtpTransport::<lettre::Tokio1Executor>::builder_dangerous(host.as_str())
}
};
let builder = builder
.port(port.as_u16())
.hello_name(ClientId::Domain(helo.into_inner()))
.timeout(Some(command_timeout))
.pool_config(pool);
let builder = match (
cfg.email.smarthost.username.as_deref(),
cfg.email.smarthost.password.as_deref(),
) {
(Some(u), Some(p)) => {
let username =
SmtpUsername::parse(u).map_err(|e| config_invalid("MAIL_SMARTHOST_USERNAME", e))?;
let password =
SmtpPassword::parse(p).map_err(|e| config_invalid("MAIL_SMARTHOST_PASSWORD", e))?;
builder.credentials(Credentials::new(
username.into_inner(),
password.expose().to_string(),
))
}
_ => builder,
};
Ok(SendMode::Smarthost {
transport: Box::new(builder.build()),
total_timeout,
})
}
fn build_direct_mx(cfg: &tranquil_config::TranquilConfig) -> Result<SendMode, SendError> {
let helo = resolve_helo(cfg)?;
let resolver = TokioAsyncResolver::tokio_from_system_conf()
.map(Arc::new)
.map_err(|e| config_invalid("system DNS configuration", e))?;
let max_concurrent = cfg.email.direct_mx.max_concurrent_sends.max(1);
Ok(SendMode::DirectMx {
resolver,
helo,
command_timeout: Duration::from_secs(cfg.email.direct_mx.command_timeout_secs),
total_timeout: Duration::from_secs(cfg.email.direct_mx.total_timeout_secs),
require_tls: cfg.email.direct_mx.require_tls,
inflight: Arc::new(Semaphore::new(max_concurrent)),
})
}
fn resolve_helo(cfg: &tranquil_config::TranquilConfig) -> Result<HeloName, SendError> {
let raw = cfg
.email
.helo_name
.clone()
.unwrap_or_else(|| cfg.server.hostname_without_port().to_string());
HeloName::parse(&raw).map_err(|e| config_invalid(&format!("HELO name {raw:?}"), e))
}
fn build_dkim(cfg: &tranquil_config::DkimConfig) -> Result<Option<DkimSigner>, SendError> {
let selector = match cfg.selector.as_deref() {
Some(s) => s,
None => return Ok(None),
};
let domain = cfg
.domain
.as_deref()
.ok_or_else(|| SendError::DkimSign("MAIL_DKIM_DOMAIN required when selector set".into()))?;
let key_path = cfg.private_key_path.as_deref().ok_or_else(|| {
SendError::DkimSign("MAIL_DKIM_KEY_PATH required when selector set".into())
})?;
let selector = DkimSelector::parse(selector)
.map_err(|e| SendError::DkimSign(format!("invalid DKIM selector: {e}")))?;
let domain = EmailDomain::parse(domain)
.map_err(|e| SendError::DkimSign(format!("invalid DKIM domain: {e}")))?;
let path = DkimKeyPath::parse(key_path)
.map_err(|e| SendError::DkimSign(format!("DKIM key path invalid: {e}")))?;
DkimSigner::load(selector, domain, path).map(Some)
}
#[async_trait]
impl CommsSender for EmailSender {
fn channel(&self) -> CommsChannel {
CommsChannel::Email
}
async fn send(&self, notification: &QueuedComms) -> Result<(), SendError> {
let mut message = message::build(&self.from, notification)?;
if let Some(signer) = &self.dkim {
signer.sign(&mut message);
}
match transport::dispatch(&self.mode, message).await {
Ok(()) => Ok(()),
Err(e) => {
warn!(comms_id = %notification.id, error = %e, "SMTP send failed");
Err(e)
}
}
}
}
+250
View File
@@ -0,0 +1,250 @@
use hickory_resolver::TokioAsyncResolver;
use hickory_resolver::error::{ResolveError, ResolveErrorKind};
use hickory_resolver::proto::op::ResponseCode;
use rand::seq::SliceRandom;
use super::types::{EmailDomain, MxHost, MxPriority, MxRecord};
use crate::sender::SendError;
pub async fn resolve(
resolver: &TokioAsyncResolver,
domain: &EmailDomain,
) -> Result<Vec<MxRecord>, SendError> {
match resolver.mx_lookup(domain.as_str()).await {
Ok(lookup) => interpret_lookup(
lookup
.iter()
.map(|mx| (mx.preference(), mx.exchange().clone())),
domain,
),
Err(e) => classify_lookup_error(e, domain),
}
}
fn interpret_lookup(
items: impl IntoIterator<Item = (u16, hickory_resolver::Name)>,
domain: &EmailDomain,
) -> Result<Vec<MxRecord>, SendError> {
let entries: Vec<_> = items.into_iter().collect();
match entries.iter().any(|(_, name)| name.is_root()) {
true => Err(SendError::DnsPermanent(format!(
"null MX record at {}: domain refuses mail",
domain.as_str()
))),
false => {
let records: Vec<MxRecord> = entries
.into_iter()
.filter_map(|(prio, name)| {
MxHost::parse(&name.to_utf8()).ok().map(|host| MxRecord {
priority: MxPriority::new(prio),
host,
})
})
.collect();
match records.is_empty() {
true => implicit_mx(domain),
false => Ok(prioritize(records)),
}
}
}
}
fn prioritize(mut records: Vec<MxRecord>) -> Vec<MxRecord> {
records.shuffle(&mut rand::thread_rng());
records.sort_by_key(|r| r.priority);
records
}
fn classify_lookup_error(
e: ResolveError,
domain: &EmailDomain,
) -> Result<Vec<MxRecord>, SendError> {
match e.kind() {
ResolveErrorKind::NoRecordsFound { response_code, .. } => match *response_code {
ResponseCode::NoError => implicit_mx(domain),
ResponseCode::NXDomain => Err(SendError::DnsPermanent(format!(
"domain {} does not exist",
domain.as_str()
))),
other => Err(SendError::DnsTransient(format!(
"MX lookup for {} failed with {other}",
domain.as_str()
))),
},
_ => Err(SendError::DnsTransient(e.to_string())),
}
}
fn implicit_mx(domain: &EmailDomain) -> Result<Vec<MxRecord>, SendError> {
MxHost::parse(domain.as_str())
.map(|host| {
vec![MxRecord {
priority: MxPriority::new(0),
host,
}]
})
.map_err(|e| SendError::DnsPermanent(format!("invalid recipient domain: {e}")))
}
#[cfg(test)]
mod tests {
use super::*;
fn record(prio: u16, host: &str) -> MxRecord {
MxRecord {
priority: MxPriority::new(prio),
host: MxHost::parse(host).unwrap(),
}
}
#[test]
fn prioritize_sorts_by_priority_ascending() {
let result = prioritize(vec![
record(20, "mx2.nel.pet"),
record(10, "mx1.nel.pet"),
record(10, "mx1b.nel.pet"),
]);
assert_eq!(result[0].priority.as_u16(), 10);
assert_eq!(result[1].priority.as_u16(), 10);
assert_eq!(result[2].priority.as_u16(), 20);
}
#[test]
fn prioritize_randomizes_equal_priority_order() {
let attempts: Vec<Vec<String>> = (0..200)
.map(|_| {
prioritize(vec![
record(10, "a.nel.pet"),
record(10, "b.nel.pet"),
record(10, "c.nel.pet"),
record(10, "d.nel.pet"),
])
.into_iter()
.map(|r| r.host.as_str().to_string())
.collect()
})
.collect();
let distinct: std::collections::HashSet<_> = attempts.iter().cloned().collect();
assert!(
distinct.len() > 1,
"equal-priority MX order should vary across calls; got only {}",
distinct.len()
);
}
#[test]
fn implicit_mx_uses_domain_as_host() {
let d = EmailDomain::parse("nel.pet").unwrap();
let result = implicit_mx(&d).unwrap();
assert_eq!(result.len(), 1);
assert_eq!(result[0].priority.as_u16(), 0);
assert_eq!(result[0].host.as_str(), "nel.pet");
}
#[test]
fn no_error_response_yields_implicit_mx() {
let d = EmailDomain::parse("nel.pet").unwrap();
let err = ResolveError::from(ResolveErrorKind::NoRecordsFound {
query: Box::new(hickory_resolver::proto::op::Query::default()),
soa: None,
negative_ttl: None,
response_code: ResponseCode::NoError,
trusted: false,
});
let result = classify_lookup_error(err, &d).unwrap();
assert_eq!(result.len(), 1);
assert_eq!(result[0].host.as_str(), "nel.pet");
}
#[test]
fn nxdomain_response_is_permanent() {
let d = EmailDomain::parse("does-not-exist.invalid").unwrap();
let err = ResolveError::from(ResolveErrorKind::NoRecordsFound {
query: Box::new(hickory_resolver::proto::op::Query::default()),
soa: None,
negative_ttl: None,
response_code: ResponseCode::NXDomain,
trusted: true,
});
match classify_lookup_error(err, &d) {
Err(SendError::DnsPermanent(_)) => {}
other => panic!("expected DnsPermanent, got {other:?}"),
}
}
#[test]
fn servfail_response_is_transient() {
let d = EmailDomain::parse("nel.pet").unwrap();
let err = ResolveError::from(ResolveErrorKind::NoRecordsFound {
query: Box::new(hickory_resolver::proto::op::Query::default()),
soa: None,
negative_ttl: None,
response_code: ResponseCode::ServFail,
trusted: false,
});
match classify_lookup_error(err, &d) {
Err(SendError::DnsTransient(_)) => {}
other => panic!("expected DnsTransient, got {other:?}"),
}
}
#[test]
fn timeout_is_transient() {
let d = EmailDomain::parse("nel.pet").unwrap();
let err = ResolveError::from(ResolveErrorKind::Timeout);
match classify_lookup_error(err, &d) {
Err(SendError::DnsTransient(_)) => {}
other => panic!("expected DnsTransient, got {other:?}"),
}
}
#[test]
fn message_variant_is_transient() {
let d = EmailDomain::parse("nel.pet").unwrap();
let err = ResolveError::from(ResolveErrorKind::Message("transient resolver glitch"));
match classify_lookup_error(err, &d) {
Err(SendError::DnsTransient(_)) => {}
other => panic!("expected DnsTransient default, got {other:?}"),
}
}
#[test]
fn null_mx_is_permanent() {
let d = EmailDomain::parse("nomail.nel.pet").unwrap();
let result = interpret_lookup(vec![(0, hickory_resolver::Name::root())], &d);
match result {
Err(SendError::DnsPermanent(msg)) => {
assert!(msg.contains("null MX"), "msg: {msg}")
}
other => panic!("expected DnsPermanent, got {other:?}"),
}
}
#[test]
fn null_mx_alongside_real_records_still_permanent() {
let d = EmailDomain::parse("mixed.nel.pet").unwrap();
let real = hickory_resolver::Name::from_ascii("mx1.nel.pet.").unwrap();
let result = interpret_lookup(vec![(10, real), (0, hickory_resolver::Name::root())], &d);
assert!(matches!(result, Err(SendError::DnsPermanent(_))));
}
#[test]
fn empty_lookup_uses_implicit_mx() {
let d = EmailDomain::parse("nel.pet").unwrap();
let result = interpret_lookup(Vec::<(u16, hickory_resolver::Name)>::new(), &d).unwrap();
assert_eq!(result.len(), 1);
assert_eq!(result[0].host.as_str(), "nel.pet");
}
#[test]
fn valid_records_pass_through_with_priority_sort() {
let d = EmailDomain::parse("nel.pet").unwrap();
let mx1 = hickory_resolver::Name::from_ascii("mx1.nel.pet.").unwrap();
let mx2 = hickory_resolver::Name::from_ascii("mx2.nel.pet.").unwrap();
let result = interpret_lookup(vec![(20, mx2), (10, mx1)], &d).unwrap();
assert_eq!(result.len(), 2);
assert_eq!(result[0].priority.as_u16(), 10);
assert_eq!(result[0].host.as_str(), "mx1.nel.pet");
assert_eq!(result[1].priority.as_u16(), 20);
}
}
@@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAtsQsUV8QpqrygsY+2+JCQ6Fw8/omM71IM2N/R8pPbzbgOl0p
78MZGsgPOQ2HSznjD0FPzsH8oO2B5Uftws04LHb2HJAYlz25+lN5cqfHAfa3fgmC
38FfwBkn7l582UtPWZ/wcBOnyCgb3yLcvJrXyrt8QxHJgvWO23ITrUVYszImbXQ6
7YGS0YhMrbixRzmo2tpm3JcIBtnHrEUMsT0NfFdfsZhTT8YbxBvA8FdODgEwx7u/
vf3J9qbi4+Kv8cvqyJuleIRSjVXPsIMnoejIn04APPKIjpMyQdnWlby7rNyQtE4+
CV+jcFjqJbE/Xilcvqxt6DirjFCvYeKYl1uHLwIDAQABAoIBAH7Mg2LA7bB0EWQh
XiL3SrnZG6BpAHAM9jaQ5RFNjua9z7suP5YUaSpnegg/FopeUuWWjmQHudl8bg5A
ZPgtoLdYoU8XubfUH19I4o1lUXBPVuaeeqn6Yw/HZCjAbSXkVdz8VbesK092ZD/e
0/4V/3irsn5lrMSq0L322yfvYKaRDFxKCF7UMnWrGcHZl6Msbv/OffLRk19uYB7t
4WGhK1zCfKIfgdLJnD0eoI6Q4wU6sJvvpyTe8NDDo8HpdAwNn3YSahSewKp9gHgg
VIQlTZUdsHxM+R+2RUwJZYj9WSTbq+s1nKICUmjQBPnWbrPW963BE5utQPFt3mOe
EWRzdsECgYEA3MBhJC1Okq+u5yrFE8plufdwNvm9fg5uYUYafvdlQiXsFTx+XDGm
FXpuWhP/bheOh1jByzPZ1rvjF57xiZjkIuzcvtePTs/b5fT82K7CydDchkc8qb0W
2dI40h+13e++sUPKYdC9aqjZHzOgl3kOlkDbyRCF3F8mNDujE49rLWcCgYEA0/MU
dX5A6VSDb5K+JCNq8vDaBKNGU8GAr2fpYAhtk/3mXLI+/Z0JN0di9ZgeNhhJr2jN
11OU/2pOButpsgnkIo2y36cOQPf5dQpSgXZke3iNDld3osuLIuPNJn/3C087AtOq
+w4YxZClZLAxiLCqX8SBVrB2IiFCQ70SJ++n8vkCgYEAzmi3rBsNEA1jblVIh1PF
wJhD/bOQ4nBd92iUV8m9jZdl4wl4YX4u/IBI9MMkIG24YIe2VOl7s9Rk5+4/jNg/
4QQ2998Y6aljxOZJEdZ+3jQELy4m49OhrTRq2ta5t/Z3CMsJTmLe6f9NXWZpr5iK
8iVdHOjtMXxqfYaR2jVNEtsCgYAl9uWUQiAoa037v0I1wO5YQ9IZgJGJUSDWynsg
C4JtPs5zji4ASY+sCipsqWnH8MPKGrC8QClxMr51ONe+30yw78a5jvfbpU9Wqpmq
vOU0xJwnlH1GeMUcY8eMfOFocjG0yOtYeubvBIDLr0/AFzz9WHp+Z69RX7m53nUR
GDlyKQKBgDGZVAbUBiB8rerqNbONBAxfipoa4IJ+ntBrFT2DtoIZNbSzaoK+nVbH
kbWMJycaV5PVOh1lfAiZeWCxQz5RcZh/RS8USnxyMG1j4dP/wLcbdasI8uRaSC6Y
hFHL5HjhLrIo0HRWySS2b2ztBI2FP1M+MaaGFPHDzm2OyZg85yr3
-----END RSA PRIVATE KEY-----
@@ -0,0 +1,164 @@
use std::sync::Arc;
use std::time::Duration;
use futures::StreamExt;
use hickory_resolver::TokioAsyncResolver;
use lettre::transport::smtp::AsyncSmtpTransport;
use lettre::transport::smtp::Error as SmtpError;
use lettre::transport::smtp::client::{Tls, TlsParameters};
use lettre::transport::smtp::extension::ClientId;
use lettre::{AsyncTransport, Message, Tokio1Executor};
use tokio::sync::Semaphore;
use super::message::recipient_domain;
use super::mx;
use super::types::{HeloName, MxRecord};
use crate::sender::SendError;
pub enum SendMode {
Smarthost {
transport: Box<AsyncSmtpTransport<Tokio1Executor>>,
total_timeout: Duration,
},
DirectMx {
resolver: Arc<TokioAsyncResolver>,
helo: HeloName,
command_timeout: Duration,
total_timeout: Duration,
require_tls: bool,
inflight: Arc<Semaphore>,
},
}
impl std::fmt::Debug for SendMode {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Smarthost { total_timeout, .. } => {
write!(f, "SendMode::Smarthost(total_timeout={total_timeout:?})")
}
Self::DirectMx {
helo, require_tls, ..
} => write!(
f,
"SendMode::DirectMx({}, require_tls={require_tls})",
helo.as_str()
),
}
}
}
pub async fn dispatch(mode: &SendMode, message: Message) -> Result<(), SendError> {
match mode {
SendMode::Smarthost {
transport,
total_timeout,
} => with_total_timeout(*total_timeout, run_send(transport, message)).await,
SendMode::DirectMx {
resolver,
helo,
command_timeout,
total_timeout,
require_tls,
inflight,
} => {
with_total_timeout(*total_timeout, async {
let _permit =
inflight.clone().acquire_owned().await.map_err(|_| {
SendError::SmtpTransient("send semaphore closed".to_string())
})?;
send_direct(
resolver.as_ref(),
helo,
*command_timeout,
*require_tls,
message,
)
.await
})
.await
}
}
}
async fn with_total_timeout<F: std::future::Future<Output = Result<(), SendError>>>(
total: Duration,
fut: F,
) -> Result<(), SendError> {
tokio::time::timeout(total, fut)
.await
.unwrap_or(Err(SendError::Timeout))
}
async fn run_send(
transport: &AsyncSmtpTransport<Tokio1Executor>,
message: Message,
) -> Result<(), SendError> {
transport
.send(message)
.await
.map(|_| ())
.map_err(classify_smtp_error)
}
async fn send_direct(
resolver: &TokioAsyncResolver,
helo: &HeloName,
command_timeout: Duration,
require_tls: bool,
message: Message,
) -> Result<(), SendError> {
let domain = recipient_domain(&message)?;
let mxs = mx::resolve(resolver, &domain).await?;
let outcome = futures::stream::iter(mxs)
.fold(None::<Result<(), SendError>>, |acc, mx_record| {
let message = message.clone();
async move {
match &acc {
Some(Ok(())) | Some(Err(SendError::SmtpPermanent(_))) => acc,
_ => Some(
attempt_one_host(mx_record, helo, command_timeout, require_tls, message)
.await,
),
}
}
})
.await;
outcome.unwrap_or_else(|| {
Err(SendError::SmtpTransient(format!(
"no MX records returned for {}",
domain.as_str()
)))
})
}
async fn attempt_one_host(
mx_record: MxRecord,
helo: &HeloName,
command_timeout: Duration,
require_tls: bool,
message: Message,
) -> Result<(), SendError> {
let host = mx_record.host.as_str().to_string();
let tls_params = TlsParameters::new(host.clone())
.map_err(|e| SendError::SmtpTransient(format!("TLS params for {host}: {e}")))?;
let tls = match require_tls {
true => Tls::Required(tls_params),
false => Tls::Opportunistic(tls_params),
};
let transport: AsyncSmtpTransport<Tokio1Executor> =
AsyncSmtpTransport::<Tokio1Executor>::builder_dangerous(&host)
.port(25)
.tls(tls)
.hello_name(ClientId::Domain(helo.as_str().to_string()))
.timeout(Some(command_timeout))
.build();
run_send(&transport, message).await
}
fn classify_smtp_error(e: SmtpError) -> SendError {
match () {
_ if e.is_permanent() => SendError::SmtpPermanent(e.to_string()),
_ if e.is_timeout() => SendError::Timeout,
_ => SendError::SmtpTransient(e.to_string()),
}
}
+291
View File
@@ -0,0 +1,291 @@
use std::path::PathBuf;
#[derive(Debug, thiserror::Error)]
pub enum ParseError {
#[error("empty value")]
Empty,
#[error("invalid character {0:?}")]
InvalidChar(char),
#[error("zero {0}")]
Zero(&'static str),
#[error("invalid TLS mode {0:?}")]
InvalidTlsMode(String),
}
fn parse_token(raw: &str, lowercase: bool, strip_trailing_dot: bool) -> Result<String, ParseError> {
let mut s = raw.trim();
if strip_trailing_dot {
s = s.trim_end_matches('.');
}
match s {
"" => Err(ParseError::Empty),
_ if s.chars().any(char::is_whitespace) => Err(ParseError::InvalidChar(' ')),
_ => Ok(match lowercase {
true => s.to_lowercase(),
false => s.to_string(),
}),
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct SmtpHost(String);
impl SmtpHost {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
parse_token(raw, true, false).map(Self)
}
pub fn as_str(&self) -> &str {
&self.0
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct SmtpPort(u16);
impl SmtpPort {
pub fn parse(raw: u16) -> Result<Self, ParseError> {
match raw {
0 => Err(ParseError::Zero("smtp port")),
n => Ok(Self(n)),
}
}
pub fn as_u16(self) -> u16 {
self.0
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct HeloName(String);
impl HeloName {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
parse_token(raw, false, false).map(Self)
}
pub fn as_str(&self) -> &str {
&self.0
}
pub fn into_inner(self) -> String {
self.0
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct EmailDomain(String);
impl EmailDomain {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
parse_token(raw, true, true).map(Self)
}
pub fn as_str(&self) -> &str {
&self.0
}
pub fn into_inner(self) -> String {
self.0
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct MxHost(String);
impl MxHost {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
parse_token(raw, true, true).map(Self)
}
pub fn as_str(&self) -> &str {
&self.0
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct MxPriority(u16);
impl MxPriority {
pub fn new(value: u16) -> Self {
Self(value)
}
pub fn as_u16(self) -> u16 {
self.0
}
}
#[derive(Debug, Clone)]
pub struct MxRecord {
pub priority: MxPriority,
pub host: MxHost,
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct DkimSelector(String);
impl DkimSelector {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
let trimmed = raw.trim();
let valid = !trimmed.is_empty() && trimmed.split('.').all(valid_subdomain);
match valid {
true => Ok(Self(trimmed.to_string())),
false => Err(ParseError::InvalidChar('?')),
}
}
pub fn into_inner(self) -> String {
self.0
}
}
fn valid_subdomain(seg: &str) -> bool {
let starts_alnum = seg
.chars()
.next()
.is_some_and(|c| c.is_ascii_alphanumeric());
let ends_alnum = seg
.chars()
.next_back()
.is_some_and(|c| c.is_ascii_alphanumeric());
let body_ok = seg.chars().all(|c| c.is_ascii_alphanumeric() || c == '-');
starts_alnum && ends_alnum && body_ok
}
#[derive(Debug, Clone)]
pub struct DkimKeyPath(PathBuf);
impl DkimKeyPath {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
let trimmed = raw.trim();
match trimmed.is_empty() {
true => Err(ParseError::Empty),
false => Ok(Self(PathBuf::from(trimmed))),
}
}
pub fn as_path(&self) -> &std::path::Path {
&self.0
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SmtpUsername(String);
impl SmtpUsername {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
match raw.is_empty() {
true => Err(ParseError::Empty),
false => Ok(Self(raw.to_string())),
}
}
pub fn into_inner(self) -> String {
self.0
}
}
#[derive(Clone)]
pub struct SmtpPassword(secrecy::SecretString);
impl SmtpPassword {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
match raw.is_empty() {
true => Err(ParseError::Empty),
false => Ok(Self(secrecy::SecretString::from(raw.to_string()))),
}
}
pub fn expose(&self) -> &str {
use secrecy::ExposeSecret;
self.0.expose_secret()
}
}
impl std::fmt::Debug for SmtpPassword {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("SmtpPassword(***)")
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TlsMode {
Implicit,
Starttls,
None,
}
impl TlsMode {
pub fn parse(raw: &str) -> Result<Self, ParseError> {
match raw.to_ascii_lowercase().as_str() {
"implicit" => Ok(Self::Implicit),
"starttls" => Ok(Self::Starttls),
"none" => Ok(Self::None),
other => Err(ParseError::InvalidTlsMode(other.to_string())),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn smtp_host_lowercases_and_trims() {
let h = SmtpHost::parse(" SMTP.NEL.PET ").unwrap();
assert_eq!(h.as_str(), "smtp.nel.pet");
}
#[test]
fn smtp_host_rejects_whitespace() {
assert!(SmtpHost::parse("a b").is_err());
}
#[test]
fn smtp_host_rejects_empty() {
assert!(SmtpHost::parse("").is_err());
assert!(SmtpHost::parse(" ").is_err());
}
#[test]
fn smtp_port_rejects_zero() {
assert!(SmtpPort::parse(0).is_err());
assert_eq!(SmtpPort::parse(587).unwrap().as_u16(), 587);
}
#[test]
fn email_domain_strips_trailing_dot() {
assert_eq!(EmailDomain::parse("Nel.pet.").unwrap().as_str(), "nel.pet");
}
#[test]
fn dkim_selector_validates() {
assert!(DkimSelector::parse("default").is_ok());
assert!(DkimSelector::parse("s1.nel.pet").is_ok());
assert!(DkimSelector::parse("s2024-q1").is_ok());
assert!(DkimSelector::parse("mailo-2024.nel.pet").is_ok());
assert!(DkimSelector::parse("a-b").is_ok());
assert!(DkimSelector::parse("").is_err());
assert!(DkimSelector::parse("a..b").is_err());
assert!(DkimSelector::parse("-leading").is_err());
assert!(DkimSelector::parse("trailing-").is_err());
assert!(DkimSelector::parse("s_under").is_err());
}
#[test]
fn tls_mode_parses_known_modes() {
assert_eq!(TlsMode::parse("STARTTLS").unwrap(), TlsMode::Starttls);
assert_eq!(TlsMode::parse("implicit").unwrap(), TlsMode::Implicit);
assert_eq!(TlsMode::parse("none").unwrap(), TlsMode::None);
assert!(TlsMode::parse("garbage").is_err());
}
#[test]
fn smtp_password_redacts_in_debug() {
let p = SmtpPassword::parse("hunter2").unwrap();
let dbg = format!("{:?}", p);
assert_eq!(dbg, "SmtpPassword(***)");
assert!(!dbg.contains("hunter2"));
}
}
+4 -2
View File
@@ -1,13 +1,15 @@
pub mod email;
mod locale;
mod sender;
mod types;
pub use email::EmailSender;
pub use locale::{
DEFAULT_LOCALE, NotificationStrings, VALID_LOCALES, format_message, get_strings,
validate_locale,
};
pub use sender::{
CommsSender, DiscordSender, EmailSender, SendError, SignalSender, TelegramSender,
is_valid_phone_number, is_valid_signal_username, mime_encode_header, sanitize_header_value,
CommsSender, DiscordSender, SendError, SignalSender, TelegramSender, is_valid_phone_number,
is_valid_signal_username,
};
pub use types::{CommsChannel, CommsStatus, CommsType, NewComms, QueuedComms};
+37 -1
View File
@@ -1,5 +1,5 @@
pub const DEFAULT_LOCALE: &str = "en";
pub const VALID_LOCALES: &[&str] = &["en", "zh", "ja", "ko", "sv", "fi"];
pub const VALID_LOCALES: &[&str] = &["en", "zh", "ja", "ko", "sv", "fi", "fr"];
pub fn validate_locale(locale: &str) -> &str {
if VALID_LOCALES.contains(&locale) {
@@ -44,6 +44,7 @@ pub fn get_strings(locale: &str) -> &'static NotificationStrings {
"ko" => &STRINGS_KO,
"sv" => &STRINGS_SV,
"fi" => &STRINGS_FI,
"fr" => &STRINGS_FR,
_ => &STRINGS_EN,
}
}
@@ -216,6 +217,34 @@ static STRINGS_FI: NotificationStrings = NotificationStrings {
channel_verification_body: "Vahvistuskoodisi on:\n{code}\n\nTai vahvista suoraan:\n{verify_link}",
};
static STRINGS_FR: NotificationStrings = NotificationStrings {
welcome_subject: "Bienvenue sur {hostname}",
welcome_body: "Bienvenue sur {hostname} !\n\nVotre identifiant est : @{handle}\n\nMerci de nous avoir rejoint.",
password_reset_subject: "Réinitialisation du mot de passe - {hostname}",
password_reset_body: "Bonjour @{handle},\n\nVotre code de réinitialisation du mot de passe est : {code}\n\nCe code expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, veuillez ignorer ce message.",
email_update_subject: "Confirmer votre nouvelle adresse e-mail - {hostname}",
email_update_body: "Bonjour @{handle},\n\nVotre code de vérification est :\n{code}\n\nCopiez le code ci-dessus et saisissez-le ici :\n{verify_page}\n\nCe code expirera dans 10 minutes.\n\nOu si vous aimez vivre dangereusement :\n{verify_link}\n\nSi vous n'avez pas demandé cela, veuillez ignorer cet e-mail.",
short_token_body: "Bonjour @{handle},\n\nVotre code de vérification est :\n{code}\n\nCe code expirera dans 15 minutes.\n\nSi vous n'avez pas demandé cela, veuillez ignorer cet e-mail.",
account_deletion_subject: "Demande de suppression de compte - {hostname}",
account_deletion_body: "Bonjour @{handle},\n\nVotre code de confirmation de suppression de compte est : {code}\n\nCe code expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, sécurisez votre compte immédiatement.",
plc_operation_subject: "{hostname} - Jeton d'opération PLC",
plc_operation_body: "Bonjour @{handle},\n\nVous avez demandé à signer une opération PLC pour votre compte.\n\nVotre jeton de vérification est : {token}\n\nCe jeton expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, vous pouvez ignorer ce message en toute sécurité.",
two_factor_code_subject: "Vérification de connexion - {hostname}",
two_factor_code_body: "Bonjour @{handle},\n\nVotre code de vérification de connexion est : {code}\n\nCe code expirera dans 10 minutes.\n\nSi vous n'avez pas demandé cela, sécurisez votre compte immédiatement.",
passkey_recovery_subject: "Récupération de compte - {hostname}",
passkey_recovery_body: "Bonjour @{handle},\n\nVous avez demandé la récupération de votre compte à clé d'accès uniquement.\n\nCliquez sur le lien ci-dessous pour définir un mot de passe temporaire et retrouver l'accès :\n{url}\n\nCe lien expirera dans 1 heure.\n\nSi vous n'avez pas demandé cela, veuillez ignorer ce message. Votre compte reste sécurisé.",
signup_verification_subject: "Vérifier votre compte - {hostname}",
signup_verification_body: "Bienvenue ! Votre code de vérification est :\n{code}\n\nCopiez le code ci-dessus et saisissez-le ici :\n{verify_page}\n\nCe code expirera dans 30 minutes.\n\nOu si vous aimez vivre dangereusement :\n{verify_link}\n\nSi vous n'avez pas créé de compte sur {hostname}, veuillez ignorer ce message.",
legacy_login_subject: "Alerte de sécurité : Connexion classique détectée - {hostname}",
legacy_login_body: "Bonjour @{handle},\n\nUne connexion à votre compte a été détectée via une application classique (comme Bluesky) qui ne prend pas en charge la vérification TOTP.\n\nDétails :\n- Date : {timestamp}\n- Adresse IP : {ip}\n\nVotre protection TOTP a été contournée pour cette connexion. La session dispose de permissions limitées pour les opérations sensibles.\n\nSi ce n'était pas vous :\n1. Changez votre mot de passe immédiatement\n2. Vérifiez vos sessions actives\n3. Envisagez de désactiver les connexions d'applications classiques dans vos paramètres de sécurité\n\nRestez vigilant,\n{hostname}",
migration_verification_subject: "Vérifier votre adresse e-mail - {hostname}",
migration_verification_body: "Bienvenue sur {hostname} !\n\nVotre compte a été migré avec succès. Pour finaliser la configuration, veuillez vérifier votre adresse e-mail.\n\nVotre code de vérification est :\n{code}\n\nCopiez le code ci-dessus et saisissez-le ici :\n{verify_page}\n\nCe code expirera dans 48 heures.\n\nOu si vous aimez vivre dangereusement :\n{verify_link}\n\nSi vous n'avez pas migré votre compte, veuillez ignorer cet e-mail.",
channel_verified_subject: "Canal de notification vérifié - {hostname}",
channel_verified_body: "Bonjour {handle},\n\n{channel} a été vérifié comme canal de notification pour votre compte sur {hostname}.",
channel_verification_subject: "Vérifier votre canal - {hostname}",
channel_verification_body: "Votre code de vérification est :\n{code}\n\nOu vérifiez directement :\n{verify_link}",
};
pub fn format_message(template: &str, vars: &[(&str, &str)]) -> String {
vars.iter()
.fold(template.to_string(), |result, (key, value)| {
@@ -233,6 +262,9 @@ mod tests {
assert_eq!(validate_locale("zh"), "zh");
assert_eq!(validate_locale("ja"), "ja");
assert_eq!(validate_locale("ko"), "ko");
assert_eq!(validate_locale("sv"), "sv");
assert_eq!(validate_locale("fi"), "fi");
assert_eq!(validate_locale("fr"), "fr");
assert_eq!(validate_locale("invalid"), DEFAULT_LOCALE);
assert_eq!(validate_locale(""), DEFAULT_LOCALE);
}
@@ -252,5 +284,9 @@ mod tests {
let zh = get_strings("zh");
assert!(zh.welcome_subject.contains("{hostname}"));
assert!(zh.welcome_body.contains("欢迎"));
let fr = get_strings("fr");
assert!(fr.welcome_subject.contains("{hostname}"));
assert!(fr.welcome_body.contains("Bienvenue"));
}
}
+60 -113
View File
@@ -1,11 +1,7 @@
use async_trait::async_trait;
use base64::{Engine, engine::general_purpose::STANDARD as BASE64};
use reqwest::Client;
use serde_json::json;
use std::process::Stdio;
use std::time::Duration;
use tokio::io::AsyncWriteExt;
use tokio::process::Command;
use super::types::{CommsChannel, QueuedComms};
@@ -21,25 +17,51 @@ pub trait CommsSender: Send + Sync {
#[derive(Debug, thiserror::Error)]
pub enum SendError {
#[error("Failed to spawn {command}: {source}")]
ProcessSpawn {
command: String,
source: std::io::Error,
},
#[error("{command} exited with non-zero status: {detail}")]
ProcessFailed { command: String, detail: String },
#[error("Channel not configured: {0:?}")]
NotConfigured(CommsChannel),
#[error("External service error: {0}")]
ExternalService(String),
#[error("Email configuration invalid: {0}")]
ConfigInvalid(String),
#[error("Invalid recipient format: {0}")]
InvalidRecipient(String),
#[error("Message construction failed: {0}")]
MessageBuild(String),
#[error("transient DNS lookup failure: {0}")]
DnsTransient(String),
#[error("permanent DNS lookup failure: {0}")]
DnsPermanent(String),
#[error("SMTP transient error: {0}")]
SmtpTransient(String),
#[error("SMTP permanent error: {0}")]
SmtpPermanent(String),
#[error("DKIM signing failed: {0}")]
DkimSign(String),
#[error("External service error: {0}")]
ExternalService(String),
#[error("Request timeout")]
Timeout,
#[error("Max retries exceeded: {0}")]
MaxRetriesExceeded(String),
}
impl SendError {
pub fn is_permanent(&self) -> bool {
match self {
Self::SmtpPermanent(_)
| Self::DnsPermanent(_)
| Self::InvalidRecipient(_)
| Self::MessageBuild(_)
| Self::DkimSign(_)
| Self::ConfigInvalid(_) => true,
Self::SmtpTransient(_)
| Self::DnsTransient(_)
| Self::Timeout
| Self::ExternalService(_)
| Self::MaxRetriesExceeded(_)
| Self::NotConfigured(_) => false,
}
}
}
fn create_http_client() -> Client {
Client::builder()
.timeout(Duration::from_secs(HTTP_TIMEOUT_SECS))
@@ -100,19 +122,6 @@ where
))
}
pub fn sanitize_header_value(value: &str) -> String {
value.replace(['\r', '\n'], " ").trim().to_string()
}
pub fn mime_encode_header(value: &str) -> String {
if value.is_ascii() {
sanitize_header_value(value)
} else {
let sanitized = sanitize_header_value(value);
format!("=?UTF-8?B?{}?=", BASE64.encode(sanitized.as_bytes()))
}
}
pub fn escape_html(text: &str) -> String {
text.replace('&', "&amp;")
.replace('<', "&lt;")
@@ -135,93 +144,6 @@ pub fn is_valid_signal_username(username: &str) -> bool {
tranquil_signal::SignalUsername::parse(username).is_ok()
}
pub struct EmailSender {
from_address: String,
from_name: String,
sendmail_path: String,
}
impl EmailSender {
pub fn new(from_address: String, from_name: String, sendmail_path: String) -> Self {
Self {
from_address,
from_name,
sendmail_path,
}
}
pub fn from_config(cfg: &tranquil_config::TranquilConfig) -> Option<Self> {
let from_address = cfg.email.from_address.clone()?;
let from_name = cfg.email.from_name.clone();
let sendmail_path = cfg.email.sendmail_path.clone();
Some(Self::new(from_address, from_name, sendmail_path))
}
pub fn format_email(&self, notification: &QueuedComms) -> String {
let subject = mime_encode_header(notification.subject.as_deref().unwrap_or("Notification"));
let recipient = sanitize_header_value(&notification.recipient);
let from_header = if self.from_name.is_empty() {
self.from_address.clone()
} else {
format!(
"{} <{}>",
sanitize_header_value(&self.from_name),
self.from_address
)
};
format!(
"From: {}\r\nTo: {}\r\nSubject: {}\r\nContent-Type: text/plain; charset=utf-8\r\nMIME-Version: 1.0\r\n\r\n{}",
from_header, recipient, subject, notification.body
)
}
}
#[async_trait]
impl CommsSender for EmailSender {
fn channel(&self) -> CommsChannel {
CommsChannel::Email
}
async fn send(&self, notification: &QueuedComms) -> Result<(), SendError> {
let email_content = self.format_email(notification);
let mut child = Command::new(&self.sendmail_path)
.arg("-t")
.arg("-oi")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.map_err(|e| SendError::ProcessSpawn {
command: self.sendmail_path.clone(),
source: e,
})?;
if let Some(mut stdin) = child.stdin.take() {
stdin
.write_all(email_content.as_bytes())
.await
.map_err(|e| SendError::ProcessSpawn {
command: self.sendmail_path.clone(),
source: e,
})?;
}
let output = child
.wait_with_output()
.await
.map_err(|e| SendError::ProcessSpawn {
command: self.sendmail_path.clone(),
source: e,
})?;
if !output.status.success() {
let stderr = String::from_utf8_lossy(&output.stderr);
return Err(SendError::ProcessFailed {
command: self.sendmail_path.clone(),
detail: stderr.to_string(),
});
}
Ok(())
}
}
const DISCORD_API_BASE: &str = "https://discord.com/api/v10";
#[derive(Clone)]
@@ -610,3 +532,28 @@ impl CommsSender for SignalSender {
))
}
}
#[cfg(test)]
mod is_permanent_matrix {
use super::{CommsChannel, SendError};
#[test]
fn permanent_variants_are_permanent() {
assert!(SendError::SmtpPermanent("x".into()).is_permanent());
assert!(SendError::DnsPermanent("x".into()).is_permanent());
assert!(SendError::InvalidRecipient("x".into()).is_permanent());
assert!(SendError::MessageBuild("x".into()).is_permanent());
assert!(SendError::DkimSign("x".into()).is_permanent());
assert!(SendError::ConfigInvalid("x".into()).is_permanent());
}
#[test]
fn transient_variants_are_not_permanent() {
assert!(!SendError::SmtpTransient("x".into()).is_permanent());
assert!(!SendError::DnsTransient("x".into()).is_permanent());
assert!(!SendError::Timeout.is_permanent());
assert!(!SendError::ExternalService("x".into()).is_permanent());
assert!(!SendError::MaxRetriesExceeded("x".into()).is_permanent());
assert!(!SendError::NotConfigured(CommsChannel::Email).is_permanent());
}
}
+158
View File
@@ -0,0 +1,158 @@
use std::time::Duration;
use chrono::Utc;
use lettre::message::Mailbox;
use lettre::transport::smtp::AsyncSmtpTransport;
use lettre::transport::smtp::extension::ClientId;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::{TcpListener, TcpStream};
use tranquil_comms::email::transport::SendMode;
use tranquil_comms::email::{EmailSender, types::HeloName};
use tranquil_comms::{CommsChannel, CommsSender, CommsStatus, CommsType, QueuedComms, SendError};
use uuid::Uuid;
fn fixture(recipient: &str, subject: &str, body: &str) -> QueuedComms {
QueuedComms {
id: Uuid::new_v4(),
user_id: None,
channel: CommsChannel::Email,
comms_type: CommsType::Welcome,
status: CommsStatus::Pending,
recipient: recipient.to_string(),
subject: Some(subject.to_string()),
body: body.to_string(),
metadata: None,
attempts: 0,
max_attempts: 3,
last_error: None,
created_at: Utc::now(),
updated_at: Utc::now(),
scheduled_for: Utc::now(),
processed_at: None,
}
}
fn build_smarthost_sender(host: &str, port: u16) -> EmailSender {
build_smarthost_sender_with_total_timeout(host, port, Duration::from_secs(10))
}
fn build_smarthost_sender_with_total_timeout(
host: &str,
port: u16,
total_timeout: Duration,
) -> EmailSender {
let from: Mailbox = "Tranquil Test <noreply@nel.pet>".parse().unwrap();
let helo = HeloName::parse("mta.nel.pet").unwrap();
let transport = AsyncSmtpTransport::<lettre::Tokio1Executor>::builder_dangerous(host)
.port(port)
.hello_name(ClientId::Domain(helo.into_inner()))
.timeout(Some(Duration::from_secs(5)))
.build();
EmailSender::new(
from,
SendMode::Smarthost {
transport: Box::new(transport),
total_timeout,
},
None,
)
}
async fn drive_stub(stream: TcpStream, rcpt_response: &'static [u8]) -> std::io::Result<()> {
let (read, mut write) = stream.into_split();
let mut reader = BufReader::new(read);
write.write_all(b"220 stub ESMTP\r\n").await?;
let mut line = String::new();
loop {
line.clear();
let n = reader.read_line(&mut line).await?;
if n == 0 {
return Ok(());
}
let upper = line.to_ascii_uppercase();
let response: &[u8] = match upper.split_whitespace().next() {
Some("EHLO") | Some("HELO") => b"250-stub\r\n250 SIZE 10240000\r\n",
Some("MAIL") => b"250 OK\r\n",
Some("RCPT") => rcpt_response,
Some("DATA") => b"354 end with .\r\n",
Some("RSET") => b"250 OK\r\n",
Some("QUIT") => b"221 bye\r\n",
_ => b"500 unknown\r\n",
};
write.write_all(response).await?;
if upper.starts_with("QUIT") {
return Ok(());
}
}
}
async fn spawn_stub(rcpt_response: &'static [u8]) -> u16 {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let _ = drive_stub(stream, rcpt_response).await;
});
port
}
#[tokio::test]
async fn rcpt_550_classifies_as_smtp_permanent() {
let port = spawn_stub(b"550 5.1.1 user unknown\r\n").await;
let sender = build_smarthost_sender("127.0.0.1", port);
let result = sender.send(&fixture("nel@nel.pet", "x", "x")).await;
match result {
Err(SendError::SmtpPermanent(_)) => {}
other => panic!("expected SmtpPermanent, got {other:?}"),
}
}
#[tokio::test]
async fn rcpt_421_classifies_as_smtp_transient() {
let port = spawn_stub(b"421 4.7.0 try again later\r\n").await;
let sender = build_smarthost_sender("127.0.0.1", port);
let result = sender.send(&fixture("nel@nel.pet", "x", "x")).await;
match result {
Err(SendError::SmtpTransient(_)) => {}
other => panic!("expected SmtpTransient, got {other:?}"),
}
}
#[tokio::test]
async fn invalid_recipient_classifies_as_invalid_recipient() {
let port = spawn_stub(b"250 OK\r\n").await;
let sender = build_smarthost_sender("127.0.0.1", port);
let result = sender.send(&fixture("not-an-address", "x", "x")).await;
match result {
Err(SendError::InvalidRecipient(_)) => {}
other => panic!("expected InvalidRecipient, got {other:?}"),
}
}
async fn spawn_silent_stub() -> u16 {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let port = listener.local_addr().unwrap().port();
tokio::spawn(async move {
let (_stream, _) = listener.accept().await.unwrap();
std::future::pending::<()>().await;
});
port
}
#[tokio::test]
async fn smarthost_silent_relay_hits_total_timeout() {
let port = spawn_silent_stub().await;
let sender =
build_smarthost_sender_with_total_timeout("127.0.0.1", port, Duration::from_millis(500));
let start = std::time::Instant::now();
let result = sender.send(&fixture("nel@nel.pet", "x", "x")).await;
let elapsed = start.elapsed();
match result {
Err(SendError::Timeout) => {}
other => panic!("expected Timeout, got {other:?}"),
}
assert!(
elapsed < Duration::from_secs(2),
"send returned in {elapsed:?}, expected close to 500ms total_timeout"
);
}
+717 -14
View File
@@ -5,6 +5,14 @@ use std::sync::OnceLock;
static CONFIG: OnceLock<TranquilConfig> = OnceLock::new();
const REMOVED_ENV_VARS: &[(&str, &str)] = &[(
"SENDMAIL_PATH",
"the sendmail-binary transport was replaced with native SMTP. \
Configure MAIL_SMARTHOST_HOST for relay delivery, or leave it unset to \
deliver directly via recipient MX records. See example.toml for the full \
MAIL_* surface.",
)];
/// Errors discovered during configuration validation.
#[derive(Debug)]
pub struct ConfigError {
@@ -96,6 +104,7 @@ pub fn load(config_path: Option<&PathBuf>) -> Result<TranquilConfig, confique::E
// Root configuration
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TranquilConfig {
#[config(nested)]
pub server: ServerConfig,
@@ -133,9 +142,6 @@ pub struct TranquilConfig {
#[config(nested)]
pub telegram: TelegramConfig,
#[config(nested)]
pub signal: SignalConfig,
#[config(nested)]
pub notifications: NotificationConfig,
@@ -162,6 +168,14 @@ impl TranquilConfig {
pub fn validate(&self, ignore_secrets: bool) -> Result<(), ConfigError> {
let mut errors = Vec::new();
// -- removed config ---------------------------------------------------
errors.extend(
REMOVED_ENV_VARS
.iter()
.filter(|(var, _)| std::env::var_os(var).is_some())
.map(|(var, guidance)| format!("{var} is no longer supported: {guidance}")),
);
// -- secrets ----------------------------------------------------------
if !ignore_secrets && !self.secrets.allow_insecure && !cfg!(test) {
if let Some(ref s) = self.secrets.jwt_secret {
@@ -210,6 +224,10 @@ impl TranquilConfig {
}
}
// -- email -----------------------------------------------------------
self.email
.validate(self.server.hostname_without_port(), &mut errors);
// -- telegram ---------------------------------------------------------
if self.telegram.bot_token.is_some() && self.telegram.webhook_secret.is_none() {
errors.push(
@@ -238,6 +256,9 @@ impl TranquilConfig {
}
}
// -- tls --------------------------------------------------------------
self.server.tls.validate(&mut errors);
// -- SSO providers ----------------------------------------------------
self.validate_sso_provider("sso.github", &self.sso.github, &mut errors);
self.validate_sso_provider("sso.google", &self.sso.google, &mut errors);
@@ -397,6 +418,7 @@ impl TranquilConfig {
// ---------------------------------------------------------------------------
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ServerConfig {
/// Public hostname of the PDS, such as `pds.example.com`.
#[config(env = "PDS_HOSTNAME")]
@@ -436,6 +458,11 @@ pub struct ServerConfig {
#[config(env = "DISABLE_RATE_LIMITING", default = false)]
pub disable_rate_limiting: bool,
/// Skip the verified-comms-channel gate for login and record writes.
/// Please keep this off unless you're an invite-only PDS!
#[config(env = "DISABLE_ACCOUNT_VERIFICATION_GATE", default = false)]
pub disable_account_verification_gate: bool,
/// List of additional banned words for handle validation.
#[config(env = "PDS_BANNED_WORDS", parse_env = split_comma_list)]
pub banned_words: Option<Vec<String>>,
@@ -459,6 +486,65 @@ pub struct ServerConfig {
/// Maximum allowed number of preferences
#[config(env = "MAX_PREFERENCES_COUNT", default = 1000)]
pub max_preferences_count: usize,
/// If you're not altering TLS config, you don't have to worry about this.
/// This is the number of trusted reverse proxies in front of Tranquil.
/// We read the client IP used for rate limiting and device records this many hops
/// from the right of the X-Forwarded-For header.
/// When left unset, Tranquil will assume:
/// - 0, if the TLS termination is happening here on Tranquil via the TLS config
/// - 1, if the TLS termination *isn't* happening here.
#[config(env = "TRUSTED_PROXY_COUNT")]
pub trusted_proxy_count: Option<usize>,
#[config(nested)]
pub tls: TlsConfig,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TlsConfig {
/// The path to the TLS cert chain.
/// If you set both this and `key_path`, the server terminates TLS itself rather than expecting
/// a reverse proxy to do it. The certificate and key reload on SIGHUP.
#[config(env = "TLS_CERT_PATH")]
pub cert_path: Option<String>,
/// Path to the TLS private key.
#[config(env = "TLS_KEY_PATH")]
pub key_path: Option<String>,
/// Serve HTTP/3 over QUIC on the same UDP port as the TCP listener.
/// Requires cert_path and key_path.
#[config(env = "TLS_HTTP3", default = false)]
pub http3: bool,
}
impl TlsConfig {
/// The certificate and key paths when both are configured.
pub fn material(&self) -> Option<(&str, &str)> {
match (self.cert_path.as_deref(), self.key_path.as_deref()) {
(Some(cert), Some(key)) => Some((cert, key)),
_ => None,
}
}
pub fn validate(&self, errors: &mut Vec<String>) {
if self.cert_path.is_some() != self.key_path.is_some() {
errors.push(
"server.tls.cert_path (TLS_CERT_PATH) and server.tls.key_path (TLS_KEY_PATH) \
must both be set to enable app-level TLS, or both be unset"
.to_string(),
);
}
if self.http3 && self.material().is_none() {
errors.push(
"server.tls.http3 (TLS_HTTP3) requires server.tls.cert_path \
and erver.tls.key_path"
.to_string(),
);
}
}
}
impl ServerConfig {
@@ -493,6 +579,7 @@ impl ServerConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct FrontendConfig {
/// Whether to enable the built in serving of the frontend.
#[config(env = "FRONTEND_ENABLED", default = true)]
@@ -505,6 +592,7 @@ pub struct FrontendConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DatabaseConfig {
/// PostgreSQL connection URL.
#[config(env = "DATABASE_URL")]
@@ -524,6 +612,7 @@ pub struct DatabaseConfig {
}
#[derive(Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SecretsConfig {
/// Secret used for signing JWTs. Must be at least 32 characters in
/// production.
@@ -650,6 +739,7 @@ impl fmt::Display for RepoBackend {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct StorageConfig {
/// Storage backend: `filesystem` or `s3`.
#[config(env = "BLOB_STORAGE_BACKEND", default = "filesystem")]
@@ -682,6 +772,7 @@ impl StorageConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct CacheConfig {
/// Cache backend: `ripple` by default, or `valkey`.
#[config(env = "CACHE_BACKEND", default = "ripple")]
@@ -696,6 +787,7 @@ pub struct CacheConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct PlcConfig {
/// Base URL of the PLC directory.
#[config(env = "PLC_DIRECTORY_URL", default = "https://plc.directory")]
@@ -715,6 +807,7 @@ pub struct PlcConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct FirehoseConfig {
/// Size of the in-memory broadcast buffer for firehose events.
#[config(env = "FIREHOSE_BUFFER_SIZE", default = 10000)]
@@ -745,6 +838,7 @@ impl FirehoseConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct EmailConfig {
/// Sender email address. When unset, email sending is disabled.
#[config(env = "MAIL_FROM_ADDRESS")]
@@ -754,12 +848,285 @@ pub struct EmailConfig {
#[config(env = "MAIL_FROM_NAME", default = "Tranquil PDS")]
pub from_name: String,
/// Path to the `sendmail` binary.
#[config(env = "SENDMAIL_PATH", default = "/usr/sbin/sendmail")]
pub sendmail_path: String,
/// HELO/EHLO name announced to remote SMTP servers. Applies to both
/// smarthost and direct-MX modes. Defaults to the server hostname.
#[config(env = "MAIL_HELO_NAME")]
pub helo_name: Option<String>,
#[config(nested)]
pub smarthost: SmarthostConfig,
#[config(nested)]
pub direct_mx: DirectMxConfig,
#[config(nested)]
pub dkim: DkimConfig,
}
impl EmailConfig {
pub fn validate(&self, server_hostname: &str, errors: &mut Vec<String>) {
match self.smarthost.tls.to_ascii_lowercase().as_str() {
"implicit" | "starttls" => {}
"none" => {
if self.smarthost.password.is_some() {
errors.push(
"email.smarthost.tls = \"none\" with email.smarthost.password set \
would transmit credentials in plaintext; use \"starttls\" or \"implicit\""
.to_string(),
);
}
}
other => errors.push(format!(
"email.smarthost.tls must be \"implicit\", \"starttls\", or \"none\", got \"{other}\""
)),
}
let smarthost_host_set = self
.smarthost
.host
.as_deref()
.is_some_and(|h| !h.is_empty());
let username_set = self.smarthost.username.is_some();
let password_set = self.smarthost.password.is_some();
if !smarthost_host_set && (username_set || password_set) {
errors.push(
"email.smarthost.username or email.smarthost.password is set but \
email.smarthost.host is empty; credentials would be silently ignored"
.to_string(),
);
}
if smarthost_host_set && username_set != password_set {
errors.push(
"email.smarthost.username and email.smarthost.password must both be set or \
both unset; otherwise authentication would silently degrade to anonymous"
.to_string(),
);
}
if self.smarthost.command_timeout_secs == 0 {
errors.push("email.smarthost.command_timeout_secs must be at least 1".to_string());
}
if self.smarthost.total_timeout_secs == 0 {
errors.push("email.smarthost.total_timeout_secs must be at least 1".to_string());
}
if self.smarthost.pool_size == 0 {
errors.push("email.smarthost.pool_size must be at least 1".to_string());
}
if self.direct_mx.max_concurrent_sends == 0 {
errors.push("email.direct_mx.max_concurrent_sends must be at least 1".to_string());
}
if self.direct_mx.command_timeout_secs == 0 {
errors.push("email.direct_mx.command_timeout_secs must be at least 1".to_string());
}
if self.direct_mx.total_timeout_secs == 0 {
errors.push("email.direct_mx.total_timeout_secs must be at least 1".to_string());
}
let dkim_set = self.dkim.selector.is_some()
|| self.dkim.domain.is_some()
|| self.dkim.private_key_path.is_some();
if dkim_set {
if self.dkim.selector.is_none() {
errors
.push("email.dkim.selector is required when any DKIM field is set".to_string());
}
if self.dkim.domain.is_none() {
errors.push("email.dkim.domain is required when any DKIM field is set".to_string());
}
if self.dkim.private_key_path.is_none() {
errors.push(
"email.dkim.private_key_path is required when any DKIM field is set"
.to_string(),
);
}
}
let Some(from_address) = self.from_address.as_deref().filter(|s| !s.is_empty()) else {
return;
};
if !looks_like_email_address(from_address) {
errors.push(format!(
"email.from_address {from_address:?} is not a valid email address"
));
}
if self.from_name.chars().any(|c| c.is_control()) {
errors.push("email.from_name must not contain control characters".to_string());
}
let helo_raw = self
.helo_name
.as_deref()
.map(str::to_string)
.unwrap_or_else(|| server_hostname.to_string());
if !is_non_whitespace_token(&helo_raw) {
errors.push(format!(
"email HELO name {helo_raw:?} must be non-empty and contain no whitespace"
));
}
if smarthost_host_set {
let host = self.smarthost.host.as_deref().unwrap_or("");
if !is_non_whitespace_token(host) {
errors.push(format!(
"email.smarthost.host {host:?} must contain no whitespace"
));
}
if self.smarthost.port == 0 {
errors.push("email.smarthost.port must be non-zero".to_string());
}
if let Some(u) = self.smarthost.username.as_deref()
&& u.is_empty()
{
errors.push("email.smarthost.username must be non-empty".to_string());
}
if let Some(p) = self.smarthost.password.as_deref()
&& p.is_empty()
{
errors.push("email.smarthost.password must be non-empty".to_string());
}
}
if let Some(selector) = self.dkim.selector.as_deref()
&& !is_valid_dkim_selector(selector)
{
errors.push(format!(
"email.dkim.selector {selector:?} must be valid subdomain syntax"
));
}
if let Some(domain) = self.dkim.domain.as_deref()
&& !is_non_whitespace_token(domain)
{
errors.push(format!(
"email.dkim.domain {domain:?} must be non-empty and contain no whitespace"
));
}
if let Some(key_path) = self.dkim.private_key_path.as_deref()
&& key_path.trim().is_empty()
{
errors.push("email.dkim.private_key_path must be non-empty".to_string());
}
}
}
fn looks_like_email_address(s: &str) -> bool {
let trimmed = s.trim();
if trimmed.is_empty() || trimmed.chars().any(char::is_whitespace) {
return false;
}
let mut parts = trimmed.split('@');
let local = parts.next().unwrap_or("");
let domain = parts.next().unwrap_or("");
parts.next().is_none() && !local.is_empty() && !domain.is_empty() && domain.contains('.')
}
fn is_non_whitespace_token(s: &str) -> bool {
let trimmed = s.trim();
!trimmed.is_empty() && !trimmed.chars().any(char::is_whitespace)
}
fn is_valid_dkim_selector(s: &str) -> bool {
let trimmed = s.trim();
!trimmed.is_empty()
&& trimmed.split('.').all(|seg| {
let starts_alnum = seg
.chars()
.next()
.is_some_and(|c| c.is_ascii_alphanumeric());
let ends_alnum = seg
.chars()
.next_back()
.is_some_and(|c| c.is_ascii_alphanumeric());
let body_ok = seg.chars().all(|c| c.is_ascii_alphanumeric() || c == '-');
starts_alnum && ends_alnum && body_ok
})
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SmarthostConfig {
/// SMTP relay host. When set, mail is delivered through this host
/// instead of resolving recipient MX records directly.
#[config(env = "MAIL_SMARTHOST_HOST")]
pub host: Option<String>,
/// SMTP relay port.
#[config(env = "MAIL_SMARTHOST_PORT", default = 587)]
pub port: u16,
/// SMTP authentication username.
#[config(env = "MAIL_SMARTHOST_USERNAME")]
pub username: Option<String>,
/// SMTP authentication password.
#[config(env = "MAIL_SMARTHOST_PASSWORD")]
pub password: Option<String>,
/// TLS mode. Valid values: "implicit", "starttls", "none". Setting "none"
/// alongside a password is rejected at startup to prevent transmitting
/// credentials in plaintext.
#[config(env = "MAIL_SMARTHOST_TLS", default = "starttls")]
pub tls: String,
/// Max size of the connection pool.
#[config(env = "MAIL_SMARTHOST_POOL_SIZE", default = 4)]
pub pool_size: u32,
/// Per-command SMTP timeout in seconds. Bounds the security handshake.
#[config(env = "MAIL_SMARTHOST_COMMAND_TIMEOUT_SECS", default = 30)]
pub command_timeout_secs: u64,
/// Total per-message timeout in seconds. Wraps the entire send so a
/// stuck relay cannot stall the comms queue.
#[config(env = "MAIL_SMARTHOST_TOTAL_TIMEOUT_SECS", default = 60)]
pub total_timeout_secs: u64,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DirectMxConfig {
/// Per-command SMTP timeout in seconds.
#[config(env = "MAIL_COMMAND_TIMEOUT_SECS", default = 30)]
pub command_timeout_secs: u64,
/// Total per-message timeout across all MX attempts in seconds.
#[config(env = "MAIL_TOTAL_TIMEOUT_SECS", default = 60)]
pub total_timeout_secs: u64,
/// Max number of concurrent direct-MX sends. Limits the load placed
/// on any single recipient MX during a backlog drain.
#[config(env = "MAIL_MAX_CONCURRENT_SENDS", default = 8)]
pub max_concurrent_sends: usize,
/// Require STARTTLS on every MX hop. When false, TLS is
/// attempted opportunistically and the session falls back to plaintext
/// if the remote does not advertise STARTTLS. Set true to refuse
/// plaintext delivery, at the cost of failing sends to MX hosts that
/// do not support TLS.
#[config(env = "MAIL_REQUIRE_TLS", default = false)]
pub require_tls: bool,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DkimConfig {
/// DKIM selector. When unset, outgoing mail is not signed.
#[config(env = "MAIL_DKIM_SELECTOR")]
pub selector: Option<String>,
/// DKIM signing domain.
#[config(env = "MAIL_DKIM_DOMAIN")]
pub domain: Option<String>,
/// Path to the DKIM private key in PEM format. Supports RSA and
/// Ed25519 keys.
#[config(env = "MAIL_DKIM_KEY_PATH")]
pub private_key_path: Option<String>,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct DiscordConfig {
/// Discord bot token. When unset, Discord integration is disabled.
#[config(env = "DISCORD_BOT_TOKEN")]
@@ -767,6 +1134,7 @@ pub struct DiscordConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TelegramConfig {
/// Telegram bot token. When unset, Telegram integration is disabled.
#[config(env = "TELEGRAM_BOT_TOKEN")]
@@ -778,14 +1146,7 @@ pub struct TelegramConfig {
}
#[derive(Debug, Config)]
pub struct SignalConfig {
/// Protocol state is stored in postgres' signal_* tables.
/// Link a device via the admin API before enabling.
#[config(env = "SIGNAL_ENABLED", default = false)]
pub enabled: bool,
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct NotificationConfig {
/// Polling interval in milliseconds for the comms queue.
#[config(env = "NOTIFICATION_POLL_INTERVAL_MS", default = 1000)]
@@ -808,6 +1169,7 @@ pub trait SsoProviderIssuerConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoConfig {
#[config(nested)]
pub github: SsoGitHubConfig,
@@ -829,6 +1191,7 @@ pub struct SsoConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoGitHubConfig {
#[config(env = "SSO_GITHUB_ENABLED", default = false)]
pub enabled: bool,
@@ -862,6 +1225,7 @@ impl SsoProviderConfig for SsoGitHubConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoDiscordConfig {
#[config(env = "SSO_DISCORD_ENABLED", default = false)]
pub enabled: bool,
@@ -895,6 +1259,7 @@ impl SsoProviderConfig for SsoDiscordConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoGoogleConfig {
#[config(env = "SSO_GOOGLE_ENABLED", default = false)]
pub enabled: bool,
@@ -928,6 +1293,7 @@ impl SsoProviderConfig for SsoGoogleConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoGitLabConfig {
#[config(env = "SSO_GITLAB_ENABLED", default = false)]
pub enabled: bool,
@@ -970,6 +1336,7 @@ impl SsoProviderIssuerConfig for SsoGitLabConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoOidcConfig {
#[config(env = "SSO_OIDC_ENABLED", default = false)]
pub enabled: bool,
@@ -1012,6 +1379,7 @@ impl SsoProviderIssuerConfig for SsoOidcConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct SsoAppleConfig {
#[config(env = "SSO_APPLE_ENABLED", default = false)]
pub enabled: bool,
@@ -1030,6 +1398,7 @@ pub struct SsoAppleConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ModerationConfig {
/// External report-handling service URL.
#[config(env = "REPORT_SERVICE_URL")]
@@ -1041,6 +1410,7 @@ pub struct ModerationConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ImportConfig {
/// Whether the PDS accepts repo imports.
#[config(env = "ACCEPTING_REPO_IMPORTS", default = true)]
@@ -1072,6 +1442,7 @@ fn split_comma_list(value: &str) -> Result<Vec<String>, std::convert::Infallible
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct RippleCacheConfig {
/// Address to bind the Ripple gossip protocol listener.
#[config(env = "RIPPLE_BIND", default = "0.0.0.0:0")]
@@ -1095,6 +1466,7 @@ pub struct RippleCacheConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct ScheduledConfig {
/// Interval in seconds between scheduled delete checks.
#[config(env = "SCHEDULED_DELETE_CHECK_INTERVAL_SECS", default = 3600)]
@@ -1142,6 +1514,7 @@ pub struct ScheduledConfig {
}
#[derive(Debug, Config)]
#[config(layer_attr(serde(deny_unknown_fields)))]
pub struct TranquilStoreConfig {
/// Directory for tranquil-store data: the metastore, eventlog, and blockstore.
#[config(
@@ -1196,3 +1569,333 @@ pub struct TranquilStoreConfig {
pub fn template() -> String {
confique::toml::template::<TranquilConfig>(confique::toml::FormatOptions::default())
}
#[cfg(test)]
mod tests {
use super::*;
fn seed_required_env() {
let required = [
("PDS_HOSTNAME", "test.local"),
("DATABASE_URL", "postgres://localhost/test"),
("TRANQUIL_PDS_ALLOW_INSECURE_SECRETS", "1"),
("INVITE_CODE_REQUIRED", "false"),
("ENABLE_PDS_HOSTED_DID_WEB", "true"),
("TRANQUIL_LEXICON_OFFLINE", "1"),
];
required
.iter()
.filter(|(k, _)| std::env::var_os(k).is_none())
.for_each(|(k, v)| unsafe { std::env::set_var(k, v) });
}
#[test]
fn load_rejects_unknown_top_level_key() {
let dir = std::env::temp_dir().join(format!(
"tranquil-config-unknown-toplevel-{}",
std::process::id()
));
std::fs::create_dir_all(&dir).expect("mkdir tempdir");
let path = dir.join("config.toml");
std::fs::write(
&path,
r#"
[server]
hostname = "test.local"
[totally_made_up]
foo = "bar"
"#,
)
.expect("write tempfile");
let result = TranquilConfig::builder().file(&path).load();
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_dir(&dir);
let err = format!("{:#}", result.expect_err("load must reject unknown key"));
assert!(
err.contains("totally_made_up"),
"expected totally_made_up in error, got {err:?}"
);
}
#[test]
fn load_rejects_unknown_nested_key() {
let dir = std::env::temp_dir().join(format!(
"tranquil-config-unknown-nested-{}",
std::process::id()
));
std::fs::create_dir_all(&dir).expect("mkdir tempdir");
let path = dir.join("config.toml");
std::fs::write(
&path,
r#"
[server]
hostname = "test.local"
not_a_real_field = "oops"
"#,
)
.expect("write tempfile");
let result = TranquilConfig::builder().file(&path).load();
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_dir(&dir);
let err = format!("{:#}", result.expect_err("load must reject unknown key"));
assert!(
err.contains("not_a_real_field"),
"expected not_a_real_field in error, got {err:?}"
);
}
#[test]
fn load_accepts_known_keys() {
let dir =
std::env::temp_dir().join(format!("tranquil-config-known-keys-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("mkdir tempdir");
let path = dir.join("config.toml");
std::fs::write(
&path,
r#"
[server]
hostname = "test.local"
port = 3000
[database]
url = "postgres://localhost/test"
[email.smarthost]
host = "smtp.example"
port = 587
"#,
)
.expect("write tempfile");
let result = TranquilConfig::builder().file(&path).load();
let _ = std::fs::remove_file(&path);
let _ = std::fs::remove_dir(&dir);
result.expect("known keys must load successfully");
}
#[test]
fn serial_validate_rejects_legacy_sendmail_path() {
seed_required_env();
unsafe { std::env::set_var("SENDMAIL_PATH", "/usr/sbin/sendmail") };
let config = TranquilConfig::builder()
.env()
.load()
.expect("load fresh config");
let result = config.validate(true);
unsafe { std::env::remove_var("SENDMAIL_PATH") };
let err = result.expect_err("validate must reject SENDMAIL_PATH");
let mentions_sendmail = err.errors.iter().any(|e| e.contains("SENDMAIL_PATH"));
assert!(
mentions_sendmail,
"errors did not mention SENDMAIL_PATH: {:?}",
err.errors
);
}
#[test]
fn serial_validate_passes_when_no_legacy_env_set() {
seed_required_env();
unsafe { std::env::remove_var("SENDMAIL_PATH") };
let config = TranquilConfig::builder()
.env()
.load()
.expect("load fresh config");
let result = config.validate(true);
let leaked_legacy = result
.as_ref()
.err()
.map(|e| e.errors.iter().any(|s| s.contains("SENDMAIL_PATH")))
.unwrap_or(false);
assert!(
!leaked_legacy,
"validate spuriously flagged SENDMAIL_PATH when unset: {:?}",
result
);
}
#[test]
fn email_address_predicate_accepts_typical_addresses() {
assert!(looks_like_email_address("alice@nel.pet"));
assert!(looks_like_email_address("a.b+tag@example.co.uk"));
}
#[test]
fn email_address_predicate_rejects_malformed() {
assert!(!looks_like_email_address(""));
assert!(!looks_like_email_address("no-at-sign"));
assert!(!looks_like_email_address("@nel.pet"));
assert!(!looks_like_email_address("alice@"));
assert!(!looks_like_email_address("alice@nel"));
assert!(!looks_like_email_address("a@b@c.com"));
assert!(!looks_like_email_address("alice @nel.pet"));
}
#[test]
fn dkim_selector_predicate_matches_subdomain_syntax() {
assert!(is_valid_dkim_selector("default"));
assert!(is_valid_dkim_selector("s2024-q1"));
assert!(is_valid_dkim_selector("mailo-2024.nel.pet"));
assert!(!is_valid_dkim_selector(""));
assert!(!is_valid_dkim_selector("a..b"));
assert!(!is_valid_dkim_selector("-leading"));
assert!(!is_valid_dkim_selector("trailing-"));
assert!(!is_valid_dkim_selector("s_under"));
}
#[test]
fn email_validate_disabled_when_from_address_unset() {
let cfg = email_config_for_test(EmailOverrides::default());
let mut errors = Vec::new();
cfg.validate("test.local", &mut errors);
assert!(errors.is_empty(), "expected no errors, got {errors:?}");
}
#[test]
fn email_validate_rejects_bad_from_address() {
let cfg = email_config_for_test(EmailOverrides {
from_address: Some("not-an-email"),
..Default::default()
});
let mut errors = Vec::new();
cfg.validate("test.local", &mut errors);
assert!(
errors.iter().any(|e| e.contains("from_address")),
"expected from_address error, got {errors:?}"
);
}
#[test]
fn email_validate_rejects_smarthost_with_bad_credentials() {
let cfg = email_config_for_test(EmailOverrides {
from_address: Some("alice@nel.pet"),
smarthost_host: Some("smtp.nel.pet"),
smarthost_username: Some(""),
smarthost_password: Some("hunter2"),
..Default::default()
});
let mut errors = Vec::new();
cfg.validate("test.local", &mut errors);
assert!(
errors.iter().any(|e| e.contains("smarthost.username")),
"expected smarthost.username error, got {errors:?}"
);
}
#[test]
fn email_validate_rejects_bad_dkim_selector() {
let cfg = email_config_for_test(EmailOverrides {
from_address: Some("alice@nel.pet"),
dkim_selector: Some("-bad"),
dkim_domain: Some("nel.pet"),
dkim_key_path: Some("/etc/dkim.key"),
..Default::default()
});
let mut errors = Vec::new();
cfg.validate("test.local", &mut errors);
assert!(
errors.iter().any(|e| e.contains("dkim.selector")),
"expected dkim.selector error, got {errors:?}"
);
}
#[test]
fn tls_validate_accepts_both_paths_unset() {
let mut errors = Vec::new();
TlsConfig {
cert_path: None,
key_path: None,
http3: false,
}
.validate(&mut errors);
assert!(errors.is_empty(), "expected no errors, got {errors:?}");
}
#[test]
fn tls_validate_accepts_both_paths_set() {
let mut errors = Vec::new();
TlsConfig {
cert_path: Some("/etc/tranquil/cert.pem".to_string()),
key_path: Some("/etc/tranquil/key.pem".to_string()),
http3: false,
}
.validate(&mut errors);
assert!(errors.is_empty(), "expected no errors, got {errors:?}");
}
#[test]
fn tls_validate_rejects_cert_without_key() {
let mut errors = Vec::new();
TlsConfig {
cert_path: Some("/etc/tranquil/cert.pem".to_string()),
key_path: None,
http3: false,
}
.validate(&mut errors);
assert!(
errors.iter().any(|e| e.contains("server.tls")),
"expected server.tls error, got {errors:?}"
);
}
#[test]
fn tls_validate_rejects_http3_without_material() {
let mut errors = Vec::new();
TlsConfig {
cert_path: None,
key_path: None,
http3: true,
}
.validate(&mut errors);
assert!(
errors.iter().any(|e| e.contains("http3")),
"expected http3 error, got {errors:?}"
);
}
#[derive(Default)]
struct EmailOverrides {
from_address: Option<&'static str>,
smarthost_host: Option<&'static str>,
smarthost_username: Option<&'static str>,
smarthost_password: Option<&'static str>,
dkim_selector: Option<&'static str>,
dkim_domain: Option<&'static str>,
dkim_key_path: Option<&'static str>,
}
fn email_config_for_test(o: EmailOverrides) -> EmailConfig {
EmailConfig {
from_address: o.from_address.map(str::to_string),
from_name: "Tranquil PDS".to_string(),
helo_name: None,
smarthost: SmarthostConfig {
host: o.smarthost_host.map(str::to_string),
port: 587,
username: o.smarthost_username.map(str::to_string),
password: o.smarthost_password.map(str::to_string),
tls: "starttls".to_string(),
pool_size: 4,
command_timeout_secs: 30,
total_timeout_secs: 60,
},
direct_mx: DirectMxConfig {
command_timeout_secs: 30,
total_timeout_secs: 60,
max_concurrent_sends: 8,
require_tls: false,
},
dkim: DkimConfig {
selector: o.dkim_selector.map(str::to_string),
domain: o.dkim_domain.map(str::to_string),
private_key_path: o.dkim_key_path.map(str::to_string),
},
}
}
}
+2
View File
@@ -244,6 +244,8 @@ pub trait InfraRepository: Send + Sync {
async fn mark_comms_failed(&self, id: Uuid, error: &str) -> Result<(), DbError>;
async fn mark_comms_failed_permanent(&self, id: Uuid, error: &str) -> Result<(), DbError>;
async fn create_invite_code(
&self,
code: &str,
+15 -18
View File
@@ -319,7 +319,6 @@ pub struct ApplyCommitInput {
#[derive(Debug, Clone)]
pub struct ApplyCommitResult {
pub seq: i64,
pub is_account_active: bool,
}
@@ -445,19 +444,15 @@ pub trait RepoRepository: Send + Sync {
async fn count_user_blocks(&self, user_id: Uuid) -> Result<i64, DbError>;
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<SequenceNumber, DbError>;
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<(), DbError>;
async fn insert_identity_event(
&self,
did: &Did,
handle: Option<&Handle>,
) -> Result<SequenceNumber, DbError>;
) -> Result<(), DbError>;
async fn insert_account_event(
&self,
did: &Did,
status: AccountStatus,
) -> Result<SequenceNumber, DbError>;
async fn insert_account_event(&self, did: &Did, status: AccountStatus) -> Result<(), DbError>;
async fn insert_sync_event(
&self,
@@ -465,7 +460,7 @@ pub trait RepoRepository: Send + Sync {
commit_cid: &CidLink,
rev: Option<&str>,
commit_bytes: &[u8],
) -> Result<SequenceNumber, DbError>;
) -> Result<(), DbError>;
async fn insert_genesis_commit_event(
&self,
@@ -475,14 +470,18 @@ pub trait RepoRepository: Send + Sync {
rev: &str,
commit_bytes: &[u8],
mst_root_bytes: &[u8],
) -> Result<SequenceNumber, DbError>;
async fn delete_sequences_except(
&self,
did: &Did,
keep_seq: SequenceNumber,
) -> Result<(), DbError>;
async fn purge_did_events_keeping_latest(&self, did: &Did) -> Result<(), DbError>;
async fn assign_pending_sequences(&self) -> Result<u64, DbError> {
Ok(0)
}
async fn flush_pending_sequences(&self) -> Result<(), DbError> {
Ok(())
}
async fn prune_events_older_than(&self, cutoff: DateTime<Utc>) -> Result<PruneCount, DbError>;
async fn get_max_seq(&self) -> Result<SequenceNumber, DbError>;
@@ -526,8 +525,6 @@ pub trait RepoRepository: Send + Sync {
async fn get_repo_root_cid_by_user_id(&self, user_id: Uuid)
-> Result<Option<CidLink>, DbError>;
async fn notify_update(&self, seq: SequenceNumber) -> Result<(), DbError>;
async fn import_repo_data(
&self,
user_id: Uuid,
@@ -563,5 +560,5 @@ pub trait RepoEventNotifier: Send + Sync {
#[async_trait]
pub trait RepoEventReceiver: Send {
async fn recv(&mut self) -> Option<i64>;
async fn recv(&mut self) -> Option<()>;
}
+3 -1
View File
@@ -44,6 +44,7 @@ pub struct UserRow {
pub deactivated_at: Option<DateTime<Utc>>,
pub takedown_ref: Option<String>,
pub is_admin: bool,
pub inbound_migration: bool,
}
#[derive(Debug, Clone)]
@@ -550,7 +551,7 @@ pub trait UserRepository: Send + Sync {
limit: i64,
) -> Result<Vec<ScheduledDeletionAccount>, DbError>;
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<i64, DbError>;
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<(), DbError>;
async fn create_password_account(
&self,
@@ -995,6 +996,7 @@ pub struct CreatePasswordAccountInput {
pub telegram_username: Option<String>,
pub signal_username: Option<String>,
pub deactivated_at: Option<DateTime<Utc>>,
pub inbound_migration: bool,
pub encrypted_key_bytes: Vec<u8>,
pub encryption_version: i32,
pub reserved_key_id: Option<Uuid>,
@@ -35,9 +35,9 @@ pub struct PostgresRepoEventReceiver {
#[async_trait]
impl RepoEventReceiver for PostgresRepoEventReceiver {
async fn recv(&mut self) -> Option<i64> {
async fn recv(&mut self) -> Option<()> {
match self.listener.recv().await {
Ok(notification) => notification.payload().parse().ok(),
Ok(_) => Some(()),
Err(_) => None,
}
}
+24 -2
View File
@@ -65,9 +65,13 @@ impl InfraRepository for PostgresInfraRepository {
SET status = 'processing', updated_at = NOW()
WHERE id IN (
SELECT id FROM comms_queue
WHERE status = 'pending'
WHERE attempts < max_attempts
AND scheduled_for <= $1
AND attempts < max_attempts
AND (
status = 'pending'
OR (status = 'processing'
AND updated_at < $1 - INTERVAL '10 minutes')
)
ORDER BY scheduled_for ASC
LIMIT $2
FOR UPDATE SKIP LOCKED
@@ -127,6 +131,24 @@ impl InfraRepository for PostgresInfraRepository {
Ok(())
}
async fn mark_comms_failed_permanent(&self, id: Uuid, error: &str) -> Result<(), DbError> {
sqlx::query!(
r#"UPDATE comms_queue
SET status = 'failed'::comms_status,
attempts = max_attempts,
last_error = $2,
updated_at = NOW()
WHERE id = $1"#,
id,
error
)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(())
}
async fn create_invite_code(
&self,
code: &str,
+1 -1
View File
@@ -374,7 +374,7 @@ impl OAuthRepository for PostgresOAuthRepository {
WHERE id IN (
SELECT id FROM oauth_token
WHERE did = $1
ORDER BY updated_at ASC
ORDER BY created_at DESC
OFFSET $2
)
"#,
+145 -88
View File
@@ -111,6 +111,64 @@ fn map_sequenced_row(r: SequencedEventRow) -> Result<SequencedEvent, DbError> {
})
}
const SEQUENCER_LOCK_KEY: i64 = 0x0074_7261_6e73_6571;
const SEQUENCER_BATCH_SIZE: i64 = 1000;
async fn notify_repo_pending(pool: &PgPool) {
if let Err(e) = sqlx::query!("NOTIFY repo_updates").execute(pool).await {
tracing::warn!(error = %e, "failed to notify repo_updates listeners");
}
}
async fn assign_one_batch(
mut tx: sqlx::Transaction<'_, sqlx::Postgres>,
pool: &PgPool,
) -> Result<i64, DbError> {
let pending_ids: Vec<i64> = sqlx::query_scalar!(
r#"SELECT id as "id!" FROM repo_seq WHERE seq IS NULL ORDER BY id LIMIT $1"#,
SEQUENCER_BATCH_SIZE
)
.fetch_all(&mut *tx)
.await
.map_err(map_sqlx_error)?;
let count = pending_ids.len() as i64;
if count == 0 {
tx.commit().await.map_err(map_sqlx_error)?;
return Ok(0);
}
let start = sqlx::query_scalar!(r#"SELECT nextval('firehose_seq') as "seq!""#)
.fetch_one(&mut *tx)
.await
.map_err(map_sqlx_error)?;
if count > 1 {
sqlx::query!("SELECT setval('firehose_seq', $1)", start + count - 1)
.fetch_one(&mut *tx)
.await
.map_err(map_sqlx_error)?;
}
let assigned = sqlx::query!(
r#"UPDATE repo_seq r
SET seq = $1::bigint + (o.ord - 1)
FROM unnest($2::bigint[]) WITH ORDINALITY AS o(id, ord)
WHERE r.id = o.id"#,
start,
&pending_ids
)
.execute(&mut *tx)
.await
.map_err(map_sqlx_error)?
.rows_affected();
tx.commit().await.map_err(map_sqlx_error)?;
if assigned > 0 {
notify_repo_pending(pool).await;
}
Ok(count)
}
pub struct PostgresRepoRepository {
pool: PgPool,
}
@@ -716,13 +774,12 @@ impl RepoRepository for PostgresRepoRepository {
Ok(rows.into_iter().map(|(cid,)| cid).collect())
}
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<SequenceNumber, DbError> {
async fn insert_commit_event(&self, data: &CommitEventData) -> Result<(), DbError> {
let (block_cids, block_data) = inline_to_paired_blocks(data.blocks.as_deref());
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
RETURNING seq
"#,
data.did.as_str(),
data.event_type.as_str(),
@@ -735,67 +792,54 @@ impl RepoRepository for PostgresRepoRepository {
data.prev_data_cid.as_ref().map(|c| c.as_str()),
data.rev
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_identity_event(
&self,
did: &Did,
handle: Option<&Handle>,
) -> Result<SequenceNumber, DbError> {
) -> Result<(), DbError> {
let handle_str = handle.map(|h| h.as_str());
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, handle)
VALUES ($1, 'identity', $2)
RETURNING seq
"#,
did.as_str(),
handle_str
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_account_event(
&self,
did: &Did,
status: AccountStatus,
) -> Result<SequenceNumber, DbError> {
async fn insert_account_event(&self, did: &Did, status: AccountStatus) -> Result<(), DbError> {
let active = status.is_active();
let status_str = status.for_firehose();
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, active, status)
VALUES ($1, 'account', $2, $3)
RETURNING seq
"#,
did.as_str(),
active,
status_str
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_sync_event(
@@ -804,18 +848,17 @@ impl RepoRepository for PostgresRepoRepository {
commit_cid: &CidLink,
rev: Option<&str>,
commit_bytes: &[u8],
) -> Result<SequenceNumber, DbError> {
) -> Result<(), DbError> {
let cid_bytes = commit_cid
.to_cid()
.map(|c| c.to_bytes())
.unwrap_or_default();
let block_cids: Vec<Vec<u8>> = vec![cid_bytes];
let block_data: Vec<Vec<u8>> = vec![commit_bytes.to_vec()];
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, rev, block_cids, block_data)
VALUES ($1, 'sync', $2, $3, $4, $5)
RETURNING seq
"#,
did.as_str(),
commit_cid.as_str(),
@@ -823,16 +866,12 @@ impl RepoRepository for PostgresRepoRepository {
&block_cids as &[Vec<u8>],
&block_data as &[Vec<u8>]
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn insert_genesis_commit_event(
@@ -843,7 +882,7 @@ impl RepoRepository for PostgresRepoRepository {
rev: &str,
commit_bytes: &[u8],
mst_root_bytes: &[u8],
) -> Result<SequenceNumber, DbError> {
) -> Result<(), DbError> {
let ops = serde_json::json!([]);
let blobs: Vec<String> = vec![];
let commit_cid_bytes = commit_cid
@@ -858,11 +897,10 @@ impl RepoRepository for PostgresRepoRepository {
let block_data: Vec<Vec<u8>> = vec![commit_bytes.to_vec(), mst_root_bytes.to_vec()];
let prev_cid: Option<&str> = None;
let seq = sqlx::query_scalar!(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, rev)
VALUES ($1, 'commit', $2, $3::TEXT, $4, $5, $6, $7, $8)
RETURNING seq
"#,
did.as_str(),
commit_cid.as_str(),
@@ -873,27 +911,22 @@ impl RepoRepository for PostgresRepoRepository {
&block_data as &[Vec<u8>],
rev
)
.fetch_one(&self.pool)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(seq.into())
notify_repo_pending(&self.pool).await;
Ok(())
}
async fn delete_sequences_except(
&self,
did: &Did,
keep_seq: SequenceNumber,
) -> Result<(), DbError> {
async fn purge_did_events_keeping_latest(&self, did: &Did) -> Result<(), DbError> {
sqlx::query!(
"DELETE FROM repo_seq WHERE did = $1 AND seq != $2",
did.as_str(),
keep_seq.as_i64()
r#"
DELETE FROM repo_seq
WHERE did = $1
AND id <> (SELECT id FROM repo_seq WHERE did = $1 ORDER BY id DESC LIMIT 1)
"#,
did.as_str()
)
.execute(&self.pool)
.await
@@ -902,6 +935,42 @@ impl RepoRepository for PostgresRepoRepository {
Ok(())
}
async fn assign_pending_sequences(&self) -> Result<u64, DbError> {
let mut total: u64 = 0;
loop {
let mut tx = self.pool.begin().await.map_err(map_sqlx_error)?;
let acquired = sqlx::query_scalar!(
r#"SELECT pg_try_advisory_xact_lock($1) as "locked!""#,
SEQUENCER_LOCK_KEY
)
.fetch_one(&mut *tx)
.await
.map_err(map_sqlx_error)?;
if !acquired {
return Ok(total);
}
let count = assign_one_batch(tx, &self.pool).await?;
total += count as u64;
if count < SEQUENCER_BATCH_SIZE {
return Ok(total);
}
}
}
async fn flush_pending_sequences(&self) -> Result<(), DbError> {
loop {
let mut tx = self.pool.begin().await.map_err(map_sqlx_error)?;
sqlx::query!("SELECT pg_advisory_xact_lock($1)", SEQUENCER_LOCK_KEY)
.execute(&mut *tx)
.await
.map_err(map_sqlx_error)?;
let count = assign_one_batch(tx, &self.pool).await?;
if count < SEQUENCER_BATCH_SIZE {
return Ok(());
}
}
}
async fn prune_events_older_than(&self, cutoff: DateTime<Utc>) -> Result<PruneCount, DbError> {
let result = sqlx::query!("DELETE FROM repo_seq WHERE created_at < $1", cutoff)
.execute(&self.pool)
@@ -965,7 +1034,7 @@ impl RepoRepository for PostgresRepoRepository {
Some(lim) => {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1
@@ -982,7 +1051,7 @@ impl RepoRepository for PostgresRepoRepository {
None => {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1
@@ -1004,7 +1073,7 @@ impl RepoRepository for PostgresRepoRepository {
) -> Result<Vec<SequencedEvent>, DbError> {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1 AND seq < $2
@@ -1024,7 +1093,7 @@ impl RepoRepository for PostgresRepoRepository {
) -> Result<Option<SequencedEvent>, DbError> {
let row = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq = $1"#,
@@ -1043,7 +1112,7 @@ impl RepoRepository for PostgresRepoRepository {
) -> Result<Vec<SequencedEvent>, DbError> {
let rows = sqlx::query_as!(
SequencedEventRow,
r#"SELECT seq, did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
r#"SELECT seq as "seq!", did, created_at, event_type as "event_type: RepoEventType", commit_cid, prev_cid, prev_data_cid,
ops, blobs, block_cids, block_data, blocks_cids, handle, active, status, rev
FROM repo_seq
WHERE seq > $1
@@ -1104,14 +1173,6 @@ impl RepoRepository for PostgresRepoRepository {
Ok(cid.map(CidLink::from))
}
async fn notify_update(&self, seq: SequenceNumber) -> Result<(), DbError> {
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq.as_i64()))
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(())
}
async fn import_repo_data(
&self,
user_id: Uuid,
@@ -1402,28 +1463,27 @@ impl RepoRepository for PostgresRepoRepository {
let event = input.commit_event;
let (event_block_cids, event_block_data) = inline_into_paired_blocks(event.blocks);
let seq: i64 = sqlx::query_scalar(
sqlx::query!(
r#"
INSERT INTO repo_seq (did, event_type, commit_cid, prev_cid, ops, blobs, block_cids, block_data, prev_data_cid, rev)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
RETURNING seq
"#,
event.did.as_str(),
event.event_type.as_str(),
event.commit_cid.as_ref().map(|c| c.as_str()),
event.prev_cid.as_ref().map(|c| c.as_str()),
event.ops,
event.blobs.as_deref(),
&event_block_cids as &[Vec<u8>],
&event_block_data as &[Vec<u8>],
event.prev_data_cid.as_ref().map(|c| c.as_str()),
event.rev
)
.bind(event.did.as_str())
.bind(event.event_type.as_str())
.bind(event.commit_cid.as_ref().map(|c| c.as_str()))
.bind(event.prev_cid.as_ref().map(|c| c.as_str()))
.bind(&event.ops)
.bind(&event.blobs)
.bind(&event_block_cids)
.bind(&event_block_data)
.bind(event.prev_data_cid.as_ref().map(|c| c.as_str()))
.bind(&event.rev)
.fetch_one(&mut *tx)
.execute(&mut *tx)
.await
.map_err(|e| ApplyCommitError::Database(e.to_string()))?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", seq))
sqlx::query!("NOTIFY repo_updates")
.execute(&mut *tx)
.await
.map_err(|e| ApplyCommitError::Database(e.to_string()))?;
@@ -1432,10 +1492,7 @@ impl RepoRepository for PostgresRepoRepository {
.await
.map_err(|e| ApplyCommitError::Database(e.to_string()))?;
Ok(tranquil_db_traits::ApplyCommitResult {
seq,
is_account_active,
})
Ok(tranquil_db_traits::ApplyCommitResult { is_account_active })
}
async fn get_users_without_blocks(&self) -> Result<Vec<UserWithoutBlocks>, DbError> {
+15 -12
View File
@@ -47,7 +47,7 @@ pub(crate) fn map_sqlx_error(e: sqlx::Error) -> DbError {
impl UserRepository for PostgresUserRepository {
async fn get_by_did(&self, did: &Did) -> Result<Option<UserRow>, DbError> {
let row = sqlx::query!(
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration
FROM users WHERE did = $1"#,
did.as_str()
)
@@ -64,12 +64,13 @@ impl UserRepository for PostgresUserRepository {
deactivated_at: r.deactivated_at,
takedown_ref: r.takedown_ref,
is_admin: r.is_admin,
inbound_migration: r.inbound_migration,
}))
}
async fn get_by_handle(&self, handle: &Handle) -> Result<Option<UserRow>, DbError> {
let row = sqlx::query!(
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin
r#"SELECT id, did, handle, email, created_at, deactivated_at, takedown_ref, is_admin, inbound_migration
FROM users WHERE handle = $1"#,
handle.as_str()
)
@@ -86,6 +87,7 @@ impl UserRepository for PostgresUserRepository {
deactivated_at: r.deactivated_at,
takedown_ref: r.takedown_ref,
is_admin: r.is_admin,
inbound_migration: r.inbound_migration,
}))
}
@@ -1863,7 +1865,7 @@ impl UserRepository for PostgresUserRepository {
async fn activate_account(&self, did: &Did) -> Result<bool, DbError> {
let result = sqlx::query!(
"UPDATE users SET deactivated_at = NULL WHERE did = $1",
"UPDATE users SET deactivated_at = NULL, inbound_migration = FALSE WHERE did = $1",
did.as_str()
)
.execute(&self.pool)
@@ -2289,7 +2291,7 @@ impl UserRepository for PostgresUserRepository {
.collect())
}
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<i64, DbError> {
async fn delete_account_with_firehose(&self, user_id: Uuid, did: &Did) -> Result<(), DbError> {
let mut tx = self.pool.begin().await.map_err(map_sqlx_error)?;
sqlx::query!("DELETE FROM blobs WHERE created_by_user = $1", user_id)
@@ -2368,11 +2370,11 @@ impl UserRepository for PostgresUserRepository {
.await
.map_err(map_sqlx_error)?;
let account_seq: i64 = sqlx::query_scalar!(
let event_id: i64 = sqlx::query_scalar!(
r#"
INSERT INTO repo_seq (did, event_type, active, status)
VALUES ($1, 'account', false, 'deleted')
RETURNING seq
RETURNING id
"#,
did.as_str()
)
@@ -2381,9 +2383,9 @@ impl UserRepository for PostgresUserRepository {
.map_err(map_sqlx_error)?;
sqlx::query!(
"DELETE FROM repo_seq WHERE did = $1 AND seq != $2",
"DELETE FROM repo_seq WHERE did = $1 AND id <> $2",
did.as_str(),
account_seq
event_id
)
.execute(&mut *tx)
.await
@@ -2391,12 +2393,12 @@ impl UserRepository for PostgresUserRepository {
tx.commit().await.map_err(map_sqlx_error)?;
sqlx::query(&format!("NOTIFY repo_updates, '{}'", account_seq))
sqlx::query!("NOTIFY repo_updates")
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(account_seq)
Ok(())
}
async fn create_password_account(
@@ -2426,8 +2428,8 @@ impl UserRepository for PostgresUserRepository {
handle, email, did, password_hash,
preferred_comms_channel,
discord_username, telegram_username, signal_username,
is_admin, deactivated_at, email_verified
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, FALSE) RETURNING id"#,
is_admin, deactivated_at, inbound_migration, email_verified
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, FALSE) RETURNING id"#,
)
.bind(input.handle.as_str())
.bind(&input.email)
@@ -2439,6 +2441,7 @@ impl UserRepository for PostgresUserRepository {
.bind(&input.signal_username)
.bind(is_first_user)
.bind(input.deactivated_at)
.bind(input.inbound_migration)
.fetch_one(&mut *tx)
.await;
@@ -310,6 +310,7 @@ pub async fn authorize_post(
State(state): State<AppState>,
_rate_limit: OAuthRateLimited<OAuthAuthorizeLimit>,
headers: HeaderMap,
client_ip: ClientIp,
Json(form): Json<AuthorizeSubmit>,
) -> Response {
let json_response = wants_json(&headers);
@@ -488,8 +489,7 @@ pub async fn authorize_post(
if !password_valid {
return show_login_error("Invalid identifier or password.", json_response);
}
let is_verified = user.channel_verification.has_any_verified();
if !is_verified {
if tranquil_api::server::verification_blocks_login(&user.channel_verification) {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &user.did).await;
let handle = resend_info
.as_ref()
@@ -617,7 +617,7 @@ pub async fn authorize_post(
let device_data = DeviceData {
session_id: SessionId::generate(),
user_agent: extract_user_agent(&headers),
ip_address: extract_client_ip(&headers, None),
ip_address: client_ip.into_string(),
last_seen_at: Utc::now(),
};
if state
@@ -854,8 +854,7 @@ pub async fn authorize_select(
);
}
};
let is_verified = user.channel_verification.has_any_verified();
if !is_verified {
if tranquil_api::server::verification_blocks_login(&user.channel_verification) {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &did).await;
return (
StatusCode::FORBIDDEN,
@@ -23,7 +23,7 @@ use tranquil_pds::rate_limit::{
};
use tranquil_pds::state::AppState;
use tranquil_pds::types::{Did, Handle, PlainPassword};
use tranquil_pds::util::extract_client_ip;
use tranquil_pds::util::ClientIp;
use tranquil_types::{AuthorizationCode, ClientId, DeviceId as DeviceIdType, RequestId};
use urlencoding::encode as url_encode;
@@ -289,9 +289,7 @@ async fn passkey_start_named(
.into_response();
}
let is_verified = user.channel_verification.has_any_verified();
if !is_verified {
if tranquil_api::server::verification_blocks_login(&user.channel_verification) {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &user.did).await;
return (
StatusCode::FORBIDDEN,
@@ -160,8 +160,10 @@ pub async fn register_complete(
.into_response();
}
let is_verified = match state.repos.user.get_session_info_by_did(&did).await {
Ok(Some(info)) => info.channel_verification.has_any_verified(),
let login_blocked = match state.repos.user.get_session_info_by_did(&did).await {
Ok(Some(info)) => {
tranquil_api::server::verification_blocks_login(&info.channel_verification)
}
Ok(None) => {
return (
StatusCode::FORBIDDEN,
@@ -189,7 +191,7 @@ pub async fn register_complete(
}
};
if !is_verified {
if login_blocked {
let resend_info = tranquil_api::server::auto_resend_verification(&state, &did).await;
return (
StatusCode::FORBIDDEN,
@@ -300,6 +302,7 @@ pub async fn register_complete(
pub async fn establish_session(
State(state): State<AppState>,
headers: HeaderMap,
client_ip: ClientIp,
auth: tranquil_pds::auth::Auth<tranquil_pds::auth::Active>,
) -> Response {
let did = &auth.did;
@@ -317,7 +320,7 @@ pub async fn establish_session(
let device_data = DeviceData {
session_id: SessionId::generate(),
user_agent: extract_user_agent(&headers),
ip_address: extract_client_ip(&headers, None),
ip_address: client_ip.into_string(),
last_seen_at: Utc::now(),
};
@@ -75,6 +75,7 @@ pub async fn authorize_2fa_post(
State(state): State<AppState>,
_rate_limit: OAuthRateLimited<OAuthAuthorizeLimit>,
headers: HeaderMap,
client_ip: ClientIp,
Json(form): Json<Authorize2faSubmit>,
) -> Response {
let json_error = |status: StatusCode, error: &str, description: &str| -> Response {
@@ -251,7 +252,7 @@ pub async fn authorize_2fa_post(
let device_data = DeviceData {
session_id: SessionId::generate(),
user_agent: extract_user_agent(&headers),
ip_address: extract_client_ip(&headers, None),
ip_address: client_ip.into_string(),
last_seen_at: Utc::now(),
};
if state
@@ -12,7 +12,7 @@ use tranquil_pds::oauth::client::{build_client_metadata, delegation_oauth_urls};
use tranquil_pds::rate_limit::{LoginLimit, OAuthRateLimited, TotpVerifyLimit};
use tranquil_pds::state::AppState;
use tranquil_pds::types::PlainPassword;
use tranquil_pds::util::extract_client_ip;
use tranquil_pds::util::ClientIp;
use tranquil_types::did_doc::{extract_handle, extract_pds_endpoint};
use tranquil_types::{Did, RequestId};
@@ -402,6 +402,7 @@ pub struct DelegationTokenAuthSubmit {
pub async fn delegation_auth_token(
State(state): State<AppState>,
headers: HeaderMap,
client_ip: ClientIp,
auth: Auth<Active>,
Json(form): Json<DelegationTokenAuthSubmit>,
) -> Response {
@@ -428,7 +429,7 @@ pub async fn delegation_auth_token(
return resp;
}
let ip = extract_client_ip(&headers, None);
let ip = client_ip.into_string();
let user_agent = tranquil_pds::util::extract_user_agent(&headers);
finalize_delegation_auth(
@@ -402,13 +402,15 @@ async fn handle_sso_login(
}
};
let is_verified = match state
let login_blocked = match state
.repos
.user
.get_session_info_by_did(&identity.did)
.await
{
Ok(Some(info)) => info.channel_verification.has_any_verified(),
Ok(Some(info)) => {
tranquil_api::server::verification_blocks_login(&info.channel_verification)
}
Ok(None) => {
tracing::error!("User not found for SSO login: {}", identity.did);
return redirect_to_error("Account not found");
@@ -419,7 +421,7 @@ async fn handle_sso_login(
}
};
if !is_verified {
if login_blocked {
tracing::warn!(
did = %identity.did,
provider = %provider.as_str(),
+3
View File
@@ -23,3 +23,6 @@ sha2 = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
uuid = { workspace = true }
[features]
native-tls-roots = []
+15 -12
View File
@@ -78,18 +78,21 @@ impl ClientMetadataCache {
Self {
cache: Arc::new(RwLock::new(HashMap::new())),
jwks_cache: Arc::new(RwLock::new(HashMap::new())),
http_client: Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.pool_max_idle_per_host(10)
.pool_idle_timeout(std::time::Duration::from_secs(90))
.user_agent(concat!(
"Tranquil-PDS/",
env!("CARGO_PKG_VERSION"),
" (ATProto; +https://tangled.org/tranquil.farm/tranquil-pds)"
))
.build()
.unwrap_or_else(|_| Client::new()),
http_client: {
let builder = Client::builder()
.timeout(std::time::Duration::from_secs(30))
.connect_timeout(std::time::Duration::from_secs(10))
.pool_max_idle_per_host(10)
.pool_idle_timeout(std::time::Duration::from_secs(90))
.user_agent(concat!(
"Tranquil-PDS/",
env!("CARGO_PKG_VERSION"),
" (ATProto; +https://tangled.org/tranquil.farm/tranquil-pds)"
));
#[cfg(feature = "native-tls-roots")]
let builder = builder.danger_accept_invalid_certs(true);
builder.build().unwrap_or_else(|_| Client::new())
},
cache_ttl_secs,
}
}
+3
View File
@@ -55,6 +55,7 @@ metrics-exporter-prometheus = { workspace = true }
multibase = { workspace = true }
multihash = { workspace = true }
p256 = { workspace = true }
parking_lot = { workspace = true }
rand = { workspace = true }
redis = { workspace = true, optional = true }
regex = { workspace = true }
@@ -90,8 +91,10 @@ s3-storage = ["tranquil-storage/s3", "dep:aws-config", "dep:aws-sdk-s3"]
s3 = ["s3-storage"]
valkey = ["tranquil-cache/valkey", "dep:redis"]
frontend = []
native-tls-roots = ["tranquil-oauth/native-tls-roots"]
[dev-dependencies]
tempfile = "3"
ciborium = { workspace = true }
ctor = { workspace = true }
testcontainers = { workspace = true }
+72 -6
View File
@@ -15,12 +15,15 @@ struct ErrorBody<'a> {
#[derive(Debug)]
pub enum ApiError {
InternalError(Option<String>),
RepoCorruption,
AuthenticationRequired,
AuthenticationFailed(Option<String>),
InvalidRequest(String),
InvalidToken(Option<String>),
ExpiredToken(Option<String>),
OAuthExpiredToken(Option<String>),
UseDpopNonce(String),
InvalidDpopProof(String),
TokenRequired,
AccountDeactivated,
AccountTakedown,
@@ -119,10 +122,34 @@ pub enum ApiError {
},
}
const MST_NODE_MISSING_MARKER: &str = "MST node not found";
impl ApiError {
pub fn is_repo_corruption(&self) -> bool {
matches!(self, Self::RepoCorruption)
}
pub fn detail_is_repo_corruption(detail: &str) -> bool {
detail.contains(tranquil_store::blockstore::BLOCK_CORRUPTION_MARKER)
|| detail.contains(MST_NODE_MISSING_MARKER)
}
pub fn from_mst_error(context: &str, e: &jacquard_repo::error::RepoError) -> Self {
let detail = format!("{e:#}");
if Self::detail_is_repo_corruption(&detail) {
tracing::warn!("{context}: repairable MST damage: {detail}");
Self::RepoCorruption
} else {
tracing::error!("{context}: {detail}");
Self::InternalError(None)
}
}
fn status_code(&self) -> StatusCode {
match self {
Self::InternalError(_) | Self::DatabaseError => StatusCode::INTERNAL_SERVER_ERROR,
Self::InternalError(_) | Self::RepoCorruption | Self::DatabaseError => {
StatusCode::INTERNAL_SERVER_ERROR
}
Self::UpstreamFailure | Self::UpstreamUnavailable(_) | Self::UpstreamErrorMsg(_) => {
StatusCode::BAD_GATEWAY
}
@@ -137,6 +164,8 @@ impl ApiError {
| Self::InvalidToken(_)
| Self::PasskeyCounterAnomaly
| Self::OAuthExpiredToken(_)
| Self::UseDpopNonce(_)
| Self::InvalidDpopProof(_)
| Self::ReauthRequired { .. } => StatusCode::UNAUTHORIZED,
Self::InvalidCode(_) => StatusCode::BAD_REQUEST,
Self::ExpiredToken(_) => StatusCode::BAD_REQUEST,
@@ -219,7 +248,9 @@ impl ApiError {
}
fn error_name(&self) -> Cow<'static, str> {
match self {
Self::InternalError(_) | Self::DatabaseError => Cow::Borrowed("InternalServerError"),
Self::InternalError(_) | Self::RepoCorruption | Self::DatabaseError => {
Cow::Borrowed("InternalServerError")
}
Self::UpstreamFailure | Self::UpstreamUnavailable(_) | Self::UpstreamErrorMsg(_) => {
Cow::Borrowed("UpstreamError")
}
@@ -236,6 +267,8 @@ impl ApiError {
Self::AuthenticationFailed(_) => Cow::Borrowed("AuthenticationFailed"),
Self::InvalidToken(_) => Cow::Borrowed("InvalidToken"),
Self::ExpiredToken(_) | Self::OAuthExpiredToken(_) => Cow::Borrowed("ExpiredToken"),
Self::UseDpopNonce(_) => Cow::Borrowed("use_dpop_nonce"),
Self::InvalidDpopProof(_) => Cow::Borrowed("invalid_dpop_proof"),
Self::TokenRequired => Cow::Borrowed("TokenRequired"),
Self::AccountDeactivated => Cow::Borrowed("AccountDeactivated"),
Self::AccountTakedown => Cow::Borrowed("AccountTakedown"),
@@ -326,6 +359,7 @@ impl ApiError {
Self::InternalError(msg) => msg
.clone()
.unwrap_or_else(|| "Internal Server Error".into()),
Self::RepoCorruption => "Internal Server Error".into(),
Self::AuthenticationFailed(msg) => msg
.clone()
.unwrap_or_else(|| "Authentication failed".into()),
@@ -335,6 +369,8 @@ impl ApiError {
Self::ExpiredToken(msg) | Self::OAuthExpiredToken(msg) => {
msg.clone().unwrap_or_else(|| "Token has expired".into())
}
Self::UseDpopNonce(_) => "DPoP nonce required".into(),
Self::InvalidDpopProof(msg) => msg.clone(),
Self::RepoNotFound(msg) => msg
.clone()
.unwrap_or_else(|| "Repository not found".into()),
@@ -560,6 +596,36 @@ impl IntoResponse for ApiError {
),
);
}
Self::UseDpopNonce(nonce) => {
match HeaderValue::from_str(nonce) {
Ok(val) => {
response
.headers_mut()
.insert(crate::util::HEADER_DPOP_NONCE, val);
}
Err(err) => {
tracing::error!(
?err,
nonce_len = nonce.len(),
"generated DPoP nonce is not a valid header value"
);
}
}
response.headers_mut().insert(
http::header::WWW_AUTHENTICATE,
HeaderValue::from_static(
"DPoP error=\"use_dpop_nonce\", error_description=\"Resource server requires nonce in DPoP proof\"",
),
);
}
Self::InvalidDpopProof(_) => {
response.headers_mut().insert(
http::header::WWW_AUTHENTICATE,
HeaderValue::from_static(
"DPoP error=\"invalid_dpop_proof\", error_description=\"Invalid DPoP proof\"",
),
);
}
_ => {}
}
response
@@ -596,6 +662,8 @@ impl From<crate::auth::TokenValidationError> for ApiError {
crate::auth::TokenValidationError::InvalidToken => {
Self::AuthenticationFailed(Some("Invalid token format".to_string()))
}
crate::auth::TokenValidationError::UseDpopNonce(nonce) => Self::UseDpopNonce(nonce),
crate::auth::TokenValidationError::InvalidDpopProof(msg) => Self::InvalidDpopProof(msg),
}
}
}
@@ -625,10 +693,8 @@ impl From<crate::auth::extractor::AuthError> for ApiError {
crate::auth::extractor::AuthError::OAuthExpiredToken(msg) => {
Self::OAuthExpiredToken(Some(msg))
}
crate::auth::extractor::AuthError::UseDpopNonce(_)
| crate::auth::extractor::AuthError::InvalidDpopProof(_) => {
Self::AuthenticationFailed(None)
}
crate::auth::extractor::AuthError::UseDpopNonce(nonce) => Self::UseDpopNonce(nonce),
crate::auth::extractor::AuthError::InvalidDpopProof(msg) => Self::InvalidDpopProof(msg),
}
}
}
+48 -2
View File
@@ -109,6 +109,35 @@ fn is_protected_method(method: &str) -> bool {
PROTECTED_METHODS.contains(method)
}
/// Fetch the `feed` generator record from the AppView and return its `did`.
async fn resolve_feed_generator_did(appview_url: &str, query: Option<&str>) -> Option<String> {
#[derive(serde::Deserialize)]
struct GetFeedQuery {
feed: String,
}
let feed = serde_urlencoded::from_str::<GetFeedQuery>(query?)
.ok()?
.feed;
let at_uri = crate::types::AtUri::new(feed).ok()?;
let repo = at_uri.did()?;
let collection = at_uri.collection()?;
let rkey = at_uri.rkey()?;
let resp = proxy_client()
.get(format!("{appview_url}/xrpc/com.atproto.repo.getRecord"))
.query(&[("repo", repo), ("collection", collection), ("rkey", rkey)])
.send()
.await
.ok()?;
if !resp.status().is_success() {
warn!(status = %resp.status(), "getFeed proxy: getRecord for feed generator failed");
return None;
}
let body: serde_json::Value = resp.json().await.ok()?;
body.get("value")?.get("did")?.as_str().map(str::to_string)
}
pub struct XrpcProxyLayer {
state: AppState,
}
@@ -287,10 +316,27 @@ async fn proxy_handler(
},
};
// getFeed must be audienced to the feed generator, not the AppView.
let (token_aud, token_lxm) = if method == "app.bsky.feed.getFeed" {
match resolve_feed_generator_did(&resolved.url, query.as_deref()).await {
Some(feed_did) => (feed_did, "app.bsky.feed.getFeedSkeleton"),
None => {
warn!(
"getFeed proxy: could not resolve feed generator DID; refusing \
to mint an AppView-audienced token"
);
return ApiError::InvalidRequest("Could not resolve feed".into())
.into_response();
}
}
} else {
(resolved.did.clone(), method)
};
match crate::auth::create_service_token(
&auth_user.did,
&resolved.did,
Some(method),
&token_aud,
Some(token_lxm),
&key_bytes,
) {
Ok(new_token) => {
@@ -21,6 +21,13 @@ pub async fn require_verified_or_delegated<'a>(
state: &AppState,
user: &'a AuthenticatedUser,
) -> Result<AccountVerified<'a>, ApiError> {
if tranquil_config::get()
.server
.disable_account_verification_gate
{
return Ok(AccountVerified { user });
}
let is_verified = state
.repos
.user
+31 -27
View File
@@ -1,11 +1,10 @@
use rand::Rng;
use serde::{Deserialize, Serialize};
use std::time::Duration;
use crate::cache::Cache;
use crate::util::{generate_token_code, normalize_token_code};
const TOKEN_TTL_SECS: u64 = 900;
const BASE32_CHARS: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum EmailTokenPurpose {
@@ -46,14 +45,6 @@ fn cache_key(did: &str, purpose: EmailTokenPurpose) -> String {
format!("email_token:{}:{}", purpose.as_str(), did)
}
fn generate_short_token() -> String {
let mut rng = rand::thread_rng();
let token: String = (0..10)
.map(|_| BASE32_CHARS[rng.gen_range(0..BASE32_CHARS.len())] as char)
.collect();
format!("{}-{}", &token[0..5], &token[5..10])
}
fn current_timestamp() -> u64 {
u64::try_from(chrono::Utc::now().timestamp()).unwrap_or(0)
}
@@ -67,9 +58,9 @@ pub async fn create_email_token(
return Err(TokenError::CacheUnavailable);
}
let token = generate_short_token();
let token = generate_token_code();
let data = TokenData {
token: token.clone(),
token: normalize_token_code(&token),
created_at: current_timestamp(),
};
@@ -108,10 +99,9 @@ pub async fn validate_email_token(
return Err(TokenError::ExpiredToken);
}
let normalized_input = token.to_uppercase().replace('-', "");
let normalized_stored = data.token.to_uppercase().replace('-', "");
let normalized_input = normalize_token_code(token);
if !constant_time_eq(normalized_input.as_bytes(), normalized_stored.as_bytes()) {
if !constant_time_eq(normalized_input.as_bytes(), data.token.as_bytes()) {
return Err(TokenError::InvalidToken);
}
@@ -260,20 +250,19 @@ mod tests {
#[tokio::test]
async fn test_token_format() {
(0..100).for_each(|_| {
let token = generate_short_token();
// The emitted token is the display form: uppercase `XXXXX-XXXXX`.
let cache = MockCache::new();
let did = "did:plc:test123";
(0..50).for_each(|_| {
let token = futures::executor::block_on(create_email_token(
&cache,
did,
EmailTokenPurpose::UpdateEmail,
))
.unwrap();
assert_eq!(token.len(), 11);
assert_eq!(&token[5..6], "-");
assert!(
token[0..5]
.chars()
.all(|c| BASE32_CHARS.contains(&(c as u8)))
);
assert!(
token[6..11]
.chars()
.all(|c| BASE32_CHARS.contains(&(c as u8)))
);
assert_eq!(token, token.to_uppercase());
});
}
@@ -292,6 +281,21 @@ mod tests {
assert!(result.is_ok());
}
#[tokio::test]
async fn test_hyphen_insensitive_validation() {
let cache = MockCache::new();
let did = "did:plc:test123";
let token = create_email_token(&cache, did, EmailTokenPurpose::UpdateEmail)
.await
.unwrap();
let no_hyphen = token.replace('-', "");
let result =
validate_email_token(&cache, did, EmailTokenPurpose::UpdateEmail, &no_hyphen).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_noop_cache_returns_unavailable() {
let cache = crate::cache::NoOpCache;
+2 -27
View File
@@ -2,7 +2,7 @@ use std::marker::PhantomData;
use axum::{
extract::{FromRequestParts, OptionalFromRequestParts, OriginalUri},
http::{StatusCode, header::AUTHORIZATION, request::Parts},
http::{header::AUTHORIZATION, request::Parts},
response::{IntoResponse, Response},
};
use tracing::{debug, error, info};
@@ -35,32 +35,7 @@ pub enum AuthError {
impl IntoResponse for AuthError {
fn into_response(self) -> Response {
match self {
Self::UseDpopNonce(nonce) => (
StatusCode::UNAUTHORIZED,
[
("DPoP-Nonce", nonce.as_str()),
("WWW-Authenticate", "DPoP error=\"use_dpop_nonce\""),
],
axum::Json(serde_json::json!({
"error": "use_dpop_nonce",
"message": "DPoP nonce required"
})),
)
.into_response(),
Self::OAuthExpiredToken(msg) => ApiError::OAuthExpiredToken(Some(msg)).into_response(),
Self::InvalidDpopProof(msg) => (
StatusCode::UNAUTHORIZED,
[("WWW-Authenticate", "DPoP error=\"invalid_dpop_proof\"")],
axum::Json(serde_json::json!({
"error": "invalid_dpop_proof",
"message": msg
})),
)
.into_response(),
Self::InsufficientScope(msg) => ApiError::InsufficientScope(Some(msg)).into_response(),
other => ApiError::from(other).into_response(),
}
ApiError::from(self).into_response()
}
}
+227 -31
View File
@@ -1,15 +1,14 @@
use chrono::Utc;
use rand::Rng;
use serde::{Deserialize, Serialize};
use std::time::Duration;
use crate::cache::Cache;
use crate::types::Did;
use crate::util::{generate_token_code, normalize_token_code};
const CHALLENGE_TTL_SECS: u64 = 300;
const MIN_REMAINING_TTL_SECS: u64 = 10;
const MAX_ATTEMPTS: u8 = 5;
const CODE_LENGTH: usize = 8;
const COOLDOWN_SECS: u64 = 60;
#[derive(Debug, Clone, Serialize, Deserialize)]
@@ -94,7 +93,8 @@ async fn validate_challenge_internal(
return Err(ValidationError::ChallengeExpired);
}
if !constant_time_eq(code.as_bytes(), data.code.as_bytes()) {
let normalized_input = normalize_token_code(code);
if !constant_time_eq(normalized_input.as_bytes(), data.code.as_bytes()) {
let updated = ChallengeData {
code: data.code,
attempts: data.attempts + 1,
@@ -127,17 +127,19 @@ fn cooldown_key(did: &str) -> String {
format!("legacy_2fa_cooldown:{}", did)
}
fn generate_code() -> String {
let mut rng = rand::thread_rng();
(0..CODE_LENGTH)
.map(|_| rng.gen_range(0..10).to_string())
.collect()
}
fn current_timestamp() -> u64 {
u64::try_from(Utc::now().timestamp()).unwrap_or(0)
}
pub fn looks_like_totp_token(code: &str) -> bool {
let c = code.trim();
(c.len() == 6 && c.bytes().all(|b| b.is_ascii_digit())) || crate::auth::is_backup_code_format(c)
}
pub fn used_totp_factor(has_totp: bool, auth_factor_token: Option<&str>) -> bool {
has_totp && auth_factor_token.is_some_and(looks_like_totp_token)
}
fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
if a.len() != b.len() {
return false;
@@ -148,6 +150,7 @@ fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
== 0
}
#[derive(Debug)]
pub enum Legacy2faOutcome {
NotRequired,
Blocked,
@@ -177,6 +180,7 @@ pub async fn process_legacy_2fa(
did: &Did,
ctx: &Legacy2faContext,
auth_factor_token: Option<&str>,
verify_totp: impl AsyncFnOnce(&str) -> bool,
) -> Result<Legacy2faOutcome, Legacy2faFlowError> {
if !ctx.requires_2fa() {
return Ok(Legacy2faOutcome::NotRequired);
@@ -192,8 +196,16 @@ pub async fn process_legacy_2fa(
Ok(Legacy2faOutcome::ChallengeSent(code))
}
Some(token) => {
validate_challenge(cache, did, token).await?;
Ok(Legacy2faOutcome::Verified)
if ctx.has_totp && looks_like_totp_token(token) {
if verify_totp(token).await {
Ok(Legacy2faOutcome::Verified)
} else {
Err(Legacy2faFlowError::Validation(ValidationError::InvalidCode))
}
} else {
validate_challenge(cache, did, token).await?;
Ok(Legacy2faOutcome::Verified)
}
}
}
}
@@ -219,11 +231,11 @@ async fn create_challenge_code(
return Err(ChallengeError::RateLimited);
}
let code = generate_code();
let display = generate_token_code();
let now = current_timestamp();
let data = ChallengeData {
code: code.clone(),
code: normalize_token_code(&display),
attempts: 0,
created_at: now,
};
@@ -244,7 +256,7 @@ async fn create_challenge_code(
.await
.map_err(|_| ChallengeError::CacheError)?;
Ok(ChallengeCode(code))
Ok(ChallengeCode(display))
}
#[derive(Debug)]
@@ -332,12 +344,46 @@ mod tests {
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
assert_eq!(code.as_str().len(), CODE_LENGTH);
assert_eq!(code.as_str().len(), 11);
let result = validate_challenge(&cache, &did, code.as_str()).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_challenge_code_format() {
let cache = MockCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
let code = code.as_str();
assert_eq!(code.len(), 11);
assert_eq!(&code[5..6], "-");
assert_eq!(code, code.to_uppercase());
}
#[tokio::test]
async fn test_case_insensitive_validation() {
let cache = MockCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
let lowercase = code.as_str().to_lowercase();
let result = validate_challenge(&cache, &did, &lowercase).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_hyphen_insensitive_validation() {
let cache = MockCache::new();
let did = Did::new("did:plc:test123".to_string()).unwrap();
let code = create_challenge(&cache, &did).await.unwrap();
let no_hyphen = code.as_str().replace('-', "");
let result = validate_challenge(&cache, &did, &no_hyphen).await;
assert!(result.is_ok());
}
#[tokio::test]
async fn test_invalid_code_rejected() {
let cache = MockCache::new();
@@ -396,15 +442,6 @@ mod tests {
assert_eq!(result.unwrap_err(), ChallengeError::CacheUnavailable);
}
#[tokio::test]
async fn test_code_generation_is_numeric() {
(0..100).for_each(|_| {
let code = generate_code();
assert!(code.chars().all(|c| c.is_ascii_digit()));
assert_eq!(code.len(), CODE_LENGTH);
});
}
#[tokio::test]
async fn test_constant_time_eq() {
assert!(constant_time_eq(b"12345678", b"12345678"));
@@ -425,7 +462,9 @@ mod tests {
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::NotRequired));
}
@@ -440,7 +479,9 @@ mod tests {
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::NotRequired));
}
@@ -455,7 +496,9 @@ mod tests {
allow_legacy_login: false,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Blocked));
}
@@ -470,7 +513,9 @@ mod tests {
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::ChallengeSent(_)));
}
@@ -485,7 +530,9 @@ mod tests {
allow_legacy_login: false,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, None).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, None, reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::ChallengeSent(_)));
}
@@ -502,7 +549,7 @@ mod tests {
let code = create_challenge(&cache, &did).await.unwrap();
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some(code.as_str()))
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some(code.as_str()), reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
@@ -532,4 +579,153 @@ mod tests {
let result = validate_challenge(&cache, &did, "12345678").await;
assert_eq!(result.unwrap_err(), ValidationError::CacheUnavailable);
}
async fn reject_totp(_code: &str) -> bool {
false
}
async fn accept_totp(_code: &str) -> bool {
true
}
#[tokio::test]
async fn test_totp_shaped_token_accepted_via_verifier() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp1".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: false,
has_totp: true,
allow_legacy_login: true,
};
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some("123456"), accept_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_totp_shaped_token_rejected_does_not_touch_email_challenge() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp2".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: true,
has_totp: true,
allow_legacy_login: true,
};
// An email challenge exists for this user.
let email_code = create_challenge(&cache, &did).await.unwrap();
// Five wrong TOTP-shaped attempts. If these incremented the email attempt
// counter, the email challenge would be exhausted (MAX_ATTEMPTS = 5).
for _ in 0..5 {
let err = process_legacy_2fa(&cache, &did, &ctx, Some("000000"), reject_totp)
.await
.unwrap_err();
assert!(matches!(
err,
Legacy2faFlowError::Validation(ValidationError::InvalidCode)
));
}
// The email challenge is still valid and consumable.
let outcome =
process_legacy_2fa(&cache, &did, &ctx, Some(email_code.as_str()), reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_email_shaped_token_routes_to_email_path_when_totp_present() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp3".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: true,
has_totp: true,
allow_legacy_login: true,
};
let email_code = create_challenge(&cache, &did).await.unwrap();
// reject_totp would fail if this routed to the verifier; it must route to email.
let outcome =
process_legacy_2fa(&cache, &did, &ctx, Some(email_code.as_str()), reject_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_backup_code_shaped_token_routes_to_verifier() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp4".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: false,
has_totp: true,
allow_legacy_login: true,
};
// No email challenge created. If this routed to email it would be
// ChallengeNotFound; Verified proves it went to the verifier.
let outcome = process_legacy_2fa(&cache, &did, &ctx, Some("ABCD2345"), accept_totp)
.await
.unwrap();
assert!(matches!(outcome, Legacy2faOutcome::Verified));
}
#[tokio::test]
async fn test_totp_shaped_token_ignored_when_no_totp() {
let cache = MockCache::new();
let did = Did::new("did:plc:totp5".to_string()).unwrap();
let ctx = Legacy2faContext {
is_app_password: false,
email_2fa_enabled: true,
has_totp: false,
allow_legacy_login: false,
};
// has_totp = false -> 6-digit token routes to email path; no challenge -> NotFound.
let err = process_legacy_2fa(&cache, &did, &ctx, Some("123456"), reject_totp)
.await
.unwrap_err();
assert!(matches!(
err,
Legacy2faFlowError::Validation(ValidationError::ChallengeNotFound)
));
}
#[tokio::test]
async fn test_looks_like_totp_token() {
// 6-digit TOTP codes
assert!(looks_like_totp_token("123456"));
assert!(looks_like_totp_token(" 000000 "));
// backup-code format (8 chars, backup alphabet)
assert!(looks_like_totp_token("ABCD2345"));
// email challenge codes normalize to 10 alphanumeric chars -> not TOTP-shaped
assert!(!looks_like_totp_token("ABCDEFGHIJ"));
assert!(!looks_like_totp_token("ABCDE-FGHIJ"));
// wrong lengths / non-digits
assert!(!looks_like_totp_token("12345"));
assert!(!looks_like_totp_token("1234567"));
assert!(!looks_like_totp_token("12345A"));
assert!(!looks_like_totp_token(""));
}
#[test]
fn test_used_totp_factor() {
// strong MFA factors completed the login -> true
assert!(used_totp_factor(true, Some("123456")));
assert!(used_totp_factor(true, Some("ABCD2345")));
// email-shaped code, or no token, or no TOTP on the account -> false
assert!(!used_totp_factor(true, Some("ABCDEFGHIJ")));
assert!(!used_totp_factor(true, None));
assert!(!used_totp_factor(false, Some("123456")));
assert!(!used_totp_factor(true, Some("")));
}
}
+12 -2
View File
@@ -106,7 +106,7 @@ struct CachedUserStatus {
is_admin: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum TokenValidationError {
AccountDeactivated,
AccountTakedown,
@@ -115,6 +115,8 @@ pub enum TokenValidationError {
TokenExpired,
OAuthTokenExpired,
InvalidToken,
UseDpopNonce(String),
InvalidDpopProof(String),
}
impl fmt::Display for TokenValidationError {
@@ -126,6 +128,8 @@ impl fmt::Display for TokenValidationError {
Self::AuthenticationFailed => write!(f, "AuthenticationFailed"),
Self::TokenExpired | Self::OAuthTokenExpired => write!(f, "ExpiredToken"),
Self::InvalidToken => write!(f, "InvalidToken"),
Self::UseDpopNonce(_) => write!(f, "use_dpop_nonce"),
Self::InvalidDpopProof(_) => write!(f, "invalid_dpop_proof"),
}
}
}
@@ -518,7 +522,7 @@ async fn validate_bearer_token_with_options_internal(
auth_source: AuthSource::OAuth,
});
} else {
return Err(TokenValidationError::TokenExpired);
return Err(TokenValidationError::OAuthTokenExpired);
}
}
@@ -613,6 +617,12 @@ pub async fn validate_token_with_dpop(
Err(crate::oauth::OAuthError::ExpiredToken(_)) => {
Err(TokenValidationError::OAuthTokenExpired)
}
Err(crate::oauth::OAuthError::UseDpopNonce(nonce)) => {
Err(TokenValidationError::UseDpopNonce(nonce))
}
Err(crate::oauth::OAuthError::InvalidDpopProof(msg)) => {
Err(TokenValidationError::InvalidDpopProof(msg))
}
Err(_) => Err(TokenValidationError::AuthenticationFailed),
}
}
+1 -1
View File
@@ -4,7 +4,7 @@ pub use tranquil_comms::{
CommsChannel, CommsSender, CommsStatus, CommsType, DEFAULT_LOCALE, DiscordSender, EmailSender,
NewComms, NotificationStrings, QueuedComms, SendError, SignalSender, TelegramSender,
VALID_LOCALES, format_message, get_strings, is_valid_phone_number, is_valid_signal_username,
mime_encode_header, sanitize_header_value, validate_locale,
validate_locale,
};
pub use service::{CommsService, repo as comms_repo, resolve_delivery_channel};
+15 -1
View File
@@ -149,13 +149,19 @@ impl CommsService {
}
}
Err(e) => {
let permanent = e.is_permanent();
let error_msg = e.to_string();
warn!(
comms_id = %comms_id,
error = %error_msg,
permanent,
"Failed to send comms"
);
if let Err(db_err) = self.mark_failed(comms_id, &error_msg).await {
let db_result = match permanent {
true => self.mark_failed_permanent(comms_id, &error_msg).await,
false => self.mark_failed(comms_id, &error_msg).await,
};
if let Err(db_err) = db_result {
error!(
comms_id = %comms_id,
error = %db_err,
@@ -173,6 +179,14 @@ impl CommsService {
async fn mark_failed(&self, id: Uuid, error: &str) -> Result<(), tranquil_db_traits::DbError> {
self.infra_repo.mark_comms_failed(id, error).await
}
async fn mark_failed_permanent(
&self,
id: Uuid,
error: &str,
) -> Result<(), tranquil_db_traits::DbError> {
self.infra_repo.mark_comms_failed_permanent(id, error).await
}
}
struct ResolvedRecipient {
+3 -3
View File
@@ -51,6 +51,8 @@ pub const BUILD_VERSION: &str = concat!(
#[cfg(not(debug_assertions))]
pub const BUILD_VERSION: &str = env!("CARGO_PKG_VERSION");
pub const GENERAL_BODY_LIMIT: usize = 16 * 1024 * 1024;
pub struct ExternalRoutes {
pub xrpc: Router<AppState>,
pub oauth: Router<AppState>,
@@ -97,9 +99,7 @@ pub fn app_with_routes(state: AppState, external: ExternalRoutes) -> Router {
.nest("/.well-known", well_known_router)
.route("/metrics", get(metrics::metrics_handler))
.merge(external.extra)
.layer(DefaultBodyLimit::max(
tranquil_config::get().server.max_blob_size as usize,
))
.layer(DefaultBodyLimit::max(GENERAL_BODY_LIMIT))
.layer(axum::middleware::map_response(rewrite_extractor_errors))
.layer(middleware::from_fn(metrics::metrics_middleware))
.layer(

Some files were not shown because too many files have changed in this diff Show More