Compare commits

..
15 Commits
Author SHA1 Message Date
Johanna LarssonandTangled 4eb59e4f6c Creating a mega test binary
Overall `just test` runs about 10-20s faster on this branch than current main, for me, with around 170s instead of around 180-190s. But the cool thing is the compile time. Making a simple edit to any file involved and running `just test` on main takes about 60s to recompile the test binaries. On this branch it takes less than 15s.

This is done by replacing a ton of separate test binaries with a single megabinary.

Only test files that looked safe have been included, avoiding ones that do stuff like setting env.
2026-09-24 13:09:23 +00:00
Johanna LarssonandTangled 0c5eda6117 Drop rustls 0.21 copy
It's not being used anyway, so just wasted CPU cycles.
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 7978246c66 Remove redundant cargo test build and set SQLX_OFFLINE=true
The pg tests were missing SQLX_OFFLINE
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled ffc8ef4e01 Disable fsync for postgres too
Summary [ 166.110s] 1453 tests run: 1453 passed (1 slow), 15 skipped

Above is before this commit, below is after.

     Summary [ 140.410s] 1453 tests run: 1453 passed, 15 skipped

The mechanism here is the same as for the metastore tests. By skipping fsync we can run the tests much faster, and fsync doesn't matter in tests. Unless you're testing postgres itself I guess
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 7c0ca21de5 Remove sleeps from tests
Replace sleeps with the existing polling function instead. Exposes that the stress test isn't much of a stress test, it probably needs more data or something, but if it's heavy it should maybe not be part of `just test`
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 5214dd1c23 Remove forced serialization where not needed
Tests already run in isolation, so this is only relevant when messing with env vars and stuff, I think!
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 8a72d2c0e7 Stop talking to docker in tests that don't need docker
`just test` doesn't use docker, but the cleanup was still pruning containers over and over. This switches to only cleaning up containers created by the tests.
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 939b5a8cc4 Stop pg tests from duplicating tranquil-store tests 2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled e483fa139f Remove tests that require postgres from just test
These suites got added but actually need the postgres setup, and so can't succeed and are just annoying everyone.
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled e87952616d Avoid compiling and running tests twice
test-unit was building a second copy of the code with different features, and then the main test command ran the same tests anyway, so we just built and ran the tests twice

Locally shaves a minute off of a full build+test, on this branch (so both runs include the fsync change etc)

We're at [ 173.444s] overall for `just test` now
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 43dfd2f2f3 Stop compiling tranqil-store for just-test
Takes off ~140s CPU time for me, but no wall time improvement. Still, less work!
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 781be29d7b Speed up bcrypt and blowfish in tests
Build optimized versions of bcrypt and blowfish to speed them up, referencing things like https://github.com/oxidecomputer/omicron/blob/main/Cargo.toml#L981-L982

Password hashing is not a dominating cost, so this doesn't have a significant impact on the overall time, but a lot of tests like `**tranquil-pds::legacy_2fa** **test_legacy_2fa_code_consumed_after_use**` speed up. It goes from 5.6s to 2.7s on my machine.
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled 06b1255733 Skip fsync when running tests
Flushing to disk all the time ensures no partial states when writing to disk during awkwardnesses like crashes in prod risking real user data. That's good!

We're also doing it while running tests. Less good! Well, at least not important!

Comparing a clean test run (`just test --no-fail-fast`) and a run on this branch, on my MacOS M4 Pro, I get:

`**Summary** [ 517.723s] **1424** tests run: **1405** **passed** (**7** **slow**), **19** **failed**, **15** **skipped**`

before and

`**Summary** [ 326.369s] **1424** tests run: **1405** **passed**, **19** **failed**, **15** **skipped**`

after.

The reason I run `--no-fail-fast` is that a bunch of tests always fail and time out and stuff
2026-09-23 17:24:44 +00:00
Johanna LarssonandTangled eebce28926 fix: handle wildcard + aud in OAuth scopes
`rpc:*?aud=did:web:api.bsky.app#bsky_appview` fails in the permissions check even though it's a valid scope. This makes it resolve correctly.

Also this commit fixes a bug where `rpc` as a bare scope is interpreted as all methods for all services, but it's really an invalid scope. Can drop it if it's a problem, but it just looks like a bug in the implementation rather than something intended?

Note that the parser treats `rpc:*` as "allow everything for no services", but aud is actually required, so it's not a valid scope? Didn't mess with it though because delegation uses it and that part of the code treats it as allow everything for every service. We should maybe split the scope parsers so we have one for delegated scopes and one for regular?

However, to avoid the same bug as bare `rpc`, I changed the enforcement step in "regular" OAuth to disallow `rpc:*` (this matches the existing behavior on main, no actual change, just a little acknowledgement).
2026-09-23 15:42:29 +00:00
LewisandTangled b260ee10c7 tranquil-store: make it seem a little less 'experimental'
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-09-23 14:54:51 +00:00
92 changed files with 560 additions and 608 deletions
+9 -33
View File
@@ -53,27 +53,15 @@ heavy-load-tests = { max-threads = 4 }
io-heavy-sim = { max-threads = 2 }
[[profile.default.overrides]]
filter = "test(/import_with_verification/) | test(/plc_migration/)"
filter = "binary(it) and test(/^ripple_cluster::/)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "binary(handle_domains)"
filter = "package(tranquil-signal) and test(/^tests::/)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "binary(ripple_cluster)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "package(tranquil-signal)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "package(tranquil-config)"
test-group = "serial-env-tests"
[[profile.default.overrides]]
filter = "binary(whole_story)"
filter = "binary(it) and test(/^whole_story::/)"
test-group = "heavy-load-tests"
[[profile.default.overrides]]
@@ -81,7 +69,7 @@ filter = "test(/two_node_stress_concurrent_load/)"
test-group = "heavy-load-tests"
[[profile.default.overrides]]
filter = "binary(repo_lifecycle)"
filter = "binary(it) and test(/^repo_lifecycle::/)"
test-group = "heavy-load-tests"
[[profile.default.overrides]]
@@ -103,7 +91,7 @@ filter = "binary(gauntlet_smoke)"
slow-timeout = { period = "300s", terminate-after = 8 }
[[profile.default.overrides]]
filter = "binary(compaction_restart) | binary(mst_refcount_integrity) | binary(gc_compaction_restart)"
filter = "binary(compaction_restart) | binary(mst_refcount_integrity) | binary(it) and test(/^gc_compaction_restart::/)"
slow-timeout = { period = "120s", terminate-after = 4 }
[[profile.default.overrides]]
@@ -117,27 +105,15 @@ slow-timeout = { period = "300s", terminate-after = 4 }
test-group = "io-heavy-sim"
[[profile.ci.overrides]]
filter = "test(/import_with_verification/) | test(/plc_migration/)"
filter = "binary(it) and test(/^ripple_cluster::/)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "binary(handle_domains)"
filter = "package(tranquil-signal) and test(/^tests::/)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "binary(ripple_cluster)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "package(tranquil-signal)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "package(tranquil-config)"
test-group = "serial-env-tests"
[[profile.ci.overrides]]
filter = "binary(whole_story)"
filter = "binary(it) and test(/^whole_story::/)"
test-group = "heavy-load-tests"
[[profile.ci.overrides]]
@@ -145,7 +121,7 @@ filter = "test(/two_node_stress_concurrent_load/)"
test-group = "heavy-load-tests"
[[profile.ci.overrides]]
filter = "binary(repo_lifecycle)"
filter = "binary(it) and test(/^repo_lifecycle::/)"
test-group = "heavy-load-tests"
[[profile.ci.overrides]]
Generated
+68 -147
View File
@@ -741,23 +741,17 @@ dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
"aws-smithy-types",
"h2 0.3.27",
"h2 0.4.13",
"http 0.2.12",
"h2",
"http 1.4.0",
"http-body 0.4.6",
"hyper 0.14.32",
"hyper 1.8.1",
"hyper-rustls 0.24.2",
"hyper-rustls 0.27.7",
"hyper",
"hyper-rustls",
"hyper-util",
"pin-project-lite",
"rustls 0.21.12",
"rustls 0.23.37",
"rustls",
"rustls-native-certs",
"rustls-pki-types",
"tokio",
"tokio-rustls 0.26.4",
"tokio-rustls",
"tower",
"tracing",
]
@@ -905,7 +899,7 @@ dependencies = [
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
"hyper",
"hyper-util",
"itoa",
"matchit",
@@ -1172,16 +1166,16 @@ dependencies = [
"home",
"http 1.4.0",
"http-body-util",
"hyper 1.8.1",
"hyper",
"hyper-named-pipe",
"hyper-rustls 0.27.7",
"hyper-rustls",
"hyper-util",
"hyperlocal",
"log",
"num",
"pin-project-lite",
"rand 0.9.2",
"rustls 0.23.37",
"rustls",
"rustls-native-certs",
"rustls-pemfile",
"rustls-pki-types",
@@ -1646,6 +1640,21 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "conquer-once"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d008a441c0f269f36ca13712528069a86a3e60dffee1d98b976eb3b0b2160b4"
dependencies = [
"conquer-util",
]
[[package]]
name = "conquer-util"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e763eef8846b13b380f37dfecda401770b0ca4e56e95170237bd7c25c7db3582"
[[package]]
name = "const-oid"
version = "0.9.6"
@@ -2882,25 +2891,6 @@ dependencies = [
"subtle",
]
[[package]]
name = "h2"
version = "0.3.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0beca50380b1fc32983fc1cb4587bfa4bb9e78fc259aad4a0032d2080309222d"
dependencies = [
"bytes",
"fnv",
"futures-core",
"futures-sink",
"futures-util",
"http 0.2.12",
"indexmap 2.13.0",
"slab",
"tokio",
"tokio-util",
"tracing",
]
[[package]]
name = "h2"
version = "0.4.13"
@@ -3259,30 +3249,6 @@ version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "hyper"
version = "0.14.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41dfc780fdec9373c01bae43289ea34c972e40ee3c9f6b3c8801a35f35586ce7"
dependencies = [
"bytes",
"futures-channel",
"futures-core",
"futures-util",
"h2 0.3.27",
"http 0.2.12",
"http-body 0.4.6",
"httparse",
"httpdate",
"itoa",
"pin-project-lite",
"socket2 0.5.10",
"tokio",
"tower-service",
"tracing",
"want",
]
[[package]]
name = "hyper"
version = "1.8.1"
@@ -3293,7 +3259,7 @@ dependencies = [
"bytes",
"futures-channel",
"futures-core",
"h2 0.4.13",
"h2",
"http 1.4.0",
"http-body 1.0.1",
"httparse",
@@ -3313,7 +3279,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73b7d8abf35697b81a825e386fc151e0d503e8cb5fcb93cc8669c376dfd6f278"
dependencies = [
"hex",
"hyper 1.8.1",
"hyper",
"hyper-util",
"pin-project-lite",
"tokio",
@@ -3321,21 +3287,6 @@ dependencies = [
"winapi",
]
[[package]]
name = "hyper-rustls"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec3efd23720e2049821a693cbc7e65ea87c72f1c58ff2f9522ff332b1491e590"
dependencies = [
"futures-util",
"http 0.2.12",
"hyper 0.14.32",
"log",
"rustls 0.21.12",
"tokio",
"tokio-rustls 0.24.1",
]
[[package]]
name = "hyper-rustls"
version = "0.27.7"
@@ -3343,13 +3294,13 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3c93eb611681b207e1fe55d5a71ecf91572ec8a6705cdb6857f7d8d5242cf58"
dependencies = [
"http 1.4.0",
"hyper 1.8.1",
"hyper",
"hyper-util",
"rustls 0.23.37",
"rustls",
"rustls-native-certs",
"rustls-pki-types",
"tokio",
"tokio-rustls 0.26.4",
"tokio-rustls",
"tower-service",
"webpki-roots 1.0.6",
]
@@ -3360,7 +3311,7 @@ version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b90d566bffbce6a75bd8b09a05aa8c2cb1fabb6cb348f8840c9e4c90a0d83b0"
dependencies = [
"hyper 1.8.1",
"hyper",
"hyper-util",
"pin-project-lite",
"tokio",
@@ -3379,7 +3330,7 @@ dependencies = [
"futures-util",
"http 1.4.0",
"http-body 1.0.1",
"hyper 1.8.1",
"hyper",
"ipnet",
"libc",
"percent-encoding",
@@ -3400,7 +3351,7 @@ checksum = "986c5ce3b994526b3cd75578e62554abd09f0899d6206de48b3e96ab34ccc8c7"
dependencies = [
"hex",
"http-body-util",
"hyper 1.8.1",
"hyper",
"hyper-util",
"pin-project-lite",
"tokio",
@@ -3931,11 +3882,11 @@ dependencies = [
"percent-encoding",
"quoted_printable",
"rsa",
"rustls 0.23.37",
"rustls",
"sha2",
"socket2 0.6.3",
"tokio",
"tokio-rustls 0.26.4",
"tokio-rustls",
"tracing",
"url",
"webpki-roots 1.0.6",
@@ -4398,7 +4349,7 @@ checksum = "dd7399781913e5393588a8d8c6a2867bf85fb38eaf2502fdce465aad2dc6f034"
dependencies = [
"base64 0.22.1",
"http-body-util",
"hyper 1.8.1",
"hyper",
"hyper-util",
"indexmap 2.13.0",
"ipnet",
@@ -5555,7 +5506,7 @@ dependencies = [
"quinn-proto",
"quinn-udp",
"rustc-hash",
"rustls 0.23.37",
"rustls",
"socket2 0.6.3",
"thiserror 2.0.18",
"tokio",
@@ -5575,7 +5526,7 @@ dependencies = [
"rand 0.9.2",
"ring",
"rustc-hash",
"rustls 0.23.37",
"rustls",
"rustls-pki-types",
"slab",
"thiserror 2.0.18",
@@ -5898,12 +5849,12 @@ dependencies = [
"encoding_rs",
"futures-core",
"futures-util",
"h2 0.4.13",
"h2",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
"hyper-rustls 0.27.7",
"hyper",
"hyper-rustls",
"hyper-util",
"js-sys",
"log",
@@ -5912,14 +5863,14 @@ dependencies = [
"percent-encoding",
"pin-project-lite",
"quinn",
"rustls 0.23.37",
"rustls",
"rustls-pki-types",
"serde",
"serde_json",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tokio-rustls 0.26.4",
"tokio-rustls",
"tokio-util",
"tower",
"tower-http",
@@ -6062,18 +6013,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "rustls"
version = "0.21.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f56a14d1f48b391359b22f731fd4bd7e43c97f3c50eee276f3aa09c94784d3e"
dependencies = [
"log",
"ring",
"rustls-webpki 0.101.7",
"sct",
]
[[package]]
name = "rustls"
version = "0.23.37"
@@ -6085,7 +6024,7 @@ dependencies = [
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki 0.103.10",
"rustls-webpki",
"subtle",
"zeroize",
]
@@ -6121,16 +6060,6 @@ dependencies = [
"zeroize",
]
[[package]]
name = "rustls-webpki"
version = "0.101.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b6275d1ee7a1cd780b64aca7726599a1dbc893b1e64144529e55c3c2f745765"
dependencies = [
"ring",
"untrusted",
]
[[package]]
name = "rustls-webpki"
version = "0.103.10"
@@ -6212,16 +6141,6 @@ version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "sct"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da046153aa2352493d6cb7da4b6e5c0c057d8a1d0a9aa8560baffdd945acd414"
dependencies = [
"ring",
"untrusted",
]
[[package]]
name = "sec1"
version = "0.3.0"
@@ -6560,6 +6479,16 @@ dependencies = [
"thiserror 2.0.18",
]
[[package]]
name = "signal-hook"
version = "0.3.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d881a16cf4426aa584979d30bd82cb33429027e42122b169753d6ef1085ed6e2"
dependencies = [
"libc",
"signal-hook-registry",
]
[[package]]
name = "signal-hook-registry"
version = "1.4.8"
@@ -6775,7 +6704,7 @@ dependencies = [
"memchr",
"once_cell",
"percent-encoding",
"rustls 0.23.37",
"rustls",
"serde",
"serde_json",
"sha2",
@@ -7120,6 +7049,7 @@ dependencies = [
"async-trait",
"bollard",
"bytes",
"conquer-once",
"docker_credential",
"either",
"etcetera 0.11.0",
@@ -7133,6 +7063,7 @@ dependencies = [
"serde",
"serde_json",
"serde_with",
"signal-hook",
"thiserror 2.0.18",
"tokio",
"tokio-stream",
@@ -7301,23 +7232,13 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "tokio-rustls"
version = "0.24.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c28327cf380ac148141087fbfb9de9d7bd4e84ab5d2c28fbc911d753de8a7081"
dependencies = [
"rustls 0.21.12",
"tokio",
]
[[package]]
name = "tokio-rustls"
version = "0.26.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1729aa945f29d91ba541258c8df89027d5792d85a8841fb65e8bf0f4ede4ef61"
dependencies = [
"rustls 0.23.37",
"rustls",
"tokio",
]
@@ -7340,10 +7261,10 @@ checksum = "d25a406cddcc431a75d3d9afc6a7c0f7428d4891dd973e4d54c56b46127bf857"
dependencies = [
"futures-util",
"log",
"rustls 0.23.37",
"rustls",
"rustls-pki-types",
"tokio",
"tokio-rustls 0.26.4",
"tokio-rustls",
"tungstenite 0.28.0",
"webpki-roots 0.26.11",
]
@@ -7474,11 +7395,11 @@ dependencies = [
"axum",
"base64 0.22.1",
"bytes",
"h2 0.4.13",
"h2",
"http 1.4.0",
"http-body 1.0.1",
"http-body-util",
"hyper 1.8.1",
"hyper",
"hyper-timeout",
"hyper-util",
"percent-encoding",
@@ -8045,7 +7966,7 @@ dependencies = [
"quinn",
"rand 0.9.2",
"rcgen",
"rustls 0.23.37",
"rustls",
"serde",
"sha2",
"thiserror 2.0.18",
@@ -8091,16 +8012,16 @@ dependencies = [
"h3-quinn",
"hex",
"http 1.4.0",
"hyper 1.8.1",
"hyper",
"hyper-util",
"quinn",
"rcgen",
"reqwest",
"rustls 0.23.37",
"rustls",
"rustls-pemfile",
"thiserror 2.0.18",
"tokio",
"tokio-rustls 0.26.4",
"tokio-rustls",
"tokio-util",
"tower",
"tracing",
@@ -8278,7 +8199,7 @@ dependencies = [
"httparse",
"log",
"rand 0.9.2",
"rustls 0.23.37",
"rustls",
"rustls-pki-types",
"sha1",
"thiserror 2.0.18",
@@ -8397,7 +8318,7 @@ dependencies = [
"base64 0.22.1",
"log",
"percent-encoding",
"rustls 0.23.37",
"rustls",
"rustls-pki-types",
"ureq-proto",
"utf8-zero",
@@ -9185,7 +9106,7 @@ dependencies = [
"futures",
"http 1.4.0",
"http-body-util",
"hyper 1.8.1",
"hyper",
"hyper-util",
"log",
"once_cell",
+7 -1
View File
@@ -64,7 +64,7 @@ bincode = { version = "2", features = ["serde"] }
anyhow = "1.0"
async-trait = "0.1"
aws-config = "1.8"
aws-sdk-s3 = "1.118"
aws-sdk-s3 = { version = "1.118", default-features = false, features = ["sigv4a", "default-https-client", "rt-tokio"] }
axum = { version = "0.8", features = ["ws", "macros"] }
base32 = "0.5"
base64 = "0.22"
@@ -170,3 +170,9 @@ lto = "fat"
codegen-units = 1
debug = 1
strip = false
[profile.dev.package.bcrypt]
opt-level = 3
[profile.dev.package.blowfish]
opt-level = 3
+1 -1
View File
@@ -90,7 +90,7 @@ tranquil-infra = { workspace = true, features = ["testing"] }
tempfile = "3"
ciborium = { workspace = true }
ctor = { workspace = true }
testcontainers = { workspace = true }
testcontainers = { workspace = true, features = ["watchdog"] }
testcontainers-modules = { workspace = true }
tranquil-ripple = { workspace = true }
tranquil-sync = { workspace = true }
+56 -10
View File
@@ -83,28 +83,41 @@ fn has_external_infra() -> bool {
|| (std::env::var("DATABASE_URL").is_ok()
&& (std::env::var("S3_ENDPOINT").is_ok() || std::env::var("BLOB_STORAGE_PATH").is_ok()))
}
#[cfg(not(feature = "external-infra"))]
fn started_container_ids() -> Vec<String> {
let db = DB_CONTAINER.get().map(|c| c.id().to_string());
#[cfg(feature = "s3")]
let s3 = S3_CONTAINER.get().map(|c| c.id().to_string());
#[cfg(not(feature = "s3"))]
let s3: Option<String> = None;
db.into_iter().chain(s3).collect()
}
#[cfg(feature = "external-infra")]
fn started_container_ids() -> Vec<String> {
Vec::new()
}
#[cfg(test)]
#[ctor::dtor]
fn cleanup() {
if let Some(temp_dir) = TEST_TEMP_DIR.get() {
let _ = std::fs::remove_dir_all(temp_dir);
}
if has_external_infra() {
let ids = started_container_ids();
if ids.is_empty() {
return;
}
if std::env::var("XDG_RUNTIME_DIR").is_ok() {
let _ = std::process::Command::new("podman")
.args(["rm", "-f", "--filter", "label=tranquil_pds_test=true"])
.args(["rm", "-f"])
.args(&ids)
.output();
}
let _ = std::process::Command::new("docker")
.args([
"container",
"prune",
"-f",
"--filter",
"label=tranquil_pds_test=true",
])
.args(["rm", "-f"])
.args(&ids)
.output();
}
@@ -734,7 +747,40 @@ pub async fn spawn_cluster(pool: Option<sqlx::PgPool>, node_count: usize) -> Vec
let first = &instances[0];
APP_PORT.set(first.port).ok();
tokio::time::sleep(Duration::from_millis(2000)).await;
let caches: Vec<Arc<dyn Cache>> = instances
.iter()
.map(|instance| instance.cache.clone().expect("cluster node has no cache"))
.collect();
futures::future::join_all(caches.iter().enumerate().map(|(i, cache)| async move {
cache
.set(
&format!("__cluster_ready_{i}"),
"1",
Duration::from_secs(60),
)
.await
.expect("cluster readiness probe failed");
}))
.await;
let deadline = tokio::time::Instant::now() + Duration::from_secs(10);
loop {
let ready = futures::future::join_all(caches.iter().flat_map(|cache| {
(0..caches.len()).map(move |i| async move {
cache.get(&format!("__cluster_ready_{i}")).await.is_some()
})
}))
.await
.into_iter()
.all(|seen| seen);
if ready {
break;
}
assert!(
tokio::time::Instant::now() < deadline,
"cluster nodes did not converge within 10s"
);
tokio::time::sleep(Duration::from_millis(50)).await;
}
instances
}
@@ -1,6 +1,4 @@
mod common;
mod helpers;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,5 +1,4 @@
mod common;
use common::{base_url, client, create_account_and_login, get_test_repos, user_id_of};
use crate::common::{base_url, client, create_account_and_login, get_test_repos, user_id_of};
use serde_json::{Value, json};
use tranquil_db_traits::{CommsChannel, CommsType};
use tranquil_types::{Did, Recipient};
@@ -1,5 +1,4 @@
mod common;
use common::{base_url, client, create_account_and_login};
use crate::common::{base_url, client, create_account_and_login};
use serde_json::{Value, json};
#[tokio::test]
@@ -1,5 +1,3 @@
mod common;
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::CommsType;
@@ -7,10 +5,10 @@ use tranquil_types::Did;
#[tokio::test]
async fn test_send_email_success() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let (access_jwt, did) = common::create_admin_account_and_login(&client).await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let (access_jwt, did) = crate::common::create_admin_account_and_login(&client).await;
let res = client
.post(format!("{}/xrpc/com.atproto.admin.sendEmail", base_url))
.bearer_auth(&access_jwt)
@@ -26,7 +24,7 @@ async fn test_send_email_success() {
assert_eq!(res.status(), StatusCode::OK);
let body: Value = res.json().await.expect("Invalid JSON");
assert_eq!(body["sent"], true);
let user_id = common::user_id_of(repos, &Did::new(did).unwrap()).await;
let user_id = crate::common::user_id_of(repos, &Did::new(did).unwrap()).await;
let comms = repos
.infra
.get_latest_comms_for_user(user_id, CommsType::AdminEmail, 1)
@@ -43,10 +41,10 @@ async fn test_send_email_success() {
#[tokio::test]
async fn test_send_email_default_subject() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let (access_jwt, did) = common::create_admin_account_and_login(&client).await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let (access_jwt, did) = crate::common::create_admin_account_and_login(&client).await;
let res = client
.post(format!("{}/xrpc/com.atproto.admin.sendEmail", base_url))
.bearer_auth(&access_jwt)
@@ -61,7 +59,7 @@ async fn test_send_email_default_subject() {
assert_eq!(res.status(), StatusCode::OK);
let body: Value = res.json().await.expect("Invalid JSON");
assert_eq!(body["sent"], true);
let user_id = common::user_id_of(repos, &Did::new(did).unwrap()).await;
let user_id = crate::common::user_id_of(repos, &Did::new(did).unwrap()).await;
let comms = repos
.infra
.get_latest_comms_for_user(user_id, CommsType::AdminEmail, 10)
@@ -83,9 +81,9 @@ async fn test_send_email_default_subject() {
#[tokio::test]
async fn test_send_email_recipient_not_found() {
let client = common::client();
let base_url = common::base_url().await;
let (access_jwt, _) = common::create_admin_account_and_login(&client).await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let (access_jwt, _) = crate::common::create_admin_account_and_login(&client).await;
let res = client
.post(format!("{}/xrpc/com.atproto.admin.sendEmail", base_url))
.bearer_auth(&access_jwt)
@@ -104,9 +102,9 @@ async fn test_send_email_recipient_not_found() {
#[tokio::test]
async fn test_send_email_missing_content() {
let client = common::client();
let base_url = common::base_url().await;
let (access_jwt, did) = common::create_admin_account_and_login(&client).await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let (access_jwt, did) = crate::common::create_admin_account_and_login(&client).await;
let res = client
.post(format!("{}/xrpc/com.atproto.admin.sendEmail", base_url))
.bearer_auth(&access_jwt)
@@ -125,9 +123,9 @@ async fn test_send_email_missing_content() {
#[tokio::test]
async fn test_send_email_missing_recipient() {
let client = common::client();
let base_url = common::base_url().await;
let (access_jwt, _) = common::create_admin_account_and_login(&client).await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let (access_jwt, _) = crate::common::create_admin_account_and_login(&client).await;
let res = client
.post(format!("{}/xrpc/com.atproto.admin.sendEmail", base_url))
.bearer_auth(&access_jwt)
@@ -144,8 +142,8 @@ async fn test_send_email_missing_recipient() {
#[tokio::test]
async fn test_send_email_requires_auth() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!("{}/xrpc/com.atproto.admin.sendEmail", base_url))
.json(&json!({
@@ -161,11 +159,11 @@ async fn test_send_email_requires_auth() {
#[tokio::test]
async fn test_send_email_rejects_garbage_stored_email() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let (access_jwt, did) = common::create_admin_account_and_login(&client).await;
let user_id = common::user_id_of(repos, &Did::new(did.clone()).unwrap()).await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let (access_jwt, did) = crate::common::create_admin_account_and_login(&client).await;
let user_id = crate::common::user_id_of(repos, &Did::new(did.clone()).unwrap()).await;
repos
.user
.update_email(user_id, "not-an-email")
@@ -1,6 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,6 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::Value;
@@ -1,5 +1,4 @@
mod common;
use common::{base_url, client, create_admin_account_and_login};
use crate::common::{base_url, client, create_admin_account_and_login};
use serde_json::Value;
#[tokio::test]
@@ -1,10 +1,7 @@
mod common;
mod helpers;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::Utc;
use common::{base_url, client, create_account_and_login, pds_endpoint};
use helpers::verify_new_account;
use crate::common::{base_url, client, create_account_and_login, pds_endpoint};
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
@@ -215,7 +212,7 @@ async fn test_oauth_admin_extractor_allows_oauth_tokens() {
let did = account["did"].as_str().unwrap().to_string();
verify_new_account(&http_client, &did).await;
let repos = common::get_test_repos().await;
let repos = crate::common::get_test_repos().await;
repos
.user
.set_admin_status(&tranquil_types::Did::new(did.clone()).unwrap(), true)
@@ -13,10 +13,8 @@
* - we maintain this code precisely because we believe everyone deserves an experience on the web that is free from this kinda language
*/
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::json;
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,7 +1,6 @@
mod common;
use base64::Engine;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use common::*;
use crate::common::*;
use k256::ecdsa::{SigningKey, signature::Signer};
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -100,7 +99,7 @@ async fn test_external_did_web_no_local_doc() {
let mock_addr = mock_uri.trim_start_matches("http://");
let did = format!("did:web:{}", mock_addr.replace(":", "%3A"));
let handle = format!("xw{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let pds_endpoint = common::pds_endpoint();
let pds_endpoint = crate::common::pds_endpoint();
let reserve_res = client
.post(format!(
@@ -203,7 +202,7 @@ async fn assert_reserved_key_placement_rejected(
let mock_addr = mock_uri.trim_start_matches("http://");
let did = format!("did:web:{}", mock_addr.replace(":", "%3A"));
let handle = format!("wm{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let pds_endpoint = common::pds_endpoint();
let pds_endpoint = crate::common::pds_endpoint();
let signing_key = reserve_signing_key(&client, base, &did).await;
let signing_multibase = signing_key
@@ -525,8 +524,8 @@ async fn test_did_web_byod_flow() {
unique_id
);
let handle = format!("by{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let pds_endpoint = common::pds_endpoint();
let pds_hostname = common::pds_hostname();
let pds_endpoint = crate::common::pds_endpoint();
let pds_hostname = crate::common::pds_hostname();
let pds_did = format!("did:web:{}", pds_hostname);
let temp_key = SigningKey::random(&mut rand::thread_rng());
@@ -579,7 +578,7 @@ async fn test_did_web_byod_flow() {
"BYOD accounts should require verification"
);
let access_jwt = common::verify_new_account(&client, returned_did).await;
let access_jwt = crate::common::verify_new_account(&client, returned_did).await;
let res = client
.get(format!(
@@ -1,11 +1,10 @@
mod common;
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::CommsType;
use tranquil_types::Did;
async fn get_email_update_token(did: &str) -> String {
let repos = common::get_test_repos().await;
let repos = crate::common::get_test_repos().await;
let parsed_did = Did::new(did.to_string()).unwrap();
let user = repos
.user
@@ -60,14 +59,14 @@ async fn create_verified_account(
assert_eq!(res.status(), StatusCode::OK);
let body: Value = res.json().await.expect("Invalid JSON");
let did = body["did"].as_str().expect("No did").to_string();
let jwt = common::verify_new_account(client, &did).await;
let jwt = crate::common::verify_new_account(client, &did).await;
(jwt, did)
}
#[tokio::test]
async fn test_request_email_update_returns_token_required() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("er{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
let (access_jwt, _) = create_verified_account(&client, base_url, &handle, &email).await;
@@ -88,9 +87,9 @@ async fn test_request_email_update_returns_token_required() {
#[tokio::test]
async fn test_update_email_flow_success() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("eu{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
let (access_jwt, did) = create_verified_account(&client, base_url, &handle, &email).await;
@@ -136,8 +135,8 @@ async fn test_update_email_flow_success() {
#[tokio::test]
async fn test_update_email_requires_token_when_verified() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("ed{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
let (access_jwt, _) = create_verified_account(&client, base_url, &handle, &email).await;
@@ -157,8 +156,8 @@ async fn test_update_email_requires_token_when_verified() {
#[tokio::test]
async fn test_update_email_same_email_noop() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("es{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
let (access_jwt, _) = create_verified_account(&client, base_url, &handle, &email).await;
@@ -179,8 +178,8 @@ async fn test_update_email_same_email_noop() {
#[tokio::test]
async fn test_update_email_invalid_token() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("eb{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
let (access_jwt, _) = create_verified_account(&client, base_url, &handle, &email).await;
@@ -214,8 +213,8 @@ async fn test_update_email_invalid_token() {
#[tokio::test]
async fn test_update_email_no_auth() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!("{}/xrpc/com.atproto.server.updateEmail", base_url))
@@ -230,8 +229,8 @@ async fn test_update_email_no_auth() {
#[tokio::test]
async fn test_update_email_invalid_format() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("ef{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
let (access_jwt, _) = create_verified_account(&client, base_url, &handle, &email).await;
@@ -248,9 +247,9 @@ async fn test_update_email_invalid_format() {
#[tokio::test]
async fn test_confirm_email_confirms_existing_email() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("ec{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
@@ -323,9 +322,9 @@ async fn test_confirm_email_confirms_existing_email() {
#[tokio::test]
async fn test_confirm_email_rejects_wrong_email() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("ew{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@jola.dev", handle);
@@ -391,8 +390,8 @@ async fn test_confirm_email_rejects_wrong_email() {
#[tokio::test]
async fn test_confirm_email_invalid_token() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("ei{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@jola.dev", handle);
@@ -433,9 +432,9 @@ async fn test_confirm_email_invalid_token() {
#[tokio::test]
async fn test_unverified_account_can_update_email_without_token() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("ev{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@nel.pet", handle);
@@ -503,9 +502,9 @@ async fn test_unverified_account_can_update_email_without_token() {
#[tokio::test]
async fn test_update_email_to_same_as_another_user_allowed() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle1 = format!("d1{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email1 = format!("{}@jola.dev", handle1);
@@ -557,8 +556,8 @@ async fn test_update_email_to_same_as_another_user_allowed() {
#[tokio::test]
async fn test_check_email_in_use_distinguishes_empty_from_invalid() {
let client = common::client();
let base = common::base_url().await;
let client = crate::common::client();
let base = crate::common::base_url().await;
let resp = client
.post(format!("{}/xrpc/_account.checkEmailInUse", base))
@@ -1,8 +1,6 @@
mod common;
use base64::Engine;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
use std::sync::{Arc, Mutex};
@@ -1,11 +1,7 @@
mod common;
mod firehose;
mod helpers;
use cid::Cid;
use common::*;
use firehose::FirehoseConsumer;
use helpers::build_car_with_signature;
use crate::common::*;
use crate::firehose::FirehoseConsumer;
use crate::helpers::build_car_with_signature;
use iroh_car::CarReader;
use k256::ecdsa::SigningKey;
use multihash::Multihash;
@@ -1,7 +1,5 @@
mod common;
use cid::Cid;
use common::*;
use crate::common::*;
use futures::{SinkExt, stream::StreamExt};
use iroh_car::CarReader;
use reqwest::StatusCode;
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_types::{Did, Nsid, Rkey};
@@ -1,7 +1,5 @@
mod common;
use chrono::Utc;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
use wiremock::matchers::{method, path};
@@ -98,7 +97,7 @@ async fn test_create_did_web_account_and_resolve() {
let did = format!("did:web:{}", mock_addr.replace(":", "%3A"));
let handle = format!("wu{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let base = base_url().await;
let pds_endpoint = common::pds_endpoint();
let pds_endpoint = crate::common::pds_endpoint();
let reserve_res = client
.post(format!(
@@ -220,7 +219,7 @@ async fn test_did_web_lifecycle() {
let handle = format!("lc{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let did = format!("did:web:{}:u:{}", mock_addr.replace(":", "%3A"), handle);
let email = format!("{}@test.com", handle);
let pds_endpoint = common::pds_endpoint();
let pds_endpoint = crate::common::pds_endpoint();
let reserve_res = client
.post(format!(
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use iroh_car::CarHeader;
use reqwest::StatusCode;
use serde_json::json;
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::{Client, StatusCode};
use serde_json::{Value, json};
use tranquil_pds::api::error::ApiError;
@@ -1,8 +1,7 @@
#![allow(unused_imports)]
mod common;
use crate::common::{base_url, client, create_account_and_login, get_test_repos};
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::{Duration, Utc};
use common::{base_url, client, create_account_and_login, get_test_repos};
use k256::SecretKey;
use k256::ecdsa::{Signature, SigningKey, signature::Signer};
use rand::rngs::OsRng;
@@ -1,6 +1,4 @@
mod common;
use common::{base_url, client, create_account_and_login, get_test_repos, user_id_of};
use crate::common::{base_url, client, create_account_and_login, get_test_repos, user_id_of};
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::{CommsChannel, CommsType};
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::{StatusCode, header};
use serde_json::{Value, json};
use std::time::Duration;
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
+85
View File
@@ -0,0 +1,85 @@
#[path = "../common/mod.rs"]
mod common;
#[path = "../firehose/mod.rs"]
mod firehose;
#[path = "../helpers/mod.rs"]
mod helpers;
#[path = "../mst_verify/mod.rs"]
mod mst_verify;
mod account_lifecycle;
mod account_notifications;
mod actor;
mod admin_email;
mod admin_invite;
mod admin_moderation;
mod admin_search;
mod admin_stats;
mod apple_sso_unit;
mod auth_extractor;
mod banned_words;
mod car_export_detects_corruption;
mod change_password;
mod commit_signing;
mod delete_account;
mod did_web;
mod dpop_unit;
mod email_update;
mod feed_proxy_aud;
mod firehose_inline_blocks;
mod firehose_validation;
mod gc_after_delete;
mod gc_compaction_restart;
mod identity;
mod image_processing;
mod import_verification;
mod invite;
mod invite_registration;
mod jwt_security;
mod legacy_2fa;
mod lifecycle_record;
mod lifecycle_session;
mod lifecycle_social;
mod moderation;
mod mst_diff_equivalence;
mod mst_firehose_e2e;
mod mst_inductive_firehose;
mod mst_integrity;
mod mst_property_fuzz;
mod mst_repair_audit;
mod mst_repair_durability;
mod mst_structural_repair;
mod notifications;
mod oauth;
mod oauth_lifecycle;
mod oauth_permission_sets;
mod oauth_scopes;
mod oauth_security;
mod oauth_token_eviction;
mod password_reset;
mod plc_operations;
mod plc_validation;
mod rate_limit;
mod record_validation;
mod repair_leaf_loss;
mod repo_batch;
mod repo_blob;
mod repo_conformance;
mod repo_lifecycle;
mod ripple_cluster;
mod scope_edge_cases;
mod security_fixes;
mod server;
mod session_management;
mod shutdown_unit;
mod signing_key;
mod sso;
mod store_parity;
mod sync_blob;
mod sync_conformance;
mod sync_deprecated;
mod sync_repo;
mod user_blocks_reachability;
mod validation_edge_cases;
mod verify_live_commit;
mod whole_story;
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,6 +1,3 @@
mod common;
mod firehose;
use std::collections::BTreeMap;
use std::io::Cursor;
use std::str::FromStr;
@@ -9,8 +6,8 @@ use std::time::Duration;
use bytes::Bytes;
use cid::Cid;
use common::*;
use firehose::{FirehoseConsumer, ParsedCommitFrame};
use crate::common::*;
use crate::firehose::{FirehoseConsumer, ParsedCommitFrame};
use iroh_car::CarReader;
use jacquard_common::smol_str::SmolStr;
use jacquard_repo::commit::Commit;
@@ -31,7 +28,7 @@ async fn car_to_blocks(car_bytes: &[u8]) -> BTreeMap<Cid, Bytes> {
blocks
}
fn op_to_verified(op: &firehose::ParsedRepoOp) -> Result<VerifiedWriteOp, String> {
fn op_to_verified(op: &crate::firehose::ParsedRepoOp) -> Result<VerifiedWriteOp, String> {
let key = SmolStr::new(&op.path);
match op.action {
RepoAction::Create => {
@@ -1,17 +1,14 @@
mod common;
mod mst_verify;
use std::collections::BTreeMap;
use std::str::FromStr;
use std::sync::Arc;
use cid::Cid;
use common::*;
use crate::common::*;
use jacquard_common::smol_str::SmolStr;
use jacquard_repo::commit::Commit;
use jacquard_repo::mst::{Mst, VerifiedWriteOp};
use jacquard_repo::storage::{BlockStore, MemoryBlockStore};
use mst_verify::{extract_event_blocks, inline_to_store};
use crate::mst_verify::{extract_event_blocks, inline_to_store};
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::{RepoEventType, SequenceNumber, SequencedEvent};
@@ -1,7 +1,3 @@
mod common;
mod firehose;
mod helpers;
use std::collections::BTreeMap;
use std::io::Cursor;
use std::sync::Arc;
@@ -9,9 +5,9 @@ use std::time::Duration;
use bytes::Bytes;
use cid::Cid;
use common::*;
use firehose::FirehoseConsumer;
use helpers::build_car_with_signature;
use crate::common::*;
use crate::firehose::FirehoseConsumer;
use crate::helpers::build_car_with_signature;
use iroh_car::CarReader;
use jacquard_repo::commit::Commit;
use jacquard_repo::mst::Mst;
@@ -1,16 +1,13 @@
mod common;
mod mst_verify;
use std::collections::HashMap;
use std::str::FromStr;
use cid::Cid;
use common::*;
use crate::common::*;
use jacquard_common::smol_str::SmolStr;
use jacquard_repo::commit::Commit;
use jacquard_repo::mst::{Mst, VerifiedWriteOp};
use jacquard_repo::storage::BlockStore;
use mst_verify::{extract_event_blocks, inline_to_store};
use crate::mst_verify::{extract_event_blocks, inline_to_store};
use rand::rngs::StdRng;
use rand::{Rng, SeedableRng};
use reqwest::StatusCode;
@@ -1,12 +1,11 @@
mod common;
use tranquil_db_traits::{CommsChannel, CommsStatus, CommsType};
use tranquil_types::{Did, Recipient};
#[tokio::test]
async fn test_enqueue_comms() {
let repos = common::get_test_repos().await;
let (_, did) = common::create_account_and_login(&common::client()).await;
let user_id = common::user_id_of(repos, &Did::new(did).unwrap()).await;
let repos = crate::common::get_test_repos().await;
let (_, did) = crate::common::create_account_and_login(&crate::common::client()).await;
let user_id = crate::common::user_id_of(repos, &Did::new(did).unwrap()).await;
repos
.infra
.enqueue_comms(
@@ -36,9 +35,9 @@ async fn test_enqueue_comms() {
#[tokio::test]
async fn test_comms_queue_status_index() {
let repos = common::get_test_repos().await;
let (_, did) = common::create_account_and_login(&common::client()).await;
let user_id = common::user_id_of(repos, &Did::new(did).unwrap()).await;
let repos = crate::common::get_test_repos().await;
let (_, did) = crate::common::create_account_and_login(&crate::common::client()).await;
let user_id = crate::common::user_id_of(repos, &Did::new(did).unwrap()).await;
let initial_count = repos
.infra
.count_comms_by_type(user_id, CommsType::PasswordReset)
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use common::{base_url, client, get_test_repos};
use helpers::verify_new_account;
use crate::common::{base_url, client, get_test_repos};
use crate::helpers::verify_new_account;
use reqwest::{StatusCode, redirect};
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
@@ -1,10 +1,7 @@
mod common;
mod helpers;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::Utc;
use common::{base_url, client};
use helpers::verify_new_account;
use crate::common::{base_url, client};
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
@@ -1,10 +1,7 @@
mod common;
mod helpers;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::Utc;
use common::{base_url, client, create_account_and_login};
use helpers::verify_new_account;
use crate::common::{base_url, client, create_account_and_login};
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
@@ -59,7 +56,7 @@ async fn setup_mock_client_metadata(redirect_uri: &str) -> MockServer {
}
async fn seed_permission_set(nsid: &str, granular_scope: &str) {
let state = common::get_test_app_state().await;
let state = crate::common::get_test_app_state().await;
let key = tranquil_pds::cache_keys::permission_set_key(
&tranquil_types::Nsid::new(nsid).unwrap(),
None,
@@ -597,7 +594,7 @@ async fn test_grant_row_keeps_include_jwt_carries_expanded() {
);
let token_id = token_id_from_jwt(&session.access_token);
let token_data = common::get_test_repos()
let token_data = crate::common::get_test_repos()
.await
.oauth
.get_token_by_id(&token_id)
@@ -916,7 +913,7 @@ async fn test_consent_post_errors_when_set_unresolvable() {
"Consent GET should succeed"
);
let state = common::get_test_app_state().await;
let state = crate::common::get_test_app_state().await;
let key = tranquil_pds::cache_keys::permission_set_key(
&tranquil_types::Nsid::new(UNRESOLVABLE_NSID).unwrap(),
None,
@@ -1211,7 +1208,7 @@ async fn test_consent_remember_persists_set_preference() {
let did: tranquil_types::Did = delegated_did.parse().expect("valid did");
let client_id_typed = tranquil_types::ClientId::new(client_id.clone());
let stored_prefs = common::get_test_repos()
let stored_prefs = crate::common::get_test_repos()
.await
.oauth
.get_scope_preferences(&did, &client_id_typed)
@@ -1595,7 +1592,7 @@ async fn test_consent_post_drops_unpresented_scope() {
let access_token = token_body["access_token"].as_str().unwrap().to_string();
let token_id = token_id_from_jwt(&access_token);
let token_data = common::get_test_repos()
let token_data = crate::common::get_test_repos()
.await
.oauth
.get_token_by_id(&token_id)
@@ -1,10 +1,7 @@
mod common;
mod helpers;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::Utc;
use common::{base_url, client, get_test_repos};
use helpers::verify_new_account;
use crate::common::{base_url, client, get_test_repos};
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
@@ -1,10 +1,8 @@
#![allow(unused_imports)]
mod common;
mod helpers;
use crate::common::{base_url, client, create_account_and_login};
use crate::helpers::verify_new_account;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::Utc;
use common::{base_url, client, create_account_and_login};
use helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use sha2::{Digest, Sha256};
@@ -1,10 +1,7 @@
mod common;
mod helpers;
use chrono::{DateTime, Duration, Utc};
use common::{base_url, client, get_test_repos};
use crate::common::{base_url, client, get_test_repos};
use futures::StreamExt;
use helpers::verify_new_account;
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_oauth::{
@@ -1,15 +1,13 @@
mod common;
mod helpers;
use helpers::verify_new_account;
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::CommsType;
#[tokio::test]
async fn test_request_password_reset_creates_code() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("pr{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@example.com", handle);
let payload = json!({
@@ -55,8 +53,8 @@ async fn test_request_password_reset_creates_code() {
#[tokio::test]
async fn test_request_password_reset_unknown_email_returns_ok() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.requestPasswordReset",
@@ -71,9 +69,9 @@ async fn test_request_password_reset_unknown_email_returns_ok() {
#[tokio::test]
async fn test_reset_password_with_valid_token() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("pr2{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@example.com", handle);
let old_password = "Oldpass123!";
@@ -167,8 +165,8 @@ async fn test_reset_password_with_valid_token() {
#[tokio::test]
async fn test_reset_password_with_invalid_token() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.resetPassword",
@@ -188,9 +186,9 @@ async fn test_reset_password_with_invalid_token() {
#[tokio::test]
async fn test_reset_password_with_expired_token() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("pr3{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@example.com", handle);
let payload = json!({
@@ -249,9 +247,9 @@ async fn test_reset_password_with_expired_token() {
#[tokio::test]
async fn test_reset_password_invalidates_sessions() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let handle = format!("pr4{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@example.com", handle);
let payload = json!({
@@ -320,8 +318,8 @@ async fn test_reset_password_invalidates_sessions() {
#[tokio::test]
async fn test_request_password_reset_empty_email() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.requestPasswordReset",
@@ -338,9 +336,9 @@ async fn test_request_password_reset_empty_email() {
#[tokio::test]
async fn test_reset_password_creates_notification() {
let repos = common::get_test_repos().await;
let client = common::client();
let base_url = common::base_url().await;
let repos = crate::common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("pr5{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let email = format!("{}@example.com", handle);
let payload = json!({
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::json;
use tranquil_types::Did;
@@ -1,5 +1,4 @@
mod common;
use common::{base_url, client};
use crate::common::{base_url, client};
use reqwest::StatusCode;
use serde_json::json;
@@ -1,9 +1,6 @@
mod common;
mod helpers;
use cid::Cid;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use jacquard_repo::commit::Commit;
use jacquard_repo::storage::BlockStore;
use serde_json::json;
@@ -1,6 +1,5 @@
mod common;
use chrono::Utc;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::{Backlink, BacklinkPath};
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::{StatusCode, header};
use serde_json::Value;
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,9 +1,6 @@
mod common;
mod firehose;
use crate::common::*;
use crate::firehose::{FirehoseConsumer, ParsedCommitFrame};
use cid::Cid;
use common::*;
use firehose::{FirehoseConsumer, ParsedCommitFrame};
use iroh_car::CarReader;
use jacquard_repo::commit::Commit;
use reqwest::StatusCode;
@@ -12,8 +9,6 @@ use std::io::Cursor;
use std::str::FromStr;
use tranquil_scopes::RepoAction;
mod helpers;
async fn create_post_record(client: &reqwest::Client, token: &str, did: &str, text: &str) -> Value {
let payload = json!({
"repo": did,
@@ -384,7 +379,7 @@ async fn test_firehose_commit_signature_verification() {
let client = client();
let (token, did) = create_account_and_login(&client).await;
let key_bytes = helpers::get_user_signing_key(&did)
let key_bytes = crate::helpers::get_user_signing_key(&did)
.await
.expect("Failed to get signing key");
let signing_key =
@@ -1,5 +1,3 @@
mod common;
use reqwest::StatusCode;
use serde_json::json;
use std::sync::Arc;
@@ -25,14 +23,14 @@ where
}
}
fn cache_for(nodes: &[common::ServerInstance], idx: usize) -> Arc<dyn Cache> {
fn cache_for(nodes: &[crate::common::ServerInstance], idx: usize) -> Arc<dyn Cache> {
nodes[idx]
.cache
.clone()
.unwrap_or_else(|| panic!("node {idx} should have a cache"))
}
fn rl_for(nodes: &[common::ServerInstance], idx: usize) -> Arc<dyn DistributedRateLimiter> {
fn rl_for(nodes: &[crate::common::ServerInstance], idx: usize) -> Arc<dyn DistributedRateLimiter> {
nodes[idx]
.distributed_rate_limiter
.clone()
@@ -41,10 +39,10 @@ fn rl_for(nodes: &[common::ServerInstance], idx: usize) -> Arc<dyn DistributedRa
#[tokio::test]
async fn cluster_formation() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
assert!(nodes.len() >= 3, "expected at least 3 cluster nodes");
let client = common::client();
let client = crate::common::client();
let results: Vec<_> = futures::future::join_all(nodes.iter().map(|node| {
let client = client.clone();
let url = node.url.clone();
@@ -71,8 +69,8 @@ async fn cluster_formation() {
#[tokio::test]
async fn cluster_any_node_access() {
let nodes = common::cluster().await;
let client = common::client();
let nodes = crate::common::cluster().await;
let client = crate::common::client();
let handle = format!("u{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let payload = serde_json::json!({
@@ -97,7 +95,7 @@ async fn cluster_any_node_access() {
.expect("no accessJwt")
.to_string();
let repos = common::get_test_repos().await;
let repos = crate::common::get_test_repos().await;
let user = repos
.user
.get_by_did(&tranquil_types::Did::new(did.clone()).unwrap())
@@ -176,7 +174,7 @@ async fn cluster_any_node_access() {
#[tokio::test]
async fn cache_convergence() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let cache_a = nodes[0].cache.as_ref().expect("node 0 should have a cache");
let cache_b = nodes[1].cache.as_ref().expect("node 1 should have a cache");
@@ -218,7 +216,7 @@ async fn cache_convergence() {
#[tokio::test]
async fn rate_limit_convergence() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let rl_a = nodes[0]
.distributed_rate_limiter
@@ -274,7 +272,7 @@ async fn rate_limit_convergence() {
#[tokio::test]
async fn delete_convergence() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let cache_0 = cache_for(nodes, 0);
let cache_1 = cache_for(nodes, 1);
@@ -308,7 +306,7 @@ async fn delete_convergence() {
#[tokio::test]
async fn three_node_transitive_convergence() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let cache_0 = cache_for(nodes, 0);
let cache_2 = cache_for(nodes, 2);
@@ -331,7 +329,7 @@ async fn three_node_transitive_convergence() {
#[tokio::test]
async fn cluster_overwrite_conflict_resolution() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let cache_0 = cache_for(nodes, 0);
let cache_1 = cache_for(nodes, 1);
let cache_2 = cache_for(nodes, 2);
@@ -374,7 +372,7 @@ async fn cluster_overwrite_conflict_resolution() {
#[tokio::test]
async fn cluster_bulk_key_convergence() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let cache_0 = cache_for(nodes, 0);
let cache_1 = cache_for(nodes, 1);
let cache_2 = cache_for(nodes, 2);
@@ -433,7 +431,7 @@ async fn cluster_bulk_key_convergence() {
#[tokio::test]
async fn cluster_concurrent_multi_node_writes() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let cache_0 = cache_for(nodes, 0);
let cache_1 = cache_for(nodes, 1);
let cache_2 = cache_for(nodes, 2);
@@ -542,7 +540,7 @@ async fn cluster_concurrent_multi_node_writes() {
#[tokio::test]
async fn cluster_rate_limit_multi_node_convergence() {
let nodes = common::cluster().await;
let nodes = crate::common::cluster().await;
let rl_0 = rl_for(nodes, 0);
let rl_1 = rl_for(nodes, 1);
let rl_2 = rl_for(nodes, 2);
@@ -632,7 +630,7 @@ fn create_account_on_node<'a>(
.expect("no accessJwt")
.to_string();
let repos = common::get_test_repos().await;
let repos = crate::common::get_test_repos().await;
let user = repos
.user
.get_by_did(&tranquil_types::Did::new(did.clone()).unwrap())
@@ -690,8 +688,8 @@ fn create_account_on_node<'a>(
#[tokio::test]
async fn cross_node_rate_limit_via_login() {
let nodes = common::cluster().await;
let client = common::client();
let nodes = crate::common::cluster().await;
let client = crate::common::client();
let now_ms = u64::try_from(
std::time::SystemTime::now()
@@ -772,8 +770,8 @@ async fn cross_node_rate_limit_via_login() {
#[tokio::test]
async fn cross_node_handle_resolution_from_cache() {
let nodes = common::cluster().await;
let client = common::client();
let nodes = crate::common::cluster().await;
let client = crate::common::client();
let cache_0 = cache_for(nodes, 0);
let fake_handle = format!("cached-{}.test", uuid::Uuid::new_v4().simple());
@@ -825,8 +823,8 @@ async fn cross_node_handle_resolution_from_cache() {
#[tokio::test]
async fn cross_node_cache_delete_observable_via_http() {
let nodes = common::cluster().await;
let client = common::client();
let nodes = crate::common::cluster().await;
let client = crate::common::client();
let cache_0 = cache_for(nodes, 0);
let cache_1 = cache_for(nodes, 1);
@@ -892,8 +890,8 @@ async fn cross_node_cache_delete_observable_via_http() {
#[tokio::test]
async fn cross_node_email_update_status() {
let nodes = common::cluster().await;
let client = common::client();
let nodes = crate::common::cluster().await;
let client = crate::common::client();
let cache_0 = cache_for(nodes, 0);
let cache_1 = cache_for(nodes, 1);
@@ -967,8 +965,8 @@ async fn cross_node_email_update_status() {
#[tokio::test]
async fn cross_node_session_revocation() {
let nodes = common::cluster().await;
let client = common::client();
let nodes = crate::common::cluster().await;
let client = crate::common::client();
let (token, _did) = create_account_on_node(&client, &nodes[0].url).await;
@@ -1,4 +1,3 @@
mod common;
use tranquil_pds::comms::{SendError, is_valid_phone_number};
use tranquil_pds::image::{ImageError, ImageProcessor};
@@ -84,7 +83,7 @@ fn test_send_error_display() {
#[tokio::test]
async fn test_signup_queue_authentication() {
use common::{base_url, client, create_account_and_login};
use crate::common::{base_url, client, create_account_and_login};
let base = base_url().await;
let http_client = client();
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::verify_new_account;
use crate::common::*;
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -1,13 +1,11 @@
mod common;
mod helpers;
use helpers::verify_new_account;
use crate::helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
#[tokio::test]
async fn test_reserve_signing_key_without_did() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -29,9 +27,9 @@ async fn test_reserve_signing_key_without_did() {
#[tokio::test]
async fn test_reserve_signing_key_with_did() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let target_did = "did:plc:test123456";
let res = client
.post(format!(
@@ -60,9 +58,9 @@ async fn test_reserve_signing_key_with_did() {
#[tokio::test]
async fn test_reserve_signing_key_stores_private_key() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -100,8 +98,8 @@ async fn test_reserve_signing_key_stores_private_key() {
#[tokio::test]
async fn test_reserve_signing_key_unique_keys() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res1 = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -131,8 +129,8 @@ async fn test_reserve_signing_key_unique_keys() {
#[tokio::test]
async fn test_reserve_signing_key_is_public() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -151,9 +149,9 @@ async fn test_reserve_signing_key_is_public() {
#[tokio::test]
async fn test_create_account_with_reserved_signing_key() {
let client = common::client();
let base_url = common::base_url().await;
let repos = common::get_test_repos().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let repos = crate::common::get_test_repos().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -203,8 +201,8 @@ async fn test_create_account_with_reserved_signing_key() {
#[tokio::test]
async fn test_create_account_with_invalid_signing_key() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let handle = format!("bk{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let res = client
.post(format!(
@@ -227,8 +225,8 @@ async fn test_create_account_with_invalid_signing_key() {
#[tokio::test]
async fn test_create_account_cannot_reuse_signing_key() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -280,8 +278,8 @@ async fn test_create_account_cannot_reuse_signing_key() {
#[tokio::test]
async fn test_reserved_key_tokens_work() {
let client = common::client();
let base_url = common::base_url().await;
let client = crate::common::client();
let base_url = crate::common::base_url().await;
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.reserveSigningKey",
@@ -1,6 +1,4 @@
mod common;
use common::{base_url, client, create_account_and_login, get_test_repos};
use crate::common::{base_url, client, create_account_and_login, get_test_repos};
use reqwest::StatusCode;
use serde_json::{Value, json};
use tranquil_db_traits::{CommsChannel, CommsType, SsoAction, SsoProviderType};
@@ -1,6 +1,3 @@
mod common;
mod helpers;
use std::sync::Arc;
use tranquil_db::PostgresRepositories;
use tranquil_db_traits::{Backlink, BacklinkPath, CommsChannel, CommsType};
@@ -85,7 +82,7 @@ async fn create_store_repos() -> Arc<PostgresRepositories> {
}
async fn create_pg_repos() -> Arc<PostgresRepositories> {
let db_url = common::get_db_connection_string().await;
let db_url = crate::common::get_db_connection_string().await;
let pool = sqlx::postgres::PgPoolOptions::new()
.max_connections(5)
.connect(&db_url)
@@ -117,7 +114,7 @@ fn test_handle(suffix: &str) -> Handle {
}
fn test_cid(seed: u8) -> CidLink {
CidLink::from_cid(&helpers::make_cid(&[seed]))
CidLink::from_cid(&crate::helpers::make_cid(&[seed]))
}
fn test_nsid(name: &str) -> Nsid {
@@ -139,7 +136,7 @@ fn test_at_uri(did: &Did, collection: &Nsid, rkey: &Rkey) -> AtUri {
}
async fn seed_user(repos: &PostgresRepositories, did: &Did, handle: &Handle) -> Uuid {
let commit_cid = CidLink::from_cid(&helpers::make_cid(did.as_str().as_bytes()));
let commit_cid = CidLink::from_cid(&crate::helpers::make_cid(did.as_str().as_bytes()));
let input = tranquil_db_traits::CreatePasswordAccountInput {
handle: handle.clone(),
email: None,
@@ -1780,7 +1777,7 @@ async fn parity_prune_events_older_than() {
let baseline = f.pg.repo.get_max_seq().await.unwrap();
f.pg.repo.insert_commit_event(&event).await.unwrap();
f.store.repo.insert_commit_event(&event).await.unwrap();
let pg_seq = common::sequenced_event_for_did(&f.pg, baseline, &did)
let pg_seq = crate::common::sequenced_event_for_did(&f.pg, baseline, &did)
.await
.seq;
@@ -1,5 +1,4 @@
mod common;
use common::*;
use crate::common::*;
use reqwest::StatusCode;
use reqwest::header;
use serde_json::Value;
@@ -1,8 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::Value;
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::Value;
@@ -1,7 +1,5 @@
mod common;
mod helpers;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use reqwest::header;
use serde_json::{Value, json};
@@ -1,8 +1,6 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use helpers::*;
use crate::common::*;
use crate::helpers::*;
use reqwest::StatusCode;
use serde_json::json;
use std::sync::LazyLock;
@@ -1,7 +1,6 @@
use bytes::Bytes;
use cid::Cid;
use std::collections::HashMap;
mod common;
#[tokio::test]
#[ignore = "depends on external live server state; run manually with --ignored"]
@@ -1,10 +1,7 @@
mod common;
mod helpers;
use chrono::Utc;
use common::*;
use crate::common::*;
use futures::{StreamExt, future::join_all};
use helpers::*;
use crate::helpers::*;
use k256::ecdsa::SigningKey;
use reqwest::{StatusCode, header};
use serde_json::{Value, json};
@@ -36,8 +36,22 @@ async fn spawn_pair(
.await
.expect("node B failed to start");
tokio::time::sleep(Duration::from_millis(2000)).await;
cache_a
.set("__ready_a", "1", Duration::from_secs(60))
.await
.expect("readiness probe on A failed");
cache_b
.set("__ready_b", "1", Duration::from_secs(60))
.await
.expect("readiness probe on B failed");
poll_until(10_000, 50, || {
let cache_a = cache_a.clone();
let cache_b = cache_b.clone();
async move {
cache_b.get("__ready_a").await.is_some() && cache_a.get("__ready_b").await.is_some()
}
})
.await;
((cache_a, rl_a), (cache_b, rl_b))
}
@@ -780,7 +794,5 @@ async fn two_node_stress_concurrent_load() {
r.unwrap_or_else(|e| panic!("task {i} panicked: {e}"));
});
tokio::time::sleep(Duration::from_secs(12)).await;
shutdown.cancel();
}
+1 -5
View File
@@ -92,11 +92,7 @@ fn blob_covers(g: &BlobScope, r: &BlobScope) -> bool {
}
fn rpc_covers(g: &RpcScope, r: &RpcScope) -> bool {
let lxm_ok = match &g.lxm {
None => true,
Some(gl) if gl == "*" => true,
Some(gl) => r.lxm.as_deref() == Some(gl.as_str()),
};
let lxm_ok = g.lxm == "*" || g.lxm == r.lxm;
let aud_ok = match &g.aud {
None => true,
Some(ga) if ga == "*" => true,
+13 -9
View File
@@ -93,7 +93,7 @@ impl BlobScope {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RpcScope {
pub lxm: Option<String>,
pub lxm: String,
pub aud: Option<String>,
}
@@ -245,9 +245,10 @@ pub fn parse_scope(scope: &str) -> ParsedScope {
let lxm = lxm_positional.or_else(|| params.get("lxm").and_then(|v| v.first().cloned()));
let aud = params.get("aud").and_then(|v| v.first().cloned());
let is_lxm_wildcard = lxm.as_deref() == Some("*") || lxm.is_none();
let is_aud_wildcard = aud.as_deref() == Some("*");
if is_lxm_wildcard && is_aud_wildcard {
let Some(lxm) = lxm else {
return ParsedScope::Unknown(scope.to_string());
};
if lxm == "*" && aud.as_deref() == Some("*") {
return ParsedScope::Unknown(scope.to_string());
}
@@ -400,7 +401,7 @@ mod tests {
let scope = parse_scope("rpc:app.bsky.feed.getTimeline?aud=did:web:api.bsky.app");
match scope {
ParsedScope::Rpc(r) => {
assert_eq!(r.lxm, Some("app.bsky.feed.getTimeline".to_string()));
assert_eq!(r.lxm, "app.bsky.feed.getTimeline");
assert_eq!(r.aud, Some("did:web:api.bsky.app".to_string()));
}
_ => panic!("Expected Rpc scope"),
@@ -510,6 +511,12 @@ mod tests {
assert!(matches!(scope4, ParsedScope::Rpc(_)));
}
#[test]
fn test_rpc_lxm_required() {
let bare = parse_scope("rpc");
assert!(matches!(bare, ParsedScope::Unknown(_)));
}
#[test]
fn test_url_encoded_aud_with_fragment() {
let scope =
@@ -527,10 +534,7 @@ mod tests {
);
match scope2 {
ParsedScope::Rpc(r) => {
assert_eq!(
r.lxm,
Some("com.atproto.moderation.createReport".to_string())
);
assert_eq!(r.lxm, "com.atproto.moderation.createReport");
assert_eq!(r.aud, Some("did:web:api.bsky.app#bsky_appview".to_string()));
}
_ => panic!("Expected Rpc scope"),
+23 -8
View File
@@ -49,10 +49,7 @@ impl ScopePermissions {
pub fn superseded_by_transition_generic(scope: &ParsedScope) -> bool {
match scope {
ParsedScope::Repo(_) | ParsedScope::Blob(_) => true,
ParsedScope::Rpc(rpc) => !rpc
.lxm
.as_deref()
.is_some_and(|lxm| lxm == "*" || lxm.starts_with("chat.bsky.")),
ParsedScope::Rpc(rpc) => !(rpc.lxm == "*" || rpc.lxm.starts_with("chat.bsky.")),
ParsedScope::Account(_)
| ParsedScope::Identity(_)
| ParsedScope::TransitionEmail
@@ -193,9 +190,9 @@ impl ScopePermissions {
let has_permission = self.find_rpc_scopes().any(|rpc_scope| {
let lxm_matches = match &rpc_scope.lxm {
None => true,
Some(scope_lxm) if scope_lxm == lxm => true,
Some(scope_lxm) if scope_lxm.ends_with(".*") => {
scope_lxm if scope_lxm == "*" => true,
scope_lxm if scope_lxm == lxm => true,
scope_lxm if scope_lxm.ends_with(".*") => {
let prefix = scope_lxm.strip_suffix(".*").unwrap();
lxm.starts_with(prefix) && lxm.chars().nth(prefix.len()) == Some('.')
}
@@ -203,7 +200,7 @@ impl ScopePermissions {
};
let aud_matches = match &rpc_scope.aud {
None => true,
None => false,
Some(scope_aud) if scope_aud == "*" => true,
Some(scope_aud) => scope_aud == aud,
};
@@ -474,6 +471,24 @@ mod tests {
assert!(!perms.allows_rpc("did:web:api.bsky.app", &c("app.bsky.feed.getAuthorFeed")));
}
#[test]
fn test_rpc_wildcard_lxm() {
let perms = ScopePermissions::from_scope_string(Some(
"rpc:*?aud=did:web:api.bsky.app#bsky_appview",
));
let aud = "did:web:api.bsky.app#bsky_appview";
let other = "did:web:other.app#bsky_appview";
assert!(perms.allows_rpc(aud, &c("app.bsky.feed.getTimeline")));
assert!(!perms.allows_rpc(other, &c("app.bsky.feed.getTimeline")));
}
#[test]
fn test_rpc_wildcard_lxm_without_aud() {
let perms = ScopePermissions::from_scope_string(Some("rpc:*"));
let aud = "did:web:api.bsky.app#bsky_appview";
assert!(!perms.allows_rpc(aud, &c("app.bsky.feed.getTimeline")));
}
#[test]
fn test_granular_account() {
let perms = ScopePermissions::from_scope_string(Some("account:email?action=read"));
+1
View File
@@ -46,6 +46,7 @@ test-harness = ["dep:tempfile"]
jemalloc = ["dep:tikv-jemallocator"]
gauntlet-cli = ["test-harness", "dep:clap", "dep:toml", "dep:tracing-subscriber", "jemalloc"]
gauntlet-jemalloc-prof = []
skip-fsync = []
[[bin]]
name = "tranquil-gauntlet"
+12
View File
@@ -267,10 +267,16 @@ impl StorageIO for RealIO {
self.lookup(id)?.write_at(buf, offset)
}
#[cfg(not(feature = "skip-fsync"))]
fn sync(&self, id: FileId) -> io::Result<()> {
self.lookup(id)?.sync_data()
}
#[cfg(feature = "skip-fsync")]
fn sync(&self, id: FileId) -> io::Result<()> {
self.lookup(id).map(|_| ())
}
fn file_size(&self, id: FileId) -> io::Result<u64> {
self.lookup(id)?.metadata().map(|m| m.len())
}
@@ -291,10 +297,16 @@ impl StorageIO for RealIO {
fs::create_dir_all(path)
}
#[cfg(not(feature = "skip-fsync"))]
fn sync_dir(&self, path: &Path) -> io::Result<()> {
fs::File::open(path)?.sync_all()
}
#[cfg(feature = "skip-fsync")]
fn sync_dir(&self, path: &Path) -> io::Result<()> {
fs::File::open(path).map(|_| ())
}
fn list_dir(&self, path: &Path) -> io::Result<Vec<PathBuf>> {
fs::read_dir(path)?
.map(|entry| entry.map(|e| e.path()))
+8 -3
View File
@@ -42,6 +42,11 @@ use self::user_hash::UserHashMap;
const CURRENT_FORMAT_VERSION: u64 = 3;
#[cfg(feature = "skip-fsync")]
const METASTORE_PERSIST_MODE: fjall::PersistMode = fjall::PersistMode::Buffer;
#[cfg(not(feature = "skip-fsync"))]
const METASTORE_PERSIST_MODE: fjall::PersistMode = fjall::PersistMode::SyncData;
#[derive(Debug, Clone)]
pub struct MetastoreConfig {
pub cache_size_bytes: u64,
@@ -243,14 +248,14 @@ impl Metastore {
repo_data.remove(records::record_by_cid_built_key().as_slice())?;
Self::migrate_blob_ownership(db, repo_data)?;
repo_data.insert(version_key, version_bytes)?;
db.persist(fjall::PersistMode::SyncData)?;
db.persist(METASTORE_PERSIST_MODE)?;
Ok(())
}
}
}
None => {
repo_data.insert(version_key, version_bytes)?;
db.persist(fjall::PersistMode::SyncData)?;
db.persist(METASTORE_PERSIST_MODE)?;
Ok(())
}
}
@@ -427,7 +432,7 @@ impl Metastore {
pub fn persist(&self) -> Result<(), MetastoreError> {
self.db
.persist(fjall::PersistMode::SyncData)
.persist(METASTORE_PERSIST_MODE)
.map_err(MetastoreError::Fjall)
}
+15 -17
View File
@@ -96,52 +96,50 @@ test-store-asan:
cargo +nightly nextest run -p tranquil-store --features tranquil-store/test-harness --target x86_64-unknown-linux-gnu
test-unit:
SQLX_OFFLINE=true cargo test --test dpop_unit --test validation_edge_cases --test scope_edge_cases
SQLX_OFFLINE=true cargo test --test it -- dpop_unit:: validation_edge_cases:: scope_edge_cases::
store_test := "SQLX_OFFLINE=true TRANQUIL_TEST_BACKEND=store TRANQUIL_PDS_ALLOW_INSECURE_SECRETS=1 DISABLE_RATE_LIMITING=1 TRANQUIL_LEXICON_OFFLINE=1 SKIP_IMPORT_VERIFICATION=true cargo nextest run -E 'not package(tranquil-store) and not binary(store_parity)'"
store_run := "SQLX_OFFLINE=true TRANQUIL_TEST_BACKEND=store TRANQUIL_PDS_ALLOW_INSECURE_SECRETS=1 DISABLE_RATE_LIMITING=1 TRANQUIL_LEXICON_OFFLINE=1 SKIP_IMPORT_VERIFICATION=true cargo nextest run --workspace --exclude tranquil-store --features tranquil-store/skip-fsync"
store_test := store_run + " -E 'not (binary(it) and test(/^store_parity::/)) and not (package(tranquil-signal) and test(/^tests::/))'"
test-auth:
{{store_test}} --test oauth --test oauth_lifecycle --test oauth_scopes --test oauth_security --test jwt_security --test session_management --test change_password --test password_reset
{{store_run}} --test it -E 'test(/^(oauth|oauth_lifecycle|oauth_scopes|oauth_security|jwt_security|session_management|change_password|password_reset)::/)'
test-admin:
{{store_test}} --test admin_email --test admin_invite --test admin_moderation --test admin_search --test admin_stats
{{store_run}} --test it -E 'test(/^(admin_email|admin_invite|admin_moderation|admin_search|admin_stats)::/)'
test-sync:
{{store_test}} --test sync_repo --test sync_blob --test sync_conformance --test sync_deprecated --test firehose_validation
{{store_run}} --test it -E 'test(/^(sync_repo|sync_blob|sync_conformance|sync_deprecated|firehose_validation)::/)'
test-repo:
{{store_test}} --test repo_batch --test repo_blob --test record_validation --test lifecycle_record
{{store_run}} --test it -E 'test(/^(repo_batch|repo_blob|record_validation|lifecycle_record)::/)'
test-identity:
{{store_test}} --test identity --test did_web --test plc_migration --test plc_operations --test plc_validation
{{store_run}} --test it --test plc_migration -E 'binary(plc_migration) | test(/^(identity|did_web|plc_operations|plc_validation)::/)'
test-account:
{{store_test}} --test lifecycle_session --test delete_account --test invite --test email_update --test account_notifications
{{store_run}} --test it -E 'test(/^(lifecycle_session|delete_account|invite|email_update|account_notifications)::/)'
test-security:
{{store_test}} --test security_fixes --test banned_words --test rate_limit --test moderation
{{store_run}} --test it -E 'test(/^(security_fixes|banned_words|rate_limit|moderation)::/)'
test-import:
{{store_test}} --test import_verification --test import_with_verification
{{store_run}} --test it --test import_with_verification -E 'binary(import_with_verification) | test(/^import_verification::/)'
test-misc:
{{store_test}} --test actor --test commit_signing --test image_processing --test lifecycle_social --test notifications --test server --test signing_key --test verify_live_commit
{{store_run}} --test it -E 'test(/^(actor|commit_signing|image_processing|lifecycle_social|notifications|server|signing_key|verify_live_commit)::/)'
test *args:
@just test-unit
{{store_test}} {{args}}
test-one name:
{{store_test}} --test {{name}}
{{store_run}} --test it -E 'test(/^{{name}}::/)'
test-full *args:
@just test-unit
@just services-up
eval "$(tranquil-dev-services env)" && SQLX_OFFLINE=true cargo nextest run --features tranquil-pds/s3 -E 'not package(tranquil-store)' {{args}}
eval "$(tranquil-dev-services env)" && SQLX_OFFLINE=true cargo nextest run --workspace --exclude tranquil-store --features tranquil-pds/s3 {{args}}
test-pg *args:
@just test-unit
./scripts/run-tests.sh {{args}}
SQLX_OFFLINE=true ./scripts/run-tests.sh {{args}}
services-up:
tranquil-dev-services up
+1 -4
View File
@@ -13,12 +13,9 @@ source "${TMPDIR:-/tmp}/tranquil_pds_test_infra.env"
echo ""
ulimit -n 65536
echo "Building test binaries..."
cargo test --no-run 2>&1 | tail -1
echo "Running tests..."
echo ""
cargo nextest run -E 'not package(tranquil-store)' "$@"
cargo nextest run --workspace --exclude tranquil-store "$@"
echo ""
echo "All tests passed."
+4 -1
View File
@@ -42,7 +42,10 @@ start_infra() {
-P \
--label tranquil_pds_test=true \
postgres:18-alpine \
-c max_connections=500 >/dev/null
-c max_connections=500 \
-c fsync=off \
-c synchronous_commit=off \
-c full_page_writes=off >/dev/null
echo "Waiting for services to be ready..."
for i in {1..30}; do
if $CONTAINER_CMD exec "${CONTAINER_PREFIX}-postgres" pg_isready -U postgres >/dev/null 2>&1; then