Compare commits

..
33 changed files with 247 additions and 242 deletions
+55
View File
@@ -4,6 +4,7 @@
In order of importance:
- If your change involves how Tranquil implements atproto make sure its correct! See more below.
- **You must run your change! Every contribution that says "here's xyz. untested." does not help the project.**
- Relevant tests to your PR must pass. The whole suite doesn't have to be proven to have run, because there are a *ton* of tests and they're quite heavy, but hopefully there are existing tests for whatever you're PRing, and if there aren't, please add those too.
- Run cargo fmt :P
@@ -16,6 +17,60 @@ Things that would also be nice but aren't like, a pain in our side:
- Big changes should be stacked PRs that are broken up into digestible pieces. Those stacked PRs should hopefully be able to be merged individually if necessary.
### How we define a "correct" PDS implementation
The atproto specs are notoriously imprecise, ambiguious,
lacks specifications for large parts of the protocol and network (even including what implementing a PDS entails!)
and is generally none specific.
This is bad.
We won't waste time here describing all the ways in which that is problematic,
the important thing for Tranquil is that this means that "follows spec" is not sufficient to describe a "correct" PDS implementation.
Thus we need to come up with a description of "correct".
In order of importance the following rules describe what "correct" means for Tranquil:
- The specs take precedence.
If the spec *is* specific enough then follow it.
Even if the reference implementation doesn't.
- If the specs aren't sufficiently specific
rely on the reference implementation, potential supporting documents or discussions,
and/or community sentiment or common sense.
If the matter is still debated and/or PBCs opinion differs from community sentiment we generally side with the community.
- Examples here include what features and APIs to implement,
here we look at what the reference implementation implements
as well as https://github.com/bluesky-social/atproto/discussions/2350 as a supporting document.
Another example is whether `include` scopes are allowed to use a `*` `aud` parameter.
Discussion here has happened in https://github.com/bluesky-social/atproto/issues/4490.
PBC has voiced an opinion that this should be disallowed,
community sentiment seems to strongly lean to allowing it. Tranquil allows it.
- Please mark locations like this with a `// SPECAMB: ...` comment explaining the ambiguity
and what parts of the reference implementation and/or supporting documents have been used as reference.
- If the reference implementation has behaviour that is only ever relevant for the Bluesky application.
Implementions of such behaviour **must** be gated behind a `bsky-support` cargo feature of the implementing crate.
- Examples here include bluesky feedgen specific service proxying behaviour,
the `app.bsky.actor.getPreferences` and `app.bsky.actor.putPreferences` APIs,
and special handling of the `X-BSKY-TOPICS` HTTP header during service proxying.
- Please add a comment next to these implementations with an explanation of the behaviour.
- Most of these behaviours are required for proper functioning of the official Bluesky client, though not all.
If the behaviour isn't required for the official client consider not implementing it.
- One such behaviour that we have a *hard rule* to never implement is default proxying to a configured Bluesky appview
for `app.bsky.*` APIs and as fallback for `com.atproto.repo.getRecord`.
Many third-party Bluesky clients rely on this behaviour, the official client used to do the same but does not anymore.
Third-party clients breaking because they don't specify an `atproto-proxy` header is thus *not* a Tranquil bug but a bug in said clients.
- Bluesky is the only application that will ever recieve application specific behaviour like this.
It does so only because such a big section of atproto usage is Bluesky
and because Bluesky is the only application that can practically rely on application specific behaviour.
Application specific behaviour for other applications may still be added to Tranquil if such behaviour is a Tranquil feature,
for example for Tranquils rudimentary banned content moderation feature,
and not something said application relies on for proper functioning.
There is bound to be edge cases that these rules don't fully cover.
Here common sense, community sentiment, furthering the goals of atproto itself, and ultimately maintainer opinion take precedence over support for any individual applicaion.
Even Bluesky.
The rules above are meant to capture Tranquils goals of being correct while being community oriented and avoiding as much "Bluesky-defaultism" as possible.
Tranquil is a community atproto PDS, *not* a company-led Bluesky (or other atproto app) PDS.
See also "Tranquil & the world" in docs/1_WELCOME_TO_TRANQUIL_PDS.md.
## Local Development
### Prerequisites
Generated
+22 -75
View File
@@ -2486,7 +2486,6 @@ checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c"
dependencies = [
"crc32fast",
"miniz_oxide",
"zlib-rs",
]
[[package]]
@@ -7630,9 +7629,8 @@ dependencies = [
[[package]]
name = "tranquil-api"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"anyhow",
"axum",
"backon",
"base32",
@@ -7666,22 +7664,19 @@ dependencies = [
"tokio",
"tracing",
"tranquil-config",
"tranquil-db",
"tranquil-db-traits",
"tranquil-lexicon",
"tranquil-pds",
"tranquil-scopes",
"tranquil-signal",
"tranquil-types",
"urlencoding",
"uuid",
"webauthn-rs",
"zip",
]
[[package]]
name = "tranquil-auth"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"anyhow",
"base32",
@@ -7705,7 +7700,7 @@ dependencies = [
[[package]]
name = "tranquil-cache"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7720,7 +7715,7 @@ dependencies = [
[[package]]
name = "tranquil-comms"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7734,7 +7729,6 @@ dependencies = [
"rsa",
"secrecy",
"serde_json",
"sqlx",
"thiserror 2.0.18",
"tokio",
"tracing",
@@ -7746,15 +7740,14 @@ dependencies = [
[[package]]
name = "tranquil-config"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"confique",
"serde",
]
[[package]]
name = "tranquil-crypto"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"aes-gcm",
"base64 0.22.1",
@@ -7770,7 +7763,7 @@ dependencies = [
[[package]]
name = "tranquil-db"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"chrono",
@@ -7787,7 +7780,7 @@ dependencies = [
[[package]]
name = "tranquil-db-traits"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7803,18 +7796,17 @@ dependencies = [
[[package]]
name = "tranquil-infra"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"bytes",
"futures",
"thiserror 2.0.18",
"tranquil-config",
]
[[package]]
name = "tranquil-lexicon"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"chrono",
"futures",
@@ -7834,7 +7826,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"anyhow",
"axum",
@@ -7857,7 +7849,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth-server"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"axum",
@@ -7892,7 +7884,7 @@ dependencies = [
[[package]]
name = "tranquil-pds"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"aes-gcm",
"anyhow",
@@ -7904,7 +7896,6 @@ dependencies = [
"base32",
"base64 0.22.1",
"bcrypt",
"bs58",
"bytes",
"chrono",
"ciborium",
@@ -7920,7 +7911,6 @@ dependencies = [
"hmac",
"http 1.4.0",
"image",
"infer",
"ipld-core",
"iroh-car",
"jacquard-common",
@@ -7962,7 +7952,6 @@ dependencies = [
"tranquil-cache",
"tranquil-comms",
"tranquil-config",
"tranquil-crypto",
"tranquil-db",
"tranquil-db-traits",
"tranquil-lexicon",
@@ -7981,12 +7970,11 @@ dependencies = [
"webauthn-rs",
"webauthn-rs-proto",
"wiremock",
"zip",
]
[[package]]
name = "tranquil-repo"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"bytes",
"cid",
@@ -7998,7 +7986,7 @@ dependencies = [
[[package]]
name = "tranquil-ripple"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"backon",
@@ -8027,7 +8015,7 @@ dependencies = [
[[package]]
name = "tranquil-scopes"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"axum",
"futures",
@@ -8044,7 +8032,7 @@ dependencies = [
[[package]]
name = "tranquil-server"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"arc-swap",
"axum",
@@ -8081,7 +8069,7 @@ dependencies = [
[[package]]
name = "tranquil-signal"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"chrono",
@@ -8089,7 +8077,6 @@ dependencies = [
"futures",
"presage",
"rand 0.9.2",
"serde",
"serde_json",
"sqlx",
"tempfile",
@@ -8097,14 +8084,13 @@ dependencies = [
"tokio",
"tokio-util",
"tracing",
"tranquil-signal",
"url",
"uuid",
]
[[package]]
name = "tranquil-storage"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"aws-config",
@@ -8121,7 +8107,7 @@ dependencies = [
[[package]]
name = "tranquil-store"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"async-trait",
"bytes",
@@ -8161,7 +8147,6 @@ dependencies = [
"tranquil-db",
"tranquil-db-traits",
"tranquil-oauth",
"tranquil-repo",
"tranquil-store",
"tranquil-types",
"uuid",
@@ -8170,7 +8155,7 @@ dependencies = [
[[package]]
name = "tranquil-sync"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"anyhow",
"axum",
@@ -8192,7 +8177,7 @@ dependencies = [
[[package]]
name = "tranquil-types"
version = "0.6.5"
version = "0.6.6"
dependencies = [
"base64 0.22.1",
"chrono",
@@ -8255,12 +8240,6 @@ version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ea3136b675547379c4bd395ca6b938e5ad3c3d20fad76e7fe85f9e0d011419c"
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.19.0"
@@ -9462,20 +9441,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "zip"
version = "7.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c42e33efc22a0650c311c2ef19115ce232583abbe80850bc8b66509ebef02de0"
dependencies = [
"crc32fast",
"flate2",
"indexmap 2.13.0",
"memchr",
"typed-path",
"zopfli",
]
[[package]]
name = "zkcredential"
version = "0.1.0"
@@ -9524,30 +9489,12 @@ dependencies = [
"zkcredential",
]
[[package]]
name = "zlib-rs"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3be3d40e40a133f9c916ee3f9f4fa2d9d63435b5fbe1bfc6d9dae0aa0ada1513"
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[package]]
name = "zune-core"
version = "0.5.1"
+1 -3
View File
@@ -26,7 +26,7 @@ members = [
]
[workspace.package]
version = "0.6.5"
version = "0.6.6"
edition = "2024"
license = "AGPL-3.0-or-later"
@@ -47,7 +47,6 @@ tranquil-db = { path = "crates/tranquil-db" }
tranquil-ripple = { path = "crates/tranquil-ripple" }
tranquil-lexicon = { path = "crates/tranquil-lexicon" }
tranquil-pds = { path = "crates/tranquil-pds" }
tranquil-server = { path = "crates/tranquil-server" }
tranquil-sync = { path = "crates/tranquil-sync" }
tranquil-oauth-server = { path = "crates/tranquil-oauth-server" }
tranquil-api = { path = "crates/tranquil-api" }
@@ -142,7 +141,6 @@ uuid = { version = "1.19", features = ["v4", "v5", "v7", "fast-rng", "serde"] }
webauthn-rs = { version = "0.5", features = ["danger-allow-state-serialisation", "danger-user-presence-only-security-keys", "conditional-ui"] }
webauthn-rs-proto = "0.5"
x509-parser = "0.18"
zip = { version = "7.0", default-features = false, features = ["deflate"] }
ciborium = "0.2"
ctor = "0.6"
+4 -4
View File
@@ -8,13 +8,10 @@ license.workspace = true
tranquil-pds = { workspace = true }
tranquil-types = { workspace = true }
tranquil-config = { workspace = true }
tranquil-db = { workspace = true }
tranquil-db-traits = { workspace = true }
tranquil-lexicon = { workspace = true, features = ["resolve"] }
tranquil-scopes = { workspace = true }
tranquil-signal = { workspace = true }
anyhow = { workspace = true }
axum = { workspace = true }
backon = { workspace = true }
base32 = { workspace = true }
@@ -50,4 +47,7 @@ tracing = { workspace = true }
urlencoding = { workspace = true }
uuid = { workspace = true }
webauthn-rs = { workspace = true }
zip = { workspace = true }
[features]
bsky = ["bsky-support"]
bsky-support = []
-3
View File
@@ -1,3 +0,0 @@
mod preferences;
pub use preferences::{get_preferences, put_preferences};
+17 -13
View File
@@ -205,20 +205,24 @@ pub async fn sequence_new_account(
{
tracing::warn!("Failed to sequence sync event for {}: {}", did, e);
}
let profile_record = serde_json::json!({
"$type": "app.bsky.actor.profile",
"displayName": display_name
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
state,
did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
// TODO: make this configurable and also deduplicate with tranquil-oauth-server/src/sso_endpoints.rs:1210
#[cfg(feature = "bsky")]
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
let profile_record = serde_json::json!({
"$type": "app.bsky.actor.profile",
"displayName": display_name
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
state,
did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
};
}
}
+26 -11
View File
@@ -1,5 +1,12 @@
pub mod actor;
// BSKY: Bluesky requires PDSs to implement its private preferences API
#[cfg(feature = "bsky-support")]
pub mod actor {
mod preferences;
pub use preferences::{get_preferences, put_preferences};
}
pub mod admin;
#[cfg(feature = "bsky")]
pub mod age_assurance;
pub mod common;
pub mod delegation;
@@ -21,7 +28,7 @@ pub fn api_routes() -> axum::Router<AppState> {
let blob_body_limit =
DefaultBodyLimit::max(tranquil_config::get().server.max_blob_size as usize);
axum::Router::new()
let router = axum::Router::new()
.route("/_health", get(server::health))
.route(
"/com.atproto.server.describeServer",
@@ -373,14 +380,6 @@ pub fn api_routes() -> axum::Router<AppState> {
post(admin::update_subject_status),
)
.route("/com.atproto.admin.sendEmail", post(admin::send_email))
.route(
"/app.bsky.actor.getPreferences",
get(actor::get_preferences),
)
.route(
"/app.bsky.actor.putPreferences",
post(actor::put_preferences),
)
.route(
"/com.atproto.temp.checkSignupQueue",
get(temp::check_signup_queue),
@@ -438,7 +437,21 @@ pub fn api_routes() -> axum::Router<AppState> {
.route(
"/_delegation.resolveController",
get(delegation::resolve_controller),
);
#[cfg(feature = "bsky-support")]
let router = router
.route(
"/app.bsky.actor.getPreferences",
get(actor::get_preferences),
)
.route(
"/app.bsky.actor.putPreferences",
post(actor::put_preferences),
);
#[cfg(feature = "bsky")]
let router = router
.route(
"/app.bsky.ageassurance.getState",
get(age_assurance::get_state),
@@ -446,7 +459,9 @@ pub fn api_routes() -> axum::Router<AppState> {
.route(
"/app.bsky.unspecced.getAgeAssuranceState",
get(age_assurance::get_age_assurance_state),
)
);
router
}
pub fn well_known_api_routes() -> axum::Router<AppState> {
@@ -1 +0,0 @@
pub use tranquil_pds::repo_ops::*;
-31
View File
@@ -1,31 +0,0 @@
use axum::{
Json,
extract::State,
response::{IntoResponse, Response},
};
use serde::Deserialize;
use tranquil_pds::api::SuccessResponse;
use tranquil_pds::state::AppState;
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct ConfirmChannelVerificationInput {
pub channel: tranquil_db_traits::CommsChannel,
pub identifier: String,
pub code: String,
}
pub async fn confirm_channel_verification(
State(state): State<AppState>,
Json(input): Json<ConfirmChannelVerificationInput>,
) -> Response {
let token_input = crate::server::VerifyTokenInput {
token: input.code,
identifier: input.identifier,
};
match crate::server::verify_token_internal(&state, token_input).await {
Ok(_output) => SuccessResponse::ok().into_response(),
Err(e) => e.into_response(),
}
}
-1
View File
@@ -5,7 +5,6 @@ edition.workspace = true
license.workspace = true
[features]
default = []
valkey = ["dep:redis"]
[dependencies]
-1
View File
@@ -19,7 +19,6 @@ reqwest = { workspace = true }
rsa = { workspace = true }
secrecy = { workspace = true }
serde_json = { workspace = true }
sqlx = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }
-1
View File
@@ -6,4 +6,3 @@ license.workspace = true
[dependencies]
confique = { workspace = true }
serde = { workspace = true }
-2
View File
@@ -5,9 +5,7 @@ edition.workspace = true
license.workspace = true
[features]
default = ["postgres"]
postgres = []
sqlite = []
[dependencies]
tranquil-db-traits = { workspace = true }
-2
View File
@@ -5,8 +5,6 @@ edition.workspace = true
license.workspace = true
[dependencies]
tranquil-config = { workspace = true }
async-trait = { workspace = true }
bytes = { workspace = true }
futures = { workspace = true }
-1
View File
@@ -5,7 +5,6 @@ edition.workspace = true
license.workspace = true
[features]
default = []
resolve = ["dep:reqwest", "dep:hickory-resolver", "dep:tokio", "dep:parking_lot", "dep:tracing", "dep:urlencoding"]
[dependencies]
+3
View File
@@ -37,3 +37,6 @@ webauthn-rs = { workspace = true }
[dev-dependencies]
async-trait = { workspace = true }
[features]
bsky = []
@@ -1209,20 +1209,23 @@ pub async fn complete_registration(
tracing::warn!("Failed to sequence account event for {}: {}", did, e);
}
let profile_record = json!({
"$type": "app.bsky.actor.profile",
"displayName": handle.as_str()
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
&state,
&did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
#[cfg(feature = "bsky")]
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
let profile_record = json!({
"$type": "app.bsky.actor.profile",
"displayName": handle.as_str()
});
if let Err(e) = tranquil_pds::repo_ops::create_record_internal(
&state,
&did,
&tranquil_pds::types::PROFILE_COLLECTION,
&tranquil_pds::types::PROFILE_RKEY,
&profile_record,
)
.await
{
tracing::warn!("Failed to create default profile for {}: {}", did, e);
};
}
let app_password = generate_app_password();
+10 -13
View File
@@ -7,7 +7,6 @@ license.workspace = true
[dependencies]
tranquil-types = { workspace = true }
tranquil-config = { workspace = true }
tranquil-crypto = { workspace = true }
tranquil-storage = { workspace = true }
tranquil-cache = { workspace = true }
tranquil-repo = { workspace = true }
@@ -29,13 +28,11 @@ axum = { workspace = true }
base32 = { workspace = true }
base64 = { workspace = true }
bcrypt = { workspace = true }
bs58 = { workspace = true }
bytes = { workspace = true }
chrono = { workspace = true }
cid = { workspace = true }
ed25519-dalek = { workspace = true }
futures = { workspace = true }
hex = { workspace = true }
futures-util = { workspace = true }
governor = { workspace = true }
hickory-resolver = { workspace = true }
@@ -43,7 +40,6 @@ hkdf = { workspace = true }
hmac = { workspace = true }
http = { workspace = true }
image = { workspace = true }
infer = { workspace = true }
ipld-core = { workspace = true }
iroh-car = { workspace = true }
jacquard-common = { workspace = true }
@@ -57,7 +53,6 @@ multihash = { workspace = true }
p256 = { workspace = true }
parking_lot = { workspace = true }
rand = { workspace = true }
redis = { workspace = true, optional = true }
regex = { workspace = true }
reqwest = { workspace = true }
serde = { workspace = true }
@@ -70,7 +65,6 @@ sqlx = { workspace = true }
subtle = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tokio-tungstenite = { workspace = true }
tokio-util = { workspace = true }
tower = { workspace = true }
tower-http = { workspace = true }
@@ -80,16 +74,14 @@ urlencoding = { workspace = true }
uuid = { workspace = true }
webauthn-rs = { workspace = true }
webauthn-rs-proto = { workspace = true }
zip = { workspace = true }
aws-config = { workspace = true, optional = true }
aws-sdk-s3 = { workspace = true, optional = true }
[features]
default = ["frontend", "s3", "valkey"]
bsky = ["bsky-support"]
bsky-support = []
external-infra = []
s3-storage = ["tranquil-storage/s3", "dep:aws-config", "dep:aws-sdk-s3"]
s3 = ["s3-storage"]
valkey = ["tranquil-cache/valkey", "dep:redis"]
postgres = ["tranquil-db/postgres"]
s3 = ["tranquil-storage/s3"]
valkey = ["tranquil-cache/valkey"]
frontend = []
native-tls-roots = ["tranquil-oauth/native-tls-roots"]
@@ -105,3 +97,8 @@ tranquil-api = { workspace = true }
tranquil-oauth-server = { workspace = true }
tracing-subscriber = { workspace = true, features = ["env-filter"] }
wiremock = { workspace = true }
hex = { workspace = true }
tokio-tungstenite = { workspace = true }
aws-config = { workspace = true }
aws-sdk-s3 = { workspace = true }
redis = { workspace = true }
+30 -23
View File
@@ -19,9 +19,7 @@ use tower::{Service, util::BoxCloneSyncService};
use tracing::{error, info, warn};
static PROTECTED_METHODS: LazyLock<HashSet<&'static str>> = LazyLock::new(|| {
[
"app.bsky.actor.getPreferences",
"app.bsky.actor.putPreferences",
let mut methods: HashSet<&str> = [
"com.atproto.admin.deleteAccount",
"com.atproto.admin.disableAccountInvites",
"com.atproto.admin.disableInviteCodes",
@@ -103,7 +101,13 @@ static PROTECTED_METHODS: LazyLock<HashSet<&'static str>> = LazyLock::new(|| {
"com.atproto.temp.dereferenceScope",
]
.into_iter()
.collect()
.collect();
// BSKY: the Bluesky preferences API must be implemented by PDSs
if cfg!(feature = "bsky-support") {
methods.insert("app.bsky.actor.getPreferences");
methods.insert("app.bsky.actor.putPreferences");
};
methods
});
fn is_protected_method(method: &str) -> bool {
@@ -111,6 +115,7 @@ fn is_protected_method(method: &str) -> bool {
}
/// Fetch the `feed` generator record from the AppView and return its `did`.
#[cfg(feature = "bsky-support")]
async fn resolve_feed_generator_did(appview_url: &str, query: Option<&str>) -> Option<Did> {
#[derive(serde::Deserialize)]
struct GetFeedQuery {
@@ -201,6 +206,7 @@ impl<S: Service<Request, Response = Response, Error = Infallible>> Service<Reque
}
// If the age assurance override is set and this is an age assurance call then we dont want to proxy even if the client requests it
#[cfg(feature = "bsky")]
if tranquil_config::get().server.age_assurance_override
&& (path.ends_with("app.bsky.ageassurance.getState")
|| path.ends_with("app.bsky.unspecced.getAgeAssuranceState"))
@@ -328,27 +334,28 @@ async fn proxy_handler(
},
};
// getFeed must be audienced to the feed generator, not the AppView.
let (token_aud, token_lxm) = if method == "app.bsky.feed.getFeed" {
match resolve_feed_generator_did(&resolved.url, query.as_deref()).await {
Some(feed_did) => (
feed_did,
"app.bsky.feed.getFeedSkeleton"
.parse::<Nsid>()
.expect("getFeedSkeleton is a valid NSID"),
),
None => {
warn!(
"getFeed proxy: could not resolve feed generator DID; refusing \
// BSKY: getFeed must be audienced to the feed generator, not the AppView.
let (token_aud, token_lxm) =
if cfg!(feature = "bsky-support") && method == "app.bsky.feed.getFeed" {
match resolve_feed_generator_did(&resolved.url, query.as_deref()).await {
Some(feed_did) => (
feed_did,
"app.bsky.feed.getFeedSkeleton"
.parse::<Nsid>()
.expect("getFeedSkeleton is a valid NSID"),
),
None => {
warn!(
"getFeed proxy: could not resolve feed generator DID; refusing \
to mint an AppView-audienced token"
);
return ApiError::InvalidRequest("Could not resolve feed".into())
.into_response();
);
return ApiError::InvalidRequest("Could not resolve feed".into())
.into_response();
}
}
}
} else {
(resolved.did.clone(), method_nsid.clone())
};
} else {
(resolved.did.clone(), method_nsid.clone())
};
match crate::auth::create_service_token(
&auth_user.did,
+11 -1
View File
@@ -147,7 +147,9 @@ impl std::fmt::Display for SsrfError {
}
impl std::error::Error for SsrfError {}
// TODO: update to match https://github.com/bluesky-social/atproto/blob/main/packages/pds/src/pipethrough.ts
// Currently spec says nothing at all!! about forwarding headers during proxying and https://github.com/bluesky-social/atproto/discussions/2350
#[cfg(feature = "bsky-support")]
pub static HEADERS_TO_FORWARD: LazyLock<[HeaderName; 4]> = LazyLock::new(|| {
[
HeaderName::from_static("accept-language"),
@@ -156,6 +158,14 @@ pub static HEADERS_TO_FORWARD: LazyLock<[HeaderName; 4]> = LazyLock::new(|| {
http::header::CONTENT_TYPE,
]
});
#[cfg(not(feature = "bsky-support"))]
pub static HEADERS_TO_FORWARD: LazyLock<[HeaderName; 3]> = LazyLock::new(|| {
[
HeaderName::from_static("accept-language"),
crate::util::HEADER_ATPROTO_ACCEPT_LABELERS,
http::header::CONTENT_TYPE,
]
});
pub static RESPONSE_HEADERS_TO_FORWARD: LazyLock<[HeaderName; 6]> = LazyLock::new(|| {
[
crate::util::HEADER_ATPROTO_REPO_REV,
@@ -1,6 +1,9 @@
use std::collections::HashSet;
use std::sync::LazyLock;
// TODO: make all of this configurable.
// PDS implementation should not impose any reserved domains imo.
// some of these are even bad to have as defaults let alone non-configurables
const ATP_SPECIFIC: &[&str] = &[
"at", "atp", "plc", "pds", "did", "repo", "tid", "nsid", "xrpc", "lex", "lexicon", "bsky",
"bluesky", "handle",
+4
View File
@@ -115,6 +115,10 @@ pub fn extract_blob_cids(record: &Value) -> Vec<crate::types::CidLink> {
use crate::types::AtUri;
use tranquil_db_traits::{Backlink, BacklinkPath};
// TODO: it really really should not be necessary to extract backlinks and store those.
// especially not in a way that isnt generic.
// figure out what the fuck is going on here
// (lewis do you remember why you did this???)
pub fn extract_backlinks(uri: &AtUri, record: &Value) -> Vec<Backlink> {
let record_type = record
.get("$type")
+1 -1
View File
@@ -2,5 +2,5 @@ pub use tranquil_storage::{
BlobStorage, FilesystemBlobStorage, StorageError, StreamUploadResult, create_blob_storage,
};
#[cfg(feature = "s3-storage")]
#[cfg(feature = "s3")]
pub use tranquil_storage::S3BlobStorage;
+2
View File
@@ -2,6 +2,8 @@ pub use tranquil_types::*;
use std::sync::LazyLock;
#[cfg(feature = "bsky")]
pub static PROFILE_COLLECTION: LazyLock<Nsid> =
LazyLock::new(|| "app.bsky.actor.profile".parse().unwrap());
#[cfg(feature = "bsky")]
pub static PROFILE_RKEY: LazyLock<Rkey> = LazyLock::new(|| "self".parse().unwrap());
+8 -1
View File
@@ -87,6 +87,7 @@ pub const HEADER_ATPROTO_ACCEPT_LABELERS: HeaderName =
pub const HEADER_ATPROTO_REPO_REV: HeaderName = HeaderName::from_static("atproto-repo-rev");
pub const HEADER_ATPROTO_CONTENT_LABELERS: HeaderName =
HeaderName::from_static("atproto-content-labelers");
#[cfg(feature = "bsky-support")]
pub const HEADER_X_BSKY_TOPICS: HeaderName = HeaderName::from_static("x-bsky-topics");
pub fn get_header_str(
@@ -247,7 +248,13 @@ pub fn build_full_url(path: &str) -> String {
let cfg = tranquil_config::get();
let normalized_path = if !path.starts_with("/xrpc/")
&& (path.starts_with("/com.atproto.")
|| path.starts_with("/app.bsky.")
// BSKY: Bluesky requires that the PDS implement some app.bsky.* endpoints so we need to deal with those here too.
// TODO: surely we can figure out a way to do this more generically?
|| (if cfg!(feature = "bsky-support") {
path.starts_with("/app.bsky.")
} else {
true
})
|| path.starts_with("/_"))
{
format!("/xrpc{path}")
@@ -138,20 +138,25 @@ fn check_banned_content(
rkey: Option<&Rkey>,
) -> Result<(), ValidationError> {
match record_type {
#[cfg(feature = "bsky")]
"app.bsky.feed.post" => {
check_post_banned_content(obj)?;
}
#[cfg(feature = "bsky")]
"app.bsky.actor.profile" => {
check_string_field(obj, "displayName")?;
check_string_field(obj, "description")?;
}
#[cfg(feature = "bsky")]
"app.bsky.graph.list" => {
check_string_field(obj, "name")?;
}
#[cfg(feature = "bsky")]
"app.bsky.graph.starterpack" => {
check_string_field(obj, "name")?;
check_string_field(obj, "description")?;
}
#[cfg(feature = "bsky")]
"app.bsky.feed.generator" => {
if let Some(rkey) = rkey
&& crate::moderation::has_explicit_slur(rkey)
@@ -167,6 +172,7 @@ fn check_banned_content(
Ok(())
}
#[cfg(feature = "bsky")]
fn check_post_banned_content(obj: &serde_json::Map<String, Value>) -> Result<(), ValidationError> {
if let Some(tags) = obj.get("tags").and_then(|v| v.as_array()) {
tags.iter().enumerate().try_for_each(|(i, tag)| {
+8 -8
View File
@@ -1,8 +1,8 @@
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
use aws_config::BehaviorVersion;
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
use aws_sdk_s3::Client as S3Client;
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
use aws_sdk_s3::config::Credentials;
use chrono::Utc;
use reqwest::{Client, StatusCode, header};
@@ -56,9 +56,9 @@ pub struct ServerInstance {
pub distributed_rate_limiter: Option<Arc<dyn DistributedRateLimiter>>,
}
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
use testcontainers::GenericImage;
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
use testcontainers::core::ContainerPort;
#[cfg(not(feature = "external-infra"))]
use testcontainers::{ContainerAsync, ImageExt, runners::AsyncRunner};
@@ -66,7 +66,7 @@ use testcontainers::{ContainerAsync, ImageExt, runners::AsyncRunner};
use testcontainers_modules::postgres::Postgres;
#[cfg(not(feature = "external-infra"))]
static DB_CONTAINER: OnceLock<ContainerAsync<Postgres>> = OnceLock::new();
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
static S3_CONTAINER: OnceLock<ContainerAsync<GenericImage>> = OnceLock::new();
#[allow(dead_code)]
@@ -195,7 +195,7 @@ async fn setup_with_external_infra() -> String {
spawn_app(database_url).await
}
#[cfg(all(not(feature = "external-infra"), not(feature = "s3-storage")))]
#[cfg(all(not(feature = "external-infra"), not(feature = "s3")))]
async fn setup_with_testcontainers() -> String {
let temp_dir = std::env::temp_dir().join(format!("tranquil-pds-test-{}", uuid::Uuid::new_v4()));
let blob_path = temp_dir.join("blobs");
@@ -227,7 +227,7 @@ async fn setup_with_testcontainers() -> String {
spawn_app(connection_string).await
}
#[cfg(all(not(feature = "external-infra"), feature = "s3-storage"))]
#[cfg(all(not(feature = "external-infra"), feature = "s3"))]
async fn setup_with_testcontainers() -> String {
let s3_container = GenericImage::new("cgr.dev/chainguard/minio", "latest")
.with_exposed_port(ContainerPort::Tcp(9000))
+4 -1
View File
@@ -41,8 +41,11 @@ tracing-subscriber = { workspace = true }
rcgen = { workspace = true }
[features]
default = ["frontend", "s3", "valkey"]
default = ["bsky", "frontend", "postgres", "s3", "valkey"]
bsky = ["bsky-support", "tranquil-pds/bsky", "tranquil-api/bsky"]
bsky-support = ["tranquil-pds/bsky-support", "tranquil-api/bsky-support"]
frontend = ["tranquil-pds/frontend"]
postgres = ["tranquil-pds/postgres"]
s3 = ["tranquil-pds/s3"]
valkey = ["tranquil-pds/valkey"]
native-tls-roots = ["tranquil-pds/native-tls-roots"]
-2
View File
@@ -17,13 +17,11 @@ tracing = { workspace = true }
tokio = { workspace = true }
tokio-util = { workspace = true }
futures = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
url = "2.5"
uuid = { workspace = true }
thiserror = { workspace = true }
[dev-dependencies]
tranquil-signal = { path = ".", features = ["fjall-store"] }
rand = "0.9"
tempfile = "3"
-1
View File
@@ -5,7 +5,6 @@ edition.workspace = true
license.workspace = true
[features]
default = []
s3 = ["dep:aws-config", "dep:aws-sdk-s3"]
[dependencies]
+1 -2
View File
@@ -59,8 +59,7 @@ tempfile = "3"
futures = { workspace = true }
tokio = { workspace = true, features = ["sync", "rt-multi-thread", "macros", "time"] }
jacquard-common = { workspace = true }
tranquil-repo = { workspace = true }
tranquil-db = { workspace = true }
tranquil-db = { workspace = true, features = [ "postgres" ] }
sqlx = { workspace = true }
k256 = { workspace = true }
rand = { workspace = true }
+1 -5
View File
@@ -4,10 +4,6 @@ version.workspace = true
edition.workspace = true
license.workspace = true
[features]
default = ["sqlx"]
sqlx = ["dep:sqlx"]
[dependencies]
base64 = { workspace = true }
chrono = { workspace = true }
@@ -16,6 +12,6 @@ jacquard-common = { workspace = true }
rand = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
sqlx = { workspace = true, optional = true }
sqlx = { workspace = true }
thiserror = { workspace = true }
uuid = { workspace = true }
+14 -22
View File
@@ -92,9 +92,9 @@ macro_rules! simple_string_newtype {
$vis:vis struct $name:ident;
) => {
$(#[$meta])*
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
#[cfg_attr(feature = "sqlx", derive(sqlx::Type), sqlx(transparent))]
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize, sqlx::Type)]
#[serde(transparent)]
#[sqlx(transparent)]
$vis struct $name(String);
impl $name {
@@ -148,9 +148,9 @@ macro_rules! validated_string_newtype {
validator = $validator:expr;
) => {
$(#[$meta])*
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize)]
#[cfg_attr(feature = "sqlx", derive(sqlx::Type), sqlx(transparent))]
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, sqlx::Type)]
#[serde(transparent)]
#[sqlx(transparent)]
$vis struct $name(String);
impl<'de> Deserialize<'de> for $name {
@@ -223,9 +223,9 @@ impl Did {
}
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize)]
#[cfg_attr(feature = "sqlx", derive(sqlx::Type), sqlx(transparent))]
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, sqlx::Type)]
#[serde(transparent)]
#[sqlx(transparent)]
pub struct Handle(String);
impl<'de> Deserialize<'de> for Handle {
@@ -523,9 +523,9 @@ validated_string_newtype! {
validator = |s| jacquard_common::types::string::Language::from_str(s).map(|v| v.as_str().to_owned()).map_err(|_| ());
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize)]
#[cfg_attr(feature = "sqlx", derive(sqlx::Type), sqlx(transparent))]
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, sqlx::Type)]
#[serde(transparent)]
#[sqlx(transparent)]
pub struct CidLink(String);
impl<'de> Deserialize<'de> for CidLink {
@@ -718,9 +718,9 @@ impl Deref for PlainPassword {
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[cfg_attr(feature = "sqlx", derive(sqlx::Type), sqlx(transparent))]
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, sqlx::Type)]
#[serde(transparent)]
#[sqlx(transparent)]
pub struct PasswordHash(String);
impl PasswordHash {
@@ -843,13 +843,9 @@ simple_string_newtype! {
pub struct InviteCode;
}
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[cfg_attr(
feature = "sqlx",
derive(sqlx::Type),
sqlx(type_name = "comms_channel", rename_all = "snake_case")
)]
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, sqlx::Type)]
#[serde(rename_all = "snake_case")]
#[sqlx(type_name = "comms_channel", rename_all = "snake_case")]
#[derive(Copy)]
pub enum CommsChannel {
Email,
@@ -885,13 +881,9 @@ impl fmt::Display for CommsChannel {
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[cfg_attr(
feature = "sqlx",
derive(sqlx::Type),
sqlx(type_name = "comms_type", rename_all = "snake_case")
)]
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, sqlx::Type)]
#[serde(rename_all = "snake_case")]
#[sqlx(type_name = "comms_type", rename_all = "snake_case")]
pub enum CommsType {
Verification,
PasswordReset,