Compare commits

...
Author SHA1 Message Date
nelind 75c341b94b chore: bump version to 0.6.7 2026-09-20 22:29:26 +02:00
nelind 0b29c40b27 docs: list jola.dev as a core team maintainer 2026-09-20 18:51:59 +02:00
Anastasia CalicoandTangled 34ece34296 feat: Implement s3_path in storage config. 2026-09-20 16:47:22 +00:00
Johanna LarssonandTangled 833356116a Set inputmode on inputs
This improves mobile behavior for the inputs without affecting other things.

Two inputs set to email, rest to URL pre recommendation in https://tangled.org/tranquil.farm/tranquil-pds/issues/140

Also disables autocapitalize, autocorrect, and spellcheck, while we're at it. All really annoying when typing in handles on a mobile phone.
2026-09-19 14:10:15 +00:00
Johanna LarssonandTangled 643e1bb902 Make ownership repair job report failures more loudly 2026-09-17 12:35:55 +00:00
Johanna LarssonandTangled 9f05ea5f31 Introduce job to repair missing blobs
My last PR introduced a migration to blobs, changing them from being unique per cid to being unique per cid+user. This is because two users can upload the same blob, get the same cid, and then previously in tranquil the second user would never actually get their ownership recorded, meaning listBlobs was missing the blob, and a migration _off_ of tranquil would leave them sans blob https://tangled.org/tranquil.farm/tranquil-pds/pulls/261/round/3

This PR attempts to add a job to repair existing instances where blobs have been deduplicated and not attributed to each owning user. It does this by pulling all records for each user, and then in batches crawling the records to find blob references. For each blob reference, it checks whether that user has its ownership recorded, and if not, repairs it by adding the entry.

To avoid re-running this potentially heavy job on every startup, I've used the config table to store a little record of whether it has been run.
2026-09-17 12:35:55 +00:00
TrezyandTangled 71cd282d1e feat: show repo actions narrowed out by client metadata on consent
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 0e40bdca19 fix: store filtered scopes on tokens so refresh can skip the client metadata check
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 156066fe1b fix: reject PAR requests missing the atproto scope
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 311530a9a9 test: give the mock client a scope so NotRegistered actually gets exercised
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 08cd3fa100 chore: remove unreachable scope format check in consent
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
TrezyandTangled 12a8712eae fix: unrecognized scopes can no longer fail PAR
Signed-off-by: Trezy <tre@trezy.com>
2026-09-16 16:20:17 +00:00
LunaandTangled cdd5fa70c9 fix: add MIME version to outgoing mail 2026-09-14 13:40:08 +00:00
xan.lolandTangled 218741050d add a period (spelling fix) 2026-09-13 22:47:12 +00:00
Johanna LarssonandTangled b3ff62c221 Simplify blob migration 2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 2088f59197 Stop deleting logo blob when switching
This also means we can clean up the last blob operations that were per cid rather than user+cid 🪓

We should be setting ourselves up to be able to go garbage collect blobs safely so deleting the logo blobs manually won't matter anyway.
2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 2fc5f2e308 Wrap in counter lock and stop deleting blobs by cid 2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 877b587481 Ensure listing missing blobs always checks per user blobs 2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 695a7d981c Allow blobs to be owned by multiple accounts
The current tranquil database design only allows each blob to be owned by one account. This means that if a second account also has that blob, tranquil skips associated the blob with the account. That works fine a lot of the time, since blobs are looked up by cid and the blob exists. However, it can lead to loss of data under certain scenarios.

One example is where I upload a blob, the blob already exists in my instance so insertion is skipped (postgres requires cid to be unique in blobs, fjall only allows one owner per blob cid), I then decide to migrate off tranquil, the blob does not come with me since it is not mine.

Another example is where an account is deleted. If a blob was uploaded for account a, then account b uploads the same blob tranquil skips storing it since it exists. Then I delete account a, now account b's blob is missing.

I accidentally stumbled upon this when I migrated my account to my own tranuil instance and list blobs now lists 2 fewer blobs than before, two images that had been uploaded by accounts already on the PDS.

ps I found record_blobs a bit confusing, at first it looked like a blob ownership table, but then it turns out to just be used for migrations!

This PR makes the blob primary key be cid+user for postgres, and updates the queries to account for there being multiple "blobs" with the same cid. For queries that just care about the blob existing, it doesn't matter "whose" blob it is, so limit 1.

Most of the work is on the metastore side. Adds ref_count to track how many are referencing the blob since we can't just check for other rows. Instead of storing blobs directly, we now store a per account cid, and the blob reference itself is shared and keyed by cid only. This means some of these operations now require updating two places, so they're done in `batch`es.

With the new layout get_blob_value becomes simpler, all blob data is a single "table" or whatever it's called, so we just grab it using cid instead of looking it up for the user.

Migrates blobs rather than maintaining two different versions of the tables, although it seems like that could be supported.

I removed a test that asserted the old behavior, and added a reasonable (?) set of new tests that assert the new behavior, including a parity test.
2026-09-12 15:34:46 +00:00
Johanna LarssonandTangled 04689cbe25 Improve logging and correct translation string 2026-09-11 19:54:37 +00:00
Johanna LarssonandTangled 3474ed588d Make legacy login alerts configurable
This adds security alerts section to the communication settings, with one new item: a toggle for enabling/disabling legacy login alerts. It's enabled by default. Disabling it means that you no longer get emails when logging in through a non-2FA enabled login flow, like createSession.

The reason I tackled this is that I have a bot account that automatically refreshes its session, using username and app password, and I'm close to having 100 of these emails in my inbox. I also tried to just disable TOTP but I wasn't allowed (I fixed that in a different PR).

English and Swedish translations were me, the rest were MyMemory. I imagine someone can improve on them after this is merged!
2026-09-11 19:54:37 +00:00
Johanna LarssonandTangled 09ba5e4521 Allow disabling TOTP even if last session was legacy
This removes the `require_legacy_session_mfa` call on the TOTP disable path. It looks at the last session, which isn't really relevant, and we're getting a fresh TOTP from the form anyway, which should be enough proof to allow disabling TOTP.

The way I bumped into this was having a "bot account" log in with app password on some app on a schdule, triggering lots of warning emails. So I tried to disable TOTP to stop the flood and wasn't allowed.
2026-09-05 18:16:56 +00:00
SteffoandTangled 6750aeccaf fix: Use "registered notification channel" where applicable 2026-09-03 13:37:09 +00:00
LewisandTangled 8c3386a3ab caddy: on-demand TLS endpoint
Lewis: May this revision serve well! <did:plc:3fwecdnvtcscjnrx2p4n7alz>
2026-09-01 16:31:10 +00:00
TrezyandTangled eba8167da8 chore: clean up supersedence shtuff
Signed-off-by: Trezy <tre@trezy.com>
2026-08-29 20:11:10 +00:00
TrezyandTangled 2e92310518 fix: allow transition:generic to be used with granular scopes
Signed-off-by: Trezy <tre@trezy.com>
2026-08-29 20:11:10 +00:00
nelindandTangled 0e82a38add fix(api): dont do rotation key validation in signPlcOperation as it blocks migrations 2026-08-29 05:42:03 +00:00
101 changed files with 3144 additions and 939 deletions
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1",
"query": "SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1 LIMIT 1",
"describe": {
"columns": [
{
@@ -30,5 +30,5 @@
false
]
},
"hash": "dd1b61d6ec81fd891d4effd3b51e6c22308b878acdc5355dfcb04c5664c9463b"
"hash": "03f129e4984e1bed9e87294adc9caf1730906d889101b9039113ec8aa234618d"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as \"total!\" FROM blobs",
"query": "SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as \"total!\"\n FROM (SELECT DISTINCT cid, size_bytes FROM blobs) t",
"describe": {
"columns": [
{
@@ -16,5 +16,5 @@
null
]
},
"hash": "0890b2c7c921005f58ed0e57b6e062b2085ce804a4cccb27b4ae2ba6711f24c4"
"hash": "155efbae4cd55f73ec0709dda7b18a76e92065e6ae4a6081bd38a19821fbfcc3"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT cid, takedown_ref FROM blobs WHERE cid = $1",
"query": "SELECT cid, takedown_ref FROM blobs WHERE cid = $1 ORDER BY takedown_ref NULLS LAST LIMIT 1",
"describe": {
"columns": [
{
@@ -24,5 +24,5 @@
true
]
},
"hash": "62942bd21d545eb15bfea4f46378b6c2ebfe12b8bc9e27c63a6c0f77a9105303"
"hash": "5996484ff0f8dbc3b278cfd01b8375dbf7bf6da8d903145b12871dda6e1fd5d9"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key as \"storage_key!\" FROM blobs b\n WHERE created_by_user = $1\n AND NOT EXISTS (\n SELECT 1 FROM blobs o\n WHERE o.cid = b.cid AND o.created_by_user <> $1\n )",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "storage_key!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "8844d942ef2810afc386e5a9838624ee07a43c380d2df31efdba5cf299aab571"
}
@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT u.id as user_id, u.did\n FROM users u\n JOIN repos r ON r.user_id = u.id\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "user_id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "did",
"type_info": "Text"
}
],
"parameters": {
"Left": []
},
"nullable": [
false,
false
]
},
"hash": "8951136879711bca5b562c34f88e691a8ee16f370f6ef9b88ddb3873ddf2b45f"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (cid) DO NOTHING RETURNING cid",
"query": "INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)\n VALUES ($1, $2, $3, $4, $5)\n ON CONFLICT (cid, created_by_user) DO NOTHING RETURNING cid",
"describe": {
"columns": [
{
@@ -22,5 +22,5 @@
false
]
},
"hash": "8afea2b745385348f4c78b51f74145d6718bfcf9a3a0c218109ec691aeb930ba"
"hash": "996e5513fb55670fe3304a6046381e377da6a187dfa3347bd285078a7b4410f2"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key FROM blobs WHERE cid = $1",
"query": "SELECT storage_key FROM blobs WHERE cid = $1 LIMIT 1",
"describe": {
"columns": [
{
@@ -18,5 +18,5 @@
false
]
},
"hash": "6131bb5b39ca81bdbb193c0a9867bead8d9f3d793ad4eca97a79d166467a5052"
"hash": "9fb9e128076b20ff067d01955221488ce7e5b886dba0529fb073c3e0461fe030"
}
@@ -1,14 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM blobs WHERE cid = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "d2990ce7f233d2489bb36a63920571c9f454a0605cc463829693d581bc0dce12"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)\n SELECT DISTINCT b.cid, b.mime_type, b.size_bytes, $1::uuid, b.storage_key\n FROM blobs b WHERE b.cid = $2\n ON CONFLICT (cid, created_by_user) DO NOTHING",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": []
},
"hash": "e248d71f595abf0207b01bc2f4e1f312d0c96b0f2f5131dfc13bfbb42a79d886"
}
@@ -1,22 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT storage_key as \"storage_key!\" FROM blobs WHERE created_by_user = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "storage_key!",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false
]
},
"hash": "f59010ecdd7f782489e0e03288a06dacd72b33d04c1e2b98475018ad25485852"
}
@@ -1,6 +1,6 @@
{
"db_name": "PostgreSQL",
"query": "SELECT rb.blob_cid, rb.record_uri\n FROM record_blobs rb\n LEFT JOIN blobs b ON rb.blob_cid = b.cid\n WHERE rb.repo_id = $1 AND b.cid IS NULL AND rb.blob_cid > $2\n ORDER BY rb.blob_cid\n LIMIT $3",
"query": "SELECT rb.blob_cid, rb.record_uri\n FROM record_blobs rb\n LEFT JOIN blobs b ON rb.blob_cid = b.cid AND b.created_by_user = $1\n WHERE rb.repo_id = $1 AND b.cid IS NULL AND rb.blob_cid > $2\n ORDER BY rb.blob_cid\n LIMIT $3",
"describe": {
"columns": [
{
@@ -26,5 +26,5 @@
false
]
},
"hash": "8e88ec169f0ef14c7207944cd4c2c0970e302c0457f9e317ec752dc13a5b1393"
"hash": "f8bb421e07e47f7b0a3b2789e368abfa6ad64152e8660e3efc7117b2d9320f22"
}
Generated
+24 -22
View File
@@ -7665,7 +7665,7 @@ dependencies = [
[[package]]
name = "tranquil-api"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"axum",
"backon",
@@ -7712,7 +7712,7 @@ dependencies = [
[[package]]
name = "tranquil-auth"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"anyhow",
"base32",
@@ -7737,7 +7737,7 @@ dependencies = [
[[package]]
name = "tranquil-cache"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7752,7 +7752,7 @@ dependencies = [
[[package]]
name = "tranquil-comms"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7777,14 +7777,16 @@ dependencies = [
[[package]]
name = "tranquil-config"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"confique",
"serde",
"tranquil-types",
]
[[package]]
name = "tranquil-crypto"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"aes-gcm",
"base64 0.22.1",
@@ -7800,7 +7802,7 @@ dependencies = [
[[package]]
name = "tranquil-db"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"chrono",
@@ -7817,7 +7819,7 @@ dependencies = [
[[package]]
name = "tranquil-db-traits"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"base64 0.22.1",
@@ -7833,7 +7835,7 @@ dependencies = [
[[package]]
name = "tranquil-infra"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"bytes",
@@ -7846,7 +7848,7 @@ dependencies = [
[[package]]
name = "tranquil-lexicon"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"chrono",
"futures",
@@ -7866,7 +7868,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"anyhow",
"axum",
@@ -7890,7 +7892,7 @@ dependencies = [
[[package]]
name = "tranquil-oauth-server"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"axum",
@@ -7926,7 +7928,7 @@ dependencies = [
[[package]]
name = "tranquil-pds"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"aes-gcm",
"anyhow",
@@ -8017,7 +8019,7 @@ dependencies = [
[[package]]
name = "tranquil-repo"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"bytes",
"cid",
@@ -8029,7 +8031,7 @@ dependencies = [
[[package]]
name = "tranquil-ripple"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"backon",
@@ -8058,7 +8060,7 @@ dependencies = [
[[package]]
name = "tranquil-scopes"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"axum",
"futures",
@@ -8075,7 +8077,7 @@ dependencies = [
[[package]]
name = "tranquil-server"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"arc-swap",
"axum",
@@ -8112,7 +8114,7 @@ dependencies = [
[[package]]
name = "tranquil-signal"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"chrono",
@@ -8133,7 +8135,7 @@ dependencies = [
[[package]]
name = "tranquil-storage"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"aws-config",
@@ -8150,7 +8152,7 @@ dependencies = [
[[package]]
name = "tranquil-store"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"async-trait",
"bytes",
@@ -8198,7 +8200,7 @@ dependencies = [
[[package]]
name = "tranquil-sync"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"anyhow",
"axum",
@@ -8220,7 +8222,7 @@ dependencies = [
[[package]]
name = "tranquil-types"
version = "0.6.6"
version = "0.6.7"
dependencies = [
"base64 0.22.1",
"chrono",
+1 -1
View File
@@ -26,7 +26,7 @@ members = [
]
[workspace.package]
version = "0.6.6"
version = "0.6.7"
edition = "2024"
license = "AGPL-3.0-or-later"
+1
View File
@@ -79,6 +79,7 @@ We currently don't have a shared space to chat and organize Tranquil things, but
- [@oyster.cafe](https://tangled.org/did:plc:3fwecdnvtcscjnrx2p4n7alz)
- [@nel.pet](https://tangled.org/did:plc:h5wsnqetncv6lu2weom35lg2)
- [@jola.dev](https://tangled.org/did:plc:bvraa6gajy4tfr3eh2sisdkr)
### Amazing contributors
@@ -66,11 +66,11 @@ pub async fn update_account_handle(
{
return Err(ApiError::InvalidHandle(None));
}
let available_domains = tranquil_config::get().server.available_user_domain_list();
let handle = if !input_handle.contains('.') {
format!("{}.{}", input_handle, &available_domains[0])
} else {
let primary = tranquil_pds::handle::ServiceDomains::for_user_handles().primary();
let handle = if input_handle.contains('.') {
input_handle.to_string()
} else {
format!("{}.{}", input_handle, primary)
};
let old_handle = state.repos.user.get_handle_by_did(did).await.ok().flatten();
let user_id = state
-42
View File
@@ -1,10 +1,8 @@
use axum::{Json, extract::State};
use serde::{Deserialize, Serialize};
use tracing::{error, warn};
use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{Admin, Auth};
use tranquil_pds::state::AppState;
use tranquil_types::CidLink;
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
@@ -183,46 +181,6 @@ pub async fn update_server_config(
}
if let Some(ref logo_cid) = req.logo_cid {
let old_logo_cid = state
.repos
.infra
.get_server_config("logo_cid")
.await
.ok()
.flatten();
let should_delete_old = match (&old_logo_cid, logo_cid.is_empty()) {
(Some(old), true) => Some(old.clone()),
(Some(old), false) if old != logo_cid => Some(old.clone()),
_ => None,
};
if let Some(old_cid_str) = should_delete_old {
match CidLink::new(old_cid_str) {
Ok(old_cid) => {
if let Ok(Some(storage_key)) = state
.repos
.infra
.get_blob_storage_key_by_cid(&old_cid)
.await
{
if let Err(e) = state.blob_store.delete(&storage_key).await {
error!("Failed to delete old logo blob from storage: {:?}", e);
}
if let Err(e) = state.repos.infra.delete_blob_by_cid(&old_cid).await {
error!("Failed to delete old logo blob record: {:?}", e);
}
}
}
Err(e) => {
warn!(
"Old logo CID in database is invalid, skipping cleanup: {:?}",
e
);
}
}
}
if logo_cid.is_empty() {
state
.repos
+13 -26
View File
@@ -132,12 +132,9 @@ pub async fn well_known_did(State(state): State<AppState>, headers: HeaderMap) -
let host_header = get_header_str(&headers, http::header::HOST).unwrap_or(hostname);
let host_without_port = host_header.split(':').next().unwrap_or(host_header);
if host_without_port != hostname_without_port {
let is_subdomain = cfg
.server
.available_user_domain_list()
.into_iter()
.chain(std::iter::once(hostname_without_port.to_string()))
.any(|d| host_without_port.ends_with(&format!(".{}", d)));
let is_subdomain = tranquil_pds::handle::ServiceDomains::served()
.split_handle(host_without_port)
.is_some();
if is_subdomain {
return serve_handle_did_doc(&state, host_without_port, hostname).await;
}
@@ -582,26 +579,16 @@ pub async fn update_handle(
"Inappropriate language in handle".into(),
)));
}
let handle_domains = tranquil_config::get().server.user_handle_domain_list();
let matched_handle_domain = handle_domains
.iter()
.filter(|d| new_handle.ends_with(&format!(".{}", d)))
.max_by_key(|d| d.len())
.cloned();
let is_domain_itself = handle_domains.iter().any(|d| d == &new_handle);
let handle: Handle = if (!new_handle.contains('.') || matched_handle_domain.is_some())
&& !is_domain_itself
{
let (short_part, full_handle) = match &matched_handle_domain {
Some(domain) => {
let suffix = format!(".{}", domain);
let short = new_handle.strip_suffix(&suffix).unwrap_or(&new_handle);
(short.to_string(), new_handle.clone())
}
None => {
let primary = &handle_domains[0];
(new_handle.clone(), format!("{}.{}", new_handle, primary))
}
let handle_domains = tranquil_pds::handle::ServiceDomains::for_user_handles();
let split = handle_domains.split_handle(&new_handle);
let is_domain_itself = handle_domains.contains(&new_handle);
let handle: Handle = if (!new_handle.contains('.') || split.is_some()) && !is_domain_itself {
let (short_part, full_handle) = match split {
Some((_domain, short)) => (short.to_string(), new_handle.clone()),
None => (
new_handle.clone(),
format!("{}.{}", new_handle, handle_domains.primary()),
),
};
if full_handle == current_handle {
let handle: Handle = match full_handle.parse() {
+1 -16
View File
@@ -9,10 +9,7 @@ use tranquil_pds::api::ApiError;
use tranquil_pds::api::error::DbResultExt;
use tranquil_pds::auth::{Auth, Permissive};
use tranquil_pds::circuit_breaker::with_circuit_breaker;
use tranquil_pds::plc::{
PlcError, PlcService, create_update_op, missing_required_rotation_key, sign_operation,
signing_key_to_did_key,
};
use tranquil_pds::plc::{PlcError, PlcService, create_update_op, sign_operation};
use tranquil_pds::state::AppState;
#[derive(Debug, Deserialize)]
@@ -118,18 +115,6 @@ pub async fn sign_plc_operation(
}
})?;
let signing_did_key = signing_key_to_did_key(&signing_key);
if let Some(rotation_keys) = unsigned_op.get("rotationKeys").and_then(Value::as_array) {
let rotation_key_strs: Vec<&str> = rotation_keys.iter().filter_map(Value::as_str).collect();
if let Some(missing) = missing_required_rotation_key(
&rotation_key_strs,
&signing_did_key,
tranquil_config::get().secrets.plc_rotation_key.as_deref(),
) {
return Err(ApiError::InvalidRequest(missing.message().into()));
}
}
let signed_op = sign_operation(&unsigned_op, &signing_key).map_err(|e| {
error!("Failed to sign PLC operation: {:?}", e);
ApiError::InternalError(None)
+8 -2
View File
@@ -467,9 +467,15 @@ pub fn api_routes() -> axum::Router<AppState> {
pub fn well_known_api_routes() -> axum::Router<AppState> {
use axum::routing::get;
axum::Router::new()
let routes = axum::Router::new()
.route("/did.json", get(identity::well_known_did))
.route("/atproto-did", get(identity::well_known_atproto_did))
.route("/atproto-did", get(identity::well_known_atproto_did));
if tranquil_config::get().server.enable_caddy_on_demand_tls {
routes.route("/caddy/ask", get(server::caddy_ask))
} else {
routes
}
}
pub fn webhook_routes() -> axum::Router<AppState> {
@@ -19,6 +19,7 @@ pub struct NotificationPrefsOutput {
pub telegram_verified: bool,
pub signal_username: Option<String>,
pub signal_verified: bool,
pub legacy_login_alerts: bool,
}
pub async fn get_notification_prefs(
@@ -32,6 +33,26 @@ pub async fn get_notification_prefs(
.await
.log_db_err("get notification prefs")?
.ok_or(ApiError::AccountNotFound)?;
let user_id = state
.repos
.user
.get_id_by_did(&auth.did)
.await
.log_db_err("get user by did")?
.ok_or(ApiError::AccountNotFound)?;
let legacy_login_alerts = state
.repos
.infra
.get_account_preferences(user_id)
.await
.log_db_err("get legacy login alert prefs")?
.iter()
.find(|(name, _)| name == "legacy_login_alerts")
.and_then(|(_, value)| value.as_bool())
.unwrap_or(true);
Ok(Json(NotificationPrefsOutput {
preferred_channel: prefs.preferred_channel,
email: prefs.email,
@@ -41,6 +62,7 @@ pub async fn get_notification_prefs(
telegram_verified: prefs.telegram_verified,
signal_username: prefs.signal_username,
signal_verified: prefs.signal_verified,
legacy_login_alerts,
}))
}
@@ -121,6 +143,7 @@ pub struct UpdateNotificationPrefsInput {
pub discord_username: Option<String>,
pub telegram_username: Option<String>,
pub signal_username: Option<String>,
pub legacy_login_alerts: Option<bool>,
}
#[derive(Serialize)]
@@ -435,6 +458,15 @@ pub async fn update_notification_prefs(
.await?;
}
if let Some(alerts) = input.legacy_login_alerts {
state
.repos
.infra
.upsert_account_preference(user_id, "legacy_login_alerts", json!(alerts))
.await
.log_db_err("update legacy login alert prefs")?;
}
Ok(Json(UpdateNotificationPrefsOutput {
success: true,
verification_required,
+9 -18
View File
@@ -148,7 +148,13 @@ pub async fn upload_blob(
size, cid_str
);
match state
if let Err(e) = state.blob_store.copy(&temp_key, &storage_key).await {
let _ = state.blob_store.delete(&temp_key).await;
error!("Failed to copy blob to final location: {:?}", e);
return Err(ApiError::InternalError(Some("Failed to store blob".into())));
}
if let Err(e) = state
.repos
.blob
.insert_blob(
@@ -160,24 +166,9 @@ pub async fn upload_blob(
)
.await
{
Ok(_) => {}
Err(e) => {
let _ = state.blob_store.delete(&temp_key).await;
error!("Failed to insert blob record: {:?}", e);
return Err(ApiError::InternalError(None));
}
};
if let Err(e) = state.blob_store.copy(&temp_key, &storage_key).await {
let _ = state.blob_store.delete(&temp_key).await;
if let Err(db_err) = state.repos.blob.delete_blob_by_cid(&cid_link).await {
error!(
"Failed to clean up orphaned blob record after copy failure: {:?}",
db_err
);
}
error!("Failed to copy blob to final location: {:?}", e);
return Err(ApiError::InternalError(Some("Failed to store blob".into())));
error!("Failed to insert blob record: {:?}", e);
return Err(ApiError::InternalError(None));
}
let _ = state.blob_store.delete(&temp_key).await;
+43
View File
@@ -0,0 +1,43 @@
use axum::extract::{Query, State};
use axum::http::StatusCode;
use serde::de::Error as _;
use serde::{Deserialize, Deserializer};
use tracing::error;
use tranquil_pds::handle::ServiceDomains;
use tranquil_pds::state::AppState;
use tranquil_pds::types::Handle;
pub struct AskedDomain(Handle);
impl<'de> Deserialize<'de> for AskedDomain {
fn deserialize<D: Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
let raw = String::deserialize(deserializer)?;
let without_root_dot = raw.strip_suffix('.').unwrap_or(&raw);
Handle::new(without_root_dot)
.map(Self)
.map_err(D::Error::custom)
}
}
#[derive(Deserialize)]
pub struct CaddyAskQuery {
pub domain: AskedDomain,
}
pub async fn caddy_ask(
State(state): State<AppState>,
Query(ask): Query<CaddyAskQuery>,
) -> StatusCode {
let AskedDomain(handle) = ask.domain;
if ServiceDomains::served().contains(handle.as_str()) {
return StatusCode::OK;
}
match state.repos.user.get_by_handle(&handle).await {
Ok(Some(_)) => StatusCode::OK,
Ok(None) => StatusCode::NOT_FOUND,
Err(e) => {
error!("caddy ask couldn't look up handle {handle}: {e:?}");
StatusCode::INTERNAL_SERVER_ERROR
}
}
}
+6 -1
View File
@@ -77,7 +77,12 @@ pub async fn describe_server(State(state): State<AppState>) -> Json<DescribeServ
let pds_hostname = &cfg.server.hostname;
Json(DescribeServerOutput {
available_user_domains: cfg.server.user_handle_domain_list(),
available_user_domains: match cfg.server.user_handle_domains.as_deref() {
Some(domains) if !domains.is_empty() => {
domains.iter().map(|d| d.as_str().to_owned()).collect()
}
_ => vec![cfg.server.hostname_without_port().to_owned()],
},
invite_code_required: cfg.server.invite_code_required,
did: format!("did:web:{}", pds_hostname),
links: DescribeServerLinks {
+2
View File
@@ -1,5 +1,6 @@
pub mod account_status;
pub mod app_password;
pub mod caddy;
pub mod email;
pub mod invite;
pub mod logo;
@@ -22,6 +23,7 @@ pub use account_status::{
request_account_delete,
};
pub use app_password::{create_app_password, list_app_passwords, revoke_app_password};
pub use caddy::caddy_ask;
pub use email::{
authorize_email_update, check_channel_verified, check_email_in_use, check_email_update_status,
check_email_verified, confirm_email, request_email_update, update_email,
+31 -17
View File
@@ -317,23 +317,37 @@ pub async fn create_session(
return Err(ApiError::InternalError(None));
}
if is_legacy_login && !used_totp_factor {
warn!(
did = %row.did,
ip = %client_ip,
"Legacy login on TOTP-enabled account - sending notification"
);
let hostname = &tranquil_config::get().server.hostname;
if let Err(e) = tranquil_pds::comms::comms_repo::enqueue_legacy_login(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
row.id,
hostname,
client_ip,
row.preferred_comms_channel,
)
.await
{
error!("Failed to queue legacy login notification: {:?}", e);
let alerts_enabled = match state.repos.infra.get_account_preferences(row.id).await {
Ok(prefs) => prefs
.iter()
.find(|(name, _)| name == "legacy_login_alerts")
.and_then(|(_, value)| value.as_bool())
.unwrap_or(true),
Err(e) => {
warn!("Failed to fetch legacy login alert preference: {:?}", e);
true
}
};
if alerts_enabled {
warn!(
did = %row.did,
ip = %client_ip,
"Legacy login on TOTP-enabled account - sending notification"
);
let hostname = &tranquil_config::get().server.hostname;
if let Err(e) = tranquil_pds::comms::comms_repo::enqueue_legacy_login(
state.repos.user.as_ref(),
state.repos.infra.as_ref(),
row.id,
hostname,
client_ip,
row.preferred_comms_channel,
)
.await
{
error!("Failed to queue legacy login notification: {:?}", e);
}
}
}
let handle = row.handle.clone();
+4 -6
View File
@@ -6,8 +6,8 @@ use tranquil_pds::api::error::{ApiError, DbResultExt};
use tranquil_pds::auth::{
Active, Auth, decrypt_totp_secret, encrypt_totp_secret, generate_backup_codes,
generate_qr_png_base64, generate_totp_secret, generate_totp_uri, hash_backup_code,
is_backup_code_format, require_legacy_session_mfa, verify_backup_code, verify_password_mfa,
verify_totp_code, verify_totp_mfa,
is_backup_code_format, verify_backup_code, verify_password_mfa, verify_totp_code,
verify_totp_mfa,
};
use tranquil_pds::rate_limit::{TotpVerifyLimit, check_user_rate_limit_with_message};
use tranquil_pds::state::AppState;
@@ -163,11 +163,9 @@ pub async fn disable_totp(
auth: Auth<Active>,
Json(input): Json<DisableTotpInput>,
) -> Result<Json<EmptyResponse>, ApiError> {
let session_mfa = require_legacy_session_mfa(&state, &auth).await?;
let _rate_limit = check_user_rate_limit_with_message::<TotpVerifyLimit>(
&state,
session_mfa.did(),
auth.did.as_str(),
"Too many verification attempts. Please try again in a few minutes.",
)
.await?;
@@ -184,7 +182,7 @@ pub async fn disable_totp(
tranquil_pds::auth::legacy_2fa::clear_challenge(state.cache.as_ref(), &auth.did).await;
info!(did = %session_mfa.did(), "TOTP disabled (verified via {} and {})", password_mfa.method(), totp_mfa.method());
info!(did = %password_mfa.did(), "TOTP disabled (verified via {} and {})", password_mfa.method(), totp_mfa.method());
Ok(Json(EmptyResponse {}))
}
+3 -1
View File
@@ -1,6 +1,6 @@
use lettre::Message;
use lettre::message::Mailbox;
use lettre::message::header::ContentType;
use lettre::message::header::{ContentType, MIME_VERSION_1_0};
use lettre::message::header::{Header, HeaderName, HeaderValue};
use uuid::Uuid;
@@ -24,6 +24,7 @@ pub(super) fn build(
.to(to)
.subject(subject)
.message_id(Some(message_id))
.header(MIME_VERSION_1_0)
.header(ContentType::TEXT_PLAIN);
let category = apply_atmos_categories
@@ -142,6 +143,7 @@ mod tests {
assert!(raw.contains("From: \"Test Sender\" <noreply@nel.pet>"));
assert!(raw.contains("To: user@nel.pet"));
assert!(raw.contains("Subject: Welcome"));
assert!(raw.contains("MIME-Version: 1.0"));
assert!(lower.contains("content-type: text/plain"));
assert!(raw.contains("Hello world."));
}
+2
View File
@@ -5,4 +5,6 @@ edition.workspace = true
license.workspace = true
[dependencies]
serde = { workspace = true }
tranquil-types = { workspace = true }
confique = { workspace = true }
+26 -21
View File
@@ -2,6 +2,7 @@ use confique::Config;
use std::fmt;
use std::path::PathBuf;
use std::sync::OnceLock;
use tranquil_types::Domain;
static CONFIG: OnceLock<TranquilConfig> = OnceLock::new();
@@ -30,7 +31,6 @@ impl fmt::Display for ConfigError {
}
impl std::error::Error for ConfigError {}
/// Initialize the global configuration. Must be called once at startup before
/// any other code accesses the configuration. Panics if called more than once.
pub fn init(config: TranquilConfig) {
@@ -224,6 +224,12 @@ impl TranquilConfig {
}
}
if let Err(e) = Domain::new(self.server.hostname_without_port()) {
errors.push(format!(
"server.hostname (PDS_HOSTNAME) must be a plain domain, {e}"
));
}
// -- email -----------------------------------------------------------
self.email
.validate(self.server.hostname_without_port(), &mut errors);
@@ -428,7 +434,7 @@ pub struct ServerConfig {
pub hostname: String,
/// Address to bind the HTTP server to.
#[config(env = "SERVER_HOST", default = "127.0.0.1")]
#[config(env = "SERVER_HOST", default = "[::1]")]
pub host: String,
/// Port to bind the HTTP server to.
@@ -438,13 +444,21 @@ pub struct ServerConfig {
/// List of domains for user handles.
/// Defaults to the PDS hostname when not set.
#[config(env = "PDS_USER_HANDLE_DOMAINS", parse_env = split_comma_list)]
pub user_handle_domains: Option<Vec<String>>,
pub user_handle_domains: Option<Vec<Domain>>,
/// Enable PDS-hosted did:web identities. Hosting did:web requires a
/// long-term commitment to serve DID documents; opt-in only.
#[config(env = "ENABLE_PDS_HOSTED_DID_WEB", default = false)]
pub enable_pds_hosted_did_web: bool,
/// The caddy on-demand TLS requires we serve
/// the endpoint `/.well-known/caddy/ask`.
/// It will be used so that caddy can create TLS
/// certs for us on the fly
/// and we don't have to do annoying wildcard certs.
#[config(env = "ENABLE_CADDY_ON_DEMAND_TLS", default = true)]
pub enable_caddy_on_demand_tls: bool,
/// iykyk!
#[config(env = "RFC_MOO_COMPLIANCE", default = false)]
pub rfc_moo_compliance: bool,
@@ -573,20 +587,6 @@ impl ServerConfig {
pub fn banned_word_list(&self) -> Vec<String> {
self.banned_words.clone().unwrap_or_default()
}
/// Returns the user handle domains, falling back to `[hostname_without_port]`.
pub fn user_handle_domain_list(&self) -> Vec<String> {
self.user_handle_domains
.as_deref()
.filter(|v| !v.is_empty())
.map(|v| v.to_vec())
.unwrap_or_else(|| vec![self.hostname_without_port().to_string()])
}
/// Alias for `user_handle_domain_list` (for callers that were using the now-removed `available_user_domains` field).
pub fn available_user_domain_list(&self) -> Vec<String> {
self.user_handle_domain_list()
}
}
#[derive(Debug, Config)]
@@ -770,6 +770,10 @@ pub struct StorageConfig {
#[config(env = "S3_ENDPOINT")]
pub s3_endpoint: Option<String>,
/// Path on the storage for the S3 blob backend.
#[config(env = "S3_PATH", default = "")]
pub s3_path: String,
/// Repository backend: `postgres` by default, or `tranquil-store`, our embedded db.
/// tranquil-store is EXPERIMENTAL!!!! RISK OF TOTAL DATA LOSS.
#[config(env = "REPO_BACKEND", default = "postgres")]
@@ -1484,12 +1488,13 @@ pub struct ImportConfig {
/// trimming whitespace and dropping empty entries.
///
/// Signature matches confique's `parse_env` expectation: `fn(&str) -> Result<T, E>`.
fn split_comma_list(value: &str) -> Result<Vec<String>, std::convert::Infallible> {
Ok(value
fn split_comma_list<T: std::str::FromStr>(value: &str) -> Result<Vec<T>, T::Err> {
value
.split(',')
.map(|item| item.trim().to_string())
.map(str::trim)
.filter(|item| !item.is_empty())
.collect())
.map(T::from_str)
.collect()
}
#[derive(Debug, Config)]
+2 -2
View File
@@ -70,12 +70,12 @@ pub trait BlobRepository: Send + Sync {
takedown_ref: Option<&str>,
) -> Result<bool, DbError>;
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<bool, DbError>;
async fn delete_blobs_by_user(&self, user_id: Uuid) -> Result<u64, DbError>;
async fn get_blob_storage_keys_by_user(&self, user_id: Uuid) -> Result<Vec<String>, DbError>;
async fn ensure_blob_ownership(&self, user_id: Uuid, cid: &CidLink) -> Result<bool, DbError>;
async fn insert_record_blobs(
&self,
repo_id: Uuid,
+1 -5
View File
@@ -1,7 +1,7 @@
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use serde::{Deserialize, Serialize};
use tranquil_types::{CidLink, Did, Handle, InviteCode};
use tranquil_types::{Did, Handle, InviteCode};
use uuid::Uuid;
use crate::DbError;
@@ -417,10 +417,6 @@ pub trait InfraRepository: Send + Sync {
async fn delete_server_config(&self, key: &str) -> Result<(), DbError>;
async fn get_blob_storage_key_by_cid(&self, cid: &CidLink) -> Result<Option<String>, DbError>;
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<(), DbError>;
async fn get_admin_account_info_by_did(
&self,
did: &Did,
+2 -2
View File
@@ -36,8 +36,8 @@ pub use repo::{
AccountStatus, ApplyCommitError, ApplyCommitInput, ApplyCommitResult, CommitEventData,
EventBlockInline, EventBlocks, FullRecordInfo, ImportBlock, ImportRecord, ImportRepoError,
PruneCount, RecordDelete, RecordInfo, RecordUpsert, RecordWithTakedown, RepoAccountInfo,
RepoEventNotifier, RepoEventReceiver, RepoEventType, RepoInfo, RepoListItem, RepoRepository,
RepoSeqEvent, RepoWithoutRev, SequencedEvent, UserNeedingRecordBlobsBackfill,
RepoEventNotifier, RepoEventReceiver, RepoEventType, RepoIdentity, RepoInfo, RepoListItem,
RepoRepository, RepoSeqEvent, RepoWithoutRev, SequencedEvent, UserNeedingRecordBlobsBackfill,
UserWithoutBlocks,
};
pub use scope::{DbScope, InvalidScopeError};
+8
View File
@@ -171,6 +171,12 @@ pub struct UserNeedingRecordBlobsBackfill {
pub did: Did,
}
#[derive(Debug, Clone)]
pub struct RepoIdentity {
pub user_id: Uuid,
pub did: Did,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RepoSeqEvent {
pub seq: SequenceNumber,
@@ -545,6 +551,8 @@ pub trait RepoRepository: Send + Sync {
limit: i64,
) -> Result<Vec<UserNeedingRecordBlobsBackfill>, DbError>;
async fn get_all_repo_identities(&self) -> Result<Vec<RepoIdentity>, DbError>;
async fn insert_record_blobs(
&self,
repo_id: Uuid,
+35 -20
View File
@@ -33,7 +33,7 @@ impl BlobRepository for PostgresBlobRepository {
let result = sqlx::query_scalar!(
r#"INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (cid) DO NOTHING RETURNING cid"#,
ON CONFLICT (cid, created_by_user) DO NOTHING RETURNING cid"#,
cid.as_str(),
mime_type,
size_bytes,
@@ -49,7 +49,7 @@ impl BlobRepository for PostgresBlobRepository {
async fn get_blob_metadata(&self, cid: &CidLink) -> Result<Option<BlobMetadata>, DbError> {
let result = sqlx::query!(
"SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1",
"SELECT storage_key, mime_type, size_bytes FROM blobs WHERE cid = $1 LIMIT 1",
cid.as_str()
)
.fetch_optional(&self.pool)
@@ -68,7 +68,7 @@ impl BlobRepository for PostgresBlobRepository {
cid: &CidLink,
) -> Result<Option<BlobWithTakedown>, DbError> {
let result = sqlx::query!(
"SELECT cid, takedown_ref FROM blobs WHERE cid = $1",
"SELECT cid, takedown_ref FROM blobs WHERE cid = $1 ORDER BY takedown_ref NULLS LAST LIMIT 1",
cid.as_str()
)
.fetch_optional(&self.pool)
@@ -86,11 +86,13 @@ impl BlobRepository for PostgresBlobRepository {
}
async fn get_blob_storage_key(&self, cid: &CidLink) -> Result<Option<String>, DbError> {
let result =
sqlx::query_scalar!("SELECT storage_key FROM blobs WHERE cid = $1", cid.as_str())
.fetch_optional(&self.pool)
.await
.map_err(map_sqlx_error)?;
let result = sqlx::query_scalar!(
"SELECT storage_key FROM blobs WHERE cid = $1 LIMIT 1",
cid.as_str()
)
.fetch_optional(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result)
}
@@ -147,7 +149,8 @@ impl BlobRepository for PostgresBlobRepository {
async fn sum_blob_storage(&self) -> Result<i64, DbError> {
let result = sqlx::query_scalar!(
r#"SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as "total!" FROM blobs"#
r#"SELECT COALESCE(SUM(size_bytes), 0)::BIGINT as "total!"
FROM (SELECT DISTINCT cid, size_bytes FROM blobs) t"#
)
.fetch_one(&self.pool)
.await
@@ -173,15 +176,6 @@ impl BlobRepository for PostgresBlobRepository {
Ok(result.rows_affected() > 0)
}
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<bool, DbError> {
let result = sqlx::query!("DELETE FROM blobs WHERE cid = $1", cid.as_str())
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result.rows_affected() > 0)
}
async fn delete_blobs_by_user(&self, user_id: Uuid) -> Result<u64, DbError> {
let result = sqlx::query!("DELETE FROM blobs WHERE created_by_user = $1", user_id)
.execute(&self.pool)
@@ -193,7 +187,12 @@ impl BlobRepository for PostgresBlobRepository {
async fn get_blob_storage_keys_by_user(&self, user_id: Uuid) -> Result<Vec<String>, DbError> {
let results = sqlx::query_scalar!(
r#"SELECT storage_key as "storage_key!" FROM blobs WHERE created_by_user = $1"#,
r#"SELECT storage_key as "storage_key!" FROM blobs b
WHERE created_by_user = $1
AND NOT EXISTS (
SELECT 1 FROM blobs o
WHERE o.cid = b.cid AND o.created_by_user <> $1
)"#,
user_id
)
.fetch_all(&self.pool)
@@ -203,6 +202,22 @@ impl BlobRepository for PostgresBlobRepository {
Ok(results)
}
async fn ensure_blob_ownership(&self, user_id: Uuid, cid: &CidLink) -> Result<bool, DbError> {
let result = sqlx::query!(
r#"INSERT INTO blobs (cid, mime_type, size_bytes, created_by_user, storage_key)
SELECT DISTINCT b.cid, b.mime_type, b.size_bytes, $1::uuid, b.storage_key
FROM blobs b WHERE b.cid = $2
ON CONFLICT (cid, created_by_user) DO NOTHING"#,
user_id,
cid.as_str()
)
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result.rows_affected() > 0)
}
async fn insert_record_blobs(
&self,
repo_id: Uuid,
@@ -238,7 +253,7 @@ impl BlobRepository for PostgresBlobRepository {
let results = sqlx::query!(
r#"SELECT rb.blob_cid, rb.record_uri
FROM record_blobs rb
LEFT JOIN blobs b ON rb.blob_cid = b.cid
LEFT JOIN blobs b ON rb.blob_cid = b.cid AND b.created_by_user = $1
WHERE rb.repo_id = $1 AND b.cid IS NULL AND rb.blob_cid > $2
ORDER BY rb.blob_cid
LIMIT $3"#,
+1 -20
View File
@@ -7,7 +7,7 @@ use tranquil_db_traits::{
InviteCodeSortOrder, InviteCodeState, InviteCodeUse, NotificationHistoryRow, PlcTokenInfo,
QueuedComms, ReservedSigningKey, ReservedSigningKeyFull, ValidatedInviteCode,
};
use tranquil_types::{CidLink, Did, InviteCode};
use tranquil_types::{Did, InviteCode};
use uuid::Uuid;
use super::col;
@@ -1010,25 +1010,6 @@ impl InfraRepository for PostgresInfraRepository {
Ok(())
}
async fn get_blob_storage_key_by_cid(&self, cid: &CidLink) -> Result<Option<String>, DbError> {
let result =
sqlx::query_scalar!("SELECT storage_key FROM blobs WHERE cid = $1", cid.as_str())
.fetch_optional(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(result)
}
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<(), DbError> {
sqlx::query!("DELETE FROM blobs WHERE cid = $1", cid.as_str())
.execute(&self.pool)
.await
.map_err(map_sqlx_error)?;
Ok(())
}
async fn get_admin_account_info_by_did(
&self,
did: &Did,
+25 -2
View File
@@ -4,8 +4,9 @@ use sqlx::PgPool;
use tranquil_db_traits::{
AccountStatus, CommitEventData, DbError, EventBlockInline, EventBlocks, FullRecordInfo,
ImportBlock, ImportRecord, ImportRepoError, PruneCount, RecordInfo, RecordWithTakedown,
RepoAccountInfo, RepoEventType, RepoInfo, RepoListItem, RepoRepository, RepoWithoutRev,
SequenceNumber, SequencedEvent, UserNeedingRecordBlobsBackfill, UserWithoutBlocks,
RepoAccountInfo, RepoEventType, RepoIdentity, RepoInfo, RepoListItem, RepoRepository,
RepoWithoutRev, SequenceNumber, SequencedEvent, UserNeedingRecordBlobsBackfill,
UserWithoutBlocks,
};
use tranquil_types::{AtUri, CidLink, Did, Handle, Nsid, Rkey, Tid};
use uuid::Uuid;
@@ -1650,6 +1651,28 @@ impl RepoRepository for PostgresRepoRepository {
.collect()
}
async fn get_all_repo_identities(&self) -> Result<Vec<RepoIdentity>, DbError> {
let rows = sqlx::query!(
r#"
SELECT u.id as user_id, u.did
FROM users u
JOIN repos r ON r.user_id = u.id
"#
)
.fetch_all(&self.pool)
.await
.map_err(map_sqlx_error)?;
rows.into_iter()
.map(|r| {
Ok(RepoIdentity {
user_id: r.user_id,
did: column(r.did, col::USERS_DID)?,
})
})
.collect()
}
async fn insert_record_blobs(
&self,
repo_id: Uuid,
@@ -1,4 +1,5 @@
use super::*;
use tranquil_scopes::{ParsedScope, parse_scope};
use tranquil_types::Nsid;
#[derive(Debug, Serialize)]
@@ -10,6 +11,7 @@ pub struct ScopeInfo {
pub display_name: String,
pub granted: Option<bool>,
pub restricted: bool,
pub superseded: bool,
#[serde(skip_serializing_if = "Option::is_none")]
pub effective_scope: Option<String>,
}
@@ -27,6 +29,7 @@ pub struct PermissionSetInfo {
pub expanded: Vec<ScopeInfo>,
pub granted: Option<bool>,
pub restricted: bool,
pub superseded: bool,
}
#[derive(Debug, Serialize)]
@@ -40,6 +43,13 @@ pub struct FailedSetInfo {
pub reason: tranquil_scopes::ResolveFailure,
}
#[derive(Debug, Serialize)]
pub struct RejectedScopeInfo {
// The scope exactly as the client requested it, which may be invalid or malformed.
pub scope: String,
pub reason: tranquil_scopes::ScopeRejection,
}
#[derive(Debug, Serialize)]
pub struct ConsentResponse {
pub request_uri: String,
@@ -49,7 +59,9 @@ pub struct ConsentResponse {
pub logo_uri: Option<String>,
pub scopes: Vec<ScopeInfo>,
pub permission_sets: Vec<PermissionSetInfo>,
pub transition_supersedes: bool,
pub failed_sets: Vec<FailedSetInfo>,
pub rejected_scopes: Vec<RejectedScopeInfo>,
pub show_consent: bool,
pub did: Did,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -152,9 +164,13 @@ pub async fn consent_get(
Some(grant) => scope_resolution::Authority::Delegated(&grant.granted_scopes),
None => scope_resolution::Authority::FullSelf,
};
let effective =
scope_resolution::resolve_effective_scopes(&*state.cache, requested_scope_str, authority)
.await;
let effective = scope_resolution::resolve_effective_scopes(
&*state.cache,
requested_scope_str,
authority,
client_metadata.as_ref().and_then(|m| m.scope.as_deref()),
)
.await;
let requested_scopes: Vec<&str> = effective.permitted.split_whitespace().collect();
let preferences = state
.repos
@@ -166,16 +182,7 @@ pub async fn consent_get(
.iter()
.map(|p| (p.scope.as_str(), p.granted))
.collect();
let presented_item_strings: Vec<String> = effective
.outcome
.passthrough
.iter()
.cloned()
.chain(effective.outcome.sets.iter().map(|g| match &g.aud {
Some(a) => format!("include:{}?aud={}", g.nsid, a),
None => format!("include:{}", g.nsid),
}))
.collect();
let presented_item_strings = effective.outcome.unexpanded_scopes();
let show_consent = should_show_consent(
state.repos.oauth.as_ref(),
&did,
@@ -185,6 +192,9 @@ pub async fn consent_get(
.await
.unwrap_or(true);
let has_granular_scopes = requested_scopes.iter().any(|s| is_granular_scope(s));
let has_transition_generic = requested_scopes
.iter()
.any(|s| matches!(parse_scope(s), ParsedScope::TransitionGeneric));
let grant_scope_str: Option<&str> =
delegation_grant.as_ref().map(|g| g.granted_scopes.as_str());
@@ -237,6 +247,8 @@ pub async fn consent_get(
)
};
let granted = pref_map.get(scope).copied();
let superseded = has_transition_generic
&& tranquil_scopes::superseded_by_transition_generic(&parse_scope(scope));
ScopeInfo {
scope: scope.to_string(),
category,
@@ -245,6 +257,7 @@ pub async fn consent_get(
display_name,
granted,
restricted,
superseded,
effective_scope,
}
};
@@ -261,12 +274,10 @@ pub async fn consent_get(
.sets
.iter()
.map(|g| {
let include_scope = match &g.aud {
Some(a) => format!("include:{}?aud={}", g.nsid, a),
None => format!("include:{}", g.nsid),
};
let include_scope = g.include_token();
let expanded: Vec<ScopeInfo> = g.expanded.iter().map(|s| make_scope_info(s)).collect();
let restricted = !expanded.is_empty() && expanded.iter().all(|s| s.restricted);
let superseded = !expanded.is_empty() && expanded.iter().all(|s| s.superseded);
PermissionSetInfo {
nsid: g.nsid.clone(),
aud: g.aud.clone(),
@@ -276,6 +287,7 @@ pub async fn consent_get(
include_scope,
expanded,
restricted,
superseded,
}
})
.collect();
@@ -291,6 +303,16 @@ pub async fn consent_get(
})
.collect();
let rejected_scopes: Vec<RejectedScopeInfo> = effective
.outcome
.rejected
.iter()
.map(|r| RejectedScopeInfo {
scope: r.scope.clone(),
reason: r.reason,
})
.collect();
let account_handle = state
.repos
.user
@@ -332,6 +354,9 @@ pub async fn consent_get(
(None, None, None, None)
};
let transition_supersedes =
scopes.iter().any(|s| s.superseded) || permission_sets.iter().any(|s| s.superseded);
Json(ConsentResponse {
request_uri: query.request_uri.clone(),
client_id: request_data.parameters.client_id.clone(),
@@ -340,7 +365,9 @@ pub async fn consent_get(
logo_uri: client_metadata.as_ref().and_then(|m| m.logo_uri.clone()),
scopes,
permission_sets,
transition_supersedes,
failed_sets,
rejected_scopes,
show_consent,
did: did.clone(),
handle: account_handle,
@@ -432,9 +459,19 @@ pub async fn consent_post(
Some(grant) => scope_resolution::Authority::Delegated(&grant.granted_scopes),
None => scope_resolution::Authority::FullSelf,
};
let effective =
scope_resolution::resolve_effective_scopes(&*state.cache, original_scope_str, authority)
.await;
let client_scope = state
.client_metadata_cache
.get(&request_data.parameters.client_id)
.await
.ok()
.and_then(|m| m.scope);
let effective = scope_resolution::resolve_effective_scopes(
&*state.cache,
original_scope_str,
authority,
client_scope.as_deref(),
)
.await;
let include_token = |nsid: &str, aud: &Option<String>| -> String {
match aud {
Some(a) => format!("include:{}?aud={}", nsid, a),
@@ -461,19 +498,7 @@ pub async fn consent_post(
),
);
}
let presented_items: Vec<String> = effective
.outcome
.passthrough
.iter()
.cloned()
.chain(
effective
.outcome
.sets
.iter()
.map(|g| include_token(&g.nsid, &g.aud)),
)
.collect();
let presented_items = effective.outcome.unexpanded_scopes();
let atproto_was_requested = presented_items.iter().any(|s| s == "atproto");
if atproto_was_requested && !form.approved_scopes.contains(&"atproto".to_string()) {
return json_error(
@@ -492,14 +517,6 @@ pub async fn consent_post(
);
}
let approved_scope_str = final_approved.join(" ");
let has_valid_scope = final_approved.iter().all(|s| is_valid_scope(s));
if !has_valid_scope {
return json_error(
StatusCode::BAD_REQUEST,
"invalid_request",
"Invalid scope format",
);
}
if form.remember {
let preferences: Vec<ScopePreference> = presented_items
.iter()
@@ -77,15 +77,6 @@ fn is_granular_scope(s: &str) -> bool {
|| s.starts_with("identity:")
}
fn is_valid_scope(s: &str) -> bool {
s == "atproto"
|| s == "transition:generic"
|| s == "transition:chat.bsky"
|| s == "transition:email"
|| is_granular_scope(s)
|| s.starts_with("include:")
}
fn extract_device_cookie(headers: &HeaderMap) -> Option<tranquil_types::DeviceId> {
headers
.get("cookie")
@@ -1,8 +1,10 @@
use tranquil_db_traits::DbScope;
use tranquil_pds::cache::Cache;
use tranquil_pds::delegation::intersect_scopes;
use tranquil_pds::delegation::{GrantCoverage, grant_coverage, intersect_scopes};
use tranquil_pds::oauth::permission_set_resolver::expand_scopes;
use tranquil_scopes::ExpansionOutcome;
use tranquil_scopes::{
ExpansionOutcome, ParsedScope, RejectedScope, RepoScope, ScopeRejection, parse_scope,
};
pub enum Authority<'a> {
FullSelf,
@@ -20,8 +22,12 @@ pub async fn resolve_effective_scopes(
cache: &dyn Cache,
requested: &str,
authority: Authority<'_>,
client_scope: Option<&str>,
) -> EffectiveScopes {
let outcome = expand_scopes(cache, requested).await;
let mut outcome = expand_scopes(cache, requested).await;
if let Some(registered) = client_scope.map(str::trim).filter(|s| !s.is_empty()) {
reject_unregistered(&mut outcome, registered);
}
let expanded = outcome.to_scope_string();
let permitted = match authority {
Authority::FullSelf => expanded,
@@ -30,6 +36,60 @@ pub async fn resolve_effective_scopes(
EffectiveScopes { permitted, outcome }
}
fn reject_unregistered(outcome: &mut ExpansionOutcome, registered: &str) {
let mut rejected = Vec::new();
let mut keep = |scope: String| match grant_coverage(registered, &scope) {
GrantCoverage::Full => Some(scope),
GrantCoverage::Narrowed(narrowed) => {
rejected.extend(narrowed_out(&scope, &narrowed).map(|scope| RejectedScope {
scope,
reason: ScopeRejection::NotRegistered,
}));
Some(narrowed)
}
GrantCoverage::Withheld => {
rejected.push(RejectedScope {
scope,
reason: ScopeRejection::NotRegistered,
});
None
}
};
outcome.passthrough = std::mem::take(&mut outcome.passthrough)
.into_iter()
.filter_map(&mut keep)
.collect();
outcome.sets = std::mem::take(&mut outcome.sets)
.into_iter()
.filter(|group| keep(group.include_token()).is_some())
.collect();
outcome.rejected.extend(rejected);
}
/// The repo actions dropped when `requested` was narrowed to `narrowed`, as a scope of their own.
/// Only repo scopes are ever narrowed; anything else yields `None`.
fn narrowed_out(requested: &str, narrowed: &str) -> Option<String> {
let (ParsedScope::Repo(requested), ParsedScope::Repo(narrowed)) =
(parse_scope(requested), parse_scope(narrowed))
else {
return None;
};
let actions: std::collections::HashSet<_> = requested
.actions
.difference(&narrowed.actions)
.copied()
.collect();
(!actions.is_empty()).then(|| {
RepoScope {
collection: requested.collection,
actions,
}
.to_scope_string()
})
}
#[cfg(test)]
mod tests {
use super::*;
@@ -65,6 +125,7 @@ mod tests {
&c,
"atproto include:io.atcr.authFullApp",
Authority::FullSelf,
None,
)
.await;
assert!(eff.permitted.contains("atproto"));
@@ -88,6 +149,7 @@ mod tests {
&c,
"atproto include:io.atcr.authFullApp",
Authority::Delegated(&granted),
None,
)
.await;
assert!(eff.permitted.contains("atproto"));
@@ -97,4 +159,106 @@ mod tests {
);
assert!(!eff.permitted.contains("identity"));
}
#[tokio::test]
async fn unrecognized_scopes_never_reach_permitted() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(&c, "atproto chat", Authority::FullSelf, None).await;
assert!(eff.permitted.split_whitespace().any(|s| s == "atproto"));
assert!(
!eff.permitted.split_whitespace().any(|s| s == "chat"),
"permitted was {:?}",
eff.permitted
);
assert_eq!(eff.outcome.rejected.len(), 1);
assert_eq!(eff.outcome.rejected[0].reason, ScopeRejection::Unrecognized);
}
#[tokio::test]
async fn scopes_absent_from_client_metadata_are_rejected() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(
&c,
"atproto identity:*",
Authority::FullSelf,
Some("atproto"),
)
.await;
assert!(!eff.permitted.split_whitespace().any(|s| s == "identity:*"));
assert_eq!(eff.outcome.rejected.len(), 1);
assert_eq!(eff.outcome.rejected[0].scope, "identity:*");
assert_eq!(
eff.outcome.rejected[0].reason,
ScopeRejection::NotRegistered
);
}
#[tokio::test]
async fn wildcard_client_registration_covers_narrower_request() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(
&c,
"atproto repo:app.bsky.feed.post?action=create",
Authority::FullSelf,
Some("atproto repo:*"),
)
.await;
assert!(eff.outcome.rejected.is_empty());
assert!(
eff.permitted
.contains("repo:app.bsky.feed.post?action=create")
);
}
#[tokio::test]
async fn absent_client_metadata_scope_constrains_nothing() {
let c = MemoryCache::new();
let eff =
resolve_effective_scopes(&c, "atproto identity:*", Authority::FullSelf, None).await;
assert!(eff.outcome.rejected.is_empty());
assert!(eff.permitted.contains("identity:*"));
}
#[tokio::test]
async fn set_expanded_scopes_bypass_the_client_registration_check() {
let c = cache_with("io.atcr.authFullApp", "identity:*").await;
let eff = resolve_effective_scopes(
&c,
"atproto include:io.atcr.authFullApp",
Authority::FullSelf,
Some("atproto include:io.atcr.authFullApp"),
)
.await;
assert!(
eff.outcome.rejected.is_empty(),
"a permission set legitimately expands to scopes the client never registered"
);
assert!(eff.permitted.contains("identity:*"));
}
#[tokio::test]
async fn actions_narrowed_out_by_client_metadata_are_reported_as_rejected() {
let c = MemoryCache::new();
let eff = resolve_effective_scopes(
&c,
"atproto repo:app.bsky.feed.post?action=create&action=delete",
Authority::FullSelf,
Some("atproto repo:*?action=create"),
)
.await;
assert!(
eff.permitted
.split_whitespace()
.any(|s| s == "repo:app.bsky.feed.post?action=create"),
"permitted was {:?}",
eff.permitted
);
assert_eq!(
eff.outcome.rejected,
vec![RejectedScope {
scope: "repo:app.bsky.feed.post?action=delete".to_string(),
reason: ScopeRejection::NotRegistered,
}]
);
}
}
@@ -5,7 +5,6 @@ use serde::{Deserialize, Serialize};
use tranquil_pds::oauth::{
AuthorizationRequestParameters, ClientAuth, CodeChallengeMethod, OAuthError, Prompt,
RequestData, RequestId, ResponseMode, ResponseType,
scopes::{ParsedScope, parse_scope},
};
use tranquil_pds::rate_limit::{OAuthParLimit, OAuthRateLimited};
use tranquil_pds::state::AppState;
@@ -84,7 +83,7 @@ pub async fn pushed_authorization_request(
let client_metadata = client_cache.get(&request.client_id).await?;
client_cache.validate_redirect_uri(&client_metadata, &request.redirect_uri)?;
let client_auth = determine_client_auth(&request)?;
let validated_scope = validate_scope(&request.scope, &client_metadata)?;
let validated_scope = normalize_scope(&request.scope)?;
let request_id = RequestId::generate();
let expires_at = Utc::now() + Duration::seconds(PAR_EXPIRY_SECONDS);
let response_mode = parse_response_mode(request.response_mode.as_deref())?;
@@ -165,10 +164,7 @@ fn determine_client_auth(request: &ParRequest) -> Result<ClientAuth, OAuthError>
Ok(ClientAuth::None)
}
fn validate_scope(
requested_scope: &Option<String>,
client_metadata: &tranquil_pds::oauth::ClientMetadata,
) -> Result<Option<String>, OAuthError> {
fn normalize_scope(requested_scope: &Option<String>) -> Result<Option<String>, OAuthError> {
let scope_str = match requested_scope {
Some(s) if !s.is_empty() => s,
_ => return Ok(Some("atproto".to_string())),
@@ -177,80 +173,14 @@ fn validate_scope(
if requested_scopes.is_empty() {
return Ok(Some("atproto".to_string()));
}
if let Some(unknown) = requested_scopes
.iter()
.find(|s| matches!(parse_scope(s), ParsedScope::Unknown(_)))
{
return Err(OAuthError::InvalidScope(format!(
"Unsupported scope: {}",
unknown
)));
}
let has_transition = requested_scopes.iter().any(|s| {
matches!(
parse_scope(s),
ParsedScope::TransitionGeneric
| ParsedScope::TransitionChat
| ParsedScope::TransitionEmail
)
});
let has_granular = requested_scopes.iter().any(|s| {
matches!(
parse_scope(s),
ParsedScope::Repo(_)
| ParsedScope::Blob(_)
| ParsedScope::Rpc(_)
| ParsedScope::Account(_)
| ParsedScope::Identity(_)
| ParsedScope::Include(_)
)
});
if has_transition && has_granular {
if !requested_scopes.contains(&"atproto") {
return Err(OAuthError::InvalidScope(
"Cannot mix transition scopes with granular scopes. Use either transition:* scopes OR granular scopes (repo:*, blob:*, rpc:*, account:*, include:*), not both.".to_string()
"The atproto scope is required".to_string(),
));
}
if let Some(client_scope) = &client_metadata.scope {
let client_scopes: Vec<&str> = client_scope.split_whitespace().collect();
if let Some(unregistered) = requested_scopes
.iter()
.find(|scope| !client_scopes.iter().any(|cs| scope_matches(cs, scope)))
{
return Err(OAuthError::InvalidScope(format!(
"Scope '{}' not registered for this client",
unregistered
)));
}
}
Ok(Some(requested_scopes.join(" ")))
}
fn scope_matches(client_scope: &str, requested_scope: &str) -> bool {
if client_scope == requested_scope {
return true;
}
fn get_resource_type(scope: &str) -> &str {
let base = scope.split('?').next().unwrap_or(scope);
base.split(':').next().unwrap_or(base)
}
let client_type = get_resource_type(client_scope);
let requested_type = get_resource_type(requested_scope);
if client_type == requested_type {
let client_base = client_scope.split('?').next().unwrap_or(client_scope);
if client_base.contains('*') {
return true;
}
}
false
}
fn parse_response_type(value: &str) -> Result<ResponseType, OAuthError> {
match value {
"code" => Ok(ResponseType::Code),
@@ -300,3 +230,45 @@ fn parse_prompt(value: Option<&str>) -> Result<Option<Prompt>, OAuthError> {
))),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn normalized(scope: Option<&str>) -> Result<Option<String>, OAuthError> {
normalize_scope(&scope.map(str::to_string))
}
#[test]
fn absent_or_blank_scope_defaults_to_atproto() {
assert_eq!(normalized(None).unwrap().as_deref(), Some("atproto"));
assert_eq!(normalized(Some("")).unwrap().as_deref(), Some("atproto"));
assert_eq!(normalized(Some(" ")).unwrap().as_deref(), Some("atproto"));
}
#[test]
fn scope_without_atproto_is_invalid() {
assert!(matches!(
normalized(Some("repo:*?action=create blob:*/*")),
Err(OAuthError::InvalidScope(_))
));
}
#[test]
fn atproto_need_not_come_first() {
assert_eq!(
normalized(Some("repo:*?action=create atproto"))
.unwrap()
.as_deref(),
Some("repo:*?action=create atproto")
);
}
#[test]
fn unrecognized_scopes_still_pass_par() {
assert_eq!(
normalized(Some("atproto chat")).unwrap().as_deref(),
Some("atproto chat")
);
}
}
@@ -157,6 +157,7 @@ pub async fn handle_authorization_code_grant(
&*state.cache,
requested_for_resolve,
authority,
client_metadata.scope.as_deref(),
)
.await;
if !effective.outcome.failures.is_empty() {
@@ -201,7 +202,10 @@ pub async fn handle_authorization_code_grant(
details: None,
code: None,
current_refresh_token: Some(refresh_token.clone()),
scope: requested_scope.clone(),
// Filtered but unexpanded: a remembered consent skips the consent screen, so the raw
// request can still hold scopes the client no longer registers. Sets stay as `include:`
// tokens so refresh re-resolves them.
scope: Some(effective.outcome.unexpanded_scopes().join(" ")),
controller_did: controller_did.clone(),
};
state
@@ -274,10 +278,13 @@ async fn recompute_resolved_scope(
Some(g) => crate::endpoints::authorize::scope_resolution::Authority::Delegated(g),
None => crate::endpoints::authorize::scope_resolution::Authority::FullSelf,
};
// No client metadata check here: `token_data.scope` was already filtered against it when
// the token was issued, so there is nothing for a re-check to remove.
let effective = crate::endpoints::authorize::scope_resolution::resolve_effective_scopes(
&*state.cache,
requested,
authority,
None,
)
.await;
if !effective.outcome.failures.is_empty() {
@@ -789,13 +789,16 @@ pub async fn check_handle_available(
}
};
let available_domains = tranquil_config::get().server.available_user_domain_list();
if let Some(ref d) = query.domain
&& !available_domains.iter().any(|ad| ad == d)
let available_domains = tranquil_pds::handle::ServiceDomains::for_user_handles();
if let Some(d) = &query.domain
&& !available_domains.contains(d.as_str())
{
return Err(ApiError::InvalidRequest("Unknown user domain".into()));
}
let domain = query.domain.as_deref().unwrap_or(&available_domains[0]);
let domain = query
.domain
.as_deref()
.unwrap_or_else(|| available_domains.primary().as_str());
let full_handle = format!("{}.{}", validated, domain);
let handle: tranquil_pds::types::Handle = match full_handle.parse() {
Ok(h) => h,
@@ -882,34 +885,33 @@ pub async fn complete_registration(
let cfg = tranquil_config::get();
let hostname = &cfg.server.hostname;
let available_domains = cfg.server.available_user_domain_list();
let available_domains = tranquil_pds::handle::ServiceDomains::for_user_handles();
let matched_domain = available_domains
.iter()
.filter(|d| input.handle.ends_with(&format!(".{}", d)))
.max_by_key(|d| d.len());
let split = available_domains.split_handle(&input.handle);
let handle: tranquil_pds::types::Handle =
if !input.handle.contains('.') || matched_domain.is_some() {
let handle_to_validate = match matched_domain {
Some(domain) => input
.handle
.strip_suffix(&format!(".{}", domain))
.unwrap_or(&input.handle),
None => &input.handle,
};
match tranquil_pds::api::validation::validate_short_handle(handle_to_validate) {
Ok(h) => format!("{}.{}", h, matched_domain.unwrap_or(&available_domains[0]))
.parse()
.map_err(|_| ApiError::InvalidHandle(None))?,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
} else {
match tranquil_pds::api::validation::validate_full_domain_handle(&input.handle) {
Ok(h) => h,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
let handle: tranquil_pds::types::Handle = if !input.handle.contains('.') || split.is_some() {
let handle_to_validate = match split {
Some((_domain, short)) => short,
None => input.handle.as_str(),
};
match tranquil_pds::api::validation::validate_short_handle(handle_to_validate) {
Ok(h) => format!(
"{}.{}",
h,
split
.map(|(d, _)| d)
.unwrap_or_else(|| available_domains.primary())
)
.parse()
.map_err(|_| ApiError::InvalidHandle(None))?,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
} else {
match tranquil_pds::api::validation::validate_full_domain_handle(&input.handle) {
Ok(h) => h,
Err(_) => return Err(ApiError::InvalidHandle(None)),
}
};
let verification_channel = input
.verification_channel
+1 -2
View File
@@ -763,8 +763,7 @@ impl From<crate::api::validation::HandleValidationError> for ApiError {
HandleValidationError::BannedWord => {
Self::InvalidHandle(Some("Inappropriate language in handle".to_string()))
}
HandleValidationError::UnusableHandleDomain
| HandleValidationError::NoHandleDomains => Self::InternalError(Some(e.to_string())),
HandleValidationError::UnusableHandleDomain => Self::InternalError(Some(e.to_string())),
_ => Self::InvalidHandle(Some(e.to_string())),
}
}
+9 -22
View File
@@ -111,7 +111,6 @@ pub enum HandleValidationError {
InvalidSyntax,
DisallowedTld,
UnusableHandleDomain,
NoHandleDomains,
}
impl std::fmt::Display for HandleValidationError {
@@ -143,9 +142,6 @@ impl std::fmt::Display for HandleValidationError {
f,
"This server's handle domain has a reserved TLD, so no handle under it is a valid atproto handle"
),
Self::NoHandleDomains => {
write!(f, "No handle domains are configured on this server")
}
}
}
}
@@ -215,21 +211,14 @@ pub fn validate_short_handle(handle: &str) -> Result<String, HandleValidationErr
}
pub fn resolve_handle_input(input: &str) -> Result<Handle, HandleValidationError> {
let available_domains = tranquil_config::get().server.available_user_domain_list();
let matched_domain = available_domains
.iter()
.filter(|d| input.ends_with(&format!(".{}", d)))
.max_by_key(|d| d.len());
let domains = crate::handle::ServiceDomains::for_user_handles();
let split = domains.split_handle(input);
if !input.contains('.') || matched_domain.is_some() {
let handle_to_validate = match matched_domain {
Some(domain) => input.strip_suffix(&format!(".{}", domain)).unwrap_or(input),
None => input,
};
let validated = validate_short_handle(handle_to_validate)?;
let domain = matched_domain
.or_else(|| available_domains.first())
.ok_or(HandleValidationError::NoHandleDomains)?;
if !input.contains('.') || split.is_some() {
let (short, domain) = split
.map(|(domain, short)| (short, domain))
.unwrap_or((input, domains.primary()));
let validated = validate_short_handle(short)?;
let handle = Handle::new(format!("{}.{}", validated, domain))
.map_err(|_| HandleValidationError::InvalidSyntax)?;
match handle.has_disallowed_tld() {
@@ -246,11 +235,9 @@ pub fn domain_forms_valid_handles(domain: &str) -> bool {
}
pub fn warn_unusable_handle_domains() {
tranquil_config::get()
.server
.user_handle_domain_list()
crate::handle::ServiceDomains::for_user_handles()
.iter()
.filter(|domain| !domain_forms_valid_handles(domain))
.filter(|domain| !domain_forms_valid_handles(domain.as_str()))
.for_each(|domain| {
tracing::error!(
domain = %domain,
+133 -16
View File
@@ -3,8 +3,16 @@ pub mod reserved;
use crate::types::{Did, Handle};
use hickory_resolver::TokioAsyncResolver;
use hickory_resolver::config::{ResolverConfig, ResolverOpts};
use std::sync::LazyLock;
use thiserror::Error;
pub use tranquil_types::Domain;
static HOSTNAME_DOMAIN: LazyLock<Domain> = LazyLock::new(|| {
Domain::new(tranquil_config::get().server.hostname_without_port())
.expect("server.hostname is validated at config load")
});
#[derive(Error, Debug)]
pub enum HandleResolutionError {
#[error("DNS lookup failed: {0}")]
@@ -85,28 +93,137 @@ pub async fn verify_handle_ownership(
}
}
pub fn is_service_domain_handle(handle: &str, hostname: &str) -> bool {
if !handle.contains('.') {
return true;
#[derive(Clone, Copy)]
pub struct ServiceDomains<'a> {
user_domains: &'a [Domain],
hostname: &'a Domain,
serve_hostname: bool,
}
impl ServiceDomains<'static> {
pub fn for_user_handles() -> Self {
Self::from_config(false)
}
pub fn served() -> Self {
Self::from_config(true)
}
fn from_config(serve_hostname: bool) -> Self {
let server = &tranquil_config::get().server;
Self {
user_domains: server.user_handle_domains.as_deref().unwrap_or_default(),
hostname: &HOSTNAME_DOMAIN,
serve_hostname,
}
}
}
impl<'a> ServiceDomains<'a> {
pub fn iter(&self) -> impl Iterator<Item = &'a Domain> {
let hostname = (self.serve_hostname || self.user_domains.is_empty())
.then_some(self.hostname)
.filter(|h| !self.user_domains.contains(h));
self.user_domains.iter().chain(hostname)
}
pub fn primary(&self) -> &'a Domain {
self.user_domains.first().unwrap_or(self.hostname)
}
pub fn contains(&self, name: &str) -> bool {
self.iter().any(|d| d.eq_name(name))
}
pub fn split_handle<'h>(&self, handle: &'h str) -> Option<(&'a Domain, &'h str)> {
self.iter()
.filter_map(|d| d.strip_from(handle).map(|short| (d, short)))
.max_by_key(|(d, _)| d.as_str().len())
}
let service_domains = tranquil_config::try_get()
.map(|c| c.server.user_handle_domain_list())
.unwrap_or_else(|| vec![hostname.to_string()]);
service_domains
.iter()
.any(|domain| handle.ends_with(&format!(".{}", domain)) || handle == domain)
}
#[cfg(test)]
mod tests {
use super::*;
use super::{Domain, ServiceDomains};
use std::sync::LazyLock;
static HOST: LazyLock<Domain> = LazyLock::new(|| "pds.oyster.cafe".parse().unwrap());
fn domains(user_domains: &[Domain], serve_hostname: bool) -> ServiceDomains<'_> {
ServiceDomains {
user_domains,
hostname: &HOST,
serve_hostname,
}
}
fn owned(list: &[&str]) -> Vec<Domain> {
list.iter().map(|d| d.parse().unwrap()).collect()
}
#[test]
fn test_is_service_domain_handle() {
assert!(is_service_domain_handle("nel.oyster.cafe", "oyster.cafe"));
assert!(is_service_domain_handle("oyster.cafe", "oyster.cafe"));
assert!(is_service_domain_handle("myhandle", "oyster.cafe"));
assert!(!is_service_domain_handle("lyna.nel.pet", "oyster.cafe"));
assert!(!is_service_domain_handle("myhandle.xyz", "oyster.cafe"));
fn thostname_until_domains_are_configured() {
assert!(domains(&[], false).contains("pds.oyster.cafe"));
assert_eq!(domains(&[], false).primary(), "pds.oyster.cafe");
let configured = owned(&["oyster.cafe"]);
assert!(!domains(&configured, false).contains("pds.oyster.cafe"));
assert!(domains(&configured, false).contains("oyster.cafe"));
}
#[test]
fn served_set_covers_hostname_and_handle_domains() {
let configured = owned(&["oyster.cafe"]);
assert!(domains(&configured, true).contains("pds.oyster.cafe"));
assert!(domains(&configured, true).contains("oyster.cafe"));
}
#[test]
fn hostname_in_list_is_yielded_once() {
let configured = owned(&["pds.oyster.cafe", "oyster.cafe"]);
let served: Vec<&str> = domains(&configured, true)
.iter()
.map(Domain::as_str)
.collect();
assert_eq!(served, ["pds.oyster.cafe", "oyster.cafe"]);
let configured = owned(&["PDS.Oyster.Cafe"]);
let served: Vec<&str> = domains(&configured, true)
.iter()
.map(Domain::as_str)
.collect();
assert_eq!(served, ["pds.oyster.cafe"]);
}
#[test]
fn matching_case_insensitive() {
let configured = owned(&["oyster.cafe"]);
assert!(domains(&configured, false).contains("Oyster.Cafe"));
let (domain, short) = domains(&configured, false)
.split_handle("NEL.OYSTER.CAFE")
.unwrap();
assert_eq!(domain, "oyster.cafe");
assert_eq!(short, "NEL");
}
#[test]
fn longest_matching_domain_wins() {
let configured = owned(&["oyster.cafe", "pets.oyster.cafe"]);
let (domain, short) = domains(&configured, false)
.split_handle("nel.pets.oyster.cafe")
.unwrap();
assert_eq!(domain, "pets.oyster.cafe");
assert_eq!(short, "nel");
}
#[test]
fn split_handle_requires_a_dot() {
let configured = owned(&["oyster.cafe"]);
assert_eq!(
domains(&configured, false).split_handle("oyster.cafe"),
None
);
assert_eq!(
domains(&configured, false).split_handle("notoyster.cafe"),
None
);
}
}
@@ -3,8 +3,8 @@ use crate::cache_keys::permission_set_key;
use serde::{Deserialize, Serialize};
use std::time::Duration;
use tranquil_scopes::{
ExpansionOutcome, FailedSet, ResolveFailure, ResolvedSetGroup, ScopeExpansionError,
fetch_and_expand, parse_include_scope,
ExpansionOutcome, FailedSet, ParsedScope, RejectedScope, ResolveFailure, ResolvedSetGroup,
ScopeExpansionError, ScopeRejection, fetch_and_expand, parse_include_scope, parse_scope,
};
use tranquil_types::Nsid;
@@ -32,6 +32,12 @@ pub async fn expand_scopes(cache: &dyn Cache, scope_string: &str) -> ExpansionOu
let mut outcome = ExpansionOutcome::default();
for tok in scope_string.split_whitespace() {
match tok.strip_prefix("include:") {
None if matches!(parse_scope(tok), ParsedScope::Unknown(_)) => {
outcome.rejected.push(RejectedScope {
scope: tok.to_string(),
reason: ScopeRejection::Unrecognized,
})
}
None => outcome.passthrough.push(tok.to_string()),
Some(rest) => {
let (nsid, aud) = parse_include_scope(rest);
@@ -236,4 +242,28 @@ mod tests {
assert_eq!(out.failures.len(), 1);
assert_eq!(out.failures[0].given_nsid, "nonexistent.fake.permissionSet");
}
#[tokio::test]
async fn unrecognized_scopes_are_rejected_not_passed_through() {
let cache = MemoryCache::new();
let out = expand_scopes(&cache, "atproto chat").await;
assert_eq!(out.passthrough, vec!["atproto".to_string()]);
assert!(
!out.flat_scopes().iter().any(|s| s == "chat"),
"an unrecognized scope must never reach the effective scope set"
);
assert_eq!(out.rejected.len(), 1);
assert_eq!(out.rejected[0].scope, "chat");
assert_eq!(out.rejected[0].reason, ScopeRejection::Unrecognized);
}
#[tokio::test]
async fn structurally_invalid_granular_scopes_are_rejected() {
let cache = MemoryCache::new();
let out = expand_scopes(&cache, "atproto rpc:*?aud=*").await;
assert_eq!(out.passthrough, vec!["atproto".to_string()]);
assert_eq!(out.rejected.len(), 1);
assert_eq!(out.rejected[0].scope, "rpc:*?aud=*");
assert_eq!(out.rejected[0].reason, ScopeRejection::Unrecognized);
}
}
+1 -1
View File
@@ -188,7 +188,7 @@ impl PlcClient {
let timeout_secs = cfg.map_or(10, |c| c.plc.timeout_secs);
let connect_timeout_secs = cfg.map_or(5, |c| c.plc.connect_timeout_secs);
let fetch_policy = tranquil_types::ReachPolicy::from_private_fetch(
cfg.map_or(false, |c| c.server.allow_private_fetch),
cfg.is_some_and(|c| c.server.allow_private_fetch),
);
let client = Client::builder()
.timeout(Duration::from_secs(timeout_secs))
+165 -1
View File
@@ -3,13 +3,16 @@ use cid::Cid;
use ipld_core::ipld::Ipld;
use jacquard_repo::commit::Commit;
use jacquard_repo::storage::BlockStore;
use std::collections::BTreeSet;
use std::str::FromStr;
use std::sync::Arc;
use std::time::Duration;
use tokio::time::interval;
use tokio_util::sync::CancellationToken;
use tracing::{debug, error, info, warn};
use tranquil_db_traits::{BlobRepository, RepoRepository, SsoRepository, UserRepository};
use tranquil_db_traits::{
BlobRepository, InfraRepository, RepoRepository, SsoRepository, UserRepository,
};
use tranquil_store::blockstore::CidBytes;
use tranquil_store::bloom::BloomFilter;
use tranquil_types::{AtUri, CidLink, Did};
@@ -307,6 +310,83 @@ async fn process_record_blobs(
Ok((user_id, did, blob_refs_found))
}
const OWNERSHIP_CHUNK_SIZE: usize = 500;
async fn process_blob_ownership(
repo_repo: &dyn RepoRepository,
blob_repo: &dyn BlobRepository,
block_store: &AnyBlockStore,
user_id: uuid::Uuid,
did: Did,
) -> Result<(uuid::Uuid, Did, u64), (uuid::Uuid, &'static str)> {
let records = repo_repo
.get_all_records(user_id)
.await
.map_err(|_| (user_id, "failed to fetch records"))?;
let mut cids: BTreeSet<CidLink> = BTreeSet::new();
for chunk in records.chunks(OWNERSHIP_CHUNK_SIZE) {
futures::future::join_all(chunk.iter().map(|record| async move {
let uri = format!("{}/{}", record.collection.as_str(), record.rkey.as_str());
let cid = match Cid::from_str(record.record_cid.as_str()) {
Ok(c) => c,
Err(e) => {
warn!(user_id = %user_id, record = %uri, error = %e, "skipping record with unparseable CID");
return None;
}
};
let block_bytes = match block_store.get(&cid).await {
Ok(Some(b)) => b,
Ok(None) => {
warn!(user_id = %user_id, record = %uri, "skipping record where block is missing in the block store");
return None;
}
Err(e) => {
warn!(user_id = %user_id, record = %uri, error = %e, "skipping record because block couldn't be read");
return None;
}
};
let record_ipld: Ipld = match serde_ipld_dagcbor::from_slice(&block_bytes) {
Ok(v) => v,
Err(e) => {
warn!(user_id = %user_id, record = %uri, error = %e, "skipping record because block couldn't be decoded");
return None;
}
};
Some(
crate::sync::import::find_blob_refs_ipld(&record_ipld, 0)
.into_iter()
.map(|blob_ref| blob_ref.cid)
.collect::<Vec<_>>(),
)
}))
.await
.into_iter()
.flatten()
.flatten()
.for_each(|cid| {
cids.insert(cid);
});
}
let mut granted = 0u64;
for cid in &cids {
if blob_repo
.ensure_blob_ownership(user_id, cid)
.await
.map_err(|_| (user_id, "failed to grant ownership"))?
{
granted += 1;
}
}
Ok((user_id, did, granted))
}
pub async fn backfill_record_blobs(repo_repo: Arc<dyn RepoRepository>, block_store: AnyBlockStore) {
let users_needing_backfill = match repo_repo.get_users_needing_record_blobs_backfill(100).await
{
@@ -352,6 +432,90 @@ pub async fn backfill_record_blobs(repo_repo: Arc<dyn RepoRepository>, block_sto
info!(success, failed, "Completed record_blobs backfill");
}
const BLOB_OWNERSHIP_BACKFILL_KEY: &str = "blob_ownership_backfilled";
pub async fn backfill_blob_ownership(
infra_repo: Arc<dyn InfraRepository>,
repo_repo: Arc<dyn RepoRepository>,
blob_repo: Arc<dyn BlobRepository>,
block_store: AnyBlockStore,
) {
match infra_repo
.get_server_config(BLOB_OWNERSHIP_BACKFILL_KEY)
.await
{
Ok(Some(_)) => return,
Ok(None) => {}
Err(e) => {
error!("Failed to read blob ownership backfill marker: {:?}", e);
return;
}
}
let repos = match repo_repo.get_all_repo_identities().await {
Ok(rows) => rows,
Err(e) => {
error!("Failed to query repos for blob ownership backfill: {:?}", e);
return;
}
};
if repos.is_empty() {
debug!("No repos need blob ownership backfill",);
return;
}
info!(
count = repos.len(),
"Backfilling blob ownership for existing repos"
);
let mut success = 0;
let mut failed = 0;
for chunk in repos.chunks(OWNERSHIP_CHUNK_SIZE) {
let results = futures::future::join_all(chunk.iter().map(|repo| {
let repo_repo = repo_repo.clone();
let blob_repo = blob_repo.clone();
let block_store = block_store.clone();
async move {
process_blob_ownership(
repo_repo.as_ref(),
blob_repo.as_ref(),
&block_store,
repo.user_id,
repo.did.clone(),
)
.await
}
}))
.await;
results.iter().for_each(|r| match r {
Ok((user_id, did, granted)) => {
if *granted > 0 {
info!(user_id = %user_id, did = %did, granted = granted, "Granted blob ownership");
}
success += 1;
}
Err((user_id, reason)) => {
warn!(user_id = %user_id, reason = reason, "Failed to backfill blob ownership");
failed += 1;
}
});
}
if let Err(e) = infra_repo
.upsert_server_config(BLOB_OWNERSHIP_BACKFILL_KEY, "1")
.await
{
error!("Failed to set blob ownership backfill marker: {:?}", e);
}
info!(success, failed, "Completed blob ownership backfill");
}
#[allow(clippy::too_many_arguments)]
pub async fn start_scheduled_tasks(
user_repo: Arc<dyn UserRepository>,
+113
View File
@@ -0,0 +1,113 @@
mod common;
use common::*;
use futures::StreamExt;
use reqwest::StatusCode;
use serde_json::{Value, json};
#[ctor::ctor]
fn enable_on_demand_tls() {
unsafe {
std::env::set_var("ENABLE_CADDY_ON_DEMAND_TLS", "true");
std::env::set_var("PDS_USER_HANDLE_DOMAINS", "handles.pds.test");
}
}
async fn create_hosted_account() -> String {
let client = client();
let short_handle = format!("caddy{}", &uuid::Uuid::new_v4().simple().to_string()[..12]);
let payload = json!({
"handle": short_handle,
"email": format!("{}@oyster.cafe", short_handle),
"password": "Testpass123!"
});
let res = client
.post(format!(
"{}/xrpc/com.atproto.server.createAccount",
base_url().await
))
.json(&payload)
.send()
.await
.expect("failed to create account");
assert_eq!(res.status(), StatusCode::OK);
let body: Value = res
.json()
.await
.expect("createAccount response wasn't JSON");
body["handle"]
.as_str()
.expect("createAccount didn't return a handle")
.to_string()
}
async fn ask(client: &reqwest::Client, domain: &str) -> StatusCode {
client
.get(format!("{}/.well-known/caddy/ask", base_url().await))
.query(&[("domain", domain)])
.send()
.await
.expect("failed to query ask endpoint")
.status()
}
#[tokio::test]
async fn test_caddy_ask_allows_hosted_handle() {
let client = client();
let handle = create_hosted_account().await;
assert_eq!(ask(&client, &handle).await, StatusCode::OK);
assert_eq!(ask(&client, &handle.to_uppercase()).await, StatusCode::OK);
assert_eq!(ask(&client, &format!("{handle}.")).await, StatusCode::OK);
}
#[tokio::test]
async fn test_caddy_ask_denies_unhosted_and_invalid_domains() {
let client = client();
let unknown = format!("ghost-{}.handles.pds.test", uuid::Uuid::new_v4().simple());
assert_eq!(ask(&client, &unknown).await, StatusCode::NOT_FOUND);
assert_eq!(ask(&client, "nel.pet").await, StatusCode::NOT_FOUND);
assert_eq!(
ask(&client, "!!not-a-handle").await,
StatusCode::BAD_REQUEST
);
assert_eq!(ask(&client, "").await, StatusCode::BAD_REQUEST);
let res = client
.get(format!("{}/.well-known/caddy/ask", base_url().await))
.send()
.await
.expect("failed to query ask endpoint");
assert_eq!(res.status(), StatusCode::BAD_REQUEST);
}
#[tokio::test]
async fn test_caddy_ask_allows_handle_domain_apexes() {
let client = client();
base_url().await;
futures::stream::iter(
tranquil_config::get()
.server
.user_handle_domains
.iter()
.flatten(),
)
.for_each(|domain| async {
assert_eq!(ask(&client, domain.as_str()).await, StatusCode::OK);
})
.await;
}
#[tokio::test]
async fn test_caddy_ask_allows_the_pds_hostname_beside_handle_domains() {
let client = client();
base_url().await;
let cfg = tranquil_config::get();
let hostname = cfg.server.hostname_without_port();
assert!(
!cfg.server
.user_handle_domains
.iter()
.flatten()
.any(|d| d == hostname),
"this test only means something if hostname is outside the handle domains"
);
assert_eq!(ask(&client, hostname).await, StatusCode::OK);
}
+7 -6
View File
@@ -1057,7 +1057,7 @@ async fn test_granular_scope_repo_create_only() {
let url = base_url().await;
let http_client = client();
let (token, did, _) =
get_oauth_token_with_scope("repo:app.bsky.feed.post?action=create blob:*/*").await;
get_oauth_token_with_scope("atproto repo:app.bsky.feed.post?action=create blob:*/*").await;
let now = chrono::Utc::now().to_rfc3339();
let create_res = http_client
.post(format!("{}/xrpc/com.atproto.repo.createRecord", url))
@@ -1111,7 +1111,7 @@ async fn test_granular_scope_wildcard_collection() {
let url = base_url().await;
let http_client = client();
let (token, did, _) = get_oauth_token_with_scope(
"repo:app.bsky.*?action=create&action=update&action=delete blob:*/*",
"atproto repo:app.bsky.*?action=create&action=update&action=delete blob:*/*",
)
.await;
let now = chrono::Utc::now().to_rfc3339();
@@ -1168,7 +1168,7 @@ async fn test_granular_scope_wildcard_collection() {
async fn test_granular_scope_email_read() {
let url = base_url().await;
let http_client = client();
let (token, did, _) = get_oauth_token_with_scope("account:email?action=read").await;
let (token, did, _) = get_oauth_token_with_scope("atproto account:email?action=read").await;
let session_res = http_client
.get(format!("{}/xrpc/com.atproto.server.getSession", url))
.bearer_auth(&token)
@@ -1189,7 +1189,7 @@ async fn test_granular_scope_email_read() {
async fn test_granular_scope_no_email_access() {
let url = base_url().await;
let http_client = client();
let (token, did, _) = get_oauth_token_with_scope("repo:*?action=create blob:*/*").await;
let (token, did, _) = get_oauth_token_with_scope("atproto repo:*?action=create blob:*/*").await;
let session_res = http_client
.get(format!("{}/xrpc/com.atproto.server.getSession", url))
.bearer_auth(&token)
@@ -1210,7 +1210,8 @@ async fn test_granular_scope_no_email_access() {
async fn test_granular_scope_rpc_specific_method() {
let url = base_url().await;
let http_client = client();
let (token, _, _) = get_oauth_token_with_scope("rpc:app.bsky.feed.getTimeline?aud=*").await;
let (token, _, _) =
get_oauth_token_with_scope("atproto rpc:app.bsky.feed.getTimeline?aud=*").await;
let allowed_res = http_client
.get(format!("{}/xrpc/com.atproto.server.getServiceAuth", url))
.bearer_auth(&token)
@@ -1275,7 +1276,7 @@ async fn test_granular_scope_rpc_aud_with_service_id() {
let url = base_url().await;
let http_client = client();
let (token, _, _) = get_oauth_token_with_scope(
"rpc:app.bsky.feed.getTimeline?aud=did:web:api.bsky.app#bsky_appview",
"atproto rpc:app.bsky.feed.getTimeline?aud=did:web:api.bsky.app#bsky_appview",
)
.await;
let allowed_res = http_client
+423 -2
View File
@@ -3,7 +3,7 @@ mod helpers;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::Utc;
use common::{base_url, client};
use common::{base_url, client, get_test_repos};
use helpers::verify_new_account;
use reqwest::StatusCode;
use serde_json::{Value, json};
@@ -22,9 +22,16 @@ fn generate_pkce() -> (String, String) {
}
async fn setup_mock_client_metadata(redirect_uri: &str) -> MockServer {
setup_mock_client_metadata_with_scope(redirect_uri, None).await
}
async fn setup_mock_client_metadata_with_scope(
redirect_uri: &str,
scope: Option<&str>,
) -> MockServer {
let mock_server = MockServer::start().await;
let client_id = mock_server.uri();
let metadata = json!({
let mut metadata = json!({
"client_id": client_id,
"client_name": "Test OAuth Scope Client",
"redirect_uris": [redirect_uri],
@@ -33,6 +40,9 @@ async fn setup_mock_client_metadata(redirect_uri: &str) -> MockServer {
"token_endpoint_auth_method": "none",
"dpop_bound_access_tokens": false
});
if let Some(scope) = scope {
metadata["scope"] = json!(scope);
}
Mock::given(method("GET"))
.and(path("/"))
.respond_with(ResponseTemplate::new(200).set_body_json(metadata))
@@ -693,3 +703,414 @@ async fn test_dereference_scope_requires_auth() {
"Should require authentication"
);
}
#[tokio::test]
async fn test_unrecognized_scope_reaches_consent_and_is_never_granted() {
let url = base_url().await;
let http_client = client();
let redirect_uri = "https://example.com/callback";
let suffix = &uuid::Uuid::new_v4().simple().to_string()[..4];
let handle = format!("badscope{}", suffix);
let email = format!("badscope{}@example.com", suffix);
let password = "BadscopePass123!";
let create_res = http_client
.post(format!("{}/xrpc/com.atproto.server.createAccount", url))
.json(&json!({ "handle": handle, "email": email, "password": password }))
.send()
.await
.expect("Account creation failed");
assert_eq!(create_res.status(), StatusCode::OK);
let account: Value = create_res.json().await.unwrap();
let user_did = account["did"].as_str().unwrap().to_string();
let _ = verify_new_account(&http_client, &user_did).await;
let mock_client = setup_mock_client_metadata(redirect_uri).await;
let client_id = mock_client.uri();
let (code_verifier, code_challenge) = generate_pkce();
let par_res = http_client
.post(format!("{}/oauth/par", url))
.form(&[
("response_type", "code"),
("client_id", &client_id),
("redirect_uri", redirect_uri),
("code_challenge", &code_challenge),
("code_challenge_method", "S256"),
("scope", "atproto chat"),
])
.send()
.await
.expect("PAR failed");
assert!(
par_res.status() == StatusCode::OK || par_res.status() == StatusCode::CREATED,
"PAR must not reject an unrecognized scope, got {}",
par_res.status()
);
let par_body: Value = par_res.json().await.unwrap();
let request_uri = par_body["request_uri"].as_str().unwrap().to_string();
let auth_res = http_client
.post(format!("{}/oauth/authorize", url))
.header("Content-Type", "application/json")
.header("Accept", "application/json")
.json(&json!({
"request_uri": request_uri,
"username": &handle,
"password": password,
"remember_device": false
}))
.send()
.await
.expect("Authorize failed");
assert_eq!(auth_res.status(), StatusCode::OK);
let auth_body: Value = auth_res.json().await.unwrap();
let location = auth_body["redirect_uri"].as_str().unwrap().to_string();
assert!(
location.contains("/oauth/consent"),
"should land on the consent screen, got {}",
location
);
let consent_get: Value = http_client
.get(format!(
"{}/oauth/authorize/consent?request_uri={}",
url, request_uri
))
.send()
.await
.expect("Consent GET failed")
.json()
.await
.unwrap();
let rejected = consent_get["rejected_scopes"].as_array().unwrap();
assert_eq!(rejected.len(), 1, "got {:?}", rejected);
assert_eq!(rejected[0]["scope"].as_str(), Some("chat"));
assert_eq!(rejected[0]["reason"].as_str(), Some("unrecognized"));
assert!(
!consent_get["scopes"]
.as_array()
.unwrap()
.iter()
.any(|s| s["scope"] == "chat"),
"an unrecognized scope must never be offered as grantable"
);
let consent_res = http_client
.post(format!("{}/oauth/authorize/consent", url))
.header("Content-Type", "application/json")
.json(&json!({
"request_uri": request_uri,
"approved_scopes": ["atproto", "chat"],
"remember": false
}))
.send()
.await
.expect("Consent POST failed");
assert_eq!(consent_res.status(), StatusCode::OK);
let consent_body: Value = consent_res.json().await.unwrap();
let location = consent_body["redirect_uri"].as_str().unwrap().to_string();
let code = location
.split("code=")
.nth(1)
.unwrap()
.split('&')
.next()
.unwrap();
let token_res = http_client
.post(format!("{}/oauth/token", url))
.form(&[
("grant_type", "authorization_code"),
("code", code),
("redirect_uri", redirect_uri),
("code_verifier", &code_verifier),
("client_id", &client_id),
])
.send()
.await
.expect("Token request failed");
assert_eq!(token_res.status(), StatusCode::OK);
let token_body: Value = token_res.json().await.unwrap();
let granted = token_body["scope"].as_str().unwrap();
assert!(
granted.split_whitespace().any(|s| s == "atproto"),
"granted scope was {:?}",
granted
);
assert!(
!granted.split_whitespace().any(|s| s == "chat"),
"an unrecognized scope leaked into the issued token: {:?}",
granted
);
}
struct PendingAuthorization {
client_id: String,
request_uri: String,
code_verifier: String,
// Where authorize sent us: the consent screen, or straight to the client with a code.
location: String,
_mock: MockServer,
}
const REDIRECT_URI: &str = "https://example.com/callback";
async fn par_and_login(
handle_prefix: &str,
requested_scope: &str,
client_scope: Option<&str>,
before_login: impl AsyncFnOnce(&str, &str),
) -> PendingAuthorization {
let url = base_url().await;
let http_client = client();
let suffix = &uuid::Uuid::new_v4().simple().to_string()[..4];
let handle = format!("{}{}", handle_prefix, suffix);
let password = format!("{}Pass123!", handle_prefix);
let create_res = http_client
.post(format!("{}/xrpc/com.atproto.server.createAccount", url))
.json(&json!({
"handle": handle,
"email": format!("{}{}@example.com", handle_prefix, suffix),
"password": password
}))
.send()
.await
.expect("Account creation failed");
assert_eq!(create_res.status(), StatusCode::OK);
let account: Value = create_res.json().await.unwrap();
let did = account["did"].as_str().unwrap().to_string();
let _ = verify_new_account(&http_client, &did).await;
let mock = setup_mock_client_metadata_with_scope(REDIRECT_URI, client_scope).await;
let client_id = mock.uri();
let (code_verifier, code_challenge) = generate_pkce();
let par_res = http_client
.post(format!("{}/oauth/par", url))
.form(&[
("response_type", "code"),
("client_id", &client_id),
("redirect_uri", REDIRECT_URI),
("code_challenge", &code_challenge),
("code_challenge_method", "S256"),
("scope", requested_scope),
])
.send()
.await
.expect("PAR failed");
assert_eq!(par_res.status(), StatusCode::CREATED, "PAR should succeed");
let par_body: Value = par_res.json().await.unwrap();
let request_uri = par_body["request_uri"].as_str().unwrap().to_string();
before_login(&did, &client_id).await;
let auth_res = http_client
.post(format!("{}/oauth/authorize", url))
.header("Content-Type", "application/json")
.header("Accept", "application/json")
.json(&json!({
"request_uri": request_uri,
"username": &handle,
"password": &password,
"remember_device": false
}))
.send()
.await
.expect("Authorize failed");
assert_eq!(auth_res.status(), StatusCode::OK);
let auth_body: Value = auth_res.json().await.unwrap();
let location = auth_body["redirect_uri"].as_str().unwrap().to_string();
PendingAuthorization {
client_id,
request_uri,
code_verifier,
location,
_mock: mock,
}
}
async fn exchange_code(pending: &PendingAuthorization, location: &str) -> Value {
let code = location
.split("code=")
.nth(1)
.expect("redirect should carry a code")
.split('&')
.next()
.unwrap();
let token_res = client()
.post(format!("{}/oauth/token", base_url().await))
.form(&[
("grant_type", "authorization_code"),
("code", code),
("redirect_uri", REDIRECT_URI),
("code_verifier", &pending.code_verifier),
("client_id", &pending.client_id),
])
.send()
.await
.expect("Token request failed");
assert_eq!(token_res.status(), StatusCode::OK);
token_res.json().await.unwrap()
}
fn has_scope(scope_str: &str, scope: &str) -> bool {
scope_str.split_whitespace().any(|s| s == scope)
}
#[tokio::test]
async fn test_par_rejects_scope_without_atproto() {
let url = base_url().await;
let mock = setup_mock_client_metadata(REDIRECT_URI).await;
let client_id = mock.uri();
let (_, code_challenge) = generate_pkce();
let par_res = client()
.post(format!("{}/oauth/par", url))
.form(&[
("response_type", "code"),
("client_id", &client_id),
("redirect_uri", REDIRECT_URI),
("code_challenge", &code_challenge),
("code_challenge_method", "S256"),
("scope", "repo:*?action=create"),
])
.send()
.await
.expect("PAR failed");
assert_eq!(par_res.status(), StatusCode::BAD_REQUEST);
let body: Value = par_res.json().await.unwrap();
assert_eq!(
body["error"].as_str(),
Some("invalid_scope"),
"got {:?}",
body
);
}
#[tokio::test]
async fn test_scope_missing_from_client_metadata_is_not_registered_on_consent() {
let pending = par_and_login(
"unreg",
"atproto identity:*",
Some("atproto"),
async |_, _| {},
)
.await;
assert!(pending.location.contains("/oauth/consent"));
let url = base_url().await;
let consent_get: Value = client()
.get(format!(
"{}/oauth/authorize/consent?request_uri={}",
url, pending.request_uri
))
.send()
.await
.expect("Consent GET failed")
.json()
.await
.unwrap();
let rejected = consent_get["rejected_scopes"].as_array().unwrap();
assert_eq!(rejected.len(), 1, "got {:?}", rejected);
assert_eq!(rejected[0]["scope"].as_str(), Some("identity:*"));
assert_eq!(rejected[0]["reason"].as_str(), Some("not_registered"));
let consent_res = client()
.post(format!("{}/oauth/authorize/consent", url))
.header("Content-Type", "application/json")
.json(&json!({
"request_uri": pending.request_uri,
"approved_scopes": ["atproto", "identity:*"],
"remember": false
}))
.send()
.await
.expect("Consent POST failed");
assert_eq!(consent_res.status(), StatusCode::OK);
let consent_body: Value = consent_res.json().await.unwrap();
let token = exchange_code(&pending, consent_body["redirect_uri"].as_str().unwrap()).await;
let granted = token["scope"].as_str().unwrap();
assert!(!has_scope(granted, "identity:*"), "granted {:?}", granted);
}
/// A remembered consent skips the consent screen, so the scope stored on the token must be
/// filtered at issuance rather than copied from the raw request. Otherwise a scope the client
/// has since dropped from its metadata survives in storage and comes back on refresh.
#[tokio::test]
async fn test_remembered_scope_later_unregistered_never_reaches_a_token() {
let pending = par_and_login(
"remember",
"atproto identity:*",
Some("atproto"),
async |did, client_id| {
let prefs =
["atproto", "identity:*"].map(|scope| tranquil_pds::oauth::db::ScopePreference {
scope: scope.to_string(),
granted: true,
});
get_test_repos()
.await
.oauth
.upsert_scope_preferences(
&did.parse().unwrap(),
&tranquil_types::ClientId::new(client_id.to_string()),
&prefs,
)
.await
.expect("seeding scope preferences failed");
},
)
.await;
assert!(
!pending.location.contains("/oauth/consent"),
"remembered consent should skip the consent screen, got {}",
pending.location
);
let token = exchange_code(&pending, &pending.location).await;
assert!(!has_scope(token["scope"].as_str().unwrap(), "identity:*"));
let token_id = {
let payload = token["access_token"]
.as_str()
.unwrap()
.split('.')
.nth(1)
.unwrap();
let claims: Value =
serde_json::from_slice(&URL_SAFE_NO_PAD.decode(payload).unwrap()).unwrap();
tranquil_types::TokenId::new(claims["sid"].as_str().expect("sid claim"))
};
let row = get_test_repos()
.await
.oauth
.get_token_by_id(&token_id)
.await
.expect("get_token_by_id query failed")
.expect("token row should exist");
let row_scope = row.scope.expect("token row should have a scope");
assert!(
!has_scope(&row_scope, "identity:*"),
"stored {:?}",
row_scope
);
let refresh_res = client()
.post(format!("{}/oauth/token", base_url().await))
.form(&[
("grant_type", "refresh_token"),
("refresh_token", token["refresh_token"].as_str().unwrap()),
("client_id", &pending.client_id),
])
.send()
.await
.expect("Refresh request failed");
assert_eq!(refresh_res.status(), StatusCode::OK);
let refreshed: Value = refresh_res.json().await.unwrap();
assert!(
!has_scope(refreshed["scope"].as_str().unwrap(), "identity:*"),
"refresh granted {:?}",
refreshed["scope"]
);
}
+157 -9
View File
@@ -986,6 +986,162 @@ async fn parity_blob_duplicate_insert() {
assert_eq!(pg_dup, store_dup);
}
#[tokio::test(flavor = "multi_thread")]
async fn parity_blob_shared_between_repos() {
let f = ParityFixture::new().await;
let did_a = test_did("shareda");
let did_b = test_did("sharedb");
let (pg_a, store_a) = seed_repos(&f, &did_a, &test_handle("shareda")).await;
let (pg_b, store_b) = seed_repos(&f, &did_b, &test_handle("sharedb")).await;
let cid = test_cid(210);
let pg_first =
f.pg.blob
.insert_blob(&cid, "image/png", 100, pg_a, "blobs/shared.png")
.await
.unwrap();
let store_first = f
.store
.blob
.insert_blob(&cid, "image/png", 100, store_a, "blobs/shared.png")
.await
.unwrap();
assert_eq!(pg_first, store_first);
let pg_second =
f.pg.blob
.insert_blob(&cid, "image/png", 100, pg_b, "blobs/shared.png")
.await
.unwrap();
let store_second = f
.store
.blob
.insert_blob(&cid, "image/png", 100, store_b, "blobs/shared.png")
.await
.unwrap();
assert_eq!(pg_second, store_second);
assert!(pg_second.is_some());
for (pg_uid, store_uid) in [(pg_a, store_a), (pg_b, store_b)] {
assert_eq!(f.pg.blob.count_blobs_by_user(pg_uid).await.unwrap(), 1);
assert_eq!(
f.store.blob.count_blobs_by_user(store_uid).await.unwrap(),
1
);
assert_eq!(
f.pg.blob
.list_blobs_by_user(pg_uid, None, 100)
.await
.unwrap(),
vec![cid.clone()]
);
assert_eq!(
f.store
.blob
.list_blobs_by_user(store_uid, None, 100)
.await
.unwrap(),
vec![cid.clone()]
);
assert!(
f.pg.blob
.get_blob_storage_keys_by_user(pg_uid)
.await
.unwrap()
.is_empty()
);
assert!(
f.store
.blob
.get_blob_storage_keys_by_user(store_uid)
.await
.unwrap()
.is_empty()
);
}
}
#[tokio::test(flavor = "multi_thread")]
async fn parity_ensure_blob_ownership() {
let f = ParityFixture::new().await;
let did_a = test_did("ensurea");
let did_b = test_did("ensureb");
let (pg_a, store_a) = seed_repos(&f, &did_a, &test_handle("ensurea")).await;
let (pg_b, store_b) = seed_repos(&f, &did_b, &test_handle("ensureb")).await;
let cid = test_cid(211);
f.pg.blob
.insert_blob(&cid, "image/png", 100, pg_a, "blobs/ensure.png")
.await
.unwrap();
f.store
.blob
.insert_blob(&cid, "image/png", 100, store_a, "blobs/ensure.png")
.await
.unwrap();
assert!(f.pg.blob.ensure_blob_ownership(pg_b, &cid).await.unwrap());
assert!(
f.store
.blob
.ensure_blob_ownership(store_b, &cid)
.await
.unwrap()
);
assert!(!f.pg.blob.ensure_blob_ownership(pg_b, &cid).await.unwrap());
assert!(
!f.store
.blob
.ensure_blob_ownership(store_b, &cid)
.await
.unwrap()
);
let absent = test_cid(212);
assert!(
!f.pg
.blob
.ensure_blob_ownership(pg_b, &absent)
.await
.unwrap()
);
assert!(
!f.store
.blob
.ensure_blob_ownership(store_b, &absent)
.await
.unwrap()
);
for (pg_uid, store_uid) in [(pg_a, store_a), (pg_b, store_b)] {
assert_eq!(f.pg.blob.count_blobs_by_user(pg_uid).await.unwrap(), 1);
assert_eq!(
f.store.blob.count_blobs_by_user(store_uid).await.unwrap(),
1
);
assert!(
f.pg.blob
.get_blob_storage_keys_by_user(pg_uid)
.await
.unwrap()
.is_empty()
);
assert!(
f.store
.blob
.get_blob_storage_keys_by_user(store_uid)
.await
.unwrap()
.is_empty()
);
}
}
#[tokio::test]
async fn parity_get_all_records() {
let f = ParityFixture::new().await;
@@ -1567,7 +1723,7 @@ async fn parity_plc_tokens() {
}
#[tokio::test]
async fn parity_blob_delete_and_takedown() {
async fn parity_blob_takedown() {
let f = ParityFixture::new().await;
let did = test_did("blobdel");
let handle = test_handle("blobdel");
@@ -1604,14 +1760,6 @@ async fn parity_blob_delete_and_takedown() {
pg_with_td.as_ref().map(|b| b.takedown_ref.as_deref()),
store_with_td.as_ref().map(|b| b.takedown_ref.as_deref())
);
f.pg.blob.delete_blob_by_cid(&cid).await.unwrap();
f.store.blob.delete_blob_by_cid(&cid).await.unwrap();
let pg_meta = f.pg.blob.get_blob_metadata(&cid).await.unwrap();
let store_meta = f.store.blob.get_blob_metadata(&cid).await.unwrap();
assert!(pg_meta.is_none());
assert!(store_meta.is_none());
}
#[tokio::test]
+1 -1
View File
@@ -48,7 +48,7 @@ pub static SCOPE_DEFINITIONS: LazyLock<HashMap<&'static str, ScopeDefinition>> =
category: ScopeCategory::Transition,
required: false,
description: "Generic transition scope for compatibility",
display_name: "Transition Access",
display_name: "Generic Access",
},
ScopeDefinition {
scope: "transition:chat.bsky",
+3 -3
View File
@@ -16,7 +16,7 @@ pub use parser::{
ParsedScope, RepoAction, RepoScope, RpcScope, parse_scope, parse_scope_string,
};
pub use permission_set::{
ExpansionOutcome, FailedSet, FetchedSet, ResolveFailure, ResolvedSetGroup, ScopeExpansionError,
fetch_and_expand, parse_include_scope,
ExpansionOutcome, FailedSet, FetchedSet, RejectedScope, ResolveFailure, ResolvedSetGroup,
ScopeExpansionError, ScopeRejection, fetch_and_expand, parse_include_scope,
};
pub use permissions::ScopePermissions;
pub use permissions::{ScopePermissions, superseded_by_transition_generic};
@@ -44,6 +44,19 @@ pub enum ResolveFailure {
EmptyPermissions,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum ScopeRejection {
Unrecognized,
NotRegistered,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct RejectedScope {
pub scope: String,
pub reason: ScopeRejection,
}
#[derive(Debug, Clone)]
pub struct FailedSet {
// NSID and aud are left as strings to avoid issues from malformed requests.
@@ -61,11 +74,21 @@ pub struct ResolvedSetGroup {
pub expanded: Vec<String>,
}
impl ResolvedSetGroup {
pub fn include_token(&self) -> String {
match &self.aud {
Some(aud) => format!("include:{}?aud={}", self.nsid, aud),
None => format!("include:{}", self.nsid),
}
}
}
#[derive(Debug, Clone, Default)]
pub struct ExpansionOutcome {
pub passthrough: Vec<String>,
pub sets: Vec<ResolvedSetGroup>,
pub failures: Vec<FailedSet>,
pub rejected: Vec<RejectedScope>,
}
impl ExpansionOutcome {
@@ -87,6 +110,16 @@ impl ExpansionOutcome {
pub fn to_scope_string(&self) -> String {
self.flat_scopes().join(" ")
}
/// The scopes that survived filtering, as requested: passthrough scopes plus the `include:`
/// token of each resolved set, without expanding the sets.
pub fn unexpanded_scopes(&self) -> Vec<String> {
self.passthrough
.iter()
.cloned()
.chain(self.sets.iter().map(ResolvedSetGroup::include_token))
.collect()
}
}
#[derive(Debug, Deserialize)]
@@ -811,6 +844,7 @@ mod tests {
given_aud: None,
reason: ResolveFailure::NotFound,
}],
rejected: vec![],
};
let flat = out.flat_scopes();
assert_eq!(
@@ -839,6 +873,7 @@ mod tests {
expanded: vec!["repo:x".into(), "rpc:io.atcr.getManifest".into()],
}],
failures: vec![],
rejected: vec![],
};
let flat = out.flat_scopes();
assert_eq!(flat, vec!["repo:x", "rpc:io.atcr.getManifest"]);
+205 -32
View File
@@ -43,7 +43,26 @@ impl ScopePermissions {
has_transition_email,
}
}
}
/// Whether holding `transition:generic` makes `scope` redundant.
pub fn superseded_by_transition_generic(scope: &ParsedScope) -> bool {
match scope {
ParsedScope::Repo(_) | ParsedScope::Blob(_) => true,
ParsedScope::Rpc(rpc) => !rpc
.lxm
.as_deref()
.is_some_and(|lxm| lxm == "*" || lxm.starts_with("chat.bsky.")),
ParsedScope::Account(_)
| ParsedScope::Identity(_)
| ParsedScope::TransitionEmail
| ParsedScope::TransitionChat => false,
ParsedScope::Include(_) => false,
ParsedScope::TransitionGeneric | ParsedScope::Atproto | ParsedScope::Unknown(_) => false,
}
}
impl ScopePermissions {
pub fn has_scope(&self, scope: &str) -> bool {
self.scopes.contains(scope)
}
@@ -158,22 +177,17 @@ impl ScopePermissions {
}
pub fn assert_rpc(&self, aud: &str, lxm: &Nsid) -> Result<(), ScopeError> {
if lxm.starts_with("chat.bsky.") {
if self.has_transition_chat {
return Ok(());
}
if self.has_transition_generic && !self.has_transition_chat {
return Err(ScopeError::InsufficientScope {
required: "transition:chat.bsky".to_string(),
message: format!(
"Chat access requires transition:chat.bsky scope to call {}",
lxm
),
});
}
let is_chat = lxm.starts_with("chat.bsky.");
if is_chat && self.has_transition_chat {
return Ok(());
}
if self.has_transition_generic {
// `transition:generic` covers every lexicon except chat. Note it does not *block* chat:
// holding it must never remove access a granular `rpc:chat.bsky.*` scope would grant on
// its own, so chat requests fall through to the granular check below rather than
// failing here.
if self.has_transition_generic && !is_chat {
return Ok(());
}
@@ -198,13 +212,24 @@ impl ScopePermissions {
});
if has_permission {
Ok(())
} else {
Err(ScopeError::InsufficientScope {
return Ok(());
}
// Point a caller holding only `transition:generic` at the scope it actually needs,
// rather than at a granular rpc scope it probably did not mean to request.
Err(match is_chat && self.has_transition_generic {
true => ScopeError::InsufficientScope {
required: "transition:chat.bsky".to_string(),
message: format!(
"Chat access requires transition:chat.bsky scope to call {}",
lxm
),
},
false => ScopeError::InsufficientScope {
required: format!("rpc:{}?aud={}", lxm, aud),
message: format!("Insufficient scope to call {} on {}", lxm, aud),
})
}
},
})
}
pub fn assert_account(
@@ -212,10 +237,6 @@ impl ScopePermissions {
attr: AccountAttr,
action: AccountAction,
) -> Result<(), ScopeError> {
if self.has_transition_generic {
return Ok(());
}
if attr == AccountAttr::Email && action == AccountAction::Read && self.has_transition_email
{
return Ok(());
@@ -245,8 +266,7 @@ impl ScopePermissions {
}
pub fn allows_email_read(&self) -> bool {
self.has_transition_generic
|| self.has_transition_email
self.has_transition_email
|| self
.find_account_scopes()
.any(|a| a.attr == AccountAttr::Email || a.attr == AccountAttr::Wildcard)
@@ -269,10 +289,6 @@ impl ScopePermissions {
}
pub fn assert_identity(&self, attr: IdentityAttr) -> Result<(), ScopeError> {
if self.has_transition_generic {
return Ok(());
}
let has_permission = self.find_identity_scopes().any(|identity_scope| {
identity_scope.attr == IdentityAttr::Wildcard || identity_scope.attr == attr
});
@@ -336,6 +352,7 @@ impl Default for ScopePermissions {
#[cfg(test)]
mod tests {
use super::*;
use crate::parser::parse_scope;
fn c(s: &str) -> Nsid {
s.parse().unwrap()
@@ -512,10 +529,10 @@ mod tests {
}
#[test]
fn test_transition_generic_grants_identity() {
fn test_transition_generic_does_not_grant_identity() {
let perms = ScopePermissions::from_scope_string(Some("transition:generic"));
assert!(perms.allows_identity(IdentityAttr::Handle));
assert!(perms.allows_identity(IdentityAttr::Wildcard));
assert!(!perms.allows_identity(IdentityAttr::Handle));
assert!(!perms.allows_identity(IdentityAttr::Wildcard));
}
#[test]
@@ -597,4 +614,160 @@ mod tests {
&c("app.bsky.feed.getAuthorFeed")
));
}
#[test]
fn transition_generic_supersedes_granular_scopes() {
for scope in [
"repo:app.bsky.feed.post?action=create",
"blob:image/png",
"rpc:app.bsky.actor.getProfile?aud=*",
] {
assert!(
superseded_by_transition_generic(&parse_scope(scope)),
"{scope} should be superseded by transition:generic"
);
}
}
#[test]
fn transition_generic_does_not_supersede_chat() {
// assert_rpc rejects chat.bsky.* when transition:generic is held without
// transition:chat.bsky, so neither the transition scope nor an rpc scope that
// could reach a chat lexicon is covered by it.
for scope in [
"transition:chat.bsky",
"rpc:chat.bsky.convo.sendMessage?aud=*",
"rpc:*?aud=did:web:api.bsky.app",
"account:email?action=manage",
"account:email?action=read",
"account:status?action=read",
"identity:handle",
"identity:*",
"transition:email",
] {
assert!(
!superseded_by_transition_generic(&parse_scope(scope)),
"{scope} must not be treated as superseded"
);
}
}
#[test]
fn transition_generic_does_not_supersede_itself_or_baseline() {
assert!(!superseded_by_transition_generic(&parse_scope(
"transition:generic"
)));
assert!(!superseded_by_transition_generic(&parse_scope("atproto")));
}
#[test]
fn superseded_matches_enforcement_for_chat_and_feed() {
// Cross-check against ScopePermissions so the two cannot drift apart.
let perms = ScopePermissions::from_scope_string(Some("atproto transition:generic"));
let feed = Nsid::new("app.bsky.feed.getTimeline").unwrap();
let chat = Nsid::new("chat.bsky.convo.sendMessage").unwrap();
assert!(perms.allows_rpc("did:web:api.bsky.app", &feed));
assert!(!perms.allows_rpc("did:web:api.bsky.app", &chat));
}
#[test]
fn granular_chat_rpc_works_without_transition_generic() {
// Baseline for the test below: on its own, a granular chat rpc scope grants chat.
let perms = ScopePermissions::from_scope_string(Some(
"atproto rpc:chat.bsky.convo.sendMessage?aud=*",
));
assert!(perms.allows_rpc("did:web:api.bsky.chat", &c("chat.bsky.convo.sendMessage")));
}
#[test]
fn transition_generic_does_not_revoke_granular_chat_rpc() {
// Adding a broader scope must never remove access. transition:generic does not cover
// chat lexicons, but it must not stop a granular chat rpc scope from doing so either.
let perms = ScopePermissions::from_scope_string(Some(
"atproto transition:generic rpc:chat.bsky.convo.sendMessage?aud=*",
));
assert!(perms.allows_rpc("did:web:api.bsky.chat", &c("chat.bsky.convo.sendMessage")));
// ...and still grants everything else it covers.
assert!(perms.allows_rpc("did:web:api.bsky.app", &c("app.bsky.feed.getTimeline")));
}
#[test]
fn transition_generic_does_not_widen_granular_chat_rpc() {
// The granular scope grants exactly one chat lexicon; transition:generic must not be
// read as covering the rest of chat.
let perms = ScopePermissions::from_scope_string(Some(
"atproto transition:generic rpc:chat.bsky.convo.sendMessage?aud=*",
));
assert!(!perms.allows_rpc("did:web:api.bsky.chat", &c("chat.bsky.convo.deleteMessage")));
}
#[test]
fn chat_denial_still_names_the_scope_the_caller_needs() {
// transition:generic alone: the useful advice is "ask for transition:chat.bsky",
// not "ask for rpc:chat.bsky.convo.listConvos".
let generic = ScopePermissions::from_scope_string(Some("atproto transition:generic"));
let err = generic
.assert_rpc("did:web:api.bsky.chat", &c("chat.bsky.convo.listConvos"))
.expect_err("chat must be denied without transition:chat.bsky");
match err {
ScopeError::InsufficientScope { required, .. } => {
assert_eq!(required, "transition:chat.bsky");
}
other => panic!("unexpected error: {other:?}"),
}
// Without transition:generic the granular scope is the right thing to name.
let bare = ScopePermissions::from_scope_string(Some("atproto"));
let err = bare
.assert_rpc("did:web:api.bsky.chat", &c("chat.bsky.convo.listConvos"))
.expect_err("chat must be denied with no rpc scope at all");
match err {
ScopeError::InsufficientScope { required, .. } => {
assert!(required.starts_with("rpc:chat.bsky.convo.listConvos"));
}
other => panic!("unexpected error: {other:?}"),
}
}
#[test]
fn transition_generic_does_not_grant_account_management() {
// "no account management actions: change handle, change email, delete or deactivate
// account, migrate account" -- atproto OAuth spec.
let perms = ScopePermissions::from_scope_string(Some("atproto transition:generic"));
assert!(!perms.allows_account(AccountAttr::Email, AccountAction::Manage));
assert!(!perms.allows_account(AccountAttr::Repo, AccountAction::Manage));
assert!(!perms.allows_account(AccountAttr::Status, AccountAction::Manage));
}
#[test]
fn transition_generic_does_not_grant_email_read() {
// Reading the account email is what transition:email is for.
let perms = ScopePermissions::from_scope_string(Some("atproto transition:generic"));
assert!(!perms.allows_email_read());
assert!(!perms.allows_account(AccountAttr::Email, AccountAction::Read));
}
#[test]
fn granular_scopes_still_grant_alongside_transition_generic() {
// Removing the short-circuit must not stop an explicitly granted scope from working.
let perms = ScopePermissions::from_scope_string(Some(
"atproto transition:generic account:email?action=manage identity:handle",
));
assert!(perms.allows_account(AccountAttr::Email, AccountAction::Manage));
assert!(perms.allows_identity(IdentityAttr::Handle));
let with_email = ScopePermissions::from_scope_string(Some(
"atproto transition:generic transition:email",
));
assert!(with_email.allows_email_read());
}
#[test]
fn transition_generic_still_grants_what_the_spec_says_it_does() {
let perms = ScopePermissions::from_scope_string(Some("atproto transition:generic"));
assert!(perms.allows_repo(RepoAction::Create, &c("app.bsky.feed.post")));
assert!(perms.allows_repo(RepoAction::Delete, &c("app.bsky.feed.post")));
assert!(perms.allows_blob("image/png"));
assert!(perms.allows_rpc("did:web:api.bsky.app", &c("app.bsky.feed.getTimeline")));
}
}
+17 -3
View File
@@ -10,7 +10,8 @@ use tranquil_pds::comms::{CommsService, DiscordSender, EmailSender, SignalSender
use tranquil_pds::crawlers::{Crawlers, start_crawlers_service};
use tranquil_pds::scheduled::{
backfill_record_blobs, backfill_repo_rev, backfill_user_blocks, start_scheduled_tasks,
backfill_blob_ownership, backfill_record_blobs, backfill_repo_rev, backfill_user_blocks,
start_scheduled_tasks,
};
use tranquil_pds::state::AppState;
@@ -77,9 +78,12 @@ async fn main() -> ExitCode {
}
config
.server
.user_handle_domain_list()
.user_handle_domains
.iter()
.filter(|d| !tranquil_pds::api::validation::domain_forms_valid_handles(d))
.flatten()
.filter(|d| {
!tranquil_pds::api::validation::domain_forms_valid_handles(d.as_str())
})
.for_each(|d| {
eprintln!(
"account creation under handle domain {d} will be rejected because its TLD is reserved"
@@ -192,11 +196,21 @@ async fn run() -> Result<(), Box<dyn std::error::Error>> {
let backfill_repo_repo = state.repos.repo.clone();
let backfill_block_store = state.block_store.clone();
let ownership_repo_repo = state.repos.repo.clone();
let ownership_infra_repo = state.repos.infra.clone();
let ownership_blob_repo = state.repos.blob.clone();
let ownership_block_store = state.block_store.clone();
tokio::spawn(async move {
tokio::join!(
backfill_repo_rev(backfill_repo_repo.clone(), backfill_block_store.clone()),
backfill_user_blocks(backfill_repo_repo.clone(), backfill_block_store.clone()),
backfill_record_blobs(backfill_repo_repo, backfill_block_store),
backfill_blob_ownership(
ownership_infra_repo,
ownership_repo_repo,
ownership_blob_repo,
ownership_block_store
)
);
});
+36 -16
View File
@@ -114,6 +114,7 @@ mod s3 {
pub struct S3BlobStorage {
client: Client,
bucket: String,
path: String,
}
impl S3BlobStorage {
@@ -125,12 +126,23 @@ mod s3 {
.clone()
.expect("storage.s3_bucket (S3_BUCKET) must be set");
let client = create_s3_client().await;
Self { client, bucket }
let path = cfg.storage.s3_path
.trim_start_matches("/")
.trim_end_matches("/")
.to_string();
Self {
client,
bucket,
path,
}
}
pub async fn with_bucket(bucket: String) -> Self {
let client = create_s3_client().await;
Self { client, bucket }
fn resolve_path(&self, key: &str) -> String {
if self.path.is_empty() {
return key.to_string()
}
format!("{}/{}", self.path, key)
}
}
@@ -165,10 +177,11 @@ mod s3 {
}
async fn put_bytes(&self, key: &str, data: Bytes) -> Result<(), StorageError> {
let path = self.resolve_path(key);
self.client
.put_object()
.bucket(&self.bucket)
.key(key)
.key(&path)
.body(ByteStream::from(data))
.send()
.await
@@ -182,11 +195,12 @@ mod s3 {
}
async fn get_bytes(&self, key: &str) -> Result<Bytes, StorageError> {
let path = self.resolve_path(key);
let resp = self
.client
.get_object()
.bucket(&self.bucket)
.key(key)
.key(&path)
.send()
.await
.map_err(|e| StorageError::Backend(e.to_string()))?;
@@ -199,12 +213,13 @@ mod s3 {
}
async fn get_head(&self, key: &str, size: usize) -> Result<Bytes, StorageError> {
let path = self.resolve_path(key);
let range = format!("bytes=0-{}", size.saturating_sub(1));
let resp = self
.client
.get_object()
.bucket(&self.bucket)
.key(key)
.key(&path)
.range(range)
.send()
.await
@@ -218,10 +233,11 @@ mod s3 {
}
async fn delete(&self, key: &str) -> Result<(), StorageError> {
let path = self.resolve_path(key);
self.client
.delete_object()
.bucket(&self.bucket)
.key(key)
.key(&path)
.send()
.await
.map_err(|e| StorageError::Backend(e.to_string()))?;
@@ -236,11 +252,12 @@ mod s3 {
) -> Result<StreamUploadResult, StorageError> {
use futures::StreamExt;
let path = self.resolve_path(key);
let create_resp = self
.client
.create_multipart_upload()
.bucket(&self.bucket)
.key(key)
.key(&path)
.send()
.await
.map_err(|e| {
@@ -263,13 +280,13 @@ mod s3 {
> {
let client = client.clone();
let bucket = bucket.to_string();
let key = key.to_string();
let path = self.resolve_path(key);
let upload_id = upload_id.to_string();
Box::pin(async move {
let resp = client
.upload_part()
.bucket(&bucket)
.key(&key)
.key(&path)
.upload_id(&upload_id)
.part_number(part_num)
.body(ByteStream::from(data))
@@ -314,7 +331,7 @@ mod s3 {
.client
.abort_multipart_upload()
.bucket(&self.bucket)
.key(key)
.key(path)
.upload_id(&upload_id)
.send()
.await;
@@ -384,10 +401,11 @@ mod s3 {
.set_parts(Some(state.completed_parts))
.build();
let path = self.resolve_path(key);
self.client
.complete_multipart_upload()
.bucket(&self.bucket)
.key(key)
.key(&path)
.upload_id(&upload_id)
.multipart_upload(completed_upload)
.send()
@@ -404,13 +422,15 @@ mod s3 {
}
async fn copy(&self, src_key: &str, dst_key: &str) -> Result<(), StorageError> {
let copy_source = format!("{}/{}", self.bucket, src_key);
let src_path = self.resolve_path(src_key);
let copy_source = format!("{}/{}", self.bucket, &src_path);
let dst_path = self.resolve_path(dst_key);
self.client
.copy_object()
.bucket(&self.bucket)
.copy_source(&copy_source)
.key(dst_key)
.copy_source(copy_source)
.key(&dst_path)
.send()
.await
.map_err(|e| StorageError::Backend(format!("Failed to copy object: {}", e)))?;
+212 -165
View File
@@ -7,7 +7,10 @@ use smallvec::SmallVec;
use uuid::Uuid;
use super::MetastoreError;
use super::blobs::{BlobMetaValue, blob_by_cid_key, blob_meta_key, blob_user_prefix, blobs_prefix};
use super::blobs::{
BlobContentValue, BlobMetaValue, blob_by_cid_key, blob_by_cid_prefix, blob_meta_key,
blob_user_prefix,
};
use super::commit_ops::{RecordBlobsValue, record_blobs_user_prefix};
use super::encoding::{KeyReader, exclusive_upper_bound};
use super::keys::{KeyTag, UserHash};
@@ -22,14 +25,21 @@ pub struct BlobOps {
db: Database,
repo_data: Keyspace,
user_hashes: Arc<UserHashMap>,
counter_lock: Arc<parking_lot::Mutex<()>>,
}
impl BlobOps {
pub fn new(db: Database, repo_data: Keyspace, user_hashes: Arc<UserHashMap>) -> Self {
pub fn new(
db: Database,
repo_data: Keyspace,
user_hashes: Arc<UserHashMap>,
counter_lock: Arc<parking_lot::Mutex<()>>,
) -> Self {
Self {
db,
repo_data,
user_hashes,
counter_lock,
}
}
@@ -47,6 +57,7 @@ impl BlobOps {
created_by_user: Uuid,
storage_key: &str,
) -> Result<Option<CidLink>, MetastoreError> {
let _guard = self.counter_lock.lock();
if size_bytes < 0 {
return Err(MetastoreError::InvalidInput(
"size_bytes must be non-negative",
@@ -55,69 +66,99 @@ impl BlobOps {
let user_hash = self.resolve_user_hash(created_by_user)?;
let cid_str = cid.as_str();
let cid_index_key = blob_by_cid_key(cid_str);
let existing = self
let marker_key = blob_meta_key(user_hash, cid_str);
if self
.repo_data
.get(cid_index_key.as_slice())
.map_err(MetastoreError::Fjall)?;
if existing.is_some() {
.get(marker_key.as_slice())
.map_err(MetastoreError::Fjall)?
.is_some()
{
return Ok(None);
}
let value = BlobMetaValue {
size_bytes,
mime_type: mime_type.to_owned(),
storage_key: storage_key.to_owned(),
takedown_ref: None,
created_at_ms: chrono::Utc::now().timestamp_millis(),
let cid_index_key = blob_by_cid_key(cid_str);
let content = match point_lookup(
&self.repo_data,
cid_index_key.as_slice(),
BlobContentValue::deserialize,
"corrupt blob_content value",
)? {
Some(mut existing) => {
existing.ref_count = existing.ref_count.saturating_add(1);
existing
}
None => BlobContentValue {
meta: BlobMetaValue {
size_bytes,
mime_type: mime_type.to_owned(),
storage_key: storage_key.to_owned(),
takedown_ref: None,
created_at_ms: chrono::Utc::now().timestamp_millis(),
},
ref_count: 1,
},
};
let primary_key = blob_meta_key(user_hash, cid_str);
let mut batch = self.db.batch();
batch.insert(&self.repo_data, primary_key.as_slice(), value.serialize());
batch.insert(&self.repo_data, marker_key.as_slice(), &[] as &[u8]);
batch.insert(
&self.repo_data,
cid_index_key.as_slice(),
user_hash.raw().to_be_bytes(),
content.serialize(),
);
batch.commit().map_err(MetastoreError::Fjall)?;
Ok(Some(cid.clone()))
}
fn lookup_user_hash_by_cid(&self, cid_str: &str) -> Result<Option<UserHash>, MetastoreError> {
let key = blob_by_cid_key(cid_str);
match self
pub fn ensure_blob_ownership(
&self,
user_id: Uuid,
cid: &CidLink,
) -> Result<bool, MetastoreError> {
let _guard = self.counter_lock.lock();
let user_hash = self.resolve_user_hash(user_id)?;
let cid_str = cid.as_str();
let marker_key = blob_meta_key(user_hash, cid_str);
if self
.repo_data
.get(key.as_slice())
.get(marker_key.as_slice())
.map_err(MetastoreError::Fjall)?
.is_some()
{
Some(raw) => {
let arr: [u8; 8] = raw
.as_ref()
.try_into()
.map_err(|_| MetastoreError::CorruptData("blob_by_cid value not 8 bytes"))?;
Ok(Some(UserHash::from_raw(u64::from_be_bytes(arr))))
}
None => Ok(None),
return Ok(false);
}
let cid_index_key = blob_by_cid_key(cid_str);
let Some(mut content) = self.get_blob_content(cid)? else {
return Ok(false);
};
content.ref_count = content.ref_count.saturating_add(1);
let mut batch = self.db.batch();
batch.insert(&self.repo_data, marker_key.as_slice(), &[] as &[u8]);
batch.insert(
&self.repo_data,
cid_index_key.as_slice(),
content.serialize(),
);
batch.commit().map_err(MetastoreError::Fjall)?;
Ok(true)
}
fn get_blob_content(&self, cid: &CidLink) -> Result<Option<BlobContentValue>, MetastoreError> {
point_lookup(
&self.repo_data,
blob_by_cid_key(cid.as_str()).as_slice(),
BlobContentValue::deserialize,
"corrupt blob_content value",
)
}
fn get_blob_value(&self, cid: &CidLink) -> Result<Option<BlobMetaValue>, MetastoreError> {
let cid_str = cid.as_str();
let user_hash = match self.lookup_user_hash_by_cid(cid_str)? {
Some(h) => h,
None => return Ok(None),
};
let key = blob_meta_key(user_hash, cid_str);
point_lookup(
&self.repo_data,
key.as_slice(),
BlobMetaValue::deserialize,
"corrupt blob_meta value",
)
Ok(self.get_blob_content(cid)?.map(|c| c.meta))
}
pub fn get_blob_metadata(
@@ -186,14 +227,14 @@ impl BlobOps {
}
pub fn sum_blob_storage(&self) -> Result<i64, MetastoreError> {
let prefix = blobs_prefix();
let prefix = blob_by_cid_prefix();
self.repo_data
.prefix(prefix.as_slice())
.try_fold(0i64, |acc, guard| {
let (_, val_bytes) = guard.into_inner().map_err(MetastoreError::Fjall)?;
let value = BlobMetaValue::deserialize(&val_bytes)
.ok_or(MetastoreError::CorruptData("corrupt blob_meta in sum"))?;
Ok::<_, MetastoreError>(acc.saturating_add(value.size_bytes))
let content = BlobContentValue::deserialize(&val_bytes)
.ok_or(MetastoreError::CorruptData("corrupt blob_content in sum"))?;
Ok::<_, MetastoreError>(acc.saturating_add(content.meta.size_bytes))
})
}
@@ -202,60 +243,27 @@ impl BlobOps {
cid: &CidLink,
takedown_ref: Option<&str>,
) -> Result<bool, MetastoreError> {
let cid_str = cid.as_str();
let user_hash = match self.lookup_user_hash_by_cid(cid_str)? {
Some(h) => h,
let _guard = self.counter_lock.lock();
let mut content = match self.get_blob_content(cid)? {
Some(c) => c,
None => return Ok(false),
};
let key = blob_meta_key(user_hash, cid_str);
let mut value = match point_lookup(
content.meta.takedown_ref = takedown_ref.map(str::to_owned);
let mut batch = self.db.batch();
batch.insert(
&self.repo_data,
key.as_slice(),
BlobMetaValue::deserialize,
"corrupt blob_meta value",
)? {
Some(v) => v,
None => return Ok(false),
};
value.takedown_ref = takedown_ref.map(str::to_owned);
let mut batch = self.db.batch();
batch.insert(&self.repo_data, key.as_slice(), value.serialize());
blob_by_cid_key(cid.as_str()).as_slice(),
content.serialize(),
);
batch.commit().map_err(MetastoreError::Fjall)?;
Ok(true)
}
pub fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<bool, MetastoreError> {
let cid_str = cid.as_str();
let user_hash = match self.lookup_user_hash_by_cid(cid_str)? {
Some(h) => h,
None => return Ok(false),
};
let primary_key = blob_meta_key(user_hash, cid_str);
let exists = self
.repo_data
.get(primary_key.as_slice())
.map_err(MetastoreError::Fjall)?
.is_some();
if !exists {
return Ok(false);
}
let cid_index_key = blob_by_cid_key(cid_str);
let mut batch = self.db.batch();
batch.remove(&self.repo_data, primary_key.as_slice());
batch.remove(&self.repo_data, cid_index_key.as_slice());
batch.commit().map_err(MetastoreError::Fjall)?;
Ok(true)
}
pub fn delete_blobs_by_user(&self, user_id: Uuid) -> Result<u64, MetastoreError> {
let _guard = self.counter_lock.lock();
let user_hash = self.resolve_user_hash(user_id)?;
let prefix = blob_user_prefix(user_hash);
let user_hash_bytes = user_hash.raw().to_be_bytes();
let (final_batch, remaining, total) = self
.repo_data
@@ -273,14 +281,25 @@ impl BlobOps {
blob_meta_key(user_hash, &cid_str).as_slice(),
);
let cid_index_key = blob_by_cid_key(&cid_str);
let owns_cid = self
.repo_data
.get(cid_index_key.as_slice())
.map_err(MetastoreError::Fjall)?
.is_some_and(|raw| raw.as_ref() == user_hash_bytes);
if owns_cid {
batch.remove(&self.repo_data, cid_index_key.as_slice());
if let Some(mut content) = point_lookup(
&self.repo_data,
cid_index_key.as_slice(),
BlobContentValue::deserialize,
"corrupt blob_content value",
)? {
content.ref_count = content.ref_count.saturating_sub(1);
if content.ref_count == 0 {
batch.remove(&self.repo_data, cid_index_key.as_slice());
} else {
batch.insert(
&self.repo_data,
cid_index_key.as_slice(),
content.serialize(),
);
}
}
let new_count = count + 1;
if new_count >= DELETE_BATCH_SIZE {
batch.commit().map_err(MetastoreError::Fjall)?;
@@ -311,11 +330,14 @@ impl BlobOps {
self.repo_data
.prefix(prefix.as_slice())
.map(|guard| {
let (_, val_bytes) = guard.into_inner().map_err(MetastoreError::Fjall)?;
let value = BlobMetaValue::deserialize(&val_bytes)
.ok_or(MetastoreError::CorruptData("corrupt blob_meta value"))?;
Ok(value.storage_key)
let (key_bytes, _) = guard.into_inner().map_err(MetastoreError::Fjall)?;
let cid = parse_blob_cid_from_key(key_bytes.as_ref())?;
Ok(self
.get_blob_content(&cid)?
.filter(|c| c.ref_count == 1)
.map(|c| c.meta.storage_key))
})
.filter_map(Result::transpose)
.collect()
}
@@ -346,10 +368,9 @@ impl BlobOps {
if acc.contains_key(&cid_str) {
return Ok(());
}
let key = blob_meta_key(user_hash, &cid_str);
let exists = self
.repo_data
.get(key.as_slice())
.get(blob_meta_key(user_hash, &cid_str).as_slice())
.map_err(MetastoreError::Fjall)?
.is_some();
if !exists {
@@ -415,17 +436,11 @@ impl BlobOps {
Ok(c) => c,
Err(e) => return Some(Err(e)),
};
let key = blob_meta_key(user_hash, cid_link.as_str());
match point_lookup(
&self.repo_data,
key.as_slice(),
BlobMetaValue::deserialize,
"corrupt blob_meta value",
) {
Ok(Some(v)) => Some(Ok(tranquil_db_traits::BlobForExport {
match self.get_blob_content(&cid_link) {
Ok(Some(c)) => Some(Ok(tranquil_db_traits::BlobForExport {
cid: cid_link,
storage_key: v.storage_key,
mime_type: v.mime_type,
storage_key: c.meta.storage_key,
mime_type: c.meta.mime_type,
})),
Ok(None) => None,
Err(e) => Some(Err(e)),
@@ -537,26 +552,6 @@ mod tests {
);
}
#[test]
fn insert_same_cid_different_user_returns_none() {
let (_dir, ms) = open_fresh();
let (user_a, _) = setup_user(&ms);
let (user_b, _) = setup_user(&ms);
let ops = ms.blob_ops();
let cid = test_cid_link(80);
assert!(
ops.insert_blob(&cid, "image/png", 100, user_a, "ka")
.unwrap()
.is_some()
);
assert!(
ops.insert_blob(&cid, "image/png", 100, user_b, "kb")
.unwrap()
.is_none()
);
}
#[test]
fn get_blob_with_takedown_no_takedown() {
let (_dir, ms) = open_fresh();
@@ -676,37 +671,6 @@ mod tests {
assert_eq!(ops.sum_blob_storage().unwrap(), 350);
}
#[test]
fn delete_blob_by_cid() {
let (_dir, ms) = open_fresh();
let (user_id, _) = setup_user(&ms);
let ops = ms.blob_ops();
let cid = test_cid_link(40);
ops.insert_blob(&cid, "image/png", 100, user_id, "k")
.unwrap();
assert!(ops.delete_blob_by_cid(&cid).unwrap());
assert!(ops.get_blob_metadata(&cid).unwrap().is_none());
assert!(!ops.delete_blob_by_cid(&cid).unwrap());
}
#[test]
fn delete_blob_cleans_up_indexes() {
let (_dir, ms) = open_fresh();
let (user_id, _) = setup_user(&ms);
let ops = ms.blob_ops();
let cid = test_cid_link(41);
ops.insert_blob(&cid, "image/png", 100, user_id, "storage/abc")
.unwrap();
assert!(ops.get_blob_storage_key(&cid).unwrap().is_some());
ops.delete_blob_by_cid(&cid).unwrap();
assert!(ops.lookup_user_hash_by_cid(cid.as_str()).unwrap().is_none());
}
#[test]
fn delete_blobs_by_user() {
let (_dir, ms) = open_fresh();
@@ -735,7 +699,7 @@ mod tests {
ops.delete_blobs_by_user(user_id).unwrap();
assert!(ops.lookup_user_hash_by_cid(cid.as_str()).unwrap().is_none());
assert!(ops.get_blob_metadata(&cid).unwrap().is_none());
}
#[test]
@@ -787,4 +751,87 @@ mod tests {
assert_eq!(ops.count_blobs_by_user(user_b).unwrap(), 1);
assert_eq!(ops.sum_blob_storage().unwrap(), 30);
}
#[test]
fn blob_shared_between_users() {
let (_dir, ms) = open_fresh();
let (user_a, _) = setup_user(&ms);
let (user_b, _) = setup_user(&ms);
let ops = ms.blob_ops();
let cid = test_cid_link(80);
assert!(
ops.insert_blob(&cid, "a/b", 10, user_a, "k")
.unwrap()
.is_some()
);
assert!(
ops.insert_blob(&cid, "a/b", 10, user_b, "k")
.unwrap()
.is_some()
);
assert!(
ops.insert_blob(&cid, "a/b", 10, user_b, "k")
.unwrap()
.is_none()
);
assert_eq!(ops.count_blobs_by_user(user_a).unwrap(), 1);
assert_eq!(ops.count_blobs_by_user(user_b).unwrap(), 1);
assert_eq!(
ops.list_blobs_by_user(user_b, None, 100).unwrap(),
vec![cid.clone()]
);
assert_eq!(ops.sum_blob_storage().unwrap(), 10);
assert!(
ops.get_blob_storage_keys_by_user(user_a)
.unwrap()
.is_empty()
);
ops.delete_blobs_by_user(user_a).unwrap();
assert_eq!(ops.count_blobs_by_user(user_b).unwrap(), 1);
assert!(ops.get_blob_metadata(&cid).unwrap().is_some());
assert_eq!(
ops.get_blob_storage_keys_by_user(user_b).unwrap(),
vec!["k".to_string()]
);
}
#[test]
fn ensure_blob_ownership_grants_to_second_user() {
let (_dir, ms) = open_fresh();
let (user_a, _) = setup_user(&ms);
let (user_b, _) = setup_user(&ms);
let ops = ms.blob_ops();
let cid = test_cid_link(81);
ops.insert_blob(&cid, "a/b", 10, user_a, "k").unwrap();
assert!(ops.ensure_blob_ownership(user_b, &cid).unwrap());
assert!(!ops.ensure_blob_ownership(user_b, &cid).unwrap());
assert_eq!(ops.count_blobs_by_user(user_b).unwrap(), 1);
assert_eq!(ops.sum_blob_storage().unwrap(), 10);
assert!(
ops.get_blob_storage_keys_by_user(user_a)
.unwrap()
.is_empty()
);
}
#[test]
fn ensure_blob_ownership_ignores_absent_blob() {
let (_dir, ms) = open_fresh();
let (user_id, _) = setup_user(&ms);
let ops = ms.blob_ops();
assert!(
!ops.ensure_blob_ownership(user_id, &test_cid_link(82))
.unwrap()
);
assert_eq!(ops.count_blobs_by_user(user_id).unwrap(), 0);
}
}
@@ -33,6 +33,33 @@ impl BlobMetaValue {
}
}
const BLOB_CONTENT_SCHEMA_VERSION: u8 = 1;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct BlobContentValue {
pub meta: BlobMetaValue,
pub ref_count: u32,
}
impl BlobContentValue {
pub fn serialize(&self) -> Vec<u8> {
let payload =
postcard::to_allocvec(self).expect("BlobContentValue serialization cannot fail");
let mut buf = Vec::with_capacity(1 + payload.len());
buf.push(BLOB_CONTENT_SCHEMA_VERSION);
buf.extend_from_slice(&payload);
buf
}
pub fn deserialize(bytes: &[u8]) -> Option<Self> {
let (&version, payload) = bytes.split_first()?;
match version {
BLOB_CONTENT_SCHEMA_VERSION => postcard::from_bytes(payload).ok(),
_ => None,
}
}
}
pub fn blob_meta_key(user_hash: UserHash, cid_str: &str) -> SmallVec<[u8; 128]> {
KeyBuilder::new()
.tag(KeyTag::BLOBS)
@@ -59,6 +86,10 @@ pub fn blob_by_cid_key(cid_str: &str) -> SmallVec<[u8; 128]> {
.build()
}
pub fn blob_by_cid_prefix() -> SmallVec<[u8; 128]> {
KeyBuilder::new().tag(KeyTag::BLOB_BY_CID).build()
}
#[cfg(test)]
mod tests {
use super::*;
+31 -42
View File
@@ -14,17 +14,18 @@ use tranquil_db_traits::{
InviteCodeSortOrder, InviteCodeUse, MigrationReactivationError, MigrationReactivationInput,
NotificationHistoryRow, NotificationPrefs, OAuthTokenWithUser, PasswordResetResult,
PlcTokenInfo, PruneCount, QueuedComms, ReactivatedAccountInfo, RecoverPasskeyAccountInput,
RecoverPasskeyAccountResult, RepoAccountInfo, RepoInfo, RepoListItem, RepoWithoutRev,
ReservedSigningKey, ReservedSigningKeyFull, ScheduledDeletionAccount, ScopePreference,
SequenceNumber, SequencedEvent, StoredBackupCode, StoredPasskey, TokenFamilyId, TotpRecord,
TotpRecordState, User2faStatus, UserAuthInfo, UserCommsPrefs, UserConfirmSignup,
UserDidWebInfo, UserEmailInfo, UserForDeletion, UserForDidDoc, UserForDidDocBuild,
UserForPasskeyRecovery, UserForPasskeySetup, UserForRecovery, UserForVerification,
UserIdAndHandle, UserIdAndPasswordHash, UserIdHandleEmail, UserInfoForAuth, UserKeyInfo,
UserKeyWithId, UserLegacyLoginPref, UserLoginCheck, UserLoginFull, UserLoginInfo,
UserNeedingRecordBlobsBackfill, UserPasswordInfo, UserResendVerification, UserResetCodeInfo,
UserRow, UserSessionInfo, UserStatus, UserVerificationInfo, UserWithKey, UserWithoutBlocks,
ValidatedInviteCode, WebauthnChallengeType,
RecoverPasskeyAccountResult, RepoAccountInfo, RepoIdentity, RepoInfo, RepoListItem,
RepoWithoutRev, ReservedSigningKey, ReservedSigningKeyFull, ScheduledDeletionAccount,
ScopePreference, SequenceNumber, SequencedEvent, StoredBackupCode, StoredPasskey,
TokenFamilyId, TotpRecord, TotpRecordState, User2faStatus, UserAuthInfo, UserCommsPrefs,
UserConfirmSignup, UserDidWebInfo, UserEmailInfo, UserForDeletion, UserForDidDoc,
UserForDidDocBuild, UserForPasskeyRecovery, UserForPasskeySetup, UserForRecovery,
UserForVerification, UserIdAndHandle, UserIdAndPasswordHash, UserIdHandleEmail,
UserInfoForAuth, UserKeyInfo, UserKeyWithId, UserLegacyLoginPref, UserLoginCheck,
UserLoginFull, UserLoginInfo, UserNeedingRecordBlobsBackfill, UserPasswordInfo,
UserResendVerification, UserResetCodeInfo, UserRow, UserSessionInfo, UserStatus,
UserVerificationInfo, UserWithKey, UserWithoutBlocks, ValidatedInviteCode,
WebauthnChallengeType,
};
use tranquil_oauth::{AuthorizedClientData, DeviceData, RequestData, TokenData};
use tranquil_types::{
@@ -782,6 +783,14 @@ impl<S: StorageIO + 'static> tranquil_db_traits::RepoRepository for MetastoreCli
recv(rx).await
}
async fn get_all_repo_identities(&self) -> Result<Vec<RepoIdentity>, DbError> {
let (tx, rx) = oneshot::channel();
self.pool.send(MetastoreRequest::Commit(Box::new(
CommitRequest::GetAllRepoIdentities { tx },
)))?;
recv(rx).await
}
async fn insert_record_blobs(
&self,
repo_id: Uuid,
@@ -875,6 +884,17 @@ impl<S: StorageIO + 'static> tranquil_db_traits::BlobRepository for MetastoreCli
recv(rx).await
}
async fn ensure_blob_ownership(&self, user_id: Uuid, cid: &CidLink) -> Result<bool, DbError> {
let (tx, rx) = oneshot::channel();
self.pool
.send(MetastoreRequest::Blob(BlobRequest::EnsureBlobOwnership {
user_id,
cid: cid.clone(),
tx,
}))?;
recv(rx).await
}
async fn get_blob_metadata(
&self,
cid: &CidLink,
@@ -975,16 +995,6 @@ impl<S: StorageIO + 'static> tranquil_db_traits::BlobRepository for MetastoreCli
recv(rx).await
}
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<bool, DbError> {
let (tx, rx) = oneshot::channel();
self.pool
.send(MetastoreRequest::Blob(BlobRequest::DeleteBlobByCid {
cid: cid.clone(),
tx,
}))?;
recv(rx).await
}
async fn delete_blobs_by_user(&self, user_id: Uuid) -> Result<u64, DbError> {
let (tx, rx) = oneshot::channel();
self.pool
@@ -2362,27 +2372,6 @@ impl<S: StorageIO + 'static> tranquil_db_traits::InfraRepository for MetastoreCl
recv(rx).await
}
async fn get_blob_storage_key_by_cid(&self, cid: &CidLink) -> Result<Option<String>, DbError> {
let (tx, rx) = oneshot::channel();
self.pool.send(MetastoreRequest::Infra(
InfraRequest::GetBlobStorageKeyByCid {
cid: cid.clone(),
tx,
},
))?;
recv(rx).await
}
async fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<(), DbError> {
let (tx, rx) = oneshot::channel();
self.pool
.send(MetastoreRequest::Infra(InfraRequest::DeleteBlobByCid {
cid: cid.clone(),
tx,
}))?;
recv(rx).await
}
async fn get_admin_account_info_by_did(
&self,
did: &Did,
@@ -26,7 +26,7 @@ use crate::io::{RealIO, StorageIO};
use tranquil_db_traits::{
ApplyCommitError, ApplyCommitInput, ApplyCommitResult, ImportBlock, ImportRecord,
ImportRepoError, UserNeedingRecordBlobsBackfill, UserWithoutBlocks,
ImportRepoError, RepoIdentity, UserNeedingRecordBlobsBackfill, UserWithoutBlocks,
};
use tranquil_types::{AtUri, CidLink, Did, Tid};
@@ -401,6 +401,21 @@ impl<S: StorageIO + 'static> CommitOps<S> {
)
}
pub fn get_all_repo_identities(&self) -> Result<Vec<RepoIdentity>, MetastoreError> {
self.scan_users(
|_, _| Ok(true),
|meta, user_id| {
let did = match meta.did {
None => Err(MetastoreError::CorruptData("repo_meta missing DID field")),
Some(d) => Did::new(d)
.map_err(|_| MetastoreError::CorruptData("corrupt repo_meta did")),
}?;
Ok(RepoIdentity { user_id, did })
},
usize::MAX,
)
}
fn scan_users_missing_prefix<T, F, P>(
&self,
make_prefix: P,
@@ -410,6 +425,33 @@ impl<S: StorageIO + 'static> CommitOps<S> {
where
F: Fn(RepoMetaValue, Uuid) -> Result<T, MetastoreError>,
P: Fn(UserHash) -> SmallVec<[u8; 128]>,
{
self.scan_users(
|ops, user_hash| match ops
.repo_data
.prefix(make_prefix(user_hash).as_slice())
.next()
{
Some(guard) => guard
.into_inner()
.map(|_| false)
.map_err(MetastoreError::Fjall),
None => Ok(true),
},
build_result,
limit,
)
}
fn scan_users<T, F, P>(
&self,
include: P,
build_result: F,
limit: usize,
) -> Result<Vec<T>, MetastoreError>
where
F: Fn(RepoMetaValue, Uuid) -> Result<T, MetastoreError>,
P: Fn(&Self, UserHash) -> Result<bool, MetastoreError>,
{
let prefix = repo_meta_prefix();
@@ -429,18 +471,10 @@ impl<S: StorageIO + 'static> CommitOps<S> {
}
};
let check_prefix = make_prefix(user_hash);
let has_entries = match self.repo_data.prefix(check_prefix.as_slice()).next() {
Some(guard) => match guard.into_inner() {
Ok(_) => true,
Err(e) => return Some(Err(MetastoreError::Fjall(e))),
},
None => false,
};
match has_entries {
true => None,
false => {
match include(self, user_hash) {
Err(e) => Some(Err(e)),
Ok(false) => None,
Ok(true) => {
let meta = match RepoMetaValue::deserialize(&val_bytes) {
Some(v) => v,
None => {
+39 -53
View File
@@ -16,17 +16,17 @@ use tranquil_db_traits::{
MigrationReactivationError, MigrationReactivationInput, NotificationHistoryRow,
NotificationPrefs, OAuthTokenWithUser, PasswordResetResult, PlcTokenInfo, QueuedComms,
ReactivatedAccountInfo, RecoverPasskeyAccountInput, RecoverPasskeyAccountResult,
RefreshSessionResult, ReservedSigningKey, ReservedSigningKeyFull, ScheduledDeletionAccount,
ScopePreference, SequenceNumber, SequencedEvent, SessionId, StoredBackupCode, StoredPasskey,
TokenFamilyId, TotpRecord, TotpRecordState, User2faStatus, UserAuthInfo, UserCommsPrefs,
UserConfirmSignup, UserDidWebInfo, UserEmailInfo, UserForDeletion, UserForDidDoc,
UserForDidDocBuild, UserForPasskeyRecovery, UserForPasskeySetup, UserForRecovery,
UserForVerification, UserIdAndHandle, UserIdAndPasswordHash, UserIdHandleEmail,
UserInfoForAuth, UserKeyInfo, UserKeyWithId, UserLegacyLoginPref, UserLoginCheck,
UserLoginFull, UserLoginInfo, UserNeedingRecordBlobsBackfill, UserPasswordInfo,
UserResendVerification, UserResetCodeInfo, UserRow, UserSessionInfo, UserStatus,
UserVerificationInfo, UserWithKey, UserWithoutBlocks, ValidatedInviteCode,
WebauthnChallengeType,
RefreshSessionResult, RepoIdentity, ReservedSigningKey, ReservedSigningKeyFull,
ScheduledDeletionAccount, ScopePreference, SequenceNumber, SequencedEvent, SessionId,
StoredBackupCode, StoredPasskey, TokenFamilyId, TotpRecord, TotpRecordState, User2faStatus,
UserAuthInfo, UserCommsPrefs, UserConfirmSignup, UserDidWebInfo, UserEmailInfo,
UserForDeletion, UserForDidDoc, UserForDidDocBuild, UserForPasskeyRecovery,
UserForPasskeySetup, UserForRecovery, UserForVerification, UserIdAndHandle,
UserIdAndPasswordHash, UserIdHandleEmail, UserInfoForAuth, UserKeyInfo, UserKeyWithId,
UserLegacyLoginPref, UserLoginCheck, UserLoginFull, UserLoginInfo,
UserNeedingRecordBlobsBackfill, UserPasswordInfo, UserResendVerification, UserResetCodeInfo,
UserRow, UserSessionInfo, UserStatus, UserVerificationInfo, UserWithKey, UserWithoutBlocks,
ValidatedInviteCode, WebauthnChallengeType,
};
use tranquil_oauth::{AuthorizedClientData, DeviceData, RequestData, TokenData};
use tranquil_types::{
@@ -499,6 +499,9 @@ pub enum CommitRequest {
limit: i64,
tx: Tx<Vec<UserNeedingRecordBlobsBackfill>>,
},
GetAllRepoIdentities {
tx: Tx<Vec<RepoIdentity>>,
},
InsertRecordBlobs {
repo_id: Uuid,
record_uris: Vec<AtUri>,
@@ -516,7 +519,8 @@ impl CommitRequest {
repo_id: user_id, ..
} => uuid_to_routing(user_hashes, user_id),
Self::GetUsersWithoutBlocks { .. }
| Self::GetUsersNeedingRecordBlobsBackfill { .. } => Routing::Global,
| Self::GetUsersNeedingRecordBlobsBackfill { .. }
| Self::GetAllRepoIdentities { .. } => Routing::Global,
}
}
}
@@ -566,6 +570,11 @@ pub enum BlobRequest {
storage_key: String,
tx: Tx<Option<CidLink>>,
},
EnsureBlobOwnership {
user_id: Uuid,
cid: CidLink,
tx: Tx<bool>,
},
GetBlobMetadata {
cid: CidLink,
tx: Tx<Option<tranquil_db_traits::BlobMetadata>>,
@@ -601,10 +610,6 @@ pub enum BlobRequest {
takedown_ref: Option<String>,
tx: Tx<bool>,
},
DeleteBlobByCid {
cid: CidLink,
tx: Tx<bool>,
},
DeleteBlobsByUser {
user_id: Uuid,
tx: Tx<u64>,
@@ -633,8 +638,8 @@ impl BlobRequest {
fn routing(&self, user_hashes: &UserHashMap) -> Routing {
match self {
Self::InsertBlob { cid, .. }
| Self::UpdateBlobTakedown { cid, .. }
| Self::DeleteBlobByCid { cid, .. } => cid_to_routing(cid),
| Self::EnsureBlobOwnership { cid, .. }
| Self::UpdateBlobTakedown { cid, .. } => cid_to_routing(cid),
Self::DeleteBlobsByUser { user_id, .. } => uuid_to_routing(user_hashes, user_id),
@@ -2013,14 +2018,6 @@ pub enum InfraRequest {
key: String,
tx: Tx<()>,
},
GetBlobStorageKeyByCid {
cid: CidLink,
tx: Tx<Option<String>>,
},
DeleteBlobByCid {
cid: CidLink,
tx: Tx<()>,
},
GetAdminAccountInfoByDid {
did: Did,
tx: Tx<Option<AdminAccountInfo>>,
@@ -2107,9 +2104,6 @@ impl InfraRequest {
| Self::GetDeletionRequestByDid { did, .. }
| Self::GetPlcTokensByDid { did, .. }
| Self::CountPlcTokensByDid { did, .. } => did_to_routing(did),
Self::GetBlobStorageKeyByCid { cid, .. } | Self::DeleteBlobByCid { cid, .. } => {
cid_to_routing(cid)
}
_ => Routing::Global,
}
}
@@ -3089,6 +3083,14 @@ fn dispatch_commit<S: StorageIO + 'static>(state: &HandlerState<S>, req: CommitR
.map_err(metastore_to_db),
);
}
CommitRequest::GetAllRepoIdentities { tx } => {
let _ = tx.send(
state
.commit_ops
.get_all_repo_identities()
.map_err(metastore_to_db),
);
}
CommitRequest::InsertRecordBlobs {
repo_id,
record_uris,
@@ -3194,6 +3196,14 @@ fn dispatch_blob<S: StorageIO + 'static>(state: &HandlerState<S>, req: BlobReque
.map_err(metastore_to_db);
let _ = tx.send(result);
}
BlobRequest::EnsureBlobOwnership { user_id, cid, tx } => {
let result = state
.metastore
.blob_ops()
.ensure_blob_ownership(user_id, &cid)
.map_err(metastore_to_db);
let _ = tx.send(result);
}
BlobRequest::GetBlobMetadata { cid, tx } => {
let result = state
.metastore
@@ -3267,14 +3277,6 @@ fn dispatch_blob<S: StorageIO + 'static>(state: &HandlerState<S>, req: BlobReque
.map_err(metastore_to_db);
let _ = tx.send(result);
}
BlobRequest::DeleteBlobByCid { cid, tx } => {
let result = state
.metastore
.blob_ops()
.delete_blob_by_cid(&cid)
.map_err(metastore_to_db);
let _ = tx.send(result);
}
BlobRequest::DeleteBlobsByUser { user_id, tx } => {
let result = state
.metastore
@@ -4309,22 +4311,6 @@ fn dispatch_infra<S: StorageIO>(state: &HandlerState<S>, req: InfraRequest) {
.map_err(metastore_to_db);
let _ = tx.send(result);
}
InfraRequest::GetBlobStorageKeyByCid { cid, tx } => {
let result = state
.metastore
.infra_ops()
.get_blob_storage_key_by_cid(&cid)
.map_err(metastore_to_db);
let _ = tx.send(result);
}
InfraRequest::DeleteBlobByCid { cid, tx } => {
let result = state
.metastore
.infra_ops()
.delete_blob_by_cid(&cid)
.map_err(metastore_to_db);
let _ = tx.send(result);
}
InfraRequest::GetAdminAccountInfoByDid { did, tx } => {
let result = state
.metastore
@@ -6,7 +6,6 @@ use smallvec::SmallVec;
use uuid::Uuid;
use super::MetastoreError;
use super::blobs::{BlobMetaValue, blob_by_cid_key, blob_meta_key};
use super::infra_schema::{
DeletionRequestValue, InviteCodeUseValue, InviteCodeValue, NotificationHistoryValue,
QueuedCommsValue, ReportValue, SigningKeyValue, account_pref_key, account_pref_prefix,
@@ -28,12 +27,11 @@ use tranquil_db_traits::{
InviteCodeState, InviteCodeUse, NotificationHistoryRow, PlcTokenInfo, QueuedComms,
ReservedSigningKey, ReservedSigningKeyFull, ValidatedInviteCode,
};
use tranquil_types::{CidLink, Did, Handle, InviteCode};
use tranquil_types::{Did, Handle, InviteCode};
pub struct InfraOps {
db: Database,
infra: Keyspace,
repo_data: Keyspace,
users: Keyspace,
user_hashes: Arc<UserHashMap>,
comms_seq: Arc<std::sync::atomic::AtomicU32>,
@@ -44,7 +42,6 @@ impl InfraOps {
pub fn new(
db: Database,
infra: Keyspace,
repo_data: Keyspace,
users: Keyspace,
user_hashes: Arc<UserHashMap>,
comms_seq: Arc<std::sync::atomic::AtomicU32>,
@@ -53,7 +50,6 @@ impl InfraOps {
Self {
db,
infra,
repo_data,
users,
user_hashes,
comms_seq,
@@ -1216,63 +1212,6 @@ impl InfraOps {
})
}
pub fn get_blob_storage_key_by_cid(
&self,
cid: &CidLink,
) -> Result<Option<String>, MetastoreError> {
let cid_str = cid.as_str();
let cid_index_key = blob_by_cid_key(cid_str);
let user_hash_raw = match self
.repo_data
.get(cid_index_key.as_slice())
.map_err(MetastoreError::Fjall)?
{
Some(raw) => {
let arr: [u8; 8] = raw
.as_ref()
.try_into()
.map_err(|_| MetastoreError::CorruptData("blob_by_cid value not 8 bytes"))?;
u64::from_be_bytes(arr)
}
None => return Ok(None),
};
let user_hash = UserHash::from_raw(user_hash_raw);
let key = blob_meta_key(user_hash, cid_str);
let val: Option<BlobMetaValue> = point_lookup(
&self.repo_data,
key.as_slice(),
BlobMetaValue::deserialize,
"corrupt blob_meta value",
)?;
Ok(val.map(|v| v.storage_key))
}
pub fn delete_blob_by_cid(&self, cid: &CidLink) -> Result<(), MetastoreError> {
let cid_str = cid.as_str();
let cid_index_key = blob_by_cid_key(cid_str);
let user_hash_raw = match self
.repo_data
.get(cid_index_key.as_slice())
.map_err(MetastoreError::Fjall)?
{
Some(raw) => {
let arr: [u8; 8] = raw
.as_ref()
.try_into()
.map_err(|_| MetastoreError::CorruptData("blob_by_cid value not 8 bytes"))?;
u64::from_be_bytes(arr)
}
None => return Ok(()),
};
let user_hash = UserHash::from_raw(user_hash_raw);
let primary_key = blob_meta_key(user_hash, cid_str);
let mut batch = self.db.batch();
batch.remove(&self.repo_data, primary_key.as_slice());
batch.remove(&self.repo_data, cid_index_key.as_slice());
batch.commit().map_err(MetastoreError::Fjall)
}
pub fn get_admin_account_info_by_did(
&self,
did: &Did,
+36 -2
View File
@@ -35,11 +35,12 @@ use std::sync::Arc;
use fjall::{Database, Keyspace};
use self::encoding::KeyReader;
use self::keys::KeyTag;
use self::partitions::Partition;
use self::user_hash::UserHashMap;
const CURRENT_FORMAT_VERSION: u64 = 2;
const CURRENT_FORMAT_VERSION: u64 = 3;
#[derive(Debug, Clone)]
pub struct MetastoreConfig {
@@ -240,6 +241,7 @@ impl Metastore {
"upgrading metastore format and rebuilding derived indexes"
);
repo_data.remove(records::record_by_cid_built_key().as_slice())?;
Self::migrate_blob_ownership(db, repo_data)?;
repo_data.insert(version_key, version_bytes)?;
db.persist(fjall::PersistMode::SyncData)?;
Ok(())
@@ -254,6 +256,38 @@ impl Metastore {
}
}
fn migrate_blob_ownership(db: &Database, repo_data: &Keyspace) -> Result<(), MetastoreError> {
let entries: Vec<(Vec<u8>, Vec<u8>)> = repo_data
.prefix(blobs::blobs_prefix().as_slice())
.map(|guard| {
let (k, v) = guard.into_inner()?;
Ok((k.as_ref().to_vec(), v.as_ref().to_vec()))
})
.collect::<Result<_, fjall::Error>>()?;
for (key_bytes, val_bytes) in entries {
let Some(meta) = blobs::BlobMetaValue::deserialize(&val_bytes) else {
continue;
};
let mut reader = KeyReader::new(&key_bytes);
reader.tag();
reader.u64();
let Some(cid_str) = reader.string() else {
continue;
};
let cid_index_key = blobs::blob_by_cid_key(cid_str.as_str());
let content = blobs::BlobContentValue { meta, ref_count: 1 };
let mut batch = db.batch();
batch.insert(repo_data, cid_index_key.as_slice(), content.serialize());
batch.insert(repo_data, key_bytes.as_slice(), &[] as &[u8]);
batch.commit()?;
}
Ok(())
}
pub fn path(&self) -> &Path {
&self.path
}
@@ -311,6 +345,7 @@ impl Metastore {
self.db.clone(),
self.partitions[Partition::RepoData.index()].clone(),
Arc::clone(&self.user_hashes),
Arc::clone(&self.counter_lock),
)
}
@@ -351,7 +386,6 @@ impl Metastore {
infra_ops::InfraOps::new(
self.db.clone(),
self.partitions[Partition::Infra.index()].clone(),
self.partitions[Partition::RepoData.index()].clone(),
self.partitions[Partition::Users.index()].clone(),
Arc::clone(&self.user_hashes),
Arc::clone(&self.comms_seq),
+12 -4
View File
@@ -344,8 +344,12 @@ fn verify_backup_detects_checksum_mismatch() {
.unwrap();
let manifest = read_manifest(backup_dir.path()).unwrap();
let first_file = &manifest.files[0];
let file_path = backup_dir.path().join(&first_file.path);
let target = manifest
.files
.iter()
.find(|f| f.size > 0)
.expect("backup manifest must list a file with content");
let file_path = backup_dir.path().join(&target.path);
let mut data = std::fs::read(&file_path).unwrap();
data.iter_mut().take(8).for_each(|b| *b ^= 0xFF);
std::fs::write(&file_path, &data).unwrap();
@@ -901,8 +905,12 @@ fn restore_fails_cleanly_on_corrupted_backup() {
.create_backup(backup_dir.path())
.unwrap();
let first_file = &manifest.files[0];
let file_path = backup_dir.path().join(&first_file.path);
let target = manifest
.files
.iter()
.find(|f| f.size > 0)
.expect("backup manifest must list a file with content");
let file_path = backup_dir.path().join(&target.path);
let mut data = std::fs::read(&file_path).unwrap();
data.iter_mut().take(16).for_each(|b| *b ^= 0xFF);
std::fs::write(&file_path, &data).unwrap();
+32
View File
@@ -330,6 +330,38 @@ pub enum HandleError {
Invalid(String),
}
validated_string_newtype! {
pub struct Domain;
error = DomainError;
label = "domain";
validator = |s| {
let normalized = s.to_ascii_lowercase();
(normalized.len() <= 253
&& !normalized.is_empty()
&& normalized.split('.').all(|label| {
!label.is_empty()
&& !label.starts_with('-')
&& !label.ends_with('-')
&& label.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'-' || b == b'_')
}))
.then_some(normalized)
.ok_or(())
};
}
impl Domain {
pub fn eq_name(&self, name: &str) -> bool {
self.as_str().eq_ignore_ascii_case(name)
}
pub fn strip_from<'h>(&self, handle: &'h str) -> Option<&'h str> {
let domain_len = self.as_str().len();
(handle.len() > domain_len + 1 && handle.as_bytes()[handle.len() - domain_len - 1] == b'.')
.then(|| &handle[..handle.len() - domain_len - 1])
.filter(|_| handle[handle.len() - domain_len..].eq_ignore_ascii_case(self.as_str()))
}
}
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum AtIdentifier {
Did(Did),
+1 -1
View File
@@ -66,7 +66,7 @@ See [example.toml](https://tangled.org/tranquil.farm/tranquil-pds/blob/main/exam
# by default, tranquil runs on port 3000.
# You can change this with the tranquil-pds.settings.server.port option in the service config.
extraConfig = ''
reverse_proxy localhost:3000
reverse_proxy [::1]:3000
'';
};
};
+20 -2
View File
@@ -10,8 +10,8 @@
#
# Can also be specified via environment variable `SERVER_HOST`.
#
# Default value: "127.0.0.1"
#host = "127.0.0.1"
# Default value: "[::1]"
#host = "[::1]"
# Port to bind the HTTP server to.
#
@@ -34,6 +34,17 @@
# Default value: false
#enable_pds_hosted_did_web = false
# The caddy on-demand TLS requires we serve
# the endpoint `/.well-known/caddy/ask`.
# It will be used so that caddy can create TLS
# certs for us on the fly
# and we don't have to do annoying wildcard certs.
#
# Can also be specified via environment variable `ENABLE_CADDY_ON_DEMAND_TLS`.
#
# Default value: true
#enable_caddy_on_demand_tls = true
# iykyk!
#
# Can also be specified via environment variable `RFC_MOO_COMPLIANCE`.
@@ -253,6 +264,13 @@
# Can also be specified via environment variable `S3_ENDPOINT`.
#s3_endpoint =
# Path on the storage for the S3 blob backend.
#
# Can also be specified via environment variable `S3_PATH`.
#
# Default value: ""
#s3_path = ""
# Repository backend: `postgres` by default, or `tranquil-store`, our embedded db.
# tranquil-store is EXPERIMENTAL!!!! RISK OF TOTAL DATA LOSS.
#
+1
View File
@@ -24,6 +24,7 @@
devShells = forAllSystems (pkgs: {
default = pkgs.callPackage ./shell.nix { };
full = pkgs.callPackage ./shells/full.nix { };
});
nixosModules = {
@@ -64,6 +64,10 @@
<input
{id}
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
value={value}
{placeholder}
{disabled}
@@ -461,6 +461,10 @@
<input
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
value={searchQuery}
oninput={(e) => onSearchInput(e.currentTarget.value)}
placeholder={$_('admin.searchPlaceholder')}
@@ -30,6 +30,7 @@
let savedDiscordUsername = $state('')
let savedTelegramUsername = $state('')
let savedSignalUsername = $state('')
let legacyLoginAlerts = $state(true)
let verifyingChannel = $state<string | null>(null)
let verificationCode = $state('')
let historyLoading = $state(true)
@@ -62,6 +63,7 @@
telegramVerified = prefs.telegramVerified
signalUsername = prefs.signalUsername ?? ''
signalVerified = prefs.signalVerified
legacyLoginAlerts = prefs.legacyLoginAlerts ?? true
savedDiscordUsername = discordUsername
savedTelegramUsername = telegramUsername
savedSignalUsername = signalUsername
@@ -85,6 +87,7 @@
discordUsername: discordUsername !== savedDiscordUsername ? discordUsername : undefined,
telegramUsername: telegramUsername !== savedTelegramUsername ? telegramUsername : undefined,
signalUsername: signalUsername !== savedSignalUsername ? signalUsername : undefined,
legacyLoginAlerts,
})
await refreshSession()
toast.success($_('comms.preferencesSaved'))
@@ -316,6 +319,25 @@
</div>
</section>
<section>
<h3>{$_('comms.securityAlerts')}</h3>
<div class="toggle-row">
<div class="toggle-info">
<span class="toggle-label">{$_('comms.legacyLoginAlerts')}</span>
<span class="toggle-description">{$_('comms.legacyLoginAlertsDescription')}</span>
</div>
<button
type="button"
class="toggle-button {legacyLoginAlerts ? 'on' : 'off'}"
onclick={() => legacyLoginAlerts = !legacyLoginAlerts}
disabled={saving}
aria-label={legacyLoginAlerts ? $_('comms.disableLegacyLoginAlerts') : $_('comms.enableLegacyLoginAlerts')}
>
<span class="toggle-slider"></span>
</button>
</div>
</section>
<div class="actions">
<button type="submit" disabled={saving}>
{saving ? $_('common.saving') : $_('comms.savePreferences')}
@@ -422,6 +422,10 @@
<input
id="controllerIdentifier"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
value={addControllerIdentifier}
oninput={(e) => onControllerInput((e.target as HTMLInputElement).value)}
onblur={() => { setTimeout(() => { showTypeahead = false }, 200) }}
@@ -541,6 +545,10 @@
<input
id="delegatedHandle"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={newDelegatedHandle}
placeholder="username"
disabled={creatingDelegated}
@@ -208,6 +208,10 @@
<input
id="new-handle"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={newHandle}
placeholder={$_('didEditor.handlePlaceholder')}
/>
@@ -342,7 +342,18 @@
<form onsubmit={handleUpdateHandle}>
<div>
<label for="new-handle-byo">{$_('settings.yourDomain')}</label>
<input id="new-handle-byo" type="text" bind:value={newHandle} placeholder={$_('settings.yourDomainPlaceholder')} disabled={handleLoading} required />
<input
id="new-handle-byo"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={newHandle}
placeholder={$_('settings.yourDomainPlaceholder')}
disabled={handleLoading}
required
/>
</div>
<button type="submit" disabled={handleLoading || !newHandle}>
{handleLoading ? $_('common.verifying') : $_('settings.verifyAndUpdate')}
@@ -377,6 +377,10 @@
<input
id="source-handle"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
placeholder={$_('migration.inbound.sourceAuth.handlePlaceholder')}
bind:value={handleInput}
disabled={loading || isResuming}
+1
View File
@@ -680,6 +680,7 @@ export const api = {
discordUsername?: string;
telegramUsername?: string;
signalUsername?: string;
legacyLoginAlerts?: boolean;
}): Promise<UpdateNotificationPrefsResponse> {
return xrpc("_account.updateNotificationPrefs", {
method: "POST",
+1
View File
@@ -232,6 +232,7 @@ export interface NotificationPrefs {
telegramVerified: boolean;
signalUsername: string | null;
signalVerified: boolean;
legacyLoginAlerts: boolean;
}
export interface NotificationHistoryItem {
+22 -5
View File
@@ -271,7 +271,7 @@
"emailUpdateFailed": "Failed to update email",
"handleUpdated": "Handle updated successfully",
"handleUpdateFailed": "Failed to update handle",
"deletionConfirmationSent": "Deletion confirmation sent to your email",
"deletionConfirmationSent": "Deletion confirmation sent to your registered notification channel",
"deletionRequestFailed": "Failed to request account deletion",
"deleteConfirmation": "Are you absolutely sure you want to delete your account? This cannot be undone.",
"deletionFailed": "Failed to delete account",
@@ -282,7 +282,7 @@
"appPasswords": {
"create": "Create",
"name": "Name",
"namePlaceholder": "App name (eg. Witchsky)",
"namePlaceholder": "App name (e.g. Witchsky)",
"created": "App Password Created",
"createdMessage": "Copy this password now. You won't be able to see it again.",
"noPasswords": "No app passwords yet",
@@ -458,7 +458,12 @@
"telegramStartBot": "Or send /start {handle} to @{botUsername} manually",
"telegramOpenLink": "Open Telegram to verify",
"discordStartBot": "DM @{botUsername} on Discord and send /start {handle}",
"discordOpenLink": "Open Discord to verify"
"discordOpenLink": "Open Discord to verify",
"securityAlerts": "Security alerts",
"legacyLoginAlerts": "Legacy login alerts",
"legacyLoginAlertsDescription": "Get notified when someone signs in to your TOTP enabled account using legacy login methods.",
"enableLegacyLoginAlerts": "Enable legacy login alerts",
"disableLegacyLoginAlerts": "Disable legacy login alerts"
},
"repoExplorer": {
"collections": "Collections",
@@ -602,6 +607,7 @@
"unavailablePermissions": "Unavailable permissions",
"unavailableLimited": "Limited by delegation",
"unavailableFailed": "Failed to load",
"unavailableRejected": "Not granted",
"setPartiallyLimited": "Some permissions in this bundle are limited by your delegation; see \"Unavailable Permissions\" to find out which permissions weren't allowed.",
"setFailureReason": {
"not_found": "This bundle could not be found and cannot be granted.",
@@ -612,6 +618,11 @@
"empty_permissions": "This bundle does not currently grant any permissions.",
"unknown": "This bundle cannot be granted."
},
"scopeRejectionReason": {
"unrecognized": "This server does not recognize this permission, so it cannot be granted.",
"not_registered": "The application did not register this permission, so it cannot be granted.",
"unknown": "This permission cannot be granted."
},
"permTable": {
"data": "Data",
"create": "Create",
@@ -634,7 +645,13 @@
"title": "Unexpected State",
"description": "The consent page is in an unexpected state. Please check the browser console for errors.",
"reload": "Reload Page"
}
},
"supersedeWarningTitle": "This app asked for broad access",
"supersedeWarningBody": "This application has requested full read and write access to your account.",
"supersededNote": "Already covered by transition:generic",
"deselectedWarningTitle": "Some permissions are disabled",
"deselectedWarningBody": "You have disabled some of the permissions requested by this app. This may cause some parts of the app to be broken or unavailable.",
"supersedeWarningBodyMixed": "This application has requested full read and write access to your account, alongside more specific permissions. The specific permissions are meaningless if you grant the application complete and total control by leaving transition:generic selected."
},
"accounts": {
"title": "Choose account",
@@ -785,7 +802,7 @@
"subtitle": "Lost access to your passkey? Enter your handle or email and we'll send you a recovery link.",
"successTitle": "Recovery Link Sent",
"successMessage": "If your account exists and is a passkey-only account, you'll receive a recovery link at your preferred notification channel.",
"successInfo": "The link will expire in 1 hour. Check your email, Discord, Telegram, or Signal depending on your account settings.",
"successInfo": "The link will expire in 1 hour. Check your registered notification channel.",
"handleOrEmail": "Handle or Email",
"emailPlaceholder": "handle or you@example.com",
"howItWorks": "How it works",
+6 -1
View File
@@ -458,7 +458,12 @@
"failedToLoad": "Asetusten lataus epäonnistui",
"failedToSave": "Asetusten tallennus epäonnistui",
"failedToVerify": "Vahvistus epäonnistui",
"failedToLoadHistory": "Viestihistorian lataus epäonnistui"
"failedToLoadHistory": "Viestihistorian lataus epäonnistui",
"securityAlerts": "Turvallisuushälytykset",
"legacyLoginAlerts": "Vanhat kirjautumisilmoitukset",
"legacyLoginAlertsDescription": "Saat ilmoituksen, kun joku kirjautuu TOTP-yhteensopivalle tilillesi käyttämällä vanhoja kirjautumistapoja.",
"enableLegacyLoginAlerts": "Ota käyttöön vanhat kirjautumisilmoitukset",
"disableLegacyLoginAlerts": "Poista käytöstä vanhat kirjautumisilmoitukset"
},
"repoExplorer": {
"collections": "Kokoelmat",
+6 -1
View File
@@ -458,7 +458,12 @@
"telegramStartBot": "Ou envoyez /start {handle} à @{botUsername} manuellement",
"telegramOpenLink": "Ouvrir Telegram pour vérifier",
"discordStartBot": "Envoyez un DM à @{botUsername} sur Discord avec /start {handle}",
"discordOpenLink": "Ouvrir Discord pour vérifier"
"discordOpenLink": "Ouvrir Discord pour vérifier",
"securityAlerts": "Alertes de sécurité",
"legacyLoginAlerts": "Anciennes alertes de connexion",
"legacyLoginAlertsDescription": "Recevez une notification lorsque quelqu'un se connecte à votre compte TOTP à l'aide de méthodes de connexion héritées.",
"enableLegacyLoginAlerts": "Activer les alertes de connexion héritées",
"disableLegacyLoginAlerts": "Désactiver les alertes de connexion héritées"
},
"repoExplorer": {
"collections": "Collections",
+6 -1
View File
@@ -458,7 +458,12 @@
"failedToLoad": "設定の読み込みに失敗しました",
"failedToSave": "設定の保存に失敗しました",
"failedToVerify": "確認に失敗しました",
"failedToLoadHistory": "メッセージ履歴の読み込みに失敗しました"
"failedToLoadHistory": "メッセージ履歴の読み込みに失敗しました",
"securityAlerts": "セキュリティ警告",
"legacyLoginAlerts": "レガシーログインアラート",
"legacyLoginAlertsDescription": "誰かが従来のログイン方法を使用してTOTP対応アカウントにサインインしたときに通知を受け取ります。",
"enableLegacyLoginAlerts": "レガシーログインアラートを有効にする",
"disableLegacyLoginAlerts": "レガシーログインアラートを無効にする"
},
"repoExplorer": {
"collections": "コレクション",
+6 -1
View File
@@ -458,7 +458,12 @@
"failedToLoad": "설정 로딩 실패",
"failedToSave": "설정 저장 실패",
"failedToVerify": "인증 실패",
"failedToLoadHistory": "메시지 기록 로딩 실패"
"failedToLoadHistory": "메시지 기록 로딩 실패",
"securityAlerts": "보안 경고",
"legacyLoginAlerts": "레거시 로그인 알림",
"legacyLoginAlertsDescription": "레거시 로그인 방법을 사용하여 누군가가 TOTP 지원 계정에 로그인하면 알림을 받습니다.",
"enableLegacyLoginAlerts": "레거시 로그인 알림 활성화",
"disableLegacyLoginAlerts": "레거시 로그인 알림 비활성화"
},
"repoExplorer": {
"collections": "컬렉션",
+6 -1
View File
@@ -458,7 +458,12 @@
"failedToLoad": "Kunde inte ladda inställningar",
"failedToSave": "Kunde inte spara inställningar",
"failedToVerify": "Verifiering misslyckades",
"failedToLoadHistory": "Kunde inte ladda meddelandehistorik"
"failedToLoadHistory": "Kunde inte ladda meddelandehistorik",
"securityAlerts": "Säkerhetsvarningar",
"legacyLoginAlerts": "Varningar för föråldrade inloggningsmetoder",
"legacyLoginAlertsDescription": "Bli notifierad när en inloggning sker med föråldrade inloggningsmetoder, när du har TOTP aktiverat.",
"enableLegacyLoginAlerts": "Aktivera föråldrad inloggningsmetodsvarning",
"disableLegacyLoginAlerts": "Avaktivera föråldrad inloggningsmetodsvarning"
},
"repoExplorer": {
"collections": "Samlingar",
+6 -1
View File
@@ -458,7 +458,12 @@
"failedToLoad": "加载偏好设置失败",
"failedToSave": "保存偏好设置失败",
"failedToVerify": "验证失败",
"failedToLoadHistory": "加载消息历史失败"
"failedToLoadHistory": "加载消息历史失败",
"securityAlerts": "安全警报",
"legacyLoginAlerts": "旧版登录提醒",
"legacyLoginAlertsDescription": "当有人使用传统登录方式登录您的启用了TOTP的帐户时,会收到通知。",
"enableLegacyLoginAlerts": "启用旧版登录警报",
"disableLegacyLoginAlerts": "禁用旧版登录警报"
},
"repoExplorer": {
"collections": "集合",
+90 -6
View File
@@ -10,6 +10,7 @@
display_name: string
granted: boolean | null
restricted?: boolean
superseded?: boolean
effective_scope?: string
}
@@ -43,6 +44,7 @@
expanded: ScopeInfo[]
granted: boolean | null
restricted?: boolean
superseded?: boolean
}
type SetFailureReason =
@@ -67,12 +69,27 @@
return known[reason] ?? 'oauth.consent.setFailureReason.unknown'
}
const SCOPE_REJECTION_LOCALE_KEYS = {
unrecognized: 'oauth.consent.scopeRejectionReason.unrecognized',
not_registered: 'oauth.consent.scopeRejectionReason.not_registered',
}
function scopeRejectionLocaleKey(reason: string): string {
const known: Partial<Record<string, string>> = SCOPE_REJECTION_LOCALE_KEYS
return known[reason] ?? 'oauth.consent.scopeRejectionReason.unknown'
}
interface FailedSetInfo {
nsid: string
aud?: string
reason: string
}
interface RejectedScopeInfo {
scope: string
reason: string
}
interface ConsentData {
request_uri: string
client_id: string
@@ -81,7 +98,9 @@
logo_uri: string | null
scopes: ScopeInfo[]
permission_sets: PermissionSetInfo[]
transition_supersedes?: boolean
failed_sets: FailedSetInfo[]
rejected_scopes: RejectedScopeInfo[]
show_consent: boolean
did: string
handle?: string
@@ -264,10 +283,32 @@
}
}
const TRANSITION_GENERIC = 'transition:generic'
let transitionGenericSelected = $derived(scopeSelections[TRANSITION_GENERIC] === true)
let anyDeselected = $derived(
Object.values(scopeSelections).some((selected) => selected === false)
)
function isSupersededNow(item: { superseded?: boolean }): boolean {
return Boolean(consentData?.transition_supersedes && item.superseded && transitionGenericSelected)
}
function handleScopeToggle(scope: string) {
const scopeInfo = consentData?.scopes.find(s => s.scope === scope)
if (scopeInfo?.required) return
scopeSelections[scope] = !scopeSelections[scope]
if (scopeInfo && isSupersededNow(scopeInfo)) return
const next = !scopeSelections[scope]
scopeSelections[scope] = next
if (scope === TRANSITION_GENERIC && next) {
for (const s of consentData?.scopes ?? []) {
if (s.superseded && !s.restricted) scopeSelections[s.scope] = true
}
for (const set of consentData?.permission_sets ?? []) {
if (set.superseded && !set.restricted) scopeSelections[set.include_scope] = true
}
}
}
const CATEGORY_ORDER = [
@@ -311,8 +352,12 @@
consentData ? (consentData.permission_sets ?? []).filter(s => s.expanded.some(e => e.restricted)) : []
)
let failedSets = $derived(consentData?.failed_sets ?? [])
let rejectedScopes = $derived(consentData?.rejected_scopes ?? [])
let hasUnavailable = $derived(
restrictedScopes.length > 0 || limitedBundles.length > 0 || failedSets.length > 0
restrictedScopes.length > 0 ||
limitedBundles.length > 0 ||
failedSets.length > 0 ||
rejectedScopes.length > 0
)
let hasGranularScopes = $derived(
@@ -460,6 +505,30 @@
<span class="consent-account-did">{consentData.did}</span>
{/if}
</div>
{#if transitionGenericSelected}
<div class="permissions-notice" role="status">
<div class="notice-header">
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="8" x2="12" y2="12"/><line x1="12" y1="16" x2="12.01" y2="16"/></svg>
<span>{$_('oauth.consent.supersedeWarningTitle')}</span>
</div>
<p class="notice-text">
{consentData.transition_supersedes
? $_('oauth.consent.supersedeWarningBodyMixed')
: $_('oauth.consent.supersedeWarningBody')}
</p>
</div>
{/if}
{#if anyDeselected}
<div class="permissions-notice" role="status">
<div class="notice-header">
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="12" y1="8" x2="12" y2="12"/><line x1="12" y1="16" x2="12.01" y2="16"/></svg>
<span>{$_('oauth.consent.deselectedWarningTitle')}</span>
</div>
<p class="notice-text">{$_('oauth.consent.deselectedWarningBody')}</p>
</div>
{/if}
</div>
<div class="permissions-panel">
@@ -482,8 +551,8 @@
<label class="scope-item" class:required={scope.required}>
<input
type="checkbox"
checked={scopeSelections[scope.scope]}
disabled={scope.required || submitting}
checked={isSupersededNow(scope) ? true : scopeSelections[scope.scope]}
disabled={scope.required || submitting || isSupersededNow(scope)}
onchange={() => handleScopeToggle(scope.scope)}
/>
<div class="scope-info">
@@ -492,6 +561,9 @@
{#if scope.required}
<span class="required-badge">{$_('oauth.consent.required')}</span>
{/if}
{#if isSupersededNow(scope)}
<span class="superseded-note">{$_('oauth.consent.supersededNote')}</span>
{/if}
</div>
</label>
{/each}
@@ -509,8 +581,8 @@
<label class="scope-item">
<input
type="checkbox"
checked={scopeSelections[set.include_scope]}
disabled={submitting}
checked={isSupersededNow(set) ? true : scopeSelections[set.include_scope]}
disabled={submitting || isSupersededNow(set)}
onchange={() => handleScopeToggle(set.include_scope)}
/>
<div class="scope-info">
@@ -600,6 +672,18 @@
</div>
{/each}
{/if}
{#if rejectedScopes.length}
<p class="unavailable-subhead">{$_('oauth.consent.unavailableRejected')}</p>
{#each rejectedScopes as r}
<div class="scope-item failed">
<div class="scope-info">
<span class="scope-name scope-raw">{r.scope}</span>
<span class="scope-description">{$_(scopeRejectionLocaleKey(r.reason))}</span>
</div>
</div>
{/each}
{/if}
</div>
{/if}
</div>
@@ -163,6 +163,10 @@
<input
id="controller-identifier"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={controllerIdentifier}
disabled={submitting}
required
+4
View File
@@ -396,6 +396,10 @@
<input
id="username"
type="text"
inputmode="url"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={username}
placeholder={handlePlaceholder}
disabled={submitting}
@@ -53,6 +53,10 @@
<input
id="identifier"
type="text"
inputmode="email"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={identifier}
placeholder={$_('requestPasskeyRecovery.emailPlaceholder')}
disabled={submitting}
@@ -75,4 +79,3 @@
<a href={getFullUrl(routes.login)}>{$_('common.backToLogin')}</a>
</p>
</div>
+4 -1
View File
@@ -120,6 +120,10 @@
<input
id="email"
type="text"
inputmode="email"
autocapitalize="none"
autocorrect="off"
spellcheck="false"
bind:value={email}
placeholder={$_('resetPassword.emailPlaceholder')}
disabled={submitting}
@@ -136,4 +140,3 @@
<a href="/app/login">{$_('common.backToLogin')}</a>
</p>
</div>
+6 -1
View File
@@ -1085,7 +1085,8 @@ button.forget-btn:hover {
color: var(--text-muted);
}
.restricted-note {
.restricted-note,
.superseded-note {
display: block;
font-size: 0.75em;
color: var(--text-muted);
@@ -1589,3 +1590,7 @@ button.forget-btn:hover {
margin-left: auto;
}
}
.scope-item:has(input:disabled:checked) .scope-name {
color: var(--text-muted);
}
@@ -0,0 +1,197 @@
import { beforeEach, describe, expect, it } from "vitest";
import { fireEvent, render, screen, waitFor } from "@testing-library/svelte";
import OAuthConsent from "../routes/OAuthConsent.svelte";
import {
clearMocks,
jsonResponse,
mockEndpoint,
setupFetchMock,
setupIndexedDBMock,
} from "./mocks.ts";
const consentPayload = {
request_uri: "urn:mock:request",
client_id: "https://example.com",
client_name: "Mixed Scope App",
client_uri: null,
logo_uri: null,
transition_supersedes: true,
scopes: [
{
scope: "atproto",
category: "Core Access",
required: true,
description: "Baseline",
display_name: "AT Protocol Access",
granted: null,
superseded: false,
},
{
scope: "transition:generic",
category: "Other",
required: false,
description: "Broad access",
display_name: "Generic Access",
granted: null,
superseded: false,
},
{
scope: "repo:app.bsky.feed.post?action=create",
category: "Other",
required: false,
description: "Create posts",
display_name: "repo:app.bsky.feed.post",
granted: null,
superseded: true,
},
{
scope: "account:email?action=manage",
category: "Other",
required: false,
description: "Manage email",
display_name: "account:email",
granted: null,
superseded: false,
},
{
scope: "transition:chat.bsky",
category: "Other",
required: false,
description: "Chat access",
display_name: "Chat Access",
granted: null,
superseded: false,
},
],
permission_sets: [],
failed_sets: [],
rejected_scopes: [],
show_consent: true,
did: "did:plc:example",
};
function boxFor(name: string): HTMLInputElement {
const label = screen.getByText(name).closest("label");
if (!label) throw new Error(`no label containing "${name}"`);
const input = label.querySelector("input[type=checkbox]");
if (!input) throw new Error(`no checkbox in label for "${name}"`);
return input as HTMLInputElement;
}
describe("OAuthConsent transition:generic supersede behaviour", () => {
beforeEach(() => {
clearMocks();
setupFetchMock();
setupIndexedDBMock();
Object.defineProperty(window.location, "search", {
value: "?request_uri=urn:mock:request",
writable: true,
configurable: true,
});
mockEndpoint("/oauth/authorize/consent", () =>
jsonResponse(consentPayload),
);
});
it("warns that the itemised scopes are redundant", async () => {
render(OAuthConsent);
await waitFor(() =>
expect(screen.getByText(/asked for broad access/i)).toBeTruthy(),
);
expect(
screen.getByText(/specific permissions are meaningless/i),
).toBeTruthy();
});
it("warns about transition:generic even when nothing is superseded", async () => {
mockEndpoint("/oauth/authorize/consent", () =>
jsonResponse({
...consentPayload,
transition_supersedes: false,
scopes: consentPayload.scopes
.filter((s) => !s.superseded)
.map((s) => ({ ...s, superseded: false })),
}),
);
render(OAuthConsent);
await waitFor(() =>
expect(screen.getByText(/asked for broad access/i)).toBeTruthy(),
);
expect(screen.getByText(/full read and write access/i)).toBeTruthy();
expect(
screen.queryByText(/specific permissions are meaningless/i),
).toBeNull();
});
it("drops the warning once transition:generic is declined", async () => {
render(OAuthConsent);
await waitFor(() => expect(boxFor("Generic Access")).toBeTruthy());
await fireEvent.click(boxFor("Generic Access"));
await waitFor(() =>
expect(screen.queryByText(/asked for broad access/i)).toBeNull(),
);
});
it("locks superseded scopes checked while transition:generic is selected", async () => {
render(OAuthConsent);
await waitFor(() => expect(boxFor("Generic Access")).toBeTruthy());
const superseded = boxFor("repo:app.bsky.feed.post");
expect(superseded.checked).toBe(true);
expect(superseded.disabled).toBe(true);
});
it("leaves scopes generic does not cover editable", async () => {
render(OAuthConsent);
await waitFor(() => expect(boxFor("Chat Access")).toBeTruthy());
expect(boxFor("Chat Access").disabled).toBe(false);
expect(boxFor("account:email").disabled).toBe(false);
});
it("warns once a requested permission is switched off", async () => {
render(OAuthConsent);
await waitFor(() => expect(boxFor("Chat Access")).toBeTruthy());
expect(screen.queryByText(/disabled some of the permissions/i)).toBeNull();
await fireEvent.click(boxFor("Chat Access"));
await waitFor(() =>
expect(
screen.getByText(/disabled some of the permissions/i),
).toBeTruthy(),
);
});
it("hands control back once transition:generic is unchecked", async () => {
render(OAuthConsent);
await waitFor(() => expect(boxFor("Generic Access")).toBeTruthy());
await fireEvent.click(boxFor("Generic Access"));
await waitFor(() =>
expect(boxFor("repo:app.bsky.feed.post").disabled).toBe(false),
);
});
it("re-locks them when transition:generic is selected again", async () => {
render(OAuthConsent);
await waitFor(() => expect(boxFor("Generic Access")).toBeTruthy());
const generic = boxFor("Generic Access");
await fireEvent.click(generic);
await waitFor(() =>
expect(boxFor("repo:app.bsky.feed.post").disabled).toBe(false),
);
await fireEvent.click(boxFor("repo:app.bsky.feed.post"));
await fireEvent.click(boxFor("Generic Access"));
await waitFor(() => {
const superseded = boxFor("repo:app.bsky.feed.post");
expect(superseded.disabled).toBe(true);
expect(superseded.checked).toBe(true);
});
});
});
+1
View File
@@ -260,6 +260,7 @@ export const mockData = {
telegramVerified: false,
signalUsername: null,
signalVerified: false,
legacyLoginAlerts: true,
...overrides,
}),
describeServer: (overrides?: Record<string, unknown>) => ({
+28 -15
View File
@@ -98,43 +98,56 @@ test-store-asan:
test-unit:
SQLX_OFFLINE=true cargo test --test dpop_unit --test validation_edge_cases --test scope_edge_cases
store_test := "SQLX_OFFLINE=true TRANQUIL_TEST_BACKEND=store TRANQUIL_PDS_ALLOW_INSECURE_SECRETS=1 DISABLE_RATE_LIMITING=1 TRANQUIL_LEXICON_OFFLINE=1 SKIP_IMPORT_VERIFICATION=true cargo nextest run -E 'not package(tranquil-store) and not binary(store_parity)'"
test-auth:
./scripts/run-tests.sh --test oauth --test oauth_lifecycle --test oauth_scopes --test oauth_security --test jwt_security --test session_management --test change_password --test password_reset
{{store_test}} --test oauth --test oauth_lifecycle --test oauth_scopes --test oauth_security --test jwt_security --test session_management --test change_password --test password_reset
test-admin:
./scripts/run-tests.sh --test admin_email --test admin_invite --test admin_moderation --test admin_search --test admin_stats
{{store_test}} --test admin_email --test admin_invite --test admin_moderation --test admin_search --test admin_stats
test-sync:
./scripts/run-tests.sh --test sync_repo --test sync_blob --test sync_conformance --test sync_deprecated --test firehose_validation
{{store_test}} --test sync_repo --test sync_blob --test sync_conformance --test sync_deprecated --test firehose_validation
test-repo:
./scripts/run-tests.sh --test repo_batch --test repo_blob --test record_validation --test lifecycle_record
{{store_test}} --test repo_batch --test repo_blob --test record_validation --test lifecycle_record
test-identity:
./scripts/run-tests.sh --test identity --test did_web --test plc_migration --test plc_operations --test plc_validation
{{store_test}} --test identity --test did_web --test plc_migration --test plc_operations --test plc_validation
test-account:
./scripts/run-tests.sh --test lifecycle_session --test delete_account --test invite --test email_update --test account_notifications
{{store_test}} --test lifecycle_session --test delete_account --test invite --test email_update --test account_notifications
test-security:
./scripts/run-tests.sh --test security_fixes --test banned_words --test rate_limit --test moderation
{{store_test}} --test security_fixes --test banned_words --test rate_limit --test moderation
test-import:
./scripts/run-tests.sh --test import_verification --test import_with_verification
{{store_test}} --test import_verification --test import_with_verification
test-misc:
./scripts/run-tests.sh --test actor --test commit_signing --test image_processing --test lifecycle_social --test notifications --test server --test signing_key --test verify_live_commit
{{store_test}} --test actor --test commit_signing --test image_processing --test lifecycle_social --test notifications --test server --test signing_key --test verify_live_commit
test *args:
@just test-unit
./scripts/run-tests.sh {{args}}
test-embedded *args:
@just test-unit
SQLX_OFFLINE=true TRANQUIL_TEST_BACKEND=store TRANQUIL_PDS_ALLOW_INSECURE_SECRETS=1 DISABLE_RATE_LIMITING=1 TRANQUIL_LEXICON_OFFLINE=1 SKIP_IMPORT_VERIFICATION=true cargo nextest run -E 'not binary(store_parity)' {{args}}
{{store_test}} {{args}}
test-one name:
./scripts/run-tests.sh --test {{name}}
{{store_test}} --test {{name}}
test-full *args:
@just test-unit
@just services-up
eval "$(tranquil-dev-services env)" && SQLX_OFFLINE=true cargo nextest run --features tranquil-pds/s3 -E 'not package(tranquil-store)' {{args}}
test-pg *args:
@just test-unit
./scripts/run-tests.sh {{args}}
services-up:
tranquil-dev-services up
services-down:
tranquil-dev-services down
infra-start:
./scripts/test-infra.sh start
@@ -0,0 +1,2 @@
ALTER TABLE blobs DROP CONSTRAINT IF EXISTS blobs_pkey;
ALTER TABLE blobs ADD PRIMARY KEY (cid, created_by_user);
+1 -1
View File
@@ -76,7 +76,7 @@ in
server = {
host = mkOption {
type = types.str;
default = "127.0.0.1";
default = "[::1]";
description = "Host for tranquil-pds to listen on";
};
-5
View File
@@ -1,7 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
INFRA_SCRIPT="$SCRIPT_DIR/test-infra.sh"
cleanup() {
echo ""
@@ -12,10 +11,6 @@ trap cleanup EXIT
"$INFRA_SCRIPT" start
source "${TMPDIR:-/tmp}/tranquil_pds_test_infra.env"
echo ""
echo "Running database migrations..."
sqlx database create 2>/dev/null || true
sqlx migrate run --source "$PROJECT_DIR/migrations"
echo ""
ulimit -n 65536
echo "Building test binaries..."
-4
View File
@@ -5,8 +5,6 @@
# repo tooling
just,
podman,
podman-compose,
# rust tooling
clippy,
@@ -38,8 +36,6 @@ mkShell {
packages = [
just
podman
podman-compose
clippy
rustfmt
+182
View File
@@ -0,0 +1,182 @@
DEV_DIR="${XDG_STATE_HOME:-$HOME/.local/state}/tranquil-dev-services"
ENV_FILE="$DEV_DIR/services.env"
PG_DIR="$DEV_DIR/pg"
PG_LOG="$DEV_DIR/pg.log"
PG_PORT=54329
PG_DATABASE=tranquil
GARAGE_DIR="$DEV_DIR/garage"
GARAGE_CONF="$GARAGE_DIR/garage.toml"
GARAGE_LOG="$DEV_DIR/garage.log"
GARAGE_PID_FILE="$DEV_DIR/garage.pid"
GARAGE_S3_PORT=3990
GARAGE_RPC_PORT=3901
GARAGE_RPC_SECRET=6465767365637265746465767365637265746465767365637265746465767365
S3_BUCKET=tranquil-dev
LOOPBACK="::1"
if [ "$(id -u)" = 0 ]; then
echo "PostgreSQL won't run as root, use a lowerclass user" >&2
exit 1
fi
write_garage_conf() {
mkdir -p "$GARAGE_DIR/meta" "$GARAGE_DIR/data"
cat > "$GARAGE_CONF" << EOF
metadata_dir = "$GARAGE_DIR/meta"
data_dir = "$GARAGE_DIR/data"
db_engine = "lmdb"
replication_factor = 1
rpc_bind_addr = "[$LOOPBACK]:$GARAGE_RPC_PORT"
rpc_public_addr = "[$LOOPBACK]:$GARAGE_RPC_PORT"
rpc_secret = "$GARAGE_RPC_SECRET"
[s3_api]
s3_region = "tranquil"
api_bind_addr = "[$LOOPBACK]:$GARAGE_S3_PORT"
root_domain = ".s3.tranquil.dev"
EOF
}
garage_cli() {
garage -c "$GARAGE_CONF" "$@"
}
port_is_open() {
echo 2>/dev/null > "/dev/tcp/$LOOPBACK/$1"
}
wait_for_port() {
local port=$1
for _ in $(seq 1 60); do
if port_is_open "$port"; then
return 0
fi
sleep 0.5
done
echo "Nothing spun up on [$LOOPBACK]:$port" >&2
return 1
}
start_postgres() {
if [ -f "$PG_DIR/postmaster.pid" ] && pg_ctl -D "$PG_DIR" status >/dev/null 2>&1; then
echo "Postgres is already running"
return
fi
if [ ! -f "$PG_DIR/PG_VERSION" ]; then
mkdir -p "$PG_DIR"
initdb -D "$PG_DIR" -U postgres --auth=trust >/dev/null
fi
if ! pg_ctl -D "$PG_DIR" -l "$PG_LOG" -w \
-o "-p $PG_PORT -k $PG_DIR -c listen_addresses=$LOOPBACK" start >/dev/null; then
echo "Postgres wouldn't start. Please inspect $PG_LOG" >&2
exit 1
fi
wait_for_port "$PG_PORT"
if ! psql -h "$LOOPBACK" -p "$PG_PORT" -U postgres -lqt 2>/dev/null | cut -d'|' -f1 | grep -qw "$PG_DATABASE"; then
createdb -h "$LOOPBACK" -p "$PG_PORT" -U postgres "$PG_DATABASE"
fi
echo "PostgreSQL is up on [$LOOPBACK]:$PG_PORT"
}
layout_version() {
garage_cli layout show 2>/dev/null |
awk '/Current cluster layout version:/{print $NF; found=1} END{if (!found) print 0}'
}
start_garage() {
if port_is_open "$GARAGE_S3_PORT"; then
echo "Garage object storage is already running"
else
write_garage_conf
garage -c "$GARAGE_CONF" server >> "$GARAGE_LOG" 2>&1 &
echo $! > "$GARAGE_PID_FILE"
wait_for_port "$GARAGE_S3_PORT"
fi
local node_id
node_id=$(garage_cli node id 2>/dev/null |
awk 'match($0, /^[0-9a-f]{64}/) {print substr($0, RSTART, RLENGTH); exit}')
if [ -z "$node_id" ]; then
echo "Couldn't read the garage node id, please inspect $GARAGE_LOG" >&2
exit 1
fi
if ! garage_cli layout show 2>/dev/null | grep -q "${node_id:0:16}"; then
garage_cli layout assign "$node_id" -z dev -c 1GB
garage_cli layout apply --version "$(($(layout_version) + 1))"
fi
if ! garage_cli bucket list | grep -qw "$S3_BUCKET"; then
garage_cli bucket create "$S3_BUCKET" >/dev/null
fi
if ! garage_cli key list | grep -qw "$S3_BUCKET"; then
garage_cli key create "$S3_BUCKET" >/dev/null
fi
garage_cli bucket allow --read --write --owner "$S3_BUCKET" --key "$S3_BUCKET" >/dev/null
echo "Garage is up on [$LOOPBACK]:$GARAGE_S3_PORT"
}
write_env() {
local access_key secret_key
access_key=$(garage_cli key info "$S3_BUCKET" | awk '/^Key ID:/{print $3}')
secret_key=$(garage_cli key info --show-secret "$S3_BUCKET" | awk '/^Secret key:/{print $3}')
if [ -z "$access_key" ] || [ -z "$secret_key" ]; then
echo "Garage didn't show credentials for key $S3_BUCKET, please see $GARAGE_LOG" >&2
exit 1
fi
cat > "$ENV_FILE" << EOF
export DATABASE_URL="postgres://postgres@[$LOOPBACK]:$PG_PORT/$PG_DATABASE"
export TRANQUIL_PDS_TEST_INFRA_READY="1"
export TRANQUIL_PDS_ALLOW_INSECURE_SECRETS="1"
export DISABLE_RATE_LIMITING="1"
export TRANQUIL_LEXICON_OFFLINE="1"
export SKIP_IMPORT_VERIFICATION="1"
export BLOB_STORAGE_BACKEND="s3"
export S3_ENDPOINT="http://[$LOOPBACK]:$GARAGE_S3_PORT"
export S3_BUCKET="$S3_BUCKET"
export AWS_ACCESS_KEY_ID="$access_key"
export AWS_SECRET_ACCESS_KEY="$secret_key"
export AWS_REGION="tranquil"
EOF
}
stop_services() {
if [ -f "$PG_DIR/postmaster.pid" ]; then
pg_ctl -D "$PG_DIR" -m fast -w stop >/dev/null 2>&1 || true
fi
if [ -f "$GARAGE_PID_FILE" ]; then
local pid
pid=$(cat "$GARAGE_PID_FILE")
if [ "$(readlink "/proc/$pid/exe" 2>/dev/null)" = "$(command -v garage)" ]; then
kill "$pid" 2>/dev/null || true
for _ in $(seq 1 60); do
kill -0 "$pid" 2>/dev/null || break
sleep 0.5
done
kill -0 "$pid" 2>/dev/null && echo "Garage $pid is taking its sweet time to exit" >&2
fi
rm -f "$GARAGE_PID_FILE"
fi
if port_is_open "$GARAGE_S3_PORT"; then
echo "Smth is still listening on [$LOOPBACK]:$GARAGE_S3_PORT, not to do with us" >&2
fi
echo "Services have been stopped"
}
case "${1:-}" in
up)
mkdir -p "$DEV_DIR"
start_postgres
start_garage
write_env
echo "env at $ENV_FILE"
;;
down)
stop_services
;;
env)
cat "$ENV_FILE"
;;
*)
echo "usage: tranquil-dev-services <up|down|env>" >&2
exit 1
;;
esac

Some files were not shown because too many files have changed in this diff Show More