This also means we can clean up the last blob operations that were per cid rather than user+cid 🪓
We should be setting ourselves up to be able to go garbage collect blobs safely so deleting the logo blobs manually won't matter anyway.
The current tranquil database design only allows each blob to be owned by one account. This means that if a second account also has that blob, tranquil skips associated the blob with the account. That works fine a lot of the time, since blobs are looked up by cid and the blob exists. However, it can lead to loss of data under certain scenarios.
One example is where I upload a blob, the blob already exists in my instance so insertion is skipped (postgres requires cid to be unique in blobs, fjall only allows one owner per blob cid), I then decide to migrate off tranquil, the blob does not come with me since it is not mine.
Another example is where an account is deleted. If a blob was uploaded for account a, then account b uploads the same blob tranquil skips storing it since it exists. Then I delete account a, now account b's blob is missing.
I accidentally stumbled upon this when I migrated my account to my own tranuil instance and list blobs now lists 2 fewer blobs than before, two images that had been uploaded by accounts already on the PDS.
ps I found record_blobs a bit confusing, at first it looked like a blob ownership table, but then it turns out to just be used for migrations!
This PR makes the blob primary key be cid+user for postgres, and updates the queries to account for there being multiple "blobs" with the same cid. For queries that just care about the blob existing, it doesn't matter "whose" blob it is, so limit 1.
Most of the work is on the metastore side. Adds ref_count to track how many are referencing the blob since we can't just check for other rows. Instead of storing blobs directly, we now store a per account cid, and the blob reference itself is shared and keyed by cid only. This means some of these operations now require updating two places, so they're done in `batch`es.
With the new layout get_blob_value becomes simpler, all blob data is a single "table" or whatever it's called, so we just grab it using cid instead of looking it up for the user.
Migrates blobs rather than maintaining two different versions of the tables, although it seems like that could be supported.
I removed a test that asserted the old behavior, and added a reasonable (?) set of new tests that assert the new behavior, including a parity test.
This adds security alerts section to the communication settings, with one new item: a toggle for enabling/disabling legacy login alerts. It's enabled by default. Disabling it means that you no longer get emails when logging in through a non-2FA enabled login flow, like createSession.
The reason I tackled this is that I have a bot account that automatically refreshes its session, using username and app password, and I'm close to having 100 of these emails in my inbox. I also tried to just disable TOTP but I wasn't allowed (I fixed that in a different PR).
English and Swedish translations were me, the rest were MyMemory. I imagine someone can improve on them after this is merged!
This removes the `require_legacy_session_mfa` call on the TOTP disable path. It looks at the last session, which isn't really relevant, and we're getting a fresh TOTP from the form anyway, which should be enough proof to allow disabling TOTP.
The way I bumped into this was having a "bot account" log in with app password on some app on a schdule, triggering lots of warning emails. So I tried to disable TOTP to stop the flood and wasn't allowed.
Remove the requirement that the user's did can be controlled by the
server rotation key and signing key. This was preventing users who
didn't want to trust the PDS with their rotation keys from activating
their accounts. Errors are removed but we issue debug messages.
Concurrent or retried com.atproto.server.refreshSession calls presenting the
same refresh token hit the reuse-detection path, which deleted the session and
returned "Refresh token has been revoked due to suspected compromise" —
logging users out at random. The legacy flow had no grace period, unlike OAuth.
Mirror the reference atproto PDS: every rotated refresh token gets a 2h grace
window measured from its own rotation time (used_refresh_tokens.used_at in
postgres; a rotated_at_ms field appended to the metastore used-marker, with
old-format markers decoding as outside the window). A refresh presenting a
recently-rotated token is served the session's current tokens, re-minted on
the fly with the same jti/expiry — signed JWTs are never persisted. Reuse
outside the window still revokes the session.
The grace lookup returns the session's encrypted signing key so the handler
verifies the presented token's signature before minting replacement tokens or
revoking a session; a forged token bearing a known jti gets a generic
rejection with no side effects.
Integration tests asserting the old replay-gets-401 behavior are reworked to
the new contract and now also cover forged-signature replays and
out-of-window revocation.
Collapse the three ad-hoc short-code generators into one canonical generator
plus a shared normalizer:
- util::generate_token_code now emits the uppercase base32 XXXXX-XXXXX display
form; new util::normalize_token_code canonicalizes user input (uppercase,
strip hyphen/whitespace).
- email_token and legacy_2fa now generate via util, store the normalized form,
and compare normalized input. Their private generate_short_token/generate_code
(and BASE32_CHARS/CODE_LENGTH) are removed.
- PLC (request/sign) and password reset inline util::generate_token_code,
persist the normalized form, email the display form, and normalize input
before lookup. The generate_plc_token/generate_reset_code wrappers are removed.
Behavior changes: legacy login-2FA codes go from 8-digit numeric to XXXXX-XXXXX;
PLC and password-reset codes go from lowercase to uppercase. All four code types
are now accepted case-/hyphen-insensitively. OAuth web-login 2FA, account
deletion, and the long verification_token blobs are intentionally untouched.
Tests: add util normalize tests + email_token/legacy_2fa case/hyphen tests;
update integration tests to expect the canonical stored form and the new
emailed format.
Johanna LarssonandTangled
LewisandTangled
nelindandTangled
Louis EscherandTangled
Edmund EdgarandTangled
LewisandTangled
aveandTangled