Johanna LarssonandTangled
695a7d981c
The current tranquil database design only allows each blob to be owned by one account. This means that if a second account also has that blob, tranquil skips associated the blob with the account. That works fine a lot of the time, since blobs are looked up by cid and the blob exists. However, it can lead to loss of data under certain scenarios. One example is where I upload a blob, the blob already exists in my instance so insertion is skipped (postgres requires cid to be unique in blobs, fjall only allows one owner per blob cid), I then decide to migrate off tranquil, the blob does not come with me since it is not mine. Another example is where an account is deleted. If a blob was uploaded for account a, then account b uploads the same blob tranquil skips storing it since it exists. Then I delete account a, now account b's blob is missing. I accidentally stumbled upon this when I migrated my account to my own tranuil instance and list blobs now lists 2 fewer blobs than before, two images that had been uploaded by accounts already on the PDS. ps I found record_blobs a bit confusing, at first it looked like a blob ownership table, but then it turns out to just be used for migrations! This PR makes the blob primary key be cid+user for postgres, and updates the queries to account for there being multiple "blobs" with the same cid. For queries that just care about the blob existing, it doesn't matter "whose" blob it is, so limit 1. Most of the work is on the metastore side. Adds ref_count to track how many are referencing the blob since we can't just check for other rows. Instead of storing blobs directly, we now store a per account cid, and the blob reference itself is shared and keyed by cid only. This means some of these operations now require updating two places, so they're done in `batch`es. With the new layout get_blob_value becomes simpler, all blob data is a single "table" or whatever it's called, so we just grab it using cid instead of looking it up for the user. Migrates blobs rather than maintaining two different versions of the tables, although it seems like that could be supported. I removed a test that asserted the old behavior, and added a reasonable (?) set of new tests that assert the new behavior, including a parity test.
Tranquil PDS
A Personal Data Server for the AT Protocol.
"A what for the what?" -> glad you asked: Bluesky, Tangled, and a bunch of other web applications use a federated protocol called AT Protocol (atproto). Your account lives on a PDS, a server that stores your posts, profile, follows, cryptographic keys, et cetera. The beauty is that a PDS is the only place your data lives permanently - so you can navigate any atproto app knowing that your data is yours and not getting locked behind any one app's walls.
We came together to make this PDS to enable and empower our users to better host their data on this shared protocol. All of our decisions as a project are guided by their usefulness to the community: PDS hosters and end-users both.
Comparatively: Bluesky the company created a "reference PDS" that we can self-host quite easily, and that's great, but Bluesky has an incentive to make software for themselves first & foremost, then secondly their software can be useful for us self-hosters. In contrast, Tranquil is not from a company, and will never be.
What's different about Tranquil PDS
It is a superset of the reference PDS, including:
- passkeys and 2FA: WebAuthn/FIDO2, TOTP, backup codes, trusted devices
- SSO login and signup
- did:web support: PDS-hosted subdomains or bring-your-own
- multi-channel communication: you can be notified via email, discord, telegram, and signal for verification and alerts
- granular OAuth scopes with a consent UI that allows unchecking specific scopes
- app passwords with the same granular permission scope system as OAuth
- account delegation: letting others manage an account with configurable permission levels
- a built-in web UI for account management, repo browsing, and admin
Unlike the ref PDS, Tranquil is a single binary with no nodejs runtime. That said, at time of writing, Tranquil does require postgres running separately.
Quick Start
cp example.toml config.toml
podman compose up db -d
just run
Configuration
See example.toml for all configuration options.
Note
The order of configuration precedence is: environment variables, then a config file passed via
--config, then/etc/tranquil-pds/config.toml, then the built-in defaults. So you can use environment variables, or a config file, or both.
Development
Run just to see available commands.
just test
just lint
Nix users can enter a devshell with nix develop, or direnv allow to auto-enter via the bundled .envrc. Pre-built artifacts including the devshell are available from our binary cache.
Production Deployment
Quick Deploy (Docker/Podman Compose)
docker-compose.prod.yaml pulls the prebuilt image atcr.io/tranquil.farm/tranquil-pds:latest. Sign in to the registry first with podman login atcr.io. The Containers guide covers building from source.
cp example.toml config.toml
Edit config.toml with your values and generate secrets with openssl rand -base64 48. Set the postgres password to match docker-compose.prod.yaml. nginx needs a TLS certificate before it starts, so follow the wildcard cert steps in the Containers guide.
podman-compose -f docker-compose.prod.yaml up -d
Installation Guides
Community
"Let's connect!" or whatever linkedin-types say
We currently don't have a shared space to chat and organize Tranquil things, but we're very interested in changing that in the near future. What do you suggest? Anything but a discord server.
Core team
Amazing contributors
- @isabelroses.com
- @quilling.dev
- @koi.rip
- @bas.sh
- @nekomimi.pet
- @islacant.win
- @a.starrysky.fyi
- @sans-self.org
- @tachyonism.tngl.sh
- @trezy.codes
- @ave.zone
- Could be your name here too!
Tranquil PDS instances in the wild!
- Tranquil Farm
- Your instance here!! Don't be a stranger.
Special thanks
This project is very grateful to @nonbinary.computer, @juliet.paris, @mary.my.id, @baileytownsend.dev, and @ptr.pet for their help and their code to lean on.
License
AGPL-3.0-or-later. Documentation is CC BY-SA 4.0. See LICENSE for details.