feat(docs): add some general PDS debugging documentation

docs/1_WELCOME_TO_TRANQUIL_PDS.md

@@ -64,3 +64,12 @@ So you want to migrate your account from a different PDS to Tranquil? The UI of
 The good thing about migrations is that for most of the rickety process, the real identity of your account hasn't actually moved, it's most of the unimportant data copied over (ie. everything but switching over your keys to say "hey this account is hosted on this specific PDS instance and not any other one"). So if something goes wrong in our migrator or any other, feel free to dip into a bit of Tranquil PDS admin and delete out the half-formed account ~~fetus~~ (sorry).
 
 One thing that **tends to not go smoothly in our own migrator** right now is **deactivation on source PDS** -> since we use OAuth for our migrator, and the reference PDS disallows using OAuth to deactivate an account at time of writing, that part of the migration will fail - which is fine, a migration doesn't *need* the source account to be deactivated - but the Bluesky AppView for example definitely likes events to come in an exact order for it to update nicely. I have gotten around its multi-hour auth cache by manually cURLing login + deactivate endpoint in terminal to the source PDS, and then doing a no-op handle update API call on the newly migrated target PDS. Like a bit of a defribulation on the firehose such that the Bluesky AppView really does in fact pick up on the move instead of just screaming about jwt issuers for a couple of hours.
+
+> 🌺 Nel
+> Backups people BACKUPS!!
+>
+> You can in many cases be generally sure that stuff will go okay and you can get your way out of a weird situation with some help, but that doesn't mean that a backup isn't good and important! some situations are genuinely mathematically impossible to get out of unless you have the proper backups in place.
+>
+> First take a backup of your data. Get a repo export and download all your blobs. [Skeetgen's export tool](https://mary-ext.github.io/skeetgen/export.html) is wonderful for this. It's made by the wonderful [Mary](https://witchsky.app/profile/did:plc:ia76kvnndjutgedggx2ibrem), it's a bit old by now and made for making archives of Bluesky posts specifically but it will take care of all your public data just fine, not just Bluesky. If Skeetgen doesn't do it for you then the same dev has also made [boat](https://boat.kelinci.net/) which has an "Export repository" tool and an "Export blobs" tool that together acomplish the same thing. If you care about your Bluesky settings, mutes and a few other odities then backup your preferences too. These are a bit more involved to get a hand of. Currently the easiest way is using the [goat](https://github.com/bluesky-social/goat) CLI tool. Specifically `goat bsky prefs export` (remember this is private data so you need to be logged in to goat to use it).
+>
+> Second make sure you have direct control over your DID (ie. not through the PDS). For PLC identities you want to add your own rotation key pair, make sure it's first in the list and *keep the private key safe* it can be used to fully take over your account if you aren't a little careful with it. boat mentioned above has a "Generate secret keys" tool and "Apply PLC operations" tool that you can use to do this. Either key type is fine. If you're on a did:web DID I trust you already have control over it, if you don't have control over it and it's domain then I trust that since you know how to get into that situation you also know enough to know how fucked you probably are.

docs/3_DEBUGGING_YOUR_SETUP.md

@@ -0,0 +1,22 @@
+PDSs are part of a distributed system with many different actors and services interacting with each other. That fundamentally means that many many things can go wrong. If you're having issues like Bluesky not picking up your posts then it's time to get your debugging hands dirty! Below is some general debugging advies as well as a list of frequent issues, how to spot them and how to fix them. If you belive your issue isn't listed below don't be afraid to contact us! We're more than willing to help if we can. Our official account is @tranquil.farm and we are additionally both available in the tangled discord in our "guest" channel (tysm tangled!) or in the atproto-touchers (somewhat less frequently however).
+
+> As always: TAKE BACKUPS!!
+> View the note on backups at the bottom of "Welcome to Tranquil PDS".
+
+# General
+As always debugging starts with good tools for investigating what is going wrong. For atproto those tools are primarily <pdsls.dev> and <debug.hose.cam>. They are your friends! Get acquainted with them.
+
+PDSls is (as the name implies) good for debugging the PDS side of things: is your PDS up? accessible to the outside world (or at least your PC)? what does the firehose coming from your PDS look like? but also other account level details: is your handle valid? what are your current rotation keys? signing key? etc.
+
+debug.hose.cam is wonderful to get your an overview of what the relay side of things looks like. Open it up and type in a handle, DID or PDS URL and it'll quickly fetch a bunch of useful debug info for you.
+
+# `seq` from PDS is behind `seq` stored by relays.
+This is one of the less common sources of issues but a real head scratcher if you don't know to look out for it. This is generally recognisible by no changes you take propagating to the network and all the relays showing your PDS as `idle` when you look at debug.hose.cam. That last one especially is a tell tale sign!!
+
+Fixing this isn't too hard. You want to figure out what `seq` the relays think your PDS is at, the easiest way to do this is dig through your browser devtools network tab and find the `com.atproto.sync.getHostStatus` calls for each relay and find the `seq` in the response.
+
+> 🌺 Nel
+>
+> #protip double check that the `seq` is actually misaligned with your PDS by getting the latest `seq` from your PDS using PDSls's firehose feature and a cursor value of `0` when connecting. If the relays don't have a `seq` that's bigger than your PDSs `seq` then this isn't your issue!
+
+Now you need to update the PDS to use a `seq` that's *at least* one above the highest `seq` any of the relays have. Currently doing this on Tranquil depends on your used storage backend. For the default Postgres repo store you want to shutdown Tranquil itself, open the DB in `psql` and run `SELECT setval('firehose_seq, <new updated seq value>');` and the start Tranquil back up. Take a few actions to make sure new events get sent out. Everything should work once all the downstream consumers have had a chance to resync. You can double check with debug.hose.cam that all the relays show your PDS as `active`. Some might need a lil push with a request crawl, luckily debug.hose.cam also makes that easy :p (ignore that it says it failed to issue a request crawl, CORS is fickle).