Introduces s3s and s3s-fs to run an in-memory s3 server in tests. With this we can promote s3 tests to run in `just test`, they won't need containers, and they're super fast. Adds `just test-s3` to run them more easily.
```
Nextest run ID db9a9530-738e-4c92-ad3a-23289252cac2 with nextest profile: default Starting 6 tests across 1 binary PASS [ 0.356s] tranquil-storage::s3 put_stream_error_aborts_upload PASS [ 0.356s] tranquil-storage::s3 put_stream_empty_aborts_upload PASS [ 0.358s] tranquil-storage::s3 copy PASS [ 0.360s] tranquil-storage::s3 put_get_head_delete PASS [ 0.360s] tranquil-storage::s3 put_stream PASS [ 0.447s] tranquil-storage::s3 put_stream_multipart ──────────── Summary [ 0.448s] 6 tests run: 6 passed, 0 skipped
```
Also optimizes more hashing functions because it matters in tests.
We've got a bunch of tests that never pass because they time out on their own, or that take 10s because they test a 10s timeout, that we can rethink. Together they introduce a lot of slowness.
This puts anything over 10s in CI into test jail.
Many of these don't finish anyway on my computer, I gave up around 40 minutes. But for the ones that did, it's a big speed up! For the others I guess they need more love.
gauntlet-pr: 67.9s with it, 120.9s without
test-store: 508.8s with it, still running at 1313s (I gave up)
test-store-sim-nightly: ran for like 40 minutes each attempt without finishing. I just gave up.
test-store-asan: only linux? but no reason why we can't apply the fix here too
Overall `just test` runs about 10-20s faster on this branch than current main, for me, with around 170s instead of around 180-190s. But the cool thing is the compile time. Making a simple edit to any file involved and running `just test` on main takes about 60s to recompile the test binaries. On this branch it takes less than 15s.
This is done by replacing a ton of separate test binaries with a single megabinary.
Only test files that looked safe have been included, avoiding ones that do stuff like setting env.
test-unit was building a second copy of the code with different features, and then the main test command ran the same tests anyway, so we just built and ran the tests twice
Locally shaves a minute off of a full build+test, on this branch (so both runs include the fsync change etc)
We're at [ 173.444s] overall for `just test` now
Flushing to disk all the time ensures no partial states when writing to disk during awkwardnesses like crashes in prod risking real user data. That's good!
We're also doing it while running tests. Less good! Well, at least not important!
Comparing a clean test run (`just test --no-fail-fast`) and a run on this branch, on my MacOS M4 Pro, I get:
`**Summary** [ 517.723s] **1424** tests run: **1405** **passed** (**7** **slow**), **19** **failed**, **15** **skipped**`
before and
`**Summary** [ 326.369s] **1424** tests run: **1405** **passed**, **19** **failed**, **15** **skipped**`
after.
The reason I run `--no-fail-fast` is that a bunch of tests always fail and time out and stuff
Johanna LarssonandTangled
LewisandTangled
Lewis
TrezyandTangled
