Commit Graph
498 Commits
Author SHA1 Message Date
Evan JarrettandClaude Opus 5 c615d7253b credhelper: make config writes non-destructive
Follow-up to 8a556a5. v3 keys accounts by DID and, unlike v2, rewrites the
config during ordinary `docker pull` traffic (migration on first read, DID
backfill after a successful get) rather than only on explicit login. That moved
three latent ways to lose a device.json onto the hot path. A user cannot recover
from any of them: once the file is gone they no longer know which accounts they
had.

save() is now atomic. It writes a temp file in the same directory, fsyncs, and
renames, so the config always holds either the previous contents or the complete
new contents. The truncate-then-write it replaced left a zero-length window that
Ctrl-C, Docker reaping the helper, or a suspend could land in. The fsync matters
on its own: on ext4/XFS a rename can become durable while the new file's data
blocks are not, which resurrects the empty-file case. The rename also replaces
the destination's mode, which incidentally repairs a config restored from a
backup as world-readable (os.WriteFile's perm argument only applies at creation).

An unreadable config no longer degrades into an empty one that the next write
commits. loadConfig returns a usable empty config so read paths can still print
something helpful, but it now marks the unrecoverable cases with
errConfigUnusable, and loadConfigForWrite refuses on that sentinel. This is the
downgrade path: the previous binary hard-gates on Version == 2, so without the
guard a reinstall of an older helper would wipe a v3 file on the first
`docker login`. loadConfig also stops explicitly on a version newer than it
understands instead of falling through the legacy probes to the same empty
config. All six write paths take the guard — get, store, erase, login, logout,
switch. login had the same warn-and-continue-then-save shape as store.

The guard keys on the sentinel rather than on any error, deliberately. The
v2 -> v3 migration returns a fully populated config alongside a "saving migrated
config" error when the directory is not writable, and every pre-v3 user passes
through that path on their next invocation; refusing there would hard-break
`docker pull` for exactly the population that is migrating.

migrateV2toV3 merges DID collisions deterministically. Two v2 entries collapse
onto one v3 key when a handle was renamed and the old entry was never cleaned up,
which is the case v3 exists to fix. The previous loop wrote both into the same
map slot in randomized iteration order, so which account survived varied run to
run, and when the stale one won, get would fail validation and remove it — the
user ended up with no account at all. Iteration is now sorted and collisions
resolve through v2EntryBeats (active entry, then the one holding a secret, then
the smaller key), with the loser donating its secret if the winner has none. The
"No account is dropped" comment was false and is now accurate.

Also deterministic: find() and upsert() resolve their scans through a shared
scanFor helper that prefers DID matches and breaks ties on the smallest key, so
two entries sharing a handle can no longer hand Docker a different secret on each
invocation. upsert additionally refuses to match a handle that already belongs to
a different DID, which previously let it overwrite an unrelated account's DID and
destroy that account's credentials. Nil map entries are tolerated throughout
rather than panicking on Docker's credential path.

resolveHandleDNS is bounded at 2s. It runs synchronously on every store and on
every get for a DID-less account, and net.LookupTXT applies no deadline of its
own, so a blackholing resolver (captive portal, split-horizon VPN) stalled
docker login and docker pull on each invocation.

Tests cover the paths that can lose credentials. Each was checked against the
pre-fix code: the collision, null-entry, DID-theft, find-determinism,
permissions, and out-of-place-write tests all fail or panic without their fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 19:19:40 -05:00
Evan JarrettandClaude Opus 4.8 8a556a5893 credhelper: key accounts by DID (v3 config) with lossless migration
The credential helper stored accounts keyed by handle, which broke on
handle renames and let the Docker `store` path (username+secret, no DID)
overwrite a good account with a DID-less one — how evan.jarrett.net on
buoy.cr ended up active with a blank DID.

Re-key everything by the stable DID, treating handle as a mutable display
label. DID is recovered client-side via standard AT-proto handle
resolution (DNS TXT _atproto.<handle> + HTTPS .well-known/atproto-did) —
no server change, no JWT, no auth, and no indigo pulled into the helper.

- resolve.go: stdlib handle->DID resolver
- config.go: v3 DID-keyed schema; find/activeAccount/upsert/rekey helpers;
  upsert never blanks a known DID; migrateV2toV3 re-keys existing files in
  place (no login lost; DID-less accounts stay provisional and self-heal)
- protocol.go: store resolves+preserves DID; get lazily backfills+re-keys;
  list reports the active account's handle (was arbitrary map iteration)
- status/switch/logout: display handle, key/compare by DID
- config_test.go: migration, upsert-never-blanks-DID, rekey, find, list,
  resolver parse helpers

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-02 13:38:45 -05:00
Evan JarrettandClaude Fable 5 37bab324d7 fix OAuth refresh-token burn on client cancellation causing sign-outs
When a Docker client canceled a slow /auth/token request mid-refresh, the
token-refresh POST was aborted client-side but completed on the PDS, which
rotated the refresh token. The rotated token was never received or persisted,
so the next refresh replayed the consumed token, got invalid_grant, and the
session (OAuth + UI) was deleted, signing the user out everywhere.

- Detach refresh POSTs from the inbound request context via a per-session
  RoundTripper (WithoutCancel + 30s cap); once a refresh starts it completes
- Persist session updates (rotated tokens, DPoP nonces) on a detached context
- Gate session deletion on IsSessionInvalidError: cancellation, timeouts, and
  transport errors no longer delete sessions; genuine invalid_grant still does
- Add phase timing to /auth/token and per-DID lock wait warnings to attribute
  the ~14s pre-refresh stalls that push requests past Docker's deadline

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-02 13:38:45 -05:00
Evan Jarrett 6e77311c22 minor ui fixes, update privacy page with manged holds list 2026-08-02 13:38:45 -05:00
Maarten RijkeandTangled 2e55352974 hold/pds: fix scan broadcaster predecessor check to compare successor did
checkPredecessor returned true for any hold with a non-empty successor field,
without verifying the successor was actually this hold.
This caused every hold running proactive scan discovery to queue manifests from
every other migrated hold on the network, producing 404 errors when the scanner
tried to fetch foreign blobs from its own S3.

Signed-off-by: Maarten Rijke <did:plc:fy4lwkc4hrd776vfkcrbzr5a>
2026-06-27 23:56:37 +03:00
lime360andTangled c0e20d7baf wrong url 2026-06-14 04:02:43 +03:00
Evan Jarrett 2eea5ff885 small fix for null columns and background pds inserts on push 2026-06-13 19:50:29 -05:00
Evan Jarrett 6758996300 add SBOM package diffing, verify hold-service captain records
- diff view gains a Packages tab with added/removed/changed/unchanged
  package tables and purl-derived type/license/upstream links
- captain records verified against the DID's atcr_hold service before
  caching (processor + batch backfill), preventing forged holds
- fix empty-handle updates clobbering cached handles and colliding on
  the UNIQUE constraint
- move fillPrevCIDs into repo.go; DirectRepoOperator is now canonical,
  repomgr kept as a test oracle
- surface read-only crew status in hold selector
- reconcile docs
2026-06-13 12:49:03 -05:00
Evan Jarrett ab4a4ebf9d admin panel long running imrovements, billing fixes, ui cleanup
1. Multiple registry domains + per-user domain preference

The biggest feature. The appview can serve several registry domains (e.g. buoy.cr, atcr.io),
and users can now pick which one shows up in their pull/push commands.

- Lexicon/record: adds registryDomain (and documents ociClient)
to the sailor profile (lexicons/.../profile.json, pkg/atproto/lexicon.go).
- DB: new registry_domain column on users (schema.sql + migration 0027),
with GetUserByDID/Handle reads, UpdateUserRegistryDomain writer,
and Jetstream caching it on profile updates (writes unconditionally so clearing propagates).
- UI/handlers: new UpdateRegistryDomainHandler + /api/profile/registry-domain route,
a <select> in the user settings panel (only shown when >1 domain configured), and resolveRegistryURL()
which falls back to the primary domain if the user's pref is stale/removed. Tests added for all of it.

2. default_hold_did removed → first managed_holds entry is the default

Consolidates two overlapping config fields into one. ServerConfig.DefaultHoldDID is gone;
PrimaryHoldDID() now returns managed_holds[0]. managed_holds is now REQUIRED.
Updated in config, validation, server wiring, test harness, example YAML, and the deploy template.

3. Admin long-running operations → generic background-job framework

New pkg/hold/admin/jobs.go introduces a reusable startJob/jobRegistry pattern
 (a detached context.Background() job + a /admin/api/jobs/{key}/status polling endpoint).
This replaces the bespoke scan-backfill goroutine state machine, and now also wraps crew tier remap and crew import
all three previously looped synchronously on the request context and got 504'd/cancelled mid-run by the reverse proxy.
 Forms switched from POST-redirect to htmx fragments (job_progress.html, job_result.html, crew_import_results.html)
 the old crew_import_results.html page and scan_backfill_progress.html partial were deleted.
This is also captured as a new rule in CLAUDE.md.

4. Cascade-delete manifest on last-tag deletion

DeleteTagHandler now, after removing the last tag pointing to a digest, cascade-deletes the manifest itself
 (PDS + DB + hold blob purge) — but only if it's not a child of a manifest list (multi-arch parent).
 New GetTagDigest and ShouldCascadeDeleteManifest queries back it, plus cascade_delete_test.go.
 Also switches tag rkey computation to the atproto.RepositoryTagToRKey helper.

5. Billing simplification

Drops the OwnerBadge config option (hold-owner supporter badge).
The user-profile template no longer special-cases an "owner" badge value (only "Captain").
Example tiers renamed to the nautical scheme (deckhand/bosun/quartermaster).

6. Build/deploy: go generate always runs via Make

make generate is now a phony target that always runs go generate ./... (regenerating cbor_gen, icon sprites, etc.),
 and build-trixie depends on it. The deploy tooling (provision.go/update.go)
drops its own runGenerate calls since the Makefile handles it.

7. New cmd/firehose-tap tool (untracked)

A standalone CLI that subscribes to a com.atproto.sync.subscribeRepos endpoint and pretty-prints events,
with emphasis on Sync 1.1 compliance fields (per-op prev CIDs, commit prevData) and a --validate CI mode.
Fits with the recent "more sync1.1 compliant" commit.
2026-06-05 20:57:25 -05:00
Evan Jarrett efabb677e4 clean up some functions to use indigo helpers. make repomgr more sync1.1 compliant 2026-05-26 20:26:29 -05:00
Anhgelus MorhtuuzhandTangled f330e4db5e Revert "remove go install path for now"
This reverts commit e81681d61b.
2026-05-25 17:02:13 +03:00
Evan Jarrett fd3132a545 create a seamark credential helper, have the appview redirect to the source if navigating to go paths 2026-05-25 09:01:46 -05:00
Evan Jarrett 2f02d3e7e5 add a way to change crew tier on admin page 2026-05-18 22:39:54 -05:00
Evan Jarrett ecd689a7e1 billing improvements 2026-05-18 22:10:20 -05:00
Evan Jarrett 63909aaca0 add webhook for storage quota percentage 2026-05-17 19:57:09 -05:00
Evan Jarrett 6b4781941b fix display size on repo page v0.1.4 2026-05-17 15:34:30 -05:00
Evan Jarrett f4acdd76eb fix purge manifest when repo owner deletes from UI 2026-05-17 15:24:34 -05:00
Evan Jarrett baa68d7a3f minor bugfix for credential helper update command 2026-05-17 13:36:22 -05:00
Evan Jarrett a66e31f521 fix install script to follow multiple curl redirects v0.1.3 2026-05-17 10:36:08 -05:00
Evan Jarrett b5495af2b6 billing bugfixes 2026-05-16 19:39:57 -05:00
Evan Jarrett 19bb66f6bb don't use docker hub for images 2026-05-16 17:58:01 -05:00
Evan Jarrett 04e10b6818 fix quota message 2026-05-16 17:20:12 -05:00
Evan Jarrett 038993c814 fix concurrency test 2026-05-16 12:18:06 -05:00
Evan Jarrett 902fba4553 convert alert to modal. go fix the codebase 2026-05-16 11:59:47 -05:00
Evan Jarrett be64dbb364 add more integration tests 2026-05-11 19:53:13 -05:00
Evan Jarrett a0cc862798 lots of new work for authenticating between appview -> hold. fixed quota handling, improve integration tests for round-trip push/pull/auth/quota checking 2026-05-11 09:20:55 -05:00
Evan Jarrett 3271ac6dcc fix labeler issues 2026-05-09 21:21:20 -05:00
Evan Jarrett 966e391a91 fix some connection issues with jetstream causing a crashloop 2026-05-09 15:47:19 -05:00
Evan Jarrett 98a2cfea59 improve UI around credential helper authorization. have the hold requestCrawl on restart. Update comments that relay_endpoints must suport listreposbycollection 2026-05-08 20:44:04 -05:00
Evan Jarrett 3533f07ecb minor bug fixes, add ability see starred repos 2026-05-06 21:55:47 -05:00
Evan Jarrett 56f3b2fc2f fix star not being filled on when you star a repo 2026-05-06 21:24:36 -05:00
Evan Jarrett 2d0872790d delete manifests correct when deleted from the PDS fixes https://tangled.org/did:plc:e3kzdezk5gsirzh7eoqplc64/issues/18 2026-05-06 21:13:00 -05:00
Evan Jarrett 419d05057e fix to artifact diff. add missing file 2026-05-04 23:18:40 -05:00
Evan Jarrett e1483f7aa5 fix favicon in navbar 2026-05-04 21:56:26 -05:00
Evan Jarrett b2d6842bb7 clean up old migration code. minor bug fixes with appview ui 2026-05-04 21:52:28 -05:00
Evan Jarrett 1ac8af74d5 fix labeler deployment 2026-05-03 20:49:56 -05:00
Evan Jarrett c2250c8787 remove binary file 2026-05-03 20:03:52 -05:00
Evan Jarrett ba05b20364 update goreleaser 2026-05-03 14:51:05 -05:00
Evan Jarrett 18b000d0c1 simplify appview label subs 2026-05-03 10:48:36 -05:00
Evan Jarrett 4328eda814 holds now listen for deletes and labelers for takedowns. GC will defer takedowns for a grace period in case of reversal 2026-05-02 23:31:41 -05:00
Evan Jarrett 7b4a2e22a2 fix more db query issues, improve labeler usage 2026-05-02 22:59:32 -05:00
Evan Jarrett ab66028151 more labeler improvements. standardize did work between labeler and hold. improve sql race conditions on local-only db 2026-05-02 22:13:53 -05:00
Evan Jarrett 410183f13b first pass at implementing a label service 2026-05-02 11:14:44 -05:00
Evan Jarrett 35aa19e2b2 fix captain record check ttl 2026-05-02 11:11:01 -05:00
Evan Jarrett 13a793ca90 improve admin tooling 2026-04-29 10:37:36 -05:00
Evan Jarrett 9af6eccc9d improvements to how scanning works, and helmchart ui 2026-04-29 10:12:25 -05:00
Evan Jarrett f9ba8ff62b pass through 429 retry-after from pds 2026-04-29 08:50:54 -05:00
Evan Jarrett e81681d61b remove go install path for now 2026-04-23 10:47:33 -05:00
Evan Jarrett dfee21e3d3 fix install instructions 2026-04-23 10:38:55 -05:00
Evan Jarrett 1bff09ecbe fix settings page 2026-04-22 21:49:51 -05:00