156 Commits
Author SHA1 Message Date
Evan Jarrett ab4a4ebf9d admin panel long running imrovements, billing fixes, ui cleanup
1. Multiple registry domains + per-user domain preference

The biggest feature. The appview can serve several registry domains (e.g. buoy.cr, atcr.io),
and users can now pick which one shows up in their pull/push commands.

- Lexicon/record: adds registryDomain (and documents ociClient)
to the sailor profile (lexicons/.../profile.json, pkg/atproto/lexicon.go).
- DB: new registry_domain column on users (schema.sql + migration 0027),
with GetUserByDID/Handle reads, UpdateUserRegistryDomain writer,
and Jetstream caching it on profile updates (writes unconditionally so clearing propagates).
- UI/handlers: new UpdateRegistryDomainHandler + /api/profile/registry-domain route,
a <select> in the user settings panel (only shown when >1 domain configured), and resolveRegistryURL()
which falls back to the primary domain if the user's pref is stale/removed. Tests added for all of it.

2. default_hold_did removed → first managed_holds entry is the default

Consolidates two overlapping config fields into one. ServerConfig.DefaultHoldDID is gone;
PrimaryHoldDID() now returns managed_holds[0]. managed_holds is now REQUIRED.
Updated in config, validation, server wiring, test harness, example YAML, and the deploy template.

3. Admin long-running operations → generic background-job framework

New pkg/hold/admin/jobs.go introduces a reusable startJob/jobRegistry pattern
 (a detached context.Background() job + a /admin/api/jobs/{key}/status polling endpoint).
This replaces the bespoke scan-backfill goroutine state machine, and now also wraps crew tier remap and crew import
all three previously looped synchronously on the request context and got 504'd/cancelled mid-run by the reverse proxy.
 Forms switched from POST-redirect to htmx fragments (job_progress.html, job_result.html, crew_import_results.html)
 the old crew_import_results.html page and scan_backfill_progress.html partial were deleted.
This is also captured as a new rule in CLAUDE.md.

4. Cascade-delete manifest on last-tag deletion

DeleteTagHandler now, after removing the last tag pointing to a digest, cascade-deletes the manifest itself
 (PDS + DB + hold blob purge) — but only if it's not a child of a manifest list (multi-arch parent).
 New GetTagDigest and ShouldCascadeDeleteManifest queries back it, plus cascade_delete_test.go.
 Also switches tag rkey computation to the atproto.RepositoryTagToRKey helper.

5. Billing simplification

Drops the OwnerBadge config option (hold-owner supporter badge).
The user-profile template no longer special-cases an "owner" badge value (only "Captain").
Example tiers renamed to the nautical scheme (deckhand/bosun/quartermaster).

6. Build/deploy: go generate always runs via Make

make generate is now a phony target that always runs go generate ./... (regenerating cbor_gen, icon sprites, etc.),
 and build-trixie depends on it. The deploy tooling (provision.go/update.go)
drops its own runGenerate calls since the Makefile handles it.

7. New cmd/firehose-tap tool (untracked)

A standalone CLI that subscribes to a com.atproto.sync.subscribeRepos endpoint and pretty-prints events,
with emphasis on Sync 1.1 compliance fields (per-op prev CIDs, commit prevData) and a --validate CI mode.
Fits with the recent "more sync1.1 compliant" commit.
2026-06-05 20:57:25 -05:00
Evan Jarrett fd3132a545 create a seamark credential helper, have the appview redirect to the source if navigating to go paths 2026-05-25 09:01:46 -05:00
Evan Jarrett baa68d7a3f minor bugfix for credential helper update command 2026-05-17 13:36:22 -05:00
Evan Jarrett 902fba4553 convert alert to modal. go fix the codebase 2026-05-16 11:59:47 -05:00
Evan Jarrett a0cc862798 lots of new work for authenticating between appview -> hold. fixed quota handling, improve integration tests for round-trip push/pull/auth/quota checking 2026-05-11 09:20:55 -05:00
Evan Jarrett 98a2cfea59 improve UI around credential helper authorization. have the hold requestCrawl on restart. Update comments that relay_endpoints must suport listreposbycollection 2026-05-08 20:44:04 -05:00
Evan Jarrett ab66028151 more labeler improvements. standardize did work between labeler and hold. improve sql race conditions on local-only db 2026-05-02 22:13:53 -05:00
Evan Jarrett 410183f13b first pass at implementing a label service 2026-05-02 11:14:44 -05:00
Evan Jarrett 13a793ca90 improve admin tooling 2026-04-29 10:37:36 -05:00
Evan Jarrett 9af6eccc9d improvements to how scanning works, and helmchart ui 2026-04-29 10:12:25 -05:00
Evan Jarrett 20202aa0df update credential helper to pull latest update from tangled directly 2026-04-04 20:59:50 -05:00
Evan Jarrett ca69c0d1c2 ai advisor poc 2026-03-29 22:31:45 -07:00
Evan Jarrett 0827219716 cleanup relay-compare script 2026-02-27 20:14:26 -06:00
Evan Jarrett 136c0a0ecc billing refactor, move billing to appview, move webhooks to appview 2026-02-26 22:28:09 -06:00
Evan Jarrett dc31ca2f35 more work on webhook, implement getMetadata endpoint for appview and link holds to a preferred appview 2026-02-22 22:49:33 -06:00
Evan Jarrett e6c2099a0f add a verify check on the relay-compare 2026-02-22 15:28:34 -06:00
Evan Jarrett 5249c9eaab add relay-compare tool 2026-02-22 12:06:02 -06:00
Evan Jarrett f90a46e0a4 begin implement supporter badges, clean up lexicons, various other changes 2026-02-20 22:12:18 -06:00
Evan Jarrett 27cf78158b vuln scanner fixes, major refactor of the credential helper. 2026-02-17 22:38:25 -06:00
Evan Jarrett f340158a79 tweaks related to did:plc, fix bluesky profile creation, update deploys to build locally then scp 2026-02-14 21:00:07 -06:00
Evan Jarrett e3843db9d8 Implement did:plc support for holds with the ability to import/export CARs.
did:plc Identity Support (pkg/hold/pds/did.go, pkg/hold/config.go, pkg/hold/server.go)

  The big feature — holds can now use did:plc identities instead of only did:web. This adds:
  - LoadOrCreateDID() — resolves hold DID by priority: config DID > did.txt on disk > create new
  - CreatePLCIdentity() — builds a genesis operation, signs with rotation key, submits to PLC directory
  - EnsurePLCCurrent() — on boot, compares local signing key + URL against PLC directory and auto-updates if they've drifted (requires rotation key)
  - New config fields: did_method (web/plc), did, plc_directory_url, rotation_key_path
  - GenerateDIDDocument() now uses the stored DID instead of always deriving did:web from URL
  - NewHoldServer wired up to call LoadOrCreateDID instead of GenerateDIDFromURL

  CAR Export/Import (pkg/hold/pds/export.go, pkg/hold/pds/import.go, cmd/hold/repo.go)

  New CLI subcommands for repo backup/restore:
  - atcr-hold repo export — streams the hold's repo as a CAR file to stdout
  - atcr-hold repo import <file>... — reads CAR files, upserts all records in a single atomic commit. Uses a bulkImportRecords method that opens a delta session, checks each record for
  create vs update, commits once, and fires repo events.
  - openHoldPDS() helper to spin up a HoldPDS from config for offline CLI operations

  Admin UI Fixes (pkg/hold/admin/)

  - Logout changed from GET to POST — nav template now uses a <form method=POST> instead of an <a> link (prevents CSRF on logout)
  - Removed return_to parameter from login flow — simplified redirect logic, auth middleware now redirects to /admin/auth/login without query params

  Config/Deploy

  - config-hold.example.yaml and deploy/upcloud/configs/hold.yaml.tmpl updated with the four new did:plc config fields
  - go.mod / go.sum — added github.com/did-method-plc/go-didplc dependency
2026-02-14 15:17:53 -06:00
Evan Jarrett de02e1f046 remove distribution from hold, add vulnerability scanning in appview.
1. Removing distribution/distribution from the Hold Service (biggest change)
  The hold service previously used distribution's StorageDriver interface for all blob operations. This replaces it with direct AWS SDK v2 calls through ATCR's own pkg/s3.S3Service:
  - New S3Service methods: Stat(), PutBytes(), Move(), Delete(), WalkBlobs(), ListPrefix() added to pkg/s3/types.go
  - Pull zone fix: Presigned URLs are now generated against the real S3 endpoint, then the host is swapped to the CDN URL post-signing (previously the CDN URL was set as the endpoint, which
  broke SigV4 signatures)
  - All hold subsystems migrated: GC, OCI uploads, XRPC handlers, profile uploads, scan broadcaster, manifest posts — all now use *s3.S3Service instead of storagedriver.StorageDriver
  - Config simplified: Removed configuration.Storage type and buildStorageConfigFromFields(); replaced with a simple S3Params() method
  - Mock expanded: MockS3Client gains an in-memory object store + 5 new methods, replacing duplicate mockStorageDriver implementations in tests (~160 lines deleted from each test file)
2. Vulnerability Scan UI in AppView (new feature)
  Displays scan results from the hold's PDS on the repository page:
  - New lexicon: io/atcr/hold/scan.json with vulnReportBlob field for storing full Grype reports
  - Two new HTMX endpoints: /api/scan-result (badge) and /api/vuln-details (modal with CVE table)
  - New templates: vuln-badge.html (severity count chips) and vuln-details.html (full CVE table with NVD/GHSA links)
  - Repository page: Lazy-loads scan badges per manifest via HTMX
  - Tests: ~590 lines of test coverage for both handlers
3. S3 Diagnostic Tool
  New cmd/s3-test/main.go (418 lines) — tests S3 connectivity with both SDK v1 and v2, including presigned URL generation, pull zone host swapping, and verbose signing debug output.
4. Deployment Tooling
  - New syncServiceUnit() for comparing/updating systemd units on servers
  - Update command now syncs config keys (adds missing keys from template) and service units with daemon-reload
5. DB Migration
  0011_fix_captain_successor_column.yaml — rebuilds hold_captain_records to add the successor column that was missed in a previous migration.
6. Documentation
  - APPVIEW-UI-FUTURE.md rewritten as a status-tracked feature inventory
  - DISTRIBUTION.md renamed to CREDENTIAL_HELPER.md
  - New REMOVING_DISTRIBUTION.md — 480-line analysis of fully removing distribution from the appview side
7. go.mod
  aws-sdk-go v1 moved from indirect to direct (needed by cmd/s3-test).
2026-02-13 15:26:24 -06:00
Evan Jarrett 92c31835e2 implement the ability to promote a hold as a successor as a way to migrate users to a new storage server 2026-02-12 20:14:19 -06:00
Evan Jarrett ef0161fb0e update settings page, move admin-panel to tailwind/daisy 2026-02-06 11:23:12 -06:00
Evan Jarrett 6a52175d70 add theme overrides 2026-02-03 20:35:13 -06:00
Evan Jarrett 34f342f637 lots of refactor and cleanup to allow for branding overrides 2026-02-02 22:42:15 -06:00
Evan Jarrett ca56a7c309 allow domain name and short name to be replaced by config 2026-01-22 14:52:30 -06:00
Evan Jarrett 57593a8683 remove the filesystem and buffered upload ability on the holds. going forward the only supported storage is s3. adds extra mocks and tests around uploading 2026-01-19 16:59:03 -06:00
Evan Jarrett d0843323fe more pagespeed fixes 2026-01-17 15:48:40 -06:00
Evan Jarrett b7ed0e7d5b more pagespeed improvements, improve routing handler logic 2026-01-17 10:38:35 -06:00
Evan Jarrett c48a763529 fixup search page to use repocard. remove hardcoded values from privacy/terms/home 2026-01-16 11:19:42 -06:00
Evan Jarrett 950b1f94d0 add mascot with new colors 2026-01-15 21:45:31 -06:00
Evan Jarrett 908e124917 more visual tweaks 2026-01-15 00:17:48 -06:00
Evan Jarrett 4c0f20a32e begin large refactor of UI to use tailwind and daisy 2026-01-14 14:42:04 -06:00
Evan Jarrett 9e600649a6 begin s3 garbage collection implementation, more envvar cleanup 2026-01-08 23:31:56 -06:00
Evan Jarrett 51f6917444 add log shipper begin envvar cleanup 2026-01-08 22:52:32 -06:00
Evan Jarrett 3409af6c67 implement hold discovery dropdown in settings. implement a data privacy export feature 2026-01-07 22:41:14 -06:00
Evan Jarrett d4b88b5105 more lint fixes. enable autofix 2026-01-06 23:56:17 -06:00
Evan Jarrett f5979b8f08 implement a basic crew management admin panel 2026-01-05 21:30:42 -06:00
Evan Jarrett f35bf2bcde fix oauth scope mismatch 2026-01-05 20:26:41 -06:00
Evan Jarrett a448e8257b fix bug creating layer records on non-tagged pushes 2026-01-05 12:08:04 -06:00
Evan Jarrett 487fc8a47e wording 2026-01-04 23:37:31 -06:00
Evan Jarrett e5e59fdcbf add tos and privacy policy 2026-01-04 23:12:14 -06:00
Evan Jarrett efef46b15a various linting fixes 2026-01-04 22:02:01 -06:00
Evan Jarrett a7175f9e3e interface{} -> any 2026-01-04 21:10:29 -06:00
Evan Jarrett aa4b32bbd6 basic implementation of quotas 2026-01-04 20:09:41 -06:00
Evan Jarrett 647c33e164 fix backoff not clearing correctly. add better logging to find out why someone is denied access (backoff, pds issue, missing record etc) 2026-01-02 14:45:55 -06:00
Evan Jarrett 045aeb2de5 add index table to mst so listRecords is more efficient 2026-01-01 21:19:38 -06:00
Evan Jarrett 74c90697a7 disable stats migration 2025-12-31 13:56:39 -06:00
Evan Jarrett 1df1bb57a4 clean up logs, delete cached data when atproto account is deleted 2025-12-31 12:21:17 -06:00