mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-25 19:54:15 +00:00
1. Multiple registry domains + per-user domain preference
The biggest feature. The appview can serve several registry domains (e.g. buoy.cr, atcr.io),
and users can now pick which one shows up in their pull/push commands.
- Lexicon/record: adds registryDomain (and documents ociClient)
to the sailor profile (lexicons/.../profile.json, pkg/atproto/lexicon.go).
- DB: new registry_domain column on users (schema.sql + migration 0027),
with GetUserByDID/Handle reads, UpdateUserRegistryDomain writer,
and Jetstream caching it on profile updates (writes unconditionally so clearing propagates).
- UI/handlers: new UpdateRegistryDomainHandler + /api/profile/registry-domain route,
a <select> in the user settings panel (only shown when >1 domain configured), and resolveRegistryURL()
which falls back to the primary domain if the user's pref is stale/removed. Tests added for all of it.
2. default_hold_did removed → first managed_holds entry is the default
Consolidates two overlapping config fields into one. ServerConfig.DefaultHoldDID is gone;
PrimaryHoldDID() now returns managed_holds[0]. managed_holds is now REQUIRED.
Updated in config, validation, server wiring, test harness, example YAML, and the deploy template.
3. Admin long-running operations → generic background-job framework
New pkg/hold/admin/jobs.go introduces a reusable startJob/jobRegistry pattern
(a detached context.Background() job + a /admin/api/jobs/{key}/status polling endpoint).
This replaces the bespoke scan-backfill goroutine state machine, and now also wraps crew tier remap and crew import
all three previously looped synchronously on the request context and got 504'd/cancelled mid-run by the reverse proxy.
Forms switched from POST-redirect to htmx fragments (job_progress.html, job_result.html, crew_import_results.html)
the old crew_import_results.html page and scan_backfill_progress.html partial were deleted.
This is also captured as a new rule in CLAUDE.md.
4. Cascade-delete manifest on last-tag deletion
DeleteTagHandler now, after removing the last tag pointing to a digest, cascade-deletes the manifest itself
(PDS + DB + hold blob purge) — but only if it's not a child of a manifest list (multi-arch parent).
New GetTagDigest and ShouldCascadeDeleteManifest queries back it, plus cascade_delete_test.go.
Also switches tag rkey computation to the atproto.RepositoryTagToRKey helper.
5. Billing simplification
Drops the OwnerBadge config option (hold-owner supporter badge).
The user-profile template no longer special-cases an "owner" badge value (only "Captain").
Example tiers renamed to the nautical scheme (deckhand/bosun/quartermaster).
6. Build/deploy: go generate always runs via Make
make generate is now a phony target that always runs go generate ./... (regenerating cbor_gen, icon sprites, etc.),
and build-trixie depends on it. The deploy tooling (provision.go/update.go)
drops its own runGenerate calls since the Makefile handles it.
7. New cmd/firehose-tap tool (untracked)
A standalone CLI that subscribes to a com.atproto.sync.subscribeRepos endpoint and pretty-prints events,
with emphasis on Sync 1.1 compliance fields (per-op prev CIDs, commit prevData) and a --validate CI mode.
Fits with the recent "more sync1.1 compliant" commit.
868 lines
36 KiB
Go
868 lines
36 KiB
Go
package atproto
|
|
|
|
//go:generate go run generate.go
|
|
|
|
import (
|
|
"crypto/sha256"
|
|
"encoding/base32"
|
|
"encoding/base64"
|
|
"encoding/json"
|
|
"fmt"
|
|
"strings"
|
|
"time"
|
|
|
|
lexutil "github.com/bluesky-social/indigo/lex/util"
|
|
)
|
|
|
|
// Collection names for ATProto records
|
|
const (
|
|
// ManifestCollection is the collection name for container manifests
|
|
ManifestCollection = "io.atcr.manifest"
|
|
|
|
// TagCollection is the collection name for image tags
|
|
TagCollection = "io.atcr.tag"
|
|
|
|
// HoldCrewCollection is the collection name for hold crew (membership) - LEGACY BYOS model
|
|
// Stored in owner's PDS for BYOS holds
|
|
HoldCrewCollection = "io.atcr.hold.crew"
|
|
|
|
// CaptainCollection is the collection name for captain records (hold ownership) - EMBEDDED PDS model
|
|
// Stored in hold's embedded PDS (singleton record at rkey "self")
|
|
CaptainCollection = "io.atcr.hold.captain"
|
|
|
|
// CrewCollection is the collection name for crew records (access control) - EMBEDDED PDS model
|
|
// Stored in hold's embedded PDS (one record per member)
|
|
// Note: Uses same collection name as HoldCrewCollection but stored in different PDS (hold's PDS vs owner's PDS)
|
|
CrewCollection = "io.atcr.hold.crew"
|
|
|
|
// LayerCollection is the collection name for container layer metadata
|
|
// Stored in hold's embedded PDS to track which layers are stored
|
|
LayerCollection = "io.atcr.hold.layer"
|
|
|
|
// StatsCollection is the collection name for repository statistics
|
|
// Stored in hold's embedded PDS to track pull/push counts per owner+repo
|
|
StatsCollection = "io.atcr.hold.stats"
|
|
|
|
// DailyStatsCollection is the collection name for daily repository statistics
|
|
// Stored in hold's embedded PDS to track daily pull/push counts per owner+repo+date
|
|
DailyStatsCollection = "io.atcr.hold.stats.daily"
|
|
|
|
// ScanCollection is the collection name for vulnerability scan results
|
|
// Stored in hold's embedded PDS to track scan results per manifest
|
|
ScanCollection = "io.atcr.hold.scan"
|
|
|
|
// ImageConfigCollection is the collection name for OCI image configs
|
|
// Stored in hold's embedded PDS, one per manifest (keyed by manifest digest hex)
|
|
ImageConfigCollection = "io.atcr.hold.image.config"
|
|
|
|
// TangledProfileCollection is the collection name for tangled profiles
|
|
// Stored in hold's embedded PDS (singleton record at rkey "self")
|
|
TangledProfileCollection = "sh.tangled.actor.profile"
|
|
|
|
// BskyPostCollection is the collection name for Bluesky posts
|
|
BskyPostCollection = "app.bsky.feed.post"
|
|
|
|
// SailorProfileCollection is the collection name for user profiles
|
|
SailorProfileCollection = "io.atcr.sailor.profile"
|
|
|
|
// StarCollection is the collection name for repository stars
|
|
StarCollection = "io.atcr.sailor.star"
|
|
|
|
// RepoPageCollection is the collection name for repository page metadata
|
|
// Stored in user's PDS with rkey = repository name
|
|
RepoPageCollection = "io.atcr.repo.page"
|
|
)
|
|
|
|
// ManifestRecord represents a container image manifest stored in ATProto
|
|
// This follows the OCI image manifest specification but stored as an ATProto record
|
|
type ManifestRecord struct {
|
|
// Type should be "io.atcr.manifest"
|
|
Type string `json:"$type"`
|
|
|
|
// Repository is the name of the repository (e.g., "myapp")
|
|
Repository string `json:"repository"`
|
|
|
|
// Digest is the content digest (e.g., "sha256:abc123...")
|
|
Digest string `json:"digest"`
|
|
|
|
// HoldDID is the DID of the hold service where blobs are stored
|
|
// This is the primary reference for hold resolution
|
|
// e.g., "did:web:hold01.atcr.io"
|
|
HoldDID string `json:"holdDid,omitempty"`
|
|
|
|
// HoldEndpoint is the hold service endpoint URL where blobs are stored (DEPRECATED)
|
|
// Kept for backward compatibility with manifests created before DID migration
|
|
// New manifests should use HoldDID instead
|
|
// This is a historical reference that doesn't change even if user's default hold changes
|
|
HoldEndpoint string `json:"holdEndpoint,omitempty"`
|
|
|
|
// MediaType is the OCI media type (e.g., "application/vnd.oci.image.manifest.v1+json")
|
|
MediaType string `json:"mediaType"`
|
|
|
|
// SchemaVersion is the OCI schema version (typically 2)
|
|
SchemaVersion int `json:"schemaVersion"`
|
|
|
|
// Config references the image configuration blob (for image manifests)
|
|
// Nil for manifest lists/indexes
|
|
Config *BlobReference `json:"config,omitempty"`
|
|
|
|
// Layers references the filesystem layers (for image manifests)
|
|
// Empty for manifest lists/indexes
|
|
Layers []BlobReference `json:"layers,omitempty"`
|
|
|
|
// Manifests references other manifests (for manifest lists/indexes)
|
|
// Empty for image manifests
|
|
Manifests []ManifestReference `json:"manifests,omitempty"`
|
|
|
|
// Annotations contains arbitrary metadata
|
|
Annotations map[string]string `json:"annotations,omitempty"`
|
|
|
|
// Subject references another manifest (for attestations, signatures, etc.)
|
|
Subject *BlobReference `json:"subject,omitempty"`
|
|
|
|
// ManifestBlob is a reference to the manifest blob stored in ATProto blob storage
|
|
ManifestBlob *ATProtoBlobRef `json:"manifestBlob,omitempty"`
|
|
|
|
// CreatedAt timestamp
|
|
CreatedAt time.Time `json:"createdAt"`
|
|
}
|
|
|
|
// BlobReference represents a reference to a blob (layer or config)
|
|
// Blobs are stored in S3 and referenced by digest
|
|
type BlobReference struct {
|
|
// MediaType of the blob
|
|
MediaType string `json:"mediaType"`
|
|
|
|
// Digest is the content digest (e.g., "sha256:abc123...")
|
|
Digest string `json:"digest"`
|
|
|
|
// Size in bytes
|
|
Size int64 `json:"size"`
|
|
|
|
// URLs where the blob can be retrieved (S3 URLs)
|
|
URLs []string `json:"urls,omitempty"`
|
|
|
|
// Annotations for the blob
|
|
Annotations map[string]string `json:"annotations,omitempty"`
|
|
}
|
|
|
|
// ManifestReference represents a reference to a manifest in a manifest list/index
|
|
type ManifestReference struct {
|
|
// MediaType of the referenced manifest
|
|
MediaType string `json:"mediaType"`
|
|
|
|
// Digest is the content digest (e.g., "sha256:abc123...")
|
|
Digest string `json:"digest"`
|
|
|
|
// Size in bytes
|
|
Size int64 `json:"size"`
|
|
|
|
// Platform describes the platform/architecture this manifest is for
|
|
Platform *Platform `json:"platform,omitempty"`
|
|
|
|
// Annotations for the manifest reference
|
|
Annotations map[string]string `json:"annotations,omitempty"`
|
|
}
|
|
|
|
// Platform describes the platform (OS/architecture) for a manifest
|
|
type Platform struct {
|
|
// Architecture is the CPU architecture (e.g., "amd64", "arm64", "arm")
|
|
Architecture string `json:"architecture"`
|
|
|
|
// OS is the operating system (e.g., "linux", "windows", "darwin")
|
|
OS string `json:"os"`
|
|
|
|
// OSVersion is the optional OS version
|
|
OSVersion string `json:"os.version,omitempty"`
|
|
|
|
// OSFeatures is an optional list of OS features
|
|
OSFeatures []string `json:"os.features,omitempty"`
|
|
|
|
// Variant is the optional CPU variant (e.g., "v7" for ARM)
|
|
Variant string `json:"variant,omitempty"`
|
|
}
|
|
|
|
// NewManifestRecord creates a new manifest record from OCI manifest JSON
|
|
func NewManifestRecord(repository, digest string, ociManifest []byte) (*ManifestRecord, error) {
|
|
// Parse the OCI manifest
|
|
var ociData struct {
|
|
SchemaVersion int `json:"schemaVersion"`
|
|
MediaType string `json:"mediaType"`
|
|
Config json.RawMessage `json:"config,omitempty"`
|
|
Layers []json.RawMessage `json:"layers,omitempty"`
|
|
Manifests []json.RawMessage `json:"manifests,omitempty"`
|
|
Subject json.RawMessage `json:"subject,omitempty"`
|
|
Annotations map[string]string `json:"annotations,omitempty"`
|
|
}
|
|
|
|
if err := json.Unmarshal(ociManifest, &ociData); err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// Detect manifest type based on media type
|
|
isManifestList := strings.Contains(ociData.MediaType, "manifest.list") ||
|
|
strings.Contains(ociData.MediaType, "image.index")
|
|
|
|
// Validate: must have either (config+layers) OR (manifests), never both
|
|
hasImageFields := len(ociData.Config) > 0 || len(ociData.Layers) > 0
|
|
hasIndexFields := len(ociData.Manifests) > 0
|
|
|
|
if hasImageFields && hasIndexFields {
|
|
return nil, fmt.Errorf("manifest cannot have both image fields (config/layers) and index fields (manifests)")
|
|
}
|
|
if !hasImageFields && !hasIndexFields {
|
|
return nil, fmt.Errorf("manifest must have either image fields (config/layers) or index fields (manifests)")
|
|
}
|
|
|
|
record := &ManifestRecord{
|
|
Type: ManifestCollection,
|
|
Repository: repository,
|
|
Digest: digest,
|
|
MediaType: ociData.MediaType,
|
|
SchemaVersion: ociData.SchemaVersion,
|
|
Annotations: ociData.Annotations,
|
|
// ManifestBlob will be set by the caller after uploading to blob storage
|
|
CreatedAt: time.Now(),
|
|
}
|
|
|
|
if isManifestList {
|
|
// Parse manifest list/index
|
|
record.Manifests = make([]ManifestReference, len(ociData.Manifests))
|
|
for i, m := range ociData.Manifests {
|
|
if err := json.Unmarshal(m, &record.Manifests[i]); err != nil {
|
|
return nil, fmt.Errorf("failed to parse manifest reference %d: %w", i, err)
|
|
}
|
|
}
|
|
} else {
|
|
// Parse image manifest
|
|
if len(ociData.Config) > 0 {
|
|
var config BlobReference
|
|
if err := json.Unmarshal(ociData.Config, &config); err != nil {
|
|
return nil, fmt.Errorf("failed to parse config: %w", err)
|
|
}
|
|
record.Config = &config
|
|
}
|
|
|
|
// Parse layers
|
|
record.Layers = make([]BlobReference, len(ociData.Layers))
|
|
for i, layer := range ociData.Layers {
|
|
if err := json.Unmarshal(layer, &record.Layers[i]); err != nil {
|
|
return nil, fmt.Errorf("failed to parse layer %d: %w", i, err)
|
|
}
|
|
}
|
|
}
|
|
|
|
// Parse subject if present (works for both types)
|
|
if len(ociData.Subject) > 0 {
|
|
var subject BlobReference
|
|
if err := json.Unmarshal(ociData.Subject, &subject); err != nil {
|
|
return nil, err
|
|
}
|
|
record.Subject = &subject
|
|
}
|
|
|
|
return record, nil
|
|
}
|
|
|
|
// TagRecord represents a tag pointing to a manifest
|
|
type TagRecord struct {
|
|
// Type should be "io.atcr.tag"
|
|
Type string `json:"$type"`
|
|
|
|
// Repository is the name of the repository
|
|
Repository string `json:"repository"`
|
|
|
|
// Tag is the tag name (e.g., "latest", "v1.0.0")
|
|
Tag string `json:"tag"`
|
|
|
|
// Manifest is the AT-URI of the manifest this tag points to
|
|
// Format: at://did:plc:xyz/io.atcr.manifest/abc123
|
|
// Preferred over ManifestDigest for new records
|
|
Manifest string `json:"manifest,omitempty"`
|
|
|
|
// MediaType is the OCI media type of the manifest this tag points to
|
|
// e.g., "application/vnd.oci.image.manifest.v1+json" or "application/vnd.oci.image.index.v1+json"
|
|
MediaType string `json:"mediaType,omitempty"`
|
|
|
|
// ManifestDigest is the digest of the manifest this tag points to (DEPRECATED)
|
|
// Kept for backward compatibility with old records
|
|
// New records should use Manifest field instead
|
|
ManifestDigest string `json:"manifestDigest,omitempty"`
|
|
|
|
// UpdatedAt timestamp
|
|
UpdatedAt time.Time `json:"updatedAt"`
|
|
}
|
|
|
|
// NewTagRecord creates a new tag record with manifest AT-URI
|
|
// did: The DID of the user (e.g., "did:plc:xyz123")
|
|
// repository: The repository name (e.g., "myapp")
|
|
// tag: The tag name (e.g., "latest", "v1.0.0")
|
|
// manifestDigest: The manifest digest (e.g., "sha256:abc123...")
|
|
func NewTagRecord(did, repository, tag, manifestDigest, mediaType string) *TagRecord {
|
|
// Build AT-URI for the manifest
|
|
// Format: at://did:plc:xyz/io.atcr.manifest/<digest-without-sha256-prefix>
|
|
manifestURI := BuildManifestURI(did, manifestDigest)
|
|
|
|
return &TagRecord{
|
|
Type: TagCollection,
|
|
Repository: repository,
|
|
Tag: tag,
|
|
Manifest: manifestURI,
|
|
MediaType: mediaType,
|
|
// Note: ManifestDigest is not set for new records (only for backward compat with old records)
|
|
UpdatedAt: time.Now(),
|
|
}
|
|
}
|
|
|
|
// HoldRecord represents a storage hold definition (BYOS)
|
|
// Users create these records to define where their blobs should be stored
|
|
type HoldRecord struct {
|
|
// Type should be "io.atcr.hold"
|
|
Type string `json:"$type"`
|
|
|
|
// Endpoint is the URL of the hold service
|
|
// e.g., "https://hold1.example.com"
|
|
Endpoint string `json:"endpoint"`
|
|
|
|
// Owner is the DID of the hold owner
|
|
Owner string `json:"owner"`
|
|
|
|
// Public controls whether this hold allows public blob reads (pulls) without auth
|
|
// Writes always require crew membership
|
|
Public bool `json:"public"`
|
|
|
|
// CreatedAt timestamp
|
|
CreatedAt time.Time `json:"createdAt"`
|
|
}
|
|
|
|
// SailorProfileRecord represents a user's profile with registry preferences
|
|
// Stored in the user's PDS to configure default hold and other settings
|
|
type SailorProfileRecord struct {
|
|
// Type should be "io.atcr.sailor.profile"
|
|
Type string `json:"$type"`
|
|
|
|
// DefaultHold is the default hold DID for blob storage
|
|
// Can be a DID (e.g., "did:web:hold01.atcr.io") or legacy URL
|
|
// URLs are migrated to DIDs on user login
|
|
// If null/empty, user has opted out of defaults
|
|
DefaultHold string `json:"defaultHold,omitempty"`
|
|
|
|
// AutoRemoveUntagged controls whether untagged manifests are automatically
|
|
// cleaned up. When true, manifests that lose all tags (e.g., after a tag
|
|
// overwrite) are deleted from PDS, and their layers are cleaned up by hold GC.
|
|
AutoRemoveUntagged bool `json:"autoRemoveUntagged,omitempty"`
|
|
|
|
// OciClient is the preferred client for pull commands (docker, podman, buildah, nerdctl, crane).
|
|
// "none" means image reference only (no `<client> pull ` prefix). Defaults to "docker" if empty.
|
|
OciClient string `json:"ociClient,omitempty"`
|
|
|
|
// RegistryDomain is the user's preferred registry domain for UI display.
|
|
// Must be one of the appview's configured registry_domains. Empty = primary (first configured).
|
|
RegistryDomain string `json:"registryDomain,omitempty"`
|
|
|
|
// AIAdvisorEnabled controls whether the AI Image Advisor feature is active for this user.
|
|
// nil = default (enabled if user has billing access), false = explicitly disabled.
|
|
AIAdvisorEnabled *bool `json:"aiAdvisorEnabled,omitempty"`
|
|
|
|
// CreatedAt timestamp
|
|
CreatedAt time.Time `json:"createdAt"`
|
|
|
|
// UpdatedAt timestamp
|
|
UpdatedAt time.Time `json:"updatedAt"`
|
|
}
|
|
|
|
// NewSailorProfileRecord creates a new sailor profile record
|
|
func NewSailorProfileRecord(defaultHold string) *SailorProfileRecord {
|
|
now := time.Now()
|
|
return &SailorProfileRecord{
|
|
Type: SailorProfileCollection,
|
|
DefaultHold: defaultHold,
|
|
CreatedAt: now,
|
|
UpdatedAt: now,
|
|
}
|
|
}
|
|
|
|
// RepoPageRecord represents repository page metadata (description + avatar)
|
|
// Stored in the user's PDS with rkey = repository name
|
|
// Users can edit this directly in their PDS to customize their repository page
|
|
type RepoPageRecord struct {
|
|
// Type should be "io.atcr.repo.page"
|
|
Type string `json:"$type"`
|
|
|
|
// Repository is the name of the repository (e.g., "myapp")
|
|
Repository string `json:"repository"`
|
|
|
|
// Description is the markdown README/description content
|
|
Description string `json:"description,omitempty"`
|
|
|
|
// Avatar is the repository avatar/icon blob reference
|
|
Avatar *ATProtoBlobRef `json:"avatar,omitempty"`
|
|
|
|
// UserEdited indicates the description was manually edited by the user
|
|
// When true, auto-population from manifest annotations is skipped on push
|
|
UserEdited bool `json:"userEdited,omitempty"`
|
|
|
|
// CreatedAt timestamp
|
|
CreatedAt time.Time `json:"createdAt"`
|
|
|
|
// UpdatedAt timestamp
|
|
UpdatedAt time.Time `json:"updatedAt"`
|
|
}
|
|
|
|
// NewRepoPageRecord creates a new repo page record
|
|
func NewRepoPageRecord(repository, description string, avatar *ATProtoBlobRef) *RepoPageRecord {
|
|
now := time.Now()
|
|
return &RepoPageRecord{
|
|
Type: RepoPageCollection,
|
|
Repository: repository,
|
|
Description: description,
|
|
Avatar: avatar,
|
|
CreatedAt: now,
|
|
UpdatedAt: now,
|
|
}
|
|
}
|
|
|
|
// StarRecord represents a user starring a repository
|
|
// Stored in the starrer's PDS (like Bluesky likes)
|
|
// Subject is an AT URI pointing to the repo page record being starred
|
|
type StarRecord struct {
|
|
// Type should be "io.atcr.sailor.star"
|
|
Type string `json:"$type"`
|
|
|
|
// Subject is the AT URI of the repo page being starred
|
|
// e.g., "at://did:plc:abc/io.atcr.repo.page/myapp"
|
|
Subject string `json:"subject"`
|
|
|
|
// CreatedAt timestamp
|
|
CreatedAt time.Time `json:"createdAt"`
|
|
}
|
|
|
|
// GetSubjectDIDAndRepository extracts the owner DID and repository name
|
|
// from the star record's subject AT URI.
|
|
func (s *StarRecord) GetSubjectDIDAndRepository() (ownerDID, repository string, err error) {
|
|
return ParseRepoPageURI(s.Subject)
|
|
}
|
|
|
|
// NewStarRecord creates a new star record with an AT URI subject
|
|
func NewStarRecord(ownerDID, repository string) *StarRecord {
|
|
return &StarRecord{
|
|
Type: StarCollection,
|
|
Subject: BuildRepoPageURI(ownerDID, repository),
|
|
CreatedAt: time.Now(),
|
|
}
|
|
}
|
|
|
|
// BuildRepoPageURI creates an AT URI for a repo page record
|
|
// e.g., BuildRepoPageURI("did:plc:abc", "myapp") → "at://did:plc:abc/io.atcr.repo.page/myapp"
|
|
func BuildRepoPageURI(ownerDID, repository string) string {
|
|
return fmt.Sprintf("at://%s/%s/%s", ownerDID, RepoPageCollection, repository)
|
|
}
|
|
|
|
// ParseRepoPageURI extracts the owner DID and repository from a repo page AT URI
|
|
func ParseRepoPageURI(uri string) (ownerDID, repository string, err error) {
|
|
if !strings.HasPrefix(uri, "at://") {
|
|
return "", "", fmt.Errorf("invalid AT URI: must start with 'at://'")
|
|
}
|
|
remainder := strings.TrimPrefix(uri, "at://")
|
|
parts := strings.SplitN(remainder, "/", 3)
|
|
if len(parts) != 3 {
|
|
return "", "", fmt.Errorf("invalid AT URI: expected 3 parts (did/collection/rkey), got %d", len(parts))
|
|
}
|
|
if parts[1] != RepoPageCollection {
|
|
return "", "", fmt.Errorf("invalid AT URI: expected collection %s, got %s", RepoPageCollection, parts[1])
|
|
}
|
|
return parts[0], parts[2], nil
|
|
}
|
|
|
|
// StarRecordKey generates a record key for a star
|
|
// Uses a simple hash to ensure uniqueness and prevent duplicate stars
|
|
func StarRecordKey(ownerDID, repository string) string {
|
|
// Use base64 encoding of "ownerDID/repository" as the record key
|
|
// This is deterministic and prevents duplicate stars
|
|
combined := ownerDID + "/" + repository
|
|
return base64.RawURLEncoding.EncodeToString([]byte(combined))
|
|
}
|
|
|
|
// ParseStarRecordKey decodes a star record key back to ownerDID and repository
|
|
func ParseStarRecordKey(rkey string) (ownerDID, repository string, err error) {
|
|
decoded, err := base64.RawURLEncoding.DecodeString(rkey)
|
|
if err != nil {
|
|
return "", "", fmt.Errorf("failed to decode star rkey: %w", err)
|
|
}
|
|
|
|
parts := strings.SplitN(string(decoded), "/", 2)
|
|
if len(parts) != 2 {
|
|
return "", "", fmt.Errorf("invalid star rkey format: %s", string(decoded))
|
|
}
|
|
|
|
return parts[0], parts[1], nil
|
|
}
|
|
|
|
// RepositoryTagToRKey converts a repository and tag to an ATProto record key
|
|
// ATProto record keys must match: ^[a-zA-Z0-9._~-]{1,512}$
|
|
func RepositoryTagToRKey(repository, tag string) string {
|
|
// Combine repository and tag to create a unique key
|
|
// Replace invalid characters: slashes become tildes (~)
|
|
// We use tilde instead of dash to avoid ambiguity with repository names that contain hyphens
|
|
key := fmt.Sprintf("%s_%s", repository, tag)
|
|
|
|
// Replace / with ~ (slash not allowed in rkeys, tilde is allowed and unlikely in repo names)
|
|
key = strings.ReplaceAll(key, "/", "~")
|
|
|
|
return key
|
|
}
|
|
|
|
// RKeyToRepositoryTag converts an ATProto record key back to repository and tag
|
|
// This is the inverse of RepositoryTagToRKey
|
|
// Note: If the tag contains underscores, this will split on the LAST underscore
|
|
func RKeyToRepositoryTag(rkey string) (repository, tag string) {
|
|
// Find the last underscore to split repository and tag
|
|
lastUnderscore := strings.LastIndex(rkey, "_")
|
|
if lastUnderscore == -1 {
|
|
// No underscore found - treat entire string as tag with empty repository
|
|
return "", rkey
|
|
}
|
|
|
|
repository = rkey[:lastUnderscore]
|
|
tag = rkey[lastUnderscore+1:]
|
|
|
|
// Convert tildes back to slashes in repository (tilde was used to encode slashes)
|
|
repository = strings.ReplaceAll(repository, "~", "/")
|
|
|
|
return repository, tag
|
|
}
|
|
|
|
// BuildManifestURI creates an AT-URI for a manifest record
|
|
// did: The DID of the user (e.g., "did:plc:xyz123")
|
|
// manifestDigest: The manifest digest (e.g., "sha256:abc123...")
|
|
// Returns: AT-URI in format "at://did:plc:xyz/io.atcr.manifest/<digest-without-sha256-prefix>"
|
|
func BuildManifestURI(did, manifestDigest string) string {
|
|
// Remove the "sha256:" prefix from the digest to get the rkey
|
|
rkey := strings.TrimPrefix(manifestDigest, "sha256:")
|
|
return fmt.Sprintf("at://%s/%s/%s", did, ManifestCollection, rkey)
|
|
}
|
|
|
|
// ParseManifestURI extracts the digest from a manifest AT-URI
|
|
// manifestURI: AT-URI in format "at://did:plc:xyz/io.atcr.manifest/<digest-without-sha256-prefix>"
|
|
// Returns: Full digest with "sha256:" prefix (e.g., "sha256:abc123...")
|
|
func ParseManifestURI(manifestURI string) (string, error) {
|
|
// Expected format: at://did:plc:xyz/io.atcr.manifest/<rkey>
|
|
if !strings.HasPrefix(manifestURI, "at://") {
|
|
return "", fmt.Errorf("invalid AT-URI format: must start with 'at://'")
|
|
}
|
|
|
|
// Remove "at://" prefix
|
|
remainder := strings.TrimPrefix(manifestURI, "at://")
|
|
|
|
// Split by "/"
|
|
parts := strings.Split(remainder, "/")
|
|
if len(parts) != 3 {
|
|
return "", fmt.Errorf("invalid AT-URI format: expected 3 parts (did/collection/rkey), got %d", len(parts))
|
|
}
|
|
|
|
// Validate collection
|
|
if parts[1] != ManifestCollection {
|
|
return "", fmt.Errorf("invalid AT-URI: expected collection %s, got %s", ManifestCollection, parts[1])
|
|
}
|
|
|
|
// The rkey is the digest without the "sha256:" prefix
|
|
// Add it back to get the full digest
|
|
rkey := parts[2]
|
|
return "sha256:" + rkey, nil
|
|
}
|
|
|
|
// GetManifestDigest extracts the digest from a TagRecord, preferring the manifest field
|
|
// Returns the digest with "sha256:" prefix (e.g., "sha256:abc123...")
|
|
func (t *TagRecord) GetManifestDigest() (string, error) {
|
|
// Prefer the new manifest field
|
|
if t.Manifest != "" {
|
|
return ParseManifestURI(t.Manifest)
|
|
}
|
|
|
|
// Fall back to the legacy manifestDigest field
|
|
if t.ManifestDigest != "" {
|
|
return t.ManifestDigest, nil
|
|
}
|
|
|
|
return "", fmt.Errorf("tag record has neither manifest nor manifestDigest field")
|
|
}
|
|
|
|
// =============================================================================
|
|
// Embedded PDS Types (Hold Service)
|
|
// =============================================================================
|
|
|
|
// CaptainRecord represents the hold's ownership and metadata
|
|
// Collection: io.atcr.hold.captain (singleton record at rkey "self")
|
|
// Stored in the hold's embedded PDS to identify the hold owner and settings
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
type CaptainRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
Owner string `json:"owner" cborgen:"owner"` // DID of hold owner
|
|
Public bool `json:"public" cborgen:"public"` // Public read access
|
|
AllowAllCrew bool `json:"allowAllCrew" cborgen:"allowAllCrew"` // Allow any authenticated user to register as crew
|
|
EnableBlueskyPosts bool `json:"enableBlueskyPosts" cborgen:"enableBlueskyPosts"` // Enable Bluesky posts when manifests are pushed (overrides env var)
|
|
DeployedAt string `json:"deployedAt" cborgen:"deployedAt"` // RFC3339 timestamp
|
|
Region string `json:"region,omitempty" cborgen:"region,omitempty"` // Deployment region (optional)
|
|
Successor string `json:"successor,omitempty" cborgen:"successor,omitempty"` // DID of successor hold (migration redirect)
|
|
}
|
|
|
|
// CrewRecord represents a crew member in the hold
|
|
// Collection: io.atcr.hold.crew (one record per member)
|
|
// Stored in the hold's embedded PDS for access control
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
// Note: Same collection name as HoldCrewRecord but stored in hold's PDS (not owner's PDS)
|
|
type CrewRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
Member string `json:"member" cborgen:"member"`
|
|
Role string `json:"role" cborgen:"role"`
|
|
Permissions []string `json:"permissions" cborgen:"permissions"`
|
|
Tier string `json:"tier,omitempty" cborgen:"tier,omitempty"` // Optional tier for quota limits (e.g., 'deckhand', 'bosun', 'quartermaster')
|
|
Plankowner bool `json:"plankowner,omitempty" cborgen:"plankowner"` // Early adopter flag - gets plankowner_crew_tier for free
|
|
AddedAt string `json:"addedAt" cborgen:"addedAt"` // RFC3339 timestamp
|
|
}
|
|
|
|
// LayerRecord represents metadata about a container layer stored in the hold
|
|
// Collection: io.atcr.hold.layer
|
|
// Stored in the hold's embedded PDS for tracking and analytics
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
type LayerRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
Digest string `json:"digest" cborgen:"digest"` // Layer digest (e.g., "sha256:abc123...")
|
|
Size int64 `json:"size" cborgen:"size"` // Size in bytes
|
|
MediaType string `json:"mediaType" cborgen:"mediaType"` // Media type (e.g., "application/vnd.oci.image.layer.v1.tar+gzip")
|
|
Manifest string `json:"manifest" cborgen:"manifest"` // AT-URI of manifest that included this layer
|
|
UserDID string `json:"userDid" cborgen:"userDid"` // DID of user who uploaded this layer
|
|
CreatedAt string `json:"createdAt" cborgen:"createdAt"` // RFC3339 timestamp
|
|
}
|
|
|
|
// NewLayerRecord creates a new layer record
|
|
// manifestURI: AT-URI of the manifest (e.g., "at://did:plc:xyz/io.atcr.manifest/abc123")
|
|
func NewLayerRecord(digest string, size int64, mediaType, userDID, manifestURI string) *LayerRecord {
|
|
return &LayerRecord{
|
|
Type: LayerCollection,
|
|
Digest: digest,
|
|
Size: size,
|
|
MediaType: mediaType,
|
|
Manifest: manifestURI,
|
|
UserDID: userDID,
|
|
CreatedAt: time.Now().Format(time.RFC3339),
|
|
}
|
|
}
|
|
|
|
// StatsRecord represents repository statistics stored in the hold's PDS
|
|
// Collection: io.atcr.hold.stats
|
|
// Stored in the hold's embedded PDS for tracking manifest pull/push counts
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
// RKey is deterministic: base32(sha256(ownerDID + "/" + repository)[:16])
|
|
type StatsRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
OwnerDID string `json:"ownerDid" cborgen:"ownerDid"` // DID of the image owner (e.g., "did:plc:xyz123")
|
|
Repository string `json:"repository" cborgen:"repository"` // Repository name (e.g., "myapp")
|
|
PullCount int64 `json:"pullCount" cborgen:"pullCount"` // Number of manifest downloads
|
|
LastPull string `json:"lastPull,omitempty" cborgen:"lastPull,omitempty"`
|
|
PushCount int64 `json:"pushCount" cborgen:"pushCount"` // Number of manifest uploads
|
|
LastPush string `json:"lastPush,omitempty" cborgen:"lastPush,omitempty"`
|
|
UpdatedAt string `json:"updatedAt" cborgen:"updatedAt"` // RFC3339 timestamp
|
|
}
|
|
|
|
// NewStatsRecord creates a new stats record
|
|
func NewStatsRecord(ownerDID, repository string) *StatsRecord {
|
|
return &StatsRecord{
|
|
Type: StatsCollection,
|
|
OwnerDID: ownerDID,
|
|
Repository: repository,
|
|
PullCount: 0,
|
|
PushCount: 0,
|
|
UpdatedAt: time.Now().Format(time.RFC3339),
|
|
}
|
|
}
|
|
|
|
// StatsRecordKey generates a deterministic record key for stats
|
|
// Uses base32 encoding of first 16 bytes of SHA-256 hash of "ownerDID/repository"
|
|
// This ensures same owner+repo always maps to same rkey
|
|
func StatsRecordKey(ownerDID, repository string) string {
|
|
combined := ownerDID + "/" + repository
|
|
hash := sha256.Sum256([]byte(combined))
|
|
// Use first 16 bytes (128 bits) for collision resistance
|
|
// Encode with base32 (alphanumeric, lowercase, no padding) for ATProto rkey compatibility
|
|
return strings.ToLower(base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(hash[:16]))
|
|
}
|
|
|
|
// DailyStatsRecord represents daily repository statistics stored in the hold's PDS
|
|
// Collection: io.atcr.hold.stats.daily
|
|
// Stored in the hold's embedded PDS for tracking daily pull/push counts
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
// RKey is deterministic: base32(sha256(ownerDID + "/" + repository + "/" + date)[:16])
|
|
type DailyStatsRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
OwnerDID string `json:"ownerDid" cborgen:"ownerDid"` // DID of the image owner
|
|
Repository string `json:"repository" cborgen:"repository"` // Repository name
|
|
Date string `json:"date" cborgen:"date"` // YYYY-MM-DD format
|
|
PullCount int64 `json:"pullCount" cborgen:"pullCount"` // Number of manifest downloads on this date
|
|
PushCount int64 `json:"pushCount" cborgen:"pushCount"` // Number of manifest uploads on this date
|
|
UpdatedAt string `json:"updatedAt" cborgen:"updatedAt"` // RFC3339 timestamp
|
|
}
|
|
|
|
// NewDailyStatsRecord creates a new daily stats record
|
|
func NewDailyStatsRecord(ownerDID, repository, date string) *DailyStatsRecord {
|
|
return &DailyStatsRecord{
|
|
Type: DailyStatsCollection,
|
|
OwnerDID: ownerDID,
|
|
Repository: repository,
|
|
Date: date,
|
|
PullCount: 0,
|
|
PushCount: 0,
|
|
UpdatedAt: time.Now().Format(time.RFC3339),
|
|
}
|
|
}
|
|
|
|
// DailyStatsRecordKey generates a deterministic record key for daily stats
|
|
// Uses base32 encoding of first 16 bytes of SHA-256 hash of "ownerDID/repository/date"
|
|
func DailyStatsRecordKey(ownerDID, repository, date string) string {
|
|
combined := ownerDID + "/" + repository + "/" + date
|
|
hash := sha256.Sum256([]byte(combined))
|
|
return strings.ToLower(base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(hash[:16]))
|
|
}
|
|
|
|
// CrewRecordKey generates a deterministic rkey from member DID
|
|
// Uses same pattern as StatsRecordKey for consistency
|
|
// This enables O(1) crew membership lookups via getRecord instead of O(n) pagination
|
|
func CrewRecordKey(memberDID string) string {
|
|
hash := sha256.Sum256([]byte(memberDID))
|
|
// Use first 16 bytes (128 bits) for collision resistance
|
|
// Encode with base32 (alphanumeric, lowercase, no padding) for ATProto rkey compatibility
|
|
return strings.ToLower(base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(hash[:16]))
|
|
}
|
|
|
|
// ScanRecord represents vulnerability scan results for a manifest
|
|
// Collection: io.atcr.hold.scan
|
|
// Stored in hold's embedded PDS to track scan results per manifest
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
// RKey is deterministic: based on manifest digest (one scan per manifest)
|
|
type ScanRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
Manifest string `json:"manifest" cborgen:"manifest"` // AT-URI of the scanned manifest (e.g., "at://did:plc:xyz/io.atcr.manifest/abc123...")
|
|
Repository string `json:"repository" cborgen:"repository"` // Repository name (e.g., "myapp")
|
|
UserDID string `json:"userDid" cborgen:"userDid"` // DID of the image owner
|
|
SbomBlob *lexutil.LexBlob `json:"sbomBlob,omitempty" cborgen:"sbomBlob"` // SBOM blob uploaded to hold's PDS blob storage
|
|
VulnReportBlob *lexutil.LexBlob `json:"vulnReportBlob,omitempty" cborgen:"vulnReportBlob"` // Grype vulnerability report blob (full CVE details)
|
|
Critical int64 `json:"critical" cborgen:"critical"` // Count of critical vulnerabilities
|
|
High int64 `json:"high" cborgen:"high"` // Count of high vulnerabilities
|
|
Medium int64 `json:"medium" cborgen:"medium"` // Count of medium vulnerabilities
|
|
Low int64 `json:"low" cborgen:"low"` // Count of low vulnerabilities
|
|
Total int64 `json:"total" cborgen:"total"` // Total vulnerability count
|
|
ScannerVersion string `json:"scannerVersion" cborgen:"scannerVersion"` // Scanner version (e.g., "atcr-scanner-v1.0.0")
|
|
ScannedAt string `json:"scannedAt" cborgen:"scannedAt"` // RFC3339 timestamp of scan completion
|
|
Status string `json:"status,omitempty" cborgen:"status,omitempty"` // Scan outcome: "ok" (default if empty), "failed", or "skipped"
|
|
Reason string `json:"reason,omitempty" cborgen:"reason,omitempty"` // Optional reason for non-ok status (e.g., unscannable artifact type)
|
|
}
|
|
|
|
// Scan record status values. An empty Status field is treated as ScanStatusOK
|
|
// for back-compat with records written before the field was introduced.
|
|
const (
|
|
ScanStatusOK = "ok"
|
|
ScanStatusFailed = "failed"
|
|
ScanStatusSkipped = "skipped"
|
|
)
|
|
|
|
// NewScanRecord creates a new scan record
|
|
// manifestDigest: the manifest digest (e.g., "sha256:abc123...")
|
|
// userDID: the DID of the image owner (used to build the manifest AT-URI)
|
|
// sbomBlob: blob reference from uploading SBOM to PDS blob storage (nil if no SBOM)
|
|
// vulnReportBlob: blob reference from uploading Grype vulnerability report (nil if no report)
|
|
func NewScanRecord(manifestDigest, repository, userDID string, sbomBlob, vulnReportBlob *lexutil.LexBlob, critical, high, medium, low, total int, scannerVersion string) *ScanRecord {
|
|
return &ScanRecord{
|
|
Type: ScanCollection,
|
|
Manifest: BuildManifestURI(userDID, manifestDigest),
|
|
Repository: repository,
|
|
UserDID: userDID,
|
|
SbomBlob: sbomBlob,
|
|
VulnReportBlob: vulnReportBlob,
|
|
Critical: int64(critical),
|
|
High: int64(high),
|
|
Medium: int64(medium),
|
|
Low: int64(low),
|
|
Total: int64(total),
|
|
ScannerVersion: scannerVersion,
|
|
ScannedAt: time.Now().Format(time.RFC3339),
|
|
Status: ScanStatusOK,
|
|
}
|
|
}
|
|
|
|
// NewSkippedScanRecord creates a scan record marking an artifact as intentionally
|
|
// not scanned (e.g., helm charts, in-toto attestations). The stale-scan loop
|
|
// leaves these records alone since the outcome won't change without a code
|
|
// change in the scanner.
|
|
func NewSkippedScanRecord(manifestDigest, repository, userDID, reason, scannerVersion string) *ScanRecord {
|
|
return &ScanRecord{
|
|
Type: ScanCollection,
|
|
Manifest: BuildManifestURI(userDID, manifestDigest),
|
|
Repository: repository,
|
|
UserDID: userDID,
|
|
ScannerVersion: scannerVersion,
|
|
ScannedAt: time.Now().Format(time.RFC3339),
|
|
Status: ScanStatusSkipped,
|
|
Reason: reason,
|
|
}
|
|
}
|
|
|
|
// NewFailedScanRecord creates a scan record marking a scan attempt as failed
|
|
// (e.g., scanner crash, OOM, network error during fetch). The stale-scan loop
|
|
// will re-queue these records on the rescan interval — failures may be
|
|
// transient.
|
|
func NewFailedScanRecord(manifestDigest, repository, userDID, reason, scannerVersion string) *ScanRecord {
|
|
return &ScanRecord{
|
|
Type: ScanCollection,
|
|
Manifest: BuildManifestURI(userDID, manifestDigest),
|
|
Repository: repository,
|
|
UserDID: userDID,
|
|
ScannerVersion: scannerVersion,
|
|
ScannedAt: time.Now().Format(time.RFC3339),
|
|
Status: ScanStatusFailed,
|
|
Reason: reason,
|
|
}
|
|
}
|
|
|
|
// ScanRecordKey generates a deterministic record key for a scan result
|
|
// Uses the manifest digest (without algorithm prefix) as the rkey
|
|
// This ensures one scan record per manifest, and re-scans upsert the record
|
|
func ScanRecordKey(manifestDigest string) string {
|
|
// Remove the "sha256:" prefix - the hex digest is already a valid rkey
|
|
return strings.TrimPrefix(manifestDigest, "sha256:")
|
|
}
|
|
|
|
// ImageConfigRecord represents an OCI image config stored in the hold's embedded PDS
|
|
// Collection: io.atcr.hold.image.config
|
|
// One record per manifest, keyed by manifest digest hex (same pattern as ScanRecord)
|
|
// Stores the full OCI config JSON so the appview can display layer history including empty layers
|
|
type ImageConfigRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
Manifest string `json:"manifest" cborgen:"manifest"` // AT-URI of the manifest
|
|
ConfigJSON string `json:"configJson" cborgen:"configJson,maxlen=1000000"` // Raw OCI image config JSON. Cap mirrors lexicon maxLength so deep image histories (Bazel, multi-stage builds with verbose created_by lines) fit.
|
|
CreatedAt string `json:"createdAt" cborgen:"createdAt"` // RFC3339 timestamp
|
|
}
|
|
|
|
// NewImageConfigRecord creates a new image config record
|
|
func NewImageConfigRecord(manifestURI, configJSON string) *ImageConfigRecord {
|
|
return &ImageConfigRecord{
|
|
Type: ImageConfigCollection,
|
|
Manifest: manifestURI,
|
|
ConfigJSON: configJSON,
|
|
CreatedAt: time.Now().Format(time.RFC3339),
|
|
}
|
|
}
|
|
|
|
// TangledProfileRecord represents a Tangled profile for the hold
|
|
// Collection: sh.tangled.actor.profile (singleton record at rkey "self")
|
|
// Stored in the hold's embedded PDS
|
|
// Uses CBOR encoding for efficient storage in hold's carstore
|
|
type TangledProfileRecord struct {
|
|
Type string `json:"$type" cborgen:"$type"`
|
|
Links []string `json:"links" cborgen:"links"`
|
|
Stats []string `json:"stats" cborgen:"stats"`
|
|
Bluesky bool `json:"bluesky" cborgen:"bluesky"`
|
|
Location string `json:"location" cborgen:"location"`
|
|
Description string `json:"description" cborgen:"description"`
|
|
PinnedRepositories []string `json:"pinnedRepositories" cborgen:"pinnedRepositories"`
|
|
}
|