mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-19 16:54:15 +00:00
Evan JarrettandClaude Opus 5
3ceedc8bb5
The aux-record sweep decided whether a record was old enough to collect from a timestamp inside the record body, which for scan records is scannedAt. Every rescan rewrites that, so the grace clock reset continuously and a hold with a rescan_interval shorter than the grace period could never collect an orphaned scan record at all. Grace now measures how long a slot has been continuously observed orphaned across analysis passes. The key deliberately excludes the CID so a rescan rewriting the record in place does not restart the clock, and the check runs last, after the manifest, reachability and co-ownership tests, so a record judged live never accrues orphan age. A pass that errors partway commits nothing rather than resetting the clock on slots it never reached. The clock is in memory, so a restart forgets every observation and each surviving orphan starts its grace again. That delays collection and can never advance it, which is the safe direction, but it does mean a hold restarting more often than the grace period will not collect aux orphans. Persisting it wants a table in pkg/hold/db and is left for later. Applied to both aux collections rather than only to scan records. For image configs the new rule is strictly more conservative, since a record cannot be observed before it is written, and a per-collection table of which timestamps are safe to trust is a thing to maintain and to get quietly wrong. One widening beyond the reported bug, called out rather than left to be found: a record whose body timestamp is unparseable used to be kept forever, because the zero time read as in-grace. It is now collected on the normal schedule, having cleared every other guard plus a full grace period. Also corrects the comment claiming these records hold no blob references. Scan records carry sbomBlob and vulnReportBlob; they live under a prefix the blob sweep never walks, so deleting the record frees nothing, and a future sweep for that space must read those fields before the record goes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U1Km3N3uUmeGaj7VbaM8PF
58 lines
2.8 KiB
Go
58 lines
2.8 KiB
Go
// Package gc implements garbage collection for the hold service.
|
|
//
|
|
// It collects two things on separate clocks: records in the hold's embedded
|
|
// PDS (layer, scan, and image config) whose manifest no longer exists in the
|
|
// owning user's PDS, and blobs in S3 that no live manifest references. Records
|
|
// go first and blobs follow later, so a user who deletes a manifest record
|
|
// directly on their PDS stops being billed for it well before the bytes are
|
|
// actually reclaimed.
|
|
package gc
|
|
|
|
import "time"
|
|
|
|
// Hardcoded defaults - keep configuration simple
|
|
const (
|
|
// gcInterval is how often GC runs (nightly)
|
|
gcInterval = 24 * time.Hour
|
|
|
|
// gcBlobGracePeriod is how old a blob must be, by its own S3 modification
|
|
// time, before GC will delete it. Blob pruning is best-effort: reclaiming
|
|
// bytes a week late costs storage, reclaiming them early can destroy
|
|
// content a client is still pushing or a takedown may yet reverse.
|
|
gcBlobGracePeriod = 7 * 24 * time.Hour
|
|
|
|
// gcRecordGracePeriod is how long a record must have looked collectable
|
|
// before GC treats a missing manifest as an intentional deletion rather
|
|
// than a race.
|
|
//
|
|
// Records are metadata, not content, so they don't need the blob window.
|
|
// What they do need is to outlast a push: blobs and layer records are
|
|
// written before the manifest reaches the user's PDS, so a record that has
|
|
// looked orphaned for less than this may name a manifest that simply
|
|
// hasn't landed yet. A day is far longer than any push and still collects
|
|
// on the next nightly run.
|
|
//
|
|
// The two sweeps start the clock differently, because only one of them has
|
|
// a write time it can trust. Layer records have unique per-write TID rkeys,
|
|
// so the rkey dates the write and the sweep measures from it. Scan and
|
|
// image-config records live at a rkey derived from the manifest digest and
|
|
// are rewritten in place — a rescan restamps io.atcr.hold.scan's only
|
|
// timestamp — so their sweep measures from the first pass that observed
|
|
// them orphaned instead. See auxOrphanState.graceElapsed.
|
|
gcRecordGracePeriod = 24 * time.Hour
|
|
|
|
// maxPreviewAgeForDelete bounds how stale a preview may be when the admin
|
|
// clicks "Delete N Orphaned Records". lastPreview is held in memory for the
|
|
// life of the process, so an open admin tab keeps that button live and
|
|
// actionable long after its findings stopped matching reality. The
|
|
// per-record checks in deleteOrphanedRecords are the real safety net; this
|
|
// just keeps the admin from acting on a picture that is hours or days old.
|
|
maxPreviewAgeForDelete = 30 * time.Minute
|
|
)
|
|
|
|
// Config holds GC configuration
|
|
type Config struct {
|
|
// Enabled controls whether the nightly GC background process runs.
|
|
Enabled bool `yaml:"enabled" comment:"Enable nightly garbage collection of orphaned blobs and records."`
|
|
}
|