mirror of
https://tangled.org/evan.jarrett.net/at-container-registry
synced 2026-09-22 10:14:15 +00:00
Evan JarrettandClaude Opus 5
a63f668de0
An audit of the scan pipeline and the hold side of scanning found several ways scanning stops without saying so. Each fix here was written test-first: a test expressing the wanted behaviour, confirmed failing for the right reason, then the change. A summary-less result crash-looped both processes. worker.go dereferenced result.Summary unconditionally, but processJob only sets it when Grype runs, and SendResult puts the nil on the wire before the scanner dies on it, so handleResult's unguarded log killed the hold too. A nil Summary now means "not scanned for vulnerabilities", deliberately distinct from "scanned, found zero" — inventing a zeroed summary would report every image as clean when Grype never ran. The hold writes a record rather than orphaning the uploaded SBOM, and the appview renders an "SBOM only" state instead of a green Clean badge. The Grype database could wedge with no way back short of a restart. All three throttles in loadVulnDatabase were guarded by vulnDB != nil, so a scanner holding no provider retried a full download on every scan under the exclusive lock. Two earlier attempts at this bug each added one more condition to the same chain; this replaces the chain with a single decision function over a state snapshot, consulted by both call sites so they cannot disagree. That disagreement was itself a bug: the 50-scan reload had never once executed. Two independent halts. An unparseable frame was dropped in silence, stranding a row that held the hold's only dispatch slot forever; it is now answered "skipped" on first delivery. The 10-minute sweep leaked the in-flight digest and wrote no record, permanently retiring one image per timeout. A digest went unvalidated into filepath.Join and os.Create, so a layer digest of sha256:../../../x wrote outside the scan directory, and nothing verified that downloaded bytes hashed to the digest naming them. Digests come from records in a user's own PDS. Both are fixed together: verification is what makes an escaping write self-defeating. Concurrency did not work on either axis. The proactive capacity gate was depth-one hold-wide, so neither extra workers nor extra scanner processes received work. Depth is now the sum of the worker counts scanners advertise on connect, the gate is scoped to proactive work, and dispatch prefers the least-loaded scanner. Disconnects no longer hand a running scan to someone else: a scanner keeps a stable per-process identity and reclaims its own rows within a grace window, while a process that truly restarted returns with a new identity and has its work reclaimed, which is correct because the restart did lose it. The hold's scanning deadline measured queueing rather than scanning, because the scanner acks on receipt and handleAck never refreshed assigned_at. A new "started" message, sent by the worker that dequeues the job, separates the two budgets. An older scanner never sends it and falls under the queueing budget, which is more forgiving than the deadline it gets today. Adds an in-process mock hold and an e2e harness that runs the real client, queue and worker pool, seeded with 84 real manifest records fetched from a live PDS. Real image layouts and the Grype database are fetched by scripts and gitignored; suites needing them skip cleanly, so the default run stays offline and fast. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U1Km3N3uUmeGaj7VbaM8PF
110 lines
4.3 KiB
Go
110 lines
4.3 KiB
Go
// Package scanner defines shared types for the ATCR scanner service.
|
|
// These types are self-contained with no imports from the root module.
|
|
package scanner
|
|
|
|
import "encoding/json"
|
|
|
|
// ScanJob represents a vulnerability scanning job received from the hold service
|
|
type ScanJob struct {
|
|
Seq int64 `json:"seq"`
|
|
ManifestDigest string `json:"manifestDigest"`
|
|
Repository string `json:"repository"`
|
|
Tag string `json:"tag"`
|
|
UserDID string `json:"userDid"`
|
|
UserHandle string `json:"userHandle"`
|
|
HoldDID string `json:"holdDid"`
|
|
HoldEndpoint string `json:"holdEndpoint"`
|
|
Tier string `json:"tier"`
|
|
Config BlobDescriptor `json:"config"`
|
|
Layers []BlobDescriptor `json:"layers"`
|
|
}
|
|
|
|
// ScanJobRaw is the raw WebSocket message with JSON config/layers
|
|
type ScanJobRaw struct {
|
|
Type string `json:"type"` // "job"
|
|
Seq int64 `json:"seq"`
|
|
ManifestDigest string `json:"manifestDigest"`
|
|
Repository string `json:"repository"`
|
|
Tag string `json:"tag"`
|
|
UserDID string `json:"userDid"`
|
|
UserHandle string `json:"userHandle"`
|
|
HoldDID string `json:"holdDid"`
|
|
HoldEndpoint string `json:"holdEndpoint"`
|
|
Tier string `json:"tier"`
|
|
Config json.RawMessage `json:"config"`
|
|
Layers json.RawMessage `json:"layers"`
|
|
}
|
|
|
|
// BlobDescriptor describes a blob (layer or config) in a container image
|
|
type BlobDescriptor struct {
|
|
Digest string `json:"digest"`
|
|
Size int64 `json:"size"`
|
|
MediaType string `json:"mediaType"`
|
|
}
|
|
|
|
// ScanResult contains the output of a completed scan
|
|
type ScanResult struct {
|
|
ManifestDigest string `json:"manifestDigest"`
|
|
SBOM []byte `json:"sbom,omitempty"`
|
|
SBOMDigest string `json:"sbomDigest,omitempty"`
|
|
VulnReport []byte `json:"vulnReport,omitempty"`
|
|
VulnDigest string `json:"vulnDigest,omitempty"`
|
|
Summary *VulnerabilitySummary `json:"summary,omitempty"`
|
|
}
|
|
|
|
// VulnerabilitySummary contains counts of vulnerabilities by severity
|
|
type VulnerabilitySummary struct {
|
|
Critical int `json:"critical"`
|
|
High int `json:"high"`
|
|
Medium int `json:"medium"`
|
|
Low int `json:"low"`
|
|
Total int `json:"total"`
|
|
}
|
|
|
|
// AckMessage is sent from scanner to hold to acknowledge job receipt
|
|
type AckMessage struct {
|
|
Type string `json:"type"` // "ack"
|
|
Seq int64 `json:"seq"`
|
|
}
|
|
|
|
// StartedMessage is sent from scanner to hold when a worker actually begins a
|
|
// scan, as distinct from AckMessage, which is sent from the WebSocket reader
|
|
// the moment the job frame arrives.
|
|
//
|
|
// The gap between the two is the depth of this scanner's own queue multiplied
|
|
// by the per-image scan time, and the hold cannot see into that queue. Without
|
|
// this message its only clock is dispatch, so its scanning deadline budgets
|
|
// queueing as well as scanning and cancels healthy work out from under a
|
|
// backlogged scanner. A hold that does not know the message ignores it.
|
|
type StartedMessage struct {
|
|
Type string `json:"type"` // "started"
|
|
Seq int64 `json:"seq"`
|
|
}
|
|
|
|
// ResultMessage is sent from scanner to hold with scan results
|
|
type ResultMessage struct {
|
|
Type string `json:"type"` // "result"
|
|
Seq int64 `json:"seq"`
|
|
SBOM string `json:"sbom,omitempty"`
|
|
VulnReport string `json:"vulnReport,omitempty"`
|
|
Summary *VulnerabilitySummary `json:"summary,omitempty"`
|
|
}
|
|
|
|
// ErrorMessage is sent from scanner to hold when a scan fails
|
|
type ErrorMessage struct {
|
|
Type string `json:"type"` // "error"
|
|
Seq int64 `json:"seq"`
|
|
Error string `json:"error"`
|
|
}
|
|
|
|
// SkippedMessage is sent from scanner to hold when an artifact is intentionally
|
|
// not scanned (e.g., helm charts, in-toto attestations). Distinct from
|
|
// ErrorMessage so the hold can mark the scan record as "skipped" rather than
|
|
// "failed" — the stale-scan loop will leave skipped records alone since the
|
|
// outcome won't change without a code change in the scanner.
|
|
type SkippedMessage struct {
|
|
Type string `json:"type"` // "skipped"
|
|
Seq int64 `json:"seq"`
|
|
Reason string `json:"reason"`
|
|
}
|