feat!: stream caller-owned items and reduce the content options

Replaces the streaming request/result API with caller-owned items and
consolidates the content options, following the Job execution performance
design.

- `Item`, `BatchSource` and `Run` replace `StreamRequest`, `StreamResult`,
  `StreamSource`, `StreamSink` and `RunStream`. Every accepted item receives
  exactly one terminal callback from a single goroutine and without doing
  I/O; an item ACP cannot process reports `Failed`, and a completion carries
  one outcome per requested target in request order with its error identity
  preserved. A graceful stop stops feeding, finishes the items in flight and
  reports the rest, and a hard stop abandons per-item accounting.
- One `HashPolicy` replaces `WithHash`, `WithSignatureCache` and
  `ForceRehash`. A stored hash is read whenever the policy uses the cache,
  reused only where the policy allows it, and refreshed only when it differs.
- A source-only `WithReadMode` selects buffered (the default) or mapped
  reads, and buffered sources open with `O_NOATIME` where the platform has
  it. `LinearDevice` keeps request ordering, append targets and abort-on-error
  but no longer selects the read mode.
- `Overwrite` becomes a target-device option, `WithReadBuffer` sets the read
  buffer, and `SelectFiles` returns the entries the command line copies,
  replacing `AccurateJob`, `WildcardJob` and the walk options.
- Copy consumers share one reference-counted chunk buffer each from a common
  pool, and the per-item write goroutines finish before the result channel
  closes, so no finished item can be dropped.

The command line tools, the documentation and the test suite move with the
API, including semantic tests for the callback contract, the stop and drain
paths, the hash policy matrix and the read modes.

README.md

@@ -4,16 +4,140 @@ An Advanced Copy Tools, with following extra features:
 - Sorted copy order, to improve tape device read performance
 - Concurrent source preparation with request-ordered writes for linear targets
 - Multi target path, read once write many
-- Read file with mmap, with small file prefetch hint
+- Buffered or memory-mapped source reads
 - JSON format job report
-- Optional SHA-256 xattr cache for hash-only streams
+- Optional SHA-256 xattr cache for content hashes
 - Can use as a golang library
 
+## Library API
+
+The caller owns the work. A `BatchSource` supplies items, and every item reports its own
+terminal outcome, so no id lookup or result plumbing is needed:
+
+```go
+type Item interface {
+	Source() string
+	Targets() []string
+
+	Completed(*Result)
+	Failed(error)
+}
+
+type BatchSource interface {
+	Next(context.Context) ([]Item, error) // returns io.EOF when the input ends
+}
+
+func Run(ctx context.Context, source BatchSource, opts ...Option) error
+```
+
+`Completed` reports the content facts of one processed item plus one outcome per requested
+target, in request order:
+
+```go
+type Result struct {
+	Source string
+
+	Size      int64
+	Mode      fs.FileMode
+	ModTime   time.Time
+	WriteTime time.Time
+
+	SHA256            []byte
+	SignatureCacheHit bool
+
+	Targets []TargetResult
+}
+
+type TargetResult struct {
+	Path      string
+	Size      int64
+	WriteTime time.Time
+	Err       error // nil when this target was written and verified
+}
+```
+
+### Item contract
+
+- Every accepted item receives exactly one terminal callback, whether it completes or
+  fails, and callbacks are delivered one at a time from a single pipeline goroutine.
+  ACP performs no I/O on the caller's behalf inside the callback, so a callback may do its
+  own.
+- `Completed` is an item ACP processed. It carries one `TargetResult` per requested target
+  in request order, and an item whose every target failed is still a completion. Target
+  errors keep their identity, so `errors.Is(err, acp.ErrTargetNoSpace)` identifies an
+  exhausted medium.
+- `Failed` is reserved for items ACP could not process at all, such as a source it cannot
+  stat or open, and for items abandoned by a stop. A failed item never reports a result.
+- A graceful stop (the caller cancels the context) stops feeding items, finishes the items
+  already being copied, abandons the rest with `Failed` carrying the stopping error,
+  starts no read for an item it has not begun, and makes `Run` return the stopping error.
+
+`Run` returns an error only for a pipeline failure, such as a `BatchSource` error, or for
+the caller's stop. Per-item failures arrive through the item callbacks.
+
+## Options
+
+| Option | Scope | Meaning |
+| --- | --- | --- |
+| `SetFromDevice(...)` | run | Source device options |
+| `SetToDevice(...)` | run | Target device options |
+| `LinearDevice(bool)` | device | One worker and request order for a tape-like device |
+| `DeviceThreads(int)` | device | Worker count, default 8, forced to 1 for a linear device |
+| `Overwrite(bool)` | target device | Replace an existing target instead of refusing it |
+| `WithReadMode(ReadBuffered\|ReadMapped)` | source device | Read sources buffered or through a memory mapping |
+| `WithReadBuffer(int)` | run | How many items ACP may hold in its read buffer, default 4096 |
+| `WithHashPolicy(HashPolicy)` | run | Content hash and stored hash policy, default `HashOff` |
+| `WithLogger(*logrus.Logger)` | run | Logger for pipeline diagnostics |
+| `WithEventHandler(EventHandler)` | run | Progress, job, and error events |
+| `WithProgressBar()` | run | Terminal progress bar handler |
+
+`WithReadMode` applies to the source device only; a target read mode is rejected. Buffered
+reads are the default and avoid updating the source access time where the platform allows
+it. Mapped reads use the `mmap` package with a fallback for empty files.
+
+## Content hash policy
+
+`HashPolicy` selects both how an item's content hash is produced and how the stored hash is
+used, because refreshing the cache requires a computed hash:
+
+| Policy | Hash | Stored hash |
+| --- | --- | --- |
+| `HashOff` | none | untouched |
+| `HashCachedOnly` | reused if stored, otherwise none | untouched |
+| `HashCachedOrRead` | reused if stored, otherwise read | untouched |
+| `HashCachedOrReadRefresh` | reused if stored, otherwise read | written when it differs |
+| `HashRead` | always read | untouched |
+| `HashReadRefresh` | always read | written when it differs |
+
+A request with targets always reads its source, so a stored hash cannot be reused there. A
+policy that neither reads content nor finds a stored hash completes the item without
+opening the source at all. A refresh publishes the stored value only when it differs from
+the computed one, which keeps an identical rewrite from touching the attribute.
+
 ## Content signature cache
 
-Library callers can opt in with `WithSignatureCache(true)` and bypass reads with `ForceRehash(true)`. ACP stores a fixed binary SHA-256, file size, and nanosecond mtime value in `user.acp.signature` on Linux and the canonical `acp.signature` user attribute on Darwin and FreeBSD.
+ACP stores a fixed binary SHA-256 plus the file size and nanosecond mtime in
+`user.acp.signature` on Linux and the canonical `acp.signature` user attribute on Darwin
+and FreeBSD.
 
-The xattr is a disposable optimization. Hash-only jobs may reuse a metadata-valid value. Jobs with targets always read and hash content, then refresh the source and every successful target before `WaitErr` or `RunStream` returns. Writers publish the hashed version's size and mtime unchanged, so an entry whose file changed after hashing simply fails the reader's metadata comparison and is treated as stale. Missing, stale, corrupt, read-only, full, or unsupported xattrs are summarized as warnings and never become copy errors. ACP's managed key is not copied as an ordinary source xattr.
+The xattr is a disposable optimization. A policy that reuses a stored hash may skip reading
+content, and a policy that refreshes rewrites the source and every successful target after
+the item finishes. Writers publish the hashed version's size and mtime unchanged, so an
+entry whose file changed after hashing simply fails the reader's metadata comparison and is
+treated as stale. Missing, stale, corrupt, read-only, full, or unsupported xattrs are
+summarized as warnings and never become copy errors. ACP's managed key is not copied as an
+ordinary source xattr.
+
+## File selection
+
+`SelectFiles` walks sources, keeps regular files, maps each source-relative path onto every
+target directory, removes repeated relative paths, and returns entries in the platform's
+path order:
+
+```go
+entries, err := acp.SelectFiles([]string{"source"}, []string{"target-a", "target-b"})
+// acp.FileEntry{Base, Path, Source, Targets}
+```
 
 # Install
 ```

TESTING.md

@@ -7,6 +7,16 @@ Run all commands from the repository root.
 - Go 1.18 or newer.
 - Enough free space under the system temporary directory for test files and compiled binaries.
 - A host file system that supports the operations exercised by the selected tests.
+- A host file system that supports the managed signature xattr for the cache tests. A test
+  that cannot store it skips itself with a diagnostic.
+
+When this module is checked out inside another Go workspace, run every command with an
+explicit empty `GOFLAGS` and `GOWORK=off` so the parent build flags and workspace do not
+leak into module resolution:
+
+```sh
+GOFLAGS= GOWORK=off go test ./...
+```
 
 ## Standard checks
 
@@ -46,8 +56,9 @@ go test -run '^TestACPCommandE2E$' -v .
 
 The library end-to-end test uses the public Go API directly with two destinations and verifies:
 
-- the complete index, prepare, copy, cleanup, and event pipeline;
+- file selection, item submission, and the complete prepare, copy, cleanup, and event pipeline;
 - regular, empty, and nested file contents at both destinations;
+- one terminal callback per submitted item with one target outcome per destination;
 - successful report status and destination sets;
 - file size and exact SHA256 values.
 
@@ -79,13 +90,57 @@ Run path and mountpoint tests:
 go test -run '^(TestComparePath|TestSourceRoot|TestFindMountpoint)$' .
 ```
 
+Run file selection tests:
+
+```sh
+go test -run '^TestSelectFiles' .
+```
+
+Run item contract tests, which pin one terminal callback per accepted item, one outcome per
+requested target in request order, failure routing, and single-goroutine callback delivery:
+
+```sh
+go test -run '^(TestRunReportsEveryAcceptedItemExactlyOnce|TestRunKeepsLinearOrderingPastAnUnprocessedItem|TestRunReportsUnprocessableItemsAsFailures|TestRunReportsOneOutcomePerRequestedTargetInOrder|TestRunReportsTargetFailureAsItemOutcome|TestRunDeliversTerminalCallbacksFromOneGoroutine|TestPrepareReportsUnstartedItemsWithoutReadingThem|TestPrepareCompletesWithoutOpeningTheSourceWhenPolicyNeedsNoContent)$' .
+```
+
+Run graceful-stop tests, which drive a source that blocks or spans several batches and
+cancel the context mid-run:
+
+```sh
+go test -run '^(TestRunStopsFeedingItemsAfterGracefulStop|TestRunReturnsStoppingErrorWhenBatchSourceEndsFirst|TestRunCancellationDrainsPrefetchedItems|TestRunReportsEveryAcceptedItemExactlyOnce)$' .
+```
+
 Run linear stream-order tests:
 
 ```sh
-go test -run '^(TestRunStreamCopiesRequestsToLinearTarget|TestForwardPreparedOrdersOnlyLinearTargets|TestRunStreamAppliesBoundedBackpressure|TestRunStreamCancellationDrainsPrefetchedJobs|TestLinearTargetStopsWhenDiskUsageEstimateIsInsufficient)$' .
+go test -run '^(TestRunCopiesItemsToLinearTarget|TestForwardPreparedOrdersOnlyLinearTargets|TestRunAppliesBoundedBackpressure|TestLinearTargetStopsWhenDiskUsageEstimateIsInsufficient|TestStoppedLinearTargetDoesNotReadBatchSource)$' .
 ```
 
-On Linux, include `TestRunStreamMapsDeviceFullToTargetNoSpace` to exercise an actual `/dev/full` write failure.
+Run the target-failure drain tests, which prove every read buffer is released and no
+goroutine deadlocks:
+
+```sh
+go test -race -run '^(TestWriteFailureDrainsBuffersAndTargets|TestDeviceFullWriteFailureDrainsQueuedBuffers)$' .
+```
+
+On Linux, include `TestRunMapsDeviceFullToTargetNoSpace` and
+`TestDeviceFullWriteFailureDrainsQueuedBuffers` to exercise actual `/dev/full` write failures:
+
+```sh
+go test -race -run '^(TestRunMapsDeviceFullToTargetNoSpace|TestDeviceFullWriteFailureDrainsQueuedBuffers)$' .
+```
+
+Run hash policy tests, which pin the reuse, read, and refresh matrix:
+
+```sh
+go test -run '^(TestRunHashPolicyMatrix|TestRunRefreshWritesOnlyWhenStoredHashDiffers|TestRunTransferAlwaysReadsAndRefreshesTargets|TestOverwriteInvalidatesSignatureWithoutCache|TestRunCorruptSignatureIsWarning|TestRunSignatureCacheZeroLength)$' .
+```
+
+Run every signature cache test, including the drain tests:
+
+```sh
+go test -run '^TestRunSignature|^TestSignatureCache' .
+```
 
 Run `acp-rewrite` tests:
 

acp.go

@@ -19,6 +19,13 @@ type Copyer struct {
 	getDiskUsageCache func(mountPoint string) *diskUsageCache
 	linearTargetEnded uint32
 	signatures        *signatureCache
+
+	// hardStop ends the pipeline without per-item accounting, for internal failures.
+	hardStop     chan struct{}
+	hardStopOnce sync.Once
+	// abandoned carries items that will never be processed so the reporting goroutine
+	// can still report them.
+	abandoned chan *baseJob
 }
 
 func New(ctx context.Context, opts ...Option) (*Copyer, error) {
@@ -41,12 +48,14 @@ func New(ctx context.Context, opts ...Option) (*Copyer, error) {
 	c := &Copyer{
 		option:    opt,
 		eventCh:   make(chan Event, 128),
+		hardStop:  make(chan struct{}),
+		abandoned: make(chan *baseJob),
 		getDevice: getDevice,
 		getDiskUsageCache: Cache(func(mountPoint string) *diskUsageCache {
 			return newDiskUsageCache(mountPoint, defaultDiskUsageFreshInterval)
 		}),
 	}
-	if opt.withSignatureCache {
+	if opt.hashPolicy.usesCache() {
 		c.signatures = newSignatureCache(signatureWorkers(opt.fromDevice, opt.toDevice))
 	}
 
@@ -57,6 +66,52 @@ func New(ctx context.Context, opts ...Option) (*Copyer, error) {
 	return c, nil
 }
 
+// stopHard ends the pipeline immediately. Per-item accounting is not promised afterwards.
+func (c *Copyer) stopHard() {
+	c.hardStopOnce.Do(func() {
+		close(c.hardStop)
+	})
+}
+
+// stopped reports whether the caller stopped the pipeline gracefully.
+func (c *Copyer) stopped(ctx context.Context) bool {
+	if ctx.Err() != nil || c.linearTargetStopped() {
+		return true
+	}
+	select {
+	case <-c.hardStop:
+		return true
+	default:
+		return false
+	}
+}
+
+// abandonment describes why an accepted item never entered processing.
+func (c *Copyer) abandonment(ctx context.Context) error {
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if c.linearTargetStopped() {
+		return ErrTargetNoSpace
+	}
+	return context.Canceled
+}
+
+// abandon hands an accepted item to the reporting goroutine, because a stage stops
+// reading its input but must not drop a job it already owns.
+func (c *Copyer) abandon(job *baseJob, reason error) bool {
+	if job == nil || job.item == nil {
+		return false
+	}
+	job.itemError = reason
+	select {
+	case c.abandoned <- job:
+		return true
+	case <-c.hardStop:
+		return false
+	}
+}
+
 func (c *Copyer) Wait() {
 	c.running.Wait()
 }
@@ -106,22 +161,17 @@ func (c *Copyer) run(ctx context.Context) error {
 	indexed, err := c.index(ctx)
 	if err != nil {
 		c.setError(err)
+		c.stopHard()
 		return err
 	}
 
-	// Run preparation, copying, and result persistence as one pipeline.
+	// Run preparation, copying, and result reporting as one pipeline. Every stage drains
+	// what it holds, so a graceful stop still reports every accepted item.
 	prepared := c.prepare(ctx, indexed)
 	copyed := c.copy(ctx, prepared)
-	sinkFailed := c.cleanupJob(ctx, cancel, copyed)
+	c.cleanup(ctx, copyed)
 
-	// Flush persisted results unless a Sink write already failed.
-	if c.streamSink != nil && !sinkFailed {
-		if err := c.streamSink.Flush(ctx); err != nil {
-			c.setError(fmt.Errorf("flush stream sink failed, %w", err))
-		}
-	}
-
-	// Drain remaining stages in dependency order. Prepared jobs retain open sources.
+	// Drain remaining stages in dependency order. A hard stop leaves jobs behind.
 	for range indexed {
 	}
 	for job := range prepared {
@@ -188,3 +238,11 @@ func (c *Copyer) reportError(src, dst string, err error) {
 	c.logf(logrus.ErrorLevel, e.Error())
 	c.submit(&EventReportError{Error: e})
 }
+
+// reportItemError reports one item's failure without failing the pipeline: the caller
+// receives the failure as an item outcome and decides what it means.
+func (c *Copyer) reportItemError(src, dst string, err error) {
+	e := &Error{Src: src, Dst: dst, Err: err}
+	c.logf(logrus.ErrorLevel, e.Error())
+	c.submit(&EventReportError{Error: e})
+}

cleanup.go

@@ -7,56 +7,88 @@ import (
 	"os"
 )
 
-func (c *Copyer) cleanupJob(ctx context.Context, cancel context.CancelFunc, copyed <-chan *baseJob) bool {
+// cleanup reports every accepted item exactly once. It is the only goroutine that
+// invokes item callbacks, so callbacks arrive one at a time and a slow callback cannot
+// stall the read pipeline.
+func (c *Copyer) cleanup(ctx context.Context, copyed <-chan *baseJob) {
 	for {
 		select {
 		case job, ok := <-copyed:
 			if !ok {
-				return false
+				return
 			}
 
-			// Restore metadata before publishing the final result.
-			for _, dst := range append([]string(nil), job.successTargets...) {
-				if err := mappingError(writeSysStat(dst, job.stat)); err != nil {
-					c.endLinearTarget(err)
-					job.fail(dst, fmt.Errorf("change info, write sys stat fail, %w", err))
-					c.reportError(job.path, dst, fmt.Errorf("change info, write sys stat fail, %w", err))
-
-					// Remove the failed target so the same operation can retry it.
-					if err := os.Remove(dst); err != nil && !errors.Is(err, os.ErrNotExist) {
-						c.reportError(job.path, dst, fmt.Errorf("delete target after metadata failure failed, %w", err))
-					}
-				}
-			}
-
-			// Refresh only signatures backed by a complete source hash.
-			shouldRefreshSignature := c.signatures != nil && job.hashValid && !job.cacheHit
-			if shouldRefreshSignature {
-				signature, err := newCachedSignature(job.hash, job.stat)
-				if err != nil {
-					c.signatures.recordFailure(job.path, err)
-				} else {
-					c.signatures.enqueue(job.path, signature)
-					for _, dst := range job.successTargets {
-						c.signatures.enqueue(dst, signature)
-					}
-				}
-			}
-
-			// Publish only results whose data and metadata lifecycle has finished.
-			job.setStatus(jobStatusFinished)
-			if c.streamSink == nil {
+			// An item with no target outcome to report could not be processed at all, so
+			// it fails instead of completing with content facts it never established.
+			if job.itemError != nil {
+				c.failedJob(job)
 				continue
 			}
-			if err := c.streamSink.Write(ctx, &StreamResult{ID: job.streamID, Job: job.report()}); err != nil {
-				// Stop upstream writes once final results can no longer be persisted.
-				c.setError(fmt.Errorf("write stream result failed, id=%d, %w", job.streamID, err))
-				cancel()
-				return true
-			}
-		case <-ctx.Done():
-			c.setError(ctx.Err())
-			return false
+			c.finishJob(job)
+		case job := <-c.abandoned:
+			c.failedJob(job)
+		case <-c.hardStop:
+			return
 		}
 	}
 }
+
+// failedJob reports an accepted item that ACP could not process at all.
+func (c *Copyer) failedJob(job *baseJob) {
+	job.item.Failed(job.itemError)
+}
+
+// refreshSignature publishes a computed signature unless the target already stores it.
+func (c *Copyer) refreshSignature(path string, signature CachedSignature) {
+	stored, ok, err := ReadCachedSignature(path)
+	if err == nil && ok && stored == signature {
+		return
+	}
+
+	c.signatures.enqueue(path, signature)
+}
+
+// finishJob publishes one finished job after restoring its metadata and refreshing the
+// content cache.
+func (c *Copyer) finishJob(job *baseJob) {
+	// An item ACP could not process at all is reported as a failure rather than as a
+	// completion with failed target outcomes.
+	if job.itemError != nil {
+		job.setStatus(jobStatusFinished)
+		job.item.Failed(job.itemError)
+		return
+	}
+
+	// Restore metadata before publishing the final result.
+	for _, dst := range append([]string(nil), job.successTargets...) {
+		if err := mappingError(writeSysStat(dst, job.stat)); err != nil {
+			c.endLinearTarget(err)
+			job.fail(dst, fmt.Errorf("change info, write sys stat fail, %w", err))
+
+			// Remove the failed target so the same operation can retry it.
+			if err := os.Remove(dst); err != nil && !errors.Is(err, os.ErrNotExist) {
+				c.reportItemError(job.path, dst, fmt.Errorf("delete target after metadata failure failed, %w", err))
+			}
+		}
+	}
+
+	// Refresh signatures backed by a complete source hash, and only where the stored
+	// value differs: an identical rewrite costs a physical xattr block on copy-on-write
+	// filesystems and leaves the cache in the same state.
+	shouldRefreshSignature := c.signatures != nil && job.hashValid && !job.cacheHit && c.hashPolicy.refreshesCache()
+	if shouldRefreshSignature {
+		signature, err := newCachedSignature(job.hash, job.stat)
+		if err != nil {
+			c.signatures.recordFailure(job.path, err)
+		} else {
+			c.refreshSignature(job.path, signature)
+			for _, dst := range job.successTargets {
+				c.refreshSignature(dst, signature)
+			}
+		}
+	}
+
+	// Publish only results whose data and metadata lifecycle has finished.
+	job.setStatus(jobStatusFinished)
+	job.item.Completed(job.result())
+}

cleanup_test.go

@@ -11,10 +11,7 @@ import (
 func TestCleanupRemovesTargetAfterMetadataFailure(t *testing.T) {
 	// Use a dangling symlink so metadata restoration fails while removal remains possible.
 	root := t.TempDir()
-	sourcePath := filepath.Join(root, "source")
-	if err := os.WriteFile(sourcePath, []byte("fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
+	sourcePath := writeSourceFile(t, root, "source", []byte("fixture"))
 	info, err := os.Stat(sourcePath)
 	if err != nil {
 		t.Fatal(err)
@@ -29,31 +26,30 @@ func TestCleanupRemovesTargetAfterMetadataFailure(t *testing.T) {
 	}
 
 	// Cleanup must discard the failed target before publishing its final result.
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
-	option := newOption()
-	if err := option.check(); err != nil {
-		t.Fatal(err)
-	}
-	copyer := &Copyer{option: option, eventCh: make(chan Event, 8)}
+	copyer := newTestCopyer(t)
+	item := newFixtureItem(sourcePath, target)
 	job := &baseJob{
 		copyer:         copyer,
+		item:           item,
 		src:            &source{base: root, path: "source"},
 		path:           sourcePath,
 		stat:           stat,
+		targets:        []string{target},
 		successTargets: []string{target},
 	}
 	copyed := make(chan *baseJob, 1)
 	copyed <- job
 	close(copyed)
-	if sinkFailed := copyer.cleanupJob(ctx, cancel, copyed); sinkFailed {
-		t.Fatal("cleanup reported an unexpected Sink failure")
-	}
+	copyer.cleanup(context.Background(), copyed)
+
 	if _, err := os.Lstat(target); !errors.Is(err, os.ErrNotExist) {
 		t.Fatalf("stat failed target error = %v, want %v", err, os.ErrNotExist)
 	}
-	report := job.report()
-	if !errors.Is(report.FailTargets[target], os.ErrNotExist) {
-		t.Fatalf("metadata failure = %v, want %v", report.FailTargets[target], os.ErrNotExist)
+	result, terminalErr := item.terminal(t)
+	if terminalErr != nil {
+		t.Fatalf("a metadata failure is a target outcome, not an item failure: %v", terminalErr)
+	}
+	if len(result.Targets) != 1 || !errors.Is(result.Targets[0].Err, os.ErrNotExist) {
+		t.Fatalf("target outcome = %#v, want %v", result.Targets, os.ErrNotExist)
 	}
 }

cmd/acp-rewrite/main.go

@@ -6,6 +6,7 @@ import (
 	"errors"
 	"flag"
 	"fmt"
+	"io"
 	"io/fs"
 	"math/rand"
 	"os"
@@ -359,6 +360,36 @@ func newTmpPath(path string, rnd *rand.Rand) (string, error) {
 	return "", fmt.Errorf("tmp path collide")
 }
 
+// rewriteSource supplies one file as the only item of a rewrite run.
+type rewriteSource struct {
+	source string
+	target string
+
+	done bool
+}
+
+func (s *rewriteSource) Next(ctx context.Context) ([]acp.Item, error) {
+	if s.done {
+		return nil, io.EOF
+	}
+	s.done = true
+
+	return []acp.Item{&rewriteItem{source: s.source, target: s.target}}, nil
+}
+
+// rewriteItem carries one file through the copy pipeline. The rewrite state and the
+// report come from events and the rewritten file, so the terminal callbacks record
+// nothing here.
+type rewriteItem struct {
+	source string
+	target string
+}
+
+func (i *rewriteItem) Source() string        { return i.source }
+func (i *rewriteItem) Targets() []string     { return []string{i.target} }
+func (i *rewriteItem) Completed(*acp.Result) {}
+func (i *rewriteItem) Failed(error)          {}
+
 func rewriteFile(ctx context.Context, entry rewriteEntry, tmpPath string) (*acp.Report, error) {
 	stat, err := os.Stat(entry.Path)
 	if err != nil {
@@ -369,19 +400,17 @@ func rewriteFile(ctx context.Context, entry rewriteEntry, tmpPath string) (*acp.
 	}
 
 	opts := []acp.Option{
-		acp.AccurateJob(entry.Path, []string{tmpPath}),
-		acp.WithHash(true),
-		acp.Overwrite(true),
+		acp.WithHashPolicy(acp.HashRead),
+		acp.SetToDevice(acp.Overwrite(true)),
 	}
 
 	handler, getter := acp.NewReportGetter()
 	opts = append(opts, acp.WithEventHandler(handler))
 
-	c, err := acp.New(ctx, opts...)
-	if err != nil {
+	items := &rewriteSource{source: entry.Path, target: tmpPath}
+	if err := acp.Run(ctx, items, opts...); err != nil {
 		return nil, err
 	}
-	c.Wait()
 
 	report := getter()
 	if report == nil {

cmd/acp/main.go

@@ -2,10 +2,14 @@ package main
 
 import (
 	"context"
+	"encoding/hex"
 	"flag"
+	"io"
 	"os"
 	"os/signal"
+	"path/filepath"
 	"strings"
+	"sync"
 
 	"github.com/klauspost/cpuid/v2"
 	"github.com/samuelncui/acp"
@@ -63,67 +67,35 @@ func main() {
 		}
 	}()
 
-	opts := make([]acp.Option, 0, 8)
-
-	useAccurate := func() bool {
-		if *noTarget {
-			return false
-		}
-		if len(sources) > 1 {
-			return false
-		}
-		if len(targetPaths) > 1 {
-			return false
-		}
-
-		dst, src := targetPaths[0], sources[0]
-		if strings.HasSuffix(dst, "/") {
-			return false
-		}
-
-		dstStat, err := os.Stat(dst)
-		if err == nil {
-			return !dstStat.IsDir()
-		}
-		if !os.IsNotExist(err) {
-			logrus.Fatalf("stat dst path fail, %s", err)
-			panic(err)
-		}
-
-		srcStat, err := os.Stat(src)
-		if err == nil {
-			return srcStat.Mode().IsRegular()
-		}
-
-		logrus.Fatalf("stat src path fail, %s", err)
-		panic(err)
-	}()
-
-	if useAccurate {
-		opts = append(opts, acp.AccurateJob(sources[0], []string{targetPaths[0]}))
+	// Select the entries to copy: one exact target path, or every file under the sources
+	// mapped onto the target directories.
+	var entries []acp.FileEntry
+	if !*noTarget && accurateTarget(sources, targetPaths) {
+		src := filepath.Clean(sources[0])
+		base, name := filepath.Split(src)
+		entries = []acp.FileEntry{{
+			Base:    base,
+			Path:    name,
+			Source:  src,
+			Targets: []string{filepath.Clean(targetPaths[0])},
+		}}
 	} else {
-		opts = append(opts, acp.WildcardJob(acp.Source(sources...), acp.Target(targetPaths...)))
+		selected, err := acp.SelectFiles(sources, targetPaths)
+		if err != nil {
+			logrus.Fatalf("unexpected exit: %s", err)
+		}
+		entries = selected
 	}
 
-	// if *continueReport != "" {
-	// 	f, err := os.Open(*continueReport)
-	// 	if err != nil {
-	// 		logrus.Fatalf("cannot open continue report file, %s", err)
-	// 	}
-
-	// 	r := new(acp.Report)
-	// 	if err := json.NewDecoder(f).Decode(r); err != nil {
-	// 		logrus.Fatalf("decode continue report file, %s", err)
-	// 	}
-
-	// 	for _, s := range r.NoSpaceSources {
-	// 		logrus.Infof("restore unfinished: base= '%s' relative_path= '%v'", s.Base, s.RelativePaths)
-	// 		opts = append(opts, acp.AccurateSource(s.Base, s.RelativePaths...))
-	// 	}
-	// }
-
-	opts = append(opts, acp.WithHash(*reportPath != ""))
-	opts = append(opts, acp.Overwrite(!*notOverwrite))
+	report := newReport()
+	opts := make([]acp.Option, 0, 8)
+	// The command line reports hashes only when it is asked for a report.
+	hashPolicy := acp.HashOff
+	if *reportPath != "" {
+		hashPolicy = acp.HashRead
+	}
+	opts = append(opts, acp.WithHashPolicy(hashPolicy))
+	opts = append(opts, acp.SetToDevice(acp.Overwrite(!*notOverwrite)))
 
 	if *withProgressBar {
 		opts = append(opts, acp.WithProgressBar())
@@ -136,31 +108,162 @@ func main() {
 		opts = append(opts, acp.SetToDevice(acp.LinearDevice(true)))
 	}
 
-	if *reportPath != "" {
-		handler, getter := acp.NewReportGetter()
-		opts = append(opts, acp.WithEventHandler(handler))
-		defer func() {
-			if *reportPath == "" {
-				return
-			}
+	opts = append(opts, acp.WithEventHandler(report.handleEvent))
+	defer func() {
+		if *reportPath == "" {
+			return
+		}
 
-			report := getter()
-			r, err := os.Create(*reportPath)
-			if err != nil {
-				logrus.Warnf("open report fail, path= '%s', err= %s", *reportPath, err)
-				logrus.Infof("report= %q", report.ToJSONString(false))
-				return
-			}
-			defer r.Close()
+		r, err := os.Create(*reportPath)
+		if err != nil {
+			logrus.Warnf("open report fail, path= '%s', err= %s", *reportPath, err)
+			logrus.Infof("report= %q", report.getter().ToJSONString(false))
+			return
+		}
+		defer r.Close()
 
-			r.Write([]byte(report.ToJSONString(*reportIndent)))
-		}()
+		r.Write([]byte(report.getter().ToJSONString(*reportIndent)))
+	}()
+
+	if err := acp.Run(ctx, &fileSource{entries: entries, report: report}, opts...); err != nil {
+		logrus.Errorf("copy failed, %s", err)
 	}
+}
 
-	c, err := acp.New(ctx, opts...)
-	if err != nil {
-		logrus.Fatalf("unexpected exit: %s", err)
+// accurateTarget reports whether one source path copies to one exact target path instead
+// of mapping a source tree onto a target directory.
+func accurateTarget(sources, targets []string) bool {
+	if len(sources) > 1 || len(targets) > 1 {
+		return false
 	}
 
-	c.Wait()
+	dst, src := targetPaths[0], sources[0]
+	if strings.HasSuffix(dst, "/") {
+		return false
+	}
+
+	dstStat, err := os.Stat(dst)
+	if err == nil {
+		return !dstStat.IsDir()
+	}
+	if !os.IsNotExist(err) {
+		logrus.Fatalf("stat dst path fail, %s", err)
+	}
+
+	srcStat, err := os.Stat(src)
+	if err == nil {
+		return srcStat.Mode().IsRegular()
+	}
+
+	logrus.Fatalf("stat src path fail, %s", err)
+	return false
+}
+
+// fileSource supplies the selected entries as caller-owned items.
+type fileSource struct {
+	entries []acp.FileEntry
+	report  *report
+}
+
+func (s *fileSource) Next(ctx context.Context) ([]acp.Item, error) {
+	if len(s.entries) == 0 {
+		return nil, io.EOF
+	}
+
+	entries := s.entries
+	s.entries = nil
+
+	items := make([]acp.Item, 0, len(entries))
+	for _, entry := range entries {
+		items = append(items, &fileItem{entry: entry, report: s.report})
+	}
+	return items, nil
+}
+
+// fileItem carries one file entry through the copy pipeline.
+type fileItem struct {
+	entry  acp.FileEntry
+	report *report
+}
+
+func (i *fileItem) Source() string { return i.entry.Source }
+
+func (i *fileItem) Targets() []string { return i.entry.Targets }
+
+// Completed records the item's outcome for the JSON report. The pipeline reports events
+// as the item advances, so the callback itself performs no I/O.
+func (i *fileItem) Completed(result *acp.Result) {
+	i.report.completed(i.entry, result)
+}
+
+func (i *fileItem) Failed(err error) {
+	i.report.failed(i.entry, err)
+}
+
+// report collects job rows and errors for the JSON report.
+type report struct {
+	lock   sync.Mutex
+	jobs   []*acp.Job
+	errors []*acp.Error
+}
+
+func newReport() *report {
+	return new(report)
+}
+
+func (r *report) handleEvent(event acp.Event) {
+	failure, ok := event.(*acp.EventReportError)
+	if !ok {
+		return
+	}
+
+	r.lock.Lock()
+	defer r.lock.Unlock()
+	r.errors = append(r.errors, failure.Error)
+}
+
+func (r *report) completed(entry acp.FileEntry, result *acp.Result) {
+	r.lock.Lock()
+	defer r.lock.Unlock()
+
+	job := &acp.Job{
+		FullPath: entry.Source,
+		Base:     entry.Base,
+		Path:     entry.Path,
+
+		Status:    acp.JobStatusFinished,
+		Size:      result.Size,
+		Mode:      result.Mode,
+		ModTime:   result.ModTime,
+		WriteTime: result.WriteTime,
+		SHA256:    hex.EncodeToString(result.SHA256),
+
+		SignatureCacheHit: result.SignatureCacheHit,
+	}
+	for _, target := range result.Targets {
+		if target.Err == nil {
+			job.SuccessTargets = append(job.SuccessTargets, target.Path)
+			continue
+		}
+		if job.FailTargets == nil {
+			job.FailTargets = make(map[string]error, len(result.Targets))
+		}
+		job.FailTargets[target.Path] = target.Err
+	}
+
+	r.jobs = append(r.jobs, job)
+}
+
+func (r *report) failed(entry acp.FileEntry, err error) {
+	r.lock.Lock()
+	defer r.lock.Unlock()
+
+	r.errors = append(r.errors, &acp.Error{Src: entry.Source, Err: err})
+}
+
+func (r *report) getter() *acp.Report {
+	r.lock.Lock()
+	defer r.lock.Unlock()
+
+	return &acp.Report{Jobs: r.jobs, Errors: r.errors}
 }

copy.go

@@ -25,6 +25,42 @@ var (
 	sha256Pool = &sync.Pool{New: func() interface{} { return sha256.New() }}
 )
 
+// chunkBuffer is one read buffer shared by the hash consumer and every target writer of
+// one item. Each consumer holds a reference and the last release returns the buffer to
+// the shared pool, so allocation churn disappears while the read-ahead window stays
+// exactly what the consumer channels allow.
+type chunkBuffer struct {
+	data []byte
+	refs int32
+}
+
+var chunkPool = sync.Pool{
+	New: func() interface{} {
+		return &chunkBuffer{data: make([]byte, batchSize)}
+	},
+}
+
+// acquireChunk returns a pooled buffer holding one producer reference.
+func acquireChunk() *chunkBuffer {
+	chunk := chunkPool.Get().(*chunkBuffer)
+	chunk.data = chunk.data[:cap(chunk.data)]
+	chunk.refs = 1
+	return chunk
+}
+
+// retain adds one consumer reference to the buffer.
+func (c *chunkBuffer) retain() *chunkBuffer {
+	atomic.AddInt32(&c.refs, 1)
+	return c
+}
+
+// release drops one reference and returns the buffer once no consumer holds it.
+func (c *chunkBuffer) release() {
+	if atomic.AddInt32(&c.refs, -1) == 0 {
+		chunkPool.Put(c)
+	}
+}
+
 func (c *Copyer) copy(ctx context.Context, prepared <-chan *writeJob) <-chan *baseJob {
 	ch := make(chan *baseJob, 128)
 
@@ -63,17 +99,20 @@ func (c *Copyer) copy(ctx context.Context, prepared <-chan *writeJob) <-chan *ba
 		go wrap(ctx, func() {
 			defer copying.Done()
 
+			// Consume prepared readers until the stage closes, so a stopped pipeline
+			// still reports the items it holds instead of dropping them. One worker
+			// writes one item at a time: that bounds target concurrency by the device
+			// thread count and keeps a linear target in request order.
 			for job := range prepared {
-				if ctx.Err() != nil {
-					job.finishSource()
-					continue
-				}
-				if c.linearTargetStopped() {
+				if c.stopped(ctx) {
 					job.finishSource()
+					if !c.abandon(job.baseJob, c.abandonment(ctx)) {
+						return
+					}
 					continue
 				}
 
-				wrap(ctx, func() { c.write(ctx, job, ch, cntr, noSpaceDevices) })
+				c.write(ctx, job, ch, cntr, noSpaceDevices)
 			}
 		})
 	}
@@ -82,7 +121,8 @@ func (c *Copyer) copy(ctx context.Context, prepared <-chan *writeJob) <-chan *ba
 }
 
 func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, cntr *counter, noSpaceDevices mapset.Set[string]) {
-	// Release the source and publish ownership only after every consumer stops.
+	// Release the source and publish ownership only after every consumer stops. A
+	// graceful stop still reports the item this stage finished.
 	var wg sync.WaitGroup
 	defer func() {
 		wg.Wait()
@@ -90,14 +130,19 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 		job.setStatus(jobStatusFinishing)
 		select {
 		case ch <- job.baseJob:
-		case <-ctx.Done():
+		case <-c.hardStop:
 		}
 	}()
 
-	// Reject source changes before creating any targets.
+	// Reject source changes before creating any targets. Every requested target is
+	// reported failed, so the item stays a completion with failed outcomes.
 	job.setStatus(jobStatusCopying)
+	if job.skipContent {
+		atomic.AddInt64(&cntr.files, 1)
+		return
+	}
 	if job.size != job.stat.size {
-		job.fail("", fmt.Errorf("source size changed, indexed=%d current=%d", job.stat.size, job.size))
+		job.failAll(fmt.Errorf("source size changed, indexed=%d current=%d", job.stat.size, job.size))
 		return
 	}
 	if job.cacheHit {
@@ -106,16 +151,16 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 		return
 	}
 
-	// Skip jobs only when every requested target device is already exhausted.
+	// Report every target failed when every requested target device is exhausted.
 	targetDevices := lo.Map(job.targets, func(target string, _ int) string { return c.getDevice(target) })
 	if len(targetDevices) > 0 && noSpaceDevices.Contains(targetDevices...) {
-		job.fail("", ErrTargetNoSpace)
+		job.failAll(ErrTargetNoSpace)
 		return
 	}
 
 	// Track progress and close every consumer after the source reader finishes.
 	atomic.AddInt64(&cntr.files, 1)
-	chans := make([]chan []byte, 0, len(job.targets)+1)
+	chans := make([]chan *chunkBuffer, 0, len(job.targets)+1)
 	defer func() {
 		for _, ch := range chans {
 			close(ch)
@@ -180,7 +225,7 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 		}
 
 		// Consume source buffers in one managed writer for this target.
-		ch := make(chan []byte, 4)
+		ch := make(chan *chunkBuffer, 4)
 		chans = append(chans, ch)
 
 		wg.Add(1)
@@ -202,7 +247,8 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 				c.endLinearTarget(rerr)
 
 				// avoid block channel
-				for range ch {
+				for chunk := range ch {
+					chunk.release()
 				}
 
 				job.fail(target, fmt.Errorf("write dst file fail, %w", rerr))
@@ -217,14 +263,16 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 					_ = file.Close()
 				}
 			}()
-			for buf := range ch {
-				n, err := file.Write(buf)
+			for chunk := range ch {
+				size := len(chunk.data)
+				n, err := file.Write(chunk.data)
+				chunk.release()
 				if err != nil {
 					rerr = fmt.Errorf("write fail, %w", err)
 					return
 				}
-				if len(buf) != n {
-					rerr = fmt.Errorf("write fail, unexpected writen bytes return, read= %d write= %d", len(buf), n)
+				if size != n {
+					rerr = fmt.Errorf("write fail, unexpected writen bytes return, read= %d write= %d", size, n)
 					return
 				}
 			}
@@ -251,11 +299,11 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 	targetWriters := len(chans)
 
 	// Add hashing as another consumer of the shared source stream.
-	if c.withHash {
+	if c.hashPolicy.producesHash() {
 		sha := sha256Pool.Get().(hash.Hash)
 		sha.Reset()
 
-		ch := make(chan []byte, 4)
+		ch := make(chan *chunkBuffer, 4)
 		chans = append(chans, ch)
 
 		wg.Add(1)
@@ -263,10 +311,16 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 			defer wg.Done()
 			defer sha256Pool.Put(sha)
 
-			for buf := range ch {
-				sha.Write(buf)
+			for chunk := range ch {
+				sha.Write(chunk.data)
+				chunk.release()
 			}
 
+			// A stopped read describes only part of the source, so it has no content hash.
+			if readErr != nil {
+				job.setHash(nil)
+				return
+			}
 			job.setHash(sha.Sum(nil))
 		})
 	}
@@ -276,45 +330,51 @@ func (c *Copyer) write(ctx context.Context, job *writeJob, ch chan<- *baseJob, c
 		return
 	}
 	var copied int64
-	copied, readErr = c.streamCopy(ctx, chans, job.reader, &cntr.bytes)
+	copied, readErr = c.streamCopy(chans, job.reader, &cntr.bytes)
 	if readErr == nil && copied != job.size {
 		readErr = fmt.Errorf("source size changed while copying, expected=%d copied=%d", job.size, copied)
 	}
-	if readErr == nil && c.withHash {
+	if readErr == nil && c.hashPolicy.producesHash() {
 		job.validateHash()
 	}
 	if readErr != nil && targetWriters == 0 {
-		job.fail("", readErr)
+		// An item with no target outcome to report against could not be processed at all.
+		c.reportItemError(job.path, "", readErr)
+		job.itemError = readErr
 	}
 }
 
-func (c *Copyer) streamCopy(ctx context.Context, dsts []chan []byte, src io.ReadCloser, bytes *int64) (int64, error) {
+// streamCopy reads the source once and hands every chunk to each consumer. A graceful
+// stop finishes the item in flight; only a pipeline failure ends the read early.
+func (c *Copyer) streamCopy(dsts []chan *chunkBuffer, src io.ReadCloser, bytes *int64) (int64, error) {
 	var copied int64
 	for {
-		buf := make([]byte, batchSize)
+		chunk := acquireChunk()
 
-		n, err := io.ReadFull(src, buf)
+		n, err := io.ReadFull(src, chunk.data)
 		if err != nil {
 			if !errors.Is(err, io.ErrUnexpectedEOF) && !errors.Is(err, io.EOF) {
+				chunk.release()
 				return copied, fmt.Errorf("slice mmap fail, %w", err)
 			}
 		}
 
-		buf = buf[:n]
+		// Every consumer releases its own reference; the producer releases the last one
+		// after the size is captured, because the buffer may be reused immediately.
+		chunk.data = chunk.data[:n]
 		for _, ch := range dsts {
-			ch <- buf
+			ch <- chunk.retain()
 		}
-
-		nr := len(buf)
-		copied += int64(nr)
-		atomic.AddInt64(bytes, int64(nr))
-		if nr < batchSize {
+		copied += int64(n)
+		atomic.AddInt64(bytes, int64(n))
+		chunk.release()
+		if n < batchSize {
 			return copied, nil
 		}
 
 		select {
-		case <-ctx.Done():
-			return copied, ctx.Err()
+		case <-c.hardStop:
+			return copied, fmt.Errorf("copy stopped by pipeline failure")
 		default:
 		}
 	}

copy_test.go

@@ -8,6 +8,8 @@ import (
 	"math"
 	"os"
 	"path/filepath"
+	"sync"
+	"sync/atomic"
 	"syscall"
 	"testing"
 	"time"
@@ -41,47 +43,105 @@ func (*failingReadCloser) Close() error {
 	return nil
 }
 
-func TestCopyEmptyFile(t *testing.T) {
+// failingContentReader delivers whole batches and then fails, so target writers still hold
+// source buffers when the read stage stops.
+type failingContentReader struct {
+	err     error
+	batches int
+	reads   int
+}
+
+func (r *failingContentReader) Read(p []byte) (int, error) {
+	r.reads++
+	if r.reads <= r.batches {
+		return copy(p, bytes.Repeat([]byte{'x'}, len(p))), nil
+	}
+	return 0, r.err
+}
+
+func (*failingContentReader) Close() error {
+	return nil
+}
+
+// trackChunkPool instruments the shared read-buffer pool so a test can prove that every
+// buffer returns to it, including buffers discarded by a failed target writer.
+func trackChunkPool(t *testing.T) {
+	t.Helper()
+
+	var (
+		lock    sync.Mutex
+		buffers []*chunkBuffer
+	)
+
+	// Empty the shared pool first so every buffer this test uses is tracked here.
+	previousNew := chunkPool.New
+	chunkPool.New = nil
+	for chunkPool.Get() != nil {
+	}
+	t.Cleanup(func() {
+		chunkPool.New = previousNew
+
+		lock.Lock()
+		defer lock.Unlock()
+		if len(buffers) == 0 {
+			t.Error("the pipeline acquired no read buffer")
+		}
+		for index, buffer := range buffers {
+			if refs := atomic.LoadInt32(&buffer.refs); refs != 0 {
+				t.Errorf("read buffer %d still holds %d references after the pipeline stopped", index, refs)
+			}
+		}
+	})
+
+	chunkPool.New = func() interface{} {
+		buffer := &chunkBuffer{data: make([]byte, batchSize)}
+		lock.Lock()
+		buffers = append(buffers, buffer)
+		lock.Unlock()
+		return buffer
+	}
+}
+
+// runWriteJob drives one prepared write Job through the copy and cleanup stages.
+func runWriteJob(copyer *Copyer, job *writeJob) error {
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	prepared := make(chan *writeJob, 1)
+	prepared <- job
+	close(prepared)
+
+	copyed := copyer.copy(ctx, prepared)
+	copyer.cleanup(ctx, copyed)
+	for range copyed {
+	}
+	return copyer.WaitErr()
+}
+
+func TestRunCopiesEmptyFile(t *testing.T) {
 	tests := []struct {
 		name string
 		opts []Option
 	}{
-		{name: "mmap"},
-		{name: "linear", opts: []Option{SetFromDevice(LinearDevice(true))}},
+		{name: "buffered source"},
+		{name: "mapped source", opts: []Option{SetFromDevice(WithReadMode(ReadMapped))}},
+		{name: "linear source", opts: []Option{SetFromDevice(LinearDevice(true))}},
 	}
 
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			dir := t.TempDir()
-			src := filepath.Join(dir, "src")
-			dst := filepath.Join(dir, "dst")
-			if err := os.WriteFile(src, nil, 0o644); err != nil {
-				t.Fatalf("write src: %v", err)
-			}
+			root := t.TempDir()
+			source := writeSourceFile(t, root, "src", nil)
+			target := filepath.Join(root, "dst")
+			item := newFixtureItem(source, target)
 
 			handler, getter := NewReportGetter()
-			opts := append([]Option{
-				AccurateJob(src, []string{dst}),
-				Overwrite(true),
-				WithEventHandler(handler),
-			}, tt.opts...)
-			copyer, err := New(context.Background(), opts...)
-			if err != nil {
-				t.Fatalf("new copyer: %v", err)
+			opts := append([]Option{WithEventHandler(handler)}, tt.opts...)
+			if err := Run(context.Background(), newSliceSource(item), opts...); err != nil {
+				t.Fatalf("run: %v", err)
 			}
 
-			done := make(chan struct{})
-			go func() {
-				copyer.Wait()
-				close(done)
-			}()
-			select {
-			case <-done:
-			case <-time.After(5 * time.Second):
-				t.Fatalf("copy empty file timed out")
-			}
-
-			info, err := os.Stat(dst)
+			info, err := os.Stat(target)
 			if err != nil {
 				t.Fatalf("stat dst: %v", err)
 			}
@@ -89,6 +149,17 @@ func TestCopyEmptyFile(t *testing.T) {
 				t.Fatalf("dst size = %d", info.Size())
 			}
 
+			result, err := item.terminal(t)
+			if err != nil {
+				t.Fatalf("item failed: %v", err)
+			}
+			if len(result.Targets) != 1 || result.Targets[0].Err != nil || result.Targets[0].Path != target {
+				t.Fatalf("targets = %v", result.Targets)
+			}
+			if result.Size != 0 {
+				t.Fatalf("result size = %d, want 0", result.Size)
+			}
+
 			report := getter()
 			if len(report.Errors) != 0 {
 				t.Fatalf("report errors = %v", report.Errors)
@@ -100,7 +171,7 @@ func TestCopyEmptyFile(t *testing.T) {
 			if job.Status != JobStatusFinished {
 				t.Fatalf("job status = %q", job.Status)
 			}
-			if len(job.SuccessTargets) != 1 || job.SuccessTargets[0] != dst {
+			if len(job.SuccessTargets) != 1 || job.SuccessTargets[0] != target {
 				t.Fatalf("success targets = %v", job.SuccessTargets)
 			}
 		})
@@ -109,7 +180,7 @@ func TestCopyEmptyFile(t *testing.T) {
 
 func TestWritePublishesFinishingJob(t *testing.T) {
 	// Build a target-free write Job so only the worker-to-cleanup handoff is exercised.
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 8)}
+	copyer := newTestCopyer(t)
 	job := newWriteJob(&baseJob{
 		copyer: copyer,
 		src:    &source{},
@@ -124,27 +195,36 @@ func TestWritePublishesFinishingJob(t *testing.T) {
 	}
 }
 
-func TestHashOnlyReadFailureFailsJob(t *testing.T) {
-	// Build a target-free hash Job whose source fails on its first read.
+func TestWriteReportsTargetlessReadFailureAsItemFailure(t *testing.T) {
+	// Build a target-free hash item whose source fails on its first read.
 	readErr := errors.New("read failed")
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 8)}
-	copyer.withHash = true
+	copyer := newTestCopyer(t, WithHashPolicy(HashRead))
+	item := newFixtureItem("source")
 	job := newWriteJob(&baseJob{
 		copyer: copyer,
+		item:   item,
 		src:    &source{},
 		path:   "source",
 		stat:   &stat{size: 1},
 	}, &failingReadCloser{err: readErr}, 1, false)
 	completed := make(chan *baseJob, 1)
 
-	// The source error must cross both the Job result and synchronous error boundary.
+	// An item with no target outcome to report could not be processed at all.
 	copyer.write(context.Background(), job, completed, new(counter), mapset.NewSet[string]())
-	report := (<-completed).report()
-	if !errors.Is(report.FailTargets[""], readErr) {
-		t.Fatalf("hash-only failure = %v, want %v", report.FailTargets[""], readErr)
+	if err := (<-completed).itemError; !errors.Is(err, readErr) {
+		t.Fatalf("item error = %v, want %v", err, readErr)
 	}
-	if err := copyer.WaitErr(); !errors.Is(err, readErr) {
-		t.Fatalf("WaitErr() = %v, want %v", err, readErr)
+
+	// Cleanup must route that item to its failure callback instead of completing it.
+	copyed := make(chan *baseJob, 1)
+	copyed <- job.baseJob
+	close(copyed)
+	copyer.cleanup(context.Background(), copyed)
+	if result, err := item.terminal(t); !errors.Is(err, readErr) {
+		t.Fatalf("terminal outcome = %#v / %v, want failure %v", result, err, readErr)
+	}
+	if err := copyer.WaitErr(); err != nil {
+		t.Fatalf("WaitErr() = %v, want nil: a failed item is an item outcome", err)
 	}
 }
 
@@ -164,10 +244,10 @@ func TestCopyClosesPreparedSourcesAfterCancellation(t *testing.T) {
 	// Queue one prefetched reader before starting an already-canceled Copy stage.
 	ctx, cancel := context.WithCancel(context.Background())
 	cancel()
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 1)}
+	copyer := newTestCopyer(t)
 	copyer.toDevice.threads = 1
 	reader := new(trackingReadCloser)
-	job := newWriteJob(nil, reader, 0, true)
+	job := newWriteJob(&baseJob{copyer: copyer, item: newFixtureItem("source")}, reader, 0, true)
 	prepared := make(chan *writeJob, 1)
 	prepared <- job
 	close(prepared)
@@ -181,30 +261,43 @@ func TestCopyClosesPreparedSourcesAfterCancellation(t *testing.T) {
 	if !job.waitConsumed(context.Background()) {
 		t.Fatal("linear source was not notified that the reader was consumed")
 	}
+
+	// The abandoned item is handed to cleanup with the stopping error.
+	select {
+	case abandoned := <-copyer.abandoned:
+		if abandoned != job.baseJob {
+			t.Fatalf("abandoned job = %p, want %p", abandoned, job.baseJob)
+		}
+		if !errors.Is(abandoned.itemError, context.Canceled) {
+			t.Fatalf("abandoned item error = %v, want %v", abandoned.itemError, context.Canceled)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("copy did not report the prefetched item")
+	}
 }
 
-func TestWriteReturnsWhenCanceledBeforePublishing(t *testing.T) {
+func TestWriteReturnsAfterHardStopWithoutPublishing(t *testing.T) {
 	// Use an unbuffered completion channel with no receiver to expose a blocked handoff.
-	ctx, cancel := context.WithCancel(context.Background())
-	cancel()
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 8)}
+	copyer := newTestCopyer(t)
+	copyer.stopHard()
 	reader := new(trackingReadCloser)
 	job := newWriteJob(&baseJob{
 		copyer: copyer,
+		item:   newFixtureItem("source"),
 		src:    &source{},
 		stat:   &stat{},
 	}, reader, 0, false)
 	done := make(chan struct{})
 	go func() {
-		copyer.write(ctx, job, make(chan *baseJob), new(counter), mapset.NewSet[string]())
+		copyer.write(context.Background(), job, make(chan *baseJob), new(counter), mapset.NewSet[string]())
 		close(done)
 	}()
 
-	// Cancellation must skip the completion handoff while retaining source cleanup.
+	// A pipeline failure skips the completion handoff while retaining source cleanup.
 	select {
 	case <-done:
 	case <-time.After(time.Second):
-		t.Fatal("write did not return after cancellation")
+		t.Fatal("write did not return after the pipeline stopped")
 	}
 	if reader.closed != 1 {
 		t.Fatalf("reader closed %d times, want 1", reader.closed)
@@ -213,32 +306,94 @@ func TestWriteReturnsWhenCanceledBeforePublishing(t *testing.T) {
 
 func TestBaseJobFailureMovesSuccessfulTarget(t *testing.T) {
 	// Seed a completed target before applying a metadata-stage failure.
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 2)}
+	copyer := newTestCopyer(t)
+	item := newFixtureItem("source", "target")
 	job := &baseJob{
-		copyer: copyer, src: &source{}, stat: &stat{}, successTargets: []string{"target"},
+		copyer: copyer, item: item, src: &source{}, stat: &stat{},
+		targets: []string{"target"}, successTargets: []string{"target"},
 	}
 
 	// A late target failure must remove the target from the successful result.
 	job.fail("target", syscall.ENOSPC)
-	report := job.report()
-	if len(report.SuccessTargets) != 0 {
+	result := job.result()
+	if len(result.Targets) != 1 || result.Targets[0].Err == nil {
+		t.Fatalf("targets = %#v, want one failed outcome", result.Targets)
+	}
+	if report := job.report(); len(report.SuccessTargets) != 0 {
 		t.Fatalf("success targets = %v, want none", report.SuccessTargets)
 	}
-	if !errors.Is(report.FailTargets["target"], syscall.ENOSPC) {
-		t.Fatalf("target failure = %v, want %v", report.FailTargets["target"], syscall.ENOSPC)
+	if !errors.Is(result.Targets[0].Err, syscall.ENOSPC) {
+		t.Fatalf("target failure = %v, want %v", result.Targets[0].Err, syscall.ENOSPC)
 	}
 }
 
-func TestFirstTargetFailureRemainsAuthoritative(t *testing.T) {
-	// Record a mapped write failure before the best-effort cleanup error.
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 4)}
-	job := &baseJob{copyer: copyer, src: &source{}, stat: &stat{}}
+func TestTargetFailureKeepsItsIdentityAsItemOutcome(t *testing.T) {
+	// Record a mapped write failure before a later, unrelated pipeline error.
+	copyer := newTestCopyer(t)
+	item := newFixtureItem("source", "target")
+	job := &baseJob{
+		copyer: copyer, item: item, src: &source{}, stat: &stat{}, targets: []string{"target"},
+	}
 	job.fail("target", mappingError(syscall.ENOSPC))
 	copyer.setError(errors.New("remove failed"))
 
-	// Secondary cleanup failures must not hide the no-space classification.
-	if err := copyer.WaitErr(); !errors.Is(err, ErrTargetNoSpace) {
-		t.Fatalf("WaitErr() = %v, want %v", err, ErrTargetNoSpace)
+	// The item outcome keeps the no-space classification the caller classifies on.
+	result := job.result()
+	if len(result.Targets) != 1 || !errors.Is(result.Targets[0].Err, ErrTargetNoSpace) {
+		t.Fatalf("target outcome = %#v, want %v", result.Targets, ErrTargetNoSpace)
+	}
+}
+
+func TestWriteFailureDrainsBuffersAndTargets(t *testing.T) {
+	// Fail the source read after a whole batch, so every target writer runs its failure
+	// drain while the pipeline still holds the failed item's buffers.
+	trackChunkPool(t)
+
+	root := t.TempDir()
+	readErr := errors.New("read failed")
+	targets := []string{filepath.Join(root, "target-1"), filepath.Join(root, "target-2")}
+	item := newFixtureItem(filepath.Join(root, "source"), targets...)
+	copyer := newTestCopyer(t, WithHashPolicy(HashRead))
+	job := newWriteJob(&baseJob{
+		copyer:  copyer,
+		item:    item,
+		src:     &source{base: root, path: "source"},
+		path:    filepath.Join(root, "source"),
+		stat:    &stat{size: batchSize, mode: 0o644},
+		targets: targets,
+	}, &failingContentReader{err: readErr, batches: 1}, batchSize, false)
+
+	// The pipeline must drain every writer, report exactly one outcome, and return.
+	done := make(chan error, 1)
+	go func() {
+		done <- runWriteJob(copyer, job)
+	}()
+	select {
+	case err := <-done:
+		if err != nil {
+			t.Fatalf("pipeline failed: %v", err)
+		}
+	case <-time.After(10 * time.Second):
+		t.Fatal("the pipeline deadlocked while draining failed target writers")
+	}
+
+	result, err := item.terminal(t)
+	if err != nil {
+		t.Fatalf("an item whose targets failed must still complete: %v", err)
+	}
+	if len(result.SHA256) != 0 {
+		t.Fatalf("SHA256 = %x, want none: the read stopped before the source ended", result.SHA256)
+	}
+	if len(result.Targets) != 2 {
+		t.Fatalf("targets = %#v, want one outcome per requested target", result.Targets)
+	}
+	for _, outcome := range result.Targets {
+		if !errors.Is(outcome.Err, readErr) {
+			t.Fatalf("target %q error = %v, want %v", outcome.Path, outcome.Err, readErr)
+		}
+		if _, statErr := os.Stat(outcome.Path); !errors.Is(statErr, os.ErrNotExist) {
+			t.Fatalf("failed target %q still exists: %v", outcome.Path, statErr)
+		}
 	}
 }
 
@@ -254,22 +409,28 @@ func TestLinearTargetStopsWhenDiskUsageEstimateIsInsufficient(t *testing.T) {
 		t.Fatal("available disk space cannot be represented by the test Job size")
 	}
 	size := usage.Available() + defaultDiskUsageFreshInterval
+	option := newOption()
+	if err := option.check(); err != nil {
+		t.Fatal(err)
+	}
 	copyer := &Copyer{
-		option: newOption(), eventCh: make(chan Event, 8),
+		option: option, eventCh: make(chan Event, 8), hardStop: make(chan struct{}),
 		getDevice:         func(string) string { return root },
 		getDiskUsageCache: func(string) *diskUsageCache { return newDiskUsageCache(root, defaultDiskUsageFreshInterval) },
 	}
 	copyer.toDevice.linear = true
+	item := newFixtureItem("source", target)
 	job := newWriteJob(&baseJob{
-		copyer: copyer, src: &source{}, path: "source", stat: &stat{size: size}, targets: []string{target},
+		copyer: copyer, item: item, src: &source{}, path: "source",
+		stat: &stat{size: size}, targets: []string{target},
 	}, io.NopCloser(bytes.NewReader(nil)), size, false)
 	completed := make(chan *baseJob, 1)
 
 	// The hardware-backed estimate must stop a linear target before the oversized write starts.
 	copyer.write(context.Background(), job, completed, new(counter), mapset.NewSet[string]())
-	report := (<-completed).report()
-	if !errors.Is(report.FailTargets[target], ErrTargetNoSpace) {
-		t.Fatalf("target failure = %v, want %v", report.FailTargets[target], ErrTargetNoSpace)
+	result := (<-completed).result()
+	if len(result.Targets) != 1 || !errors.Is(result.Targets[0].Err, ErrTargetNoSpace) {
+		t.Fatalf("target outcome = %#v, want %v", result.Targets, ErrTargetNoSpace)
 	}
 	if !copyer.linearTargetStopped() {
 		t.Fatal("linear target continued after the capacity estimate was exhausted")
@@ -279,18 +440,47 @@ func TestLinearTargetStopsWhenDiskUsageEstimateIsInsufficient(t *testing.T) {
 	}
 }
 
-func TestStoppedLinearTargetDoesNotReadStreamSource(t *testing.T) {
-	// Mark a linear target exhausted before its stream indexer requests more work.
-	source := new(sliceStreamSource)
-	copyer := &Copyer{option: newOption(), eventCh: make(chan Event, 2)}
-	copyer.streamSource = source
-	copyer.toDevice.linear = true
+func TestStoppedLinearTargetDoesNotReadBatchSource(t *testing.T) {
+	// Mark a linear target exhausted before its batch indexer requests more work.
+	source := newSliceSource()
+	copyer := newTestCopyer(t, SetToDevice(LinearDevice(true)))
+	copyer.batch = source
 	copyer.endLinearTarget(ErrTargetNoSpace)
 
-	// A stopped target closes the index stream without consuming another request.
-	for range copyer.indexStream(context.Background()) {
+	// A stopped target closes the index without consuming another batch.
+	indexed, err := copyer.index(context.Background())
+	if err != nil {
+		t.Fatal(err)
 	}
-	if source.index != 0 {
-		t.Fatalf("source requests = %d, want 0", source.index)
+	for range indexed {
+	}
+	if calls := source.calls(); calls != 0 {
+		t.Fatalf("source calls = %d, want 0", calls)
+	}
+}
+
+func TestIndexReportsBatchSourceFailureAndYieldsAcceptedItems(t *testing.T) {
+	// A source error is a pipeline failure, and items accepted before it stay owned.
+	sourceErr := errors.New("source failed")
+	copyer := newTestCopyer(t)
+	item := newFixtureItem("source")
+	copyer.batch = &sliceSource{batches: [][]Item{{item}}, err: sourceErr}
+
+	indexed, err := copyer.index(context.Background())
+	if err != nil {
+		t.Fatal(err)
+	}
+	accepted := 0
+	for job := range indexed {
+		accepted++
+		if job.item != item {
+			t.Fatalf("indexed item = %v, want the submitted item", job.item)
+		}
+	}
+	if accepted != 1 {
+		t.Fatalf("indexed items = %d, want 1", accepted)
+	}
+	if err := copyer.WaitErr(); !errors.Is(err, sourceErr) {
+		t.Fatalf("WaitErr() = %v, want %v", err, sourceErr)
 	}
 }

files.go

@@ -0,0 +1,123 @@
+package acp
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+
+	"github.com/sirupsen/logrus"
+)
+
+// FileEntry describes one source file and the targets its source-relative path maps to.
+type FileEntry struct {
+	// Base is the directory the relative path is resolved against.
+	Base string
+	// Path is the source path relative to Base.
+	Path string
+	// Source is the resolved source file path.
+	Source string
+	// Targets holds one resolved target path per requested target directory.
+	Targets []string
+}
+
+// SelectFiles walks every source, keeps regular files, maps each source-relative path
+// onto every target directory, and returns the entries in physical order. A source may
+// be a directory or a regular file, and every target must exist and be a directory.
+//
+// The result is what the acp command line copies: repeated relative paths are removed so
+// one source tree cannot overwrite another's file, and the order follows the platform's
+// path ordering so a linear target receives files in the order the medium stores them.
+func SelectFiles(sources, targets []string) ([]FileEntry, error) {
+	dsts := make([]string, 0, len(targets))
+	for _, target := range targets {
+		target = filepath.Clean(target)
+		if target == "" {
+			continue
+		}
+
+		info, err := os.Stat(target)
+		if err != nil {
+			return nil, fmt.Errorf("check dst path '%s', %w", target, err)
+		}
+		if !info.IsDir() {
+			return nil, fmt.Errorf("dst path is not a dir, path= '%s'", target)
+		}
+
+		dsts = append(dsts, target)
+	}
+	if len(sources) == 0 {
+		return nil, fmt.Errorf("source path not found")
+	}
+
+	roots := make([]*source, 0, len(sources))
+	for _, path := range sources {
+		path = filepath.Clean(path)
+		base, name := filepath.Split(path)
+		roots = append(roots, &source{base: base, path: name})
+	}
+	sort.Slice(roots, func(i, j int) bool {
+		return comparePath(roots[i].path, roots[j].path) < 0
+	})
+	for _, root := range roots {
+		if _, err := os.Stat(root.src()); err != nil {
+			return nil, fmt.Errorf("check src path '%s', %w", root.src(), err)
+		}
+	}
+
+	entries := make([]FileEntry, 0, 64)
+	var walk func(src *source)
+	walk = func(src *source) {
+		path := src.src()
+
+		info, err := os.Stat(path)
+		if err != nil {
+			logrus.WithError(err).Errorf("walk get stat, path= '%s'", path)
+			return
+		}
+
+		mode := info.Mode()
+		if mode.IsRegular() {
+			targets := make([]string, 0, len(dsts))
+			for _, dst := range dsts {
+				targets = append(targets, src.dst(dst))
+			}
+			entries = append(entries, FileEntry{Base: src.base, Path: src.path, Source: path, Targets: targets})
+			return
+		}
+		if mode&UnexpectFileMode != 0 {
+			return
+		}
+
+		files, err := os.ReadDir(path)
+		if err != nil {
+			logrus.WithError(err).Errorf("walk read dir, path= '%s'", path)
+			return
+		}
+		for _, file := range files {
+			walk(src.append(file.Name()))
+		}
+	}
+	for _, root := range roots {
+		walk(root)
+	}
+
+	sort.Slice(entries, func(i, j int) bool {
+		return comparePath(entries[i].Path, entries[j].Path) < 0
+	})
+
+	// Repeated relative paths would write the same target twice, so keep the first.
+	filtered := entries[:0]
+	previous := ""
+	for index, entry := range entries {
+		if index > 0 && entry.Path == previous {
+			logrus.Errorf("same relative path, ignored, path= '%s'", entry.Source)
+			continue
+		}
+
+		filtered = append(filtered, entry)
+		previous = entry.Path
+	}
+
+	return filtered, nil
+}

files_test.go

@@ -0,0 +1,141 @@
+package acp
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+func TestSelectFilesMapsSourcesOntoTargets(t *testing.T) {
+	root := t.TempDir()
+	source := filepath.Join(root, "source")
+	writeSourceFile(t, source, "plain.txt", []byte("plain"))
+	writeSourceFile(t, source, "empty.txt", nil)
+	writeSourceFile(t, source, filepath.Join("nested", "data"), []byte("nested"))
+	if err := os.MkdirAll(filepath.Join(source, "emptydir"), 0o755); err != nil {
+		t.Fatal(err)
+	}
+
+	targets := []string{filepath.Join(root, "target-a"), filepath.Join(root, "target-b")}
+	for _, target := range targets {
+		if err := os.MkdirAll(target, 0o755); err != nil {
+			t.Fatal(err)
+		}
+	}
+
+	entries, err := SelectFiles([]string{source}, targets)
+	if err != nil {
+		t.Fatalf("select files: %v", err)
+	}
+
+	// Directories are not entries, and the platform's path order covers the result.
+	want := []string{
+		"empty.txt",
+		filepath.Join("nested", "data"),
+		"plain.txt",
+	}
+	if len(entries) != len(want) {
+		t.Fatalf("entries = %d, want %d: %+v", len(entries), len(want), entries)
+	}
+	for index, entry := range entries {
+		relative := filepath.Join("source", want[index])
+		if entry.Path != relative {
+			t.Fatalf("entry %d path = %q, want %q", index, entry.Path, relative)
+		}
+		if base, _ := filepath.Split(source); entry.Base != base {
+			t.Fatalf("entry %d base = %q, want %q", index, entry.Base, base)
+		}
+		if entry.Source != filepath.Join(source, want[index]) {
+			t.Fatalf("entry %d source = %q", index, entry.Source)
+		}
+		if len(entry.Targets) != len(targets) {
+			t.Fatalf("entry %d targets = %v, want one per target directory", index, entry.Targets)
+		}
+		for targetIndex, target := range targets {
+			if got, wantTarget := entry.Targets[targetIndex], filepath.Join(target, relative); got != wantTarget {
+				t.Fatalf("entry %d target %d = %q, want %q", index, targetIndex, got, wantTarget)
+			}
+		}
+	}
+}
+
+func TestSelectFilesMapsASingleFileOntoTargets(t *testing.T) {
+	root := t.TempDir()
+	source := writeSourceFile(t, root, "plain.txt", []byte("plain"))
+	target := filepath.Join(root, "target")
+	if err := os.MkdirAll(target, 0o755); err != nil {
+		t.Fatal(err)
+	}
+
+	entries, err := SelectFiles([]string{source}, []string{target})
+	if err != nil {
+		t.Fatalf("select files: %v", err)
+	}
+	if len(entries) != 1 {
+		t.Fatalf("entries = %+v, want one", entries)
+	}
+	entry := entries[0]
+	base, _ := filepath.Split(source)
+	if entry.Source != source || entry.Path != "plain.txt" || entry.Base != base {
+		t.Fatalf("entry = %+v", entry)
+	}
+	if len(entry.Targets) != 1 || entry.Targets[0] != filepath.Join(target, "plain.txt") {
+		t.Fatalf("targets = %v", entry.Targets)
+	}
+}
+
+func TestSelectFilesRemovesRepeatedRelativePaths(t *testing.T) {
+	// Two source trees with the same name would write the same target twice, so the
+	// caller receives one entry per relative path.
+	root := t.TempDir()
+	first := filepath.Join(root, "first", "tree")
+	second := filepath.Join(root, "second", "tree")
+	writeSourceFile(t, first, "data.txt", []byte("first"))
+	writeSourceFile(t, second, "data.txt", []byte("second"))
+	target := filepath.Join(root, "target")
+	if err := os.MkdirAll(target, 0o755); err != nil {
+		t.Fatal(err)
+	}
+
+	entries, err := SelectFiles([]string{second, first}, []string{target})
+	if err != nil {
+		t.Fatalf("select files: %v", err)
+	}
+	if len(entries) != 1 {
+		t.Fatalf("entries = %+v, want one entry per relative path", entries)
+	}
+	entry := entries[0]
+	if entry.Path != filepath.Join("tree", "data.txt") {
+		t.Fatalf("entry path = %q", entry.Path)
+	}
+	if entry.Source != filepath.Join(first, "data.txt") && entry.Source != filepath.Join(second, "data.txt") {
+		t.Fatalf("entry source = %q, want one of the submitted trees", entry.Source)
+	}
+}
+
+func TestSelectFilesRejectsInvalidInput(t *testing.T) {
+	root := t.TempDir()
+	target := filepath.Join(root, "target")
+	if err := os.MkdirAll(target, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	file := writeSourceFile(t, root, "plain.txt", []byte("plain"))
+
+	tests := []struct {
+		name    string
+		sources []string
+		targets []string
+	}{
+		{name: "no source", sources: nil, targets: []string{target}},
+		{name: "missing source", sources: []string{filepath.Join(root, "missing")}, targets: []string{target}},
+		{name: "missing target", sources: []string{file}, targets: []string{filepath.Join(root, "missing-target")}},
+		{name: "target is a file", sources: []string{file}, targets: []string{file}},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			if _, err := SelectFiles(test.sources, test.targets); err == nil {
+				t.Fatal("SelectFiles() error = nil")
+			}
+		})
+	}
+}

fs.go

@@ -1,7 +1,10 @@
 package acp
 
 import (
+	"errors"
 	"fmt"
+	"io/fs"
+	"os"
 	"path/filepath"
 	"strings"
 
@@ -38,6 +41,17 @@ func getMountpointCache() (func(string) string, error) {
 	}), nil
 }
 
+// openSource opens a source for buffered reading without updating its access time.
+// O_NOATIME needs ownership or privilege, so a refused open falls back to an ordinary one.
+func openSource(path string) (*os.File, error) {
+	file, err := os.OpenFile(path, os.O_RDONLY|openNoAtime, 0)
+	if err == nil || !errors.Is(err, fs.ErrPermission) {
+		return file, err
+	}
+
+	return os.Open(path)
+}
+
 func findMountpoint(path string, mountPoints []string) string {
 	matched := ""
 	for _, mountPoint := range mountPoints {

full_linux_test.go

@@ -8,29 +8,80 @@ import (
 	"os"
 	"path/filepath"
 	"testing"
+	"time"
 )
 
-func TestRunStreamMapsDeviceFullToTargetNoSpace(t *testing.T) {
+func TestRunMapsDeviceFullToTargetNoSpace(t *testing.T) {
 	// Route a disposable target symlink to Linux's deterministic ENOSPC device.
 	root := t.TempDir()
-	input := filepath.Join(root, "source")
+	input := writeSourceFile(t, root, "source", []byte("fixture"))
 	target := filepath.Join(root, "target")
-	if err := os.WriteFile(input, []byte("fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
 	if err := os.Symlink("/dev/full", target); err != nil {
 		t.Fatal(err)
 	}
-	source := &sliceStreamSource{requests: []*StreamRequest{{
-		ID: 1, Source: input, Targets: []string{target},
-	}}}
+	item := newFixtureItem(input, target)
 
-	// The synchronous stream boundary must preserve the portable no-space sentinel.
-	err := RunStream(
-		context.Background(), source, new(collectingStreamSink),
-		Overwrite(true), SetToDevice(LinearDevice(true)),
+	// A target failure is an item outcome that keeps the portable no-space sentinel.
+	err := Run(
+		context.Background(),
+		newSliceSource(item),
+		SetToDevice(Overwrite(true), LinearDevice(true)),
 	)
-	if !errors.Is(err, ErrTargetNoSpace) {
-		t.Fatalf("RunStream() error = %v, want %v", err, ErrTargetNoSpace)
+	if err != nil {
+		t.Fatalf("Run() error = %v, want nil", err)
+	}
+	result, terminalErr := item.terminal(t)
+	if terminalErr != nil {
+		t.Fatalf("item failed: %v", terminalErr)
+	}
+	if len(result.Targets) != 1 || !errors.Is(result.Targets[0].Err, ErrTargetNoSpace) {
+		t.Fatalf("target outcome = %#v, want %v", result.Targets, ErrTargetNoSpace)
+	}
+}
+
+func TestDeviceFullWriteFailureDrainsQueuedBuffers(t *testing.T) {
+	// A full device fails the first write while the producer still feeds the writer, which
+	// is the path that must release every queued read buffer.
+	trackChunkPool(t)
+
+	root := t.TempDir()
+	target := filepath.Join(root, "target")
+	if err := os.Symlink("/dev/full", target); err != nil {
+		t.Fatal(err)
+	}
+
+	size := int64(batchSize) * 4
+	readErr := errors.New("read failed")
+	item := newFixtureItem(filepath.Join(root, "source"), target)
+	copyer := newTestCopyer(t, SetToDevice(Overwrite(true)))
+	job := newWriteJob(&baseJob{
+		copyer:  copyer,
+		item:    item,
+		src:     &source{base: root, path: "source"},
+		path:    filepath.Join(root, "source"),
+		stat:    &stat{size: size, mode: 0o644},
+		targets: []string{target},
+	}, &failingContentReader{err: readErr, batches: 4}, size, false)
+
+	// The writer must fail, release its queued buffers, and let the pipeline return.
+	done := make(chan error, 1)
+	go func() {
+		done <- runWriteJob(copyer, job)
+	}()
+	select {
+	case err := <-done:
+		if err != nil {
+			t.Fatalf("pipeline failed: %v", err)
+		}
+	case <-time.After(10 * time.Second):
+		t.Fatal("the pipeline deadlocked while draining a full target")
+	}
+
+	result, terminalErr := item.terminal(t)
+	if terminalErr != nil {
+		t.Fatalf("an item whose targets failed must still complete: %v", terminalErr)
+	}
+	if len(result.Targets) != 1 || !errors.Is(result.Targets[0].Err, ErrTargetNoSpace) {
+		t.Fatalf("target outcome = %#v, want %v", result.Targets, ErrTargetNoSpace)
 	}
 }

hash_policy.go

@@ -0,0 +1,84 @@
+package acp
+
+import "fmt"
+
+// HashPolicy selects how an item's content hash is produced and how the stored hash
+// cache is used. Hashing and the cache are one policy, because refreshing the cache
+// requires a computed hash: a miss under a refresh-enabled value must read.
+type HashPolicy uint8
+
+const (
+	// HashOff produces no hash and leaves the cache untouched.
+	HashOff HashPolicy = iota
+	// HashCachedOnly reuses a stored hash; a miss leaves the item without one.
+	HashCachedOnly
+	// HashCachedOrRead reuses a stored hash; a miss reads and hashes without refreshing.
+	HashCachedOrRead
+	// HashCachedOrReadRefresh reuses a stored hash; a miss reads, hashes and refreshes.
+	HashCachedOrReadRefresh
+	// HashRead always reads and hashes; the cache is not refreshed.
+	HashRead
+	// HashReadRefresh always reads and hashes, and refreshes the cache.
+	HashReadRefresh
+)
+
+func (p HashPolicy) String() string {
+	switch p {
+	case HashOff:
+		return "off"
+	case HashCachedOnly:
+		return "cached-only"
+	case HashCachedOrRead:
+		return "cached-or-read"
+	case HashCachedOrReadRefresh:
+		return "cached-or-read-refresh"
+	case HashRead:
+		return "read"
+	case HashReadRefresh:
+		return "read-refresh"
+	}
+	return fmt.Sprintf("unknown(%d)", uint8(p))
+}
+
+// valid reports whether the value is one of the defined policies.
+func (p HashPolicy) valid() bool {
+	return p <= HashReadRefresh
+}
+
+// readsContent reports whether the policy needs the source content.
+func (p HashPolicy) readsContent() bool {
+	switch p {
+	case HashCachedOrRead, HashCachedOrReadRefresh, HashRead, HashReadRefresh:
+		return true
+	}
+	return false
+}
+
+// producesHash reports whether a read item receives a content hash.
+func (p HashPolicy) producesHash() bool {
+	return p != HashOff
+}
+
+// usesCache reports whether stored hashes are read at all.
+func (p HashPolicy) usesCache() bool {
+	return p != HashOff
+}
+
+// reusesCache reports whether a stored hash may replace the hash of an item that does not
+// write its source.
+func (p HashPolicy) reusesCache() bool {
+	switch p {
+	case HashCachedOnly, HashCachedOrRead, HashCachedOrReadRefresh:
+		return true
+	}
+	return false
+}
+
+// refreshesCache reports whether a computed hash is written back to the cache.
+func (p HashPolicy) refreshesCache() bool {
+	switch p {
+	case HashCachedOrReadRefresh, HashReadRefresh:
+		return true
+	}
+	return false
+}

index.go

@@ -2,14 +2,11 @@ package acp
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"io"
 	"os"
 	"path/filepath"
-	"sort"
-	"sync"
-	"sync/atomic"
-	"time"
 )
 
 const (
@@ -20,35 +17,16 @@ type counter struct {
 	bytes, files int64
 }
 
+// index reads caller-owned items in batches and turns them into jobs. It stops asking the
+// batch source for work as soon as the pipeline stops, and reports the items of the batch
+// it already holds rather than dropping them.
 func (c *Copyer) index(ctx context.Context) (<-chan *baseJob, error) {
-	if c.streamSource != nil {
-		return c.indexStream(ctx), nil
+	if c.batch == nil {
+		return nil, fmt.Errorf("batch source is nil")
 	}
 
-	jobs, err := c.walk(ctx)
-	if err != nil {
-		return nil, err
-	}
-
-	ch := make(chan *baseJob, 128)
-	go wrap(ctx, func() {
-		defer close(ch)
-
-		for order, job := range jobs {
-			job.order = uint64(order)
-			select {
-			case <-ctx.Done():
-				return
-			case ch <- job:
-			}
-		}
-	})
-
-	return ch, nil
-}
-
-func (c *Copyer) indexStream(ctx context.Context) <-chan *baseJob {
-	ch := make(chan *baseJob, 128)
+	// The channel capacity is the read buffer, so no separate prefetch stage is needed.
+	ch := make(chan *baseJob, c.readBuffer)
 	go wrap(ctx, func() {
 		defer close(ch)
 
@@ -57,217 +35,104 @@ func (c *Copyer) indexStream(ctx context.Context) <-chan *baseJob {
 		defer func() {
 			c.submit(&EventUpdateCount{Bytes: bytes, Files: files, Finished: true})
 		}()
+
 		for {
-			if c.linearTargetStopped() {
+			// A stop ends the feed without asking the caller for another batch.
+			if c.stopped(ctx) {
 				return
 			}
-			request, err := c.streamSource.Next(ctx)
+
+			batch, err := c.batch.Next(ctx)
 			if err != nil {
-				if err != io.EOF {
-					c.reportError("", "", fmt.Errorf("read stream source failed, %w", err))
+				if !errors.Is(err, io.EOF) {
+					err = fmt.Errorf("read batch source failed, %w", err)
+					c.reportError("", "", err)
+					c.setError(err)
 				}
 				return
 			}
-			if request == nil {
-				c.reportError("", "", fmt.Errorf("read stream source failed, request is nil"))
-				return
-			}
 
-			sourcePath := filepath.Clean(request.Source)
-			info, err := os.Stat(sourcePath)
-			if err != nil {
-				c.reportError(sourcePath, "", fmt.Errorf("stream job get stat failed, %w", err))
-				return
-			}
-			if !info.Mode().IsRegular() {
-				c.reportError(sourcePath, "", fmt.Errorf("stream job source is not a regular file"))
-				return
-			}
-			stat, err := newStat(sourcePath, info)
-			if err != nil {
-				c.reportError(sourcePath, "", fmt.Errorf("read stream job stat failed, %w", err))
-				return
-			}
-
-			job := &baseJob{
-				copyer:   c,
-				src:      &source{base: filepath.Dir(sourcePath), path: filepath.Base(sourcePath)},
-				path:     sourcePath,
-				stat:     stat,
-				targets:  append([]string(nil), request.Targets...),
-				streamID: request.ID,
-				order:    order,
-			}
-			c.submit(&EventUpdateJob{job.report()})
-			bytes += stat.size
-			files++
-
-			select {
-			case <-ctx.Done():
-				c.setError(ctx.Err())
-				return
-			case ch <- job:
+			for index, item := range batch {
+				if !c.accept(ch, item, order, &bytes, &files) {
+					c.abandonItems(ctx, batch[index:])
+					return
+				}
 				order++
 			}
+			c.submit(&EventUpdateCount{Bytes: bytes, Files: files})
 		}
 	})
-	return ch
+	return ch, nil
 }
 
-func (c *Copyer) walk(ctx context.Context) ([]*baseJob, error) {
-	done := make(chan struct{})
-	var reporting sync.WaitGroup
-	reporting.Add(1)
-	defer func() {
-		close(done)
-		reporting.Wait()
-	}()
-
-	cntr := new(counter)
-	go wrap(ctx, func() {
-		defer reporting.Done()
-		ticker := time.NewTicker(time.Second)
-		defer ticker.Stop()
-		for {
-			select {
-			case <-ticker.C:
-				c.submit(&EventUpdateCount{Bytes: atomic.LoadInt64(&cntr.bytes), Files: atomic.LoadInt64(&cntr.files)})
-			case <-done:
-				c.submit(&EventUpdateCount{Bytes: atomic.LoadInt64(&cntr.bytes), Files: atomic.LoadInt64(&cntr.files), Finished: true})
-				return
-			}
-		}
-	})
-
-	jobs := make([]*baseJob, 0, 64)
-	appendJob := func(job *baseJob) {
-		if !job.stat.mode.IsRegular() {
-			c.reportError(
-				job.path, "",
-				fmt.Errorf(
-					"unexpected file mode, not regular file, mode= %s",
-					job.stat.mode,
-				),
-			)
-			return
-		}
-
-		c.submit(&EventUpdateJob{job.report()})
-		jobs = append(jobs, job)
-		atomic.AddInt64(&cntr.files, 1)
-		atomic.AddInt64(&cntr.bytes, job.stat.size)
+// accept turns one item into a job and hands it to the read buffer. It reports false when
+// the pipeline stopped before the job was handed over.
+func (c *Copyer) accept(ch chan<- *baseJob, item Item, order uint64, bytes, files *int64) bool {
+	if item == nil {
+		c.reportError("", "", fmt.Errorf("read batch source failed, item is nil"))
+		return false
 	}
 
-	var walk func(src *source, dsts []string)
-	walk = func(src *source, dsts []string) {
-		path := src.src()
-
-		fi, err := os.Stat(path)
-		if err != nil {
-			c.reportError(path, "", fmt.Errorf("walk get stat, %w", err))
-			return
-		}
-
-		mode := fi.Mode()
-		if mode.IsRegular() {
-			targets := make([]string, 0, len(dsts))
-			for _, d := range dsts {
-				targets = append(targets, src.dst(d))
-			}
-
-			stat, err := newStat(path, fi)
-			if err != nil {
-				c.reportError(path, "", fmt.Errorf("read sys stat, %w", err))
-				return
-			}
-
-			appendJob(&baseJob{
-				copyer:  c,
-				src:     src,
-				path:    path,
-				stat:    stat,
-				targets: targets,
-			})
-			return
-		}
-		if mode&UnexpectFileMode != 0 {
-			return
-		}
-
-		files, err := os.ReadDir(path)
-		if err != nil {
-			c.reportError(path, "", fmt.Errorf("walk read dir, %w", err))
-			return
-		}
-		for _, file := range files {
-			walk(src.append(file.Name()), dsts)
-		}
+	job, err := c.newJob(item, order)
+	if err != nil {
+		c.reportItemError(item.Source(), "", err)
+		job = &baseJob{copyer: c, item: item, order: order, targets: itemTargets(item)}
+		job.itemError = err
+	} else {
+		*files++
+		*bytes += job.stat.size
 	}
 
-	results := make([]*baseJob, 0, 64)
-	for _, j := range c.wildcardJobs {
-		for _, s := range j.src {
-			walk(s, j.dst)
-		}
-
-		if len(jobs) == 0 {
-			continue
-		}
-
-		joined, err := c.joinJobs(jobs)
-		if err != nil {
-			return nil, err
-		}
-
-		results = append(results, joined...)
-		jobs = jobs[:0]
+	select {
+	case ch <- job:
+		return true
+	case <-c.hardStop:
+		return false
 	}
-
-	for _, j := range c.accurateJobs {
-		fi, err := os.Stat(j.src)
-		if err != nil {
-			c.reportError(j.src, "", fmt.Errorf("accurate job get stat, %w", err))
-			continue
-		}
-		if !fi.Mode().IsRegular() {
-			continue
-		}
-
-		stat, err := newStat(j.src, fi)
-		if err != nil {
-			c.reportError(j.src, "", fmt.Errorf("read sys stat, %w", err))
-			continue
-		}
-
-		appendJob(&baseJob{
-			copyer:  c,
-			src:     &source{base: "/", path: j.src},
-			path:    j.src,
-			stat:    stat,
-			targets: j.dsts,
-		})
-	}
-	results = append(results, jobs...)
-
-	return results, nil
 }
 
-func (c *Copyer) joinJobs(jobs []*baseJob) ([]*baseJob, error) {
-	sort.Slice(jobs, func(i int, j int) bool {
-		return comparePath(jobs[i].src.path, jobs[j].src.path) < 0
-	})
-
-	var last *baseJob
-	filtered := make([]*baseJob, 0, len(jobs))
-	for _, job := range jobs {
-		if last != nil && last.src.path == job.src.path {
-			c.reportError(last.path, "", fmt.Errorf("same relative path, ignored, '%s'", job.path))
+// abandonItems reports accepted items that the pipeline stopped before processing.
+func (c *Copyer) abandonItems(ctx context.Context, items []Item) {
+	reason := c.abandonment(ctx)
+	for _, item := range items {
+		if item == nil {
 			continue
 		}
+		c.abandon(&baseJob{copyer: c, item: item, targets: itemTargets(item)}, reason)
+	}
+}
 
-		filtered = append(filtered, job)
-		last = job
+// newJob resolves one item's source facts. A source that cannot be described is reported
+// as a failed item, not as a pipeline failure.
+func (c *Copyer) newJob(item Item, order uint64) (*baseJob, error) {
+	path := filepath.Clean(item.Source())
+	info, err := os.Stat(path)
+	if err != nil {
+		return nil, fmt.Errorf("get source stat failed, source= '%s', %w", path, err)
+	}
+	if !info.Mode().IsRegular() {
+		return nil, fmt.Errorf("source is not a regular file, source= '%s', mode= %s", path, info.Mode())
 	}
 
-	return filtered, nil
+	stat, err := newStat(path, info)
+	if err != nil {
+		return nil, fmt.Errorf("read source stat failed, source= '%s', %w", path, err)
+	}
+
+	job := &baseJob{
+		copyer:  c,
+		item:    item,
+		src:     &source{base: filepath.Dir(path), path: filepath.Base(path)},
+		path:    path,
+		stat:    stat,
+		targets: itemTargets(item),
+		order:   order,
+	}
+	c.submit(&EventUpdateJob{job.report()})
+
+	return job, nil
+}
+
+func itemTargets(item Item) []string {
+	return append([]string(nil), item.Targets()...)
 }

item.go

@@ -0,0 +1,91 @@
+package acp
+
+import (
+	"context"
+	"fmt"
+	"io/fs"
+	"time"
+)
+
+// Item is one unit of work owned by the caller.
+//
+// The caller implements Item over whatever state it owns, so the item is the handle for
+// its own result and no id lookup or payload plumbing is needed.
+type Item interface {
+	// Source returns the exact path to read.
+	Source() string
+
+	// Targets returns the destinations of a copy. An empty slice hashes the source
+	// without writing it anywhere.
+	Targets() []string
+
+	// Completed reports the item's terminal outcome. ACP calls it exactly once for
+	// every accepted item, from a single goroutine, and never waits for I/O on it.
+	Completed(*Result)
+
+	// Failed reports an item ACP could not process at all. It also reports items that
+	// were accepted but abandoned by a graceful stop, with the stopping error.
+	Failed(error)
+}
+
+// BatchSource supplies items serially. Next returns io.EOF when the input ends, and
+// must be cheap: ACP calls it from the loop that feeds the read buffer.
+type BatchSource interface {
+	Next(context.Context) ([]Item, error)
+}
+
+// Result reports the content facts of one completed item and one outcome per requested
+// target. An item whose every target failed is still a completed item, which is how
+// "not written" is distinguished from "could not be processed".
+type Result struct {
+	Source string
+
+	Size      int64
+	Mode      fs.FileMode
+	ModTime   time.Time
+	WriteTime time.Time
+
+	SHA256            []byte
+	SignatureCacheHit bool
+
+	// Targets holds one outcome per requested target, in request order.
+	Targets []TargetResult
+}
+
+// TargetResult is the outcome of one requested target.
+type TargetResult struct {
+	Path      string
+	Size      int64
+	WriteTime time.Time
+
+	// Err is nil when the target was written and verified. Target failures keep their
+	// error identity, so errors.Is(err, ErrTargetNoSpace) identifies an exhausted
+	// medium.
+	Err error
+}
+
+// Run copies caller-owned items until the batch source ends or the pipeline fails.
+//
+// A graceful stop (the caller cancels the context) stops feeding items, finishes the
+// items already in flight, reports every accepted item through its terminal callback,
+// and returns the stopping error.
+func Run(ctx context.Context, source BatchSource, opts ...Option) error {
+	if source == nil {
+		return fmt.Errorf("run failed, batch source is nil")
+	}
+
+	copyer, err := New(ctx, append(opts, withBatchSource(source))...)
+	if err != nil {
+		return fmt.Errorf("run failed, %w", err)
+	}
+	if err := copyer.WaitErr(); err != nil {
+		return fmt.Errorf("run failed, %w", err)
+	}
+
+	// The pipeline drained, so a cancelled caller learns why it stopped even when the
+	// batch source ended its input instead of reporting the cancelation itself.
+	if err := ctx.Err(); err != nil {
+		return fmt.Errorf("run stopped, %w", err)
+	}
+	return nil
+}

item_test.go

@@ -0,0 +1,996 @@
+package acp
+
+import (
+	"bytes"
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"errors"
+	"fmt"
+	"io"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"runtime"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+// fixtureItem is a caller-owned item that records every terminal callback, so a test can
+// assert the item contract without a report or a sink.
+type fixtureItem struct {
+	source  string
+	targets []string
+
+	// hook observes each terminal callback before it is recorded.
+	hook func(*Result, error)
+
+	lock     sync.Mutex
+	results  []*Result
+	failures []error
+}
+
+func newFixtureItem(source string, targets ...string) *fixtureItem {
+	return &fixtureItem{source: source, targets: targets}
+}
+
+func (i *fixtureItem) Source() string { return i.source }
+
+func (i *fixtureItem) Targets() []string { return i.targets }
+
+func (i *fixtureItem) Completed(result *Result) { i.record(result, nil) }
+
+func (i *fixtureItem) Failed(err error) { i.record(nil, err) }
+
+func (i *fixtureItem) record(result *Result, err error) {
+	if i.hook != nil {
+		i.hook(result, err)
+	}
+
+	i.lock.Lock()
+	defer i.lock.Unlock()
+	if err != nil {
+		i.failures = append(i.failures, err)
+		return
+	}
+	i.results = append(i.results, result)
+}
+
+// terminal returns the item's single terminal outcome.
+func (i *fixtureItem) terminal(t *testing.T) (*Result, error) {
+	t.Helper()
+
+	i.lock.Lock()
+	defer i.lock.Unlock()
+	if total := len(i.results) + len(i.failures); total != 1 {
+		t.Fatalf("item %q received %d terminal callbacks, want exactly 1", i.source, total)
+	}
+	if len(i.failures) == 1 {
+		return nil, i.failures[0]
+	}
+	return i.results[0], nil
+}
+
+// callbackCount returns how many terminal callbacks the item received.
+func (i *fixtureItem) callbackCount() int {
+	i.lock.Lock()
+	defer i.lock.Unlock()
+	return len(i.results) + len(i.failures)
+}
+
+// callbackOrder records the terminal callback order of several items.
+type callbackOrder struct {
+	lock  sync.Mutex
+	items []*fixtureItem
+}
+
+func (o *callbackOrder) record(item *fixtureItem) {
+	o.lock.Lock()
+	defer o.lock.Unlock()
+	o.items = append(o.items, item)
+}
+
+func (o *callbackOrder) snapshot() []*fixtureItem {
+	o.lock.Lock()
+	defer o.lock.Unlock()
+	return append([]*fixtureItem(nil), o.items...)
+}
+
+// sliceSource replays prepared batches and then ends the input.
+type sliceSource struct {
+	batches [][]Item
+	err     error
+
+	lock  sync.Mutex
+	index int
+	nexts int
+}
+
+func newSliceSource(items ...Item) *sliceSource {
+	return &sliceSource{batches: [][]Item{items}}
+}
+
+func (s *sliceSource) Next(context.Context) ([]Item, error) {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+
+	s.nexts++
+	if s.index < len(s.batches) {
+		batch := s.batches[s.index]
+		s.index++
+		return batch, nil
+	}
+	if s.err != nil {
+		return nil, s.err
+	}
+	return nil, io.EOF
+}
+
+func (s *sliceSource) calls() int {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+	return s.nexts
+}
+
+// blockingSource delivers one batch and then blocks until the context ends, which models a
+// caller that stops feeding items when it stops the pipeline.
+type blockingSource struct {
+	batch []Item
+	// endErr is returned once the context ends and defaults to the context error.
+	endErr error
+
+	lock  sync.Mutex
+	nexts int
+}
+
+func (s *blockingSource) Next(ctx context.Context) ([]Item, error) {
+	s.lock.Lock()
+	s.nexts++
+	first := s.nexts == 1
+	s.lock.Unlock()
+
+	if first {
+		return s.batch, nil
+	}
+
+	<-ctx.Done()
+	if s.endErr != nil {
+		return nil, s.endErr
+	}
+	return nil, ctx.Err()
+}
+
+func (s *blockingSource) calls() int {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+	return s.nexts
+}
+
+// endlessSource produces one item per call until the caller cancels the context.
+type endlessSource struct {
+	input  string
+	target string
+
+	lock     sync.Mutex
+	produced []*fixtureItem
+}
+
+func (s *endlessSource) Next(ctx context.Context) ([]Item, error) {
+	select {
+	case <-ctx.Done():
+		return nil, ctx.Err()
+	default:
+	}
+
+	s.lock.Lock()
+	defer s.lock.Unlock()
+
+	index := len(s.produced) + 1
+	item := newFixtureItem(s.input, filepath.Join(s.target, fmt.Sprintf("%04d", index)))
+	s.produced = append(s.produced, item)
+	return []Item{item}, nil
+}
+
+func (s *endlessSource) items() []*fixtureItem {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+	return append([]*fixtureItem(nil), s.produced...)
+}
+
+// writeSourceFile writes one source file below dir and returns its path.
+func writeSourceFile(t *testing.T, dir, name string, content []byte) string {
+	t.Helper()
+
+	path := filepath.Join(dir, name)
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		t.Fatalf("create source directory: %v", err)
+	}
+	if err := os.WriteFile(path, content, 0o644); err != nil {
+		t.Fatalf("write source file %q: %v", path, err)
+	}
+	return path
+}
+
+// newTestCopyer builds the pipeline state a stage-level test needs without starting it.
+func newTestCopyer(t *testing.T, opts ...Option) *Copyer {
+	t.Helper()
+
+	option := newOption()
+	for _, opt := range opts {
+		if opt == nil {
+			continue
+		}
+		option = opt(option)
+	}
+	if err := option.check(); err != nil {
+		t.Fatalf("check options: %v", err)
+	}
+
+	device := t.TempDir()
+	return &Copyer{
+		option:    option,
+		eventCh:   make(chan Event, 256),
+		hardStop:  make(chan struct{}),
+		abandoned: make(chan *baseJob, 16),
+		getDevice: func(string) string { return device },
+		getDiskUsageCache: func(string) *diskUsageCache {
+			return newDiskUsageCache(device, defaultDiskUsageFreshInterval)
+		},
+	}
+}
+
+// cancelOnFirstCopy stops the pipeline when the first item starts copying.
+func cancelOnFirstCopy(cancel context.CancelFunc) EventHandler {
+	var once sync.Once
+	return func(event Event) {
+		update, ok := event.(*EventUpdateJob)
+		if !ok || update.Job.Status != JobStatusCopying {
+			return
+		}
+		once.Do(cancel)
+	}
+}
+
+func TestRunCopiesItemsToLinearTarget(t *testing.T) {
+	// Build caller-owned items without constructing copy options per file.
+	root := t.TempDir()
+	order := new(callbackOrder)
+	items := make([]Item, 0, 3)
+	for index, content := range []string{"first", "second", "third"} {
+		name := string(rune('a'+index)) + ".txt"
+		source := writeSourceFile(t, root, filepath.Join("source", name), []byte(content))
+		item := newFixtureItem(source, filepath.Join(root, "target", name))
+		item.hook = func(*Result, error) { order.record(item) }
+		items = append(items, item)
+	}
+
+	// Keep the target serialized while allowing source preparation to complete in any order.
+	if err := Run(
+		context.Background(),
+		newSliceSource(items...),
+		WithHashPolicy(HashRead),
+		SetToDevice(LinearDevice(true)),
+	); err != nil {
+		t.Fatal(err)
+	}
+
+	// A linear target reports items in request order.
+	reported := order.snapshot()
+	if len(reported) != len(items) {
+		t.Fatalf("received %d callbacks, want %d", len(reported), len(items))
+	}
+	for index, item := range reported {
+		if want := items[index].(*fixtureItem); item != want {
+			t.Fatalf("callback %d reported %q, want %q", index, item.source, want.source)
+		}
+
+		result, err := item.terminal(t)
+		if err != nil {
+			t.Fatalf("item %q failed: %v", item.source, err)
+		}
+		if len(result.Targets) != 1 || result.Targets[0].Err != nil || len(result.SHA256) == 0 {
+			t.Fatalf("unexpected result: %#v", result)
+		}
+	}
+}
+
+func TestForwardPreparedOrdersOnlyLinearTargets(t *testing.T) {
+	newJob := func(order uint64, path string) *writeJob {
+		return newWriteJob(
+			&baseJob{order: order, path: path},
+			io.NopCloser(strings.NewReader("fixture")),
+			int64(len("fixture")),
+			false,
+		)
+	}
+	tests := []struct {
+		name    string
+		linear  bool
+		wantIDs []string
+	}{
+		{name: "linear request order", linear: true, wantIDs: []string{"1", "3"}},
+		{name: "random completion order", wantIDs: []string{"3", "1"}},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			// Deliver later preparation results first; a failed or skipped middle request has no write Job.
+			copyer := &Copyer{option: &option{
+				fromDevice: &deviceOption{threads: 3},
+				toDevice:   &deviceOption{linear: test.linear},
+			}}
+			completed := make(chan prepareResult, 3)
+			completed <- prepareResult{order: 2, job: newJob(2, "3")}
+			completed <- prepareResult{order: 0, job: newJob(0, "1")}
+			completed <- prepareResult{order: 1}
+			close(completed)
+			prepared := make(chan *writeJob, 3)
+
+			// Forwarding must reorder only the serialized target path.
+			copyer.forwardPrepared(context.Background(), completed, prepared)
+			close(prepared)
+			var ids []string
+			for job := range prepared {
+				ids = append(ids, job.path)
+				job.finishSource()
+			}
+			if len(ids) != len(test.wantIDs) {
+				t.Fatalf("prepared paths = %v, want %v", ids, test.wantIDs)
+			}
+			for index := range ids {
+				if ids[index] != test.wantIDs[index] {
+					t.Fatalf("prepared paths = %v, want %v", ids, test.wantIDs)
+				}
+			}
+		})
+	}
+}
+
+func TestRunHashesWithoutTargets(t *testing.T) {
+	// Submit one targetless item through the same batch interface.
+	content := []byte("hash-only fixture")
+	source := writeSourceFile(t, t.TempDir(), "source.txt", content)
+	item := newFixtureItem(source)
+
+	// Verify ACP reads the source once and reports its SHA-256 without creating a target.
+	if err := Run(context.Background(), newSliceSource(item), WithHashPolicy(HashRead)); err != nil {
+		t.Fatal(err)
+	}
+	result, err := item.terminal(t)
+	if err != nil {
+		t.Fatalf("item failed: %v", err)
+	}
+
+	wantHash := sha256.Sum256(content)
+	if got := hex.EncodeToString(result.SHA256); got != hex.EncodeToString(wantHash[:]) {
+		t.Fatalf("SHA256 = %q, want %q", got, hex.EncodeToString(wantHash[:]))
+	}
+	if len(result.Targets) != 0 {
+		t.Fatalf("hash-only targets = %v, want none", result.Targets)
+	}
+	if result.SignatureCacheHit {
+		t.Fatal("hash-only item reported a signature cache hit")
+	}
+}
+
+func TestRunReturnsBatchSourceError(t *testing.T) {
+	// Verify that a batch source failure crosses the synchronous Run boundary.
+	sourceErr := errors.New("source failed")
+	err := Run(context.Background(), &sliceSource{err: sourceErr}, WithHashPolicy(HashOff))
+	if !errors.Is(err, sourceErr) {
+		t.Fatalf("Run() error = %v, want %v", err, sourceErr)
+	}
+
+	// Verify that items accepted before the failure still receive their outcome.
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	items := []*fixtureItem{
+		newFixtureItem(input, filepath.Join(root, "target-1.txt")),
+		newFixtureItem(input, filepath.Join(root, "target-2.txt")),
+	}
+	err = Run(
+		context.Background(),
+		&sliceSource{batches: [][]Item{{items[0]}, {items[1]}}, err: sourceErr},
+		SetToDevice(Overwrite(true)),
+	)
+	if !errors.Is(err, sourceErr) {
+		t.Fatalf("Run() error = %v, want %v", err, sourceErr)
+	}
+	for _, item := range items {
+		result, terminalErr := item.terminal(t)
+		if terminalErr != nil {
+			t.Fatalf("item %q failed: %v", item.source, terminalErr)
+		}
+		if len(result.Targets) != 1 || result.Targets[0].Err != nil {
+			t.Fatalf("item %q targets = %v", item.source, result.Targets)
+		}
+	}
+}
+
+func TestRunReportsTargetFailureAsItemOutcome(t *testing.T) {
+	// Force target creation to fail while the item still completes.
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	target := writeSourceFile(t, root, "target.txt", []byte("existing"))
+	item := newFixtureItem(input, target)
+
+	// A failed target is an item outcome with its own error identity, not a Run error.
+	if err := Run(context.Background(), newSliceSource(item)); err != nil {
+		t.Fatalf("Run() error = %v, want nil", err)
+	}
+	result, err := item.terminal(t)
+	if err != nil {
+		t.Fatalf("item failed: %v", err)
+	}
+	if len(result.Targets) != 1 {
+		t.Fatalf("targets = %v, want one outcome", result.Targets)
+	}
+	if !errors.Is(result.Targets[0].Err, fs.ErrExist) {
+		t.Fatalf("target error = %v, want %v", result.Targets[0].Err, fs.ErrExist)
+	}
+
+	// The refused target must keep its previous content.
+	content, err := os.ReadFile(target)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !bytes.Equal(content, []byte("existing")) {
+		t.Fatalf("target content = %q, want %q", content, "existing")
+	}
+}
+
+func TestRunReportsOneOutcomePerRequestedTargetInOrder(t *testing.T) {
+	// Request two targets where only the second one can be written.
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	refused := writeSourceFile(t, root, "refused.txt", []byte("existing"))
+	written := filepath.Join(root, "written.txt")
+	item := newFixtureItem(input, refused, written)
+
+	if err := Run(context.Background(), newSliceSource(item)); err != nil {
+		t.Fatalf("Run() error = %v, want nil", err)
+	}
+	result, err := item.terminal(t)
+	if err != nil {
+		t.Fatalf("an item with a failed target must still complete: %v", err)
+	}
+	if len(result.Targets) != 2 {
+		t.Fatalf("targets = %v, want one outcome per requested target", result.Targets)
+	}
+	if got := result.Targets[0]; got.Path != refused || !errors.Is(got.Err, fs.ErrExist) {
+		t.Fatalf("target 0 = %#v, want %q refused with %v", got, refused, fs.ErrExist)
+	}
+	if got := result.Targets[1]; got.Path != written || got.Err != nil {
+		t.Fatalf("target 1 = %#v, want %q written", got, written)
+	}
+
+	// The successful target must hold the source content.
+	content, err := os.ReadFile(written)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !bytes.Equal(content, []byte("fixture")) {
+		t.Fatalf("written content = %q, want %q", content, "fixture")
+	}
+}
+
+func TestRunReportsUnprocessableItemsAsFailures(t *testing.T) {
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	directory := filepath.Join(root, "directory")
+	if err := os.MkdirAll(directory, 0o755); err != nil {
+		t.Fatal(err)
+	}
+
+	// A source that cannot be described, opened, or read never completes.
+	tests := []struct {
+		name    string
+		source  string
+		target  []string
+		wantErr error
+		anyErr  bool
+	}{
+		{name: "missing source", source: filepath.Join(root, "missing"), wantErr: fs.ErrNotExist},
+		{name: "directory source", source: directory, anyErr: true},
+		{name: "successful item", source: input, target: []string{filepath.Join(root, "target.txt")}},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			item := newFixtureItem(test.source, test.target...)
+			if err := Run(context.Background(), newSliceSource(item)); err != nil {
+				t.Fatalf("Run() error = %v, want nil", err)
+			}
+
+			result, err := item.terminal(t)
+			if test.wantErr != nil || test.anyErr {
+				if err == nil {
+					t.Fatalf("item completed with %#v, want a failure", result)
+				}
+				if test.wantErr != nil && !errors.Is(err, test.wantErr) {
+					t.Fatalf("item error = %v, want %v", err, test.wantErr)
+				}
+				return
+			}
+			if err != nil {
+				t.Fatalf("item failed: %v", err)
+			}
+			if result.Source != filepath.Clean(test.source) {
+				t.Fatalf("result source = %q, want %q", result.Source, filepath.Clean(test.source))
+			}
+		})
+	}
+}
+
+func TestPrepareReportsUnstartedItemsWithoutReadingThem(t *testing.T) {
+	tests := []struct {
+		name      string
+		itemError error
+		want      error
+	}{
+		{
+			name:      "source rejected while indexing",
+			itemError: fmt.Errorf("get source stat failed, %w", fs.ErrNotExist),
+			want:      fs.ErrNotExist,
+		},
+		{name: "pipeline stopped", want: context.Canceled},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			ctx, cancel := context.WithCancel(context.Background())
+			if test.itemError == nil {
+				cancel()
+			}
+			defer cancel()
+
+			copyer := newTestCopyer(t)
+			// The source does not exist, so only a pipeline that still starts a read can
+			// report anything other than the failure this stage owes the caller.
+			item := newFixtureItem(filepath.Join(t.TempDir(), "removed"), filepath.Join(t.TempDir(), "target"))
+			job := &baseJob{
+				copyer:    copyer,
+				item:      item,
+				src:       &source{base: filepath.Dir(item.source), path: filepath.Base(item.source)},
+				path:      item.source,
+				order:     0,
+				itemError: test.itemError,
+			}
+
+			indexed := make(chan *baseJob, 1)
+			indexed <- job
+			close(indexed)
+
+			done := make(chan struct{})
+			go func() {
+				defer close(done)
+				for range copyer.prepare(ctx, indexed) {
+					t.Error("prepare handed out a reader for an item that never started")
+				}
+			}()
+
+			select {
+			case abandoned := <-copyer.abandoned:
+				if abandoned != job {
+					t.Fatalf("abandoned job = %p, want %p", abandoned, job)
+				}
+				// Cleanup owns terminal callbacks, so deliver the one this item is owed.
+				copyer.failedJob(abandoned)
+			case <-time.After(5 * time.Second):
+				t.Fatal("prepare did not report the unstarted item")
+			}
+
+			select {
+			case <-done:
+			case <-time.After(5 * time.Second):
+				t.Fatal("prepare did not finish after the stop")
+			}
+
+			if _, err := item.terminal(t); !errors.Is(err, test.want) {
+				t.Fatalf("item error = %v, want %v", err, test.want)
+			}
+		})
+	}
+}
+
+func TestPrepareCompletesWithoutOpeningTheSourceWhenPolicyNeedsNoContent(t *testing.T) {
+	tests := []struct {
+		name   string
+		policy HashPolicy
+	}{
+		{name: "off", policy: HashOff},
+		{name: "cached only without entry", policy: HashCachedOnly},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			copyer := newTestCopyer(t, WithHashPolicy(test.policy))
+			if test.policy.usesCache() {
+				copyer.signatures = newSignatureCache(1)
+				t.Cleanup(func() { copyer.signatures.closeAndWait() })
+			}
+
+			// The source only has to be described: a policy that needs no content must not
+			// open it, so a missing path cannot turn the item into a failure.
+			item := newFixtureItem(filepath.Join(t.TempDir(), "removed"))
+			job := &baseJob{
+				copyer: copyer,
+				item:   item,
+				src:    &source{base: filepath.Dir(item.source), path: filepath.Base(item.source)},
+				path:   item.source,
+				stat:   &stat{size: 7, mode: 0o644},
+				order:  0,
+			}
+			indexed := make(chan *baseJob, 1)
+			indexed <- job
+			close(indexed)
+
+			var prepared []*writeJob
+			for write := range copyer.prepare(context.Background(), indexed) {
+				if !write.skipContent {
+					t.Fatal("prepare opened a source whose policy needs no content")
+				}
+				prepared = append(prepared, write)
+			}
+			if len(prepared) != 1 {
+				t.Fatalf("prepared jobs = %d, want 1", len(prepared))
+			}
+
+			if err := runWriteJob(copyer, prepared[0]); err != nil {
+				t.Fatalf("pipeline failed: %v", err)
+			}
+			result, err := item.terminal(t)
+			if err != nil {
+				t.Fatalf("item failed: %v", err)
+			}
+			if len(result.SHA256) != 0 || result.SignatureCacheHit {
+				t.Fatalf("result = %#v, want no hash and no cache hit", result)
+			}
+			if result.Size != 7 {
+				t.Fatalf("result size = %d, want the indexed size", result.Size)
+			}
+		})
+	}
+}
+
+func TestRunKeepsLinearOrderingPastAnUnprocessedItem(t *testing.T) {
+	// A linear writer advances per request order, so an item that preparation never reads
+	// must still advance that order: otherwise the items behind it lose their callback.
+	root := t.TempDir()
+	missing := newFixtureItem(filepath.Join(root, "missing"))
+	goodSource := writeSourceFile(t, root, "good.txt", []byte("good"))
+	goodTarget := filepath.Join(root, "target", "good.txt")
+	good := newFixtureItem(goodSource, goodTarget)
+
+	if err := Run(
+		context.Background(),
+		newSliceSource(missing, good),
+		SetToDevice(LinearDevice(true)),
+	); err != nil {
+		t.Fatalf("Run() error = %v, want nil", err)
+	}
+
+	if _, err := missing.terminal(t); !errors.Is(err, fs.ErrNotExist) {
+		t.Fatalf("rejected item error = %v, want %v", err, fs.ErrNotExist)
+	}
+	result, err := good.terminal(t)
+	if err != nil {
+		t.Fatalf("item behind the rejected one failed: %v", err)
+	}
+	if len(result.Targets) != 1 || result.Targets[0].Err != nil || result.Targets[0].Path != goodTarget {
+		t.Fatalf("targets = %#v, want %q written", result.Targets, goodTarget)
+	}
+}
+
+func TestRunReportsEveryAcceptedItemExactlyOnce(t *testing.T) {
+	tests := []struct {
+		name string
+		opts []Option
+	}{
+		{name: "buffered target"},
+		{name: "linear target", opts: []Option{SetToDevice(LinearDevice(true))}},
+	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			root := t.TempDir()
+			const total = 16
+
+			items := make([]Item, 0, total)
+			for index := 0; index < total; index++ {
+				name := fmt.Sprintf("%02d.txt", index)
+				source := writeSourceFile(t, root, name, []byte(strings.Repeat(name, 64)))
+				items = append(items, newFixtureItem(source, filepath.Join(root, "target", name)))
+			}
+
+			ctx, cancel := context.WithCancel(context.Background())
+			defer cancel()
+
+			opts := []Option{WithReadBuffer(4), WithEventHandler(cancelOnFirstCopy(cancel))}
+			opts = append(opts, test.opts...)
+			err := Run(ctx, newSliceSource(items...), opts...)
+			if !errors.Is(err, context.Canceled) {
+				t.Fatalf("Run() error = %v, want %v", err, context.Canceled)
+			}
+
+			// Every accepted item owns exactly one terminal outcome, and a stop abandons
+			// the rest with the stopping error instead of dropping them.
+			completed, abandoned := 0, 0
+			for _, submitted := range items {
+				item := submitted.(*fixtureItem)
+				if got := item.callbackCount(); got != 1 {
+					t.Fatalf("item %q received %d callbacks, want 1", item.source, got)
+				}
+				result, itemErr := item.terminal(t)
+				if itemErr != nil {
+					if !errors.Is(itemErr, context.Canceled) {
+						t.Fatalf("abandoned item %q error = %v, want %v", item.source, itemErr, context.Canceled)
+					}
+					abandoned++
+					continue
+				}
+				if len(result.Targets) != 1 || result.Targets[0].Err != nil {
+					t.Fatalf("completed item %q targets = %v", item.source, result.Targets)
+				}
+				completed++
+			}
+			if completed == 0 {
+				t.Fatal("no item finished after the stop")
+			}
+			if abandoned == 0 {
+				t.Fatal("no item was abandoned after the stop")
+			}
+		})
+	}
+}
+
+func TestRunDeliversTerminalCallbacksFromOneGoroutine(t *testing.T) {
+	// ACP delivers terminal callbacks one at a time and from a single goroutine, so a
+	// caller may perform its own I/O there without excluding the pipeline.
+	root := t.TempDir()
+	const total = 32
+
+	var (
+		lock        sync.Mutex
+		goroutines  = map[string]struct{}{}
+		callbackErr []error
+		overlapped  int32
+		inFlight    int32
+	)
+
+	items := make([]Item, 0, total)
+	for index := 0; index < total; index++ {
+		name := fmt.Sprintf("%02d.txt", index)
+		content := []byte(strings.Repeat(name, 32))
+		source := writeSourceFile(t, root, name, content)
+		target := filepath.Join(root, "target", name)
+		item := newFixtureItem(source, target)
+		item.hook = func(result *Result, err error) {
+			if atomic.AddInt32(&inFlight, 1) != 1 {
+				atomic.AddInt32(&overlapped, 1)
+			}
+			id := currentGoroutineID()
+			lock.Lock()
+			goroutines[id] = struct{}{}
+			lock.Unlock()
+
+			// The callback owns its own I/O, and every target it sees is already final.
+			if result != nil {
+				for _, outcome := range result.Targets {
+					if outcome.Err != nil {
+						continue
+					}
+					got, readErr := os.ReadFile(outcome.Path)
+					if readErr != nil {
+						lock.Lock()
+						callbackErr = append(callbackErr, readErr)
+						lock.Unlock()
+						continue
+					}
+					if !bytes.Equal(got, content) {
+						lock.Lock()
+						callbackErr = append(callbackErr, fmt.Errorf("target %q content = %q, want %q", outcome.Path, got, content))
+						lock.Unlock()
+					}
+				}
+			}
+
+			runtime.Gosched()
+			atomic.AddInt32(&inFlight, -1)
+		}
+		items = append(items, item)
+	}
+
+	if err := Run(context.Background(), newSliceSource(items...), WithHashPolicy(HashRead)); err != nil {
+		t.Fatal(err)
+	}
+
+	if overlapped != 0 {
+		t.Fatalf("%d callbacks overlapped another callback", overlapped)
+	}
+	lock.Lock()
+	defer lock.Unlock()
+	if len(goroutines) != 1 {
+		t.Fatalf("callbacks arrived from %d goroutines, want 1", len(goroutines))
+	}
+	if len(callbackErr) != 0 {
+		t.Fatalf("callback I/O failed: %v", callbackErr)
+	}
+}
+
+func TestRunAppliesBoundedBackpressure(t *testing.T) {
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", nil)
+
+	const (
+		total       = int64(256)
+		readBuffer  = 8
+		consumedMax = total / 2
+	)
+	var consumed int64
+	source := &boundedItemSource{
+		input: input, target: filepath.Join(root, "target"), total: total, consumed: &consumed,
+	}
+
+	// A bounded read buffer must stop the source from producing the whole input up front.
+	if err := Run(
+		context.Background(),
+		source,
+		WithReadBuffer(readBuffer),
+		SetToDevice(LinearDevice(true)),
+	); err != nil {
+		t.Fatal(err)
+	}
+	if got := atomic.LoadInt64(&consumed); got != total {
+		t.Fatalf("consumed %d items, want %d", got, total)
+	}
+	if maximum := atomic.LoadInt64(&source.maximum); maximum >= consumedMax {
+		t.Fatalf("maximum outstanding items = %d, want less than %d", maximum, consumedMax)
+	}
+}
+
+// boundedItemSource produces one item per call and records how far production ran ahead of
+// the terminal callbacks.
+type boundedItemSource struct {
+	input    string
+	target   string
+	total    int64
+	consumed *int64
+	maximum  int64
+	produced int64
+}
+
+func (s *boundedItemSource) Next(context.Context) ([]Item, error) {
+	id := atomic.AddInt64(&s.produced, 1)
+	if id > s.total {
+		return nil, io.EOF
+	}
+
+	outstanding := id - atomic.LoadInt64(s.consumed)
+	for {
+		maximum := atomic.LoadInt64(&s.maximum)
+		if outstanding <= maximum || atomic.CompareAndSwapInt64(&s.maximum, maximum, outstanding) {
+			break
+		}
+	}
+
+	item := newFixtureItem(s.input, filepath.Join(s.target, fmt.Sprintf("%04d", id)))
+	item.hook = func(*Result, error) { atomic.AddInt64(s.consumed, 1) }
+	return []Item{item}, nil
+}
+
+func TestRunStopsFeedingItemsAfterGracefulStop(t *testing.T) {
+	// Let the source block on its next batch after producing one copy item.
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	item := newFixtureItem(input, filepath.Join(root, "target.txt"))
+	source := &blockingSource{batch: []Item{item}}
+
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+	// The first completed item stops the pipeline while the source waits for work.
+	item.hook = func(*Result, error) { cancel() }
+
+	done := make(chan error, 1)
+	go func() {
+		done <- Run(ctx, source)
+	}()
+	select {
+	case err := <-done:
+		if !errors.Is(err, context.Canceled) {
+			t.Fatalf("Run() error = %v, want %v", err, context.Canceled)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("Run did not release the batch source after the caller cancelled")
+	}
+	if calls := source.calls(); calls != 2 {
+		t.Fatalf("source calls = %d, want 2", calls)
+	}
+	if _, err := item.terminal(t); err != nil {
+		t.Fatalf("item failed: %v", err)
+	}
+}
+
+func TestRunReturnsStoppingErrorWhenBatchSourceEndsFirst(t *testing.T) {
+	// A source that ends its input after the caller cancels must not hide the stop.
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	item := newFixtureItem(input, filepath.Join(root, "target.txt"))
+	source := &blockingSource{batch: []Item{item}, endErr: io.EOF}
+
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	done := make(chan error, 1)
+	go func() {
+		done <- Run(ctx, source, WithEventHandler(cancelOnFirstCopy(cancel)))
+	}()
+	select {
+	case err := <-done:
+		if !errors.Is(err, context.Canceled) {
+			t.Fatalf("Run() error = %v, want %v", err, context.Canceled)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("Run did not stop after the caller cancelled")
+	}
+	if calls := source.calls(); calls != 2 {
+		t.Fatalf("source calls = %d, want 2", calls)
+	}
+}
+
+func TestRunCancellationDrainsPrefetchedItems(t *testing.T) {
+	// Feed work until preparation starts so cancellation occurs with an active pipeline.
+	root := t.TempDir()
+	input := writeSourceFile(t, root, "source.txt", []byte("fixture"))
+	ctx, cancel := context.WithCancel(context.Background())
+	defer cancel()
+
+	source := &endlessSource{input: input, target: filepath.Join(root, "target")}
+	cancelWhenPreparing := func(event Event) {
+		update, ok := event.(*EventUpdateJob)
+		if !ok || update.Job.Status != JobStatusPreparing {
+			return
+		}
+		cancel()
+	}
+
+	// The canceled pipeline must drain its queues and return the context error.
+	done := make(chan error, 1)
+	go func() {
+		done <- Run(
+			ctx,
+			source,
+			SetToDevice(LinearDevice(true)),
+			WithEventHandler(cancelWhenPreparing),
+		)
+	}()
+	select {
+	case err := <-done:
+		if !errors.Is(err, context.Canceled) {
+			t.Fatalf("Run() error = %v, want %v", err, context.Canceled)
+		}
+	case <-time.After(5 * time.Second):
+		t.Fatal("Run did not return after cancellation")
+	}
+
+	// Every produced item must still report exactly one terminal outcome.
+	items := source.items()
+	if len(items) == 0 {
+		t.Fatal("the source produced no items")
+	}
+	for _, item := range items {
+		if got := item.callbackCount(); got != 1 {
+			t.Fatalf("item %q received %d callbacks, want 1", item.source, got)
+		}
+	}
+}
+
+// currentGoroutineID reads the runtime's goroutine identifier so a test can prove that
+// callbacks arrive from one goroutine. Production code must never depend on it.
+func currentGoroutineID() string {
+	buf := make([]byte, 64)
+	n := runtime.Stack(buf, false)
+	fields := strings.Fields(string(buf[:n]))
+	if len(fields) < 2 {
+		return ""
+	}
+	return fields[1]
+}

job.go

@@ -3,7 +3,6 @@ package acp
 import (
 	"context"
 	"encoding/hex"
-	"fmt"
 	"io"
 	"io/fs"
 	"sync"
@@ -37,12 +36,16 @@ var (
 )
 
 type baseJob struct {
-	copyer   *Copyer
-	src      *source
-	path     string
-	stat     *stat
-	streamID int64
-	order    uint64
+	copyer *Copyer
+	item   Item
+	src    *source
+	path   string
+	stat   *stat
+	order  uint64
+
+	// itemError reports an item ACP could not process at all through its failure
+	// callback instead of a completion.
+	itemError error
 
 	lock      sync.Mutex
 	writeTime time.Time
@@ -101,7 +104,9 @@ func (j *baseJob) success(path string) {
 }
 
 func (j *baseJob) fail(path string, err error) {
-	j.copyer.setError(fmt.Errorf("copy failed, source=%q target=%q, %w", j.path, path, err))
+	// One failed target is an item outcome, not a pipeline failure: the caller classifies
+	// target errors and decides whether the operation continues.
+	j.copyer.reportItemError(j.path, path, err)
 	j.lock.Lock()
 	defer j.lock.Unlock()
 
@@ -140,11 +145,62 @@ func (j *baseJob) report() *Job {
 	}
 }
 
+// failAll marks every requested target failed with one error, which reports a job that
+// could not be written anywhere as a completed item with failed target outcomes.
+func (j *baseJob) failAll(err error) {
+	j.copyer.reportItemError(j.path, "", err)
+
+	j.lock.Lock()
+	defer j.lock.Unlock()
+
+	if len(j.targets) == 0 {
+		j.failedTargets = map[string]error{"": err}
+		return
+	}
+	for _, target := range j.targets {
+		if j.failedTargets == nil {
+			j.failedTargets = make(map[string]error, len(j.targets))
+		}
+		j.failedTargets[target] = err
+	}
+	j.copyer.submit(&EventUpdateJob{j.report()})
+}
+
+// result reports the content facts of a finished job and one outcome per requested
+// target, in request order.
+func (j *baseJob) result() *Result {
+	j.lock.Lock()
+	defer j.lock.Unlock()
+
+	result := &Result{
+		Source:            j.path,
+		Size:              j.stat.size,
+		Mode:              j.stat.mode,
+		ModTime:           j.stat.modTime,
+		WriteTime:         j.writeTime,
+		SHA256:            append([]byte(nil), j.hash...),
+		SignatureCacheHit: j.cacheHit,
+		Targets:           make([]TargetResult, 0, len(j.targets)),
+	}
+	for _, target := range j.targets {
+		outcome := TargetResult{Path: target, Size: j.stat.size, WriteTime: j.writeTime}
+		if err, failed := j.failedTargets[target]; failed {
+			outcome.Err = err
+		}
+		result.Targets = append(result.Targets, outcome)
+	}
+
+	return result
+}
+
 type writeJob struct {
 	*baseJob
 	reader   io.ReadCloser
 	size     int64
 	consumed chan struct{}
+
+	// skipContent marks an item whose policy never reads the source content.
+	skipContent bool
 }
 
 func newWriteJob(job *baseJob, src io.ReadCloser, size int64, waitConsumed bool) *writeJob {
@@ -160,7 +216,9 @@ func newWriteJob(job *baseJob, src io.ReadCloser, size int64, waitConsumed bool)
 }
 
 func (wj *writeJob) finishSource() {
-	_ = wj.reader.Close()
+	if wj.reader != nil {
+		_ = wj.reader.Close()
+	}
 
 	if wj.consumed != nil {
 		close(wj.consumed)

library_e2e_test.go

@@ -5,14 +5,90 @@ import (
 	"context"
 	"crypto/sha256"
 	"encoding/hex"
+	"io"
 	"os"
 	"path/filepath"
+	"sync"
 	"testing"
 	"time"
 
 	"github.com/samuelncui/acp"
 )
 
+// entryItem is the caller-owned item the library end-to-end test feeds to acp.Run.
+type entryItem struct {
+	entry acp.FileEntry
+
+	lock   sync.Mutex
+	result *acp.Result
+	err    error
+	count  int
+}
+
+func (i *entryItem) Source() string { return i.entry.Source }
+
+func (i *entryItem) Targets() []string { return i.entry.Targets }
+
+func (i *entryItem) Completed(result *acp.Result) {
+	i.lock.Lock()
+	defer i.lock.Unlock()
+	i.result = result
+	i.count++
+}
+
+func (i *entryItem) Failed(err error) {
+	i.lock.Lock()
+	defer i.lock.Unlock()
+	i.err = err
+	i.count++
+}
+
+// outcome returns the item's single terminal outcome.
+func (i *entryItem) outcome(t *testing.T) (*acp.Result, error) {
+	t.Helper()
+
+	i.lock.Lock()
+	defer i.lock.Unlock()
+	if i.count != 1 {
+		t.Fatalf("item %q received %d terminal callbacks, want 1", i.entry.Source, i.count)
+	}
+	return i.result, i.err
+}
+
+// entrySource supplies the selected entries as one batch of caller-owned items.
+type entrySource struct {
+	entries []acp.FileEntry
+
+	lock  sync.Mutex
+	items []*entryItem
+}
+
+func (s *entrySource) Next(context.Context) ([]acp.Item, error) {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+
+	if len(s.entries) == 0 {
+		return nil, io.EOF
+	}
+
+	entries := s.entries
+	s.entries = nil
+
+	items := make([]acp.Item, 0, len(entries))
+	for _, entry := range entries {
+		item := &entryItem{entry: entry}
+		s.items = append(s.items, item)
+		items = append(items, item)
+	}
+	return items, nil
+}
+
+func (s *entrySource) submitted() []*entryItem {
+	s.lock.Lock()
+	defer s.lock.Unlock()
+	return append([]*entryItem(nil), s.items...)
+}
+
 func TestACPLibraryE2E(t *testing.T) {
 	if testing.Short() {
 		t.Skip("skipping end-to-end test in short mode")
@@ -45,29 +121,27 @@ func TestACPLibraryE2E(t *testing.T) {
 		}
 	}
 
+	// Select the entries the public API copies, then submit them as caller-owned items.
+	entries, err := acp.SelectFiles([]string{source}, targets)
+	if err != nil {
+		t.Fatalf("select files: %v", err)
+	}
+	if len(entries) != len(files) {
+		t.Fatalf("selected entries = %d, want %d", len(entries), len(files))
+	}
+	items := &entrySource{entries: entries}
+
 	handler, getter := acp.NewReportGetter()
 	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
 	defer cancel()
-	copyer, err := acp.New(
+	if err := acp.Run(
 		ctx,
-		acp.WildcardJob(acp.Source(source), acp.Target(targets...)),
-		acp.WithHash(true),
-		acp.Overwrite(true),
+		items,
+		acp.WithHashPolicy(acp.HashRead),
+		acp.SetToDevice(acp.Overwrite(true)),
 		acp.WithEventHandler(handler),
-	)
-	if err != nil {
-		t.Fatalf("create copyer: %v", err)
-	}
-
-	done := make(chan struct{})
-	go func() {
-		copyer.Wait()
-		close(done)
-	}()
-	select {
-	case <-done:
-	case <-ctx.Done():
-		t.Fatalf("wait for copyer: %v", ctx.Err())
+	); err != nil {
+		t.Fatalf("run: %v", err)
 	}
 
 	for _, target := range targets {
@@ -129,4 +203,33 @@ func TestACPLibraryE2E(t *testing.T) {
 			t.Fatalf("job %q SHA256 = %q, want %q", sourcePath, job.SHA256, want)
 		}
 	}
+
+	// Every submitted item owns one completion with one outcome per requested target.
+	submitted := items.submitted()
+	if len(submitted) != len(files) {
+		t.Fatalf("submitted items = %d, want %d", len(submitted), len(files))
+	}
+	for _, item := range submitted {
+		result, err := item.outcome(t)
+		if err != nil {
+			t.Fatalf("item %q failed: %v", item.entry.Source, err)
+		}
+		if result.Source != filepath.Clean(item.entry.Source) {
+			t.Fatalf("item %q result source = %q", item.entry.Source, result.Source)
+		}
+		if len(result.Targets) != len(targets) {
+			t.Fatalf("item %q targets = %v, want %d", item.entry.Source, result.Targets, len(targets))
+		}
+		for index, outcome := range result.Targets {
+			if outcome.Path != item.entry.Targets[index] || outcome.Err != nil {
+				t.Fatalf("item %q target %d = %#v", item.entry.Source, index, outcome)
+			}
+		}
+
+		data := files[item.entry.Source[len(source)+1:]]
+		sum := sha256.Sum256(data)
+		if !bytes.Equal(result.SHA256, sum[:]) {
+			t.Fatalf("item %q SHA256 = %x, want %x", item.entry.Source, result.SHA256, sum)
+		}
+	}
 }

opt.go

@@ -32,20 +32,20 @@ func comparePath(a, b string) int {
 	return strings.Compare(a, b)
 }
 
+// defaultReadBuffer is how many items ACP may hold in its read buffer when the caller
+// does not set one.
+const defaultReadBuffer = 4096
+
 type option struct {
-	accurateJobs []*accurateJob
-	wildcardJobs []*wildcardJob
-	streamSource StreamSource
-	streamSink   StreamSink
+	batch BatchSource
 
 	fromDevice *deviceOption
 	toDevice   *deviceOption
 
 	createFlag int
-	withHash   bool
+	hashPolicy HashPolicy
 
-	withSignatureCache bool
-	forceRehash        bool
+	readBuffer int
 
 	logger       *logrus.Logger
 	eventHanders []EventHandler
@@ -56,27 +56,28 @@ func newOption() *option {
 		fromDevice: new(deviceOption),
 		toDevice:   new(deviceOption),
 		createFlag: os.O_WRONLY | os.O_CREATE | os.O_EXCL,
+		readBuffer: defaultReadBuffer,
 	}
 }
 
 func (o *option) check() error {
-	for _, job := range o.wildcardJobs {
-		if err := job.check(); err != nil {
-			return err
-		}
-	}
-
 	if err := o.fromDevice.check(); err != nil {
 		return fmt.Errorf("check source device failed, %w", err)
 	}
 	if err := o.toDevice.check(); err != nil {
 		return fmt.Errorf("check target device failed, %w", err)
 	}
-	if o.withSignatureCache {
-		o.withHash = true
+	if o.toDevice.readMode != ReadBuffered {
+		return fmt.Errorf("read mode is a source option, mode= %s", o.toDevice.readMode)
 	}
-	if o.forceRehash && !o.withSignatureCache {
-		return fmt.Errorf("force rehash requires signature cache")
+	if o.readBuffer < 1 {
+		return fmt.Errorf("read buffer must be at least one item, items=%d", o.readBuffer)
+	}
+	if o.toDevice.overwrite {
+		o.createFlag = os.O_WRONLY | os.O_CREATE | os.O_TRUNC
+	}
+	if !o.hashPolicy.valid() {
+		return fmt.Errorf("unknown hash policy, policy= %s", o.hashPolicy)
 	}
 	if o.logger == nil {
 		o.logger = logrus.StandardLogger()
@@ -87,18 +88,6 @@ func (o *option) check() error {
 
 type Option func(*option) *option
 
-type accurateJob struct {
-	src  string
-	dsts []string
-}
-
-func AccurateJob(src string, dsts []string) Option {
-	return func(o *option) *option {
-		o.accurateJobs = append(o.accurateJobs, &accurateJob{src: filepath.Clean(src), dsts: dsts})
-		return o
-	}
-}
-
 func SetFromDevice(opts ...DeviceOption) Option {
 	return func(o *option) *option {
 		for _, opt := range opts {
@@ -123,14 +112,26 @@ func SetToDevice(opts ...DeviceOption) Option {
 	}
 }
 
-func Overwrite(b bool) Option {
-	return func(o *option) *option {
-		if b {
-			o.createFlag = os.O_WRONLY | os.O_CREATE | os.O_TRUNC
-			return o
-		}
+func Overwrite(b bool) DeviceOption {
+	return func(d *deviceOption) *deviceOption {
+		d.overwrite = b
+		return d
+	}
+}
 
-		o.createFlag = os.O_WRONLY | os.O_CREATE | os.O_EXCL
+// WithReadBuffer sets how many items ACP may hold in its read buffer. A Job runner
+// passes its configured read buffer here.
+func WithReadBuffer(items int) Option {
+	return func(o *option) *option {
+		o.readBuffer = items
+		return o
+	}
+}
+
+// withBatchSource installs the caller's item source.
+func withBatchSource(source BatchSource) Option {
+	return func(o *option) *option {
+		o.batch = source
 		return o
 	}
 }
@@ -139,26 +140,11 @@ func WithProgressBar() Option {
 	return WithEventHandler(NewProgressBar())
 }
 
-func WithHash(b bool) Option {
+// WithHashPolicy sets the content hash policy, which also decides how the stored hash
+// cache is read and refreshed. It defaults to HashOff.
+func WithHashPolicy(policy HashPolicy) Option {
 	return func(o *option) *option {
-		o.withHash = b
-		return o
-	}
-}
-
-// WithSignatureCache enables content-signature reads and writes through xattrs.
-// It also enables hashing so cache misses can be refreshed.
-func WithSignatureCache(enabled bool) Option {
-	return func(o *option) *option {
-		o.withSignatureCache = enabled
-		return o
-	}
-}
-
-// ForceRehash bypasses signature-cache reads while still refreshing the cache.
-func ForceRehash(force bool) Option {
-	return func(o *option) *option {
-		o.forceRehash = force
+		o.hashPolicy = policy
 		return o
 	}
 }

opt_device.go

@@ -2,9 +2,31 @@ package acp
 
 import "fmt"
 
+// ReadMode selects how a source device reads its content.
+type ReadMode uint8
+
+const (
+	// ReadBuffered reads through the kernel's buffered path, which is the default.
+	ReadBuffered ReadMode = iota
+	// ReadMapped reads through a memory mapping.
+	ReadMapped
+)
+
+func (m ReadMode) String() string {
+	switch m {
+	case ReadBuffered:
+		return "buffered"
+	case ReadMapped:
+		return "mapped"
+	}
+	return fmt.Sprintf("unknown(%d)", uint8(m))
+}
+
 type deviceOption struct {
-	linear  bool
-	threads int
+	linear    bool
+	threads   int
+	overwrite bool
+	readMode  ReadMode
 }
 
 func (do *deviceOption) check() error {
@@ -35,3 +57,12 @@ func DeviceThreads(threads int) DeviceOption {
 		return d
 	}
 }
+
+// WithReadMode selects how a source device reads its content. It applies to the source
+// device only and defaults to buffered reads.
+func WithReadMode(mode ReadMode) DeviceOption {
+	return func(d *deviceOption) *deviceOption {
+		d.readMode = mode
+		return d
+	}
+}

opt_test.go

@@ -9,17 +9,15 @@ import (
 )
 
 func TestSourceRoot(t *testing.T) {
+	// The file-system root splits into an empty relative path, which must still resolve as
+	// the root itself and join children onto it.
 	root := string(filepath.Separator)
-	job := Source(root)(new(wildcardJob))
-	if len(job.src) != 1 {
-		t.Fatalf("sources = %d", len(job.src))
-	}
-
-	src := job.src[0]
-	if got := src.src(); got != root {
+	base, name := filepath.Split(filepath.Clean(root))
+	if got := (&source{base: base, path: name}).src(); got != root {
 		t.Fatalf("source root = %q, want %q", got, root)
 	}
 
+	src := &source{base: base, path: name}
 	target := filepath.Join(root, "target")
 	child := src.append("file")
 	if got := child.src(); got != filepath.Join(root, "file") {
@@ -137,3 +135,55 @@ func TestNewRejectsNegativeDeviceThreads(t *testing.T) {
 		})
 	}
 }
+
+func TestNewRejectsInvalidOptions(t *testing.T) {
+	tests := []struct {
+		name   string
+		option Option
+		want   string
+	}{
+		{
+			name:   "empty read buffer",
+			option: WithReadBuffer(0),
+			want:   "read buffer must be at least one item",
+		},
+		{
+			name:   "target read mode",
+			option: SetToDevice(WithReadMode(ReadMapped)),
+			want:   "read mode is a source option",
+		},
+		{
+			name:   "unknown hash policy",
+			option: WithHashPolicy(HashPolicy(0xff)),
+			want:   "unknown hash policy",
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			copyer, err := New(context.Background(), test.option)
+			if err == nil {
+				copyer.Wait()
+				t.Fatal("New() error = nil")
+			}
+			if !strings.Contains(err.Error(), test.want) {
+				t.Fatalf("New() error = %q, want %q", err, test.want)
+			}
+		})
+	}
+}
+
+func TestRunRejectsNilBatchSource(t *testing.T) {
+	// Run owns the pipeline lifecycle, so it must reject an input it cannot read.
+	if err := Run(context.Background(), nil); err == nil {
+		t.Fatal("Run() error = nil, want a rejected batch source")
+	}
+}
+
+func TestRunRejectsInvalidOptions(t *testing.T) {
+	// Read mode belongs to the source device, so a target read mode must not start a run.
+	err := Run(context.Background(), newSliceSource(), SetToDevice(WithReadMode(ReadMapped)))
+	if err == nil || !strings.Contains(err.Error(), "read mode is a source option") {
+		t.Fatalf("Run() error = %v, want a rejected target read mode", err)
+	}
+}

opt_wildcard.go

@@ -1,106 +0,0 @@
-package acp
-
-import (
-	"fmt"
-	"os"
-	"path/filepath"
-	"sort"
-	"strings"
-)
-
-type wildcardJob struct {
-	src []*source
-	dst []string
-}
-
-func (job *wildcardJob) check() error {
-	filteredDst := make([]string, 0, len(job.dst))
-	for _, p := range job.dst {
-		p = strings.TrimSpace(p)
-		if p == "" {
-			continue
-		}
-		p = filepath.Clean(p)
-
-		dstStat, err := os.Stat(p)
-		if err != nil {
-			return fmt.Errorf("check dst path '%s', %w", p, err)
-		}
-		if !dstStat.IsDir() {
-			return fmt.Errorf("dst path is not a dir")
-		}
-
-		filteredDst = append(filteredDst, p)
-	}
-	job.dst = filteredDst
-
-	if len(job.src) == 0 {
-		return fmt.Errorf("source path not found")
-	}
-	sort.Slice(job.src, func(i, j int) bool {
-		return comparePath(job.src[i].path, job.src[j].path) < 0
-	})
-	for _, s := range job.src {
-		src := s.src()
-		if _, err := os.Stat(src); err != nil {
-			return fmt.Errorf("check src path '%s', %w", src, err)
-		}
-	}
-
-	return nil
-}
-
-func WildcardJob(opts ...WildcardJobOption) Option {
-	return func(o *option) *option {
-		j := new(wildcardJob)
-		for _, opt := range opts {
-			j = opt(j)
-		}
-
-		if len(j.src) == 0 {
-			return o
-		}
-
-		o.wildcardJobs = append(o.wildcardJobs, j)
-		return o
-	}
-}
-
-type WildcardJobOption func(*wildcardJob) *wildcardJob
-
-func Source(paths ...string) WildcardJobOption {
-	return func(j *wildcardJob) *wildcardJob {
-		for _, p := range paths {
-			p = filepath.Clean(p)
-			base, name := filepath.Split(p)
-			j.src = append(j.src, &source{base: base, path: name})
-		}
-		return j
-	}
-}
-
-func SourceWithPath(base string, paths ...string) WildcardJobOption {
-	return func(j *wildcardJob) *wildcardJob {
-		for _, p := range paths {
-			j.src = append(j.src, &source{base: base, path: p})
-		}
-		return j
-	}
-}
-
-// Deprecated: use SourceWithPath instead
-func AccurateSource(base string, paths ...[]string) WildcardJobOption {
-	return func(j *wildcardJob) *wildcardJob {
-		for _, p := range paths {
-			j.src = append(j.src, &source{base: base, path: filepath.Join(p...)})
-		}
-		return j
-	}
-}
-
-func Target(paths ...string) WildcardJobOption {
-	return func(j *wildcardJob) *wildcardJob {
-		j.dst = append(j.dst, paths...)
-		return j
-	}
-}

opt_windows_test.go

@@ -2,15 +2,15 @@
 
 package acp
 
-import "testing"
+import (
+	"path/filepath"
+	"testing"
+)
 
 func TestSourceWindowsPath(t *testing.T) {
-	job := Source(`C:\src\file.txt`)(new(wildcardJob))
-	if len(job.src) != 1 {
-		t.Fatalf("sources = %d", len(job.src))
-	}
+	base, name := filepath.Split(`C:\src\file.txt`)
+	src := &source{base: base, path: name}
 
-	src := job.src[0]
 	if got := src.src(); got != `C:\src\file.txt` {
 		t.Fatalf("source = %q", got)
 	}
@@ -20,12 +20,9 @@ func TestSourceWindowsPath(t *testing.T) {
 }
 
 func TestSourceWindowsRoot(t *testing.T) {
-	job := Source(`C:\`)(new(wildcardJob))
-	if len(job.src) != 1 {
-		t.Fatalf("sources = %d", len(job.src))
-	}
+	base, name := filepath.Split(filepath.Clean(`C:\`))
+	src := &source{base: base, path: name}
 
-	src := job.src[0]
 	if got := src.src(); got != `C:\` {
 		t.Fatalf("source root = %q", got)
 	}

prepare.go

@@ -5,7 +5,6 @@ import (
 	"context"
 	"fmt"
 	"io"
-	"os"
 	"sync"
 
 	"github.com/samuelncui/acp/mmap"
@@ -35,58 +34,71 @@ func (c *Copyer) prepare(ctx context.Context, indexed <-chan *baseJob) <-chan *w
 		go wrap(ctx, func() {
 			defer workers.Done()
 
-			// Consume indexed jobs until cancellation or source exhaustion.
+			// Consume indexed jobs until source exhaustion. A stopped pipeline drains
+			// its input without reading it, so no accepted item is dropped.
 			for {
-				select {
-				case <-ctx.Done():
+				job, ok := <-indexed
+				if !ok {
 					return
-				case job, ok := <-indexed:
-					if !ok {
+				}
+
+				// An item that was already rejected while indexing needs no source.
+				if job.itemError != nil {
+					if !c.abandonUnprepared(ctx, completed, job, job.itemError) {
 						return
 					}
-					result := prepareResult{order: job.order}
-					if c.toDevice.linear {
-						result.release = make(chan struct{})
+					continue
+				}
+				if c.stopped(ctx) {
+					if !c.abandonUnprepared(ctx, completed, job, c.abandonment(ctx)) {
+						return
 					}
-					if c.linearTargetStopped() {
-						if !sendPrepareResult(ctx, completed, result) {
-							return
-						}
-						continue
+					continue
+				}
+
+				result := prepareResult{order: job.order}
+				if c.toDevice.linear {
+					result.release = make(chan struct{})
+				}
+
+				// Enter preparation. A stored hash is read whenever the policy uses the
+				// cache and is reused only where the policy allows it.
+				job.setStatus(jobStatusPreparing)
+				var file io.ReadCloser
+				var size int64
+				reuseEligible := len(job.targets) == 0 && c.hashPolicy.reusesCache()
+				if c.signatures != nil && reuseEligible {
+					if hash, ok := c.signatures.lookup(job.path, job.stat); ok {
+						job.setCachedHash(hash)
+						file = io.NopCloser(bytes.NewReader(nil))
+						size = job.stat.size
 					}
+				}
 
-					// Enter preparation and let eligible targetless jobs reuse a valid cache entry.
-					job.setStatus(jobStatusPreparing)
-					var file io.ReadCloser
-					var size int64
-					cacheEligible := c.signatures != nil && !c.forceRehash && len(job.targets) == 0
-					if cacheEligible {
-						if hash, ok := c.signatures.lookup(job.path, job.stat); ok {
-							job.setCachedHash(hash)
-							file = io.NopCloser(bytes.NewReader(nil))
-							size = job.stat.size
-						}
+				// A policy that never reads content completes without opening the source.
+				if file == nil && len(job.targets) == 0 && !c.hashPolicy.readsContent() {
+					// No stored hash was reused here, so the item has no hash and is not
+					// reported as a cache hit.
+					job.setHash(nil)
+					wj := newWriteJob(job, nil, 0, c.fromDevice.linear)
+					wj.skipContent = true
+					result.job = wj
+					if !c.sendPrepareResult(completed, result) {
+						return
 					}
+					if !wj.waitConsumed(ctx) {
+						return
+					}
+					continue
+				}
 
-					// Cache misses and every transfer open the real source content.
-					if file == nil {
-						var err error
-						file, size, err = func(path string) (io.ReadCloser, int64, error) {
-							// Keep linear sources on ordinary descriptors for ordered reads.
-							if c.fromDevice.linear {
-								file, err := os.Open(path)
-								if err != nil {
-									return nil, 0, fmt.Errorf("open src file fail, %w", err)
-								}
-								fileInfo, err := file.Stat()
-								if err != nil {
-									_ = file.Close()
-									return nil, 0, fmt.Errorf("get src file stat fail, %w", err)
-								}
-								return file, fileInfo.Size(), nil
-							}
-
-							// Use mmap-backed readers for random-access sources, with an empty-file fallback.
+				// Cache misses and every transfer open the real source content.
+				if file == nil {
+					var err error
+					file, size, err = func(path string) (io.ReadCloser, int64, error) {
+						// Only a caller that asks for it reads through a mapping, with an
+						// empty-file fallback.
+						if c.fromDevice.readMode == ReadMapped {
 							readerAt, err := mmap.Open(path)
 							if err != nil {
 								return nil, 0, fmt.Errorf("open src file by mmap fail, %w", err)
@@ -99,28 +111,38 @@ func (c *Copyer) prepare(ctx context.Context, indexed <-chan *baseJob) <-chan *w
 							}
 
 							return mmap.NewReader(readerAt), int64(readerAt.Len()), nil
-						}(job.path)
-						if err != nil {
-							c.reportError(job.path, "", err)
-							job.fail("", err)
-							job.setStatus(jobStatusFinished)
-							if !sendPrepareResult(ctx, completed, result) {
-								return
-							}
-							continue
 						}
-					}
 
-					// Transfer the prepared reader to the ordering stage before waiting on a linear source.
-					wj := newWriteJob(job, file, size, c.fromDevice.linear)
-					result.job = wj
-					if !sendPrepareResult(ctx, completed, result) {
-						return
-					}
-					if !wj.waitConsumed(ctx) {
-						return
+						// Every other source is read buffered, without updating its access time.
+						file, err := openSource(path)
+						if err != nil {
+							return nil, 0, fmt.Errorf("open src file fail, %w", err)
+						}
+						fileInfo, err := file.Stat()
+						if err != nil {
+							_ = file.Close()
+							return nil, 0, fmt.Errorf("get src file stat fail, %w", err)
+						}
+						return file, fileInfo.Size(), nil
+					}(job.path)
+					if err != nil {
+						c.reportItemError(job.path, "", err)
+						if !c.abandonUnprepared(ctx, completed, job, err) {
+							return
+						}
+						continue
 					}
 				}
+
+				// Transfer the prepared reader to the ordering stage before waiting on a linear source.
+				wj := newWriteJob(job, file, size, c.fromDevice.linear)
+				result.job = wj
+				if !c.sendPrepareResult(completed, result) {
+					return
+				}
+				if !wj.waitConsumed(ctx) {
+					return
+				}
 			}
 		})
 	}
@@ -139,10 +161,29 @@ func (c *Copyer) prepare(ctx context.Context, indexed <-chan *baseJob) <-chan *w
 	return prepared
 }
 
-func sendPrepareResult(ctx context.Context, completed chan<- prepareResult, result prepareResult) bool {
+// abandonUnprepared reports an accepted item that preparation will never read and keeps
+// request ordering intact by publishing an empty preparation result for it.
+func (c *Copyer) abandonUnprepared(
+	ctx context.Context,
+	completed chan<- prepareResult,
+	job *baseJob,
+	reason error,
+) bool {
+	if !c.abandon(job, reason) {
+		return false
+	}
+
+	result := prepareResult{order: job.order}
+	if c.toDevice.linear {
+		result.release = make(chan struct{})
+	}
+	return c.sendPrepareResult(completed, result)
+}
+
+func (c *Copyer) sendPrepareResult(completed chan<- prepareResult, result prepareResult) bool {
 	select {
 	case completed <- result:
-	case <-ctx.Done():
+	case <-c.hardStop:
 		result.finish()
 		return false
 	}
@@ -152,7 +193,7 @@ func sendPrepareResult(ctx context.Context, completed chan<- prepareResult, resu
 	select {
 	case <-result.release:
 		return true
-	case <-ctx.Done():
+	case <-c.hardStop:
 		return false
 	}
 }
@@ -170,12 +211,14 @@ func (result prepareResult) finish() {
 	result.releaseWorker()
 }
 
+// forwardPrepared hands prepared readers to the copy stage. A graceful stop keeps
+// forwarding so the copy stage can report the items it holds.
 func (c *Copyer) forwardPrepared(
 	ctx context.Context,
 	completed <-chan prepareResult,
 	prepared chan<- *writeJob,
 ) {
-	// Close readers that remain owned by this stage when cancellation stops forwarding.
+	// Close readers that remain owned by this stage when a hard stop ends forwarding.
 	defer func() {
 		for result := range completed {
 			result.finish()
@@ -190,7 +233,7 @@ func (c *Copyer) forwardPrepared(
 			}
 			select {
 			case prepared <- result.job:
-			case <-ctx.Done():
+			case <-c.hardStop:
 				result.finish()
 				return
 			}
@@ -208,7 +251,7 @@ func (c *Copyer) forwardPrepared(
 	}()
 	for {
 		select {
-		case <-ctx.Done():
+		case <-c.hardStop:
 			return
 		case result, ok := <-completed:
 			if !ok {
@@ -234,7 +277,7 @@ func (c *Copyer) forwardPrepared(
 			select {
 			case prepared <- result.job:
 				result.releaseWorker()
-			case <-ctx.Done():
+			case <-c.hardStop:
 				result.finish()
 				return
 			}

signature_test.go

@@ -4,7 +4,6 @@ import (
 	"bytes"
 	"context"
 	"crypto/sha256"
-	"encoding/hex"
 	"errors"
 	"fmt"
 	"os"
@@ -127,134 +126,190 @@ func TestSignatureCacheWritePreservesACPSnapshot(t *testing.T) {
 	}
 }
 
-func TestRunStreamSignatureCacheHitStaleAndForce(t *testing.T) {
-	// Create one stable source whose cache lifecycle can be observed across runs.
-	content := []byte("cache fixture")
-	input := filepath.Join(t.TempDir(), "source.txt")
-	if err := os.WriteFile(input, content, 0o644); err != nil {
-		t.Fatal(err)
-	}
-	wantHash := sha256.Sum256(content)
+func TestRunHashPolicyMatrix(t *testing.T) {
+	// Every policy reuses, reads, or ignores the stored hash, and only a refresh writes.
+	// A seeded entry is metadata-valid but wrong, so a policy that reads content must
+	// report the computed hash instead of the stored one.
+	requireSignatureXattrSupport(t)
 
-	// The first hash populates the cache before RunStream returns.
-	first, firstSummary := runSignatureHash(t, input, false)
-	if first.SignatureCacheHit {
-		t.Fatal("first hash unexpectedly hit the signature cache")
-	}
-	if first.SHA256 != hex.EncodeToString(wantHash[:]) {
-		t.Fatalf("first SHA256 = %q, want %q", first.SHA256, hex.EncodeToString(wantHash[:]))
-	}
-	signature, valid, err := ReadCachedSignature(input)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if !valid {
-		t.Skip("temporary filesystem does not support signature xattrs")
-	}
-	if signature.SHA256 != wantHash || firstSummary.Writes != 1 {
-		t.Fatalf("populated signature = %#v, summary = %#v", signature, firstSummary)
+	content := []byte("policy fixture")
+	stale := []byte("stale content")
+
+	tests := []struct {
+		name       string
+		policy     HashPolicy
+		seeded     bool
+		wantHash   string
+		wantHit    bool
+		wantHits   int64
+		wantMisses int64
+		wantWrites int64
+		wantStored string
+	}{
+		{name: "off", policy: HashOff, wantHash: "none", wantStored: "none"},
+		{name: "off with stored entry", policy: HashOff, seeded: true, wantHash: "none", wantStored: "seeded"},
+		{name: "cached only miss", policy: HashCachedOnly, wantHash: "none", wantMisses: 1, wantStored: "none"},
+		{
+			name: "cached only hit", policy: HashCachedOnly, seeded: true,
+			wantHash: "seeded", wantHit: true, wantHits: 1, wantStored: "seeded",
+		},
+		{
+			name: "cached or read miss", policy: HashCachedOrRead,
+			wantHash: "computed", wantMisses: 1, wantStored: "none",
+		},
+		{
+			name: "cached or read hit", policy: HashCachedOrRead, seeded: true,
+			wantHash: "seeded", wantHit: true, wantHits: 1, wantStored: "seeded",
+		},
+		{
+			name: "cached or read refresh miss", policy: HashCachedOrReadRefresh,
+			wantHash: "computed", wantMisses: 1, wantWrites: 1, wantStored: "computed",
+		},
+		{
+			name: "cached or read refresh hit", policy: HashCachedOrReadRefresh, seeded: true,
+			wantHash: "seeded", wantHit: true, wantHits: 1, wantStored: "seeded",
+		},
+		{
+			name: "read", policy: HashRead, seeded: true,
+			wantHash: "computed", wantStored: "seeded",
+		},
+		{
+			name: "read refresh", policy: HashReadRefresh, seeded: true,
+			wantHash: "computed", wantWrites: 1, wantStored: "computed",
+		},
 	}
 
-	// Unchanged metadata uses the cached SHA-256 without reading content.
-	second, secondSummary := runSignatureHash(t, input, false)
-	if !second.SignatureCacheHit || second.SHA256 != first.SHA256 {
-		t.Fatalf("cached job = %#v, want cache hit with SHA256 %q", second, first.SHA256)
-	}
-	if secondSummary.Hits != 1 || secondSummary.Writes != 0 {
-		t.Fatalf("cache-hit summary = %#v", secondSummary)
-	}
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			input := writeSourceFile(t, t.TempDir(), "source.txt", content)
+			var seeded CachedSignature
+			if test.seeded {
+				seeded = seedStaleSignature(t, input, stale)
+			}
 
-	// A metadata change makes the old signature stale and refreshes it.
-	info, err := os.Stat(input)
-	if err != nil {
-		t.Fatal(err)
-	}
-	changed := info.ModTime().Add(2 * time.Second)
-	if err := os.Chtimes(input, changed, changed); err != nil {
-		t.Fatal(err)
-	}
-	stale, staleSummary := runSignatureHash(t, input, false)
-	if stale.SignatureCacheHit || staleSummary.Stale != 1 || staleSummary.Writes != 1 {
-		t.Fatalf("stale job = %#v, summary = %#v", stale, staleSummary)
-	}
+			result, summary := runSignatureHash(t, input, test.policy)
 
-	// Force rehash bypasses the now-valid cache and still refreshes it.
-	forced, forcedSummary := runSignatureHash(t, input, true)
-	if forced.SignatureCacheHit || forcedSummary.Hits != 0 || forcedSummary.Writes != 1 {
-		t.Fatalf("forced job = %#v, summary = %#v", forced, forcedSummary)
+			switch test.wantHash {
+			case "none":
+				if len(result.SHA256) != 0 {
+					t.Fatalf("SHA256 = %x, want none", result.SHA256)
+				}
+			case "seeded":
+				if !bytes.Equal(result.SHA256, seeded.SHA256[:]) {
+					t.Fatalf("SHA256 = %x, want the stored %x", result.SHA256, seeded.SHA256)
+				}
+			case "computed":
+				want := sha256.Sum256(content)
+				if !bytes.Equal(result.SHA256, want[:]) {
+					t.Fatalf("SHA256 = %x, want %x", result.SHA256, want)
+				}
+			}
+			if result.SignatureCacheHit != test.wantHit {
+				t.Fatalf("SignatureCacheHit = %t, want %t", result.SignatureCacheHit, test.wantHit)
+			}
+			if summary.Hits != test.wantHits || summary.Misses != test.wantMisses ||
+				summary.Writes != test.wantWrites || summary.Failures != 0 {
+				t.Fatalf("summary = %#v, want hits=%d misses=%d writes=%d", summary, test.wantHits, test.wantMisses, test.wantWrites)
+			}
+
+			stored, valid := readStoredSignature(t, input)
+			switch test.wantStored {
+			case "none":
+				if valid {
+					t.Fatalf("stored signature = %#v, want none", stored)
+				}
+			case "seeded":
+				if !valid || stored != seeded {
+					t.Fatalf("stored signature = %#v, valid=%t, want %#v", stored, valid, seeded)
+				}
+			case "computed":
+				want := sha256.Sum256(content)
+				if !valid || stored.SHA256 != want {
+					t.Fatalf("stored signature = %#v, valid=%t, want %x", stored, valid, want)
+				}
+			}
+		})
 	}
 }
 
-func TestRunStreamTransferAlwaysHashesAndRefreshesTargets(t *testing.T) {
+func TestRunRefreshWritesOnlyWhenStoredHashDiffers(t *testing.T) {
+	content := []byte("refresh fixture")
+	input := writeSourceFile(t, t.TempDir(), "source.txt", content)
+	requireSignatureXattrSupport(t)
+
+	// The first refresh stores the computed hash.
+	result, summary := runSignatureHash(t, input, HashReadRefresh)
+	want := sha256.Sum256(content)
+	if !bytes.Equal(result.SHA256, want[:]) || summary.Writes != 1 {
+		t.Fatalf("first refresh = %x / %#v", result.SHA256, summary)
+	}
+	stored, valid := readStoredSignature(t, input)
+	if !valid {
+		t.Skip("temporary filesystem does not support signature xattrs")
+	}
+
+	// An identical rewrite leaves the stored entry untouched.
+	if _, summary = runSignatureHash(t, input, HashReadRefresh); summary.Writes != 0 {
+		t.Fatalf("identical refresh wrote %d signatures, want 0", summary.Writes)
+	}
+	if again, valid := readStoredSignature(t, input); !valid || again != stored {
+		t.Fatalf("stored signature changed to %#v, want %#v", again, stored)
+	}
+
+	// A changed source content produces a different hash, which must be republished.
+	changed := []byte("changed fixture")
+	if len(changed) != len(content) {
+		t.Fatalf("test fixture requires equal content lengths, got %d and %d", len(changed), len(content))
+	}
+	if err := os.WriteFile(input, changed, 0o644); err != nil {
+		t.Fatal(err)
+	}
+	result, summary = runSignatureHash(t, input, HashReadRefresh)
+	want = sha256.Sum256(changed)
+	if !bytes.Equal(result.SHA256, want[:]) || summary.Writes != 1 {
+		t.Fatalf("changed refresh = %x / %#v", result.SHA256, summary)
+	}
+}
+
+func TestRunTransferAlwaysReadsAndRefreshesTargets(t *testing.T) {
 	// Prepare an existing target so the transfer exercises overwrite invalidation.
 	content := []byte("transfer fixture")
 	root := t.TempDir()
-	input := filepath.Join(root, "source.txt")
-	target := filepath.Join(root, "target.txt")
-	if err := os.WriteFile(input, content, 0o644); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.WriteFile(target, []byte("old"), 0o644); err != nil {
-		t.Fatal(err)
-	}
+	input := writeSourceFile(t, root, "source.txt", content)
+	target := writeSourceFile(t, root, "target.txt", []byte("old"))
 
-	// Seed a metadata-valid but incorrect cache entry to prove transfer ignores it.
-	info, err := os.Stat(input)
-	if err != nil {
-		t.Fatal(err)
-	}
-	wrong := CachedSignature{Size: info.Size(), MtimeNS: info.ModTime().UnixNano(), SHA256: sha256.Sum256([]byte("wrong"))}
-	file, err := os.Open(input)
-	if err != nil {
-		t.Fatal(err)
-	}
-	err = writeSignatureXattr(file, encodeCachedSignature(wrong))
-	_ = file.Close()
-	if err != nil {
-		t.Skipf("temporary filesystem does not support signature xattrs: %v", err)
-	}
-	targetInfo, err := os.Stat(target)
-	if err != nil {
-		t.Fatal(err)
-	}
-	targetFile, err := os.Open(target)
-	if err != nil {
-		t.Fatal(err)
-	}
-	err = writeSignatureXattr(targetFile, encodeCachedSignature(CachedSignature{
-		Size: targetInfo.Size(), MtimeNS: targetInfo.ModTime().UnixNano(), SHA256: sha256.Sum256([]byte("old")),
-	}))
-	_ = targetFile.Close()
-	if err != nil {
-		t.Skipf("temporary filesystem does not support target signature xattrs: %v", err)
-	}
+	// Seed metadata-valid but incorrect entries to prove a transfer ignores them.
+	seedStaleSignature(t, input, []byte("wrong source"))
+	seedStaleSignature(t, target, []byte("wrong target"))
 
-	// Copy real content while the source advertises a deliberately incorrect cache entry.
-	sink := new(collectingStreamSink)
+	// A request with targets always reads its source, so reuse cannot apply there.
+	item := newFixtureItem(input, target)
 	var summary SignatureCacheSummary
 	handler := func(event Event) {
 		if update, ok := event.(*EventSignatureCacheSummary); ok {
 			summary = update.Summary
 		}
 	}
-	err = RunStream(
+	if err := Run(
 		context.Background(),
-		&sliceStreamSource{requests: []*StreamRequest{{ID: 1, Source: input, Targets: []string{target}}}},
-		sink,
-		Overwrite(true),
-		WithSignatureCache(true),
+		newSliceSource(item),
+		WithHashPolicy(HashCachedOrReadRefresh),
+		SetToDevice(Overwrite(true)),
 		WithEventHandler(handler),
-	)
-	if err != nil {
+	); err != nil {
 		t.Fatal(err)
 	}
+
 	wantHash := sha256.Sum256(content)
-	job := sink.results[0].Job
-	if job.SignatureCacheHit || job.SHA256 != hex.EncodeToString(wantHash[:]) {
-		t.Fatalf("transfer job = %#v", job)
+	result, err := item.terminal(t)
+	if err != nil {
+		t.Fatalf("item failed: %v", err)
+	}
+	if result.SignatureCacheHit || !bytes.Equal(result.SHA256, wantHash[:]) {
+		t.Fatalf("transfer result = %#v, want the computed hash %x", result, wantHash)
 	}
 
-	// Verify both bytes and refreshed source and target signature facts.
+	// Both bytes and refreshed source and target signatures must be present.
 	got, err := os.ReadFile(target)
 	if err != nil {
 		t.Fatal(err)
@@ -263,10 +318,7 @@ func TestRunStreamTransferAlwaysHashesAndRefreshesTargets(t *testing.T) {
 		t.Fatalf("target content = %q, want %q", got, content)
 	}
 	for _, path := range []string{input, target} {
-		signature, valid, err := ReadCachedSignature(path)
-		if err != nil {
-			t.Fatal(err)
-		}
+		signature, valid := readStoredSignature(t, path)
 		if !valid || signature.SHA256 != wantHash {
 			t.Fatalf("signature for %q = %#v, valid=%t", path, signature, valid)
 		}
@@ -279,15 +331,9 @@ func TestRunStreamTransferAlwaysHashesAndRefreshesTargets(t *testing.T) {
 func TestOverwriteInvalidatesSignatureWithoutCache(t *testing.T) {
 	// Give the old and new content identical metadata so only explicit invalidation is safe.
 	root := t.TempDir()
-	source := filepath.Join(root, "source.txt")
-	target := filepath.Join(root, "target.txt")
+	source := writeSourceFile(t, root, "source.txt", []byte("new"))
+	target := writeSourceFile(t, root, "target.txt", []byte("old"))
 	modified := time.Unix(100, 123)
-	if err := os.WriteFile(source, []byte("new"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.WriteFile(target, []byte("old"), 0o644); err != nil {
-		t.Fatal(err)
-	}
 	for _, path := range []string{source, target} {
 		if err := os.Chtimes(path, modified, modified); err != nil {
 			t.Fatal(err)
@@ -295,26 +341,20 @@ func TestOverwriteInvalidatesSignatureWithoutCache(t *testing.T) {
 	}
 
 	// Attach the old signature after the shared metadata has been fixed.
+	seeded := CachedSignature{Size: 3, MtimeNS: modified.UnixNano(), SHA256: sha256.Sum256([]byte("old"))}
 	file, err := os.Open(target)
 	if err != nil {
 		t.Fatal(err)
 	}
-	err = writeSignatureXattr(file, encodeCachedSignature(CachedSignature{
-		Size: 3, MtimeNS: modified.UnixNano(), SHA256: sha256.Sum256([]byte("old")),
-	}))
+	err = writeSignatureXattr(file, encodeCachedSignature(seeded))
 	_ = file.Close()
 	if err != nil {
 		t.Skipf("temporary filesystem does not support signature xattrs: %v", err)
 	}
 
-	// Overwrite without enabling cache writes and require the old xattr to disappear.
-	err = RunStream(
-		context.Background(),
-		&sliceStreamSource{requests: []*StreamRequest{{ID: 1, Source: source, Targets: []string{target}}}},
-		new(collectingStreamSink),
-		Overwrite(true),
-	)
-	if err != nil {
+	// A run without a hash policy must still drop the stale stored signature.
+	item := newFixtureItem(source, target)
+	if err := Run(context.Background(), newSliceSource(item), SetToDevice(Overwrite(true))); err != nil {
 		t.Fatal(err)
 	}
 	if signature, valid, err := ReadCachedSignature(target); err != nil {
@@ -324,12 +364,9 @@ func TestOverwriteInvalidatesSignatureWithoutCache(t *testing.T) {
 	}
 }
 
-func TestRunStreamCorruptSignatureIsWarning(t *testing.T) {
+func TestRunCorruptSignatureIsWarning(t *testing.T) {
 	// Seed an undecodable managed xattr on an otherwise valid source.
-	input := filepath.Join(t.TempDir(), "source.txt")
-	if err := os.WriteFile(input, nil, 0o644); err != nil {
-		t.Fatal(err)
-	}
+	input := writeSourceFile(t, t.TempDir(), "source.txt", nil)
 	file, err := os.Open(input)
 	if err != nil {
 		t.Fatal(err)
@@ -341,39 +378,39 @@ func TestRunStreamCorruptSignatureIsWarning(t *testing.T) {
 	}
 
 	// Fall back to content hashing while surfacing only an aggregate warning.
-	job, summary := runSignatureHash(t, input, false)
-	if job.SignatureCacheHit || summary.Failures != 1 || summary.Misses != 1 || summary.Writes != 1 {
-		t.Fatalf("job = %#v, summary = %#v", job, summary)
+	result, summary := runSignatureHash(t, input, HashCachedOrReadRefresh)
+	want := sha256.Sum256(nil)
+	if result.SignatureCacheHit || !bytes.Equal(result.SHA256, want[:]) {
+		t.Fatalf("result = %#v, want the computed hash %x", result, want)
+	}
+	if summary.Failures != 1 || summary.Misses != 1 || summary.Writes != 1 {
+		t.Fatalf("summary = %#v", summary)
 	}
 	if summary.FirstError == "" || len(summary.Samples) != 1 || summary.Samples[0] != input {
 		t.Fatalf("warning details = %#v", summary)
 	}
 }
 
-func TestRunStreamSignatureCacheZeroLength(t *testing.T) {
+func TestRunSignatureCacheZeroLength(t *testing.T) {
 	// Populate the valid SHA-256 signature of an empty file.
-	input := filepath.Join(t.TempDir(), "empty")
-	if err := os.WriteFile(input, nil, 0o644); err != nil {
-		t.Fatal(err)
-	}
+	input := writeSourceFile(t, t.TempDir(), "empty", nil)
 
 	// Hash the empty content and require the asynchronous cache write to drain.
-	first, _ := runSignatureHash(t, input, false)
-	if first.SHA256 != hex.EncodeToString(sha256.New().Sum(nil)) {
-		t.Fatalf("empty SHA256 = %q", first.SHA256)
+	first, summary := runSignatureHash(t, input, HashCachedOrReadRefresh)
+	if want := sha256.Sum256(nil); !bytes.Equal(first.SHA256, want[:]) {
+		t.Fatalf("empty SHA256 = %x, want %x", first.SHA256, want)
 	}
-	_, valid, err := ReadCachedSignature(input)
-	if err != nil {
-		t.Fatal(err)
+	if summary.Writes != 1 {
+		t.Fatalf("empty cache summary = %#v", summary)
 	}
-	if !valid {
+	if _, valid := readStoredSignature(t, input); !valid {
 		t.Skip("temporary filesystem does not support signature xattrs")
 	}
 
 	// Reuse the zero-length cache entry without reopening content.
-	second, summary := runSignatureHash(t, input, false)
+	second, summary := runSignatureHash(t, input, HashCachedOrReadRefresh)
 	if !second.SignatureCacheHit || summary.Hits != 1 {
-		t.Fatalf("empty cache job = %#v, summary = %#v", second, summary)
+		t.Fatalf("empty cache result = %#v, summary = %#v", second, summary)
 	}
 }
 
@@ -394,40 +431,34 @@ func TestSignatureCacheWarningSamplesAreBounded(t *testing.T) {
 	}
 }
 
-func TestRunStreamSignatureCacheDrainsConcurrentWrites(t *testing.T) {
+func TestRunSignatureCacheDrainsConcurrentWrites(t *testing.T) {
 	// Build more sources than the writer pool can process at once.
 	root := t.TempDir()
-	source := new(sliceStreamSource)
+	items := make([]Item, 0, 32)
 	want := make(map[string][32]byte)
 	for idx := 0; idx < 32; idx++ {
-		path := filepath.Join(root, fmt.Sprintf("%02d.bin", idx))
-		content := []byte(fmt.Sprintf("concurrent signature %d", idx))
-		if err := os.WriteFile(path, content, 0o644); err != nil {
+		path := writeSourceFile(t, root, fmt.Sprintf("%02d.bin", idx), []byte(fmt.Sprintf("concurrent signature %d", idx)))
+		items = append(items, newFixtureItem(path))
+		content, err := os.ReadFile(path)
+		if err != nil {
 			t.Fatal(err)
 		}
-		source.requests = append(source.requests, &StreamRequest{ID: int64(idx + 1), Source: path})
 		want[path] = sha256.Sum256(content)
 	}
 
 	// Run concurrent hashing and require every asynchronous xattr write to drain.
-	sink := new(collectingStreamSink)
-	if err := RunStream(
-		context.Background(), source, sink,
-		WithSignatureCache(true),
+	if err := Run(
+		context.Background(),
+		newSliceSource(items...),
+		WithHashPolicy(HashReadRefresh),
 		SetFromDevice(DeviceThreads(4)),
 	); err != nil {
 		t.Fatal(err)
 	}
-	if len(sink.results) != len(want) {
-		t.Fatalf("received %d results, want %d", len(sink.results), len(want))
-	}
 
-	// Confirm all cache writes are visible after RunStream returns.
+	// Confirm all cache writes are visible after Run returns.
 	for path, expected := range want {
-		signature, valid, err := ReadCachedSignature(path)
-		if err != nil {
-			t.Fatal(err)
-		}
+		signature, valid := readStoredSignature(t, path)
 		if !valid {
 			t.Skip("temporary filesystem does not support signature xattrs")
 		}
@@ -437,69 +468,56 @@ func TestRunStreamSignatureCacheDrainsConcurrentWrites(t *testing.T) {
 	}
 }
 
-type cancelingSignatureSink struct {
-	cancel context.CancelFunc
-	path   string
-}
+func TestRunSignatureCacheDrainsAfterCancellation(t *testing.T) {
+	// Cancel only after the first completed item has left the pipeline.
+	content := []byte("cancellation fixture")
+	path := writeSourceFile(t, t.TempDir(), "cancellation.bin", content)
 
-func (s *cancelingSignatureSink) Write(_ context.Context, result *StreamResult) error {
-	s.path = result.Job.FullPath
-	s.cancel()
-	return nil
-}
-
-func (*cancelingSignatureSink) Flush(context.Context) error {
-	return nil
-}
-
-func TestRunStreamSignatureCacheDrainsAfterCancellation(t *testing.T) {
-	// Cancel only after the first completed result has entered the Sink.
-	path := filepath.Join(t.TempDir(), "cancellation.bin")
-	if err := os.WriteFile(path, []byte("cancellation fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-
-	// Run a blocking source so the Sink controls the cancellation boundary.
 	ctx, cancel := context.WithCancel(context.Background())
-	sink := &cancelingSignatureSink{cancel: cancel}
-	err := RunStream(ctx, &blockingStreamSource{
-		request: &StreamRequest{ID: 1, Source: path},
-	}, sink, WithSignatureCache(true))
+	defer cancel()
+
+	item := newFixtureItem(path)
+	item.hook = func(*Result, error) { cancel() }
+	source := &blockingSource{batch: []Item{item}, endErr: nil}
+
+	err := Run(ctx, source, WithHashPolicy(HashReadRefresh))
 	if !errors.Is(err, context.Canceled) {
-		t.Fatalf("RunStream() error = %v, want context cancellation", err)
+		t.Fatalf("Run() error = %v, want context cancellation", err)
 	}
 
-	// The completed result's cache write must drain despite caller cancellation.
-	if sink.path == "" {
-		t.Fatal("cancellation sink received no completed result")
-	}
-	_, valid, err := ReadCachedSignature(sink.path)
-	if err != nil {
-		t.Fatal(err)
+	// The completed item's cache write must drain despite caller cancellation.
+	result, terminalErr := item.terminal(t)
+	if terminalErr != nil {
+		t.Fatalf("item failed: %v", terminalErr)
 	}
+	signature, valid := readStoredSignature(t, path)
 	if !valid {
 		t.Skip("temporary filesystem does not support signature xattrs")
 	}
+	if signature.SHA256 != sha256.Sum256(content) || !bytes.Equal(result.SHA256, signature.SHA256[:]) {
+		t.Fatalf("signature = %x, result = %x", signature.SHA256, result.SHA256)
+	}
 }
 
-func TestRunStreamSignatureCacheIsBestEffortForReadOnlyFile(t *testing.T) {
+func TestRunSignatureCacheIsBestEffortForReadOnlyFile(t *testing.T) {
 	// Hash content successfully even if the filesystem rejects the cache write.
-	path := filepath.Join(t.TempDir(), "readonly.bin")
-	content := []byte("read-only signature fixture")
-	if err := os.WriteFile(path, content, 0o444); err != nil {
+	path := writeSourceFile(t, t.TempDir(), "readonly.bin", []byte("read-only signature fixture"))
+	if err := os.Chmod(path, 0o444); err != nil {
 		t.Fatal(err)
 	}
-	job, summary := runSignatureHash(t, path, true)
-	want := sha256.Sum256(content)
-	if job.SHA256 != hex.EncodeToString(want[:]) {
-		t.Fatalf("SHA256 = %q, want %q", job.SHA256, hex.EncodeToString(want[:]))
-	}
-
-	// Accept either a valid owner-writable xattr or a non-fatal warning.
-	signature, valid, err := ReadCachedSignature(path)
+	content, err := os.ReadFile(path)
 	if err != nil {
 		t.Fatal(err)
 	}
+	want := sha256.Sum256(content)
+
+	result, summary := runSignatureHash(t, path, HashReadRefresh)
+	if !bytes.Equal(result.SHA256, want[:]) {
+		t.Fatalf("SHA256 = %x, want %x", result.SHA256, want)
+	}
+
+	// Accept either a valid owner-writable xattr or a non-fatal warning.
+	signature, valid := readStoredSignature(t, path)
 	if valid && signature.SHA256 != want {
 		t.Fatalf("signature = %x, want %x", signature.SHA256, want)
 	}
@@ -508,11 +526,11 @@ func TestRunStreamSignatureCacheIsBestEffortForReadOnlyFile(t *testing.T) {
 	}
 }
 
-func runSignatureHash(t *testing.T, input string, force bool) (*Job, SignatureCacheSummary) {
+// runSignatureHash runs one targetless item with the requested policy and returns its
+// result together with the aggregate cache summary.
+func runSignatureHash(t *testing.T, input string, policy HashPolicy) (*Result, SignatureCacheSummary) {
 	t.Helper()
 
-	// Capture the aggregate event while collecting the single stream result.
-	sink := new(collectingStreamSink)
 	var summary SignatureCacheSummary
 	handler := func(event Event) {
 		if update, ok := event.(*EventSignatureCacheSummary); ok {
@@ -520,21 +538,72 @@ func runSignatureHash(t *testing.T, input string, force bool) (*Job, SignatureCa
 		}
 	}
 
-	// Run one targetless hash with the requested cache-read policy.
-	if err := RunStream(
+	item := newFixtureItem(input)
+	if err := Run(
 		context.Background(),
-		&sliceStreamSource{requests: []*StreamRequest{{ID: 1, Source: input}}},
-		sink,
-		WithSignatureCache(true),
-		ForceRehash(force),
+		newSliceSource(item),
+		WithHashPolicy(policy),
 		WithEventHandler(handler),
 	); err != nil {
 		t.Fatal(err)
 	}
-	if len(sink.results) != 1 {
-		t.Fatalf("received %d results, want 1", len(sink.results))
+	result, err := item.terminal(t)
+	if err != nil {
+		t.Fatalf("item failed: %v", err)
 	}
+	return result, summary
+}
 
-	// Return only after RunStream has drained cache writes and emitted its summary.
-	return sink.results[0].Job, summary
+// seedStaleSignature stores a metadata-valid signature for content the file does not hold,
+// so a policy that reads content reports a different hash than the stored one.
+func seedStaleSignature(t *testing.T, path string, content []byte) CachedSignature {
+	t.Helper()
+
+	info, err := os.Stat(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	stale := CachedSignature{
+		Size:    info.Size(),
+		MtimeNS: info.ModTime().UnixNano(),
+		SHA256:  sha256.Sum256(content),
+	}
+	file, err := os.Open(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = writeSignatureXattr(file, encodeCachedSignature(stale))
+	_ = file.Close()
+	if err != nil {
+		t.Skipf("temporary filesystem does not support signature xattrs: %v", err)
+	}
+	return stale
+}
+
+// requireSignatureXattrSupport skips a test when the temporary filesystem cannot store the
+// managed attribute at all.
+func requireSignatureXattrSupport(t *testing.T) {
+	t.Helper()
+
+	path := writeSourceFile(t, t.TempDir(), "probe", []byte("probe"))
+	file, err := os.Open(path)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = writeSignatureXattr(file, encodeCachedSignature(CachedSignature{Size: 5, MtimeNS: 1}))
+	_ = file.Close()
+	if err != nil {
+		t.Skipf("temporary filesystem does not support signature xattrs: %v", err)
+	}
+}
+
+// readStoredSignature returns the signature ACP currently stores for path.
+func readStoredSignature(t *testing.T, path string) (CachedSignature, bool) {
+	t.Helper()
+
+	signature, valid, err := ReadCachedSignature(path)
+	if err != nil {
+		t.Fatalf("read cached signature for %q: %v", path, err)
+	}
+	return signature, valid
 }

stream.go

@@ -1,54 +0,0 @@
-package acp
-
-import (
-	"context"
-	"fmt"
-)
-
-// StreamRequest describes one exact source and its destinations.
-type StreamRequest struct {
-	ID      int64
-	Source  string
-	Targets []string
-}
-
-// StreamResult identifies the final report for one streamed request.
-type StreamResult struct {
-	ID  int64
-	Job *Job
-}
-
-// StreamSource supplies requests serially. It returns io.EOF when exhausted.
-type StreamSource interface {
-	Next(context.Context) (*StreamRequest, error)
-}
-
-// StreamSink consumes final results serially and flushes after the pipeline drains.
-type StreamSink interface {
-	Write(context.Context, *StreamResult) error
-	Flush(context.Context) error
-}
-
-// RunStream copies a bounded stream without retaining a whole-job report.
-func RunStream(ctx context.Context, source StreamSource, sink StreamSink, opts ...Option) error {
-	if source == nil {
-		return fmt.Errorf("run stream failed, source is nil")
-	}
-	if sink == nil {
-		return fmt.Errorf("run stream failed, sink is nil")
-	}
-
-	streamOption := func(option *option) *option {
-		option.streamSource = source
-		option.streamSink = sink
-		return option
-	}
-	copyer, err := New(ctx, append(opts, streamOption)...)
-	if err != nil {
-		return fmt.Errorf("run stream failed, %w", err)
-	}
-	if err := copyer.WaitErr(); err != nil {
-		return fmt.Errorf("run stream failed, %w", err)
-	}
-	return nil
-}

stream_test.go

@@ -1,410 +0,0 @@
-package acp
-
-import (
-	"context"
-	"crypto/sha256"
-	"encoding/hex"
-	"errors"
-	"fmt"
-	"io"
-	"os"
-	"path/filepath"
-	"strings"
-	"sync/atomic"
-	"testing"
-	"time"
-)
-
-type sliceStreamSource struct {
-	requests []*StreamRequest
-	index    int
-	err      error
-}
-
-func (s *sliceStreamSource) Next(context.Context) (*StreamRequest, error) {
-	if s.index < len(s.requests) {
-		request := s.requests[s.index]
-		s.index++
-		return request, nil
-	}
-	if s.err != nil {
-		return nil, s.err
-	}
-	return nil, io.EOF
-}
-
-type collectingStreamSink struct {
-	results  []*StreamResult
-	writes   int
-	flushes  int
-	err      error
-	flushErr error
-}
-
-func (s *collectingStreamSink) Write(_ context.Context, result *StreamResult) error {
-	s.writes++
-	if s.err != nil {
-		return s.err
-	}
-	s.results = append(s.results, result)
-	return nil
-}
-
-func (s *collectingStreamSink) Flush(context.Context) error {
-	s.flushes++
-	return s.flushErr
-}
-
-func TestRunStreamCopiesRequestsToLinearTarget(t *testing.T) {
-	// Create a source stream without building ACP options per file.
-	root := t.TempDir()
-	source := new(sliceStreamSource)
-	for index, content := range []string{"first", "second", "third"} {
-		input := filepath.Join(root, "source", string(rune('a'+index))+".txt")
-		output := filepath.Join(root, "target", string(rune('a'+index))+".txt")
-		if err := os.MkdirAll(filepath.Dir(input), 0o755); err != nil {
-			t.Fatal(err)
-		}
-		if err := os.WriteFile(input, []byte(content), 0o644); err != nil {
-			t.Fatal(err)
-		}
-		source.requests = append(source.requests, &StreamRequest{ID: int64(index + 1), Source: input, Targets: []string{output}})
-	}
-	sink := new(collectingStreamSink)
-
-	// Keep the target serialized while allowing source preparation to complete in any order.
-	if err := RunStream(context.Background(), source, sink, WithHash(true), SetToDevice(LinearDevice(true))); err != nil {
-		t.Fatal(err)
-	}
-	if len(sink.results) != len(source.requests) {
-		t.Fatalf("received %d results, want %d", len(sink.results), len(source.requests))
-	}
-	for index, result := range sink.results {
-		if result.ID != int64(index+1) {
-			t.Fatalf("result %d has ID %d, want %d", index, result.ID, index+1)
-		}
-		if result.Job.Status != JobStatusFinished || len(result.Job.SuccessTargets) != 1 || result.Job.SHA256 == "" {
-			t.Fatalf("unexpected result: %#v", result.Job)
-		}
-	}
-	if sink.flushes != 1 {
-		t.Fatalf("sink flushed %d times, want 1", sink.flushes)
-	}
-}
-
-func TestForwardPreparedOrdersOnlyLinearTargets(t *testing.T) {
-	newJob := func(order uint64, id int64) *writeJob {
-		return newWriteJob(
-			&baseJob{order: order, streamID: id},
-			io.NopCloser(strings.NewReader("fixture")),
-			int64(len("fixture")),
-			false,
-		)
-	}
-	tests := []struct {
-		name    string
-		linear  bool
-		wantIDs []int64
-	}{
-		{name: "linear request order", linear: true, wantIDs: []int64{1, 3}},
-		{name: "random completion order", wantIDs: []int64{3, 1}},
-	}
-	for _, test := range tests {
-		t.Run(test.name, func(t *testing.T) {
-			// Deliver later preparation results first; a failed or skipped middle request has no write Job.
-			copyer := &Copyer{option: &option{
-				fromDevice: &deviceOption{threads: 3},
-				toDevice:   &deviceOption{linear: test.linear},
-			}}
-			completed := make(chan prepareResult, 3)
-			completed <- prepareResult{order: 2, job: newJob(2, 3)}
-			completed <- prepareResult{order: 0, job: newJob(0, 1)}
-			completed <- prepareResult{order: 1}
-			close(completed)
-			prepared := make(chan *writeJob, 3)
-
-			// Forwarding must reorder only the serialized target path.
-			copyer.forwardPrepared(context.Background(), completed, prepared)
-			close(prepared)
-			var ids []int64
-			for job := range prepared {
-				ids = append(ids, job.streamID)
-				job.finishSource()
-			}
-			if len(ids) != len(test.wantIDs) {
-				t.Fatalf("prepared IDs = %v, want %v", ids, test.wantIDs)
-			}
-			for index := range ids {
-				if ids[index] != test.wantIDs[index] {
-					t.Fatalf("prepared IDs = %v, want %v", ids, test.wantIDs)
-				}
-			}
-		})
-	}
-}
-
-func TestRunStreamHashesWithoutTargets(t *testing.T) {
-	// Submit one targetless request through the same bounded stream interface.
-	content := []byte("hash-only fixture")
-	input := filepath.Join(t.TempDir(), "source.txt")
-	if err := os.WriteFile(input, content, 0o644); err != nil {
-		t.Fatal(err)
-	}
-	source := &sliceStreamSource{requests: []*StreamRequest{{ID: 1, Source: input}}}
-	sink := new(collectingStreamSink)
-
-	// Verify ACP reads the source once and reports its SHA-256 without creating a target.
-	if err := RunStream(context.Background(), source, sink, WithHash(true)); err != nil {
-		t.Fatal(err)
-	}
-	if len(sink.results) != 1 {
-		t.Fatalf("received %d results, want 1", len(sink.results))
-	}
-	job := sink.results[0].Job
-	wantHash := sha256.Sum256(content)
-	if job.Status != JobStatusFinished || job.SHA256 != hex.EncodeToString(wantHash[:]) {
-		t.Fatalf("unexpected hash-only result: %#v", job)
-	}
-	if len(job.SuccessTargets) != 0 || len(job.FailTargets) != 0 {
-		t.Fatalf("hash-only targets = success:%v fail:%v", job.SuccessTargets, job.FailTargets)
-	}
-}
-
-func TestRunStreamReturnsSourceAndSinkErrors(t *testing.T) {
-	// Verify that source failures cross the synchronous stream interface.
-	sourceErr := errors.New("source failed")
-	err := RunStream(context.Background(), &sliceStreamSource{err: sourceErr}, new(collectingStreamSink))
-	if !errors.Is(err, sourceErr) {
-		t.Fatalf("RunStream() error = %v, want %v", err, sourceErr)
-	}
-
-	// Verify that persistence failures are returned after the pipeline drains.
-	root := t.TempDir()
-	input := filepath.Join(root, "source.txt")
-	if err := os.WriteFile(input, []byte("fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-	sinkErr := errors.New("sink failed")
-	sink := &collectingStreamSink{err: sinkErr}
-	err = RunStream(context.Background(), &sliceStreamSource{requests: []*StreamRequest{
-		{ID: 1, Source: input, Targets: []string{filepath.Join(root, "target-1.txt")}},
-		{ID: 2, Source: input, Targets: []string{filepath.Join(root, "target-2.txt")}},
-		{ID: 3, Source: input, Targets: []string{filepath.Join(root, "target-3.txt")}},
-	}}, sink, Overwrite(true))
-	if !errors.Is(err, sinkErr) {
-		t.Fatalf("RunStream() error = %v, want %v", err, sinkErr)
-	}
-	if sink.writes != 1 || sink.flushes != 0 {
-		t.Fatalf("sink calls = writes:%d flushes:%d, want writes:1 flushes:0", sink.writes, sink.flushes)
-	}
-
-	// Verify that final persistence failures cross the synchronous stream interface.
-	flushErr := errors.New("flush failed")
-	sink = &collectingStreamSink{flushErr: flushErr}
-	err = RunStream(context.Background(), &sliceStreamSource{requests: []*StreamRequest{{
-		ID: 4, Source: input,
-	}}}, sink, WithHash(true))
-	if !errors.Is(err, flushErr) {
-		t.Fatalf("RunStream() error = %v, want %v", err, flushErr)
-	}
-	if sink.writes != 1 || sink.flushes != 1 {
-		t.Fatalf("sink calls = writes:%d flushes:%d, want writes:1 flushes:1", sink.writes, sink.flushes)
-	}
-}
-
-func TestRunStreamReturnsTargetFailure(t *testing.T) {
-	// Force target creation to fail while allowing the Sink to accept the final Job result.
-	root := t.TempDir()
-	input := filepath.Join(root, "source.txt")
-	target := filepath.Join(root, "target.txt")
-	if err := os.WriteFile(input, []byte("fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-	if err := os.WriteFile(target, []byte("existing"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-	sink := new(collectingStreamSink)
-
-	// A failed copy is a RunStream error even when the Sink records the failed result.
-	err := RunStream(context.Background(), &sliceStreamSource{requests: []*StreamRequest{{
-		ID: 1, Source: input, Targets: []string{target},
-	}}}, sink)
-	if !errors.Is(err, os.ErrExist) {
-		t.Fatalf("RunStream() error = %v, want %v", err, os.ErrExist)
-	}
-	if sink.writes != 1 || len(sink.results) != 1 {
-		t.Fatalf("sink results = writes:%d results:%d, want writes:1 results:1", sink.writes, len(sink.results))
-	}
-	if len(sink.results[0].Job.FailTargets) != 1 {
-		t.Fatalf("failed targets = %v, want one target", sink.results[0].Job.FailTargets)
-	}
-}
-
-type boundedStreamSource struct {
-	input    string
-	target   string
-	total    int64
-	produced int64
-	consumed *int64
-	maximum  int64
-}
-
-func (s *boundedStreamSource) Next(context.Context) (*StreamRequest, error) {
-	id := atomic.AddInt64(&s.produced, 1)
-	if id > s.total {
-		return nil, io.EOF
-	}
-	outstanding := id - atomic.LoadInt64(s.consumed)
-	for {
-		maximum := atomic.LoadInt64(&s.maximum)
-		if outstanding <= maximum || atomic.CompareAndSwapInt64(&s.maximum, maximum, outstanding) {
-			break
-		}
-	}
-	return &StreamRequest{
-		ID: id, Source: s.input, Targets: []string{filepath.Join(s.target, fmt.Sprintf("%04d", id))},
-	}, nil
-}
-
-type countingStreamSink struct {
-	consumed int64
-}
-
-func (s *countingStreamSink) Write(context.Context, *StreamResult) error {
-	atomic.AddInt64(&s.consumed, 1)
-	return nil
-}
-
-func (*countingStreamSink) Flush(context.Context) error {
-	return nil
-}
-
-func TestRunStreamAppliesBoundedBackpressure(t *testing.T) {
-	root := t.TempDir()
-	input := filepath.Join(root, "source.txt")
-	if err := os.WriteFile(input, nil, 0o644); err != nil {
-		t.Fatal(err)
-	}
-	const total = int64(1024)
-	sink := new(countingStreamSink)
-	source := &boundedStreamSource{
-		input: input, target: filepath.Join(root, "target"), total: total, consumed: &sink.consumed,
-	}
-	if err := RunStream(context.Background(), source, sink, SetToDevice(LinearDevice(true))); err != nil {
-		t.Fatal(err)
-	}
-	if consumed := atomic.LoadInt64(&sink.consumed); consumed != total {
-		t.Fatalf("consumed %d results, want %d", consumed, total)
-	}
-	if maximum := atomic.LoadInt64(&source.maximum); maximum >= total/2 {
-		t.Fatalf("maximum outstanding requests = %d, want less than %d", maximum, total/2)
-	}
-}
-
-type untilCanceledStreamSource struct {
-	input  string
-	target string
-	nextID int64
-}
-
-type blockingStreamSource struct {
-	request *StreamRequest
-	calls   int
-}
-
-func (s *blockingStreamSource) Next(ctx context.Context) (*StreamRequest, error) {
-	s.calls++
-	if s.calls == 1 {
-		return s.request, nil
-	}
-
-	<-ctx.Done()
-	return nil, ctx.Err()
-}
-
-func TestRunStreamCancelsSourceAfterSinkFailure(t *testing.T) {
-	// Let the source block on its next request after producing one copy Job.
-	root := t.TempDir()
-	input := filepath.Join(root, "source.txt")
-	if err := os.WriteFile(input, []byte("fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-	source := &blockingStreamSource{request: &StreamRequest{
-		ID: 1, Source: input, Targets: []string{filepath.Join(root, "target.txt")},
-	}}
-	sinkErr := errors.New("sink failed")
-
-	// A Sink error must cancel the blocked Source instead of copying unpersisted work forever.
-	done := make(chan error, 1)
-	go func() {
-		done <- RunStream(context.Background(), source, &collectingStreamSink{err: sinkErr})
-	}()
-	select {
-	case err := <-done:
-		if !errors.Is(err, sinkErr) {
-			t.Fatalf("RunStream() error = %v, want %v", err, sinkErr)
-		}
-	case <-time.After(5 * time.Second):
-		t.Fatal("RunStream did not cancel its Source after the Sink failed")
-	}
-	if source.calls != 2 {
-		t.Fatalf("source calls = %d, want 2", source.calls)
-	}
-}
-
-func (s *untilCanceledStreamSource) Next(ctx context.Context) (*StreamRequest, error) {
-	select {
-	case <-ctx.Done():
-		return nil, ctx.Err()
-	default:
-	}
-
-	s.nextID++
-	return &StreamRequest{
-		ID:      s.nextID,
-		Source:  s.input,
-		Targets: []string{filepath.Join(s.target, fmt.Sprintf("%04d", s.nextID))},
-	}, nil
-}
-
-func TestRunStreamCancellationDrainsPrefetchedJobs(t *testing.T) {
-	// Feed work until Prepare starts so cancellation occurs with an active pipeline.
-	root := t.TempDir()
-	input := filepath.Join(root, "source.txt")
-	if err := os.WriteFile(input, []byte("fixture"), 0o644); err != nil {
-		t.Fatal(err)
-	}
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
-	source := &untilCanceledStreamSource{input: input, target: filepath.Join(root, "target")}
-	cancelWhenPreparing := func(event Event) {
-		update, ok := event.(*EventUpdateJob)
-		if !ok {
-			return
-		}
-		if update.Job.Status != JobStatusPreparing {
-			return
-		}
-		cancel()
-	}
-
-	// The canceled pipeline must drain its queues and return the context error.
-	done := make(chan error, 1)
-	go func() {
-		done <- RunStream(
-			ctx,
-			source,
-			new(collectingStreamSink),
-			SetToDevice(LinearDevice(true)),
-			WithEventHandler(cancelWhenPreparing),
-		)
-	}()
-	select {
-	case err := <-done:
-		if !errors.Is(err, context.Canceled) {
-			t.Fatalf("RunStream() error = %v, want %v", err, context.Canceled)
-		}
-	case <-time.After(5 * time.Second):
-		t.Fatal("RunStream did not return after cancellation")
-	}
-}

syscall_noatime_linux.go

@@ -0,0 +1,8 @@
+//go:build linux
+
+package acp
+
+import "syscall"
+
+// openNoAtime keeps source reads out of each file's access time on Linux.
+const openNoAtime = syscall.O_NOATIME

syscall_noatime_other.go

@@ -0,0 +1,6 @@
+//go:build !linux
+
+package acp
+
+// openNoAtime has no portable equivalent outside Linux; the flag is absent there.
+const openNoAtime = 0