3 Commits

Include renames

Author	SHA1	Message	Date
Ben McClelland	9a73e222ca	fix: harden posix conditional publish locking ... Fail closed on unexpected advisory-lock errors instead of silently reducing cross-process exclusion to a local mutex. Make local publish-slot waits honor request cancellation. Move version snapshots under the publish lock so concurrent versioned PUTs preserve publication order. Add regression coverage for canceled lock waiters. Also add an option to disable flock files and only rely on in process locking.	2026-08-31 19:00:00 -07:00
Ben McClelland	fce9d9bf69	fix: build failure from conflicting merges	2026-08-28 09:56:40 -07:00
Carl MercierandClaude Fable 5	f7e13d71ce	fix(posix): make conditional PUT evaluation and publication atomic per key ... S3 conditional writes (If-Match / If-None-Match: *) were evaluated with a check-then-act pattern: PutObjectWithPostFunc and CompleteMultipartUploadWithCopy read the current etag, evaluated the precondition, and only later published the replacement via link/rename. Under concurrent conditional PUTs to the same key, many writers could read the same old state, all pass the check, and all succeed, breaking compare-and-swap coordination built on conditional writes (observed by Buzz's git object-store A3 conformance probe: 32-way races returned up to 32 winners instead of exactly 1x2xx + 31x412). Introduce a per-object publish lock (lockObjectPublish) that every object publication path holds across condition re-evaluation, metadata stores, and the final link: - Exclusion is an advisory file lock (flock on unix, LockFileEx on windows) on bucket/.sgwtmp/objlock/<shard>, where shard is the first byte of sha256(key). The gateway is stateless and multiple gateway processes may share one filesystem, so a process-local mutex alone is not sufficient; file locks provide cross-process and (via NFSv4 lock semantics) cross-client exclusion. A process-local striped mutex is taken alongside so in-process contention never thrashes the filesystem lock. Lock files are empty, bounded (max 256 per bucket), and never unlinked to avoid the unlink/recreate flock race that cannot be detected reliably on NFS. - Request bodies are staged to the temp file before the lock is taken; the lock covers only the short commit phase. The kernel releases the lock on close or process death, so failures and crashes cannot leave a stale lock. If the filesystem does not support advisory locking (e.g. NFS mounted with -o nolock), the gateway falls back to process-local exclusion and warns once. - The pre-staging precondition check is kept as an advisory fast-fail; the authoritative check runs under the lock. Unconditional PUTs, directory-object PUTs, CopyObject (via PutObject), scoutfs (via PutObjectWithPostFunc/CompleteMultipartUploadWithCopy), and multipart completion all participate. - For conditional writes on versioned buckets the version snapshot is deferred until the precondition is confirmed under the lock, so losing writers no longer create spurious version snapshots. - The postprocess hook now runs before any path-based metadata is written, so a failing hook no longer leaves stale sidecar attributes (previously the new etag was visible in sidecar mode even when publication failed). Add regression tests covering 32-way If-Match and If-None-Match: * races for both ordinary PUT and multipart completion, mixed conditional/unconditional races, sequential semantics, per-key independence, and cleanup after failed publication, for both xattr and sidecar metadata modes. Add an external SigV4 test (tests/conditionalrace) that replicates the Buzz A3 probe against a live gateway, optionally across two gateway processes sharing one backend directory. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-10 17:10:21 -05:00