mirror of
https://github.com/versity/versitygw.git
synced 2026-07-02 16:54:25 +00:00
Ben McClelland
ef8f1b987f
STILL NOT FIXED: in backend/meta/sidecar.go CommitMetadata() Between pathIno(dataPath) != myIno returning false (check passes) and os.Rename executing, another goroutine can win link() and install a new inode at dataPath. This goroutine then proceeds to rename its stale attributes over the winner's freshly committed ones. The comment's safety argument — "the true winner's CommitMetadata will overwrite any partially committed attributes before it finishes" — only holds if the winner runs entirely after this goroutine's renames. If the winner already finished its renames before this goroutine's stale os.Rename executes, the loser's data corrupts the final metadata silently. This is a fundamental TOCTOU problem. There is no POSIX syscall for "rename only if this inode still owns the destination", so it can't be fixed with filesystem operations alone. The real fix would require serializing CommitMetadata calls per object ---- Previously, StoreAttribute wrote metadata (checksums, etc.) directly to the final per-object sidecar path during upload. Concurrent uploads of the same object would race to write into the same directory, causing one upload's metadata to be silently overwritten by another's data file, or vice-versa. On Linux, O_TMPFILE fd-number reuse made this worse: after link() closed the fd, the inode-number slot could be immediately reused by a different goroutine. Fix by staging sidecar metadata in a per-upload temporary directory named .sgwtmp.<pid>.<inode> instead of the final path, then atomically committing it after the data file has been linked. tmpSidecarID() identifies an in-flight upload by PID + inode (Unix) or PID + temp filename (Windows), matching the token produced by tmpfile.SidecarToken() in the posix backend so staging and commit find the same directory. StoreAttribute now writes attributes into the inode-keyed temp sidecar dir with a retry loop (up to 5 attempts) to handle the inode-reuse edge case where a concurrent CommitMetadata RemoveAll races with a fresh MkdirAll on the same directory name. CommitMetadata moves the staged attributes to the final object path one-by-one via atomic rename. Before each rename it re-verifies that the data file's inode still matches the token; if not, a later concurrent upload won the race and this goroutine aborts cleanly, leaving the winner's metadata intact.