mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-28 12:17:02 +00:00
c1c1752f88
Uploading sinks have internal semaphore limiting the maximum number of uploading parts and pieces with the value of two. This approach has several drawbacks. 1. The number is random. It could as well be three, four and any other 2. Jumbo upload in fact violates this parallelizm, because it applies to maximum number of pieces _and_ maximum number of parts in each piece that can be uploaded in parallels. Thus jumbo upload results in four parts in parallel. 3. Multiple uploads don't sync with each other, so uploading N objects would result in N * 2 (or even N * 4 with jumbo) uploads in parallel. 4. Single upload could benefit from using more sockets if no other uploads happen in parallel. IOW -- limit should be shard-wide, not single-upload-wide Previous patches already put the per-shard parallelizm under (some) control, so this semaphore is in fact used as a way to collect background uploading fibers on final flush and thus can be replaced with a gate. As a side effect, this fixes an issue that writes-after-flush shouldn't happen (see #13320) -- when flushed the upload gate is closed and subsequent writes would hit gate-closed error. Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>