mirror of
https://github.com/scylladb/scylladb.git
synced 2026-05-22 15:52:13 +00:00
f4fd5c7207
The jumbo sink is there to upload files that can be potentially larger than 50Gb (10000*5Mb). For that the sink uploads a set of so called "pieces" -- files up to 50Gb each -- then uses the copy-upload APi call to squash the pieces together. After copying the piece is removed. In case of a crash while uploading pieces remain in the bucket forever which is not great. This patch tags pieces with 'kind=piece' tag in order to tell pieces from regular objects. This can be used, for example, by setting up the lifecycle tag-based policy and collect dangling pieces eventually. fixes: #13670 Signed-off-by: Pavel Emelyanov <xemul@scylladb.com> Closes scylladb/scylladb#16023