Commit Graph

5 Commits

Author SHA1 Message Date
Calle Wilund
03408b185e utils::gcp::object_storage: Fix buffer alignment reordering trailing data
Fixes #26874

Due to certain people (me) not being able to tell forward from backward,
the data alignment to ensure partial uploads adhere to the 256k-align
rule would potentially _reorder_ trailing buffers generated iff the
source buffers input into the sink are small enough. Which, as a fun fact,
they are in backup upload.

Change the unit test to use raw sink IO and add two unit tests (of which
the smaller size provokes the bug) that checks the same 64k buf segmented
upload backup uses.

Closes scylladb/scylladb#26938
2025-11-21 09:36:13 +02:00
Calle Wilund
bd1304972c utils::gcp::storage: Add merge objects operation
Allows merging 1-32 smaller files into a destination.
2025-10-13 08:53:27 +00:00
Calle Wilund
da36a9d78e boost::gcs_storage_test: reindent
Remove redundant indentation/moosewings.
2025-10-13 08:53:27 +00:00
Calle Wilund
1356f60c69 boost::gcs_storage_test: Convert to use fixture
Instead of test-local server/endpoint etc, use the gcs test fixture,
with the added bonus of a suite-shared one for additional speed.
2025-10-13 08:53:27 +00:00
Calle Wilund
21adfd8a60 test::boost::gcp_object_storage_test: Initial unit tests for GCP obj storage
Allows testing using either local mock server (installed or using docker),
or real GCP project (not tested as of writing this).

v2: Try podman if docker unavail
v3: Ensure we check log output on fake-gcs, because when using podman, the
    published port will be connectible even though the actual server is not
    up yet.
v4: Use ephermal port forward in docker/podman to allow us running parallel
    instances. Also adjust credentials and port finding in test.
v5: Re-ensure no parallel tests for this: We seem to time out in podman
    trying to fetch image for X parallel tests
v6: Remove the ephermal port stuff. Because of course this does not work
    with our podman-in-podman. Do brute-force port speculation instead.
v7: Up timeout for server start to allow docker pull.
v8: Fix string check error
v9: Add explicit docker image version
2025-09-01 18:14:20 +00:00