mirror of
https://github.com/versity/versitygw.git
synced 2026-08-29 04:06:56 +00:00
Nils LegerandGitHub
c638503b90
* feat(azure): carry Azure blob marker in continuation token for pagination Azure blob markers are opaque values only Azure may mint, so an S3 object key can never be passed back to Azure as a listing marker. Paging the underlying Azure listing with an S3 key therefore failed or re-scanned the whole prefix on every page. Introduce azMarkerToken, an opaque S3 continuation token that carries the Azure marker alongside the last returned key: the Azure marker resumes the blob listing where it stopped, and the last key filters out already-returned entries. Tokens are versioned with a "vgw1." prefix; anything without it is treated as a plain key, so tokens from older versions and hand-crafted markers keep working. The shared listBlobs helper now backs both ListObjects and ListObjectsV2, applying the S3 marker and delimiter client side. ListObjectsV2 pages efficiently via the token; ListObjects (v1) has no token to carry state and walks the prefix from the start each page. Add unit tests (token round-trip, marker-resumed pagination, delimiter and common-prefix handling, multipart filtering) driven by a fake Azure container, and integration tests covering full and delimited pagination. Signed-off-by: Nils Leger <nils.leger@getflip.com> * fix: token is now bound to delimiter too Signed-off-by: Nils Leger <nils.leger@getflip.com> --------- Signed-off-by: Nils Leger <nils.leger@getflip.com>