fix: add gcs compatibility flag to fix s3proxy GCS SigV4 signature mismatch

The AWS SDK v2 includes Accept-Encoding in SigV4 signed headers
which causes GCS to return a SignatureDoesNotMatch error because
GCS rewrites that header internally before verifying the signature.

Add a --gcs-compatibility / VGW_S3_GCS_COMPATIBILITY option for the s3proxy
backend that injects two Smithy finalize-layer middlewares: one removes
Accept-Encoding from the request immediately before the Signing step, and
a second restores it after signing so the header is still sent on the wire.

see: https://github.com/aws/aws-sdk-go-v2/issues/1816

This can be removed once GCS fixes this incompatibility.
This commit is contained in:
Ben McClelland
2026-04-14 10:53:26 -07:00
parent 393477aafd
commit 9816c2fdb3
4 changed files with 86 additions and 2 deletions
+3 -1
View File
@@ -58,6 +58,7 @@ type S3Proxy struct {
sslSkipVerify bool
usePathStyle bool
debug bool
gcsCompatibility bool
}
var _ backend.Backend = &S3Proxy{}
@@ -70,7 +71,7 @@ func NewWithClient(ctx context.Context, client *s3.Client, metaBucket string) (*
return s, s.validate(ctx)
}
func New(ctx context.Context, access, secret, endpoint, region, metaBucket string, anonymousCredentials, disableChecksum, disableDataIntegrityCheck, sslSkipVerify, usePathStyle, debug bool) (*S3Proxy, error) {
func New(ctx context.Context, access, secret, endpoint, region, metaBucket string, anonymousCredentials, disableChecksum, disableDataIntegrityCheck, sslSkipVerify, usePathStyle, debug, gcsCompatibility bool) (*S3Proxy, error) {
s := &S3Proxy{
access: access,
secret: secret,
@@ -83,6 +84,7 @@ func New(ctx context.Context, access, secret, endpoint, region, metaBucket strin
sslSkipVerify: sslSkipVerify,
usePathStyle: usePathStyle,
debug: debug,
gcsCompatibility: gcsCompatibility,
}
client, err := s.getClientWithCtx(ctx)
if err != nil {