fix(s3): stop S3 Tables routes from swallowing buckets named "buckets" or "get-table" (#9566)

* fix(s3): stop S3 Tables routes from swallowing buckets named "buckets" or "get-table"

The S3 Tables REST endpoints share top-level paths with the regular S3
API (/buckets for ListTableBuckets/CreateTableBucket, /get-table for
GetTable). They are registered first on the same router as the bucket
subrouter, so a path-style request such as GET /buckets?list-type=2 on
a bucket actually named "buckets" matched ListTableBuckets and returned
JSON. AWS SDK V2 (and Hadoop s3a / Spark) then failed XML parsing with
"Unexpected character '{' (code 123) in prolog".

Disambiguate by requiring the AWS V4 credential scope to name the
s3tables service on the colliding routes. Regular S3 SDKs sign with
service=s3, S3 Tables SDKs sign with service=s3tables, and the scope is
present in both the Authorization header and the X-Amz-Credential query
parameter for presigned URLs, so the matcher works for both flavors.

ARN-bearing S3 Tables routes (/buckets/<arn>, /namespaces/<arn>, etc.)
already cannot collide because colons are not valid in bucket names, so
they are left untouched.

* fix(s3): accept AWS JSON RPC content type as S3 Tables intent signal

The Iceberg catalog integration tests send unsigned PUT /buckets with
Content-Type: application/x-amz-json-1.1 to create table buckets. With
only the credential-scope check, those requests fell through to the
regular S3 CreateBucket handler and the suite went red on this branch.

Extend the matcher so a request is recognized as S3 Tables when either:

  - its AWS V4 credential scope names SERVICE=s3tables; or
  - it carries the canonical AWS JSON RPC 1.1 content type and is
    unsigned (a request explicitly signed for SERVICE=s3 still wins).

The regular S3 SDKs do not send application/x-amz-json-1.1, so the
signal is safe for the colliding paths (/buckets, /get-table).

Also add an AWS SDK V2 for Go integration test under
test/s3/sdk_v2_routing/ that drives the SDK's own XML deserializer
against a bucket literally named "buckets" and "get-table" — the SDK
errors before the test asserts if the server returns the wrong body
shape. Wired up via .github/workflows/s3-sdk-v2-routing-tests.yml,
mirroring the etag/acl workflow.

* s3api: extend service matcher to all S3 Tables routes; simplify scope check

- Apply serviceMatcher to every S3 Tables route, not just the bare-path
  ones. ARN-bearing paths could otherwise be hit by an S3 object key
  that starts with arn:aws:s3tables:..., inside a bucket named
  "buckets", "namespaces", "tables", or "tag". One matcher everywhere
  closes both collision classes.
- Replace strings.Split + index lookup with strings.Contains for the
  credential-scope check. The scope shape is fixed at
  AK/DATE/REGION/SERVICE/aws4_request, slashes only delimit components,
  and access keys are alphanumeric — so /s3tables/ matches iff SERVICE
  is exactly s3tables. Existing unit cases (including the
  access-key-substring case) still pass.
- Read the GetObject body in the SDK v2 routing test with io.ReadAll;
  the single Read could return short and make the equality check flaky.

* s3api: drop content-type fallback; sign s3 tables harness traffic instead

The content-type fallback in isS3TablesSignedRequest let an anonymous
regular-S3 request whose body type is application/x-amz-json-1.1 hit
an S3 Tables route when the path-style object key happened to be
shaped like an S3 Tables ARN (e.g. PutObject on bucket "buckets"
with key arn:aws:s3tables:.../bucket/foo/policy). Narrow the matcher
back to the AWS V4 credential scope so only requests signed for
SERVICE=s3tables match the S3 Tables routes.

Update the Iceberg catalog test harness — the only caller still
sending unsigned PUT /buckets — to sign with SERVICE=s3tables. The
mini instance runs in default-allow mode, so the signature itself is
not verified; only the credential scope matters for the route match.

Drop the stale unit cases for the JSON-RPC content-type signal and
the routing test that exercised unsigned harness traffic.

test/s3/sdk_v2_routing/Makefile

@@ -0,0 +1,41 @@
+# AWS SDK V2 Route Disambiguation Integration Tests
+#
+# Pins the regression for the route collision between the regular S3 API
+# and the S3 Tables REST API on shared top-level paths (/buckets,
+# /get-table). The tests use the real AWS SDK V2 for Go so the SDK's own
+# XML deserializer is the assertion — a JSON body produces an SDK error
+# before any test code runs.
+#
+# Prerequisites:
+#   - SeaweedFS running with S3 API enabled on port 8333
+#   - Go 1.21+
+#
+# Usage:
+#   make test         - Run the SDK V2 routing tests
+#   make test-verbose - Run with verbose output
+#   make clean        - Clean test cache
+
+.PHONY: all test test-verbose clean help
+
+S3_ENDPOINT ?= http://127.0.0.1:8333
+
+all: test
+
+test:
+	@echo "Running SDK V2 routing tests against $(S3_ENDPOINT)..."
+	S3_ENDPOINT=$(S3_ENDPOINT) go test -v -timeout 5m ./...
+
+test-verbose:
+	S3_ENDPOINT=$(S3_ENDPOINT) go test -v -timeout 5m -count=1 ./...
+
+clean:
+	go clean -testcache
+
+help:
+	@echo "AWS SDK V2 Route Disambiguation Tests"
+	@echo "Targets:"
+	@echo "  test          Run the routing tests"
+	@echo "  test-verbose  Run with verbose output"
+	@echo "  clean         Clean test cache"
+	@echo "Environment Variables:"
+	@echo "  S3_ENDPOINT   S3 endpoint URL (default: http://127.0.0.1:8333)"

test/s3/sdk_v2_routing/s3_sdk_v2_routing_test.go

@@ -0,0 +1,210 @@
+// Package sdkv2routing_test exercises route disambiguation between the
+// regular S3 API and the S3 Tables REST API on top-level paths the two
+// share (/buckets, /get-table). The bug it pins:
+//
+// When a user has an S3 bucket named "buckets" (or "get-table"), a
+// path-style ListObjectsV2 request sent by AWS SDK V2 / Hadoop s3a /
+// Spark would be routed to the S3 Tables ListTableBuckets handler and
+// receive a JSON body. AWS SDK V2 then fails XML parsing with
+// "Unexpected character '{' (code 123) in prolog".
+//
+// These tests use the real AWS SDK V2 for Go, so the SDK's own
+// deserializer is the assertion: if the server returns the wrong
+// content type, the SDK errors out before any test assertion runs.
+package sdkv2routing_test
+
+import (
+	"context"
+	"io"
+	"os"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/aws/aws-sdk-go-v2/aws"
+	"github.com/aws/aws-sdk-go-v2/config"
+	"github.com/aws/aws-sdk-go-v2/credentials"
+	"github.com/aws/aws-sdk-go-v2/service/s3"
+	"github.com/stretchr/testify/require"
+)
+
+const (
+	defaultEndpoint  = "http://127.0.0.1:8333"
+	defaultAccessKey = "some_access_key1"
+	defaultSecretKey = "some_secret_key1"
+	defaultRegion    = "us-east-1"
+)
+
+func getS3Client(t *testing.T) *s3.Client {
+	t.Helper()
+
+	endpoint := os.Getenv("S3_ENDPOINT")
+	if endpoint == "" {
+		endpoint = defaultEndpoint
+	}
+	accessKey := os.Getenv("AWS_ACCESS_KEY_ID")
+	if accessKey == "" {
+		accessKey = defaultAccessKey
+	}
+	secretKey := os.Getenv("AWS_SECRET_ACCESS_KEY")
+	if secretKey == "" {
+		secretKey = defaultSecretKey
+	}
+	region := os.Getenv("AWS_REGION")
+	if region == "" {
+		region = defaultRegion
+	}
+
+	cfg, err := config.LoadDefaultConfig(context.TODO(),
+		config.WithRegion(region),
+		config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKey, secretKey, "")),
+		config.WithEndpointResolverWithOptions(aws.EndpointResolverWithOptionsFunc(
+			func(service, region string, options ...interface{}) (aws.Endpoint, error) {
+				return aws.Endpoint{
+					URL:               endpoint,
+					SigningRegion:     defaultRegion,
+					HostnameImmutable: true,
+				}, nil
+			})),
+	)
+	require.NoError(t, err)
+
+	return s3.NewFromConfig(cfg, func(o *s3.Options) {
+		o.UsePathStyle = true
+	})
+}
+
+// ensureBucket creates bucket if it doesn't already exist. It tolerates
+// BucketAlreadyOwnedByYou / BucketAlreadyExists so the tests are
+// idempotent across local re-runs.
+func ensureBucket(t *testing.T, ctx context.Context, client *s3.Client, bucket string) {
+	t.Helper()
+	_, err := client.CreateBucket(ctx, &s3.CreateBucketInput{Bucket: aws.String(bucket)})
+	if err == nil {
+		return
+	}
+	msg := err.Error()
+	if strings.Contains(msg, "BucketAlreadyOwnedByYou") || strings.Contains(msg, "BucketAlreadyExists") {
+		return
+	}
+	t.Fatalf("CreateBucket(%q) failed: %v", bucket, err)
+}
+
+// deleteBucket best-effort cleans up a bucket and any objects in it.
+// Test does not fail if cleanup fails — the next run is idempotent.
+func deleteBucket(ctx context.Context, client *s3.Client, bucket string) {
+	paginator := s3.NewListObjectsV2Paginator(client, &s3.ListObjectsV2Input{Bucket: aws.String(bucket)})
+	for paginator.HasMorePages() {
+		page, err := paginator.NextPage(ctx)
+		if err != nil {
+			break
+		}
+		for _, obj := range page.Contents {
+			client.DeleteObject(ctx, &s3.DeleteObjectInput{Bucket: aws.String(bucket), Key: obj.Key})
+		}
+	}
+	client.DeleteBucket(ctx, &s3.DeleteBucketInput{Bucket: aws.String(bucket)})
+}
+
+// TestListObjectsV2_OnBucketNamedBuckets is the direct reproducer for
+// issue #9559: Spark / Hadoop s3a does a ListObjectsV2 against bucket
+// "buckets" via AWS SDK V2, which fails with
+// "Could not parse XML response. ... Unexpected character '{' (code 123)
+// in prolog" when SeaweedFS routes the request to the JSON-returning
+// ListTableBuckets handler. The SDK's response deserializer is the
+// real assertion here — a JSON body produces an SDK error before we
+// reach require.NoError.
+func TestListObjectsV2_OnBucketNamedBuckets(t *testing.T) {
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	client := getS3Client(t)
+
+	const bucket = "buckets"
+	ensureBucket(t, ctx, client, bucket)
+	t.Cleanup(func() { deleteBucket(ctx, client, bucket) })
+
+	out, err := client.ListObjectsV2(ctx, &s3.ListObjectsV2Input{
+		Bucket: aws.String(bucket),
+		Prefix: aws.String("logs/"),
+	})
+	require.NoError(t, err, "AWS SDK V2 must parse the response as XML")
+	require.NotNil(t, out)
+}
+
+// TestPutGetObject_OnBucketNamedBuckets exercises the full read/write
+// round-trip on the colliding bucket name. PutObject and GetObject go
+// through different routes than ListObjectsV2, and verifying them
+// guards against future regressions that re-route only some verbs.
+func TestPutGetObject_OnBucketNamedBuckets(t *testing.T) {
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	client := getS3Client(t)
+
+	const bucket = "buckets"
+	const key = "logs/hello.txt"
+	const body = "hello from issue 9559"
+
+	ensureBucket(t, ctx, client, bucket)
+	t.Cleanup(func() { deleteBucket(ctx, client, bucket) })
+
+	_, err := client.PutObject(ctx, &s3.PutObjectInput{
+		Bucket: aws.String(bucket),
+		Key:    aws.String(key),
+		Body:   strings.NewReader(body),
+	})
+	require.NoError(t, err)
+
+	out, err := client.GetObject(ctx, &s3.GetObjectInput{Bucket: aws.String(bucket), Key: aws.String(key)})
+	require.NoError(t, err)
+	defer out.Body.Close()
+
+	got, err := io.ReadAll(out.Body)
+	require.NoError(t, err)
+	require.Equal(t, body, string(got))
+}
+
+// TestListObjectsV2_OnBucketNamedGetTable covers the second colliding
+// path. The S3 Tables GET /get-table endpoint shares its path with a
+// bucket literally named "get-table", which is a legal S3 bucket name.
+func TestListObjectsV2_OnBucketNamedGetTable(t *testing.T) {
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	client := getS3Client(t)
+
+	const bucket = "get-table"
+	ensureBucket(t, ctx, client, bucket)
+	t.Cleanup(func() { deleteBucket(ctx, client, bucket) })
+
+	out, err := client.ListObjectsV2(ctx, &s3.ListObjectsV2Input{Bucket: aws.String(bucket)})
+	require.NoError(t, err)
+	require.NotNil(t, out)
+}
+
+// TestCreateAndListBuckets_ServiceLevel verifies the SDK's service-level
+// ListBuckets still parses as XML when a bucket named "buckets" exists.
+// ListBuckets goes through GET / (root) which is unaffected by the
+// /buckets route collision — this is a guard against the matcher
+// accidentally widening to top-level paths.
+func TestCreateAndListBuckets_ServiceLevel(t *testing.T) {
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	client := getS3Client(t)
+
+	const bucket = "buckets"
+	ensureBucket(t, ctx, client, bucket)
+	t.Cleanup(func() { deleteBucket(ctx, client, bucket) })
+
+	out, err := client.ListBuckets(ctx, &s3.ListBucketsInput{})
+	require.NoError(t, err)
+	require.NotNil(t, out)
+
+	found := false
+	for _, b := range out.Buckets {
+		if aws.ToString(b.Name) == bucket {
+			found = true
+			break
+		}
+	}
+	require.True(t, found, "bucket %q must appear in service-level ListBuckets", bucket)
+}
+

test/s3tables/catalog/iceberg_catalog_test.go

@@ -17,6 +17,9 @@ import (
 	"testing"
 	"time"
 
+	"github.com/aws/aws-sdk-go/aws/credentials"
+	v4 "github.com/aws/aws-sdk-go/aws/signer/v4"
+
 	"github.com/seaweedfs/seaweedfs/test/testutil"
 )
 
@@ -451,19 +454,22 @@ func icebergPath(prefix, path string) string {
 	return withPrefix
 }
 
-// createTableBucket creates a table bucket via the S3Tables REST API
+// createTableBucket creates a table bucket via the S3Tables REST API.
+// The request is AWS V4 signed for SERVICE=s3tables so the S3 Tables
+// route matcher accepts it; signing with regular SERVICE=s3 would let
+// the request fall through to the S3 CreateBucket handler.
 func createTableBucket(t *testing.T, env *TestEnvironment, bucketName string) {
 	t.Helper()
 
-	// Use S3Tables REST API to create the bucket
 	endpoint := fmt.Sprintf("http://localhost:%d/buckets", env.s3Port)
-
 	reqBody := fmt.Sprintf(`{"name":"%s"}`, bucketName)
+
 	req, err := http.NewRequest(http.MethodPut, endpoint, strings.NewReader(reqBody))
 	if err != nil {
 		t.Fatalf("Failed to create request: %v", err)
 	}
 	req.Header.Set("Content-Type", "application/x-amz-json-1.1")
+	signS3TablesRequest(t, req, reqBody)
 
 	resp, err := http.DefaultClient.Do(req)
 	if err != nil {
@@ -480,6 +486,20 @@ func createTableBucket(t *testing.T, env *TestEnvironment, bucketName string) {
 	t.Logf("Created table bucket %s", bucketName)
 }
 
+// signS3TablesRequest signs req with AWS V4 for SERVICE=s3tables. The
+// underlying weed mini instance runs in default-allow mode so the
+// signature itself is not verified; only the credential scope matters,
+// because the S3 Tables route matcher requires SERVICE=s3tables to
+// distinguish S3 Tables traffic from regular S3 calls on the same paths.
+func signS3TablesRequest(t *testing.T, req *http.Request, body string) {
+	t.Helper()
+	creds := credentials.NewStaticCredentials("test-ak", "test-sk", "")
+	signer := v4.NewSigner(creds)
+	if _, err := signer.Sign(req, strings.NewReader(body), "s3tables", "us-east-1", time.Now()); err != nil {
+		t.Fatalf("Failed to sign S3 Tables request: %v", err)
+	}
+}
+
 // randomSuffix returns a short random hex suffix for unique resource naming.
 func randomSuffix() string {
 	return fmt.Sprintf("%x", time.Now().UnixNano()&0xffffffff)