test(s3tables): add Apache Doris Iceberg catalog integration test (#9307)

* test(s3tables): add Apache Doris Iceberg catalog integration test

Adds an end-to-end smoke test that boots the apache/doris all-in-one
container, registers SeaweedFS as an external Iceberg REST catalog
(OAuth2 client_credentials), and validates metadata visibility plus the
parquet read path against tables seeded via the Iceberg REST API and a
PyIceberg writer container, mirroring the existing Trino, Spark, and
Dremio coverage. Wires the test into a new s3-tables-tests workflow job.

* test(s3tables): document weed shell -master flag format and fill in helper docstrings

Restores the explanatory comment on createTableBucket about the
host:port.grpcPort ServerAddress format used by `weed shell -master`
(produced by pb.NewServerAddress) so the dot separator isn't mistaken
for a typo, and adds doc comments for createIcebergNamespace,
createIcebergTable, doIcebergJSONRequest, requireDorisRuntime, and
hasDocker.
This commit is contained in:
Chris Lu
2026-05-04 00:28:30 -07:00
committed by GitHub
parent 90b706984e
commit 1de741737d
5 changed files with 1193 additions and 0 deletions
+71
View File
@@ -261,6 +261,77 @@ jobs:
path: test/s3tables/catalog_dremio/test-output.log
retention-days: 3
doris-iceberg-catalog-tests:
name: Doris Iceberg Catalog Integration Tests
runs-on: ubuntu-22.04
timeout-minutes: 35
steps:
- name: Check out code
uses: actions/checkout@v6
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version-file: 'go.mod'
id: go
- name: Set up Docker
uses: docker/setup-buildx-action@v4
- name: Pre-pull Doris image
run: docker pull apache/doris:doris-all-in-one-2.1.0
- name: Pre-pull Python image for PyIceberg writer
run: docker pull python:3.11-slim
- name: Run go mod tidy
run: go mod tidy
- name: Install SeaweedFS
run: |
go install -buildvcs=false ./weed
- name: Run Doris Iceberg Catalog Integration Tests
timeout-minutes: 30
working-directory: test/s3tables/catalog_doris
run: |
set -x
set -o pipefail
echo "=== System Information ==="
uname -a
free -h
df -h
docker info
echo "=== Starting Doris Iceberg Catalog Tests ==="
go test -v -timeout 25m . 2>&1 | tee test-output.log || {
echo "Doris Iceberg catalog integration tests failed"
exit 1
}
- name: Show test output on failure
if: failure()
working-directory: test/s3tables/catalog_doris
run: |
echo "=== Test Output ==="
if [ -f test-output.log ]; then
tail -200 test-output.log
fi
echo "=== Process information ==="
ps aux | grep -E "(weed|test|docker|doris)" || true
echo "=== Doris containers ==="
docker ps -a --filter "name=seaweed-doris" || true
- name: Upload test logs on failure
if: failure()
uses: actions/upload-artifact@v7
with:
name: doris-iceberg-catalog-test-logs
path: test/s3tables/catalog_doris/test-output.log
retention-days: 3
polaris-integration-tests:
name: Polaris Integration Tests
runs-on: ubuntu-22.04