mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-21 22:56:55 +00:00
Chris LuandGitHub
85abf3ca88
* feat(shell): s3.lifecycle.run-shard for manual Phase 3 dispatch Subscribes to the filer meta-log filtered to one (bucket, key-prefix-hash) shard, routes events through the compiled lifecycle engine, and dispatches due actions to the S3 server's LifecycleDelete RPC. Persists the per-shard cursor to /etc/s3/lifecycle/cursors/shard-NN.json so subsequent runs resume. Operator-runnable harness for end-to-end Phase 3 validation while the plugin-worker auto-scheduler is still pending. EventBudget bounds a single invocation; flags expose dispatch + checkpoint cadence. Discovers buckets by walking the configured DirBuckets path and reading each bucket entry's Extended[s3-bucket-lifecycle-configuration-xml] through lifecycle_xml.ParseCanonical. All compiled actions are seeded BootstrapComplete=true so the run dispatches whatever fires immediately; production bootstrap walks set this incrementally per bucket. * test(s3/lifecycle): integration test driving the run-shard shell command Spins up 'weed mini', creates a bucket with a 1-day expiration on a prefix, PUTs the target object, then rewrites the entry's Mtime via filer UpdateEntry to 30 days ago. Runs 's3.lifecycle.run-shard' for every shard via 'weed shell' subprocess and asserts the backdated object is deleted within 30s, and the in-prefix-but-recent object remains. The S3 API rejects Expiration.Days < 1, so 'wait a day' is unworkable. Backdating via the filer's gRPC sidesteps that constraint while still exercising the real Reader -> Router -> Schedule -> Dispatcher -> LifecycleDelete RPC path end-to-end. Wires a new s3-lifecycle-tests job into s3-go-tests.yml. The test runs all 16 shards because ShardID(bucket, key) is hash-based and the test shouldn't couple to that detail; running every shard keeps the test independent of the hash function. * fix(shell/s3.lifecycle.run-shard): address review findings - Reject negative -events explicitly. Help text already defines 0 as unbounded; negative budgets created ambiguous behavior in pipeline.Run. - Bound the gRPC dial with a 30s timeout instead of context.Background() so an unreachable S3 endpoint doesn't hang the shell. - Paginate the bucket listing in loadLifecycleCompileInputs. SeaweedList takes a single-RPC limit; the prior 4096 silently dropped buckets past that page on large clusters. Loop with startFrom until a page comes back short. - Surface parse errors instead of swallowing them. Buckets with malformed lifecycle XML now print the first three errors verbatim and a count for the rest, so an operator running this command for diagnostics can find what's wrong. * feat(shell/s3.lifecycle.run-shard): -shards range/set with one subscription Adds -shards "lo-hi" or "a,b,c" to the manual run command and threads the same model through Reader and Pipeline. - reader.Reader gains ShardPredicate (func(int) bool) and StartTsNs; ShardID stays for the single-shard short form. Event carries the computed ShardID so consumers can route per-shard without rehashing. - dispatcher.Pipeline gains Shards []int. When set, Run holds one Cursor + Schedule + Dispatcher per shard, opens one filer SubscribeMetadata stream with a predicate covering the whole set, and routes events into the matching shard's schedule from a single dispatch goroutine — no per-shard goroutine fan-out. - shell command parses -shard or -shards (mutually exclusive), formats progress messages with a contiguous-range label when applicable, and validates against ShardCount. Integration test now uses -shards 0-15 (one subprocess invocation) instead of a 16-iteration loop. * fix(s3/lifecycle): allow Reader with StartTsNs=0 + Cursor=nil The reader rejected the legitimate 'fresh subscription from epoch' state when called from a fresh Pipeline.Run on a multi-shard worker (no cursor file yet, all shards' MinTsNs=0). The downstream SubscribeMetadata call handles SinceNs=0 fine; the up-front check was over-defensive and broke the auto-scheduler completely (CI showed 5-second-cadence retries with this exact error). * fix(s3/lifecycle): schedule from ModTime not eventTime A backdated or out-of-band entry update has eventTime ≈ now while ModTime is far in the past; eventTime+Delay would push the dispatch into the future even though the rule already fires. ModTime+Delay is the correct fire moment. The dispatcher's identity-CAS still catches drift between schedule and dispatch. * fix(s3/lifecycle): -runtime cap on run-shard so it exits on quiet shards The CI integration test sets -events 200 expecting the subprocess to return after 200 in-shard events. But -events counts only events that pass the shard filter; the test produces ~5 such events (bucket create, lifecycle PUT, two object PUTs, mtime backdate), so the reader stays in stream.Recv forever and runShellCommand hangs the test deadline. - weed/shell/command_s3_lifecycle_run_shard.go: add -runtime D flag. When > 0, Pipeline.Run runs under context.WithTimeout(D); on expiry the reader/dispatcher drain cleanly and the cursor saves. - weed/s3api/s3lifecycle/dispatcher/pipeline.go: treat context.DeadlineExceeded the same as context.Canceled at exit (both are graceful shutdown signals). * test(s3/lifecycle): pass -runtime 10s to run-shard Pair with the new -runtime flag so the subprocess exits cleanly after 10s instead of waiting for an event budget that never lands on quiet shards. * refactor(s3/lifecycle): extract HashExtended to s3lifecycle pkg The worker's router needs the same length-prefixed sha256 of the entry's Extended map; pulling it out of the s3api private file lets both sides import it. * fix(s3/lifecycle): worker captures ExtendedHash for identity-CAS Without this, the dispatcher sends ExpectedIdentity.ExtendedHash = nil while the live entry on the server has a non-nil hash, so every dispatch returns NOOP_RESOLVED:STALE_IDENTITY and nothing is ever deleted. * fix(s3/lifecycle): identity HeadFid via GetFileIdString Meta-log events go through BeforeEntrySerialization, which clears FileChunk.FileId and writes the Fid struct instead. Reading .FileId directly returns "" on the worker side while the server's freshly fetched entry still has a populated string, so the identity-CAS would mismatch and every expiration ended in NOOP_RESOLVED:STALE_IDENTITY. * fix(s3/lifecycle): treat gRPC Canceled/DeadlineExceeded as graceful exit errors.Is doesn't unwrap a gRPC status error back to the stdlib ctx errors, so a subscription that ends because runCtx was canceled was being logged as a fatal reader error. Check status.Code as well so the shell's -runtime cap exits cleanly. * fix(test/s3/lifecycle): pass the gRPC port (not HTTP) to run-shard run-shard's -s3 flag dials the LifecycleDelete gRPC service, which listens on s3.port + 10000. The integration test was passing the HTTP port instead, so the dispatcher's RPC just timed out and the shell command exited under -runtime with no work done. * chore(test/s3/lifecycle): drop emoji from Makefile output * docs(test/s3/lifecycle): correct '-shards 0-15' wording * fix(s3/lifecycle): reject out-of-range shard IDs in Pipeline.Run The shell's parseShardsSpec already validates, but a programmatic caller (scheduler, future worker config) shouldn't be able to silently produce no-op states by passing -1 or 99. * fix(s3/lifecycle): bound drain + final-save with their own timeouts Shutdown was using context.Background, so a stuck dispatcher RPC or filer save could keep Pipeline.Run from ever returning. * fix(test/s3/lifecycle): drop self-killing pkill in stop-server The pkill pattern \"weed mini -dir=...\" is also in the running shell's argv (it's the recipe body), so pkill -f matches its own bash and the recipe exits with Terminated. CI test job passed but the cleanup step failed with exit 2. The PID file is sufficient on its own. * docs(test/s3/lifecycle): document S3_GRPC_ENDPOINT env var
642 lines
18 KiB
YAML
642 lines
18 KiB
YAML
name: "S3 Go Tests"
|
|
|
|
on:
|
|
pull_request:
|
|
|
|
concurrency:
|
|
group: ${{ github.head_ref }}/s3-go-tests
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
defaults:
|
|
run:
|
|
working-directory: weed
|
|
|
|
jobs:
|
|
s3-versioning-tests:
|
|
name: S3 Versioning Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 30
|
|
strategy:
|
|
matrix:
|
|
test-type: ["quick", "comprehensive"]
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Versioning Tests - ${{ matrix.test-type }}
|
|
timeout-minutes: 25
|
|
working-directory: test/s3/versioning
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
df -h
|
|
echo "=== Starting Tests ==="
|
|
|
|
# Run tests with automatic server management
|
|
# The test-with-server target handles server startup/shutdown automatically
|
|
if [ "${{ matrix.test-type }}" = "quick" ]; then
|
|
# Override TEST_PATTERN for quick tests only
|
|
make test-with-server TEST_PATTERN="TestBucketListReturnDataVersioning|TestVersioningBasicWorkflow|TestVersioningDeleteMarkers"
|
|
else
|
|
# Run all versioning tests
|
|
make test-with-server
|
|
fi
|
|
|
|
- name: Show server logs on failure
|
|
if: failure()
|
|
working-directory: test/s3/versioning
|
|
run: |
|
|
echo "=== Server Logs ==="
|
|
if [ -f weed-test.log ]; then
|
|
echo "Last 100 lines of server logs:"
|
|
tail -100 weed-test.log
|
|
else
|
|
echo "No server log file found"
|
|
fi
|
|
|
|
echo "=== Test Environment ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
netstat -tlnp | grep -E "(8333|9333|8080)" || true
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-versioning-test-logs-${{ matrix.test-type }}
|
|
path: test/s3/versioning/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-versioning-compatibility:
|
|
name: S3 Versioning Compatibility Test
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run Core Versioning Test (Python s3tests equivalent)
|
|
timeout-minutes: 15
|
|
working-directory: test/s3/versioning
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
|
|
# Run the specific test that is equivalent to the Python s3tests
|
|
make test-with-server || {
|
|
echo "❌ Test failed, checking logs..."
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Server logs ==="
|
|
tail -100 weed-test.log
|
|
fi
|
|
echo "=== Process information ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
exit 1
|
|
}
|
|
|
|
- name: Upload server logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-versioning-compatibility-logs
|
|
path: test/s3/versioning/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-cors-compatibility:
|
|
name: S3 CORS Compatibility Test
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run Core CORS Test (AWS S3 compatible)
|
|
timeout-minutes: 15
|
|
working-directory: test/s3/cors
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
|
|
# Run the specific test that is equivalent to AWS S3 CORS behavior
|
|
make test-with-server || {
|
|
echo "❌ Test failed, checking logs..."
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Server logs ==="
|
|
tail -100 weed-test.log
|
|
fi
|
|
echo "=== Process information ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
exit 1
|
|
}
|
|
|
|
- name: Upload server logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-cors-compatibility-logs
|
|
path: test/s3/cors/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-retention-tests:
|
|
name: S3 Retention Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 30
|
|
strategy:
|
|
matrix:
|
|
test-type: ["quick", "comprehensive"]
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Retention Tests - ${{ matrix.test-type }}
|
|
timeout-minutes: 25
|
|
working-directory: test/s3/retention
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
df -h
|
|
echo "=== Starting Tests ==="
|
|
|
|
# Run tests with automatic server management
|
|
# The test-with-server target handles server startup/shutdown automatically
|
|
if [ "${{ matrix.test-type }}" = "quick" ]; then
|
|
# Override TEST_PATTERN for quick tests only
|
|
make test-with-server TEST_PATTERN="TestBasicRetentionWorkflow|TestRetentionModeCompliance|TestLegalHoldWorkflow"
|
|
else
|
|
# Run all retention tests
|
|
make test-with-server
|
|
fi
|
|
|
|
- name: Show server logs on failure
|
|
if: failure()
|
|
working-directory: test/s3/retention
|
|
run: |
|
|
echo "=== Server Logs ==="
|
|
if [ -f weed-test.log ]; then
|
|
echo "Last 100 lines of server logs:"
|
|
tail -100 weed-test.log
|
|
else
|
|
echo "No server log file found"
|
|
fi
|
|
|
|
echo "=== Test Environment ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
netstat -tlnp | grep -E "(8333|9333|8080)" || true
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-retention-test-logs-${{ matrix.test-type }}
|
|
path: test/s3/retention/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-lifecycle-tests:
|
|
name: S3 Lifecycle Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 15
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Lifecycle Tests
|
|
timeout-minutes: 12
|
|
working-directory: test/s3/lifecycle
|
|
run: |
|
|
set -x
|
|
make test-with-server
|
|
|
|
- name: Show server logs on failure
|
|
if: failure()
|
|
working-directory: test/s3/lifecycle
|
|
run: |
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Last 200 lines of server logs ==="
|
|
tail -200 weed-test.log
|
|
fi
|
|
ps aux | grep -E "(weed|test)" || true
|
|
netstat -tlnp 2>/dev/null | grep -E "(8333|9333|8080|8888)" || true
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-lifecycle-test-logs
|
|
path: test/s3/lifecycle/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-checksum-tests:
|
|
name: S3 Checksum Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Checksum Tests
|
|
timeout-minutes: 16
|
|
working-directory: test/s3/checksum
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
df -h
|
|
echo "=== Starting Tests ==="
|
|
make test-with-server
|
|
|
|
- name: Show server logs on failure
|
|
if: failure()
|
|
working-directory: test/s3/checksum
|
|
run: |
|
|
echo "=== Server Logs ==="
|
|
if [ -f weed-test.log ]; then
|
|
echo "Last 100 lines of server logs:"
|
|
tail -100 weed-test.log
|
|
else
|
|
echo "No server log file found"
|
|
fi
|
|
|
|
echo "=== Test Environment ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
netstat -tlnp | grep -E "(8333|9333|8080)" || true
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-checksum-test-logs
|
|
path: test/s3/checksum/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-cors-tests:
|
|
name: S3 CORS Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 30
|
|
strategy:
|
|
matrix:
|
|
test-type: ["quick", "comprehensive"]
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 CORS Tests - ${{ matrix.test-type }}
|
|
timeout-minutes: 25
|
|
working-directory: test/s3/cors
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
df -h
|
|
echo "=== Starting Tests ==="
|
|
|
|
# Run tests with automatic server management
|
|
# The test-with-server target handles server startup/shutdown automatically
|
|
if [ "${{ matrix.test-type }}" = "quick" ]; then
|
|
# Override TEST_PATTERN for quick tests only
|
|
make test-with-server TEST_PATTERN="TestCORSConfigurationManagement|TestServiceLevelCORS|TestCORSBasicWorkflow"
|
|
else
|
|
# Run all CORS tests
|
|
make test-with-server
|
|
fi
|
|
|
|
- name: Show server logs on failure
|
|
if: failure()
|
|
working-directory: test/s3/cors
|
|
run: |
|
|
echo "=== Server Logs ==="
|
|
if [ -f weed-test.log ]; then
|
|
echo "Last 100 lines of server logs:"
|
|
tail -100 weed-test.log
|
|
else
|
|
echo "No server log file found"
|
|
fi
|
|
|
|
echo "=== Test Environment ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
netstat -tlnp | grep -E "(8333|9333|8080)" || true
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-cors-test-logs-${{ matrix.test-type }}
|
|
path: test/s3/cors/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-retention-worm:
|
|
name: S3 Retention WORM Integration Test
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run WORM Integration Tests
|
|
timeout-minutes: 15
|
|
working-directory: test/s3/retention
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
|
|
# Run the WORM integration tests with automatic server management
|
|
# The test-with-server target handles server startup/shutdown automatically
|
|
make test-with-server TEST_PATTERN="TestWORM|TestRetentionExtendedAttributes|TestRetentionConcurrentOperations" || {
|
|
echo "❌ WORM integration test failed, checking logs..."
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Server logs ==="
|
|
tail -100 weed-test.log
|
|
fi
|
|
echo "=== Process information ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
exit 1
|
|
}
|
|
|
|
- name: Upload server logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-retention-worm-logs
|
|
path: test/s3/retention/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-versioning-stress:
|
|
name: S3 Versioning Stress Test
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 35
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Versioning Stress Tests
|
|
timeout-minutes: 30
|
|
working-directory: test/s3/versioning
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
|
|
# Start server for stress tests
|
|
make start-server
|
|
|
|
# Run stress tests (concurrent operations)
|
|
make test-versioning-stress || {
|
|
echo "❌ Stress test failed, checking logs..."
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Server logs ==="
|
|
tail -200 weed-test.log
|
|
fi
|
|
make stop-server
|
|
make clean
|
|
exit 1
|
|
}
|
|
|
|
# Run pagination stress tests (>1000 versions)
|
|
echo "=== Running pagination stress tests ==="
|
|
make test-versioning-pagination-stress || {
|
|
echo "❌ Pagination stress test failed, checking logs..."
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Server logs ==="
|
|
tail -200 weed-test.log
|
|
fi
|
|
make stop-server
|
|
make clean
|
|
exit 1
|
|
}
|
|
|
|
make stop-server
|
|
make clean
|
|
|
|
- name: Upload stress test logs
|
|
if: always()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-versioning-stress-logs
|
|
path: test/s3/versioning/weed-test*.log
|
|
retention-days: 7
|
|
|
|
s3-tagging-tests:
|
|
# CI job for S3 object tagging tests
|
|
name: S3 Tagging Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Tagging Tests
|
|
timeout-minutes: 15
|
|
working-directory: test/s3/tagging
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
|
|
# Set environment variables for the test
|
|
export S3_ENDPOINT="http://localhost:8006"
|
|
export S3_ACCESS_KEY="0555b35654ad1656d804"
|
|
export S3_SECRET_KEY="h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q=="
|
|
export AWS_ACCESS_KEY_ID="$S3_ACCESS_KEY"
|
|
export AWS_SECRET_ACCESS_KEY="$S3_SECRET_KEY"
|
|
|
|
# Run the specific test that is equivalent to AWS S3 tagging behavior
|
|
make test-with-server || {
|
|
echo "❌ Test failed, checking logs..."
|
|
if [ -f weed-test.log ]; then
|
|
echo "=== Server logs ==="
|
|
tail -100 weed-test.log
|
|
fi
|
|
echo "=== Process information ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
exit 1
|
|
}
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-tagging-test-logs
|
|
path: test/s3/tagging/weed-test*.log
|
|
retention-days: 3
|
|
|
|
s3-remote-cache-tests:
|
|
name: S3 Remote Cache Tests
|
|
runs-on: ubuntu-22.04
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Check out code
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Set up Go
|
|
uses: actions/setup-go@v6
|
|
with:
|
|
go-version-file: 'go.mod'
|
|
id: go
|
|
|
|
- name: Install SeaweedFS
|
|
run: |
|
|
go install -buildvcs=false
|
|
|
|
- name: Run S3 Remote Cache Tests
|
|
timeout-minutes: 15
|
|
working-directory: test/s3/remote_cache
|
|
run: |
|
|
set -x
|
|
echo "=== System Information ==="
|
|
uname -a
|
|
free -h
|
|
|
|
# Run the remote cache integration tests
|
|
# Tests singleflight deduplication for caching remote objects
|
|
make test-with-server || {
|
|
echo "❌ Test failed, checking logs..."
|
|
if [ -f primary-weed.log ]; then
|
|
echo "=== Primary server logs ==="
|
|
tail -100 primary-weed.log
|
|
fi
|
|
if [ -f remote-weed.log ]; then
|
|
echo "=== Remote server logs ==="
|
|
tail -100 remote-weed.log
|
|
fi
|
|
echo "=== Process information ==="
|
|
ps aux | grep -E "(weed|test)" || true
|
|
exit 1
|
|
}
|
|
|
|
- name: Upload test logs on failure
|
|
if: failure()
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: s3-remote-cache-test-logs
|
|
path: |
|
|
test/s3/remote_cache/primary-weed.log
|
|
test/s3/remote_cache/remote-weed.log
|
|
retention-days: 3
|
|
|
|
# Removed SSE-C integration tests and compatibility job |