mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-09-07 16:46:59 +00:00
Chris LuandGitHub
f720f559cb
* ci(kafka-loadtest): retry apt-get to survive Ubuntu mirror flakes
The Kafka Quick Test workflow's Docker build of Dockerfile.loadtest
keeps hitting "Connection failed [IP: ...]" on archive.ubuntu.com /
security.ubuntu.com mid-build, e.g.:
failed to solve: process "/bin/sh -c apt-get update && \
apt-get install -y ca-certificates curl jq bash netcat ..."
did not complete successfully: exit code: 100
Same class of failure PR #9106 fixed for pjdfstest. Apply the same
two retry knobs to Dockerfile.loadtest and Dockerfile.seektest so a
transient mirror flake retries five times with a 30s timeout instead
of failing the whole workflow.
(The fuller pjdfstest fix also restructured the build to use
docker/build-push-action with type=gha cache. Doing that for
kafka-client-loadtest would mean rewriting the make/docker-compose
build path; defer until the apt-retry alone proves insufficient.)
* ci(kafka-loadtest): also drop recommends/suggests + apt-get clean
Address PR review (gemini-code-assist): fully align with PR #9106's
pattern by adding --no-install-recommends / --no-install-suggests so
the runtime images stay small and don't pull in extra packages, plus
apt-get clean before rm -rf /var/lib/apt/lists/* in Dockerfile.seektest.
* ci(kafka-loadtest): use alpine / maven base images instead of apt
The previous rounds of apt-retry / apt-clean knobs aren't enough:
the Ubuntu mirror is persistently unreachable from the GitHub runner
for minutes at a time, which blows past the 5-retry / 30-second
Acquire configuration and still kills the build (see run 24551809614).
Switch both runtime images so no apt fetch is needed at all:
- Dockerfile.loadtest now runs on alpine:3.20. All runtime deps
(ca-certificates, curl, jq, bash, netcat-openbsd) are in the
Alpine main repo, fetched from Alpine's CDN rather than the
Ubuntu archive that keeps going dark.
- Dockerfile.seektest now uses maven:3.9-eclipse-temurin-11, which
ships JDK 11 and Maven preinstalled — no apt-get maven step.
This also means the runtime images no longer care about
Acquire::Retries / DEBIAN_FRONTEND / apt-get clean, so those lines
are removed with the apt call they were configuring.
Kafka Gateway Tests with SMQ Integration
This directory contains tests for the SeaweedFS Kafka Gateway with full SeaweedMQ (SMQ) integration.
Test Types
Unit Tests (./unit/)
- Basic gateway functionality
- Protocol compatibility
- No SeaweedFS backend required
- Uses mock handlers
Integration Tests (./integration/)
- Mock Mode (default): Uses in-memory handlers for protocol testing
- SMQ Mode (with
SEAWEEDFS_MASTERS): Uses real SeaweedFS backend for full integration
E2E Tests (./e2e/)
- End-to-end workflows
- Automatically detects SMQ availability
- Falls back to mock mode if SMQ unavailable
Running Tests Locally
Quick Protocol Testing (Mock Mode)
# Run all integration tests with mock backend
cd test/kafka
go test ./integration/...
# Run specific test
go test -v ./integration/ -run TestClientCompatibility
Full Integration Testing (SMQ Mode)
Requires running SeaweedFS instance:
- Start SeaweedFS with MQ support:
# Terminal 1: Start SeaweedFS server
weed server -ip="127.0.0.1" -ip.bind="0.0.0.0" -dir=/tmp/seaweedfs-data -master.port=9333 -volume.port=8081 -filer.port=8888 -filer=true -master.peers=none
# Terminal 2: Start MQ broker
weed mq.broker -master="127.0.0.1:9333" -ip="127.0.0.1" -port=17777
- Run tests with SMQ backend:
cd test/kafka
SEAWEEDFS_MASTERS=127.0.0.1:9333 go test ./integration/...
# Run specific SMQ integration tests
SEAWEEDFS_MASTERS=127.0.0.1:9333 go test -v ./integration/ -run TestSMQIntegration
Test Broker Startup
If you're having broker startup issues:
# Debug broker startup locally
./scripts/test-broker-startup.sh
CI/CD Integration
GitHub Actions Jobs
- Unit Tests - Fast protocol tests with mock backend
- Integration Tests - Mock mode by default
- E2E Tests (with SMQ) - Full SeaweedFS + MQ broker stack
- Client Compatibility (with SMQ) - Tests different Kafka clients against real backend
- Consumer Group Tests (with SMQ) - Tests consumer group persistence
- SMQ Integration Tests - Dedicated SMQ-specific functionality tests
What Gets Tested with SMQ
When SEAWEEDFS_MASTERS is available, tests exercise:
- Real Message Persistence - Messages stored in SeaweedFS volumes
- Offset Persistence - Consumer group offsets stored in SeaweedFS filer
- Topic Persistence - Topic metadata persisted in SeaweedFS filer
- Consumer Group Coordination - Distributed coordinator assignment
- Cross-Client Compatibility - Sarama, kafka-go with real backend
- Broker Discovery - Gateway discovers MQ brokers via masters
Test Infrastructure
testutil.NewGatewayTestServerWithSMQ(t, mode)
Smart gateway creation that automatically:
- Detects SMQ availability via
SEAWEEDFS_MASTERS - Uses production handler when available
- Falls back to mock when unavailable
- Provides timeout protection against hanging
Modes:
SMQRequired- Skip test if SMQ unavailableSMQAvailable- Use SMQ if available, otherwise mockSMQUnavailable- Always use mock
Timeout Protection
Gateway creation includes timeout protection to prevent CI hanging:
- 20 second timeout for
SMQRequiredmode - 15 second timeout for
SMQAvailablemode - Clear error messages when broker discovery fails
Debugging Failed Tests
CI Logs to Check
- "SeaweedFS master is up" - Master started successfully
- "SeaweedFS filer is up" - Filer ready
- "SeaweedFS MQ broker is up" - Broker started successfully
- Broker/Server logs - Shown on broker startup failure
Local Debugging
- Run
./scripts/test-broker-startup.shto test broker startup - Check logs at
/tmp/weed-*.log - Test individual components:
# Test master curl http://127.0.0.1:9333/cluster/status # Test filer curl http://127.0.0.1:8888/status # Test broker nc -z 127.0.0.1 17777
Common Issues
- Broker fails to start: Check filer is ready before starting broker
- Gateway timeout: Broker discovery fails, check broker is accessible
- Test hangs: Timeout protection not working, reduce timeout values
Architecture
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Kafka Client │───▶│ Kafka Gateway │───▶│ SeaweedMQ Broker│
│ (Sarama, │ │ (Protocol │ │ (Message │
│ kafka-go) │ │ Handler) │ │ Persistence) │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ SeaweedFS Filer │ │ SeaweedFS Master│
│ (Offset Storage)│ │ (Coordination) │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────┐
│ SeaweedFS Volumes │
│ (Message Storage) │
└─────────────────────────────────────────┘
This architecture ensures full integration testing of the entire Kafka → SeaweedFS message path.