mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-07-25 01:22:39 +00:00
490379bff3
* Add GitHub Actions workflow for codespell on master * Add rudimentary codespell config * Tune codespell config: skip generated code, ignore camelCase, whitelist domain terms Add camelCase/PascalCase regex to ignore common Go/Rust/JS identifiers like allLocations, publishErr, ReadInside, FlushInterval. Also skip templ-generated *_templ.go files, and whitelist a handful of short/domain-specific words (visibles, fo, te, ser, bject, unparseable, keep-alives, tread, anc, ue) that show up as false positives across the tree. Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix ambiguous typos and protect false positives Fixes typos that codespell reports with multiple candidate suggestions (so `codespell -w` cannot auto-apply them), plus one inline pragma and one config entry to protect legitimate identifiers. Manual fixes (single correct answer chosen from context): - pattens -> patterns (5x) in filer/upload/shell flag help strings - finded -> found (2x) in tarantool storage.lua comment - spacify -> specify (2x) in helm chart values.yaml comment - wether -> whether in skiplist.go docstring - simpe -> simple in mq schema test case name False-positive protection: - Add `//codespell:ignore` next to `source GET's` (possessive of HTTP verb) in s3api_object_handlers_copy_stream.go - Whitelist `auther` in .codespellrc — it's a local variable meaning "authenticator" in weed/security/tls.go, not a typo of "author". Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Extend codespell ignore list: .git-meta path and thirdparty groupId Also skip `.git-meta` (scratch dir for commit messages that may contain typo words verbatim) and whitelist `thirdparty` — it appears as the literal Maven groupId `org.apache.hadoop.thirdparty` in hdfs3 poms and cannot be renamed. Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [DATALAD RUNCMD] Fix non-ambiguous typos with codespell -w Auto-applied fixes to the 44 remaining single-suggestion typos across docs, comments, log messages, tests, config, and one Java pom. === Do not change lines below === { "chain": [], "cmd": "uvx codespell -w", "exit": 0, "extra_inputs": [], "inputs": [], "outputs": [], "pwd": "." } ^^^ Do not change lines above ^^^ * Revert breaking codespell fixes; whitelist unknwon and atleast Two of the auto-applied `codespell -w` fixes were false positives that would break the build/tests: - go.mod: `github.com/unknwon/goconfig` is a real Go module path — the upstream author's GitHub handle is literally `unknwon`. Renaming to `unknown` would fail dependency resolution. - test/benchmark/fuse_db/bin/{sqlite_verify.py,run_mysql.sh,run_sqlite.sh}: `atleast` is a literal CLI mode value (a string constant compared and passed as a positional argument). Rewriting to `at least` splits it into two arguments and breaks the mode check. Reverted those files and whitelisted both words in .codespellrc so future runs won't re-suggest the same broken fixes. Co-Authored-By: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Code 2.1.217 / Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SeaweedFS S3 Java SDK Compatibility Tests
This project contains Java-based integration tests for SeaweedFS S3 API compatibility.
Overview
Tests are provided for both AWS SDK v1 and v2 to ensure compatibility with the various SDK versions commonly used in production.
SDK Versions
| SDK | Version | Notes |
|---|---|---|
| AWS SDK v1 for Java | 1.12.600 | Legacy SDK, less strict ETag validation |
| AWS SDK v2 for Java | 2.20.127 | Modern SDK with strict checksum validation |
Running Tests
Prerequisites
-
SeaweedFS running with S3 API enabled:
weed server -s3 -
Java 18+ and Maven
Run All Tests
mvn test
Run Specific Tests
# Run only ETag validation tests (AWS SDK v2)
mvn test -Dtest=ETagValidationTest
# Run with custom endpoint
mvn test -Dtest=ETagValidationTest -DS3_ENDPOINT=http://localhost:8333
Environment Variables
| Variable | Default | Description |
|---|---|---|
S3_ENDPOINT |
http://127.0.0.1:8333 |
S3 API endpoint URL |
S3_ACCESS_KEY |
some_access_key1 |
Access key ID |
S3_SECRET_KEY |
some_secret_key1 |
Secret access key |
S3_REGION |
us-east-1 |
AWS region |
Test Coverage
ETagValidationTest (AWS SDK v2)
Tests for GitHub Issue #7768 - ETag format validation.
| Test | Description |
|---|---|
testSmallFilePutObject |
Verify small files return pure MD5 ETag |
testLargeFilePutObject_Issue7768 |
Critical: Verify large files (>8MB) return pure MD5 ETag |
testExtraLargeFilePutObject |
Verify very large files (>24MB) return pure MD5 ETag |
testMultipartUploadETag |
Verify multipart uploads return composite ETag |
testETagConsistency |
Verify ETag consistency across PUT/HEAD/GET |
testMultipleLargeFileUploads |
Stress test multiple large uploads |
Background: Issue #7768
AWS SDK v2 for Java includes checksum validation that decodes the ETag as hexadecimal. When SeaweedFS returned composite ETags (<md5>-<count>) for regular PutObject with internally auto-chunked files, the SDK failed with:
java.lang.IllegalArgumentException: Invalid base 16 character: '-'
Per AWS S3 specification:
PutObject: ETag is always a pure MD5 hex string (32 chars)CompleteMultipartUpload: ETag is composite format (<md5>-<partcount>)
The fix ensures SeaweedFS follows this specification.
Project Structure
src/
├── main/java/com/seaweedfs/s3/
│ ├── PutObject.java # Example PutObject with SDK v1
│ └── HighLevelMultipartUpload.java
└── test/java/com/seaweedfs/s3/
├── PutObjectTest.java # Basic SDK v1 test
└── ETagValidationTest.java # Comprehensive SDK v2 ETag tests
Validated SDK Versions
This Java test project validates:
- ✅ AWS SDK v2 for Java 2.20.127+
- ✅ AWS SDK v1 for Java 1.12.600+
Go SDK validation is performed by separate test suites:
- See Go ETag Tests for AWS SDK v2 for Go tests
- See test/s3/SDK_COMPATIBILITY.md for full SDK compatibility matrix