mirrors/seaweedfs
1
0
Fork 0
mirror of https://github.com/seaweedfs/seaweedfs.git synced 2026-08-28 11:56:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
e482e67971c91e60c3e2e5b219fec146e9d994bc
seaweedfs/test/tus
T
History
Chris LuandGitHub 44115c1051 filer: stop TUS uploads from turning into garbage (#10945) 
* filer: store TUS sub-chunks through the regular chunk writer

A TUS sub-chunk was written with one assigned file id, retried up to
three times against that same id, and abandoned on failure: an attempt
that had landed on some replicas left a needle no session record and no
entry ever references, unreclaimable by vacuum.

dataToChunkWithSSE, which the regular write path uses per chunk, assigns
a fresh file id per attempt and hands back the file ids of failed
attempts, which are now freed the way the regular write path frees them.

* filer: retry a chunk write on a fresh volume when the server 5xxs

The filer's chunk writer assigns a fresh file id per attempt but only
retried transient network errors, so a volume filling up and turning
read-only mid-write failed the whole request even though the very next
assignment would have landed elsewhere. Every other write client already
routes this through ShouldReassignUpload; the filer's own write path now
does the same, for regular uploads and TUS sub-chunks alike.

* filer: export the chunk deletion queue

The filer test harness in weed/server builds filer.Filer as a struct
literal, so any code path reaching DeleteChunks dereferenced a nil
queue. Exported like the neighboring DeletionRetryQueue so the harness
can arm it.

* filer: complete a TUS upload whose chunk records overlap

A PATCH retried while its predecessor was still storing a sub-chunk -
a proxy timeout with an immediate retry is enough - records the same
range twice. HEAD computes Upload-Offset as the covered watermark and
reported the upload fully received, but completion demanded exactly
adjacent records and failed every attempt: the client concluded success
from offset == length, no entry was created, and the session eventually
expired, turning the entire upload into deleted needles for the vacuum
to chew through.

Completion now validates gapless coverage with the same watermark HEAD
uses. A record extending coverage joins the entry - the read path
resolves partial overlaps by ModifiedTsNs, and the raced copies carry
identical bytes - while a fully covered duplicate is freed once the
entry lands.

* filer: allow one mutating TUS request per session at a time

Nothing stopped two PATCHes from writing the same range concurrently:
both loaded the same offset, both passed the conflict check, and both
recorded their sub-chunks. A client whose request timed out in a proxy
retries immediately while the server side is still storing the buffered
sub-chunk, which is exactly that race.

A session now accepts one PATCH or DELETE at a time, the way tusd locks
uploads; a concurrent one is refused with 423 Locked, which TUS clients
retry, and HEAD keeps answering so progress polling is unaffected. The
chunk state is loaded under the claim, so a retried PATCH sees every
record its predecessor left and conflicts cleanly instead of duplicating
data.

* test: cover a TUS PATCH raced by its own retry

Stalls a PATCH mid-body over a raw connection, retries the same range
while it is in flight, and expects the retry refused with 423 Locked;
the upload then resumes from the reported offset and the final content
must be intact.

* filer: never free a TUS duplicate the entry still references

Coverage is computed from ranges, so a record fully covered by another
is treated as a duplicate no matter which needle it names. A malformed
record naming a file id the entry keeps would have had that needle freed
right after the entry landed - the corruption this change set exists to
stop. The duplicates are now freed in one batch, skipping any file id
the entry references; their records go with the session directory.

* test: bound the raw TUS connection reads

http.ReadResponse on the stalled PATCH's connection blocked until the
whole go test timeout if the filer never answered.

* filer: free the needles of chunk write attempts a retry replaced

A volume server stores the needle locally and only then fans out to the
replicas, so a replication failure 5xxs with the data already written.
Each attempt assigns its own file id, so once a later attempt lands
elsewhere nothing references the earlier ones: the caller only sees the
chunk that succeeded, and the failed ids were dropped.

They are now freed the way the caller frees them when the whole write
fails. Retrying on a 5xx makes this reachable on every read-only or full
volume, which is exactly the condition that filled the reporter's
volumes.
2026-08-25 09:24:51 -07:00
..
Makefile
Add TUS protocol support for resumable uploads (#7592)
2025-12-14 21:56:07 -08:00
README.md
Add TUS protocol support for resumable uploads (#7592)
2025-12-14 21:56:07 -08:00
tus_integration_test.go
filer: stop TUS uploads from turning into garbage (#10945)
2026-08-25 09:24:51 -07:00

README.md

TUS Protocol Integration Tests

This directory contains integration tests for the TUS (resumable upload) protocol support in SeaweedFS Filer.

Overview

TUS is an open protocol for resumable file uploads over HTTP. It allows clients to upload files in chunks and resume uploads after network failures or interruptions.

Why TUS?

  • Resumable uploads: Resume interrupted uploads without re-sending data
  • Chunked uploads: Upload large files in smaller pieces
  • Simple protocol: Standard HTTP methods with custom headers
  • Wide client support: Libraries available for JavaScript, Python, Go, and more

TUS Protocol Endpoints

Method Path Description
OPTIONS /.tus/ Server capability discovery
POST /.tus/{path} Create new upload session
HEAD /.tus/.uploads/{id} Get current upload offset
PATCH /.tus/.uploads/{id} Upload data at offset
DELETE /.tus/.uploads/{id} Cancel upload

TUS Headers

Request Headers:

  • Tus-Resumable: 1.0.0 - Protocol version (required)
  • Upload-Length - Total file size in bytes (required on POST)
  • Upload-Offset - Current byte offset (required on PATCH)
  • Upload-Metadata - Base64-encoded key-value pairs (optional)
  • Content-Type: application/offset+octet-stream (required on PATCH)

Response Headers:

  • Tus-Resumable - Protocol version
  • Tus-Version - Supported versions
  • Tus-Extension - Supported extensions
  • Tus-Max-Size - Maximum upload size
  • Upload-Offset - Current byte offset
  • Location - Upload URL (on POST)

Enabling TUS

TUS protocol support is enabled by default at /.tus path. You can customize the path using the -tusBasePath flag:

# Start filer with default TUS path (/.tus)
weed filer -master=localhost:9333

# Use a custom path (leading slash added automatically if missing)
weed filer -master=localhost:9333 -tusBasePath=/.uploads/tus

# Disable TUS by setting empty path
weed filer -master=localhost:9333 -tusBasePath=

Test Structure

Integration Tests

The tests cover:

  1. Basic Functionality

    • TestTusOptionsHandler - Capability discovery
    • TestTusBasicUpload - Simple complete upload
    • TestTusCreationWithUpload - Creation-with-upload extension

  2. Chunked Uploads

    • TestTusChunkedUpload - Upload in multiple chunks

  3. Resumable Uploads

    • TestTusHeadRequest - Offset tracking
    • TestTusResumeAfterInterruption - Resume after failure

  4. Error Handling

    • TestTusInvalidOffset - Offset mismatch (409 Conflict)
    • TestTusUploadNotFound - Missing upload (404 Not Found)
    • TestTusDeleteUpload - Upload cancellation

Running Tests

Prerequisites

  1. Build SeaweedFS:
make build-weed
# or
cd ../../weed && go build -o weed

Using Makefile

# Show available targets
make help

# Run all tests with automatic server management
make test-with-server

# Run all tests (requires running server)
make test

# Run specific test categories
make test-basic      # Basic upload tests
make test-chunked    # Chunked upload tests
make test-resume     # Resume/HEAD tests
make test-errors     # Error handling tests

# Manual testing
make manual-start    # Start SeaweedFS for manual testing
make manual-stop     # Stop and cleanup

Using Go Test Directly

# Run all TUS tests
go test -v ./test/tus/...

# Run specific test
go test -v ./test/tus -run TestTusBasicUpload

# Skip integration tests (short mode)
go test -v -short ./test/tus/...

Debug

# View server logs
make debug-logs

# Check process and port status
make debug-status

Test Environment

Each test run:

  1. Starts a SeaweedFS cluster (master, volume, filer)
  2. Creates uploads using TUS protocol
  3. Verifies files are stored correctly
  4. Cleans up test data

Default Ports

Service Port
Master 19333
Volume 18080
Filer 18888

Configuration

Override defaults via environment or Makefile variables:

FILER_PORT=8889 MASTER_PORT=9334 make test

Example Usage

Create Upload

curl -X POST http://localhost:18888/.tus/mydir/file.txt \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Length: 1000" \
  -H "Upload-Metadata: filename dGVzdC50eHQ="

Upload Data

curl -X PATCH http://localhost:18888/.tus/.uploads/{upload-id} \
  -H "Tus-Resumable: 1.0.0" \
  -H "Upload-Offset: 0" \
  -H "Content-Type: application/offset+octet-stream" \
  --data-binary @file.txt

Check Offset

curl -I http://localhost:18888/.tus/.uploads/{upload-id} \
  -H "Tus-Resumable: 1.0.0"

Cancel Upload

curl -X DELETE http://localhost:18888/.tus/.uploads/{upload-id} \
  -H "Tus-Resumable: 1.0.0"

TUS Extensions Supported

  • creation: Create new uploads with POST
  • creation-with-upload: Send data in creation request
  • termination: Cancel uploads with DELETE

Architecture

Client                         Filer                      Volume Servers
  |                              |                              |
  |-- POST /.tus/path/file.mp4 ->|                              |
  |                              |-- Create session dir ------->|
  |<-- 201 Location: /.../{id} --|                              |
  |                              |                              |
  |-- PATCH /.tus/.uploads/{id} >|                              |
  |   Upload-Offset: 0           |-- Assign volume ------------>|
  |   [chunk data]               |-- Upload chunk ------------->|
  |<-- 204 Upload-Offset: N -----|                              |
  |                              |                              |
  |   (network failure)          |                              |
  |                              |                              |
  |-- HEAD /.tus/.uploads/{id} ->|                              |
  |<-- Upload-Offset: N ---------|                              |
  |                              |                              |
  |-- PATCH (resume) ----------->|-- Upload remaining -------->|
  |<-- 204 (complete) -----------|-- Assemble final file ----->|

Comparison with S3 Multipart

Feature TUS S3 Multipart
Protocol Custom HTTP headers S3 API
Session Init POST with Upload-Length CreateMultipartUpload
Upload Data PATCH with offset UploadPart with partNumber
Resume HEAD to get offset ListParts
Complete Automatic at final offset CompleteMultipartUpload
Ordering Sequential (offset-based) Parallel (part numbers)

Related Resources

Reference in New Issue View Git Blame Copy Permalink