mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-18 13:17:08 +00:00
docs(note): add production-setup slide deck
Marp-based markdown deck walking through the three-layer production topology (masters, volumes, filers + DB, S3 gateways, admin + workers) plus an erasure-coding note. Uses the object-store-layout diagram on the overview slide. Makefile renders PDF/HTML/PPTX via marp-cli.
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
MARP ?= npx @marp-team/marp-cli
|
||||
SRC := seaweedfs-production-setup.md
|
||||
PDF := $(SRC:.md=.pdf)
|
||||
HTML := $(SRC:.md=.html)
|
||||
PPTX := $(SRC:.md=.pptx)
|
||||
FLAGS := --allow-local-files
|
||||
|
||||
.PHONY: all pdf html pptx watch clean
|
||||
|
||||
all: pdf
|
||||
|
||||
pdf: $(PDF)
|
||||
|
||||
html: $(HTML)
|
||||
|
||||
pptx: $(PPTX)
|
||||
|
||||
$(PDF): $(SRC)
|
||||
$(MARP) $(SRC) --pdf $(FLAGS) -o $@
|
||||
|
||||
$(HTML): $(SRC)
|
||||
$(MARP) $(SRC) --html $(FLAGS) -o $@
|
||||
|
||||
$(PPTX): $(SRC)
|
||||
$(MARP) $(SRC) --pptx $(FLAGS) -o $@
|
||||
|
||||
watch:
|
||||
$(MARP) $(SRC) --watch --html $(FLAGS)
|
||||
|
||||
clean:
|
||||
rm -f $(PDF) $(HTML) $(PPTX)
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 738 KiB |
@@ -0,0 +1,118 @@
|
||||
---
|
||||
marp: true
|
||||
theme: default
|
||||
paginate: true
|
||||
---
|
||||
|
||||
# SeaweedFS Production Setup
|
||||
|
||||
A walkthrough of the recommended deployment topology
|
||||
|
||||
- Storage layer
|
||||
- File access layer
|
||||
- Backend operations
|
||||
- Erasure coding for durability
|
||||
|
||||
---
|
||||
|
||||

|
||||
|
||||
---
|
||||
|
||||
# Storage Layer
|
||||
|
||||
**3 Masters + Volume Servers**
|
||||
|
||||
- **Masters (run 3)**
|
||||
- Form a Raft quorum; tolerate 1 failure
|
||||
- Track volume locations and cluster topology
|
||||
- Lightweight; not on the data path for reads/writes after lookup
|
||||
|
||||
- **Volume Servers (scale out)**
|
||||
- Hold the actual needle data on local disks
|
||||
- Add more servers for capacity and throughput
|
||||
- Heartbeat to masters; clients talk to them directly for I/O
|
||||
|
||||
Rule of thumb: start with 3 masters, then grow volume servers with data.
|
||||
|
||||
---
|
||||
|
||||
# File Access Layer
|
||||
|
||||
**Filer + DB for metadata**
|
||||
|
||||
- Filer serves POSIX-like directory and file metadata
|
||||
- Metadata store options: PostgreSQL, MySQL, Redis, Cassandra, TiKV, etc.
|
||||
- Run **3 filers** for HA
|
||||
- Filers are stateless relative to each other — state lives in the DB
|
||||
|
||||
**S3 Servers**
|
||||
|
||||
- Expose the S3 API on top of the filer
|
||||
- Scale the count based on throughput requirements
|
||||
- Can be co-located with filers or run independently
|
||||
|
||||
---
|
||||
|
||||
# Backend Operations
|
||||
|
||||
**1 Admin Server + a few Workers**
|
||||
|
||||
- **Admin server (1 is enough)**
|
||||
- Coordinates background jobs: balancing, EC encoding, vacuum, replication fixes
|
||||
- **Not on the data path** — a restart does not affect object store reads/writes
|
||||
- No HA required for normal operation
|
||||
|
||||
- **Workers (a few)**
|
||||
- Execute long-running tasks issued by the admin server
|
||||
- Horizontally scalable based on maintenance workload
|
||||
|
||||
Keeps housekeeping off the hot path.
|
||||
|
||||
---
|
||||
|
||||
# Production Topology Summary
|
||||
|
||||
| Component | Count | Role |
|
||||
|----------------|----------------------|-----------------------------------|
|
||||
| Master | 3 | Raft quorum, topology, volume map |
|
||||
| Volume Server | N (scale out) | Stores data needles |
|
||||
| Filer | 3 | Metadata gateway |
|
||||
| Metadata DB | HA cluster | Persists filer metadata |
|
||||
| S3 Server | N (by throughput) | S3 API frontend |
|
||||
| Admin | 1 | Background job coordinator |
|
||||
| Worker | A few | Executes admin-scheduled tasks |
|
||||
|
||||
---
|
||||
|
||||
# Erasure Coding for Durability
|
||||
|
||||
Prefer **many volume servers** so EC shards spread across failure domains.
|
||||
|
||||
**Example: 8 volume servers, EC 5+3**
|
||||
|
||||
- Each volume becomes 5 data shards + 3 parity shards
|
||||
- Shards are placed on 8 distinct volume servers
|
||||
- **Tolerates up to 3 simultaneous volume server failures**
|
||||
- Storage overhead: 1.6x vs 3x for full replication
|
||||
|
||||
```
|
||||
Volume -> [D1][D2][D3][D4][D5] + [P1][P2][P3]
|
||||
| | | | | | | |
|
||||
VS1 VS2 VS3 VS4 VS5 VS6 VS7 VS8
|
||||
```
|
||||
|
||||
More volume servers = more EC layouts available (e.g. 10+4, 6+3).
|
||||
|
||||
---
|
||||
|
||||
# Getting Started Checklist
|
||||
|
||||
1. Provision 3 master nodes (small boxes are fine)
|
||||
2. Provision volume servers sized to your data footprint
|
||||
3. Stand up an HA metadata DB, then 3 filers pointing at it
|
||||
4. Add S3 servers sized to required throughput
|
||||
5. Run 1 admin + a few workers for background maintenance
|
||||
6. Once you have enough volume servers, enable EC (e.g. 5+3 on 8 servers)
|
||||
|
||||
Scale each layer independently as usage grows.
|
||||
Reference in New Issue
Block a user