Files
seaweedfs/other
Rushikesh Deshpande df1a25fd3e feat: add Prometheus metrics for replication operations (#10006)
* feat: add Prometheus metrics for replication operations

Adds 5 metrics to instrument volume server replication (write/delete):
- Operations counter with success/failure labels
- Duration histogram for latency tracking
- Targets gauge for replica fanout
- Failures counter with error reason labels
- Under-replicated volumes gauge on master

* fix: record replication duration histogram only when replicaCount > 0

* fix: update replication targets gauge for all operations including zero

* fix: ensure symmetric replication success/failure counting and proper metrics updates

* fix: change VolumeServerReplicationTargets from Gauge to Histogram

- Replace .Set() with .Observe() in store_replicate.go (2 occurrences)
- Update test to use CollectAndCount for histogram assertion
- Rename TestReplicationTargetsGauge -> TestReplicationTargetsHistogram
- Update documentation to reflect Histogram type and PromQL examples

* Add comments to replication metrics and improve test coverage

* metrics: add replication panels to grafana dashboard

Master row gets an under-replicated volumes timeseries; Volume Servers
row gets replication operations, failures-by-reason, p99 duration, and
average fan-out panels for the new replication metrics.

* metrics: name the replication duration histogram replication_seconds

Match the volumeServer convention (request_seconds, vacuuming_seconds)
rather than the admin/lifecycle _duration_seconds spelling.

* metrics: guard replication fan-out panel against divide-by-zero

clamp_min the _count rate so the avg-targets ratio reads 0 instead of
NaN when there are no replication events in the window.

---------

Co-authored-by: Ubuntu User <ubuntu@example.com>
Co-authored-by: Chris Lu <chris.lu@gmail.com>
2026-06-19 11:05:43 -07:00
..