metrics: guard time()-based dashboard panels against unset gauges (#9965)

* metrics: guard time()-based panels against unset (zero) gauges

Several dashboard panels compute "time() - <gauge>" (uptime, time-since-
last-scrub, lifecycle cursor lag, last daily walk). When the underlying
gauge is unset it reports 0, so the panel rendered ~56 years (time since
the Unix epoch). This is the common case for "Volume Server Uptime": the
Rust volume server doesn't set SeaweedFS_volumeServer_start_time_seconds,
and Go components expose it registered-but-zero.

Guard each such expression with "> 0" so unset/zero series drop out (the
panel shows no data) instead of rendering a nonsensical epoch value.
Affected panels: Master Uptime, Volume Server Uptime, Time Since Last
Scrub, Lifecycle Cursor Lag, Time Since Last Daily Walk.

* metrics: guard Filer Sync Offset Lag and Metadata Subscription Lag panels too

Extend the > 0 guard to the two remaining time()-<gauge> panels that
share the same unset/zero failure mode (filerSync sync offset and the
metadata subscribe last-send timestamp), so they don't render ~56-year
lag when the gauge is absent.
This commit is contained in:
Chris Lu
2026-06-14 14:41:18 -07:00
committed by GitHub
parent d47cc45b1f
commit 0e9f702152
+7 -7
View File
@@ -1750,7 +1750,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - SeaweedFS_master_start_time_seconds{cluster=~\"$cluster\"}",
"expr": "time() - (SeaweedFS_master_start_time_seconds{cluster=~\"$cluster\"} > 0)",
"range": true,
"refId": "A",
"legendFormat": "{{instance}}"
@@ -3585,7 +3585,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - max by (mode) (SeaweedFS_volumeServer_scrub_last_time_seconds{cluster=~\"$cluster\"})",
"expr": "time() - max by (mode) (SeaweedFS_volumeServer_scrub_last_time_seconds{cluster=~\"$cluster\"} > 0)",
"range": true,
"refId": "A",
"legendFormat": "{{mode}}"
@@ -3681,7 +3681,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - SeaweedFS_volumeServer_start_time_seconds{cluster=~\"$cluster\"}",
"expr": "time() - (SeaweedFS_volumeServer_start_time_seconds{cluster=~\"$cluster\"} > 0)",
"range": true,
"refId": "A",
"legendFormat": "{{instance}}"
@@ -5064,7 +5064,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - SeaweedFS_filerSync_sync_offset{cluster=~\"$cluster\"} / 1e9",
"expr": "time() - (SeaweedFS_filerSync_sync_offset{cluster=~\"$cluster\"} > 0) / 1e9",
"range": true,
"refId": "A",
"legendFormat": "{{clientName}} {{path}}"
@@ -5160,7 +5160,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - max by (clientName, path) (SeaweedFS_filer_last_send_timestamp_of_subscribe{cluster=~\"$cluster\"}) / 1e9",
"expr": "time() - max by (clientName, path) (SeaweedFS_filer_last_send_timestamp_of_subscribe{cluster=~\"$cluster\"} > 0) / 1e9",
"range": true,
"refId": "A",
"legendFormat": "{{clientName}} {{path}}"
@@ -6804,7 +6804,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - max by (shard) (SeaweedFS_s3_lifecycle_cursor_min_ts_ns{cluster=~\"$cluster\"}) / 1e9",
"expr": "time() - max by (shard) (SeaweedFS_s3_lifecycle_cursor_min_ts_ns{cluster=~\"$cluster\"} > 0) / 1e9",
"range": true,
"refId": "A",
"legendFormat": "shard {{shard}}"
@@ -7199,7 +7199,7 @@
"uid": "${DS_PROMETHEUS}"
},
"editorMode": "code",
"expr": "time() - max by (shard) (SeaweedFS_s3_lifecycle_daily_run_last_walked_ns{cluster=~\"$cluster\"}) / 1e9",
"expr": "time() - max by (shard) (SeaweedFS_s3_lifecycle_daily_run_last_walked_ns{cluster=~\"$cluster\"} > 0) / 1e9",
"range": true,
"refId": "A",
"legendFormat": "shard {{shard}}"