Remove the unused collectGaugeCount helper and assert specific label
values survive after each deletion using testutil.ToFloat64.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Replace blanket Reset() with targeted DeleteLabelValues to avoid
briefly wiping metrics for schedules that still exist. Track known
schedules in a set and only delete stale entries on each resync.
Extract the wait.Until closure into resyncBackupMetrics() so tests
can call it directly without goroutine timing. Replace hand-rolled
collectGaugeCount helper with testutil.CollectAndCount.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Remove the exported method that was only used in tests. Use the
existing Metrics() getter to access the gauge directly in the
backup controller test instead.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
The periodic backup metrics resync in updateTotalBackupMetric only set
backupLastSuccessfulTimestamp values but never removed stale entries.
When a schedule was deleted and its backups removed, the gauge persisted
until the Velero pod was restarted.
Reset the gauge before re-setting current values so that deleted
schedules are pruned automatically each resync cycle.
Fixes#9239
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Add a method to reset all backupLastSuccessfulTimestamp gauge values.
This will be used by the backup controller's periodic resync to prune
stale metrics for deleted schedules.
Fixes#9239
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Add a new Prometheus gauge metric that exposes the expected interval
between consecutive scheduled backups. This enables dynamic alerting
thresholds per schedule backups.
Signed-off-by: Quang Ngo <quang.ngo@canonical.com>
Explain that the duration histogram tracks distribution of individual
job durations, not accumulated sums, to address reviewer concerns.
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
- Rename metric constants from maintenance_job_* to repo_maintenance_*
- Update metric help text to clarify these are for repo maintenance
- Rename functions: RegisterMaintenanceJob* → RegisterRepoMaintenance*
- Update all test references to use new names
Addresses review comments from @Lyndon-Li on PR #9414
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
Adds three new Prometheus metrics to track backup repository
maintenance job execution:
- velero_maintenance_job_success_total: Counter for successful jobs
- velero_maintenance_job_failure_total: Counter for failed jobs
- velero_maintenance_job_duration_seconds: Histogram for job duration
Metrics use repository_name label to identify specific BackupRepositories.
Duration is recorded for both successful and failed jobs (when job runs),
but not when job fails to start.
Includes comprehensive unit and integration tests.
Fixes#9225
Signed-off-by: Shubham Pampattiwar <spampatt@redhat.com>
* update revision of go-hclog to match go.mod requirement
Signed-off-by: Steve Kriss <krisss@vmware.com>
* update prometheus dep to prepare for go module migration
Signed-off-by: Steve Kriss <krisss@vmware.com>
rename variables to reflect the metric name
fix comments for exported methods
explicitly record per schedule per schedule metric values
initialize metrics and change variable name to match with that of metric
add metric for recording failed volume snapshots
use singular variable instead of plural
remove extra field for failed snapshots, calculate using existing fields
initialize failure metric and rename methods
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
* add a metrics package to handle metric registration and publishing
* add a metricsAddress field to the server struct
* make metrics a part of the server
* start a metrics endpoint as part of starting the controllers
* instrument backup_controller to report metrics
* update cli-reference docs
* update example deployments with prometheus annotations
* update 'pkg/install' tooling with prometheus annotations
Signed-off-by: Ashish Amarnath <ashish.amarnath@gmail.com>
Shubham Pampattiwar
Quang Ngo
Daniel JiangandGitHub
Tiger KaovilaiandXun Jiang/Bruce Jiang
eveneast
Matthieu MORELandGitHub
allenxu404
Xun Jiang
yanggang
Nilesh Akhade
Lyndon-Li
Tobias Giese
Taeuk KimandGitHub
Ashish AmarnathandGitHub
kathpeonyandGitHub
Steve Kriss
Fábio Franco UechiandSteve Kriss
Fábio Franco UechiandNolan Brubaker
Steve KrissandKubeKween
Shubheksha Jalan
Nolan Brubaker
Alex Lemaresquier
Ashish Amarnath