Configurable Kopia Maintenance Interval

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

comment update

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

comment

Signed-off-by: Tiger Kaovilai <tkaovila@redhat.com>

changelogs/unreleased/8581-kaovilai

@@ -0,0 +1 @@
+Configurable Kopia Maintenance Interval. backup-repository-configmap adds an option for configurable`fullMaintenanceInterval` where fastGC (12 hours), and eagerGC (6 hours) allowing for faster removal of deleted velero backups from kopia repo.

pkg/repository/udmrepo/kopialib/lib_repo.go

@@ -600,6 +600,29 @@ func writeInitParameters(ctx context.Context, repoOption udmrepo.RepoOptions, lo
 			logger.Infof("Quick maintenance interval change from %v to %v", p.QuickCycle.Interval, overwriteQuickMaintainInterval)
 			p.QuickCycle.Interval = overwriteQuickMaintainInterval
 		}
+		// the repoOption.StorageOptions are set via
+		// udmrepo.WithStoreOptions -> udmrepo.GetStoreOptions (interface)
+		// -> pkg/repository/provider.GetStoreOptions(param interface{}) -> pkg/repository/provider.getStorageVariables(..., backupRepoConfig)
+		// where backupRepoConfig comes from param.(RepoParam).BackupRepo.Spec.RepositoryConfig map[string]string
+		// where RepositoryConfig comes from pkg/controller/getBackupRepositoryConfig(...)
+		// where it gets a configMap name from pkg/cmd/server/config/Config.BackupRepoConfig
+		// which gets set via velero server flag `backup-repository-configmap` "The name of ConfigMap containing backup repository configurations."
+		// and data stored as json under ConfigMap.Data[repoType] where repoType is BackupRepository.Spec.RepositoryType: either kopia or restic
+		// repoOption.StorageOptions[udmrepo.StoreOptionKeyFullMaintenanceInterval] would for example look like
+		// configMapName.data.kopia: {"fullMaintenanceInterval": "eagerGC"}
+		fullMaintIntervalOption := udmrepo.FullMaintenanceIntervalOptions(repoOption.StorageOptions[udmrepo.StoreOptionKeyFullMaintenanceInterval])
+		if fullMaintIntervalOption == udmrepo.FastGC {
+			logger.Infof("Full maintenance interval change from %v to %v", p.FullCycle.Interval, udmrepo.FastGCInterval)
+			p.FullCycle.Interval = udmrepo.FastGCInterval
+		}
+		if fullMaintIntervalOption == udmrepo.EagerGC {
+			logger.Infof("Full maintenance interval change from %v to %v", p.FullCycle.Interval, udmrepo.EagerGCInterval)
+			p.FullCycle.Interval = udmrepo.EagerGCInterval
+		}
+		if fullMaintIntervalOption == udmrepo.NormalGC {
+			logger.Infof("Full maintenance interval change from %v to %v", p.FullCycle.Interval, udmrepo.NormalGCInterval)
+			p.FullCycle.Interval = udmrepo.NormalGCInterval
+		}
 
 		p.Owner = r.ClientOptions().UsernameAtHost()
 

pkg/repository/udmrepo/repo_options.go

@@ -20,6 +20,7 @@ import (
 	"os"
 	"path/filepath"
 	"strings"
+	"time"
 )
 
 const (
@@ -70,8 +71,19 @@ const (
 	ThrottleOptionListOps       = "listOPS"
 	ThrottleOptionUploadBytes   = "uploadBytes"
 	ThrottleOptionDownloadBytes = "downloadBytes"
+	// FullMaintenanceInterval will overwrite kopia maintenance interval
+	// options are fastGC for 12 hours, eagerGC for 6 hours, normalGC for 24 hours
+	StoreOptionKeyFullMaintenanceInterval                                = "fullMaintenanceInterval"
+	FastGC                                FullMaintenanceIntervalOptions = "fastGC"
+	FastGCInterval                        time.Duration                  = 12 * time.Hour
+	EagerGC                               FullMaintenanceIntervalOptions = "eagerGC"
+	EagerGCInterval                       time.Duration                  = 6 * time.Hour
+	NormalGC                              FullMaintenanceIntervalOptions = "normalGC"
+	NormalGCInterval                      time.Duration                  = 24 * time.Hour
 )
 
+type FullMaintenanceIntervalOptions string
+
 const (
 	defaultUsername = "default"
 	defaultDomain   = "default"

site/content/docs/main/repository-maintenance.md

@@ -130,8 +130,34 @@ velero install --default-repo-maintain-frequency <DURATION>
 ```
 For Kopia the default maintenance frequency is 1 hour, and Restic is 7 * 24 hours.
 
+### Full Maintenance Interval customization
+The full maintenance interval defaults to kopia defaults of 24 hours. Velero provide three override options under `fullMaintenanceInterval` configuration using `backup-repository-configmap` ConfigMap provided to velero install commands allowing for faster removal of deleted velero backups from kopia repo.
+- normalGC: 24 hours
+- fastGC: 12 hours
+- eagerGC: 6 hours
+
+Example of the `backup-repository-configmap` ConfigMap for the above scenario is as below:
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: <config-name>
+  namespace: velero
+data:
+  <repository-type-1>: |
+    {
+      "fullMaintenanceInterval": fastGC
+    }
+  <repository-type-2>: |
+    {
+      "fullMaintenanceInterval": normalGC
+    }        
+```
+
+Per kopia [Maintenance Safety](https://kopia.io/docs/advanced/maintenance/#maintenance-safety), it is expected that velero backup deletion will not result in immediate kopia repository data removal. Reducing full maintenance interval using above options should help reduce time taken to remove blobs not in use.
+
 ### Others
 Maintenance jobs will inherit the labels, annotations, toleration, nodeSelector, service account, image, environment variables, cloud-credentials etc. from Velero deployment.
 
 [1]: velero-install.md#usage
-[2]: node-agent-concurrency.md
+[2]: node-agent-concurrency.md