Merge branch 'main' into issue-fix-6663

changelogs/unreleased/6917-27149chen

@@ -0,0 +1 @@
+Support JSON Merge Patch and Strategic Merge Patch in Resource Modifiers

changelogs/unreleased/6962-blackpiglet

@@ -0,0 +1 @@
+Add the PV backup information design document.

changelogs/unreleased/6995-kaovilai

@@ -0,0 +1 @@
+Fix unified repository (kopia) s3 credentials profile selection

changelogs/unreleased/7004-kaovilai

@@ -0,0 +1 @@
+restore: Use warning when Create IsAlreadyExist and Get error

changelogs/unreleased/7022-allenxu404

@@ -0,0 +1 @@
+Fix inconsistent behavior of Backup and Restore hook execution

changelogs/unreleased/7026-blackpiglet

@@ -0,0 +1 @@
+Add HealthCheckNodePort deletion logic for Service restore.

changelogs/unreleased/7028-shubham-pampattiwar

@@ -0,0 +1 @@
+Add description markers for dataupload and datadownload CRDs

changelogs/unreleased/7034-ywk253100

@@ -0,0 +1 @@
+Read information from the credential specified by BSL

changelogs/unreleased/7038-Lyndon-Li

@@ -0,0 +1 @@
+Fix issue #7027, data mover backup exposer should not assume the first volume as the backup volume in backup pod

changelogs/unreleased/7041-blackpiglet

@@ -0,0 +1 @@
+Remove the Velero generated client.

changelogs/unreleased/7051-blackpiglet

@@ -0,0 +1 @@
+Remove dependency of generated client part 3.

config/crd/v1/bases/velero.io_restores.yaml

@@ -186,6 +186,12 @@ spec:
                                     - Continue
                                     - Fail
                                     type: string
+                                  waitForReady:
+                                    description: WaitForReady ensures command will
+                                      be launched when container is Ready instead
+                                      of Running.
+                                    nullable: true
+                                    type: boolean
                                   waitTimeout:
                                     description: WaitTimeout defines the maximum amount
                                       of time Velero should wait for the container

config/crd/v2alpha1/bases/velero.io_datadownloads.yaml

@@ -48,6 +48,8 @@ spec:
     name: v2alpha1
     schema:
       openAPIV3Schema:
+        description: DataDownload acts as the protocol between data mover plugins
+          and data mover controller for the datamover restore operation
         properties:
           apiVersion:
             description: 'APIVersion defines the versioned schema of this representation

config/crd/v2alpha1/bases/velero.io_datauploads.yaml

@@ -49,6 +49,8 @@ spec:
     name: v2alpha1
     schema:
       openAPIV3Schema:
+        description: DataUpload acts as the protocol between data mover plugins and
+          data mover controller for the datamover backup operation
         properties:
           apiVersion:
             description: 'APIVersion defines the versioned schema of this representation

design/merge-patch-and-strategic-in-resource-modifier.md

@@ -88,10 +88,10 @@ Example of StrategicPatches in ResourceModifierRule
 version: v1
 resourceModifierRules:
 - conditions:
-  groupResource: pods
-  resourceNameRegex: "^my-pod$"
-  namespaces:
-  - ns1
+    groupResource: pods
+    resourceNameRegex: "^my-pod$"
+    namespaces:
+    - ns1
   strategicPatches:
   - patchData: |
       {

design/pv_backup_info.md

@@ -0,0 +1,192 @@
+# PersistentVolume backup information design
+
+## Abstract
+Create a new metadata file in the backup repository's backup name sub-directory to store the backup-including PVC and PV information. The information includes the way of backing up the PVC and PV data, snapshot information, and status. The needed snapshot status can also be recorded there, but the Velero-Native snapshot plugin doesn't provide a way to get the snapshot size from the API, so it's possible that not all snapshot size information is available.
+
+This new additional metadata file is needed when:
+* Get a summary of the backup's PVC and PV information, including how the data in them is backed up, or whether the data in them is skipped from backup.
+* Find out how the PVC and PV should be restored in the restore process.
+* Retrieve the PV's snapshot information for backup.
+
+## Background
+There is already a [PR](https://github.com/vmware-tanzu/velero/pull/6496) to track the skipped PVC in the backup. This design will depend on it and go further to get a summary of PVC and PV information, then persist into a metadata file in the backup repository.
+
+In the restore process, the Velero server needs to decide how the PV resource should be restored according to how the PV is backed up. The current logic is to check whether it's backed up by Velero-native snapshot, by file-system backup, or having `DeletionPolicy` set as `Delete`.
+
+The checks are made by the backup-generated PVBs or Snapshots. There is no generic way to find this information, and the CSI backup and Snapshot data movement backup are not covered.
+
+Another thing that needs noticing is when describing the backup, there is no generic way to find the PV's snapshot information.
+
+## Goals
+- Create a new metadata file to store backup's PVCs and PVs information and volume data backing up method. The file can be used to let downstream consumers generate a summary.
+- Create a generic way to let the Velero server know how the PV resources are backed up.
+- Create a generic way to let the Velero server find the PV corresponding snapshot information.
+
+## Non Goals
+- Unify how to get snapshot size information for all PV backing-up methods, and all other currently not ready PVs' information.
+
+## High-Level Design
+Create _backup-name_-volumes-info.json metadata file in the backup's repository. This file will be encoded to contain all the PVC and PV information included in the backup. The information covers whether the PV or PVC's data is skipped during backup, how its data is backed up, and the backed-up detail information.
+
+Please notice that the new metadata file includes all skipped volume information. This is used to address [the second phase needs of skipped volumes information](https://github.com/vmware-tanzu/velero/issues/5834#issuecomment-1526624211).
+
+The `restoreItem` function can decode the _backup-name_-volumes-info.json file to determine how to handle the PV resource. 
+
+## Detailed Design
+
+### The VolumeInfo structure
+_backup-name_-volumes-info.json file is a structure that contains an array of structure `VolumeInfo` and a VolumeInfo version parameter. The version is used to support multiple VolumeInfo version, and make the backward-compatible support possible. The current version is `1`.
+
+The 1 version of `VolumeInfo` definition is:
+``` golang
+type VolumeInfoV1 struct {
+    PVCName        string    // The PVC's name. The format should be <namespace-name>/<PVC-name>
+    PVName         string    // The PV name.
+    BackupMethod   string    // The way the volume data is backed up. The valid value includes `VeleroNativeSnapshot`, `PodVolumeBackup`, `CSISnapshot` and `Skipped`.
+    SnapshotDataMovement bool // Whether the volume's snapshot data is moved to specified storage.
+
+    SkippedReason   string    // The reason for the volume is skipped in the backup.
+    StartTimestamp  *metav1.Time // Snapshot starts timestamp.
+
+    CSISnapshotInfo          CSISnapshotInfo
+    SnapshotDataMoveInfo     SnapshotDataMoveInfo
+    NativeSnapshotInfo       VeleroNativeSnapshotInfo
+    PVBInfo                  PodVolumeBackupInfo
+}
+
+// CSISnapshotInfo is used for displaying the CSI snapshot status
+type CSISnapshotInfo struct {
+    SnapshotHandle  string       // The actual snapshot ID. It can be the cloud provider's snapshot ID or the file-system uploader's snapshot.
+    Size            int64        // The snapshot corresponding volume size. Some of the volume backup methods cannot retrieve the data by current design, for example, the Velero native snapshot.
+
+    Driver          string  // The name of the CSI driver.
+    VSCName         string // The name of the VolumeSnapshotContent. 
+}
+
+// SnapshotDataMoveInfo is used for displaying the snapshot data mover status.
+type SnapshotDataMoveInfo struct {
+    DataMover        string    // The data mover used by the backup. The valid values are `velero` and ``(equals to `velero`).
+    UploaderType     string    // The type of the uploader that uploads the snapshot data. The valid values are `kopia` and `restic`. It's useful for file-system backup and snapshot data mover.
+    RetainedSnapshot string    // The name or ID of the snapshot associated object(SAO).
+}
+
+// VeleroNativeSnapshotInfo is used for displaying the Velero native snapshot status.
+type VeleroNativeSnapshotInfo struct {
+    SnapshotHandle      string       // The actual snapshot ID. It can be the cloud provider's snapshot ID or the file-system uploader's snapshot.
+    Size                int64        // The snapshot corresponding volume size. Some of the volume backup methods cannot retrieve the data by current design, for example, the Velero native snapshot.
+
+    VolumeType string    // The cloud provider snapshot volume type.
+    VolumeAZ   string    // The cloud provider snapshot volume's availability zones.
+    IOPS       string    // The cloud provider snapshot volume's IOPS.
+}
+
+// PodVolumeBackupInfo is used for displaying the PodVolumeBackup snapshot status.
+type PodVolumeBackupInfo struct {
+    SnapshotHandle      string       // The actual snapshot ID. It can be the cloud provider's snapshot ID or the file-system uploader's snapshot.
+    Size                int64        // The snapshot corresponding volume size. Some of the volume backup methods cannot retrieve the data by current design, for example, the Velero native snapshot.
+
+    UploaderType  string    // The type of the uploader that uploads the data. The valid values are `kopia` and `restic`. It's useful for file-system backup and snapshot data mover.
+    VolumeName    string // The PVC's corresponding volume name used by Pod
+    PodName       string // The Pod name mounting this PVC. The format should be <namespace-name>/<pod-name>.
+}
+```
+
+To make Velero support multiple versions of VolumeInfo, Velero needs to create a structure to include the version and the versioned volume information together. The information is persisted in the metadata file during backup creation. When reading the VolumeInfo metadata file, Velero reads the version information first by un-marshalling the metadata file by structure `VolumeInfoVersion`, then decide to use which version of the VolumeInfos according to the version value.
+
+If there is need to introduce a non compatible change to the VolumeInfo, a new version of `VolumeInfos` and `VolumeInfo` are needed. For example, version 2 is created, then `VolumeInfosV2` and `VolumeInfoV2` structures are needed.
+
+Only when there a non-backward-compatible change introduced in the `VolumeInfo` structure, the `version`'s version will be incremented.
+
+``` golang
+type VolumeInfoVersion struct {
+    Version string 
+}
+
+type VolumeInfosV2 struct {
+    Infos   []VolumeInfoV2
+    Version string // VolumeInfo structure's version information.
+}
+
+type VolumeInfoV2 struct {
+    ...
+}
+```
+
+### How the VolumeInfo array is generated.
+The function `persistBackup` has `backup *pkgbackup.Request` in parameters.
+From it, the `VolumeSnapshots`, `PodVolumeBackups`, `CSISnapshots`, `itemOperationsList`, and `SkippedPVTracker` can be read. All of them will be iterated and merged into the `VolumeInfo` array, and then persisted into backup repository in function `persistBackup`.
+
+Please notice that the change happened in async operations are not reflected in the new metadata file. The file only covers the volume changes happen in the Velero server process scope.
+
+A new methods are added to BackupStore to download the VolumeInfo metadata file.
+Uploading the metadata file is covered in the exiting `PutBackup` method.
+
+``` golang
+type BackupStore interface {
+    ...
+    GetVolumeInfos(name string) ([]*VolumeInfo, error)
+    ...
+}
+```
+
+### How the VolumeInfo array is used.
+
+#### Generate the PVC backed-up information summary
+The downstream tools can use this VolumeInfo array to format and display their volume information. This is in the scope of this feature.
+
+#### Retrieve volume backed-up information for `velero backup describe` command
+The `velero backup describe` can also use this VolumeInfo array structure to display the volume information. The snapshot data mover volume should use this structure at first, then the Velero native snapshot, CSI snapshot, and PodVolumeBackup can also use this structure. The detailed implementation is also not in this feature's scope.
+
+#### Let restore know how to restore the PV
+In the function `restoreItem`, it will determine whether to restore the PV resource by checking it in the Velero native snapshots list, PodVolumeBackup list, and its DeletionPolicy. This logic is still kept. The logic will be used when the new `VolumeInfo` metadata cannot be found to support backward compatibility.
+
+``` golang
+	if groupResource == kuberesource.PersistentVolumes {
+		switch {
+		case hasSnapshot(name, ctx.volumeSnapshots):
+            ...
+        case hasPodVolumeBackup(obj, ctx):
+            ...
+        case hasDeleteReclaimPolicy(obj.Object):
+            ...
+        default:
+            ...
+```
+
+After introducing the VolumeInfo array, the following logic will be added.
+``` golang
+	if groupResource == kuberesource.PersistentVolumes {
+        volumeInfo := GetVolumeInfo(pvName)
+		switch volumeInfo.BackupMethod {
+		case VeleroNativeSnapshot:
+            ...
+        case PodVolumeBackup:
+            ...
+        case CSISnapshot:
+            ...
+        case Skipped:
+            // Check whether the Velero server should restore the PV depending on the DeletionPolicy setting.
+        default:
+            // Need to check whether the volume is backed up by the SnapshotDataMover.
+            if volumeInfo.SnapshotDataMovement:
+```
+
+### How the VolumeInfo metadata file is deleted
+_backup-name_-volumes-info.json file is deleted during backup deletion.
+
+## Alternatives Considered
+The restore process needs more information about how the PVs are backed up to determine whether this PV should be restored. The released branches also need a similar function, but backporting a new feature into previous releases may not be a good idea, so according to [Anshul Ahuja's suggestion](https://github.com/vmware-tanzu/velero/issues/6595#issuecomment-1731081580), adding more cases here to support checking PV backed-up by CSI plugin and CSI snapshot data mover: https://github.com/vmware-tanzu/velero/blob/5ff5073cc3f364bafcfbd26755e2a92af68ba180/pkg/restore/restore.go#L1206-L1324.
+
+## Security Considerations
+There should be no security impact introduced by this design.
+
+## Compatibility
+After this design is implemented, there should be no impact on the existing [skipped PVC summary feature](https://github.com/vmware-tanzu/velero/pull/6496).
+
+To support older version backup, which doesn't have the VolumeInfo metadata file, the old logic, which is checking the Velero native snapshots list, PodVolumeBackup list, and PVC DeletionPolicy, is still kept, and supporting CSI snapshots and snapshot data mover logic will be added too.
+
+## Implementation
+This will be implemented in the Velero v1.13 development cycle.
+
+## Open Issues
+There are no open issues identified by now.

go.mod

@@ -64,6 +64,7 @@ require (
 	k8s.io/metrics v0.25.6
 	k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed
 	sigs.k8s.io/controller-runtime v0.12.2
+	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd
 	sigs.k8s.io/yaml v1.3.0
 )
 
@@ -184,7 +185,6 @@ require (
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	k8s.io/component-base v0.24.2 // indirect
 	k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 // indirect
-	sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect
 	sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect
 )
 

go.sum

@@ -1428,8 +1428,8 @@ sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.30/go.mod h1:fEO7lR
 sigs.k8s.io/controller-runtime v0.12.2 h1:nqV02cvhbAj7tbt21bpPpTByrXGn2INHRsi39lXy9sE=
 sigs.k8s.io/controller-runtime v0.12.2/go.mod h1:qKsk4WE6zW2Hfj0G4v10EnNB2jMG1C+NTb8h+DwCoU0=
 sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2/go.mod h1:B+TnT182UBxE84DiCz4CVE26eOSDAeYCpfDnC2kdKMY=
-sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k=
-sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
+sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=
+sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
 sigs.k8s.io/kustomize/api v0.8.11/go.mod h1:a77Ls36JdfCWojpUqR6m60pdGY1AYFix4AH83nJtY1g=
 sigs.k8s.io/kustomize/api v0.11.4/go.mod h1:k+8RsqYbgpkIrJ4p9jcdPqe8DprLxFUUO0yNOq8C+xI=
 sigs.k8s.io/kustomize/kyaml v0.11.0/go.mod h1:GNMwjim4Ypgp/MueD3zXHLRJEjz7RvtPae0AwlvEMFM=

hack/verify-generated-crd-code.sh

@@ -19,7 +19,7 @@ HACK_DIR=$(dirname "${BASH_SOURCE}")
 ${HACK_DIR}/update-3generated-crd-code.sh
 
 # ensure no changes to generated CRDs
-if [! git diff --exit-code config/crd/v1/crds/crds.go config/crd/v2alpha1/crds/crds.go >/dev/null]; then
+if ! git diff --exit-code config/crd/v1/crds/crds.go config/crd/v2alpha1/crds/crds.go &> /dev/null; then
   # revert changes to state before running CRD generation to stay consistent
   # with code-generator `--verify-only` option which discards generated changes
   git checkout config/crd

internal/hook/wait_exec_hook_handler.go

@@ -50,6 +50,11 @@ type DefaultListWatchFactory struct {
 	PodsGetter cache.Getter
 }
 
+type HookErrInfo struct {
+	Namespace string
+	Err       error
+}
+
 func (d *DefaultListWatchFactory) NewListWatch(namespace string, selector fields.Selector) cache.ListerWatcher {
 	return cache.NewListWatchFromClient(d.PodsGetter, "pods", namespace, selector)
 }
@@ -158,8 +163,8 @@ func (e *DefaultWaitExecHookHandler) HandleHooks(
 				if hook.Hook.WaitTimeout.Duration != 0 && time.Since(waitStart) > hook.Hook.WaitTimeout.Duration {
 					err := fmt.Errorf("hook %s in container %s expired before executing", hook.HookName, hook.Hook.Container)
 					hookLog.Error(err)
+					errors = append(errors, err)
 					if hook.Hook.OnError == velerov1api.HookErrorModeFail {
-						errors = append(errors, err)
 						cancel()
 						return
 					}
@@ -172,8 +177,9 @@ func (e *DefaultWaitExecHookHandler) HandleHooks(
 				}
 				if err := e.PodCommandExecutor.ExecutePodCommand(hookLog, podMap, pod.Namespace, pod.Name, hook.HookName, eh); err != nil {
 					hookLog.WithError(err).Error("Error executing hook")
+					err = fmt.Errorf("hook %s in container %s failed to execute, err: %v", hook.HookName, hook.Hook.Container, err)
+					errors = append(errors, err)
 					if hook.Hook.OnError == velerov1api.HookErrorModeFail {
-						errors = append(errors, err)
 						cancel()
 						return
 					}
@@ -204,10 +210,9 @@ func (e *DefaultWaitExecHookHandler) HandleHooks(
 	podWatcher.Run(ctx.Done())
 
 	// There are some cases where this function could return with unexecuted hooks: the pod may
-	// be deleted, a hook with OnError mode Fail could fail, or it may timeout waiting for
+	// be deleted, a hook could fail, or it may timeout waiting for
 	// containers to become ready.
-	// Each unexecuted hook is logged as an error but only hooks with OnError mode Fail return
-	// an error from this function.
+	// Each unexecuted hook is logged as an error and this error will be returned from this function.
 	for _, hooks := range byContainer {
 		for _, hook := range hooks {
 			if hook.executed {
@@ -222,9 +227,7 @@ func (e *DefaultWaitExecHookHandler) HandleHooks(
 				},
 			)
 			hookLog.Error(err)
-			if hook.Hook.OnError == velerov1api.HookErrorModeFail {
-				errors = append(errors, err)
-			}
+			errors = append(errors, err)
 		}
 	}
 

internal/hook/wait_exec_hook_handler_test.go

@@ -209,10 +209,10 @@ func TestWaitExecHandleHooks(t *testing.T) {
 						Result(),
 				},
 			},
-			expectedErrors: []error{errors.New("pod hook error")},
+			expectedErrors: []error{errors.New("hook <from-annotation> in container container1 failed to execute, err: pod hook error")},
 		},
 		{
-			name: "should return no error when hook from annotation fails with on error mode continue",
+			name: "should return error when hook from annotation fails with on error mode continue",
 			initialPod: builder.ForPod("default", "my-pod").
 				ObjectMeta(builder.WithAnnotations(
 					podRestoreHookCommandAnnotationKey, "/usr/bin/foo",
@@ -278,7 +278,7 @@ func TestWaitExecHandleHooks(t *testing.T) {
 						Result(),
 				},
 			},
-			expectedErrors: nil,
+			expectedErrors: []error{errors.New("hook <from-annotation> in container container1 failed to execute, err: pod hook error")},
 		},
 		{
 			name: "should return no error when hook from annotation executes after 10ms wait for container to start",
@@ -422,7 +422,7 @@ func TestWaitExecHandleHooks(t *testing.T) {
 			},
 		},
 		{
-			name:          "should return no error when spec hook with wait timeout expires with OnError mode Continue",
+			name:          "should return error when spec hook with wait timeout expires with OnError mode Continue",
 			groupResource: "pods",
 			initialPod: builder.ForPod("default", "my-pod").
 				Containers(&v1.Container{
@@ -435,7 +435,7 @@ func TestWaitExecHandleHooks(t *testing.T) {
 					},
 				}).
 				Result(),
-			expectedErrors: nil,
+			expectedErrors: []error{errors.New("hook my-hook-1 in container container1 in pod default/my-pod not executed: context deadline exceeded")},
 			byContainer: map[string][]PodExecRestoreHook{
 				"container1": {
 					{
@@ -515,8 +515,8 @@ func TestWaitExecHandleHooks(t *testing.T) {
 			sharedHooksContextTimeout: time.Millisecond,
 		},
 		{
-			name:           "should return no error when shared hooks context is canceled before spec hook with OnError mode Continue executes",
-			expectedErrors: nil,
+			name:           "should return error when shared hooks context is canceled before spec hook with OnError mode Continue executes",
+			expectedErrors: []error{errors.New("hook my-hook-1 in container container1 in pod default/my-pod not executed: context deadline exceeded")},
 			groupResource:  "pods",
 			initialPod: builder.ForPod("default", "my-pod").
 				Containers(&v1.Container{

internal/resourcemodifiers/json_merge_patch.go

@@ -0,0 +1,45 @@
+package resourcemodifiers
+
+import (
+	"fmt"
+
+	jsonpatch "github.com/evanphx/json-patch"
+	"github.com/sirupsen/logrus"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"sigs.k8s.io/yaml"
+)
+
+type JSONMergePatch struct {
+	PatchData string `json:"patchData,omitempty"`
+}
+
+type JSONMergePatcher struct {
+	patches []JSONMergePatch
+}
+
+func (p *JSONMergePatcher) Patch(u *unstructured.Unstructured, _ logrus.FieldLogger) (*unstructured.Unstructured, error) {
+	objBytes, err := u.MarshalJSON()
+	if err != nil {
+		return nil, fmt.Errorf("error in marshaling object %s", err)
+	}
+
+	for _, patch := range p.patches {
+		patchBytes, err := yaml.YAMLToJSON([]byte(patch.PatchData))
+		if err != nil {
+			return nil, fmt.Errorf("error in converting YAML to JSON %s", err)
+		}
+
+		objBytes, err = jsonpatch.MergePatch(objBytes, patchBytes)
+		if err != nil {
+			return nil, fmt.Errorf("error in applying JSON Patch: %s", err.Error())
+		}
+	}
+
+	updated := &unstructured.Unstructured{}
+	err = updated.UnmarshalJSON(objBytes)
+	if err != nil {
+		return nil, fmt.Errorf("error in unmarshalling modified object %s", err.Error())
+	}
+
+	return updated, nil
+}

internal/resourcemodifiers/json_merge_patch_test.go

@@ -0,0 +1,41 @@
+package resourcemodifiers
+
+import (
+	"testing"
+
+	"github.com/sirupsen/logrus"
+	"github.com/stretchr/testify/assert"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"k8s.io/apimachinery/pkg/runtime"
+	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
+)
+
+func TestJsonMergePatchFailure(t *testing.T) {
+	tests := []struct {
+		name string
+		data string
+	}{
+		{
+			name: "patch with bad yaml",
+			data: "a: b:",
+		},
+		{
+			name: "patch with bad json",
+			data: `{"a"::1}`,
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			scheme := runtime.NewScheme()
+			err := clientgoscheme.AddToScheme(scheme)
+			assert.NoError(t, err)
+			pt := &JSONMergePatcher{
+				patches: []JSONMergePatch{{PatchData: tt.data}},
+			}
+
+			u := &unstructured.Unstructured{}
+			_, err = pt.Patch(u, logrus.New())
+			assert.Error(t, err)
+		})
+	}
+}

internal/resourcemodifiers/json_patch.go

@@ -0,0 +1,96 @@
+package resourcemodifiers
+
+import (
+	"errors"
+	"fmt"
+	"strconv"
+	"strings"
+
+	jsonpatch "github.com/evanphx/json-patch"
+	"github.com/sirupsen/logrus"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+)
+
+type JSONPatch struct {
+	Operation string `json:"operation"`
+	From      string `json:"from,omitempty"`
+	Path      string `json:"path"`
+	Value     string `json:"value,omitempty"`
+}
+
+func (p *JSONPatch) ToString() string {
+	if addQuotes(p.Value) {
+		return fmt.Sprintf(`{"op": "%s", "from": "%s", "path": "%s", "value": "%s"}`, p.Operation, p.From, p.Path, p.Value)
+	}
+	return fmt.Sprintf(`{"op": "%s", "from": "%s", "path": "%s", "value": %s}`, p.Operation, p.From, p.Path, p.Value)
+}
+
+func addQuotes(value string) bool {
+	if value == "" {
+		return true
+	}
+	// if value is null, then don't add quotes
+	if value == "null" {
+		return false
+	}
+	// if value is a boolean, then don't add quotes
+	if _, err := strconv.ParseBool(value); err == nil {
+		return false
+	}
+	// if value is a json object or array, then don't add quotes.
+	if strings.HasPrefix(value, "{") || strings.HasPrefix(value, "[") {
+		return false
+	}
+	// if value is a number, then don't add quotes
+	if _, err := strconv.ParseFloat(value, 64); err == nil {
+		return false
+	}
+	return true
+}
+
+type JSONPatcher struct {
+	patches []JSONPatch `yaml:"patches"`
+}
+
+func (p *JSONPatcher) Patch(u *unstructured.Unstructured, logger logrus.FieldLogger) (*unstructured.Unstructured, error) {
+	modifiedObjBytes, err := p.applyPatch(u)
+	if err != nil {
+		if errors.Is(err, jsonpatch.ErrTestFailed) {
+			logger.Infof("Test operation failed for JSON Patch %s", err.Error())
+			return u.DeepCopy(), nil
+		}
+		return nil, fmt.Errorf("error in applying JSON Patch %s", err.Error())
+	}
+
+	updated := &unstructured.Unstructured{}
+	err = updated.UnmarshalJSON(modifiedObjBytes)
+	if err != nil {
+		return nil, fmt.Errorf("error in unmarshalling modified object %s", err.Error())
+	}
+
+	return updated, nil
+}
+
+func (p *JSONPatcher) applyPatch(u *unstructured.Unstructured) ([]byte, error) {
+	patchBytes := p.patchArrayToByteArray()
+	jsonPatch, err := jsonpatch.DecodePatch(patchBytes)
+	if err != nil {
+		return nil, fmt.Errorf("error in decoding json patch %s", err.Error())
+	}
+
+	objBytes, err := u.MarshalJSON()
+	if err != nil {
+		return nil, fmt.Errorf("error in marshaling object %s", err.Error())
+	}
+
+	return jsonPatch.Apply(objBytes)
+}
+
+func (p *JSONPatcher) patchArrayToByteArray() []byte {
+	var patches []string
+	for _, patch := range p.patches {
+		patches = append(patches, patch.ToString())
+	}
+	patchesStr := strings.Join(patches, ",\n\t")
+	return []byte(fmt.Sprintf(`[%s]`, patchesStr))
+}

internal/resourcemodifiers/resource_modifiers.go

@@ -18,16 +18,16 @@ package resourcemodifiers
 import (
 	"fmt"
 	"regexp"
-	"strconv"
-	"strings"
 
 	jsonpatch "github.com/evanphx/json-patch"
+	"github.com/gobwas/glob"
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
 	v1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/apimachinery/pkg/runtime"
 	"sigs.k8s.io/yaml"
 
 	"github.com/vmware-tanzu/velero/pkg/util/collections"
@@ -38,11 +38,9 @@ const (
 	ResourceModifierSupportedVersionV1 = "v1"
 )
 
-type JSONPatch struct {
-	Operation string `json:"operation"`
-	From      string `json:"from,omitempty"`
-	Path      string `json:"path"`
-	Value     string `json:"value,omitempty"`
+type MatchRule struct {
+	Path  string `json:"path,omitempty"`
+	Value string `json:"value,omitempty"`
 }
 
 type Conditions struct {
@@ -50,11 +48,14 @@ type Conditions struct {
 	GroupResource     string                `json:"groupResource"`
 	ResourceNameRegex string                `json:"resourceNameRegex,omitempty"`
 	LabelSelector     *metav1.LabelSelector `json:"labelSelector,omitempty"`
+	Matches           []MatchRule           `json:"matches,omitempty"`
 }
 
 type ResourceModifierRule struct {
-	Conditions Conditions  `json:"conditions"`
-	Patches    []JSONPatch `json:"patches"`
+	Conditions       Conditions            `json:"conditions"`
+	Patches          []JSONPatch           `json:"patches,omitempty"`
+	MergePatches     []JSONMergePatch      `json:"mergePatches,omitempty"`
+	StrategicPatches []StrategicMergePatch `json:"strategicPatches,omitempty"`
 }
 
 type ResourceModifiers struct {
@@ -83,10 +84,10 @@ func GetResourceModifiersFromConfig(cm *v1.ConfigMap) (*ResourceModifiers, error
 	return resModifiers, nil
 }
 
-func (p *ResourceModifiers) ApplyResourceModifierRules(obj *unstructured.Unstructured, groupResource string, log logrus.FieldLogger) []error {
+func (p *ResourceModifiers) ApplyResourceModifierRules(obj *unstructured.Unstructured, groupResource string, scheme *runtime.Scheme, log logrus.FieldLogger) []error {
 	var errs []error
 	for _, rule := range p.ResourceModifierRules {
-		err := rule.Apply(obj, groupResource, log)
+		err := rule.apply(obj, groupResource, scheme, log)
 		if err != nil {
 			errs = append(errs, err)
 		}
@@ -95,13 +96,22 @@ func (p *ResourceModifiers) ApplyResourceModifierRules(obj *unstructured.Unstruc
 	return errs
 }
 
-func (r *ResourceModifierRule) Apply(obj *unstructured.Unstructured, groupResource string, log logrus.FieldLogger) error {
-	namespaceInclusion := collections.NewIncludesExcludes().Includes(r.Conditions.Namespaces...)
-	if !namespaceInclusion.ShouldInclude(obj.GetNamespace()) {
-		return nil
+func (r *ResourceModifierRule) apply(obj *unstructured.Unstructured, groupResource string, scheme *runtime.Scheme, log logrus.FieldLogger) error {
+	ns := obj.GetNamespace()
+	if ns != "" {
+		namespaceInclusion := collections.NewIncludesExcludes().Includes(r.Conditions.Namespaces...)
+		if !namespaceInclusion.ShouldInclude(ns) {
+			return nil
+		}
 	}
 
-	if r.Conditions.GroupResource != groupResource {
+	g, err := glob.Compile(r.Conditions.GroupResource, '.')
+	if err != nil {
+		log.Errorf("Bad glob pattern of groupResource in condition, groupResource: %s, err: %s", r.Conditions.GroupResource, err)
+		return err
+	}
+
+	if !g.Match(groupResource) {
 		return nil
 	}
 
@@ -125,87 +135,82 @@ func (r *ResourceModifierRule) Apply(obj *unstructured.Unstructured, groupResour
 		}
 	}
 
-	patches, err := r.PatchArrayToByteArray()
+	match, err := matchConditions(obj, r.Conditions.Matches, log)
 	if err != nil {
 		return err
+	} else if !match {
+		log.Info("Conditions do not match, skip it")
+		return nil
 	}
+
 	log.Infof("Applying resource modifier patch on %s/%s", obj.GetNamespace(), obj.GetName())
-	err = ApplyPatch(patches, obj, log)
+	err = r.applyPatch(obj, scheme, log)
 	if err != nil {
 		return err
 	}
 	return nil
 }
 
-// PatchArrayToByteArray converts all JsonPatch to string array with the format of jsonpatch.Patch and then convert it to byte array
-func (r *ResourceModifierRule) PatchArrayToByteArray() ([]byte, error) {
-	var patches []string
-	for _, patch := range r.Patches {
-		patches = append(patches, patch.ToString())
+func matchConditions(u *unstructured.Unstructured, rules []MatchRule, _ logrus.FieldLogger) (bool, error) {
+	if len(rules) == 0 {
+		return true, nil
 	}
-	patchesStr := strings.Join(patches, ",\n\t")
-	return []byte(fmt.Sprintf(`[%s]`, patchesStr)), nil
-}
 
-func (p *JSONPatch) ToString() string {
-	if addQuotes(p.Value) {
-		return fmt.Sprintf(`{"op": "%s", "from": "%s", "path": "%s", "value": "%s"}`, p.Operation, p.From, p.Path, p.Value)
-	}
-	return fmt.Sprintf(`{"op": "%s", "from": "%s", "path": "%s", "value": %s}`, p.Operation, p.From, p.Path, p.Value)
-}
+	var fixed []JSONPatch
+	for _, rule := range rules {
+		if rule.Path == "" {
+			return false, fmt.Errorf("path is required for match rule")
+		}
 
-func ApplyPatch(patch []byte, obj *unstructured.Unstructured, log logrus.FieldLogger) error {
-	jsonPatch, err := jsonpatch.DecodePatch(patch)
-	if err != nil {
-		return fmt.Errorf("error in decoding json patch %s", err.Error())
+		fixed = append(fixed, JSONPatch{
+			Operation: "test",
+			Path:      rule.Path,
+			Value:     rule.Value,
+		})
 	}
-	objBytes, err := obj.MarshalJSON()
-	if err != nil {
-		return fmt.Errorf("error in marshaling object %s", err.Error())
-	}
-	modifiedObjBytes, err := jsonPatch.Apply(objBytes)
+
+	p := &JSONPatcher{patches: fixed}
+	_, err := p.applyPatch(u)
 	if err != nil {
 		if errors.Is(err, jsonpatch.ErrTestFailed) {
-			log.Infof("Test operation failed for JSON Patch %s", err.Error())
-			return nil
+			return false, nil
 		}
-		return fmt.Errorf("error in applying JSON Patch %s", err.Error())
+		return false, err
 	}
-	err = obj.UnmarshalJSON(modifiedObjBytes)
-	if err != nil {
-		return fmt.Errorf("error in unmarshalling modified object %s", err.Error())
-	}
-	return nil
+
+	return true, nil
 }
 
 func unmarshalResourceModifiers(yamlData []byte) (*ResourceModifiers, error) {
 	resModifiers := &ResourceModifiers{}
 	err := yaml.UnmarshalStrict(yamlData, resModifiers)
 	if err != nil {
-		return nil, fmt.Errorf("failed to decode yaml data into resource modifiers  %v", err)
+		return nil, fmt.Errorf("failed to decode yaml data into resource modifiers, err: %s", err)
 	}
 	return resModifiers, nil
 }
 
-func addQuotes(value string) bool {
-	if value == "" {
-		return true
+type patcher interface {
+	Patch(u *unstructured.Unstructured, logger logrus.FieldLogger) (*unstructured.Unstructured, error)
+}
+
+func (r *ResourceModifierRule) applyPatch(u *unstructured.Unstructured, scheme *runtime.Scheme, logger logrus.FieldLogger) error {
+	var p patcher
+	if len(r.Patches) > 0 {
+		p = &JSONPatcher{patches: r.Patches}
+	} else if len(r.MergePatches) > 0 {
+		p = &JSONMergePatcher{patches: r.MergePatches}
+	} else if len(r.StrategicPatches) > 0 {
+		p = &StrategicMergePatcher{patches: r.StrategicPatches, scheme: scheme}
+	} else {
+		return fmt.Errorf("no patch data found")
 	}
-	// if value is null, then don't add quotes
-	if value == "null" {
-		return false
+
+	updated, err := p.Patch(u, logger)
+	if err != nil {
+		return fmt.Errorf("error in applying patch %s", err)
 	}
-	// if value is a boolean, then don't add quotes
-	if _, err := strconv.ParseBool(value); err == nil {
-		return false
-	}
-	// if value is a json object or array, then don't add quotes.
-	if strings.HasPrefix(value, "{") || strings.HasPrefix(value, "[") {
-		return false
-	}
-	// if value is a number, then don't add quotes
-	if _, err := strconv.ParseFloat(value, 64); err == nil {
-		return false
-	}
-	return true
+
+	u.SetUnstructuredContent(updated.Object)
+	return nil
 }

internal/resourcemodifiers/resource_modifiers_test.go

@@ -24,6 +24,10 @@ import (
 	v1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/serializer/yaml"
+	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
+	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
 )
 
 func TestGetResourceModifiersFromConfig(t *testing.T) {
@@ -131,6 +135,128 @@ func TestGetResourceModifiersFromConfig(t *testing.T) {
 		},
 	}
 
+	cm5 := &v1.ConfigMap{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "test-configmap",
+			Namespace: "test-namespace",
+		},
+		Data: map[string]string{
+			"sub.yml": "version: v1\nresourceModifierRules:\n- conditions:\n    groupResource: pods\n    namespaces:\n    - ns1\n    matches:\n    - path: /metadata/annotations/foo\n      value: bar\n  mergePatches:\n  - patchData: |\n      metadata:\n        annotations:\n          foo: null",
+		},
+	}
+
+	rules5 := &ResourceModifiers{
+		Version: "v1",
+		ResourceModifierRules: []ResourceModifierRule{
+			{
+				Conditions: Conditions{
+					GroupResource: "pods",
+					Namespaces: []string{
+						"ns1",
+					},
+					Matches: []MatchRule{
+						{
+							Path:  "/metadata/annotations/foo",
+							Value: "bar",
+						},
+					},
+				},
+				MergePatches: []JSONMergePatch{
+					{
+						PatchData: "metadata:\n  annotations:\n    foo: null",
+					},
+				},
+			},
+		},
+	}
+
+	cm6 := &v1.ConfigMap{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "test-configmap",
+			Namespace: "test-namespace",
+		},
+		Data: map[string]string{
+			"sub.yml": "version: v1\nresourceModifierRules:\n- conditions:\n    groupResource: pods\n    namespaces:\n    - ns1\n  strategicPatches:\n  - patchData: |\n      spec:\n        containers:\n        - name: nginx\n          image: repo2/nginx",
+		},
+	}
+
+	rules6 := &ResourceModifiers{
+		Version: "v1",
+		ResourceModifierRules: []ResourceModifierRule{
+			{
+				Conditions: Conditions{
+					GroupResource: "pods",
+					Namespaces: []string{
+						"ns1",
+					},
+				},
+				StrategicPatches: []StrategicMergePatch{
+					{
+						PatchData: "spec:\n  containers:\n  - name: nginx\n    image: repo2/nginx",
+					},
+				},
+			},
+		},
+	}
+
+	cm7 := &v1.ConfigMap{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "test-configmap",
+			Namespace: "test-namespace",
+		},
+		Data: map[string]string{
+			"sub.yml": "version: v1\nresourceModifierRules:\n- conditions:\n    groupResource: pods\n    namespaces:\n    - ns1\n  mergePatches:\n  - patchData: |\n      {\"metadata\":{\"annotations\":{\"foo\":null}}}",
+		},
+	}
+
+	rules7 := &ResourceModifiers{
+		Version: "v1",
+		ResourceModifierRules: []ResourceModifierRule{
+			{
+				Conditions: Conditions{
+					GroupResource: "pods",
+					Namespaces: []string{
+						"ns1",
+					},
+				},
+				MergePatches: []JSONMergePatch{
+					{
+						PatchData: `{"metadata":{"annotations":{"foo":null}}}`,
+					},
+				},
+			},
+		},
+	}
+
+	cm8 := &v1.ConfigMap{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "test-configmap",
+			Namespace: "test-namespace",
+		},
+		Data: map[string]string{
+			"sub.yml": "version: v1\nresourceModifierRules:\n- conditions:\n    groupResource: pods\n    namespaces:\n    - ns1\n  strategicPatches:\n  - patchData: |\n      {\"spec\":{\"containers\":[{\"name\": \"nginx\",\"image\": \"repo2/nginx\"}]}}",
+		},
+	}
+
+	rules8 := &ResourceModifiers{
+		Version: "v1",
+		ResourceModifierRules: []ResourceModifierRule{
+			{
+				Conditions: Conditions{
+					GroupResource: "pods",
+					Namespaces: []string{
+						"ns1",
+					},
+				},
+				StrategicPatches: []StrategicMergePatch{
+					{
+						PatchData: `{"spec":{"containers":[{"name": "nginx","image": "repo2/nginx"}]}}`,
+					},
+				},
+			},
+		},
+	}
+
 	type args struct {
 		cm *v1.ConfigMap
 	}
@@ -180,6 +306,38 @@ func TestGetResourceModifiersFromConfig(t *testing.T) {
 			want:    nil,
 			wantErr: true,
 		},
+		{
+			name: "complex yaml data with json merge patch",
+			args: args{
+				cm: cm5,
+			},
+			want:    rules5,
+			wantErr: false,
+		},
+		{
+			name: "complex yaml data with strategic merge patch",
+			args: args{
+				cm: cm6,
+			},
+			want:    rules6,
+			wantErr: false,
+		},
+		{
+			name: "complex json data with json merge patch",
+			args: args{
+				cm: cm7,
+			},
+			want:    rules7,
+			wantErr: false,
+		},
+		{
+			name: "complex json data with strategic merge patch",
+			args: args{
+				cm: cm8,
+			},
+			want:    rules8,
+			wantErr: false,
+		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
@@ -502,6 +660,38 @@ func TestResourceModifiers_ApplyResourceModifierRules(t *testing.T) {
 			wantErr: false,
 			wantObj: deployNginxTwoReplica.DeepCopy(),
 		},
+		{
+			name: "nginx deployment: Empty Resource Regex",
+			fields: fields{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "deployments.apps",
+							Namespaces:    []string{"foo"},
+						},
+						Patches: []JSONPatch{
+							{
+								Operation: "test",
+								Path:      "/spec/replicas",
+								Value:     "1",
+							},
+							{
+								Operation: "replace",
+								Path:      "/spec/replicas",
+								Value:     "2",
+							},
+						},
+					},
+				},
+			},
+			args: args{
+				obj:           deployNginxOneReplica.DeepCopy(),
+				groupResource: "deployments.apps",
+			},
+			wantErr: false,
+			wantObj: deployNginxTwoReplica.DeepCopy(),
+		},
 		{
 			name: "nginx deployment: Empty Resource Regex  and namespaces list",
 			fields: fields{
@@ -719,7 +909,7 @@ func TestResourceModifiers_ApplyResourceModifierRules(t *testing.T) {
 				Version:               tt.fields.Version,
 				ResourceModifierRules: tt.fields.ResourceModifierRules,
 			}
-			got := p.ApplyResourceModifierRules(tt.args.obj, tt.args.groupResource, logrus.New())
+			got := p.ApplyResourceModifierRules(tt.args.obj, tt.args.groupResource, nil, logrus.New())
 
 			assert.Equal(t, tt.wantErr, len(got) > 0)
 			assert.Equal(t, *tt.wantObj, *tt.args.obj)
@@ -727,6 +917,633 @@ func TestResourceModifiers_ApplyResourceModifierRules(t *testing.T) {
 	}
 }
 
+var podYAMLWithNginxImage = `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod1
+  namespace: fake
+spec:
+  containers:
+  - image: nginx
+    name: nginx
+`
+
+var podYAMLWithNginx1Image = `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod1
+  namespace: fake
+spec:
+  containers:
+  - image: nginx1
+    name: nginx
+`
+
+var podYAMLWithNFSVolume = `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod1
+  namespace: fake
+spec:
+  containers:
+  - image: fake
+    name: fake
+    volumeMounts:
+    - mountPath: /fake1
+      name: vol1
+    - mountPath: /fake2
+      name: vol2
+  volumes:
+  - name: vol1
+    nfs:
+      path: /fake2
+  - name: vol2
+    emptyDir: {}
+`
+
+var podYAMLWithPVCVolume = `
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod1
+  namespace: fake
+spec:
+  containers:
+  - image: fake
+    name: fake
+    volumeMounts:
+    - mountPath: /fake1
+      name: vol1
+    - mountPath: /fake2
+      name: vol2
+  volumes:
+  - name: vol1
+    persistentVolumeClaim:
+      claimName: pvc1
+  - name: vol2
+    emptyDir: {}
+`
+
+var svcYAMLWithPort8000 = `
+apiVersion: v1
+kind: Service
+metadata:
+  name: svc1
+  namespace: fake
+spec:
+  ports:
+  - name: fake1
+    port: 8001
+    protocol: TCP
+    targetPort: 8001
+  - name: fake
+    port: 8000
+    protocol: TCP
+    targetPort: 8000
+  - name: fake2
+    port: 8002
+    protocol: TCP
+    targetPort: 8002
+`
+
+var svcYAMLWithPort9000 = `
+apiVersion: v1
+kind: Service
+metadata:
+  name: svc1
+  namespace: fake
+spec:
+  ports:
+  - name: fake1
+    port: 8001
+    protocol: TCP
+    targetPort: 8001
+  - name: fake
+    port: 9000
+    protocol: TCP
+    targetPort: 9000
+  - name: fake2
+    port: 8002
+    protocol: TCP
+    targetPort: 8002
+`
+
+var cmYAMLWithLabelAToB = `
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cm1
+  namespace: fake
+  labels:
+    a: b
+    c: d
+`
+
+var cmYAMLWithLabelAToC = `
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cm1
+  namespace: fake
+  labels:
+    a: c
+    c: d
+`
+
+var cmYAMLWithoutLabelA = `
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cm1
+  namespace: fake
+  labels:
+    c: d
+`
+
+func TestResourceModifiers_ApplyResourceModifierRules_StrategicMergePatch(t *testing.T) {
+	scheme := runtime.NewScheme()
+	utilruntime.Must(clientgoscheme.AddToScheme(scheme))
+	unstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)
+	o1, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithNFSVolume), nil, nil)
+	assert.NoError(t, err)
+	podWithNFSVolume := o1.(*unstructured.Unstructured)
+
+	o2, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithPVCVolume), nil, nil)
+	assert.NoError(t, err)
+	podWithPVCVolume := o2.(*unstructured.Unstructured)
+
+	o3, _, err := unstructuredSerializer.Decode([]byte(svcYAMLWithPort8000), nil, nil)
+	assert.NoError(t, err)
+	svcWithPort8000 := o3.(*unstructured.Unstructured)
+
+	o4, _, err := unstructuredSerializer.Decode([]byte(svcYAMLWithPort9000), nil, nil)
+	assert.NoError(t, err)
+	svcWithPort9000 := o4.(*unstructured.Unstructured)
+
+	o5, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithNginxImage), nil, nil)
+	assert.NoError(t, err)
+	podWithNginxImage := o5.(*unstructured.Unstructured)
+
+	o6, _, err := unstructuredSerializer.Decode([]byte(podYAMLWithNginx1Image), nil, nil)
+	assert.NoError(t, err)
+	podWithNginx1Image := o6.(*unstructured.Unstructured)
+
+	tests := []struct {
+		name          string
+		rm            *ResourceModifiers
+		obj           *unstructured.Unstructured
+		groupResource string
+		wantErr       bool
+		wantObj       *unstructured.Unstructured
+	}{
+		{
+			name: "update image",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "pods",
+							Namespaces:    []string{"fake"},
+						},
+						StrategicPatches: []StrategicMergePatch{
+							{
+								PatchData: `{"spec":{"containers":[{"name":"nginx","image":"nginx1"}]}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           podWithNginxImage.DeepCopy(),
+			groupResource: "pods",
+			wantErr:       false,
+			wantObj:       podWithNginx1Image.DeepCopy(),
+		},
+		{
+			name: "update image with yaml format",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "pods",
+							Namespaces:    []string{"fake"},
+						},
+						StrategicPatches: []StrategicMergePatch{
+							{
+								PatchData: `spec:
+  containers:
+  - name: nginx
+    image: nginx1`,
+							},
+						},
+					},
+				},
+			},
+			obj:           podWithNginxImage.DeepCopy(),
+			groupResource: "pods",
+			wantErr:       false,
+			wantObj:       podWithNginx1Image.DeepCopy(),
+		},
+		{
+			name: "replace nfs with pvc in volume",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "pods",
+							Namespaces:    []string{"fake"},
+						},
+						StrategicPatches: []StrategicMergePatch{
+							{
+								PatchData: `{"spec":{"volumes":[{"nfs":null,"name":"vol1","persistentVolumeClaim":{"claimName":"pvc1"}}]}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           podWithNFSVolume.DeepCopy(),
+			groupResource: "pods",
+			wantErr:       false,
+			wantObj:       podWithPVCVolume.DeepCopy(),
+		},
+		{
+			name: "replace any other volume source with pvc in volume",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "pods",
+							Namespaces:    []string{"fake"},
+						},
+						StrategicPatches: []StrategicMergePatch{
+							{
+								PatchData: `{"spec":{"volumes":[{"$retainKeys":["name","persistentVolumeClaim"],"name":"vol1","persistentVolumeClaim":{"claimName":"pvc1"}}]}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           podWithNFSVolume.DeepCopy(),
+			groupResource: "pods",
+			wantErr:       false,
+			wantObj:       podWithPVCVolume.DeepCopy(),
+		},
+		{
+			name: "update a service port",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "services",
+							Namespaces:    []string{"fake"},
+						},
+						StrategicPatches: []StrategicMergePatch{
+							{
+								PatchData: `{"spec":{"$setElementOrder/ports":[{"port":8001},{"port":9000},{"port":8002}],"ports":[{"name":"fake","port":9000,"protocol":"TCP","targetPort":9000},{"$patch":"delete","port":8000}]}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           svcWithPort8000.DeepCopy(),
+			groupResource: "services",
+			wantErr:       false,
+			wantObj:       svcWithPort9000.DeepCopy(),
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, scheme, logrus.New())
+
+			assert.Equal(t, tt.wantErr, len(got) > 0)
+			assert.Equal(t, *tt.wantObj, *tt.obj)
+		})
+	}
+}
+
+func TestResourceModifiers_ApplyResourceModifierRules_JSONMergePatch(t *testing.T) {
+	unstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)
+	o1, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToB), nil, nil)
+	assert.NoError(t, err)
+	cmWithLabelAToB := o1.(*unstructured.Unstructured)
+
+	o2, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToC), nil, nil)
+	assert.NoError(t, err)
+	cmWithLabelAToC := o2.(*unstructured.Unstructured)
+
+	o3, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithoutLabelA), nil, nil)
+	assert.NoError(t, err)
+	cmWithoutLabelA := o3.(*unstructured.Unstructured)
+
+	tests := []struct {
+		name          string
+		rm            *ResourceModifiers
+		obj           *unstructured.Unstructured
+		groupResource string
+		wantErr       bool
+		wantObj       *unstructured.Unstructured
+	}{
+		{
+			name: "update labels",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "configmaps",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":"c"}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToC.DeepCopy(),
+		},
+		{
+			name: "update labels in yaml format",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "configmaps",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `metadata:
+  labels:
+    a: c`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToC.DeepCopy(),
+		},
+		{
+			name: "delete labels",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "configmaps",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":null}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithoutLabelA.DeepCopy(),
+		},
+		{
+			name: "add labels",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "configmaps",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":"b"}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithoutLabelA.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToB.DeepCopy(),
+		},
+		{
+			name: "delete non-existing labels",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "configmaps",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":null}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithoutLabelA.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithoutLabelA.DeepCopy(),
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, nil, logrus.New())
+
+			assert.Equal(t, tt.wantErr, len(got) > 0)
+			assert.Equal(t, *tt.wantObj, *tt.obj)
+		})
+	}
+}
+
+func TestResourceModifiers_wildcard_in_GroupResource(t *testing.T) {
+	unstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)
+	o1, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToB), nil, nil)
+	assert.NoError(t, err)
+	cmWithLabelAToB := o1.(*unstructured.Unstructured)
+
+	o2, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToC), nil, nil)
+	assert.NoError(t, err)
+	cmWithLabelAToC := o2.(*unstructured.Unstructured)
+
+	tests := []struct {
+		name          string
+		rm            *ResourceModifiers
+		obj           *unstructured.Unstructured
+		groupResource string
+		wantErr       bool
+		wantObj       *unstructured.Unstructured
+	}{
+		{
+			name: "match all groups and resources",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "*",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":"c"}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToC.DeepCopy(),
+		},
+		{
+			name: "match all resources in group apps",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "*.apps",
+							Namespaces:    []string{"fake"},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":"c"}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "fake.apps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToC.DeepCopy(),
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, nil, logrus.New())
+
+			assert.Equal(t, tt.wantErr, len(got) > 0)
+			assert.Equal(t, *tt.wantObj, *tt.obj)
+		})
+	}
+}
+
+func TestResourceModifiers_conditional_patches(t *testing.T) {
+	unstructuredSerializer := yaml.NewDecodingSerializer(unstructured.UnstructuredJSONScheme)
+	o1, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToB), nil, nil)
+	assert.NoError(t, err)
+	cmWithLabelAToB := o1.(*unstructured.Unstructured)
+
+	o2, _, err := unstructuredSerializer.Decode([]byte(cmYAMLWithLabelAToC), nil, nil)
+	assert.NoError(t, err)
+	cmWithLabelAToC := o2.(*unstructured.Unstructured)
+
+	tests := []struct {
+		name          string
+		rm            *ResourceModifiers
+		obj           *unstructured.Unstructured
+		groupResource string
+		wantErr       bool
+		wantObj       *unstructured.Unstructured
+	}{
+		{
+			name: "match conditions and apply patches",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "*",
+							Namespaces:    []string{"fake"},
+							Matches: []MatchRule{
+								{
+									Path:  "/metadata/labels/a",
+									Value: "b",
+								},
+							},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":"c"}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToC.DeepCopy(),
+		},
+		{
+			name: "mismatch conditions and skip patches",
+			rm: &ResourceModifiers{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "*",
+							Namespaces:    []string{"fake"},
+							Matches: []MatchRule{
+								{
+									Path:  "/metadata/labels/a",
+									Value: "c",
+								},
+							},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":"c"}}}`,
+							},
+						},
+					},
+				},
+			},
+			obj:           cmWithLabelAToB.DeepCopy(),
+			groupResource: "configmaps",
+			wantErr:       false,
+			wantObj:       cmWithLabelAToB.DeepCopy(),
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got := tt.rm.ApplyResourceModifierRules(tt.obj, tt.groupResource, nil, logrus.New())
+
+			assert.Equal(t, tt.wantErr, len(got) > 0)
+			assert.Equal(t, *tt.wantObj, *tt.obj)
+		})
+	}
+}
+
 func TestJSONPatch_ToString(t *testing.T) {
 	type fields struct {
 		Operation string

internal/resourcemodifiers/resource_modifiers_validator.go

@@ -24,6 +24,21 @@ func (r *ResourceModifierRule) Validate() error {
 	if err := r.Conditions.Validate(); err != nil {
 		return err
 	}
+
+	count := 0
+	for _, size := range []int{
+		len(r.Patches),
+		len(r.MergePatches),
+		len(r.StrategicPatches),
+	} {
+		if size != 0 {
+			count++
+		}
+		if count >= 2 {
+			return fmt.Errorf("only one of patches, mergePatches, strategicPatches can be specified")
+		}
+	}
+
 	for _, patch := range r.Patches {
 		if err := patch.Validate(); err != nil {
 			return err

internal/resourcemodifiers/resource_modifiers_validator_test.go

@@ -129,6 +129,32 @@ func TestResourceModifiers_Validate(t *testing.T) {
 			},
 			wantErr: true,
 		},
+		{
+			name: "More than one patch type in a rule",
+			fields: fields{
+				Version: "v1",
+				ResourceModifierRules: []ResourceModifierRule{
+					{
+						Conditions: Conditions{
+							GroupResource: "*",
+						},
+						Patches: []JSONPatch{
+							{
+								Operation: "test",
+								Path:      "/spec/storageClassName",
+								Value:     "premium",
+							},
+						},
+						MergePatches: []JSONMergePatch{
+							{
+								PatchData: `{"metadata":{"labels":{"a":null}}}`,
+							},
+						},
+					},
+				},
+			},
+			wantErr: true,
+		},
 	}
 
 	for _, tt := range tests {

internal/resourcemodifiers/strategic_merge_patch.go

@@ -0,0 +1,143 @@
+package resourcemodifiers
+
+import (
+	"fmt"
+	"net/http"
+
+	"github.com/sirupsen/logrus"
+	apierrors "k8s.io/apimachinery/pkg/api/errors"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/schema"
+	"k8s.io/apimachinery/pkg/util/mergepatch"
+	"k8s.io/apimachinery/pkg/util/strategicpatch"
+	"k8s.io/apimachinery/pkg/util/validation/field"
+	kubejson "sigs.k8s.io/json"
+	"sigs.k8s.io/yaml"
+)
+
+type StrategicMergePatch struct {
+	PatchData string `json:"patchData,omitempty"`
+}
+
+type StrategicMergePatcher struct {
+	patches []StrategicMergePatch
+	scheme  *runtime.Scheme
+}
+
+func (p *StrategicMergePatcher) Patch(u *unstructured.Unstructured, _ logrus.FieldLogger) (*unstructured.Unstructured, error) {
+	gvk := u.GetObjectKind().GroupVersionKind()
+	schemaReferenceObj, err := p.scheme.New(gvk)
+	if err != nil {
+		return nil, err
+	}
+
+	origin := u.DeepCopy()
+	updated := u.DeepCopy()
+	for _, patch := range p.patches {
+		patchBytes, err := yaml.YAMLToJSON([]byte(patch.PatchData))
+		if err != nil {
+			return nil, fmt.Errorf("error in converting YAML to JSON %s", err)
+		}
+
+		err = strategicPatchObject(origin, patchBytes, updated, schemaReferenceObj)
+		if err != nil {
+			return nil, fmt.Errorf("error in applying Strategic Patch %s", err.Error())
+		}
+
+		origin = updated.DeepCopy()
+	}
+
+	return updated, nil
+}
+
+// strategicPatchObject applies a strategic merge patch of `patchBytes` to
+// `originalObject` and stores the result in `objToUpdate`.
+// It additionally returns the map[string]interface{} representation of the
+// `originalObject` and `patchBytes`.
+// NOTE: Both `originalObject` and `objToUpdate` are supposed to be versioned.
+func strategicPatchObject(
+	originalObject runtime.Object,
+	patchBytes []byte,
+	objToUpdate runtime.Object,
+	schemaReferenceObj runtime.Object,
+) error {
+	originalObjMap, err := runtime.DefaultUnstructuredConverter.ToUnstructured(originalObject)
+	if err != nil {
+		return err
+	}
+
+	patchMap := make(map[string]interface{})
+	var strictErrs []error
+	strictErrs, err = kubejson.UnmarshalStrict(patchBytes, &patchMap)
+	if err != nil {
+		return apierrors.NewBadRequest(err.Error())
+	}
+
+	if err := applyPatchToObject(originalObjMap, patchMap, objToUpdate, schemaReferenceObj, strictErrs); err != nil {
+		return err
+	}
+	return nil
+}
+
+// applyPatchToObject applies a strategic merge patch of <patchMap> to
+// <originalMap> and stores the result in <objToUpdate>.
+// NOTE: <objToUpdate> must be a versioned object.
+func applyPatchToObject(
+	originalMap map[string]interface{},
+	patchMap map[string]interface{},
+	objToUpdate runtime.Object,
+	schemaReferenceObj runtime.Object,
+	strictErrs []error,
+) error {
+	patchedObjMap, err := strategicpatch.StrategicMergeMapPatch(originalMap, patchMap, schemaReferenceObj)
+	if err != nil {
+		return interpretStrategicMergePatchError(err)
+	}
+
+	// Rather than serialize the patched map to JSON, then decode it to an object, we go directly from a map to an object
+	converter := runtime.DefaultUnstructuredConverter
+	if err := converter.FromUnstructuredWithValidation(patchedObjMap, objToUpdate, true); err != nil {
+		strictError, isStrictError := runtime.AsStrictDecodingError(err)
+		switch {
+		case !isStrictError:
+			// disregard any sttrictErrs, because it's an incomplete
+			// list of strict errors given that we don't know what fields were
+			// unknown because StrategicMergeMapPatch failed.
+			// Non-strict errors trump in this case.
+			return apierrors.NewInvalid(schema.GroupKind{}, "", field.ErrorList{
+				field.Invalid(field.NewPath("patch"), fmt.Sprintf("%+v", patchMap), err.Error()),
+			})
+		//case validationDirective == metav1.FieldValidationWarn:
+		//	addStrictDecodingWarnings(requestContext, append(strictErrs, strictError.Errors()...))
+		default:
+			strictDecodingError := runtime.NewStrictDecodingError(append(strictErrs, strictError.Errors()...))
+			return apierrors.NewInvalid(schema.GroupKind{}, "", field.ErrorList{
+				field.Invalid(field.NewPath("patch"), fmt.Sprintf("%+v", patchMap), strictDecodingError.Error()),
+			})
+		}
+	} else if len(strictErrs) > 0 {
+		switch {
+		//case validationDirective == metav1.FieldValidationWarn:
+		//	addStrictDecodingWarnings(requestContext, strictErrs)
+		default:
+			return apierrors.NewInvalid(schema.GroupKind{}, "", field.ErrorList{
+				field.Invalid(field.NewPath("patch"), fmt.Sprintf("%+v", patchMap), runtime.NewStrictDecodingError(strictErrs).Error()),
+			})
+		}
+	}
+
+	return nil
+}
+
+// interpretStrategicMergePatchError interprets the error type and returns an error with appropriate HTTP code.
+func interpretStrategicMergePatchError(err error) error {
+	switch err {
+	case mergepatch.ErrBadJSONDoc, mergepatch.ErrBadPatchFormatForPrimitiveList, mergepatch.ErrBadPatchFormatForRetainKeys, mergepatch.ErrBadPatchFormatForSetElementOrderList, mergepatch.ErrUnsupportedStrategicMergePatchFormat:
+		return apierrors.NewBadRequest(err.Error())
+	case mergepatch.ErrNoListOfLists, mergepatch.ErrPatchContentNotMatchRetainKeys:
+		return apierrors.NewGenericServerResponse(http.StatusUnprocessableEntity, "", schema.GroupResource{}, "", err.Error(), 0, false)
+	default:
+		return err
+	}
+}

internal/resourcemodifiers/strategic_merge_patch_test.go

@@ -0,0 +1,52 @@
+package resourcemodifiers
+
+import (
+	"testing"
+
+	"github.com/sirupsen/logrus"
+	"github.com/stretchr/testify/assert"
+	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/schema"
+	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
+)
+
+func TestStrategicMergePatchFailure(t *testing.T) {
+	tests := []struct {
+		name string
+		data string
+		kind string
+	}{
+		{
+			name: "patch with unknown kind",
+			data: "{}",
+			kind: "BadKind",
+		},
+		{
+			name: "patch with bad yaml",
+			data: "a: b:",
+			kind: "Pod",
+		},
+		{
+			name: "patch with bad json",
+			data: `{"a"::1}`,
+			kind: "Pod",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			scheme := runtime.NewScheme()
+			err := clientgoscheme.AddToScheme(scheme)
+			assert.NoError(t, err)
+			pt := &StrategicMergePatcher{
+				patches: []StrategicMergePatch{{PatchData: tt.data}},
+				scheme:  scheme,
+			}
+
+			u := &unstructured.Unstructured{}
+			u.SetGroupVersionKind(schema.GroupVersionKind{Version: "v1", Kind: tt.kind})
+			_, err = pt.Patch(u, logrus.New())
+			assert.Error(t, err)
+		})
+	}
+}

internal/storage/storagelocation_test.go

@@ -26,8 +26,8 @@ import (
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/scheme"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
+	"github.com/vmware-tanzu/velero/pkg/util"
 )
 
 func TestIsReadyToValidate(t *testing.T) {
@@ -163,7 +163,7 @@ func TestListBackupStorageLocations(t *testing.T) {
 		t.Run(tt.name, func(t *testing.T) {
 			g := NewWithT(t)
 
-			client := fake.NewClientBuilder().WithScheme(scheme.Scheme).WithRuntimeObjects(tt.backupLocations).Build()
+			client := fake.NewClientBuilder().WithScheme(util.VeleroScheme).WithRuntimeObjects(tt.backupLocations).Build()
 			if tt.expectError {
 				_, err := ListBackupStorageLocations(context.Background(), client, "ns-1")
 				g.Expect(err).NotTo(BeNil())

pkg/apis/velero/v1/backup_types.go

@@ -261,12 +261,12 @@ type ExecHook struct {
 type HookErrorMode string
 
 const (
-	// HookErrorModeContinue means that an error from a hook is acceptable, and the backup can
-	// proceed.
+	// HookErrorModeContinue means that an error from a hook is acceptable and the backup/restore can
+	// proceed with the rest of hooks' execution. This backup/restore should be in `PartiallyFailed` status.
 	HookErrorModeContinue HookErrorMode = "Continue"
 
-	// HookErrorModeFail means that an error from a hook is problematic, and the backup should be in
-	// error.
+	// HookErrorModeFail means that an error from a hook is problematic and Velero should stop executing following hooks.
+	// This backup/restore should be in `PartiallyFailed` status.
 	HookErrorModeFail HookErrorMode = "Fail"
 )
 

pkg/apis/velero/v1/zz_generated.deepcopy.go

@@ -784,6 +784,11 @@ func (in *ExecRestoreHook) DeepCopyInto(out *ExecRestoreHook) {
 	}
 	out.ExecTimeout = in.ExecTimeout
 	out.WaitTimeout = in.WaitTimeout
+	if in.WaitForReady != nil {
+		in, out := &in.WaitForReady, &out.WaitForReady
+		*out = new(bool)
+		**out = **in
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ExecRestoreHook.

pkg/apis/velero/v2alpha1/data_download_types.go

@@ -131,6 +131,7 @@ type DataDownloadStatus struct {
 // +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp",description="Time duration since this DataDownload was created"
 // +kubebuilder:printcolumn:name="Node",type="string",JSONPath=".status.node",description="Name of the node where the DataDownload is processed"
 
+// DataDownload acts as the protocol between data mover plugins and data mover controller for the datamover restore operation
 type DataDownload struct {
 	metav1.TypeMeta `json:",inline"`
 

pkg/apis/velero/v2alpha1/data_upload_types.go

@@ -161,6 +161,7 @@ type DataUploadStatus struct {
 // +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp",description="Time duration since this DataUpload was created"
 // +kubebuilder:printcolumn:name="Node",type="string",JSONPath=".status.node",description="Name of the node where the DataUpload is processed"
 
+// DataUpload acts as the protocol between data mover plugins and data mover controller for the datamover backup operation
 type DataUpload struct {
 	metav1.TypeMeta `json:",inline"`
 

pkg/client/factory.go

@@ -33,7 +33,6 @@ import (
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	velerov2alpha1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
 )
 
 //go:generate mockery --name Factory
@@ -42,9 +41,6 @@ import (
 type Factory interface {
 	// BindFlags binds common flags (--kubeconfig, --namespace) to the passed-in FlagSet.
 	BindFlags(flags *pflag.FlagSet)
-	// Client returns a VeleroClient. It uses the following priority to specify the cluster
-	// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
-	Client() (clientset.Interface, error)
 	// KubeClient returns a Kubernetes client. It uses the following priority to specify the cluster
 	// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
 	KubeClient() (kubernetes.Interface, error)
@@ -115,19 +111,6 @@ func (f *factory) ClientConfig() (*rest.Config, error) {
 	return Config(f.kubeconfig, f.kubecontext, f.baseName, f.clientQPS, f.clientBurst)
 }
 
-func (f *factory) Client() (clientset.Interface, error) {
-	clientConfig, err := f.ClientConfig()
-	if err != nil {
-		return nil, err
-	}
-
-	veleroClient, err := clientset.NewForConfig(clientConfig)
-	if err != nil {
-		return nil, errors.WithStack(err)
-	}
-	return veleroClient, nil
-}
-
 func (f *factory) KubeClient() (kubernetes.Interface, error) {
 	clientConfig, err := f.ClientConfig()
 	if err != nil {

pkg/client/factory_test.go

@@ -112,11 +112,6 @@ func TestFactory(t *testing.T) {
 			assert.Equal(t, test.burst, clientConfig.Burst)
 			strings.Contains(clientConfig.UserAgent, test.baseName)
 
-			client, _ := f.Client()
-			_, e := client.Discovery().ServerGroups()
-			assert.Contains(t, e.Error(), fmt.Sprintf("Get \"%s/api?timeout=", test.expectedHost))
-			assert.NotNil(t, client)
-
 			kubeClient, _ := f.KubeClient()
 			group := kubeClient.NodeV1().RESTClient().APIVersion().Group
 			assert.NotNil(t, kubeClient)

pkg/client/mocks/Factory.go

@@ -1,4 +1,4 @@
-// Code generated by mockery v2.30.1. DO NOT EDIT.
+// Code generated by mockery v2.20.0. DO NOT EDIT.
 
 package mocks
 
@@ -13,8 +13,6 @@ import (
 	pkgclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	rest "k8s.io/client-go/rest"
-
-	versioned "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
 )
 
 // Factory is an autogenerated mock type for the Factory type
@@ -27,32 +25,6 @@ func (_m *Factory) BindFlags(flags *pflag.FlagSet) {
 	_m.Called(flags)
 }
 
-// Client provides a mock function with given fields:
-func (_m *Factory) Client() (versioned.Interface, error) {
-	ret := _m.Called()
-
-	var r0 versioned.Interface
-	var r1 error
-	if rf, ok := ret.Get(0).(func() (versioned.Interface, error)); ok {
-		return rf()
-	}
-	if rf, ok := ret.Get(0).(func() versioned.Interface); ok {
-		r0 = rf()
-	} else {
-		if ret.Get(0) != nil {
-			r0 = ret.Get(0).(versioned.Interface)
-		}
-	}
-
-	if rf, ok := ret.Get(1).(func() error); ok {
-		r1 = rf()
-	} else {
-		r1 = ret.Error(1)
-	}
-
-	return r0, r1
-}
-
 // ClientConfig provides a mock function with given fields:
 func (_m *Factory) ClientConfig() (*rest.Config, error) {
 	ret := _m.Called()
@@ -212,12 +184,13 @@ func (_m *Factory) SetClientQPS(_a0 float32) {
 	_m.Called(_a0)
 }
 
-// NewFactory creates a new instance of Factory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
-// The first argument is typically a *testing.T value.
-func NewFactory(t interface {
+type mockConstructorTestingTNewFactory interface {
 	mock.TestingT
 	Cleanup(func())
-}) *Factory {
+}
+
+// NewFactory creates a new instance of Factory. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
+func NewFactory(t mockConstructorTestingTNewFactory) *Factory {
 	mock := &Factory{}
 	mock.Mock.Test(t)
 

pkg/client/retry.go

@@ -25,20 +25,14 @@ import (
 )
 
 func CreateRetryGenerateName(client kbclient.Client, ctx context.Context, obj kbclient.Object) error {
-	return CreateRetryGenerateNameWithFunc(obj, func() error {
-		return client.Create(ctx, obj, &kbclient.CreateOptions{})
-	})
-}
-
-func CreateRetryGenerateNameWithFunc(obj kbclient.Object, createFn func() error) error {
 	retryCreateFn := func() error {
 		// needed to ensure that the name from the failed create isn't left on the object between retries
 		obj.SetName("")
-		return createFn()
+		return client.Create(ctx, obj, &kbclient.CreateOptions{})
 	}
 	if obj.GetGenerateName() != "" && obj.GetName() == "" {
 		return retry.OnError(retry.DefaultRetry, apierrors.IsAlreadyExists, retryCreateFn)
 	} else {
-		return createFn()
+		return client.Create(ctx, obj, &kbclient.CreateOptions{})
 	}
 }

pkg/cmd/cli/debug/debug.go

@@ -30,9 +30,12 @@ import (
 	"github.com/spf13/cobra"
 	"github.com/spf13/pflag"
 	"github.com/vmware-tanzu/crash-diagnostics/exec"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	appsv1 "k8s.io/api/apps/v1"
+	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/client-go/tools/clientcmd"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
+	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
 	"github.com/vmware-tanzu/velero/pkg/cmd"
 )
@@ -110,30 +113,32 @@ func (o *option) complete(f client.Factory, fs *pflag.FlagSet) error {
 }
 
 func (o *option) validate(f client.Factory) error {
-	kubeClient, err := f.KubeClient()
+	crClient, err := f.KubebuilderClient()
 	if err != nil {
 		return err
 	}
-	l, err := kubeClient.AppsV1().Deployments(o.namespace).List(context.TODO(), metav1.ListOptions{
-		LabelSelector: "component=velero",
+	deploymentList := new(appsv1.DeploymentList)
+	selector, err := labels.Parse("component=velero")
+	cmd.CheckError(err)
+	err = crClient.List(context.TODO(), deploymentList, &ctrlclient.ListOptions{
+		Namespace:     o.namespace,
+		LabelSelector: selector,
 	})
 	if err != nil {
 		return errors.Wrap(err, "failed to check velero deployment")
 	}
-	if len(l.Items) == 0 {
+	if len(deploymentList.Items) == 0 {
 		return fmt.Errorf("velero deployment does not exist in namespace: %s", o.namespace)
 	}
-	veleroClient, err := f.Client()
-	if err != nil {
-		return err
-	}
 	if len(o.backup) > 0 {
-		if _, err := veleroClient.VeleroV1().Backups(o.namespace).Get(context.TODO(), o.backup, metav1.GetOptions{}); err != nil {
+		backup := new(velerov1api.Backup)
+		if err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: o.namespace, Name: o.backup}, backup); err != nil {
 			return err
 		}
 	}
 	if len(o.restore) > 0 {
-		if _, err := veleroClient.VeleroV1().Restores(o.namespace).Get(context.TODO(), o.restore, metav1.GetOptions{}); err != nil {
+		restore := new(velerov1api.Restore)
+		if err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: o.namespace, Name: o.restore}, restore); err != nil {
 			return err
 		}
 	}

pkg/cmd/cli/nodeagent/server.go

@@ -490,7 +490,7 @@ func (s *nodeAgentServer) markInProgressPVRsFailed(client ctrlclient.Client) {
 }
 
 func (s *nodeAgentServer) getDataPathConcurrentNum(defaultNum int, logger logrus.FieldLogger) int {
-	configs, err := nodeagent.GetConfigs(s.ctx, s.namespace, s.kubeClient.CoreV1())
+	configs, err := nodeagent.GetConfigs(s.ctx, s.namespace, s.kubeClient)
 	if err != nil {
 		logger.WithError(err).Warn("Failed to get node agent configs")
 		return defaultNum

pkg/cmd/cli/repo/get.go

@@ -21,6 +21,8 @@ import (
 
 	"github.com/spf13/cobra"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
@@ -38,19 +40,25 @@ func NewGetCommand(f client.Factory, use string) *cobra.Command {
 			err := output.ValidateFlags(c)
 			cmd.CheckError(err)
 
-			veleroClient, err := f.Client()
+			crClient, err := f.KubebuilderClient()
 			cmd.CheckError(err)
 
 			var repos *api.BackupRepositoryList
 			if len(args) > 0 {
 				repos = new(api.BackupRepositoryList)
 				for _, name := range args {
-					repo, err := veleroClient.VeleroV1().BackupRepositories(f.Namespace()).Get(context.TODO(), name, metav1.GetOptions{})
+					repo := new(api.BackupRepository)
+					err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: f.Namespace(), Name: name}, repo)
 					cmd.CheckError(err)
 					repos.Items = append(repos.Items, *repo)
 				}
 			} else {
-				repos, err = veleroClient.VeleroV1().BackupRepositories(f.Namespace()).List(context.TODO(), listOptions)
+				selector := labels.NewSelector()
+				if listOptions.LabelSelector != "" {
+					selector, err = labels.Parse(listOptions.LabelSelector)
+					cmd.CheckError(err)
+				}
+				err = crClient.List(context.TODO(), repos, &ctrlclient.ListOptions{LabelSelector: selector})
 				cmd.CheckError(err)
 			}
 

pkg/cmd/cli/restore/logs.go

@@ -24,7 +24,7 @@ import (
 
 	"github.com/spf13/cobra"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
@@ -49,13 +49,11 @@ func NewLogsCommand(f client.Factory) *cobra.Command {
 		Run: func(c *cobra.Command, args []string) {
 			restoreName := args[0]
 
-			veleroClient, err := f.Client()
-			cmd.CheckError(err)
-
 			kbClient, err := f.KubebuilderClient()
 			cmd.CheckError(err)
 
-			restore, err := veleroClient.VeleroV1().Restores(f.Namespace()).Get(context.TODO(), restoreName, metav1.GetOptions{})
+			restore := new(velerov1api.Restore)
+			err = kbClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: f.Namespace(), Name: restoreName}, restore)
 			if apierrors.IsNotFound(err) {
 				cmd.Exit("Restore %q does not exist.", restoreName)
 			} else if err != nil {

pkg/cmd/cli/schedule/create.go

@@ -115,7 +115,7 @@ func (o *CreateOptions) Complete(args []string, f client.Factory) error {
 func (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {
 	var orders map[string]string
 
-	veleroClient, err := f.Client()
+	crClient, err := f.KubebuilderClient()
 	if err != nil {
 		return err
 	}
@@ -171,7 +171,7 @@ func (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {
 		return err
 	}
 
-	_, err = veleroClient.VeleroV1().Schedules(schedule.Namespace).Create(context.TODO(), schedule, metav1.CreateOptions{})
+	err = crClient.Create(context.TODO(), schedule)
 	if err != nil {
 		return err
 	}

pkg/cmd/cli/schedule/describe.go

@@ -22,6 +22,8 @@ import (
 
 	"github.com/spf13/cobra"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	v1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
@@ -36,19 +38,25 @@ func NewDescribeCommand(f client.Factory, use string) *cobra.Command {
 		Use:   use + " [NAME1] [NAME2] [NAME...]",
 		Short: "Describe schedules",
 		Run: func(c *cobra.Command, args []string) {
-			veleroClient, err := f.Client()
+			crClient, err := f.KubebuilderClient()
 			cmd.CheckError(err)
 
 			var schedules *v1.ScheduleList
 			if len(args) > 0 {
 				schedules = new(v1.ScheduleList)
 				for _, name := range args {
-					schedule, err := veleroClient.VeleroV1().Schedules(f.Namespace()).Get(context.TODO(), name, metav1.GetOptions{})
+					schedule := new(v1.Schedule)
+					err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: f.Namespace(), Name: name}, schedule)
 					cmd.CheckError(err)
 					schedules.Items = append(schedules.Items, *schedule)
 				}
 			} else {
-				schedules, err = veleroClient.VeleroV1().Schedules(f.Namespace()).List(context.TODO(), listOptions)
+				selector := labels.NewSelector()
+				if listOptions.LabelSelector != "" {
+					selector, err = labels.Parse(listOptions.LabelSelector)
+					cmd.CheckError(err)
+				}
+				err = crClient.List(context.TODO(), schedules, &ctrlclient.ListOptions{LabelSelector: selector})
 				cmd.CheckError(err)
 			}
 

pkg/cmd/cli/schedule/get.go

@@ -21,6 +21,8 @@ import (
 
 	"github.com/spf13/cobra"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/labels"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
@@ -38,19 +40,25 @@ func NewGetCommand(f client.Factory, use string) *cobra.Command {
 			err := output.ValidateFlags(c)
 			cmd.CheckError(err)
 
-			veleroClient, err := f.Client()
+			crClient, err := f.KubebuilderClient()
 			cmd.CheckError(err)
 
 			var schedules *api.ScheduleList
 			if len(args) > 0 {
 				schedules = new(api.ScheduleList)
 				for _, name := range args {
-					schedule, err := veleroClient.VeleroV1().Schedules(f.Namespace()).Get(context.TODO(), name, metav1.GetOptions{})
+					schedule := new(api.Schedule)
+					err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: name, Namespace: f.Namespace()}, schedule)
 					cmd.CheckError(err)
 					schedules.Items = append(schedules.Items, *schedule)
 				}
 			} else {
-				schedules, err = veleroClient.VeleroV1().Schedules(f.Namespace()).List(context.TODO(), listOptions)
+				selector := labels.NewSelector()
+				if listOptions.LabelSelector != "" {
+					selector, err = labels.Parse(listOptions.LabelSelector)
+					cmd.CheckError(err)
+				}
+				err := crClient.List(context.TODO(), schedules, &ctrlclient.ListOptions{LabelSelector: selector})
 				cmd.CheckError(err)
 			}
 

pkg/cmd/cli/schedule/pause.go

@@ -25,6 +25,7 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/labels"
 	kubeerrs "k8s.io/apimachinery/pkg/util/errors"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
@@ -63,7 +64,7 @@ func NewPauseCommand(f client.Factory, use string) *cobra.Command {
 }
 
 func runPause(f client.Factory, o *cli.SelectOptions, paused bool) error {
-	client, err := f.Client()
+	crClient, err := f.KubebuilderClient()
 	if err != nil {
 		return err
 	}
@@ -75,7 +76,8 @@ func runPause(f client.Factory, o *cli.SelectOptions, paused bool) error {
 	switch {
 	case len(o.Names) > 0:
 		for _, name := range o.Names {
-			schedule, err := client.VeleroV1().Schedules(f.Namespace()).Get(context.TODO(), name, metav1.GetOptions{})
+			schedule := new(velerov1api.Schedule)
+			err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: name, Namespace: f.Namespace()}, schedule)
 			if err != nil {
 				errs = append(errs, errors.WithStack(err))
 				continue
@@ -83,11 +85,16 @@ func runPause(f client.Factory, o *cli.SelectOptions, paused bool) error {
 			schedules = append(schedules, schedule)
 		}
 	default:
-		selector := labels.Everything().String()
+		selector := labels.Everything()
 		if o.Selector.LabelSelector != nil {
-			selector = o.Selector.String()
+			convertedSelector, err := metav1.LabelSelectorAsSelector(o.Selector.LabelSelector)
+			if err != nil {
+				return errors.WithStack(err)
+			}
+			selector = convertedSelector
 		}
-		res, err := client.VeleroV1().Schedules(f.Namespace()).List(context.TODO(), metav1.ListOptions{
+		res := new(velerov1api.ScheduleList)
+		err := crClient.List(context.TODO(), res, &ctrlclient.ListOptions{
 			LabelSelector: selector,
 		})
 		if err != nil {
@@ -113,7 +120,7 @@ func runPause(f client.Factory, o *cli.SelectOptions, paused bool) error {
 			continue
 		}
 		schedule.Spec.Paused = paused
-		if _, err := client.VeleroV1().Schedules(schedule.Namespace).Update(context.TODO(), schedule, metav1.UpdateOptions{}); err != nil {
+		if err := crClient.Update(context.TODO(), schedule); err != nil {
 			return errors.Wrapf(err, "failed to update schedule %s", schedule.Name)
 		}
 		fmt.Printf("Schedule %s %s successfully\n", schedule.Name, msg)

pkg/cmd/cli/snapshotlocation/create.go

@@ -124,12 +124,12 @@ func (o *CreateOptions) Run(c *cobra.Command, f client.Factory) error {
 		return err
 	}
 
-	client, err := f.Client()
+	client, err := f.KubebuilderClient()
 	if err != nil {
 		return err
 	}
 
-	if _, err := client.VeleroV1().VolumeSnapshotLocations(volumeSnapshotLocation.Namespace).Create(context.TODO(), volumeSnapshotLocation, metav1.CreateOptions{}); err != nil {
+	if err := client.Create(context.TODO(), volumeSnapshotLocation); err != nil {
 		return errors.WithStack(err)
 	}
 

pkg/cmd/cli/snapshotlocation/get.go

@@ -21,6 +21,7 @@ import (
 
 	"github.com/spf13/cobra"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	kbclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/client"
@@ -36,18 +37,19 @@ func NewGetCommand(f client.Factory, use string) *cobra.Command {
 		Run: func(c *cobra.Command, args []string) {
 			err := output.ValidateFlags(c)
 			cmd.CheckError(err)
-			veleroClient, err := f.Client()
+			client, err := f.KubebuilderClient()
 			cmd.CheckError(err)
-			var locations *api.VolumeSnapshotLocationList
+			locations := new(api.VolumeSnapshotLocationList)
+
 			if len(args) > 0 {
-				locations = new(api.VolumeSnapshotLocationList)
 				for _, name := range args {
-					location, err := veleroClient.VeleroV1().VolumeSnapshotLocations(f.Namespace()).Get(context.TODO(), name, metav1.GetOptions{})
+					location := new(api.VolumeSnapshotLocation)
+					err := client.Get(context.TODO(), kbclient.ObjectKey{Namespace: f.Namespace(), Name: name}, location)
 					cmd.CheckError(err)
 					locations.Items = append(locations.Items, *location)
 				}
 			} else {
-				locations, err = veleroClient.VeleroV1().VolumeSnapshotLocations(f.Namespace()).List(context.TODO(), listOptions)
+				err = client.List(context.TODO(), locations, &kbclient.ListOptions{Namespace: f.Namespace()})
 				cmd.CheckError(err)
 			}
 			_, err = output.PrintWithFormat(c, locations)

pkg/cmd/server/plugin/plugin.go

@@ -152,12 +152,12 @@ func newPodVolumeRestoreItemAction(f client.Factory) plugincommon.HandlerInitial
 			return nil, err
 		}
 
-		veleroClient, err := f.Client()
+		crClient, err := f.KubebuilderClient()
 		if err != nil {
 			return nil, err
 		}
 
-		return restore.NewPodVolumeRestoreAction(logger, client.CoreV1().ConfigMaps(f.Namespace()), veleroClient.VeleroV1().PodVolumeBackups(f.Namespace())), nil
+		return restore.NewPodVolumeRestoreAction(logger, client.CoreV1().ConfigMaps(f.Namespace()), crClient), nil
 	}
 }
 

pkg/cmd/server/server.go

@@ -66,7 +66,6 @@ import (
 	"github.com/vmware-tanzu/velero/pkg/controller"
 	velerodiscovery "github.com/vmware-tanzu/velero/pkg/discovery"
 	"github.com/vmware-tanzu/velero/pkg/features"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
 	"github.com/vmware-tanzu/velero/pkg/itemoperationmap"
 	"github.com/vmware-tanzu/velero/pkg/metrics"
 	"github.com/vmware-tanzu/velero/pkg/nodeagent"
@@ -250,12 +249,12 @@ type server struct {
 	metricsAddress        string
 	kubeClientConfig      *rest.Config
 	kubeClient            kubernetes.Interface
-	veleroClient          clientset.Interface
 	discoveryClient       discovery.DiscoveryInterface
 	discoveryHelper       velerodiscovery.Helper
 	dynamicClient         dynamic.Interface
 	csiSnapshotClient     *snapshotv1client.Clientset
 	csiSnapshotLister     snapshotv1listers.VolumeSnapshotLister
+	crClient              ctrlclient.Client
 	ctx                   context.Context
 	cancelFunc            context.CancelFunc
 	logger                logrus.FieldLogger
@@ -295,12 +294,12 @@ func newServer(f client.Factory, config serverConfig, logger *logrus.Logger) (*s
 		return nil, err
 	}
 
-	veleroClient, err := f.Client()
+	dynamicClient, err := f.DynamicClient()
 	if err != nil {
 		return nil, err
 	}
 
-	dynamicClient, err := f.DynamicClient()
+	crClient, err := f.KubebuilderClient()
 	if err != nil {
 		return nil, err
 	}
@@ -375,14 +374,20 @@ func newServer(f client.Factory, config serverConfig, logger *logrus.Logger) (*s
 		return nil, err
 	}
 
+	var discoveryClient *discovery.DiscoveryClient
+	if discoveryClient, err = discovery.NewDiscoveryClientForConfig(clientConfig); err != nil {
+		cancelFunc()
+		return nil, err
+	}
+
 	s := &server{
 		namespace:             f.Namespace(),
 		metricsAddress:        config.metricsAddress,
 		kubeClientConfig:      clientConfig,
 		kubeClient:            kubeClient,
-		veleroClient:          veleroClient,
-		discoveryClient:       veleroClient.Discovery(),
+		discoveryClient:       discoveryClient,
 		dynamicClient:         dynamicClient,
+		crClient:              crClient,
 		ctx:                   ctx,
 		cancelFunc:            cancelFunc,
 		logger:                logger,
@@ -727,6 +732,11 @@ func (s *server) runControllers(defaultVolumeSnapshotLocations map[string]string
 		s.logger.Fatal(err, "unable to create controller", "controller", controller.BackupStorageLocation)
 	}
 
+	pvbInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov1api.PodVolumeBackup{})
+	if err != nil {
+		s.logger.Fatal(err, "fail to get controller-runtime informer from manager for PVB")
+	}
+
 	if _, ok := enabledRuntimeControllers[controller.Backup]; ok {
 		backupper, err := backup.NewKubernetesBackupper(
 			s.mgr.GetClient(),
@@ -736,10 +746,8 @@ func (s *server) runControllers(defaultVolumeSnapshotLocations map[string]string
 			podvolume.NewBackupperFactory(
 				s.repoLocker,
 				s.repoEnsurer,
-				s.veleroClient,
-				s.kubeClient.CoreV1(),
-				s.kubeClient.CoreV1(),
-				s.kubeClient.CoreV1(),
+				s.crClient,
+				pvbInformer,
 				s.logger,
 			),
 			s.config.podVolumeOperationTimeout,
@@ -818,10 +826,8 @@ func (s *server) runControllers(defaultVolumeSnapshotLocations map[string]string
 			podvolume.NewBackupperFactory(
 				s.repoLocker,
 				s.repoEnsurer,
-				s.veleroClient,
-				s.kubeClient.CoreV1(),
-				s.kubeClient.CoreV1(),
-				s.kubeClient.CoreV1(),
+				s.crClient,
+				pvbInformer,
 				s.logger,
 			),
 			s.config.podVolumeOperationTimeout,
@@ -909,6 +915,11 @@ func (s *server) runControllers(defaultVolumeSnapshotLocations map[string]string
 		}
 	}
 
+	pvrInformer, err := s.mgr.GetCache().GetInformer(s.ctx, &velerov1api.PodVolumeRestore{})
+	if err != nil {
+		s.logger.Fatal(err, "fail to get controller-runtime informer from manager for PVR")
+	}
+
 	if _, ok := enabledRuntimeControllers[controller.Restore]; ok {
 		restorer, err := restore.NewKubernetesRestorer(
 			s.discoveryHelper,
@@ -918,10 +929,9 @@ func (s *server) runControllers(defaultVolumeSnapshotLocations map[string]string
 			podvolume.NewRestorerFactory(
 				s.repoLocker,
 				s.repoEnsurer,
-				s.veleroClient,
-				s.kubeClient.CoreV1(),
-				s.kubeClient.CoreV1(),
 				s.kubeClient,
+				s.crClient,
+				pvrInformer,
 				s.logger,
 			),
 			s.config.podVolumeOperationTimeout,

pkg/discovery/helper_test.go

@@ -30,7 +30,6 @@ import (
 	clientgotesting "k8s.io/client-go/testing"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	discoverymocks "github.com/vmware-tanzu/velero/pkg/discovery/mocks"
 	"github.com/vmware-tanzu/velero/pkg/features"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
 	"github.com/vmware-tanzu/velero/pkg/util/logging"
@@ -548,39 +547,31 @@ func TestHelper_refreshServerPreferredResources(t *testing.T) {
 	}
 
 	tests := []struct {
-		name          string
-		isGetResError bool
+		name        string
+		expectedErr error
 	}{
 		{
-			name: "success get preferred resources",
+			name:        "success get preferred resources",
+			expectedErr: nil,
 		},
 		{
-			name:          "failed to get preferred resources",
-			isGetResError: true,
+			name:        "failed to get preferred resources",
+			expectedErr: errors.New("Failed to discover preferred resources"),
 		},
 	}
 
 	for _, tc := range tests {
 		t.Run(tc.name, func(t *testing.T) {
-			fakeClient := discoverymocks.NewServerResourcesInterface(t)
-
-			if tc.isGetResError {
-				fakeClient.On("ServerPreferredResources").Return(nil, errors.New("Failed to discover preferred resources"))
-			} else {
-				fakeClient.On("ServerPreferredResources").Return(apiList, nil)
-			}
+			fakeClient := velerotest.NewFakeServerResourcesInterface(apiList, []*metav1.APIGroup{}, map[schema.GroupVersion]error{}, tc.expectedErr)
 
 			resources, err := refreshServerPreferredResources(fakeClient, logrus.New())
 
-			if tc.isGetResError {
+			if tc.expectedErr != nil {
 				assert.NotNil(t, err)
-				assert.Nil(t, resources)
 			} else {
 				assert.Nil(t, err)
 				assert.NotNil(t, resources)
 			}
-
-			fakeClient.AssertExpectations(t)
 		})
 	}
 }
@@ -612,41 +603,31 @@ func TestHelper_refreshServerGroupsAndResources(t *testing.T) {
 		},
 	}
 	tests := []struct {
-		name          string
-		isGetResError bool
+		name        string
+		expectedErr error
 	}{
 		{
 			name: "success get service groups and resouorces",
 		},
 		{
-			name:          "failed to service groups and resouorces",
-			isGetResError: true,
+			name:        "failed to service groups and resouorces",
+			expectedErr: errors.New("Failed to discover service groups and resouorces"),
 		},
 	}
 
 	for _, tc := range tests {
 		t.Run(tc.name, func(t *testing.T) {
-			fakeClient := discoverymocks.NewServerResourcesInterface(t)
-
-			if tc.isGetResError {
-				fakeClient.On("ServerGroupsAndResources").Return(nil, nil, errors.New("Failed to discover service groups and resouorces"))
-			} else {
-				fakeClient.On("ServerGroupsAndResources").Return(apiGroup, apiList, nil)
-			}
+			fakeClient := velerotest.NewFakeServerResourcesInterface(apiList, apiGroup, map[schema.GroupVersion]error{}, tc.expectedErr)
 
 			serverGroups, serverResources, err := refreshServerGroupsAndResources(fakeClient, logrus.New())
 
-			if tc.isGetResError {
+			if tc.expectedErr != nil {
 				assert.NotNil(t, err)
-				assert.Nil(t, serverGroups)
-				assert.Nil(t, serverResources)
 			} else {
 				assert.Nil(t, err)
 				assert.NotNil(t, serverGroups)
 				assert.NotNil(t, serverResources)
 			}
-
-			fakeClient.AssertExpectations(t)
 		})
 	}
 }

pkg/discovery/mocks/Helper.go

@@ -1,4 +1,4 @@
-// Code generated by mockery v2.30.1. DO NOT EDIT.
+// Code generated by mockery v2.20.0. DO NOT EDIT.
 
 package mocks
 
@@ -140,12 +140,13 @@ func (_m *Helper) ServerVersion() *version.Info {
 	return r0
 }
 
-// NewHelper creates a new instance of Helper. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
-// The first argument is typically a *testing.T value.
-func NewHelper(t interface {
+type mockConstructorTestingTNewHelper interface {
 	mock.TestingT
 	Cleanup(func())
-}) *Helper {
+}
+
+// NewHelper creates a new instance of Helper. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
+func NewHelper(t mockConstructorTestingTNewHelper) *Helper {
 	mock := &Helper{}
 	mock.Mock.Test(t)
 
@@ -153,84 +154,3 @@ func NewHelper(t interface {
 
 	return mock
 }
-
-// serverResourcesInterface is an autogenerated mock type for the serverResourcesInterface type
-type serverResourcesInterface struct {
-	mock.Mock
-}
-
-// ServerGroupsAndResources provides a mock function with given fields:
-func (_m *serverResourcesInterface) ServerGroupsAndResources() ([]*v1.APIGroup, []*v1.APIResourceList, error) {
-	ret := _m.Called()
-
-	var r0 []*v1.APIGroup
-	var r1 []*v1.APIResourceList
-	var r2 error
-	if rf, ok := ret.Get(0).(func() ([]*v1.APIGroup, []*v1.APIResourceList, error)); ok {
-		return rf()
-	}
-	if rf, ok := ret.Get(0).(func() []*v1.APIGroup); ok {
-		r0 = rf()
-	} else {
-		if ret.Get(0) != nil {
-			r0 = ret.Get(0).([]*v1.APIGroup)
-		}
-	}
-
-	if rf, ok := ret.Get(1).(func() []*v1.APIResourceList); ok {
-		r1 = rf()
-	} else {
-		if ret.Get(1) != nil {
-			r1 = ret.Get(1).([]*v1.APIResourceList)
-		}
-	}
-
-	if rf, ok := ret.Get(2).(func() error); ok {
-		r2 = rf()
-	} else {
-		r2 = ret.Error(2)
-	}
-
-	return r0, r1, r2
-}
-
-// ServerPreferredResources provides a mock function with given fields:
-func (_m *serverResourcesInterface) ServerPreferredResources() ([]*v1.APIResourceList, error) {
-	ret := _m.Called()
-
-	var r0 []*v1.APIResourceList
-	var r1 error
-	if rf, ok := ret.Get(0).(func() ([]*v1.APIResourceList, error)); ok {
-		return rf()
-	}
-	if rf, ok := ret.Get(0).(func() []*v1.APIResourceList); ok {
-		r0 = rf()
-	} else {
-		if ret.Get(0) != nil {
-			r0 = ret.Get(0).([]*v1.APIResourceList)
-		}
-	}
-
-	if rf, ok := ret.Get(1).(func() error); ok {
-		r1 = rf()
-	} else {
-		r1 = ret.Error(1)
-	}
-
-	return r0, r1
-}
-
-type mockConstructorTestingTnewServerResourcesInterface interface {
-	mock.TestingT
-	Cleanup(func())
-}
-
-// NewServerResourcesInterface creates a new instance of serverResourcesInterface. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
-func NewServerResourcesInterface(t mockConstructorTestingTnewServerResourcesInterface) *serverResourcesInterface {
-	mock := &serverResourcesInterface{}
-	mock.Mock.Test(t)
-
-	t.Cleanup(func() { mock.AssertExpectations(t) })
-
-	return mock
-}

pkg/exposer/csi_snapshot.go

@@ -204,6 +204,7 @@ func (e *csiSnapshotExposer) GetExposed(ctx context.Context, ownerObject corev1.
 
 	backupPodName := ownerObject.Name
 	backupPVCName := ownerObject.Name
+	volumeName := string(ownerObject.UID)
 
 	curLog := e.log.WithFields(logrus.Fields{
 		"owner": ownerObject.Name,
@@ -232,7 +233,20 @@ func (e *csiSnapshotExposer) GetExposed(ctx context.Context, ownerObject corev1.
 
 	curLog.WithField("backup pvc", backupPVCName).Info("Backup PVC is bound")
 
-	return &ExposeResult{ByPod: ExposeByPod{HostingPod: pod, VolumeName: pod.Spec.Volumes[0].Name}}, nil
+	i := 0
+	for i = 0; i < len(pod.Spec.Volumes); i++ {
+		if pod.Spec.Volumes[i].Name == volumeName {
+			break
+		}
+	}
+
+	if i == len(pod.Spec.Volumes) {
+		return nil, errors.Errorf("backup pod %s doesn't have the expected backup volume", pod.Name)
+	}
+
+	curLog.WithField("pod", pod.Name).Infof("Backup volume is found in pod at index %v", i)
+
+	return &ExposeResult{ByPod: ExposeByPod{HostingPod: pod, VolumeName: volumeName}}, nil
 }
 
 func (e *csiSnapshotExposer) CleanUp(ctx context.Context, ownerObject corev1.ObjectReference, vsName string, sourceNamespace string) {

pkg/exposer/csi_snapshot_test.go

@@ -37,6 +37,8 @@ import (
 	velerov1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
 	"github.com/vmware-tanzu/velero/pkg/util/boolptr"
+
+	clientFake "sigs.k8s.io/controller-runtime/pkg/client/fake"
 )
 
 type reactor struct {
@@ -442,3 +444,180 @@ func TestExpose(t *testing.T) {
 		})
 	}
 }
+
+func TestGetExpose(t *testing.T) {
+	backup := &velerov1.Backup{
+		TypeMeta: metav1.TypeMeta{
+			APIVersion: velerov1.SchemeGroupVersion.String(),
+			Kind:       "Backup",
+		},
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: velerov1.DefaultNamespace,
+			Name:      "fake-backup",
+			UID:       "fake-uid",
+		},
+	}
+
+	backupPod := &corev1.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: backup.Namespace,
+			Name:      backup.Name,
+		},
+		Spec: corev1.PodSpec{
+			Volumes: []corev1.Volume{
+				{
+					Name: "fake-volume",
+				},
+				{
+					Name: "fake-volume-2",
+				},
+				{
+					Name: string(backup.UID),
+				},
+			},
+		},
+	}
+
+	backupPodWithoutVolume := &corev1.Pod{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: backup.Namespace,
+			Name:      backup.Name,
+		},
+		Spec: corev1.PodSpec{
+			Volumes: []corev1.Volume{
+				{
+					Name: "fake-volume-1",
+				},
+				{
+					Name: "fake-volume-2",
+				},
+			},
+		},
+	}
+
+	backupPVC := &corev1.PersistentVolumeClaim{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: backup.Namespace,
+			Name:      backup.Name,
+		},
+		Spec: corev1.PersistentVolumeClaimSpec{
+			VolumeName: "fake-pv-name",
+		},
+	}
+
+	backupPV := &corev1.PersistentVolume{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "fake-pv-name",
+		},
+	}
+
+	scheme := runtime.NewScheme()
+	corev1.AddToScheme(scheme)
+
+	tests := []struct {
+		name            string
+		kubeClientObj   []runtime.Object
+		ownerBackup     *velerov1.Backup
+		exposeWaitParam CSISnapshotExposeWaitParam
+		Timeout         time.Duration
+		err             string
+		expectedResult  *ExposeResult
+	}{
+		{
+			name:        "backup pod is not found",
+			ownerBackup: backup,
+			exposeWaitParam: CSISnapshotExposeWaitParam{
+				NodeName: "fake-node",
+			},
+		},
+		{
+			name:        "wait pvc bound fail",
+			ownerBackup: backup,
+			exposeWaitParam: CSISnapshotExposeWaitParam{
+				NodeName: "fake-node",
+			},
+			kubeClientObj: []runtime.Object{
+				backupPod,
+			},
+			Timeout: time.Second,
+			err:     "error to wait backup PVC bound, fake-backup: error to wait for rediness of PVC: error to get pvc velero/fake-backup: persistentvolumeclaims \"fake-backup\" not found",
+		},
+		{
+			name:        "backup volume not found in pod",
+			ownerBackup: backup,
+			exposeWaitParam: CSISnapshotExposeWaitParam{
+				NodeName: "fake-node",
+			},
+			kubeClientObj: []runtime.Object{
+				backupPodWithoutVolume,
+				backupPVC,
+				backupPV,
+			},
+			Timeout: time.Second,
+			err:     "backup pod fake-backup doesn't have the expected backup volume",
+		},
+		{
+			name:        "succeed",
+			ownerBackup: backup,
+			exposeWaitParam: CSISnapshotExposeWaitParam{
+				NodeName: "fake-node",
+			},
+			kubeClientObj: []runtime.Object{
+				backupPod,
+				backupPVC,
+				backupPV,
+			},
+			Timeout: time.Second,
+			expectedResult: &ExposeResult{
+				ByPod: ExposeByPod{
+					HostingPod: backupPod,
+					VolumeName: string(backup.UID),
+				},
+			},
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			fakeKubeClient := fake.NewSimpleClientset(test.kubeClientObj...)
+
+			fakeClientBuilder := clientFake.NewClientBuilder()
+			fakeClientBuilder = fakeClientBuilder.WithScheme(scheme)
+
+			fakeClient := fakeClientBuilder.WithRuntimeObjects(test.kubeClientObj...).Build()
+
+			exposer := csiSnapshotExposer{
+				kubeClient: fakeKubeClient,
+				log:        velerotest.NewLogger(),
+			}
+
+			var ownerObject corev1.ObjectReference
+			if test.ownerBackup != nil {
+				ownerObject = corev1.ObjectReference{
+					Kind:       test.ownerBackup.Kind,
+					Namespace:  test.ownerBackup.Namespace,
+					Name:       test.ownerBackup.Name,
+					UID:        test.ownerBackup.UID,
+					APIVersion: test.ownerBackup.APIVersion,
+				}
+			}
+
+			test.exposeWaitParam.NodeClient = fakeClient
+
+			result, err := exposer.GetExposed(context.Background(), ownerObject, test.Timeout, &test.exposeWaitParam)
+			if test.err == "" {
+				assert.NoError(t, err)
+
+				if test.expectedResult == nil {
+					assert.Nil(t, result)
+				} else {
+					assert.NoError(t, err)
+					assert.Equal(t, test.expectedResult.ByPod.VolumeName, result.ByPod.VolumeName)
+					assert.Equal(t, test.expectedResult.ByPod.HostingPod.Name, result.ByPod.HostingPod.Name)
+				}
+			} else {
+				assert.EqualError(t, err, test.err)
+			}
+		})
+	}
+}

pkg/nodeagent/node_agent.go

@@ -23,13 +23,13 @@ import (
 
 	"github.com/pkg/errors"
 	v1 "k8s.io/api/core/v1"
-	"k8s.io/client-go/kubernetes"
-
-	"github.com/vmware-tanzu/velero/pkg/util/kube"
-
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
+	"k8s.io/apimachinery/pkg/labels"
+	"k8s.io/client-go/kubernetes"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
+
+	"github.com/vmware-tanzu/velero/pkg/util/kube"
 )
 
 const (
@@ -76,12 +76,18 @@ func IsRunning(ctx context.Context, kubeClient kubernetes.Interface, namespace s
 }
 
 // IsRunningInNode checks if the node agent pod is running properly in a specified node. If not, return the error found
-func IsRunningInNode(ctx context.Context, namespace string, nodeName string, podClient corev1client.PodsGetter) error {
+func IsRunningInNode(ctx context.Context, namespace string, nodeName string, crClient ctrlclient.Client) error {
 	if nodeName == "" {
 		return errors.New("node name is empty")
 	}
 
-	pods, err := podClient.Pods(namespace).List(ctx, metav1.ListOptions{LabelSelector: fmt.Sprintf("name=%s", daemonSet)})
+	pods := new(v1.PodList)
+	parsedSelector, err := labels.Parse(fmt.Sprintf("name=%s", daemonSet))
+	if err != nil {
+		return errors.Wrap(err, "fail to parse selector")
+	}
+
+	err = crClient.List(ctx, pods, &ctrlclient.ListOptions{LabelSelector: parsedSelector})
 	if err != nil {
 		return errors.Wrap(err, "failed to list daemonset pods")
 	}
@@ -108,8 +114,8 @@ func GetPodSpec(ctx context.Context, kubeClient kubernetes.Interface, namespace
 	return &ds.Spec.Template.Spec, nil
 }
 
-func GetConfigs(ctx context.Context, namespace string, cmClient corev1client.ConfigMapsGetter) (*Configs, error) {
-	cm, err := cmClient.ConfigMaps(namespace).Get(ctx, configName, metav1.GetOptions{})
+func GetConfigs(ctx context.Context, namespace string, kubeClient kubernetes.Interface) (*Configs, error) {
+	cm, err := kubeClient.CoreV1().ConfigMaps(namespace).Get(ctx, configName, metav1.GetOptions{})
 	if err != nil {
 		if apierrors.IsNotFound(err) {
 			return nil, nil

pkg/persistence/object_store.go

@@ -27,13 +27,14 @@ import (
 
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
+	"k8s.io/apimachinery/pkg/runtime/serializer"
 	kerrors "k8s.io/apimachinery/pkg/util/errors"
 
 	"github.com/vmware-tanzu/velero/internal/credentials"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/scheme"
 	"github.com/vmware-tanzu/velero/pkg/itemoperation"
 	"github.com/vmware-tanzu/velero/pkg/plugin/velero"
+	"github.com/vmware-tanzu/velero/pkg/util"
 	"github.com/vmware-tanzu/velero/pkg/volume"
 )
 
@@ -302,7 +303,9 @@ func (s *objectBackupStore) GetBackupMetadata(name string) (*velerov1api.Backup,
 		return nil, errors.WithStack(err)
 	}
 
-	decoder := scheme.Codecs.UniversalDecoder(velerov1api.SchemeGroupVersion)
+	codecFactory := serializer.NewCodecFactory(util.VeleroScheme)
+
+	decoder := codecFactory.UniversalDecoder(velerov1api.SchemeGroupVersion)
 	obj, _, err := decoder.Decode(data, nil, nil)
 	if err != nil {
 		return nil, errors.WithStack(err)

pkg/podvolume/backupper.go

@@ -26,13 +26,13 @@ import (
 	corev1api "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/util/sets"
-	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
 	"k8s.io/client-go/tools/cache"
+	ctrlcache "sigs.k8s.io/controller-runtime/pkg/cache"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	"github.com/vmware-tanzu/velero/internal/resourcepolicies"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	veleroclient "github.com/vmware-tanzu/velero/pkg/client"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
 	"github.com/vmware-tanzu/velero/pkg/label"
 	"github.com/vmware-tanzu/velero/pkg/nodeagent"
 	"github.com/vmware-tanzu/velero/pkg/repository"
@@ -50,10 +50,7 @@ type backupper struct {
 	ctx          context.Context
 	repoLocker   *repository.RepoLocker
 	repoEnsurer  *repository.Ensurer
-	veleroClient clientset.Interface
-	pvcClient    corev1client.PersistentVolumeClaimsGetter
-	pvClient     corev1client.PersistentVolumesGetter
-	podClient    corev1client.PodsGetter
+	crClient     ctrlclient.Client
 	uploaderType string
 
 	results     map[string]chan *velerov1api.PodVolumeBackup
@@ -103,32 +100,31 @@ func newBackupper(
 	ctx context.Context,
 	repoLocker *repository.RepoLocker,
 	repoEnsurer *repository.Ensurer,
-	podVolumeBackupInformer cache.SharedIndexInformer,
-	veleroClient clientset.Interface,
-	pvcClient corev1client.PersistentVolumeClaimsGetter,
-	pvClient corev1client.PersistentVolumesGetter,
-	podClient corev1client.PodsGetter,
+	pvbInformer ctrlcache.Informer,
+	crClient ctrlclient.Client,
 	uploaderType string,
+	backup *velerov1api.Backup,
 	log logrus.FieldLogger,
 ) *backupper {
 	b := &backupper{
 		ctx:          ctx,
 		repoLocker:   repoLocker,
 		repoEnsurer:  repoEnsurer,
-		veleroClient: veleroClient,
-		pvcClient:    pvcClient,
-		pvClient:     pvClient,
-		podClient:    podClient,
+		crClient:     crClient,
 		uploaderType: uploaderType,
 
 		results: make(map[string]chan *velerov1api.PodVolumeBackup),
 	}
 
-	podVolumeBackupInformer.AddEventHandler(
+	pvbInformer.AddEventHandler(
 		cache.ResourceEventHandlerFuncs{
 			UpdateFunc: func(_, obj interface{}) {
 				pvb := obj.(*velerov1api.PodVolumeBackup)
 
+				if pvb.GetLabels()[velerov1api.BackupUIDLabel] != string(backup.UID) {
+					return
+				}
+
 				if pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseCompleted || pvb.Status.Phase == velerov1api.PodVolumeBackupPhaseFailed {
 					b.resultsLock.Lock()
 					defer b.resultsLock.Unlock()
@@ -153,7 +149,8 @@ func resultsKey(ns, name string) string {
 
 func (b *backupper) getMatchAction(resPolicies *resourcepolicies.Policies, pvc *corev1api.PersistentVolumeClaim, volume *corev1api.Volume) (*resourcepolicies.Action, error) {
 	if pvc != nil {
-		pv, err := b.pvClient.PersistentVolumes().Get(context.TODO(), pvc.Spec.VolumeName, metav1.GetOptions{})
+		pv := new(corev1api.PersistentVolume)
+		err := b.crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: pvc.Spec.VolumeName}, pv)
 		if err != nil {
 			return nil, errors.Wrapf(err, "error getting pv for pvc %s", pvc.Spec.VolumeName)
 		}
@@ -173,7 +170,7 @@ func (b *backupper) BackupPodVolumes(backup *velerov1api.Backup, pod *corev1api.
 	}
 	log.Infof("pod %s/%s has volumes to backup: %v", pod.Namespace, pod.Name, volumesToBackup)
 
-	err := nodeagent.IsRunningInNode(b.ctx, backup.Namespace, pod.Spec.NodeName, b.podClient)
+	err := nodeagent.IsRunningInNode(b.ctx, backup.Namespace, pod.Spec.NodeName, b.crClient)
 	if err != nil {
 		return nil, nil, []error{err}
 	}
@@ -213,7 +210,8 @@ func (b *backupper) BackupPodVolumes(backup *velerov1api.Backup, pod *corev1api.
 	for _, podVolume := range pod.Spec.Volumes {
 		podVolumes[podVolume.Name] = podVolume
 		if podVolume.PersistentVolumeClaim != nil {
-			pvc, err := b.pvcClient.PersistentVolumeClaims(pod.Namespace).Get(context.TODO(), podVolume.PersistentVolumeClaim.ClaimName, metav1.GetOptions{})
+			pvc := new(corev1api.PersistentVolumeClaim)
+			err := b.crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: pod.Namespace, Name: podVolume.PersistentVolumeClaim.ClaimName}, pvc)
 			if err != nil {
 				errs = append(errs, errors.Wrap(err, "error getting persistent volume claim for volume"))
 				continue
@@ -263,7 +261,7 @@ func (b *backupper) BackupPodVolumes(backup *velerov1api.Backup, pod *corev1api.
 
 		// hostPath volumes are not supported because they're not mounted into /var/lib/kubelet/pods, so our
 		// daemonset pod has no way to access their data.
-		isHostPath, err := isHostPathVolume(&volume, pvc, b.pvClient.PersistentVolumes())
+		isHostPath, err := isHostPathVolume(&volume, pvc, b.crClient)
 		if err != nil {
 			errs = append(errs, errors.Wrap(err, "error checking if volume is a hostPath volume"))
 			continue
@@ -303,11 +301,7 @@ func (b *backupper) BackupPodVolumes(backup *velerov1api.Backup, pod *corev1api.
 		}
 
 		volumeBackup := newPodVolumeBackup(backup, pod, volume, repoIdentifier, b.uploaderType, pvc)
-		// TODO: once backupper is refactored to use controller-runtime, just pass client instead of anonymous func
-		if err := veleroclient.CreateRetryGenerateNameWithFunc(volumeBackup, func() error {
-			_, err := b.veleroClient.VeleroV1().PodVolumeBackups(volumeBackup.Namespace).Create(context.TODO(), volumeBackup, metav1.CreateOptions{})
-			return err
-		}); err != nil {
+		if err := veleroclient.CreateRetryGenerateName(b.crClient, b.ctx, volumeBackup); err != nil {
 			errs = append(errs, err)
 			continue
 		}
@@ -339,13 +333,9 @@ ForEachVolume:
 	return podVolumeBackups, pvcSummary, errs
 }
 
-type pvGetter interface {
-	Get(ctx context.Context, name string, opts metav1.GetOptions) (*corev1api.PersistentVolume, error)
-}
-
 // isHostPathVolume returns true if the volume is either a hostPath pod volume or a persistent
 // volume claim on a hostPath persistent volume, or false otherwise.
-func isHostPathVolume(volume *corev1api.Volume, pvc *corev1api.PersistentVolumeClaim, pvGetter pvGetter) (bool, error) {
+func isHostPathVolume(volume *corev1api.Volume, pvc *corev1api.PersistentVolumeClaim, crClient ctrlclient.Client) (bool, error) {
 	if volume.HostPath != nil {
 		return true, nil
 	}
@@ -354,7 +344,8 @@ func isHostPathVolume(volume *corev1api.Volume, pvc *corev1api.PersistentVolumeC
 		return false, nil
 	}
 
-	pv, err := pvGetter.Get(context.TODO(), pvc.Spec.VolumeName, metav1.GetOptions{})
+	pv := new(corev1api.PersistentVolume)
+	err := crClient.Get(context.TODO(), ctrlclient.ObjectKey{Name: pvc.Spec.VolumeName}, pv)
 	if err != nil {
 		return false, errors.WithStack(err)
 	}
@@ -421,7 +412,3 @@ func newPodVolumeBackup(backup *velerov1api.Backup, pod *corev1api.Pod, volume c
 
 	return pvb
 }
-
-func errorOnly(_ interface{}, err error) error {
-	return err
-}

pkg/podvolume/backupper_factory.go

@@ -18,17 +18,14 @@ package podvolume
 
 import (
 	"context"
-	"fmt"
 
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
-	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
 	"k8s.io/client-go/tools/cache"
+	ctrlcache "sigs.k8s.io/controller-runtime/pkg/cache"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
-	velerov1informers "github.com/vmware-tanzu/velero/pkg/generated/informers/externalversions/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/repository"
 )
 
@@ -41,48 +38,31 @@ type BackupperFactory interface {
 func NewBackupperFactory(
 	repoLocker *repository.RepoLocker,
 	repoEnsurer *repository.Ensurer,
-	veleroClient clientset.Interface,
-	pvcClient corev1client.PersistentVolumeClaimsGetter,
-	pvClient corev1client.PersistentVolumesGetter,
-	podClient corev1client.PodsGetter,
+	crClient ctrlclient.Client,
+	pvbInformer ctrlcache.Informer,
 	log logrus.FieldLogger,
 ) BackupperFactory {
 	return &backupperFactory{
-		repoLocker:   repoLocker,
-		repoEnsurer:  repoEnsurer,
-		veleroClient: veleroClient,
-		pvcClient:    pvcClient,
-		pvClient:     pvClient,
-		podClient:    podClient,
-		log:          log,
+		repoLocker:  repoLocker,
+		repoEnsurer: repoEnsurer,
+		crClient:    crClient,
+		pvbInformer: pvbInformer,
+		log:         log,
 	}
 }
 
 type backupperFactory struct {
-	repoLocker   *repository.RepoLocker
-	repoEnsurer  *repository.Ensurer
-	veleroClient clientset.Interface
-	pvcClient    corev1client.PersistentVolumeClaimsGetter
-	pvClient     corev1client.PersistentVolumesGetter
-	podClient    corev1client.PodsGetter
-	log          logrus.FieldLogger
+	repoLocker  *repository.RepoLocker
+	repoEnsurer *repository.Ensurer
+	crClient    ctrlclient.Client
+	pvbInformer ctrlcache.Informer
+	log         logrus.FieldLogger
 }
 
 func (bf *backupperFactory) NewBackupper(ctx context.Context, backup *velerov1api.Backup, uploaderType string) (Backupper, error) {
-	informer := velerov1informers.NewFilteredPodVolumeBackupInformer(
-		bf.veleroClient,
-		backup.Namespace,
-		0,
-		cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc},
-		func(opts *metav1.ListOptions) {
-			opts.LabelSelector = fmt.Sprintf("%s=%s", velerov1api.BackupUIDLabel, backup.UID)
-		},
-	)
+	b := newBackupper(ctx, bf.repoLocker, bf.repoEnsurer, bf.pvbInformer, bf.crClient, uploaderType, backup, bf.log)
 
-	b := newBackupper(ctx, bf.repoLocker, bf.repoEnsurer, informer, bf.veleroClient, bf.pvcClient, bf.pvClient, bf.podClient, uploaderType, bf.log)
-
-	go informer.Run(ctx.Done())
-	if !cache.WaitForCacheSync(ctx.Done(), informer.HasSynced) {
+	if !cache.WaitForCacheSync(ctx.Done(), bf.pvbInformer.HasSynced) {
 		return nil, errors.New("timed out waiting for caches to sync")
 	}
 

pkg/podvolume/backupper_test.go

@@ -23,7 +23,6 @@ import (
 	"testing"
 	"time"
 
-	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
@@ -32,17 +31,15 @@ import (
 	"k8s.io/apimachinery/pkg/runtime"
 	ctrlfake "sigs.k8s.io/controller-runtime/pkg/client/fake"
 
-	"k8s.io/client-go/kubernetes"
-	kubefake "k8s.io/client-go/kubernetes/fake"
 	clientTesting "k8s.io/client-go/testing"
+	"k8s.io/client-go/tools/cache"
 
 	"github.com/vmware-tanzu/velero/internal/resourcepolicies"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
-	velerofake "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/fake"
 	"github.com/vmware-tanzu/velero/pkg/repository"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
+	"github.com/vmware-tanzu/velero/pkg/util/kube"
 )
 
 func TestIsHostPathVolume(t *testing.T) {
@@ -101,15 +98,14 @@ func TestIsHostPathVolume(t *testing.T) {
 			VolumeName: "pv-1",
 		},
 	}
-	pvGetter := &fakePVGetter{
-		pv: &corev1api.PersistentVolume{
-			ObjectMeta: metav1.ObjectMeta{
-				Name: "pv-1",
-			},
-			Spec: corev1api.PersistentVolumeSpec{},
+	pv := &corev1api.PersistentVolume{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "pv-1",
 		},
+		Spec: corev1api.PersistentVolumeSpec{},
 	}
-	isHostPath, err = isHostPathVolume(vol, pvc, pvGetter)
+	crClient1 := velerotest.NewFakeControllerRuntimeClient(t, pv)
+	isHostPath, err = isHostPathVolume(vol, pvc, crClient1)
 	assert.Nil(t, err)
 	assert.False(t, isHostPath)
 
@@ -130,35 +126,23 @@ func TestIsHostPathVolume(t *testing.T) {
 			VolumeName: "pv-1",
 		},
 	}
-	pvGetter = &fakePVGetter{
-		pv: &corev1api.PersistentVolume{
-			ObjectMeta: metav1.ObjectMeta{
-				Name: "pv-1",
-			},
-			Spec: corev1api.PersistentVolumeSpec{
-				PersistentVolumeSource: corev1api.PersistentVolumeSource{
-					HostPath: &corev1api.HostPathVolumeSource{},
-				},
+	pv = &corev1api.PersistentVolume{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "pv-1",
+		},
+		Spec: corev1api.PersistentVolumeSpec{
+			PersistentVolumeSource: corev1api.PersistentVolumeSource{
+				HostPath: &corev1api.HostPathVolumeSource{},
 			},
 		},
 	}
-	isHostPath, err = isHostPathVolume(vol, pvc, pvGetter)
+	crClient2 := velerotest.NewFakeControllerRuntimeClient(t, pv)
+
+	isHostPath, err = isHostPathVolume(vol, pvc, crClient2)
 	assert.Nil(t, err)
 	assert.True(t, isHostPath)
 }
 
-type fakePVGetter struct {
-	pv *corev1api.PersistentVolume
-}
-
-func (g *fakePVGetter) Get(ctx context.Context, name string, opts metav1.GetOptions) (*corev1api.PersistentVolume, error) {
-	if g.pv != nil {
-		return g.pv, nil
-	}
-
-	return nil, errors.New("item not found")
-}
-
 func Test_backupper_BackupPodVolumes_log_test(t *testing.T) {
 	type args struct {
 		backup          *velerov1api.Backup
@@ -322,6 +306,7 @@ func createPVBObj(fail bool, withSnapshot bool, index int, uploaderType string)
 func TestBackupPodVolumes(t *testing.T) {
 	scheme := runtime.NewScheme()
 	velerov1api.AddToScheme(scheme)
+	corev1api.AddToScheme(scheme)
 
 	ctxWithCancel, cancel := context.WithCancel(context.Background())
 	defer cancel()
@@ -509,40 +494,6 @@ func TestBackupPodVolumes(t *testing.T) {
 			uploaderType:  "kopia",
 			bsl:           "fake-bsl",
 		},
-		{
-			name: "create PVB fail",
-			volumes: []string{
-				"fake-volume-1",
-				"fake-volume-2",
-			},
-			sourcePod: createPodObj(true, true, true, 2),
-			kubeClientObj: []runtime.Object{
-				createNodeAgentPodObj(true),
-				createPVCObj(1),
-				createPVCObj(2),
-				createPVObj(1, false),
-				createPVObj(2, false),
-			},
-			ctlClientObj: []runtime.Object{
-				createBackupRepoObj(),
-			},
-			veleroReactors: []reactor{
-				{
-					verb:     "create",
-					resource: "podvolumebackups",
-					reactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {
-						return true, nil, errors.New("fake-create-error")
-					},
-				},
-			},
-			runtimeScheme: scheme,
-			uploaderType:  "kopia",
-			bsl:           "fake-bsl",
-			errs: []string{
-				"fake-create-error",
-				"fake-create-error",
-			},
-		},
 		{
 			name: "context cancelled",
 			ctx:  ctxWithCancel,
@@ -630,23 +581,28 @@ func TestBackupPodVolumes(t *testing.T) {
 				fakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)
 			}
 
-			fakeCtlClient := fakeClientBuilder.WithRuntimeObjects(test.ctlClientObj...).Build()
+			objList := append(test.ctlClientObj, test.veleroClientObj...)
+			objList = append(objList, test.kubeClientObj...)
+			fakeCtrlClient := fakeClientBuilder.WithRuntimeObjects(objList...).Build()
 
-			fakeKubeClient := kubefake.NewSimpleClientset(test.kubeClientObj...)
-			var kubeClient kubernetes.Interface = fakeKubeClient
-
-			fakeVeleroClient := velerofake.NewSimpleClientset(test.veleroClientObj...)
-			for _, reactor := range test.veleroReactors {
-				fakeVeleroClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)
+			fakeCRWatchClient := velerotest.NewFakeControllerRuntimeWatchClient(t, test.kubeClientObj...)
+			lw := kube.InternalLW{
+				Client:     fakeCRWatchClient,
+				Namespace:  velerov1api.DefaultNamespace,
+				ObjectList: new(velerov1api.PodVolumeBackupList),
 			}
-			var veleroClient versioned.Interface = fakeVeleroClient
 
-			ensurer := repository.NewEnsurer(fakeCtlClient, velerotest.NewLogger(), time.Millisecond)
+			pvbInformer := cache.NewSharedIndexInformer(&lw, &velerov1api.PodVolumeBackup{}, 0, cache.Indexers{})
+
+			go pvbInformer.Run(ctx.Done())
+			require.True(t, cache.WaitForCacheSync(ctx.Done(), pvbInformer.HasSynced))
+
+			ensurer := repository.NewEnsurer(fakeCtrlClient, velerotest.NewLogger(), time.Millisecond)
 
 			backupObj := builder.ForBackup(velerov1api.DefaultNamespace, "fake-backup").Result()
 			backupObj.Spec.StorageLocation = test.bsl
 
-			factory := NewBackupperFactory(repository.NewRepoLocker(), ensurer, veleroClient, kubeClient.CoreV1(), kubeClient.CoreV1(), kubeClient.CoreV1(), velerotest.NewLogger())
+			factory := NewBackupperFactory(repository.NewRepoLocker(), ensurer, fakeCtrlClient, pvbInformer, velerotest.NewLogger())
 			bp, err := factory.NewBackupper(ctx, backupObj, test.uploaderType)
 
 			require.NoError(t, err)

pkg/podvolume/restorer.go

@@ -27,12 +27,12 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/util/wait"
 	"k8s.io/client-go/kubernetes"
-	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
 	"k8s.io/client-go/tools/cache"
+	ctrlcache "sigs.k8s.io/controller-runtime/pkg/cache"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	veleroclient "github.com/vmware-tanzu/velero/pkg/client"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
 	"github.com/vmware-tanzu/velero/pkg/label"
 	"github.com/vmware-tanzu/velero/pkg/nodeagent"
 	"github.com/vmware-tanzu/velero/pkg/repository"
@@ -54,13 +54,11 @@ type Restorer interface {
 }
 
 type restorer struct {
-	ctx          context.Context
-	repoLocker   *repository.RepoLocker
-	repoEnsurer  *repository.Ensurer
-	veleroClient clientset.Interface
-	pvcClient    corev1client.PersistentVolumeClaimsGetter
-	podClient    corev1client.PodsGetter
-	kubeClient   kubernetes.Interface
+	ctx         context.Context
+	repoLocker  *repository.RepoLocker
+	repoEnsurer *repository.Ensurer
+	kubeClient  kubernetes.Interface
+	crClient    ctrlclient.Client
 
 	resultsLock    sync.Mutex
 	results        map[string]chan *velerov1api.PodVolumeRestore
@@ -72,30 +70,30 @@ func newRestorer(
 	ctx context.Context,
 	repoLocker *repository.RepoLocker,
 	repoEnsurer *repository.Ensurer,
-	podVolumeRestoreInformer cache.SharedIndexInformer,
-	veleroClient clientset.Interface,
-	pvcClient corev1client.PersistentVolumeClaimsGetter,
-	podClient corev1client.PodsGetter,
+	pvrInformer ctrlcache.Informer,
 	kubeClient kubernetes.Interface,
+	crClient ctrlclient.Client,
+	restore *velerov1api.Restore,
 	log logrus.FieldLogger,
 ) *restorer {
 	r := &restorer{
-		ctx:          ctx,
-		repoLocker:   repoLocker,
-		repoEnsurer:  repoEnsurer,
-		veleroClient: veleroClient,
-		pvcClient:    pvcClient,
-		podClient:    podClient,
-		kubeClient:   kubeClient,
+		ctx:         ctx,
+		repoLocker:  repoLocker,
+		repoEnsurer: repoEnsurer,
+		kubeClient:  kubeClient,
+		crClient:    crClient,
 
 		results: make(map[string]chan *velerov1api.PodVolumeRestore),
 		log:     log,
 	}
 
-	podVolumeRestoreInformer.AddEventHandler(
+	pvrInformer.AddEventHandler(
 		cache.ResourceEventHandlerFuncs{
 			UpdateFunc: func(_, obj interface{}) {
 				pvr := obj.(*velerov1api.PodVolumeRestore)
+				if pvr.GetLabels()[velerov1api.RestoreUIDLabel] != string(restore.UID) {
+					return
+				}
 
 				if pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseCompleted || pvr.Status.Phase == velerov1api.PodVolumeRestorePhaseFailed {
 					r.resultsLock.Lock()
@@ -169,7 +167,8 @@ func (r *restorer) RestorePodVolumes(data RestoreData) []error {
 		var pvc *corev1api.PersistentVolumeClaim
 		if ok {
 			if volumeObj.PersistentVolumeClaim != nil {
-				pvc, err = r.pvcClient.PersistentVolumeClaims(data.Pod.Namespace).Get(context.TODO(), volumeObj.PersistentVolumeClaim.ClaimName, metav1.GetOptions{})
+				pvc := new(corev1api.PersistentVolumeClaim)
+				err := r.crClient.Get(context.TODO(), ctrlclient.ObjectKey{Namespace: data.Pod.Namespace, Name: volumeObj.PersistentVolumeClaim.ClaimName}, pvc)
 				if err != nil {
 					errs = append(errs, errors.Wrap(err, "error getting persistent volume claim for volume"))
 					continue
@@ -179,10 +178,7 @@ func (r *restorer) RestorePodVolumes(data RestoreData) []error {
 
 		volumeRestore := newPodVolumeRestore(data.Restore, data.Pod, data.BackupLocation, volume, backupInfo.snapshotID, repoIdentifier, backupInfo.uploaderType, data.SourceNamespace, pvc)
 
-		// TODO: once restorer is refactored to use controller-runtime, just pass client instead of anonymous func
-		if err := veleroclient.CreateRetryGenerateNameWithFunc(volumeRestore, func() error {
-			return errorOnly(r.veleroClient.VeleroV1().PodVolumeRestores(volumeRestore.Namespace).Create(context.TODO(), volumeRestore, metav1.CreateOptions{}))
-		}); err != nil {
+		if err := veleroclient.CreateRetryGenerateName(r.crClient, r.ctx, volumeRestore); err != nil {
 			errs = append(errs, errors.WithStack(err))
 			continue
 		}
@@ -214,7 +210,7 @@ func (r *restorer) RestorePodVolumes(data RestoreData) []error {
 		} else if err != nil {
 			r.log.WithError(err).Error("Failed to check node-agent pod status, disengage")
 		} else {
-			err = nodeagent.IsRunningInNode(checkCtx, data.Restore.Namespace, nodeName, r.podClient)
+			err = nodeagent.IsRunningInNode(checkCtx, data.Restore.Namespace, nodeName, r.crClient)
 			if err != nil {
 				r.log.WithField("node", nodeName).WithError(err).Error("node-agent pod is not running in node, abort the restore")
 				r.nodeAgentCheck <- errors.Wrapf(err, "node-agent pod is not running in node %s", nodeName)

pkg/podvolume/restorer_factory.go

@@ -18,18 +18,15 @@ package podvolume
 
 import (
 	"context"
-	"fmt"
 
 	"github.com/pkg/errors"
 	"github.com/sirupsen/logrus"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/kubernetes"
-	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
 	"k8s.io/client-go/tools/cache"
+	ctrlcache "sigs.k8s.io/controller-runtime/pkg/cache"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
-	velerov1informers "github.com/vmware-tanzu/velero/pkg/generated/informers/externalversions/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/repository"
 )
 
@@ -41,47 +38,33 @@ type RestorerFactory interface {
 
 func NewRestorerFactory(repoLocker *repository.RepoLocker,
 	repoEnsurer *repository.Ensurer,
-	veleroClient clientset.Interface,
-	pvcClient corev1client.PersistentVolumeClaimsGetter,
-	podClient corev1client.PodsGetter,
 	kubeClient kubernetes.Interface,
+	crClient ctrlclient.Client,
+	pvrInformer ctrlcache.Informer,
 	log logrus.FieldLogger) RestorerFactory {
 	return &restorerFactory{
-		repoLocker:   repoLocker,
-		repoEnsurer:  repoEnsurer,
-		veleroClient: veleroClient,
-		pvcClient:    pvcClient,
-		podClient:    podClient,
-		kubeClient:   kubeClient,
-		log:          log,
+		repoLocker:  repoLocker,
+		repoEnsurer: repoEnsurer,
+		kubeClient:  kubeClient,
+		crClient:    crClient,
+		pvrInformer: pvrInformer,
+		log:         log,
 	}
 }
 
 type restorerFactory struct {
-	repoLocker   *repository.RepoLocker
-	repoEnsurer  *repository.Ensurer
-	veleroClient clientset.Interface
-	pvcClient    corev1client.PersistentVolumeClaimsGetter
-	podClient    corev1client.PodsGetter
-	kubeClient   kubernetes.Interface
-	log          logrus.FieldLogger
+	repoLocker  *repository.RepoLocker
+	repoEnsurer *repository.Ensurer
+	kubeClient  kubernetes.Interface
+	crClient    ctrlclient.Client
+	pvrInformer ctrlcache.Informer
+	log         logrus.FieldLogger
 }
 
 func (rf *restorerFactory) NewRestorer(ctx context.Context, restore *velerov1api.Restore) (Restorer, error) {
-	informer := velerov1informers.NewFilteredPodVolumeRestoreInformer(
-		rf.veleroClient,
-		restore.Namespace,
-		0,
-		cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc},
-		func(opts *metav1.ListOptions) {
-			opts.LabelSelector = fmt.Sprintf("%s=%s", velerov1api.RestoreUIDLabel, restore.UID)
-		},
-	)
+	r := newRestorer(ctx, rf.repoLocker, rf.repoEnsurer, rf.pvrInformer, rf.kubeClient, rf.crClient, restore, rf.log)
 
-	r := newRestorer(ctx, rf.repoLocker, rf.repoEnsurer, informer, rf.veleroClient, rf.pvcClient, rf.podClient, rf.kubeClient, rf.log)
-
-	go informer.Run(ctx.Done())
-	if !cache.WaitForCacheSync(ctx.Done(), informer.HasSynced) {
+	if !cache.WaitForCacheSync(ctx.Done(), rf.pvrInformer.HasSynced) {
 		return nil, errors.New("timed out waiting for cache to sync")
 	}
 

pkg/podvolume/restorer_test.go

@@ -22,7 +22,6 @@ import (
 	"testing"
 	"time"
 
-	"github.com/pkg/errors"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 	appv1 "k8s.io/api/apps/v1"
@@ -31,15 +30,14 @@ import (
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/client-go/kubernetes"
 	kubefake "k8s.io/client-go/kubernetes/fake"
-	clientTesting "k8s.io/client-go/testing"
+	"k8s.io/client-go/tools/cache"
 	ctrlfake "sigs.k8s.io/controller-runtime/pkg/client/fake"
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
-	velerofake "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/fake"
 	"github.com/vmware-tanzu/velero/pkg/repository"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
+	"github.com/vmware-tanzu/velero/pkg/util/kube"
 )
 
 func TestGetVolumesRepositoryType(t *testing.T) {
@@ -162,6 +160,7 @@ type expectError struct {
 func TestRestorePodVolumes(t *testing.T) {
 	scheme := runtime.NewScheme()
 	velerov1api.AddToScheme(scheme)
+	corev1api.AddToScheme(scheme)
 
 	ctxWithCancel, cancel := context.WithCancel(context.Background())
 	defer cancel()
@@ -264,42 +263,6 @@ func TestRestorePodVolumes(t *testing.T) {
 				},
 			},
 		},
-		{
-			name: "create pvb fail",
-			pvbs: []*velerov1api.PodVolumeBackup{
-				createPVBObj(true, true, 1, "kopia"),
-				createPVBObj(true, true, 2, "kopia"),
-			},
-			kubeClientObj: []runtime.Object{
-				createNodeAgentDaemonset(),
-				createPVCObj(1),
-				createPVCObj(2),
-			},
-			ctlClientObj: []runtime.Object{
-				createBackupRepoObj(),
-			},
-			veleroReactors: []reactor{
-				{
-					verb:     "create",
-					resource: "podvolumerestores",
-					reactorFunc: func(action clientTesting.Action) (handled bool, ret runtime.Object, err error) {
-						return true, nil, errors.New("fake-create-error")
-					},
-				},
-			},
-			restoredPod:     createPodObj(true, true, true, 2),
-			sourceNamespace: "fake-ns",
-			bsl:             "fake-bsl",
-			runtimeScheme:   scheme,
-			errs: []expectError{
-				{
-					err: "fake-create-error",
-				},
-				{
-					err: "fake-create-error",
-				},
-			},
-		},
 		{
 			name: "create pvb fail",
 			ctx:  ctxWithCancel,
@@ -407,22 +370,32 @@ func TestRestorePodVolumes(t *testing.T) {
 				fakeClientBuilder = fakeClientBuilder.WithScheme(test.runtimeScheme)
 			}
 
-			fakeCtlClient := fakeClientBuilder.WithRuntimeObjects(test.ctlClientObj...).Build()
+			objClient := append(test.ctlClientObj, test.kubeClientObj...)
+			objClient = append(objClient, test.veleroClientObj...)
+
+			fakeCRClient := velerotest.NewFakeControllerRuntimeClient(t, objClient...)
 
 			fakeKubeClient := kubefake.NewSimpleClientset(test.kubeClientObj...)
 			var kubeClient kubernetes.Interface = fakeKubeClient
 
-			fakeVeleroClient := velerofake.NewSimpleClientset(test.veleroClientObj...)
-			for _, reactor := range test.veleroReactors {
-				fakeVeleroClient.Fake.PrependReactor(reactor.verb, reactor.resource, reactor.reactorFunc)
+			fakeCRWatchClient := velerotest.NewFakeControllerRuntimeWatchClient(t, test.kubeClientObj...)
+			lw := kube.InternalLW{
+				Client:     fakeCRWatchClient,
+				Namespace:  velerov1api.DefaultNamespace,
+				ObjectList: new(velerov1api.PodVolumeRestoreList),
 			}
-			var veleroClient versioned.Interface = fakeVeleroClient
 
-			ensurer := repository.NewEnsurer(fakeCtlClient, velerotest.NewLogger(), time.Millisecond)
+			pvrInformer := cache.NewSharedIndexInformer(&lw, &velerov1api.PodVolumeBackup{}, 0, cache.Indexers{})
+
+			go pvrInformer.Run(ctx.Done())
+			require.True(t, cache.WaitForCacheSync(ctx.Done(), pvrInformer.HasSynced))
+
+			ensurer := repository.NewEnsurer(fakeCRClient, velerotest.NewLogger(), time.Millisecond)
 
 			restoreObj := builder.ForRestore(velerov1api.DefaultNamespace, "fake-restore").Result()
 
-			factory := NewRestorerFactory(repository.NewRepoLocker(), ensurer, veleroClient, kubeClient.CoreV1(), kubeClient.CoreV1(), kubeClient, velerotest.NewLogger())
+			factory := NewRestorerFactory(repository.NewRepoLocker(), ensurer, kubeClient,
+				fakeCRClient, pvrInformer, velerotest.NewLogger())
 			rs, err := factory.NewRestorer(ctx, restoreObj)
 
 			require.NoError(t, err)

pkg/repository/config/aws.go

@@ -62,7 +62,7 @@ func GetS3ResticEnvVars(config map[string]string) (map[string]string, error) {
 		result[awsSecretKeyEnvVar] = creds.SecretAccessKey
 		result[awsSessTokenEnvVar] = creds.SessionToken
 		result[awsCredentialsFileEnvVar] = ""
-		result[awsProfileEnvVar] = ""
+		result[awsProfileEnvVar] = "" // profile is not needed since we have the credentials from profile via GetS3Credentials
 		result[awsConfigFileEnvVar] = ""
 	}
 
@@ -87,6 +87,7 @@ func GetS3Credentials(config map[string]string) (*aws.Credentials, error) {
 			// as credentials of a BSL
 			awsconfig.WithSharedConfigFiles([]string{credentialsFile}))
 	}
+	opts = append(opts, awsconfig.WithSharedConfigProfile(config[awsProfileKey]))
 
 	cfg, err := awsconfig.LoadDefaultConfig(context.Background(), opts...)
 	if err != nil {

pkg/repository/config/aws_test.go

@@ -17,8 +17,11 @@ limitations under the License.
 package config
 
 import (
+	"os"
+	"reflect"
 	"testing"
 
+	"github.com/aws/aws-sdk-go-v2/aws"
 	"github.com/stretchr/testify/require"
 )
 
@@ -63,3 +66,81 @@ func TestGetS3ResticEnvVars(t *testing.T) {
 		})
 	}
 }
+
+func TestGetS3CredentialsCorrectlyUseProfile(t *testing.T) {
+	type args struct {
+		config             map[string]string
+		secretFileContents string
+	}
+	tests := []struct {
+		name    string
+		args    args
+		want    *aws.Credentials
+		wantErr bool
+	}{
+		{
+			name: "Test GetS3Credentials use profile correctly",
+			args: args{
+				config: map[string]string{
+					"profile": "some-profile",
+				},
+				secretFileContents: `[default]
+	aws_access_key_id = default-access-key-id
+	aws_secret_access_key = default-secret-access-key
+	[profile some-profile]
+	aws_access_key_id = some-profile-access-key-id
+	aws_secret_access_key = some-profile-secret-access-key
+	`,
+			},
+			want: &aws.Credentials{
+				AccessKeyID:     "some-profile-access-key-id",
+				SecretAccessKey: "some-profile-secret-access-key",
+			},
+		},
+		{
+			name: "Test GetS3Credentials default to default profile",
+			args: args{
+				config: map[string]string{},
+				secretFileContents: `[default]
+	aws_access_key_id = default-access-key-id
+	aws_secret_access_key = default-secret-access-key
+	[profile some-profile]
+	aws_access_key_id = some-profile-access-key-id
+	aws_secret_access_key = some-profile-secret-access-key
+	`,
+			},
+			want: &aws.Credentials{
+				AccessKeyID:     "default-access-key-id",
+				SecretAccessKey: "default-secret-access-key",
+			},
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			tmpFile, err := os.CreateTemp("", "velero-test-aws-credentials")
+			defer os.Remove(tmpFile.Name())
+			if err != nil {
+				t.Errorf("GetS3Credentials() error = %v", err)
+				return
+			}
+			// write the contents of the secret file to the temp file
+			_, err = tmpFile.WriteString(tt.args.secretFileContents)
+			if err != nil {
+				t.Errorf("GetS3Credentials() error = %v", err)
+				return
+			}
+			tt.args.config["credentialsFile"] = tmpFile.Name()
+			got, err := GetS3Credentials(tt.args.config)
+			if (err != nil) != tt.wantErr {
+				t.Errorf("GetS3Credentials() error = %v, wantErr %v", err, tt.wantErr)
+				return
+			}
+			if !reflect.DeepEqual(got.AccessKeyID, tt.want.AccessKeyID) {
+				t.Errorf("GetS3Credentials() got = %v, want %v", got.AccessKeyID, tt.want.AccessKeyID)
+			}
+			if !reflect.DeepEqual(got.SecretAccessKey, tt.want.SecretAccessKey) {
+				t.Errorf("GetS3Credentials() got = %v, want %v", got.SecretAccessKey, tt.want.SecretAccessKey)
+			}
+		})
+	}
+}

pkg/repository/provider/unified_repo.go

@@ -53,7 +53,7 @@ var getGCPCredentials = repoconfig.GetGCPCredentials
 var getS3BucketRegion = repoconfig.GetAWSBucketRegion
 
 type localFuncTable struct {
-	getStorageVariables   func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error)
+	getStorageVariables   func(*velerov1api.BackupStorageLocation, string, string, credentials.FileStore) (map[string]string, error)
 	getStorageCredentials func(*velerov1api.BackupStorageLocation, credentials.FileStore) (map[string]string, error)
 }
 
@@ -347,7 +347,7 @@ func (urp *unifiedRepoProvider) GetStoreOptions(param interface{}) (map[string]s
 		return map[string]string{}, errors.Errorf("invalid parameter, expect %T, actual %T", RepoParam{}, param)
 	}
 
-	storeVar, err := funcTable.getStorageVariables(repoParam.BackupLocation, urp.repoBackend, repoParam.BackupRepo.Spec.VolumeNamespace)
+	storeVar, err := funcTable.getStorageVariables(repoParam.BackupLocation, urp.repoBackend, repoParam.BackupRepo.Spec.VolumeNamespace, urp.credentialGetter.FromFile)
 	if err != nil {
 		return map[string]string{}, errors.Wrap(err, "error to get storage variables")
 	}
@@ -447,7 +447,8 @@ func getStorageCredentials(backupLocation *velerov1api.BackupStorageLocation, cr
 	return result, nil
 }
 
-func getStorageVariables(backupLocation *velerov1api.BackupStorageLocation, repoBackend string, repoName string) (map[string]string, error) {
+func getStorageVariables(backupLocation *velerov1api.BackupStorageLocation, repoBackend string, repoName string,
+	credentialFileStore credentials.FileStore) (map[string]string, error) {
 	result := make(map[string]string)
 
 	backendType := repoconfig.GetBackendType(backupLocation.Spec.Provider, backupLocation.Spec.Config)
@@ -459,6 +460,13 @@ func getStorageVariables(backupLocation *velerov1api.BackupStorageLocation, repo
 	if config == nil {
 		config = map[string]string{}
 	}
+	if backupLocation.Spec.Credential != nil {
+		credsFile, err := credentialFileStore.Path(backupLocation.Spec.Credential)
+		if err != nil {
+			return map[string]string{}, errors.WithStack(err)
+		}
+		config[repoconfig.CredentialsFileKey] = credsFile
+	}
 
 	bucket := strings.Trim(config["bucket"], "/")
 	prefix := strings.Trim(config["prefix"], "/")

pkg/repository/provider/unified_repo_test.go

@@ -437,11 +437,12 @@ func TestGetStorageVariables(t *testing.T) {
 		},
 	}
 
+	credFileStore := new(credmock.FileStore)
 	for _, tc := range testCases {
 		t.Run(tc.name, func(t *testing.T) {
 			getS3BucketRegion = tc.getS3BucketRegion
 
-			actual, err := getStorageVariables(&tc.backupLocation, tc.repoBackend, tc.repoName)
+			actual, err := getStorageVariables(&tc.backupLocation, tc.repoBackend, tc.repoName, credFileStore)
 
 			require.Equal(t, tc.expected, actual)
 
@@ -530,7 +531,7 @@ func TestGetStoreOptions(t *testing.T) {
 				BackupRepo:     &velerov1api.BackupRepository{},
 			},
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, errors.New("fake-error-2")
 				},
 			},
@@ -544,7 +545,7 @@ func TestGetStoreOptions(t *testing.T) {
 				BackupRepo:     &velerov1api.BackupRepository{},
 			},
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -604,7 +605,7 @@ func TestPrepareRepo(t *testing.T) {
 			repoService:     new(reposervicenmocks.BackupRepoService),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, errors.New("fake-store-option-error")
 				},
 			},
@@ -615,7 +616,7 @@ func TestPrepareRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -635,7 +636,7 @@ func TestPrepareRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -656,7 +657,7 @@ func TestPrepareRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -733,7 +734,7 @@ func TestForget(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -757,7 +758,7 @@ func TestForget(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -785,7 +786,7 @@ func TestForget(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -877,7 +878,7 @@ func TestInitRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -895,7 +896,7 @@ func TestInitRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -965,7 +966,7 @@ func TestConnectToRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -983,7 +984,7 @@ func TestConnectToRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -1057,7 +1058,7 @@ func TestBoostRepoConnect(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -1084,7 +1085,7 @@ func TestBoostRepoConnect(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -1110,7 +1111,7 @@ func TestBoostRepoConnect(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -1197,7 +1198,7 @@ func TestPruneRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {
@@ -1215,7 +1216,7 @@ func TestPruneRepo(t *testing.T) {
 			getter:          new(credmock.SecretStore),
 			credStoreReturn: "fake-password",
 			funcTable: localFuncTable{
-				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string) (map[string]string, error) {
+				getStorageVariables: func(*velerov1api.BackupStorageLocation, string, string, velerocredentials.FileStore) (map[string]string, error) {
 					return map[string]string{}, nil
 				},
 				getStorageCredentials: func(*velerov1api.BackupStorageLocation, velerocredentials.FileStore) (map[string]string, error) {

pkg/restore/pod_volume_restore_action.go

@@ -27,11 +27,11 @@ import (
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/runtime"
 	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
+	ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
 
 	veleroimage "github.com/vmware-tanzu/velero/internal/velero"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
-	velerov1client "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/typed/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/label"
 	"github.com/vmware-tanzu/velero/pkg/plugin/framework/common"
 	"github.com/vmware-tanzu/velero/pkg/plugin/velero"
@@ -47,16 +47,16 @@ const (
 )
 
 type PodVolumeRestoreAction struct {
-	logger                logrus.FieldLogger
-	client                corev1client.ConfigMapInterface
-	podVolumeBackupClient velerov1client.PodVolumeBackupInterface
+	logger   logrus.FieldLogger
+	client   corev1client.ConfigMapInterface
+	crClient ctrlclient.Client
 }
 
-func NewPodVolumeRestoreAction(logger logrus.FieldLogger, client corev1client.ConfigMapInterface, podVolumeBackupClient velerov1client.PodVolumeBackupInterface) *PodVolumeRestoreAction {
+func NewPodVolumeRestoreAction(logger logrus.FieldLogger, client corev1client.ConfigMapInterface, crClient ctrlclient.Client) *PodVolumeRestoreAction {
 	return &PodVolumeRestoreAction{
-		logger:                logger,
-		client:                client,
-		podVolumeBackupClient: podVolumeBackupClient,
+		logger:   logger,
+		client:   client,
+		crClient: crClient,
 	}
 }
 
@@ -86,9 +86,11 @@ func (a *PodVolumeRestoreAction) Execute(input *velero.RestoreItemActionExecuteI
 
 	log := a.logger.WithField("pod", kube.NamespaceAndName(&pod))
 
-	opts := label.NewListOptionsForBackup(input.Restore.Spec.BackupName)
-	podVolumeBackupList, err := a.podVolumeBackupClient.List(context.TODO(), opts)
-	if err != nil {
+	opts := &ctrlclient.ListOptions{
+		LabelSelector: label.NewSelectorForBackup(input.Restore.Spec.BackupName),
+	}
+	podVolumeBackupList := new(velerov1api.PodVolumeBackupList)
+	if err := a.crClient.List(context.TODO(), podVolumeBackupList, opts); err != nil {
 		return nil, errors.WithStack(err)
 	}
 

pkg/restore/pod_volume_restore_action_test.go

@@ -17,7 +17,6 @@ limitations under the License.
 package restore
 
 import (
-	"context"
 	"sort"
 	"testing"
 
@@ -25,7 +24,6 @@ import (
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 	corev1api "k8s.io/api/core/v1"
-	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/client-go/kubernetes/fake"
@@ -34,7 +32,6 @@ import (
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
 	"github.com/vmware-tanzu/velero/pkg/buildinfo"
-	velerofake "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/fake"
 	"github.com/vmware-tanzu/velero/pkg/plugin/velero"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
 	"github.com/vmware-tanzu/velero/pkg/util/kube"
@@ -131,7 +128,7 @@ func TestPodVolumeRestoreActionExecute(t *testing.T) {
 		name             string
 		pod              *corev1api.Pod
 		podFromBackup    *corev1api.Pod
-		podVolumeBackups []*velerov1api.PodVolumeBackup
+		podVolumeBackups []runtime.Object
 		want             *corev1api.Pod
 	}{
 		{
@@ -179,7 +176,7 @@ func TestPodVolumeRestoreActionExecute(t *testing.T) {
 					builder.WithAnnotations("snapshot.velero.io/not-used", "")).
 				InitContainers(builder.ForContainer("first-container", "").Result()).
 				Result(),
-			podVolumeBackups: []*velerov1api.PodVolumeBackup{
+			podVolumeBackups: []runtime.Object{
 				builder.ForPodVolumeBackup(veleroNs, "pvb-1").
 					PodName("my-pod").
 					PodNamespace("ns-1").
@@ -225,7 +222,7 @@ func TestPodVolumeRestoreActionExecute(t *testing.T) {
 					builder.ForVolume("vol-2").PersistentVolumeClaimSource("pvc-2").Result(),
 				).
 				Result(),
-			podVolumeBackups: []*velerov1api.PodVolumeBackup{
+			podVolumeBackups: []runtime.Object{
 				builder.ForPodVolumeBackup(veleroNs, "pvb-1").
 					PodName("my-pod").
 					PodNamespace("original-ns").
@@ -259,12 +256,7 @@ func TestPodVolumeRestoreActionExecute(t *testing.T) {
 	for _, tc := range tests {
 		t.Run(tc.name, func(t *testing.T) {
 			clientset := fake.NewSimpleClientset()
-			clientsetVelero := velerofake.NewSimpleClientset()
-
-			for _, podVolumeBackup := range tc.podVolumeBackups {
-				_, err := clientsetVelero.VeleroV1().PodVolumeBackups(veleroNs).Create(context.TODO(), podVolumeBackup, metav1.CreateOptions{})
-				require.NoError(t, err)
-			}
+			crClient := velerotest.NewFakeControllerRuntimeClient(t, tc.podVolumeBackups...)
 
 			unstructuredPod, err := runtime.DefaultUnstructuredConverter.ToUnstructured(tc.pod)
 			require.NoError(t, err)
@@ -294,7 +286,7 @@ func TestPodVolumeRestoreActionExecute(t *testing.T) {
 			a := NewPodVolumeRestoreAction(
 				logrus.StandardLogger(),
 				clientset.CoreV1().ConfigMaps(veleroNs),
-				clientsetVelero.VeleroV1().PodVolumeBackups(veleroNs),
+				crClient,
 			)
 
 			// method under test

pkg/restore/pv_restorer_test.go

@@ -28,8 +28,6 @@ import (
 
 	api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/fake"
-	informers "github.com/vmware-tanzu/velero/pkg/generated/informers/externalversions"
 	providermocks "github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks/volumesnapshotter/v1"
 	vsv1 "github.com/vmware-tanzu/velero/pkg/plugin/velero/volumesnapshotter/v1"
 	velerotest "github.com/vmware-tanzu/velero/pkg/test"
@@ -116,11 +114,6 @@ func TestExecutePVAction_NoSnapshotRestores(t *testing.T) {
 
 	for _, tc := range tests {
 		t.Run(tc.name, func(t *testing.T) {
-			var (
-				client                   = fake.NewSimpleClientset()
-				snapshotLocationInformer = informers.NewSharedInformerFactory(client, 0).Velero().V1().VolumeSnapshotLocations()
-			)
-
 			r := &pvRestorer{
 				logger:     velerotest.NewLogger(),
 				restorePVs: tc.restore.Spec.RestorePVs,
@@ -132,7 +125,7 @@ func TestExecutePVAction_NoSnapshotRestores(t *testing.T) {
 			}
 
 			for _, loc := range tc.locations {
-				require.NoError(t, snapshotLocationInformer.Informer().GetStore().Add(loc))
+				require.NoError(t, r.kbclient.Create(context.TODO(), loc))
 			}
 
 			res, err := r.executePVAction(tc.obj)

pkg/restore/restore.go

@@ -311,7 +311,7 @@ func (kr *kubernetesRestorer) RestoreWithResolvers(
 		discoveryHelper:                kr.discoveryHelper,
 		resourcePriorities:             kr.resourcePriorities,
 		resourceRestoreHooks:           resourceRestoreHooks,
-		hooksErrs:                      make(chan error),
+		hooksErrs:                      make(chan hook.HookErrInfo),
 		waitExecHookHandler:            waitExecHookHandler,
 		hooksContext:                   hooksCtx,
 		hooksCancelFunc:                hooksCancelFunc,
@@ -360,7 +360,7 @@ type restoreContext struct {
 	discoveryHelper                discovery.Helper
 	resourcePriorities             Priorities
 	hooksWaitGroup                 sync.WaitGroup
-	hooksErrs                      chan error
+	hooksErrs                      chan hook.HookErrInfo
 	resourceRestoreHooks           []hook.ResourceRestoreHook
 	waitExecHookHandler            hook.WaitExecHookHandler
 	hooksContext                   go_context.Context
@@ -655,8 +655,8 @@ func (ctx *restoreContext) execute() (results.Result, results.Result) {
 		ctx.hooksWaitGroup.Wait()
 		close(ctx.hooksErrs)
 	}()
-	for err := range ctx.hooksErrs {
-		errs.Velero = append(errs.Velero, err.Error())
+	for errInfo := range ctx.hooksErrs {
+		errs.Add(errInfo.Namespace, errInfo.Err)
 	}
 	ctx.log.Info("Done waiting for all post-restore exec hooks to complete")
 
@@ -1474,7 +1474,7 @@ func (ctx *restoreContext) restoreItem(obj *unstructured.Unstructured, groupReso
 	}
 
 	if ctx.resourceModifiers != nil {
-		if errList := ctx.resourceModifiers.ApplyResourceModifierRules(obj, groupResource.String(), ctx.log); errList != nil {
+		if errList := ctx.resourceModifiers.ApplyResourceModifierRules(obj, groupResource.String(), ctx.kbClient.Scheme(), ctx.log); errList != nil {
 			for _, err := range errList {
 				errs.Add(namespace, err)
 			}
@@ -1532,18 +1532,17 @@ func (ctx *restoreContext) restoreItem(obj *unstructured.Unstructured, groupReso
 	}
 
 	if restoreErr != nil {
-		// check for the existence of the object in cluster, if no error then it implies that object exists
-		// and if err then we want to judge whether there is an existing error in the previous creation.
-		// if so, we will return the 'get' error.
-		// otherwise, we will return the original creation error.
+		// check for the existence of the object that failed creation due to alreadyExist in cluster, if no error then it implies that object exists.
+		// and if err then itemExists remains false as we were not able to confirm the existence of the object via Get call or creation call.
+		// We return the get error as a warning to notify the user that the object could exist in cluster and we were not able to confirm it.
 		if !ctx.disableInformerCache {
 			fromCluster, err = ctx.getResource(groupResource, obj, namespace, name)
 		} else {
 			fromCluster, err = resourceClient.Get(name, metav1.GetOptions{})
 		}
 		if err != nil && isAlreadyExistsError {
-			ctx.log.Errorf("Error retrieving in-cluster version of %s: %v", kube.NamespaceAndName(obj), err)
-			errs.Add(namespace, err)
+			ctx.log.Warnf("Unable to retrieve in-cluster version of %s: %v, object won't be restored by velero or have restore labels, and existing resource policy is not applied", kube.NamespaceAndName(obj), err)
+			warnings.Add(namespace, err)
 			return warnings, errs, itemExists
 		}
 	}
@@ -1899,10 +1898,11 @@ func (ctx *restoreContext) waitExec(createdObj *unstructured.Unstructured) {
 		// on the ctx.podVolumeErrs channel.
 		defer ctx.hooksWaitGroup.Done()
 
+		podNs := createdObj.GetNamespace()
 		pod := new(v1.Pod)
 		if err := runtime.DefaultUnstructuredConverter.FromUnstructured(createdObj.UnstructuredContent(), &pod); err != nil {
 			ctx.log.WithError(err).Error("error converting unstructured pod")
-			ctx.hooksErrs <- err
+			ctx.hooksErrs <- hook.HookErrInfo{Namespace: podNs, Err: err}
 			return
 		}
 		execHooksByContainer, err := hook.GroupRestoreExecHooks(
@@ -1912,7 +1912,7 @@ func (ctx *restoreContext) waitExec(createdObj *unstructured.Unstructured) {
 		)
 		if err != nil {
 			ctx.log.WithError(err).Errorf("error getting exec hooks for pod %s/%s", pod.Namespace, pod.Name)
-			ctx.hooksErrs <- err
+			ctx.hooksErrs <- hook.HookErrInfo{Namespace: podNs, Err: err}
 			return
 		}
 
@@ -1922,7 +1922,7 @@ func (ctx *restoreContext) waitExec(createdObj *unstructured.Unstructured) {
 
 			for _, err := range errs {
 				// Errors are already logged in the HandleHooks method.
-				ctx.hooksErrs <- err
+				ctx.hooksErrs <- hook.HookErrInfo{Namespace: podNs, Err: err}
 			}
 		}
 	}()

pkg/restore/service_action.go

@@ -66,6 +66,9 @@ func (a *ServiceAction) Execute(input *velero.RestoreItemActionExecuteInput) (*v
 		if err := deleteNodePorts(service); err != nil {
 			return nil, err
 		}
+		if err := deleteHealthCheckNodePort(service); err != nil {
+			return nil, err
+		}
 	}
 
 	res, err := runtime.DefaultUnstructuredConverter.ToUnstructured(service)
@@ -76,6 +79,72 @@ func (a *ServiceAction) Execute(input *velero.RestoreItemActionExecuteInput) (*v
 	return velero.NewRestoreItemActionExecuteOutput(&unstructured.Unstructured{Object: res}), nil
 }
 
+func deleteHealthCheckNodePort(service *corev1api.Service) error {
+	// Check service type and external traffic policy setting,
+	// if the setting is not applicable for HealthCheckNodePort, return early.
+	if service.Spec.ExternalTrafficPolicy != corev1api.ServiceExternalTrafficPolicyTypeLocal ||
+		service.Spec.Type != corev1api.ServiceTypeLoadBalancer {
+		return nil
+	}
+
+	// HealthCheckNodePort is already 0, return.
+	if service.Spec.HealthCheckNodePort == 0 {
+		return nil
+	}
+
+	// Search HealthCheckNodePort from server's last-applied-configuration
+	// annotation(HealthCheckNodePort is specified by `kubectl apply` command)
+	lastAppliedConfig, ok := service.Annotations[annotationLastAppliedConfig]
+	if ok {
+		appliedServiceUnstructured := new(map[string]interface{})
+		if err := json.Unmarshal([]byte(lastAppliedConfig), appliedServiceUnstructured); err != nil {
+			return errors.WithStack(err)
+		}
+
+		healthCheckNodePort, exist, err := unstructured.NestedFloat64(*appliedServiceUnstructured, "spec", "healthCheckNodePort")
+		if err != nil {
+			return errors.WithStack(err)
+		}
+
+		// Found healthCheckNodePort in lastAppliedConfig annotation,
+		// and the value is not 0. No need to delete, return.
+		if exist && healthCheckNodePort != 0 {
+			return nil
+		}
+	}
+
+	// Search HealthCheckNodePort from ManagedFields(HealthCheckNodePort
+	// is specified by `kubectl apply --server-side` command).
+	for _, entry := range service.GetManagedFields() {
+		if entry.FieldsV1 == nil {
+			continue
+		}
+		fields := new(map[string]interface{})
+		if err := json.Unmarshal(entry.FieldsV1.Raw, fields); err != nil {
+			return errors.WithStack(err)
+		}
+
+		_, exist, err := unstructured.NestedMap(*fields, "f:spec", "f:healthCheckNodePort")
+		if err != nil {
+			return errors.WithStack(err)
+		}
+		if !exist {
+			continue
+		}
+		// Because the format in ManagedFields is `f:healthCheckNodePort: {}`,
+		// cannot get the value, check whether exists is enough.
+		// Found healthCheckNodePort in ManagedFields.
+		// No need to delete. Return.
+		return nil
+	}
+
+	// Cannot find HealthCheckNodePort from Annotation and
+	// ManagedFields, which means it's auto-generated. Delete it.
+	service.Spec.HealthCheckNodePort = 0
+
+	return nil
+}
+
 func deleteNodePorts(service *corev1api.Service) error {
 	if service.Spec.Type == corev1api.ServiceTypeExternalName {
 		return nil

pkg/restore/service_action_test.go

@@ -36,7 +36,8 @@ import (
 func svcJSON(ports ...corev1api.ServicePort) string {
 	svc := corev1api.Service{
 		Spec: corev1api.ServiceSpec{
-			Ports: ports,
+			HealthCheckNodePort: 8080,
+			Ports:               ports,
 		},
 	}
 
@@ -486,6 +487,164 @@ func TestServiceActionExecute(t *testing.T) {
 				},
 			},
 		},
+		{
+			name: "If PreserveNodePorts is True in restore spec then HealthCheckNodePort always preserved.",
+			obj: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+					Ports: []corev1api.ServicePort{
+						{
+							Name:     "http",
+							Port:     80,
+							NodePort: 8080,
+						},
+						{
+							Name:     "hepsiburada",
+							NodePort: 9025,
+						},
+					},
+				},
+			},
+			restore: builder.ForRestore(api.DefaultNamespace, "").PreserveNodePorts(true).Result(),
+			expectedRes: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+					Ports: []corev1api.ServicePort{
+						{
+							Name:     "http",
+							Port:     80,
+							NodePort: 8080,
+						},
+						{
+							Name:     "hepsiburada",
+							NodePort: 9025,
+						},
+					},
+				},
+			},
+		},
+		{
+			name: "If PreserveNodePorts is False in restore spec then HealthCheckNodePort should be cleaned.",
+			obj: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+			restore: builder.ForRestore(api.DefaultNamespace, "").PreserveNodePorts(false).Result(),
+			expectedRes: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   0,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+		},
+		{
+			name: "If PreserveNodePorts is false in restore spec, but service is not expected, then HealthCheckNodePort should be kept.",
+			obj: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeCluster,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+			restore: builder.ForRestore(api.DefaultNamespace, "").PreserveNodePorts(false).Result(),
+			expectedRes: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeCluster,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+		},
+		{
+			name: "If PreserveNodePorts is false in restore spec, but HealthCheckNodePort can be found in Annotation, then it should be kept.",
+			obj: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:        "svc-1",
+					Annotations: map[string]string{annotationLastAppliedConfig: svcJSON()},
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+			restore: builder.ForRestore(api.DefaultNamespace, "").PreserveNodePorts(false).Result(),
+			expectedRes: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:        "svc-1",
+					Annotations: map[string]string{annotationLastAppliedConfig: svcJSON()},
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+		},
+		{
+			name: "If PreserveNodePorts is false in restore spec, but HealthCheckNodePort can be found in ManagedFields, then it should be kept.",
+			obj: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+					ManagedFields: []metav1.ManagedFieldsEntry{
+						{
+							FieldsV1: &metav1.FieldsV1{
+								Raw: []byte(`{"f:spec":{"f:healthCheckNodePort":{}}}`),
+							},
+						},
+					},
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+			restore: builder.ForRestore(api.DefaultNamespace, "").PreserveNodePorts(false).Result(),
+			expectedRes: corev1api.Service{
+				ObjectMeta: metav1.ObjectMeta{
+					Name: "svc-1",
+					ManagedFields: []metav1.ManagedFieldsEntry{
+						{
+							FieldsV1: &metav1.FieldsV1{
+								Raw: []byte(`{"f:spec":{"f:healthCheckNodePort":{}}}`),
+							},
+						},
+					},
+				},
+				Spec: corev1api.ServiceSpec{
+					HealthCheckNodePort:   8080,
+					ExternalTrafficPolicy: corev1api.ServiceExternalTrafficPolicyTypeLocal,
+					Type:                  corev1api.ServiceTypeLoadBalancer,
+				},
+			},
+		},
 	}
 
 	for _, test := range tests {

pkg/test/api_server.go

@@ -24,14 +24,11 @@ import (
 	discoveryfake "k8s.io/client-go/discovery/fake"
 	dynamicfake "k8s.io/client-go/dynamic/fake"
 	kubefake "k8s.io/client-go/kubernetes/fake"
-
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/fake"
 )
 
 // APIServer contains in-memory fakes for all of the relevant
 // Kubernetes API server clients.
 type APIServer struct {
-	VeleroClient    *fake.Clientset
 	KubeClient      *kubefake.Clientset
 	DynamicClient   *dynamicfake.FakeDynamicClient
 	DiscoveryClient *DiscoveryClient
@@ -43,7 +40,6 @@ func NewAPIServer(t *testing.T) *APIServer {
 	t.Helper()
 
 	var (
-		veleroClient  = fake.NewSimpleClientset()
 		kubeClient    = kubefake.NewSimpleClientset()
 		dynamicClient = dynamicfake.NewSimpleDynamicClientWithCustomListKinds(runtime.NewScheme(),
 			map[schema.GroupVersionResource]string{
@@ -65,7 +61,6 @@ func NewAPIServer(t *testing.T) *APIServer {
 	)
 
 	return &APIServer{
-		VeleroClient:    veleroClient,
 		KubeClient:      kubeClient,
 		DynamicClient:   dynamicClient,
 		DiscoveryClient: discoveryClient,

pkg/test/fake_controller_runtime_client.go

@@ -21,6 +21,7 @@ import (
 
 	snapshotv1api "github.com/kubernetes-csi/external-snapshotter/client/v4/apis/volumesnapshot/v1"
 	"github.com/stretchr/testify/require"
+	appsv1api "k8s.io/api/apps/v1"
 	corev1api "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/runtime"
 	"sigs.k8s.io/controller-runtime/pkg/client"
@@ -38,6 +39,8 @@ func NewFakeControllerRuntimeClientBuilder(t *testing.T) *k8sfake.ClientBuilder
 	require.NoError(t, err)
 	err = corev1api.AddToScheme(scheme)
 	require.NoError(t, err)
+	err = appsv1api.AddToScheme(scheme)
+	require.NoError(t, err)
 	err = snapshotv1api.AddToScheme(scheme)
 	require.NoError(t, err)
 	return k8sfake.NewClientBuilder().WithScheme(scheme)
@@ -51,7 +54,13 @@ func NewFakeControllerRuntimeClient(t *testing.T, initObjs ...runtime.Object) cl
 	require.NoError(t, err)
 	err = corev1api.AddToScheme(scheme)
 	require.NoError(t, err)
+	err = appsv1api.AddToScheme(scheme)
+	require.NoError(t, err)
 	err = snapshotv1api.AddToScheme(scheme)
 	require.NoError(t, err)
 	return k8sfake.NewClientBuilder().WithScheme(scheme).WithRuntimeObjects(initObjs...).Build()
 }
+
+func NewFakeControllerRuntimeWatchClient(t *testing.T, initObjs ...runtime.Object) client.WithWatch {
+	return NewFakeControllerRuntimeClientBuilder(t).WithRuntimeObjects(initObjs...).Build()
+}

pkg/uploader/provider/kopia_test.go

@@ -35,12 +35,12 @@ import (
 	"github.com/vmware-tanzu/velero/internal/credentials"
 	"github.com/vmware-tanzu/velero/internal/credentials/mocks"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/scheme"
 	"github.com/vmware-tanzu/velero/pkg/repository"
 	udmrepo "github.com/vmware-tanzu/velero/pkg/repository/udmrepo"
 	udmrepomocks "github.com/vmware-tanzu/velero/pkg/repository/udmrepo/mocks"
 	"github.com/vmware-tanzu/velero/pkg/uploader"
 	"github.com/vmware-tanzu/velero/pkg/uploader/kopia"
+	"github.com/vmware-tanzu/velero/pkg/util"
 )
 
 type FakeBackupProgressUpdater struct {
@@ -64,7 +64,7 @@ func (f *FakeRestoreProgressUpdater) UpdateProgress(p *uploader.Progress) {}
 func TestRunBackup(t *testing.T) {
 	var kp kopiaProvider
 	kp.log = logrus.New()
-	updater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: kp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()}
+	updater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: kp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}
 
 	testCases := []struct {
 		name           string
@@ -121,7 +121,7 @@ func TestRunBackup(t *testing.T) {
 func TestRunRestore(t *testing.T) {
 	var kp kopiaProvider
 	kp.log = logrus.New()
-	updater := FakeRestoreProgressUpdater{PodVolumeRestore: &velerov1api.PodVolumeRestore{}, Log: kp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()}
+	updater := FakeRestoreProgressUpdater{PodVolumeRestore: &velerov1api.PodVolumeRestore{}, Log: kp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}
 
 	testCases := []struct {
 		name            string

pkg/uploader/provider/provider_test.go

@@ -28,7 +28,7 @@ import (
 	"github.com/vmware-tanzu/velero/internal/credentials"
 	"github.com/vmware-tanzu/velero/internal/credentials/mocks"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/scheme"
+	"github.com/vmware-tanzu/velero/pkg/util"
 )
 
 type NewUploaderProviderTestCase struct {
@@ -42,7 +42,7 @@ type NewUploaderProviderTestCase struct {
 func TestNewUploaderProvider(t *testing.T) {
 	// Mock objects or dependencies
 	ctx := context.Background()
-	client := fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()
+	client := fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()
 	repoIdentifier := "repoIdentifier"
 	bsl := &velerov1api.BackupStorageLocation{}
 	backupRepo := &velerov1api.BackupRepository{}

pkg/uploader/provider/restic_test.go

@@ -33,9 +33,9 @@ import (
 	"github.com/vmware-tanzu/velero/internal/credentials"
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	"github.com/vmware-tanzu/velero/pkg/builder"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/scheme"
 	"github.com/vmware-tanzu/velero/pkg/restic"
 	"github.com/vmware-tanzu/velero/pkg/uploader"
+	"github.com/vmware-tanzu/velero/pkg/util"
 	"github.com/vmware-tanzu/velero/pkg/util/filesystem"
 )
 
@@ -148,7 +148,7 @@ func TestResticRunBackup(t *testing.T) {
 				tc.volMode = uploader.PersistentVolumeFilesystem
 			}
 			if !tc.nilUpdater {
-				updater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: tc.rp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()}
+				updater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: tc.rp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}
 				_, _, err = tc.rp.RunBackup(context.Background(), "var", "", map[string]string{}, false, parentSnapshot, tc.volMode, &updater)
 			} else {
 				_, _, err = tc.rp.RunBackup(context.Background(), "var", "", map[string]string{}, false, parentSnapshot, tc.volMode, nil)
@@ -221,7 +221,7 @@ func TestResticRunRestore(t *testing.T) {
 			}
 			var err error
 			if !tc.nilUpdater {
-				updater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: tc.rp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(scheme.Scheme).Build()}
+				updater := FakeBackupProgressUpdater{PodVolumeBackup: &velerov1api.PodVolumeBackup{}, Log: tc.rp.log, Ctx: context.Background(), Cli: fake.NewClientBuilder().WithScheme(util.VeleroScheme).Build()}
 				err = tc.rp.RunRestore(context.Background(), "", "var", tc.volMode, &updater)
 			} else {
 				err = tc.rp.RunRestore(context.Background(), "", "var", tc.volMode, nil)

pkg/util/encode/encode.go

@@ -25,9 +25,10 @@ import (
 
 	"github.com/pkg/errors"
 	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/runtime/serializer"
 
 	v1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
-	"github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned/scheme"
+	"github.com/vmware-tanzu/velero/pkg/util"
 )
 
 // Encode converts the provided object to the specified format
@@ -56,8 +57,11 @@ func To(obj runtime.Object, format string, w io.Writer) error {
 // Only objects registered in the velero scheme, or objects with their TypeMeta set will have valid encoders.
 func EncoderFor(format string, obj runtime.Object) (runtime.Encoder, error) {
 	var encoder runtime.Encoder
+
+	codecFactory := serializer.NewCodecFactory(util.VeleroScheme)
+
 	desiredMediaType := fmt.Sprintf("application/%s", format)
-	serializerInfo, found := runtime.SerializerInfoForMediaType(scheme.Codecs.SupportedMediaTypes(), desiredMediaType)
+	serializerInfo, found := runtime.SerializerInfoForMediaType(codecFactory.SupportedMediaTypes(), desiredMediaType)
 	if !found {
 		return nil, errors.Errorf("unable to locate an encoder for %q", desiredMediaType)
 	}
@@ -69,7 +73,7 @@ func EncoderFor(format string, obj runtime.Object) (runtime.Encoder, error) {
 	if !obj.GetObjectKind().GroupVersionKind().Empty() {
 		return encoder, nil
 	}
-	encoder = scheme.Codecs.EncoderForVersion(encoder, v1.SchemeGroupVersion)
+	encoder = codecFactory.EncoderForVersion(encoder, v1.SchemeGroupVersion)
 	return encoder, nil
 }
 

pkg/util/scheme.go

@@ -0,0 +1,19 @@
+package util
+
+import (
+	"k8s.io/apimachinery/pkg/runtime"
+	utilruntime "k8s.io/apimachinery/pkg/util/runtime"
+
+	v1 "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
+	"github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1"
+)
+
+var VeleroScheme = runtime.NewScheme()
+
+func init() {
+	localSchemeBuilder := runtime.SchemeBuilder{
+		v1.AddToScheme,
+		v2alpha1.AddToScheme,
+	}
+	utilruntime.Must(localSchemeBuilder.AddToScheme(VeleroScheme))
+}

site/content/docs/main/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/main/file-system-backup.md

@@ -63,7 +63,7 @@ repository on AWS S3, the full backup repo path for namespace1 would be `https:/
 for namespace2 would be `https://s3-us-west-2.amazonaws.com/bucket/kopia/ns2`.  
 
 There may be additional installation steps depending on the cloud provider plugin you are using. You should refer to the 
-[plugin specific documentation](supported-providers.md) for the must up to date information.  
+[plugin specific documentation](supported-providers.md) for the most up to date information.  
 
 **Note:** Currently, Velero creates a secret named `velero-repo-credentials` in the velero install namespace, containing a default backup repository password.
 You can update the secret with your own password encoded as base64 prior to the first backup (i.e., FS Backup, data mover) targeting to the backup repository. The value of the key to update is

site/content/docs/main/restore-reference.md

@@ -280,21 +280,23 @@ There are two ways to delete a Restore object:
 1. Deleting with `velero restore delete` will delete the Custom Resource representing the restore, along with its individual log and results files. It will not delete any objects that were created by the restore in your cluster.
 2. Deleting with `kubectl -n velero delete restore` will delete the Custom Resource representing the restore. It will not delete restore log or results files from object storage, or any objects that were created during the restore in your cluster.
 
-## What happens to NodePorts when restoring Services
+## What happens to NodePorts and HealthCheckNodePort when restoring Services
 
-During a restore, Velero deletes **Auto assigned** NodePorts by default and Services get new **auto assigned** nodePorts after restore.
+During a restore, Velero deletes **Auto assigned** NodePorts and HealthCheckNodePort by default and Services get new **auto assigned** nodePorts and healthCheckNodePort after restore.
 
-Velero auto detects **explicitly specified** NodePorts using **`last-applied-config`** annotation and they are **preserved** after restore. NodePorts can be explicitly specified as `.spec.ports[*].nodePort` field on Service definition.
+Velero auto detects **explicitly specified** NodePorts using **`last-applied-config`** annotation and **`managedFields`**. They are **preserved** after restore. NodePorts can be explicitly specified as `.spec.ports[*].nodePort` field on Service definition.
 
-### Always Preserve NodePorts
+Velero will do the same to the `HealthCheckNodePort` as `NodePorts`.
 
-It is not always possible to set nodePorts explicitly on some big clusters because of operational complexity. As the Kubernetes [NodePort documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) states, "if you want a specific port number, you can specify a value in the `nodePort` field. The control plane will either allocate you that port or report that the API transaction failed. This means that you need to take care of possible port collisions yourself. You also have to use a valid port number, one that's inside the range configured for NodePort use.""
+### Always Preserve NodePorts and HealthCheckNodePort
+
+It is not always possible to set nodePorts and healthCheckNodePort explicitly on some big clusters because of operational complexity. As the Kubernetes [NodePort documentation](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) states, "if you want a specific port number, you can specify a value in the `nodePort` field. The control plane will either allocate you that port or report that the API transaction failed. This means that you need to take care of possible port collisions yourself. You also have to use a valid port number, one that's inside the range configured for NodePort use.""
 
 The clusters which are not explicitly specifying nodePorts may still need to restore original NodePorts in the event of a disaster. Auto assigned nodePorts are typically defined on Load Balancers located in front of cluster. Changing all these nodePorts on Load Balancers is another operation complexity you are responsible for updating after disaster if nodePorts are changed.
 
-Use the `velero restore create ` command's `--preserve-nodeports` flag to preserve Service nodePorts always, regardless of whether nodePorts are explicitly specified or not. This flag is used for preserving the original nodePorts from a backup and can be used as `--preserve-nodeports` or `--preserve-nodeports=true`. If this flag is present, Velero will not remove the nodePorts when restoring a Service, but will try to use the nodePorts from the backup.
+Use the `velero restore create ` command's `--preserve-nodeports` flag to preserve Service nodePorts and healthCheckNodePort always, regardless of whether nodePorts are explicitly specified or not. This flag is used for preserving the original nodePorts and healthCheckNodePort from a backup and can be used as `--preserve-nodeports` or `--preserve-nodeports=true`. If this flag is present, Velero will not remove the nodePorts and healthCheckNodePort when restoring a Service, but will try to use the nodePorts from the backup.
 
-Trying to preserve nodePorts may cause port conflicts when restoring on situations below:
+Trying to preserve nodePorts and healthCheckNodePort may cause port conflicts when restoring on situations below:
 
 - If the nodePort from the backup is already allocated on the target cluster then Velero prints error log as shown below and continues the restore operation.
 

site/content/docs/main/restore-resource-modifiers.md

@@ -105,3 +105,83 @@ resourceModifierRules:
 -  Update a container's image using a json patch with positional arrays
 kubectl patch pod valid-pod -type='json' -p='[{"op": "replace", "path": "/spec/containers/0/image", "value":"new image"}]'
 - Before creating the resource modifier yaml, you can try it out using kubectl patch command. The same commands should work as it is.
+
+#### JSON Merge Patch
+You can modify a resource using JSON Merge Patch
+```yaml
+version: v1
+resourceModifierRules:
+- conditions:
+    groupResource: pods
+    namespaces:
+    - ns1
+  mergePatches:
+  - patchData: |
+      {
+        "metadata": {
+          "annotations": {
+            "foo": null
+          }
+        }
+      }
+```
+- The above configmap will apply the Merge Patch to all the pods in namespace ns1 and remove the annotation `foo` from the pods.
+- Both json and yaml format are supported for the patchData.
+- For more details, please refer to [this doc](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/)
+
+#### Strategic Merge Patch
+You can modify a resource using Strategic Merge Patch
+```yaml
+version: v1
+resourceModifierRules:
+- conditions:
+    groupResource: pods
+    resourceNameRegex: "^my-pod$"
+    namespaces:
+    - ns1
+  strategicPatches:
+  - patchData: |
+      {
+        "spec": {
+          "containers": [
+            {
+              "name": "nginx",
+              "image": "repo2/nginx"
+            }
+          ]
+        }
+      }
+```
+- The above configmap will apply the Strategic Merge Patch to the pod with name my-pod in namespace ns1 and update the image of container nginx to `repo2/nginx`.
+- Both json and yaml format are supported for the patchData.
+- For more details, please refer to [this doc](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/)
+
+
+### Conditional Patches in ALL Patch Types
+A new field `matches` is added in conditions to support conditional patches.
+
+Example of matches in conditions
+```yaml
+version: v1
+resourceModifierRules:
+- conditions:
+    groupResource: persistentvolumeclaims.storage.k8s.io
+    matches:
+    - path: "/spec/storageClassName"
+      value: "premium"
+  mergePatches:
+  - patchData: |
+      {
+        "metadata": {
+          "annotations": {
+            "foo": null
+          }
+        }
+      }
+```
+- The above configmap will apply the Merge Patch to all the PVCs in all namespaces with storageClassName premium and remove the annotation `foo` from the PVCs.
+- You can specify multiple rules in the `matches` list. The patch will be applied only if all the matches are satisfied.
+
+### Wildcard Support for GroupResource
+The user can specify a wildcard for groupResource in the conditions' struct. This will allow the user to apply the patches for all the resources of a particular group or all resources in all groups. For example, `*.apps` will apply to all the resources in the `apps` group, `*` will apply to all the resources in core group, `*.*` will apply to all the resources in all groups.
+- If both `*.groupName` and `namespaces` are specified, the patches will be applied to all the namespaced resources in this group in the specified namespaces and all the cluster resources in this group.

site/content/docs/v1.10/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.11/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.12/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.5/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.6/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.7/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.8/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

site/content/docs/v1.9/contributions/tencent-config.md

@@ -17,7 +17,7 @@ You can deploy Velero on Tencent [TKE](https://cloud.tencent.com/document/produc
 
 Create an object bucket for Velero to store backups in the Tencent Cloud COS console. For how to create, please refer to Tencent Cloud COS [Create a bucket](https://cloud.tencent.com/document/product/436/13309) usage instructions.
 
-Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315.E5.8D.95.E4.B8.AA.E6.8E.88.E6.9D.83) Tencent user instructions.
+Set access to the bucket through the object storage console, the bucket needs to be **read** and **written**, so the account is granted data reading, data writing permissions. For how to configure, see the [permission access settings](https://cloud.tencent.com/document/product/436/13315) Tencent user instructions.
 
 ## Get bucket access credentials
 

test/pkg/client/factory.go

@@ -34,16 +34,12 @@ import (
 
 	velerov1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v1"
 	velerov2alpha1api "github.com/vmware-tanzu/velero/pkg/apis/velero/v2alpha1"
-	clientset "github.com/vmware-tanzu/velero/pkg/generated/clientset/versioned"
 )
 
 // Factory knows how to create a VeleroClient and Kubernetes client.
 type Factory interface {
 	// BindFlags binds common flags (--kubeconfig, --namespace) to the passed-in FlagSet.
 	BindFlags(flags *pflag.FlagSet)
-	// Client returns a VeleroClient. It uses the following priority to specify the cluster
-	// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
-	Client() (clientset.Interface, error)
 	// KubeClient returns a Kubernetes client. It uses the following priority to specify the cluster
 	// configuration: --kubeconfig flag, KUBECONFIG environment variable, in-cluster configuration.
 	KubeClient() (kubernetes.Interface, error)
@@ -114,19 +110,6 @@ func (f *factory) ClientConfig() (*rest.Config, error) {
 	return Config(f.kubeconfig, f.kubecontext, f.baseName, f.clientQPS, f.clientBurst)
 }
 
-func (f *factory) Client() (clientset.Interface, error) {
-	clientConfig, err := f.ClientConfig()
-	if err != nil {
-		return nil, err
-	}
-
-	veleroClient, err := clientset.NewForConfig(clientConfig)
-	if err != nil {
-		return nil, errors.WithStack(err)
-	}
-	return veleroClient, nil
-}
-
 func (f *factory) KubeClient() (kubernetes.Interface, error) {
 	clientConfig, err := f.ClientConfig()
 	if err != nil {