Commit Graph

582 Commits

Author SHA1 Message Date
Carlisia
a71e43b2b7 Split velero plugin client into its own package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-19 16:05:37 -07:00
Steve Kriss
1eac10ca9f Merge pull request #1288 from carlisia/c-plugins-II
Split plugin framework into its own package
2019-03-19 16:44:58 -06:00
Carlisia
7dfe58d37f Split plugin framework into its own package
Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-19 15:36:31 -07:00
Steve Kriss
7d66fc31bd pkg/restic: check for & skip hostPath PVC/PVs
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-19 08:57:47 -06:00
Steve Kriss
183bea369d make resticrepositories non-restorable resources (#1296)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-18 11:38:37 -07:00
KubeKween
73514a003b Move plugin interfaces to same package (#1264)
* Move plugin interfaces to same package

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-14 16:35:06 -04:00
Steve Kriss
7674332313 pass --log-level to plugins (#1278)
Plumb the log level through to plugin processes


Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-14 10:53:36 -07:00
Steve Kriss
409116fce8 add basic plugin panic handlers (#1270)
* add server-side panic handlers to all plugin methods

Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-13 14:07:52 -04:00
Nolan Brubaker
503b112638 Add location resources and tests (#1277)
Add locations and tests to install package

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-03-13 11:23:00 -06:00
tsturzl
89ca2571f3 AWS zone on volume IDs
Signed-off-by: Travis Sturzl<travis@metismachine.com>
2019-03-12 13:04:33 -06:00
Nolan Brubaker
394548afcd Merge pull request #1254 from skriss/remove-wait-for-pv
remove restore code that waits for a PV to become Available
2019-03-11 13:20:59 -04:00
KubeKween
4ee41a13a0 Merge pull request #1261 from asaf-erlich/patch-1
Update ark restore to not open every single file open during extracti…
2019-03-08 14:59:09 -08:00
asaf-erlich
4041044a93 Update ark restore to not open every single file open during extraction of the data
Original error was:

```
ark -n <redacted> restore logs <redacted>
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=backups.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=restores.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Starting restore of backup backup/<redacted>" logSource="pkg/restore/restore.go:342"
time="2019-03-06T18:31:06Z" level=info msg="error unzipping and extracting: open /tmp/604421455/resources/rolebindings.rbac.authorization.k8s.io/namespaces/<redacted>/<redacted>: too many open files" logSource="pkg/restore/restore.go:346"
```

Downloading the directory from s3 and untarring it I found 1036 files. The ulimit -n output says 1024. This is our team's best guess at a root cause. But the code fixed in the PR definitely is holding all the files open until the method closes: https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1-8d070894cb01

Please note my go code abilities are not great and I did not test this. I just edited the file in github. All I did was remove the defer and put fileClose after the copy is done. Theoretically this should only hold one file open at a time now. Let me know if you want me to do any further steps.

Thank you,
-Asaf

Signed-off-by: Asaf Erlich <aerlich@groupon.com>
2019-03-07 11:23:35 -05:00
Michal Wieczorek
1354e2b6ff Add original item to restore plugin interface
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2019-03-05 17:09:42 -08:00
Steve Kriss
e29aa74a23 remove restore code that waits for a PV to become Available
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-05 17:04:52 -07:00
Nolan Brubaker
ce3f43e876 Merge pull request #1251 from skriss/backup-extractor
move backup extraction logic to its own type
2019-03-05 16:37:48 -05:00
Nolan Brubaker
5912fe66e5 Merge pull request #1250 from skriss/extract-pv-restorer
move pvRestorer and tests to their own files
2019-03-05 16:37:32 -05:00
DheerajSShetty
1b031f0cc4 Improve describe output
* Move Phase to right under Metadata(name/namespace/label/annotations)
 * Move Validation errors: section right after Phase: section and only
   show it if the item has a phase of FailedValidation
 * For restores move Warnings and Errors under Validation errors. Do not
   show Warnings or Errors if there are none.

Signed-off-by: DheerajSShetty <dheerajs@vmware.com>

Fixes #987
2019-03-04 13:21:18 -08:00
Steve Kriss
88e6a740f2 move pvRestorer and tests to their own files
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:07:25 -07:00
Steve Kriss
0fec56f488 move backup extraction logic to its own type
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:05:58 -07:00
KubeKween
e21940bee1 Merge pull request #1231 from skriss/k8s-1.12-deps
update kubernetes and azure dependencies to 1.12
2019-02-28 15:09:06 -08:00
Nolan Brubaker
fcf21813a5 Merge pull request #1246 from skriss/preserve-storageclass
when restoring a PV, don't remove its spec.storageClassName
2019-02-28 18:02:20 -05:00
KubeKween
65f3926caa Merge pull request #1146 from skriss/replace-map-utils-final
replace ark's map_utils.go with structured types and apimachinery's unstructured helpers
2019-02-28 14:37:07 -08:00
Steve Kriss
31501b79b2 when deleting snapshot, don't error if it doesn't exist
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss
6bf837b233 address breaking changes in Azure SDK
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss
58e471bda0 fix breaking changes
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss
61eab7dca3 update generated code using 1.12 generator
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss
21c57c46b3 when restoring a PV, don't remove its spec.storageClassName
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 14:34:36 -07:00
KubeKween
7353294b7f Merge pull request #1236 from DheerajSShetty/squashed_feature
defer closing the ReadCloser in ObjectStoreGRPCServer.GetObject
2019-02-28 13:12:05 -08:00
DheerajSShetty
032aaac508 defer closing the ReadCloser in ObjectStoreGRPCServer.GetObject
Signed-off-by: DheerajSShetty <dheerajs@vmware.com>

Fixes #1093
2019-02-28 11:51:10 -08:00
KubeKween
03b8f5397f Merge pull request #1203 from skriss/v0.11-changes
v0.11 changes
2019-02-28 07:01:23 -08:00
Steve Kriss
e3e76c2067 pkg/restic: fix concurrency bugs causing dupe repos, panics
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-25 18:30:26 -08:00
Nolan Brubaker
49eeeb04f0 Merge pull request #1204 from skriss/deprecation-notices
add deprecation notices to pkg/apis/ark/v1 types
2019-02-12 16:02:47 -05:00
Steve Kriss
e1d414338c Merge pull request #1200 from nrb/fix-1183
Fix restoring GCP regional disks
2019-02-12 13:48:42 -07:00
Nolan Brubaker
0ffaeb949d Fix restoring GCP regional disks
To create a regional disk, the URLs for the zones in which the disk is
replicated must be provided to the GCP API.

Fixes #1183

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-12 15:42:50 -05:00
Steve Kriss
ed73be44fd Merge pull request #1206 from nzoueidi/master
Add UTC time to --schedule
2019-02-12 09:19:53 -07:00
Naeil ZOUEIDI
988ce573c0 Add UTC time to --schedule
Signed-off-by: Naeil Ezzoueidi <naeilzoueidi@ubuntu.com>
2019-02-12 11:04:10 -05:00
Steve Kriss
780dc4551f fix compile err & test
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:45:04 -07:00
Steve Kriss
32835c63f6 code review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss
86c5c25d13 code review: remove obsolete commented code
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss
250f109c41 delete pkg/util/collections/map_utils.go & tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss
d8e9b772ff move MergeMaps func into pkg/restore where it's used
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:48 -07:00
Steve Kriss
88fc6e2141 pkg/controller: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:24 -07:00
Steve Kriss
38ad7d71f5 pkg/restore: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:20:41 -07:00
Steve Kriss
e91c841c59 pkg/backup: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:18:51 -07:00
Steve Kriss
902c0f797f pkg/podexec: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:18:37 -07:00
Steve Kriss
296dd6617e pkg/cloudprovider: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:17:33 -07:00
Steve Kriss
4cd8170386 add deprecation notices to pkg/apis/ark/v1 types
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 15:44:52 -07:00
Steve Kriss
0df30c1e89 ark->velero in issue templates & bug cmd
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 12:43:51 -07:00
KubeKween
378011baf6 Merge pull request #1153 from daved/fix/1142-restore_log
Clarify conditional nature of restore in log
2019-02-11 09:22:25 -08:00