Commit Graph
573 Commits
Author SHA1 Message Date
tsturzl 89ca2571f3 AWS zone on volume IDs
Signed-off-by: Travis Sturzl<travis@metismachine.com>
2019-03-12 13:04:33 -06:00
Nolan BrubakerandGitHub 394548afcd Merge pull request #1254 from skriss/remove-wait-for-pv
remove restore code that waits for a PV to become Available
2019-03-11 13:20:59 -04:00
KubeKweenandGitHub 4ee41a13a0 Merge pull request #1261 from asaf-erlich/patch-1
Update ark restore to not open every single file open during extracti…
2019-03-08 14:59:09 -08:00
asaf-erlichandAsaf Erlich 4041044a93 Update ark restore to not open every single file open during extraction of the data
Original error was:

```
ark -n <redacted> restore logs <redacted>
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=backups.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=restores.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Starting restore of backup backup/<redacted>" logSource="pkg/restore/restore.go:342"
time="2019-03-06T18:31:06Z" level=info msg="error unzipping and extracting: open /tmp/604421455/resources/rolebindings.rbac.authorization.k8s.io/namespaces/<redacted>/<redacted>: too many open files" logSource="pkg/restore/restore.go:346"
```

Downloading the directory from s3 and untarring it I found 1036 files. The ulimit -n output says 1024. This is our team's best guess at a root cause. But the code fixed in the PR definitely is holding all the files open until the method closes: https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1-8d070894cb01

Please note my go code abilities are not great and I did not test this. I just edited the file in github. All I did was remove the defer and put fileClose after the copy is done. Theoretically this should only hold one file open at a time now. Let me know if you want me to do any further steps.

Thank you,
-Asaf

Signed-off-by: Asaf Erlich <aerlich@groupon.com>
2019-03-07 11:23:35 -05:00
Michal WieczorekandCarlisia 1354e2b6ff Add original item to restore plugin interface
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2019-03-05 17:09:42 -08:00
Steve Kriss e29aa74a23 remove restore code that waits for a PV to become Available
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-05 17:04:52 -07:00
Nolan BrubakerandGitHub ce3f43e876 Merge pull request #1251 from skriss/backup-extractor
move backup extraction logic to its own type
2019-03-05 16:37:48 -05:00
Nolan BrubakerandGitHub 5912fe66e5 Merge pull request #1250 from skriss/extract-pv-restorer
move pvRestorer and tests to their own files
2019-03-05 16:37:32 -05:00
DheerajSShetty 1b031f0cc4 Improve describe output
* Move Phase to right under Metadata(name/namespace/label/annotations)
 * Move Validation errors: section right after Phase: section and only
   show it if the item has a phase of FailedValidation
 * For restores move Warnings and Errors under Validation errors. Do not
   show Warnings or Errors if there are none.

Signed-off-by: DheerajSShetty <dheerajs@vmware.com>

Fixes #987
2019-03-04 13:21:18 -08:00
Steve Kriss 88e6a740f2 move pvRestorer and tests to their own files
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:07:25 -07:00
Steve Kriss 0fec56f488 move backup extraction logic to its own type
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:05:58 -07:00
KubeKweenandGitHub e21940bee1 Merge pull request #1231 from skriss/k8s-1.12-deps
update kubernetes and azure dependencies to 1.12
2019-02-28 15:09:06 -08:00
Nolan BrubakerandGitHub fcf21813a5 Merge pull request #1246 from skriss/preserve-storageclass
when restoring a PV, don't remove its spec.storageClassName
2019-02-28 18:02:20 -05:00
KubeKweenandGitHub 65f3926caa Merge pull request #1146 from skriss/replace-map-utils-final
replace ark's map_utils.go with structured types and apimachinery's unstructured helpers
2019-02-28 14:37:07 -08:00
Steve Kriss 31501b79b2 when deleting snapshot, don't error if it doesn't exist
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss 6bf837b233 address breaking changes in Azure SDK
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss 58e471bda0 fix breaking changes
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss 61eab7dca3 update generated code using 1.12 generator
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 15:33:05 -07:00
Steve Kriss 21c57c46b3 when restoring a PV, don't remove its spec.storageClassName
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 14:34:36 -07:00
KubeKweenandGitHub 7353294b7f Merge pull request #1236 from DheerajSShetty/squashed_feature
defer closing the ReadCloser in ObjectStoreGRPCServer.GetObject
2019-02-28 13:12:05 -08:00
DheerajSShetty 032aaac508 defer closing the ReadCloser in ObjectStoreGRPCServer.GetObject
Signed-off-by: DheerajSShetty <dheerajs@vmware.com>

Fixes #1093
2019-02-28 11:51:10 -08:00
KubeKweenandGitHub 03b8f5397f Merge pull request #1203 from skriss/v0.11-changes
v0.11 changes
2019-02-28 07:01:23 -08:00
Steve Kriss e3e76c2067 pkg/restic: fix concurrency bugs causing dupe repos, panics
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-25 18:30:26 -08:00
Nolan BrubakerandGitHub 49eeeb04f0 Merge pull request #1204 from skriss/deprecation-notices
add deprecation notices to pkg/apis/ark/v1 types
2019-02-12 16:02:47 -05:00
Steve KrissandGitHub e1d414338c Merge pull request #1200 from nrb/fix-1183
Fix restoring GCP regional disks
2019-02-12 13:48:42 -07:00
Nolan Brubaker 0ffaeb949d Fix restoring GCP regional disks
To create a regional disk, the URLs for the zones in which the disk is
replicated must be provided to the GCP API.

Fixes #1183

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-12 15:42:50 -05:00
Steve KrissandGitHub ed73be44fd Merge pull request #1206 from nzoueidi/master
Add UTC time to --schedule
2019-02-12 09:19:53 -07:00
Naeil ZOUEIDI 988ce573c0 Add UTC time to --schedule
Signed-off-by: Naeil Ezzoueidi <naeilzoueidi@ubuntu.com>
2019-02-12 11:04:10 -05:00
Steve Kriss 780dc4551f fix compile err & test
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:45:04 -07:00
Steve Kriss 32835c63f6 code review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss 86c5c25d13 code review: remove obsolete commented code
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss 250f109c41 delete pkg/util/collections/map_utils.go & tests
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss d8e9b772ff move MergeMaps func into pkg/restore where it's used
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:48 -07:00
Steve Kriss 88fc6e2141 pkg/controller: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:24 -07:00
Steve Kriss 38ad7d71f5 pkg/restore: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:20:41 -07:00
Steve Kriss e91c841c59 pkg/backup: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:18:51 -07:00
Steve Kriss 902c0f797f pkg/podexec: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:18:37 -07:00
Steve Kriss 296dd6617e pkg/cloudprovider: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:17:33 -07:00
Steve Kriss 4cd8170386 add deprecation notices to pkg/apis/ark/v1 types
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 15:44:52 -07:00
Steve Kriss 0df30c1e89 ark->velero in issue templates & bug cmd
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 12:43:51 -07:00
KubeKweenandGitHub 378011baf6 Merge pull request #1153 from daved/fix/1142-restore_log
Clarify conditional nature of restore in log
2019-02-11 09:22:25 -08:00
Daved b2b1ee44ea Clarify restore log when object unchanged
Signed-off-by: Daved <daved@codemodus.com>
2019-02-11 09:12:07 -08:00
Steve KrissandGitHub 4583aa7078 Merge pull request #826 from nrb/fix-691
Wait for namespace to terminate before restoring
2019-02-07 11:07:25 -07:00
Nolan BrubakerandNolan Brubaker 890202f2e4 Wait for PV/namespace to delete before restore
If a PV already exists, wait for it, it's associated PVC, and associated
namespace to be deleted before attempting to restore it.

If a namespace already exists, wait for it to be deleted before
attempting to restore it.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-06 15:43:50 -05:00
Nolan BrubakerandGitHub 3c7737c8b1 Merge pull request #1166 from skriss/tweak-server-log-version
velero server: log version and git SHA at startup
2019-02-06 15:28:51 -05:00
Nolan BrubakerandGitHub 52ecc45ec8 Merge pull request #1167 from skriss/logging-and-misc-cleanup
Logging and misc cleanup
2019-02-06 14:45:08 -05:00
Nolan BrubakerandGitHub 46e87661c0 Merge pull request #1171 from skriss/restic-stats
use restic stats instead of check to check repo existence
2019-02-06 14:28:07 -05:00
Steve Kriss 723cda2697 use restic stats instead of check to check repo existence
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-06 12:14:11 -07:00
Nolan BrubakerandGitHub 5f0ff026b0 Merge pull request #1156 from skriss/restic-v0.9.4
upgrade to restic v0.9.4 and replace --hostname with --host
2019-02-06 14:03:44 -05:00
Nolan Brubaker c1a817b4e9 Use current info in bug template
The binary, deployment, and namespace have yet to change for most users,
so make sure the bug template has the correct information when filing.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-05 16:11:56 -05:00