Commit Graph

206 Commits

Author SHA1 Message Date
Steve Kriss
bb9c3f6a1a rename BlockStore to VolumeSnapshotter
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-27 14:55:28 -06:00
Steve Kriss
a111eed2af update license headers to Velero contributors (#1302)
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-20 12:32:48 -07:00
KubeKween
73514a003b Move plugin interfaces to same package (#1264)
* Move plugin interfaces to same package

Signed-off-by: Carlisia <carlisiac@vmware.com>
2019-03-14 16:35:06 -04:00
Nolan Brubaker
394548afcd Merge pull request #1254 from skriss/remove-wait-for-pv
remove restore code that waits for a PV to become Available
2019-03-11 13:20:59 -04:00
KubeKween
4ee41a13a0 Merge pull request #1261 from asaf-erlich/patch-1
Update ark restore to not open every single file open during extracti…
2019-03-08 14:59:09 -08:00
asaf-erlich
4041044a93 Update ark restore to not open every single file open during extraction of the data
Original error was:

```
ark -n <redacted> restore logs <redacted>
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=backups.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=restores.ark.heptio.com logSource="pkg/restore/restore.go:124"
time="2019-03-06T18:31:06Z" level=info msg="Starting restore of backup backup/<redacted>" logSource="pkg/restore/restore.go:342"
time="2019-03-06T18:31:06Z" level=info msg="error unzipping and extracting: open /tmp/604421455/resources/rolebindings.rbac.authorization.k8s.io/namespaces/<redacted>/<redacted>: too many open files" logSource="pkg/restore/restore.go:346"
```

Downloading the directory from s3 and untarring it I found 1036 files. The ulimit -n output says 1024. This is our team's best guess at a root cause. But the code fixed in the PR definitely is holding all the files open until the method closes: https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1-8d070894cb01

Please note my go code abilities are not great and I did not test this. I just edited the file in github. All I did was remove the defer and put fileClose after the copy is done. Theoretically this should only hold one file open at a time now. Let me know if you want me to do any further steps.

Thank you,
-Asaf

Signed-off-by: Asaf Erlich <aerlich@groupon.com>
2019-03-07 11:23:35 -05:00
Michal Wieczorek
1354e2b6ff Add original item to restore plugin interface
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2019-03-05 17:09:42 -08:00
Steve Kriss
e29aa74a23 remove restore code that waits for a PV to become Available
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-05 17:04:52 -07:00
Nolan Brubaker
ce3f43e876 Merge pull request #1251 from skriss/backup-extractor
move backup extraction logic to its own type
2019-03-05 16:37:48 -05:00
Steve Kriss
88e6a740f2 move pvRestorer and tests to their own files
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:07:25 -07:00
Steve Kriss
0fec56f488 move backup extraction logic to its own type
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-03-01 15:05:58 -07:00
Nolan Brubaker
fcf21813a5 Merge pull request #1246 from skriss/preserve-storageclass
when restoring a PV, don't remove its spec.storageClassName
2019-02-28 18:02:20 -05:00
Steve Kriss
21c57c46b3 when restoring a PV, don't remove its spec.storageClassName
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-28 14:34:36 -07:00
Steve Kriss
780dc4551f fix compile err & test
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:45:04 -07:00
Steve Kriss
32835c63f6 code review feedback
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss
86c5c25d13 code review: remove obsolete commented code
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:49 -07:00
Steve Kriss
d8e9b772ff move MergeMaps func into pkg/restore where it's used
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:21:48 -07:00
Steve Kriss
38ad7d71f5 pkg/restore: remove usage of pkg/util/collections
Signed-off-by: Steve Kriss <krisss@vmware.com>
2019-02-11 16:20:41 -07:00
KubeKween
378011baf6 Merge pull request #1153 from daved/fix/1142-restore_log
Clarify conditional nature of restore in log
2019-02-11 09:22:25 -08:00
Daved
b2b1ee44ea Clarify restore log when object unchanged
Signed-off-by: Daved <daved@codemodus.com>
2019-02-11 09:12:07 -08:00
Nolan Brubaker
890202f2e4 Wait for PV/namespace to delete before restore
If a PV already exists, wait for it, it's associated PVC, and associated
namespace to be deleted before attempting to restore it.

If a namespace already exists, wait for it to be deleted before
attempting to restore it.

Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-06 15:43:50 -05:00
Nolan Brubaker
43714caaec Rename Ark to Velero!!!
Signed-off-by: Nolan Brubaker <brubakern@vmware.com>
2019-02-04 17:35:22 -05:00
Andy Goldstein
62d8c642d2 Remove default token from all service accounts
Instead of only removing the default token from a service account when
it already exists in the cluster, always remove it. If the service
account already exists, continue to do the merging logic.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-12-04 11:41:18 -05:00
Michal Wieczorek
43cd2ca36f Delete spec.priority in pod restore action
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2018-11-22 09:54:49 +01:00
Steve Kriss
449cac5806 Merge pull request #1032 from mwieczorek/plugins
Move clearing up of metadata before plugin's actions
2018-11-08 12:00:33 -07:00
Andy Goldstein
3788014552 Fix check for non-found PV
We were checking for nil, but were getting back an empty
*unstructured.Unstructured{} instead, along with a NotFound error.
Change the logic to check for the NotFound error instead of a nil
object.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-11-08 10:56:14 -05:00
Michal Wieczorek
7abe115674 Move clearing up of metadata before plugin's actions
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2018-11-02 09:56:33 +01:00
Nolan Brubaker
6231aaa875 Verify PV doesn't exist before creating new volume
Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-10-30 13:57:00 -04:00
Steve Kriss
195e6aaf00 fix bug preventing PV snapshots from v0.10 backups from restoring
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 12:44:05 -06:00
Andy Goldstein
bca585162f Run 'make update' to update formatting
Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
2018-10-23 13:08:42 -04:00
Steve Kriss
90d9be59d3 support restoring/deleting legacy backups with .status.volumeBackups
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-23 09:55:40 -06:00
Wayne Witzel III
406b50a71b update restore process using snapshot locations
Signed-off-by: Wayne Witzel III <wayne@riotousliving.com>
2018-10-17 13:40:42 -06:00
Steve Kriss
38e86ceff5 pod restore action: check initContainers exist before processing
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-11 13:44:02 -06:00
KubeKween
d579784692 Merge pull request #910 from james-powis/fix_restic_restore_service_user_token
Drop volumes matching name ServiceAccountName-token-
2018-10-09 11:32:23 -07:00
James Powis
30369c2ad5 Drop volumeMounts from initContainers if SAToken
Signed-off-by: James Powis <powisj@gmail.com>
2018-10-05 16:00:58 -06:00
Steve Kriss
d009163b67 update restic to support multiple backup storage locations
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-05 09:10:35 -06:00
Steve Kriss
0c88eefc0d bug: fix restic restores when using namespace mappings
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-10-03 08:09:55 -06:00
Michal Wieczorek
0f1c5c283f Enable restoring resources with ownerReference set
Signed-off-by: Michal Wieczorek <wieczorek-michal@wp.pl>
2018-10-02 12:04:51 +02:00
KubeKween
94b8fae15a Merge pull request #891 from skriss/fix-restore-logs
remove restore log helper for accurate line #'s
2018-10-01 21:02:17 -07:00
Steve Kriss
729a688b36 remove restore log helper for accurate line #'s
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-09-30 14:45:32 -06:00
ffd2subroutine
9fdf85130a Minor code cleanup
Signed-off-by: ffd2subroutine <ffd2subroutine@users.noreply.github.com>
2018-09-30 20:43:11 +02:00
Shubheksha Jalan
e3222a9e3f move code dealing with node ports into a separate function
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 18:08:57 +05:30
Shubheksha Jalan
d95b18bad8 ignore spec.ports not being there for services of type ExternalName
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 01:35:13 +05:30
Shubheksha Jalan
907a9fbdd8 fix error during restore when spec.ports are not found
Signed-off-by: Shubheksha Jalan <jshubheksha@gmail.com>
2018-09-25 01:02:47 +05:30
James Powis
a5cb2ce4b7 Change from regex matching default-token to prefix SA-token-
Signed-off-by: James Powis <powisj@gmail.com>
2018-09-19 13:06:10 -06:00
Steve Kriss
8a5d6f9111 Merge pull request #781 from nrb/fix-777
Fix map merging logic
2018-08-22 11:29:02 -07:00
Nolan Brubaker
ea50ebf2b5 Fix map merging logic
Fixes #777

Signed-off-by: Nolan Brubaker <nolan@heptio.com>
2018-08-21 15:30:30 -04:00
Nolan Brubaker
65cd5c602f Merge pull request #754 from skriss/namespace-restore-label
update how we label restored objects
2018-08-09 15:17:50 -04:00
Steve Kriss
95b2b90006 add backup name label to restored objects
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-08 16:51:33 -07:00
Steve Kriss
0c3ac67b6d start using a namespaced label on restored objects, deprecate old label
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-08-08 16:34:39 -07:00