asaf-erlichandAsaf Erlich
4041044a93
Original error was: ``` ark -n <redacted> restore logs <redacted> time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=nodes logSource="pkg/restore/restore.go:124" time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events logSource="pkg/restore/restore.go:124" time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=events.events.k8s.io logSource="pkg/restore/restore.go:124" time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=backups.ark.heptio.com logSource="pkg/restore/restore.go:124" time="2019-03-06T18:31:06Z" level=info msg="Not including resource" groupResource=restores.ark.heptio.com logSource="pkg/restore/restore.go:124" time="2019-03-06T18:31:06Z" level=info msg="Starting restore of backup backup/<redacted>" logSource="pkg/restore/restore.go:342" time="2019-03-06T18:31:06Z" level=info msg="error unzipping and extracting: open /tmp/604421455/resources/rolebindings.rbac.authorization.k8s.io/namespaces/<redacted>/<redacted>: too many open files" logSource="pkg/restore/restore.go:346" ``` Downloading the directory from s3 and untarring it I found 1036 files. The ulimit -n output says 1024. This is our team's best guess at a root cause. But the code fixed in the PR definitely is holding all the files open until the method closes: https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1-8d070894cb01 Please note my go code abilities are not great and I did not test this. I just edited the file in github. All I did was remove the defer and put fileClose after the copy is done. Theoretically this should only hold one file open at a time now. Let me know if you want me to do any further steps. Thank you, -Asaf Signed-off-by: Asaf Erlich <aerlich@groupon.com>
Overview
Velero (formerly Heptio Ark) gives you tools to back up and restore your Kubernetes cluster resources and persistent volumes. Velero lets you:
- Take backups of your cluster and restore in case of loss.
- Copy cluster resources to other clusters.
- Replicate your production environment for development and testing environments.
Velero consists of:
- A server that runs on your cluster
- A command-line client that runs locally
You can run Velero in clusters on a cloud provider or on-premises. For detailed information, see Compatible Storage Providers.
Installation
We strongly recommend that you use an official release of Velero. The tarballs for each release contain the command-line client and version-specific sample YAML files for deploying Velero to your cluster. Follow the instructions under the Install section of our documentation to get started.
The code and sample YAML files in the master branch of the Velero repository are under active development and are not guaranteed to be stable. Use them at your own risk!
More information
The documentation provides a getting started guide, plus information about building from source, architecture, extending Velero, and more.
Please use the version selector at the top of the site to ensure you are using the appropriate documentation for your version of Velero.
Troubleshooting
If you encounter issues, review the troubleshooting docs, file an issue, or talk to us on the #velero channel on the Kubernetes Slack server.
Contributing
Thanks for taking the time to join our community and start contributing!
Feedback and discussion are available on the mailing list.
Before you start
- Please familiarize yourself with the Code of Conduct before contributing.
- See CONTRIBUTING.md for instructions on the developer certificate of origin that we require.
- Read how we're using ZenHub for project and roadmap planning
Pull requests
- We welcome pull requests. Feel free to dig through the issues and jump in.
Changelog
See the list of releases to find out about feature changes.