diff --git a/README.md b/README.md index a38b8ed7..a7fa2f6b 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ Download [latest release](https://github.com/versity/versitygw/releases) | Linux/amd64 | Linux/arm64 | MacOS/amd64 | MacOS/arm64 | BSD/amd64 | BSD/arm64 | |:-----------:|:-----------:|:-----------:|:-----------:|:---------:|:---------:| | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | ✔️ | - + ### Use Cases * Turn your local filesystem into an S3 server with a single command! * Proxy S3 requests to S3 storage @@ -45,11 +45,11 @@ Versity Gateway, a simple to use tool for seamless inline translation between AW The server translates incoming S3 API requests and transforms them into equivalent operations to the backend service. By leveraging this gateway server, applications can interact with the S3-compatible API on top of already existing storage systems. This project enables leveraging existing infrastructure investments while seamlessly integrating with S3-compatible systems, offering increased flexibility and compatibility in managing data storage. -The Versity Gateway is focused on performance, simplicity, and expandability. The Versity Gateway is designed with modularity in mind, enabling future extensions to support additional backend storage systems. At present, the Versity Gateway supports any generic POSIX file backend storage, Versity’s open source ScoutFS filesystem, Azure Blob Storage, and other S3 servers. +The Versity Gateway is focused on performance, simplicity, and expandability. The Versity Gateway is designed with modularity in mind, enabling future extensions to support additional backend storage systems. At present, the Versity Gateway supports any generic POSIX file backend storage, Versity’s open source ScoutFS filesystem, Azure Blob Storage, and other S3 servers. -The gateway is completely stateless. Multiple Versity Gateway instances may be deployed in a cluster to increase aggregate throughput. The Versity Gateway’s stateless architecture allows any request to be serviced by any gateway thereby distributing workloads and enhancing performance. Load balancers may be used to evenly distribute requests across the cluster of gateways for optimal performance. +The gateway is completely stateless. Multiple Versity Gateway instances may be deployed in a cluster to increase aggregate throughput. The Versity Gateway’s stateless architecture allows any request to be serviced by any gateway thereby distributing workloads and enhancing performance. Load balancers may be used to evenly distribute requests across the cluster of gateways for optimal performance. -The S3 HTTP(S) server and routing is implemented using the [Fiber](https://gofiber.io) web framework. This framework is actively developed with a focus on performance. S3 API compatibility leverages the official [aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) whenever possible for maximum service compatibility with AWS S3. +The S3 HTTP(S) server and routing is implemented using the [Fiber](https://gofiber.io) web framework. This framework is actively developed with a focus on performance. S3 API compatibility leverages the official [aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) whenever possible for maximum service compatibility with AWS S3. ## Getting Started See the [Quickstart](https://github.com/versity/versitygw/wiki/Quickstart) documentation. @@ -111,7 +111,7 @@ documentation for running within Docker. A Helm chart is provided to easily run Versity in Kubernetes environments: ```sh -helm install versitygw oci://ghcr.io/versity/versitygw/charts/versity:latest +helm install versitygw oci://ghcr.io/versity/versitygw/charts/versitygw ``` Please refer to the [chart's README](./chart/README.md) for more information and configuration parameters. @@ -125,9 +125,8 @@ Please refer to the [chart's README](./chart/README.md) for more information and info@versity.com
+1 844 726 8826 -### @versitysoftware -[![linkedin](https://github.com/versity/versitygw/blob/assets/assets/linkedin.jpg)](https://www.linkedin.com/company/versity/)   +### @versitysoftware +[![linkedin](https://github.com/versity/versitygw/blob/assets/assets/linkedin.jpg)](https://www.linkedin.com/company/versity/)   [![twitter](https://github.com/versity/versitygw/blob/assets/assets/twitter.jpg)](https://twitter.com/VersitySoftware)   [![facebook](https://github.com/versity/versitygw/blob/assets/assets/facebook.jpg)](https://www.facebook.com/versitysoftware)   [![instagram](https://github.com/versity/versitygw/blob/assets/assets/instagram.jpg)](https://www.instagram.com/versitysoftware/)   - diff --git a/chart/Chart.yaml b/chart/Chart.yaml index 99efa99b..d6aefa4b 100644 --- a/chart/Chart.yaml +++ b/chart/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: versitygw description: A Helm chart for deploying the Versity S3 Gateway on Kubernetes type: application -version: 0.1.0 +version: 0.2.0 sources: - https://github.com/versity/versitygw icon: https://raw.githubusercontent.com/versity/versitygw/main/webui/web/assets/images/Versity-logo-blue-horizontal.png diff --git a/chart/README.md b/chart/README.md index 715e5ea8..c5db7bca 100644 --- a/chart/README.md +++ b/chart/README.md @@ -2,6 +2,8 @@ Versity is an S3-compatible storage gateway that proxies S3 API requests to a variety of backend storage systems. +> **Note**: the chart is currently in development state and breaking changes (with regards to the Helm values structure or the chart behavior) may occur until we reach a 1.0 release of the Helm chart. + ## Overview [versitygw](https://github.com/versity/versitygw) is an S3-compatible gateway that fronts POSIX filesystems, ScoutFS, S3, Azure Blob Storage, or custom plugin backends. This chart deploys versitygw on Kubernetes as a Deployment and Service, with optional support for TLS termination, Ingress, HTTPRoutes, certificate provisioning (via `cert-manager` CRDs), IAM, an Admin API, a browser-based WebUI, persistent storage, and NetworkPolicy. @@ -26,22 +28,53 @@ helm install my-versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ > **Production note:** Passing credentials via `--set` stores them in Helm's release history. For production deployments, create a Kubernetes Secret in advance and reference it with `auth.existingSecret=`. The Secret must contain the keys `rootAccessKeyId` and `rootSecretAccessKey`. +## Upgrading + +The versioning of this Helm chart and of `versitygw` itself are currently not coupled to each other. + +By default, the Helm chart uses the `latest` tag for the versitygw container image. +For production and multi-replica deployment, it is strongly recommended to always pin a specific version, like so: + +```yaml +# values.yaml + +image: + repository: ghcr.io/versity/versitygw + tag: "v1.2.0" +``` + +To upgrade the `versitygw` version, only the `image.tag` value needs to be adjusted and the Helm charts needs to be re-deployed (with the same values), e.g.: + +```sh +helm upgrade my-versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ + --reuse-values \ + --set image.tag=v1.3.1 +``` + +To upgrade only the Helm chart, use the following command: + +```sh +helm upgrade my-versitygw oci://ghcr.io/versity/versitygw/charts/versitygw \ + --reuse-values \ + --version 0.2.0 +``` + ## Backend Storage The `gateway.backend.type` value selects the storage backend. Use `gateway.backend.args` to pass backend-specific arguments. | Backend | Description | Example `gateway.backend.args` | -|---------|-------------|-------------------------------| -| `posix` | POSIX-compatible local or network filesystem (default) | `/mnt/data` | -| `scoutfs` | [ScoutFS](https://scoutfs.org/) high-performance filesystem | `/mnt/scoutfs` | -| `s3` | Proxy to an existing S3-compatible object store | `--access KEY --secret SECRET --endpoint https://s3.example.com` | -| `azure` | Azure Blob Storage | `--account myaccount --key mykey` | -| `plugin` | Custom backend via shared library plugin | `/path/to/plugin.so` | +|---------|-------------|--------------------------------| +| [posix](https://github.com/versity/versitygw/wiki/POSIX-Backend) | POSIX-compatible local or network filesystem (default) | `/mnt/data` | +| [scoutfs](https://github.com/versity/versitygw/wiki/ScoutFS-Backend) | [ScoutFS](https://scoutfs.org/) high-performance filesystem | `/mnt/scoutfs` | +| [s3](https://github.com/versity/versitygw/wiki/S3-Backend) | Proxy to an existing S3-compatible object store | `--access KEY --secret SECRET --endpoint https://s3.example.com` | +| [azure](https://github.com/versity/versitygw/wiki/AzureBlob-Backend) | Azure Blob Storage | `--account myaccount --key mykey` | +| [plugin](https://github.com/versity/versitygw/wiki/Plugin-Backend) | Custom backend via shared library plugin | `/path/to/plugin.so` | ## Optional Features | Feature | Key values | -|---------|-----------| +|---------|------------| | **TLS** | `tls.enabled=true` — serve HTTPS; supply a TLS Secret via `certificate.secretName` or let cert-manager provision one | | **cert-manager** | `certificate.create=true`, `certificate.issuerRef`, `certificate.dnsNames` | | **Ingress** | `ingress.enabled=true`, `ingress.className`, `ingress.hosts`, `ingress.tls` | @@ -52,9 +85,18 @@ The `gateway.backend.type` value selects the storage backend. Use `gateway.backe | **Persistence** | `persistence.enabled=true` — provisions a PVC for backend data and IAM storage; defaults to `10Gi` | | **NetworkPolicy** | `networkPolicy.enabled=true` — restricts ingress to selected pods/namespaces; allows all egress | -## Multi-Replica Deployments +## Scaling and Persistence -When setting `replicaCount` greater than 1, the underlying storage must support concurrent access. Set `persistence.accessMode=ReadWriteMany` and use a storage class that supports it (e.g. NFS, CephFS, or a cloud-managed `RWX` provisioner). +By default, this chart enables persistence via a `PersistentVolumeClaim` (PVC) to ensure data consistency and prevent data loss. + +### Horizontal Scaling (replicas > 1) + +When scaling `versitygw` horizontally by setting `replicaCount` greater than 1, special care must be taken regarding the storage backend: + +- **POSIX or Internal IAM**: These backends store state locally on the filesystem. + - Using **ReadWriteOnce (RWO)**: All replicas must be scheduled on the **same Kubernetes node** to share the same volume. This is useful for process-level concurrency (e.g., when using high-performance local block storage) but limits high availability across nodes. + - Using **ReadWriteMany (RWX)**: Replicas can be distributed across **multiple nodes** in the cluster. This is the recommended approach for true horizontal scaling and high availability. When using RWX, it is also recommended to use pod anti-affinity (via `affinity` in `values.yaml`) to ensure pods are distributed across nodes/zones. +- **Stateless Backends (S3, Azure)**: If you are using a stateless storage backend (e.g. proxying to another S3 store) **and** you are either not using IAM or using an external IAM provider (e.g. LDAP, Vault), persistence can be safely disabled by setting `persistence.enabled=false`. ## Configuration diff --git a/chart/templates/deployment.yaml b/chart/templates/deployment.yaml index ca06cffc..d4f5ee18 100644 --- a/chart/templates/deployment.yaml +++ b/chart/templates/deployment.yaml @@ -1,3 +1,9 @@ +{{- /* Safety check: multiple replicas with local state must have persistence enabled */}} +{{- if and (gt (int .Values.replicaCount) 1) (not .Values.persistence.enabled) }} + {{- if or (eq .Values.gateway.backend.type "posix") (and .Values.iam.enabled (eq .Values.iam.type "internal")) }} + {{- fail "Multiple replicas with POSIX backend or Internal IAM require persistence.enabled=true to prevent data loss and inconsistency across pods. If using a stateless backend (e.g. S3, Azure) and external IAM, set persistence.enabled=false." }} + {{- end }} +{{- end }} apiVersion: apps/v1 kind: Deployment metadata: diff --git a/chart/values.yaml b/chart/values.yaml index ca08013a..50f8b02e 100644 --- a/chart/values.yaml +++ b/chart/values.yaml @@ -281,7 +281,7 @@ certificate: persistence: # If disabled, an emptyDir (ephemeral!) will be used instead of a PVC. # NOTE: Data will be lost on pod restart if persistence is disabled. - enabled: false + enabled: true # Use an existing PVC instead of creating a new one. claimName: "" size: 10Gi