Commit Graph

157 Commits

Author SHA1 Message Date
Ben McClelland 3e4c31f14a Merge pull request #1474 from versity/sis/bucket-lifecycle-configuration-actions-not-implemented
feat: adds not implemented routes for bucket lifecycle configuration actions
2025-08-20 17:09:59 -07:00
niksis02 6b450a5c11 feat: adds not implemented routes for bucket request payment actions
Closes #1455

Adds `NotImplemented` routes for bucket request payment S3 actions:
- `PutBucketRequestPayment`
- `GetBucketRequestPayment`
2025-08-21 00:54:31 +04:00
niksis02 5f28a7449e feat: adds not implemented routes for bucket logging actions
Closes #1444

Adds `NotImplemented` routes for bucket logging S3 actions:
- `PutBucketLogging`
- `GetBucketLogging`
2025-08-20 21:07:09 +04:00
niksis02 025b0ee3c8 feat: adds not implemented routes for bucket lifecycle configuration actions
Closes #1443

Adds `NotImplemented` routes for bucket lifecycle configuration S3 actions.
- `PutBucketLifecycleConfiguration`
- `GetBucketLifecycleConfiguration`
- `DeleteBucketLifecycle`
2025-08-20 20:48:58 +04:00
niksis02 09031a30e5 feat: bucket cors implementation
Closes #1003

**Changes Introduced:**

1. **S3 Bucket CORS Actions**

   * Implemented the following S3 bucket CORS APIs:

     * `PutBucketCors` – Configure CORS rules for a bucket.
     * `GetBucketCors` – Retrieve the current CORS configuration for a bucket.
     * `DeleteBucketCors` – Remove CORS configuration from a bucket.

2. **CORS Preflight Handling**

   * Added an `OPTIONS` endpoint to handle browser preflight requests.
   * The endpoint evaluates incoming requests against bucket CORS rules and returns the appropriate `Access-Control-*` headers.

3. **CORS Middleware**

   * Implemented middleware that:

     * Checks if a bucket has CORS configured.
     * Detects the `Origin` header in the request.
     * Adds the necessary `Access-Control-*` headers to the response when the request matches the bucket CORS configuration.
2025-08-20 20:45:09 +04:00
Ben McClelland dafe099d9b fix: iam ldap reconnect after network disconnects
Handle LDAP connection failures by attempting to reconnect.
This should resolve the issue of connections being closed by
the LDAP server after a period of inactivity.

Fixes #1464
2025-08-19 18:17:12 -07:00
niksis02 24b88e20e0 feat: adds not implemented routes for bucket inventory configuration actions
Closes #1440

Adds `NotImplemented` routes for bucket inventory configuration S3 actions:
- `PutBucketInventoryConfiguration`
- `GetBucketInventoryConfiguration`
- `ListBucketInventoryConfigurations`
- `DeleteBucketInventoryConfiguration`
2025-08-19 21:49:38 +04:00
niksis02 cdccdcc4d6 feat: adds not implemented routes for bucket intelligent tiering actions
Closes #1440

Adds `NotImplemented` routes for intelligent tiering S3 actions:
- `PutBucketIntelligentTieringConfiguration`
- `GetBucketIntelligentTieringConfiguration`
- `ListBucketIntelligentTieringConfigurations`
- `DeleteBucketIntelligentTieringConfiguration`
2025-08-19 21:23:05 +04:00
niksis02 ed92ad3daa feat: adds not implemented routes for bucket ecryption actions
Closes #1439

Adds `NotImplemented` routes for bucket encryption S3 actions:

- `PutBucketEncryption`
- `GetBucketEncryption`
- `DeleteBucketEncryption`
2025-08-19 20:30:02 +04:00
niksis02 8db196634b fix: adds not implemented routes for bucket analytics s3 actions.
Fixes #1433
Fixes #1437
Fixes #1438

Adds 4 routes to return `NotImplemented` for bucket analytics `S3` actions:

- `PutBucketAnalyticsConfiguration`
- `GetBucketAnalyticsConfiguration`
- `DeleteBucketAnalyticsConfiguration`
- `ListBucketAnalyticsConfiguration`
2025-08-19 02:14:31 +04:00
niksis02 e5850ff11f feat: adds copy source validation for x-amz-copy-source header.
Fixes #1388
Fixes #1389
Fixes #1390
Fixes #1401

Adds the `x-amz-copy-source` header validation for `CopyObject` and `UploadPartCopy` in front-end.
The error:
```
	ErrInvalidCopySource: {
		Code:           "InvalidArgument",
		Description:    "Copy Source must mention the source bucket and key: sourcebucket/sourcekey.",
		HTTPStatusCode: http.StatusBadRequest,
	},
```
is now deprecated.

The conditional read/write headers validation in `CopyObject` should come with #821 and #822.
2025-07-22 14:40:11 -07:00
niksis02 7f9ab35347 feat: implements unit tests for object PUT controllers 2025-07-22 20:55:22 +04:00
niksis02 866b07b98f feat: implementes unit tests for all the bucket action controllers. 2025-07-22 20:55:22 +04:00
niksis02 5be9e3bd1e feat: a total refactoring of the gateway middlewares by lowering them from server to router level. 2025-07-22 20:55:22 +04:00
Ben McClelland 08ccf821f9 fix: refresh expired iam vault tokens when needed
The IAM vault client stores an access token once authenticated,
but this token will expire after a certain amount of time set
by the server generating the token. Once this token is expired
or revoked, it can no longer be use by the vault client. So
the client should try to refresh the token with any errors
indicating expired or revoked tokens.

Fixes #976
2025-07-17 09:32:40 -07:00
Maksim Loviagin e39ab6f0ee feat: split the vault mount path into kv and auth 2025-07-15 18:57:44 +00:00
Ben McClelland f295df2217 fix: add new auth method to update ownership within acl
Add helper util auth.UpdateBucketACLOwner() that sets new
default ACL based on new owner and removes old bucket policy.

The ChangeBucketOwner() remains in the backend.Backend
interface in case there is ever a backend that needs to manage
ownership in some other way than with bucket ACLs. The arguments
are changing to clarify the updated owner. This will break any
plugins implementing the old interface. They should use the new
auth.UpdateBucketACLOwner() or implement the corresponding
change specific for the backend.
2025-07-09 16:16:34 -07:00
Ben McClelland 91b904d10f fix: add retry for iam freeipa http requests
The IPA service connections have been seen to not always work
correctly on the first network connection attempt. Add retry
logic for errors that appear to be transient network issues.
2025-07-07 22:28:58 -07:00
niksis02 458db64e2d feat: implements public bucket access.
This implementation introduces **public buckets**, which are accessible without signature-based authentication.

There are two ways to grant public access to a bucket:

* **Bucket ACLs**
* **Bucket Policies**

Only `Get` and `List` operations are permitted on public buckets. All **write operations** require authentication, regardless of whether public access is granted through an ACL or a policy.

The implementation includes an `AuthorizePublicBucketAccess` middleware, which checks if public access has been granted to the bucket. If so, authentication middlewares are skipped. For unauthenticated requests, appropriate errors are returned based on the specific S3 action.

---

**1. Bucket-Level Operations:**

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test"
    }
  ]
}
```

**2. Object-Level Operations:**

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test/*"
    }
  ]
}
```

**3. Both Bucket and Object-Level Operations:**

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test"
    },
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test/*"
    }
  ]
}
```

---

```sh
aws s3api create-bucket --bucket test --object-ownership BucketOwnerPreferred
aws s3api put-bucket-acl --bucket test --acl public-read
```
2025-07-02 00:11:10 +04:00
Yuriy Halytskyy 925f89465e fix: IPA IAM use http proxy from environment 2025-05-20 09:33:10 -07:00
niksis02 4334f869f2 feat: makes the user role editable in /update-user iam endpoint
Closes #1295

Makes the user `role` mutable in /update-user admin endpoint.
Integrates the changes in the `admin update-user` cli command, by adding the `role` flag for a user role modification.
2025-05-14 23:10:15 +04:00
Ben McClelland e98f7763d0 fix: use createtemp()/rename() for iam internal files
This cleans up a previous fix to #630 to use a better temp/rename
scheme thats less likely to have bad side effects.

The test for the previous issue still passes these cases, and we
will be less liekly to find a case where the file doesnt exist
or corrpted backup files.
2025-05-03 12:39:06 -07:00
Ben McClelland eb72905401 Merge pull request #1200 from versity/sis/obj-lock-actions-invalid-body
fix: Changes the error type to MalformedXML for PutObjectRetention and PutObjectLegalHold empty or invalid body
2025-04-08 20:55:15 -07:00
niksis02 20d00f7f6d fix: Changes the error type to MalformedXML for PutObjectRetention and PutObjectLegalHold empty or invalid body
Fixes #1185
Fixes #1191

`PutObjectLegalHold` and `PutObjectRetention` should return `MalformedXML` if the request body is empty or invalid.`
2025-04-08 19:01:00 +04:00
niksis02 2b7a71568c fix: Changes the GetObjectLegalHold result root tag to LegalHold
Fixes #1193

Changes the xml root tag element to `LegalHold` in `GetObjectLegalHold` response.
2025-04-08 18:26:26 +04:00
niksis02 30f319bc92 fix: Hanldes the case when Statement is missing in PutBucketPolicy json document.
If `Statement` field is missing in the json document in `PutBucketPolicy` body, the gateway returns `Missing required field Statement` error description alongside with `MalformedPolicy` error code.
2025-03-31 18:23:33 +04:00
niksis02 90033845ad feat: Implements bucket cors actions in FE to return not implemented.
Implements the bucket cors s3 actions in FE to return `NotImplemented` error.
Actions implemented:
- `PutBucketCors`
- `GetBucketCors`
- `DeleteBucketCors`

`Note`: no logic is implemented for the actions in any backend and no input or output data validation is added.
2025-03-27 20:47:51 +04:00
niksis02 4219d00fd7 fix: Changes the GetUserAccount method behavior in single iam mode to return root user account, if the root user access is provided.
Fixes #977

Changes the `GetUserAccount` method implementation in isma single user mode to return the root account, if the root user account is requested and ErrAdminUserNotFound otherwise. In result the `ChangeBucketOwner` admin api behavior is changed for the root user to be able to change the bucket owner to root in iam single user mode.
2025-03-20 16:23:56 +04:00
niksis02 d82da64942 fix: Fixes the bucket policy json document validation errors for invalid jsons.
Fixes #965

Changes the returned error description to `Policies must be valid JSON and the first byte must be '{'` for invalid bucket policy json documents, which doesn't start with `{`.
The gateway returns `This policy contains invalid Json` error description, if the document starts with `{`, but still isn't valid json.

Implements the `policyErr` string type which implements the `error` interface, to handle the policy json document validation errors, by avoiding staticchecker warnings.
2025-03-20 01:08:47 +04:00
Ben McClelland 22cecea1c0 fix: add err to ldap user/group conversion errors 2025-03-19 08:48:01 -07:00
niksis02 9a02c474b1 fix: Implements a generic wildcard match for bucket policy resources 2025-03-03 20:04:31 +04:00
niksis02 30f3fac4e1 fix: Prioritize explicit deny in bucket policy statements 2025-03-01 01:14:12 +04:00
Ben McClelland a3338dbd34 fix: return default bucket acl if none exists
We were trying to parse a non existing acl and returning an
internal server error due to invalid json acl data.

If the bucket acl does not exist, return a default acl with the
root account as the owner.

This fixes #1060, but does not address the invalid acl format
from s3cmd reported in #963.
2025-02-12 16:38:04 -08:00
Ben McClelland fd4bb8ffbc Merge pull request #1053 from versity/ben/fix_ipa_quoting
Potential fix for code scanning alert no. 15: Potentially unsafe quoting
2025-02-03 09:17:19 -08:00
Nick Jones 86fe01ede0 Update iam_vault.go
clarify error message
2025-02-03 12:50:25 +13:00
Ben McClelland 4add647501 fix: potentially unsafe quoting in ipa iam
CodeQL flagged a possible unsafe quoting in ipa iam code.

Code that constructs a quoted string literal containing user-provided data needs to ensure that this data does not itself contain a quote. Otherwise the embedded data could (accidentally or intentionally) terminate the string literal early and thereby change the structure of the overall string, with potentially severe consequences. If, for example, the string is later used as part of an operating-system command or database query, an attacker may be able to craft input data that injects a malicious command.

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-01-31 16:38:21 -08:00
niksis02 c37a22ffe1 fix: Fixes the AccessControlPolicy Grantee Type unmarshalling, Adds request body validation for the PutBucketAcl action 2025-01-13 23:44:46 +04:00
Ben McClelland fe584a8f63 fix: minor code cleanup for ipa iam
Some minor code cleanup and removal of extra print output.
2025-01-08 16:58:54 -08:00
yhal-nesi ee315276f6 Implement IPA IAM backend (#1005)
feat: FreeIPA IAM implementation

This implements the GetUserAccount() IAM support for accounts stored
within FreeIPA service. This is not implementing any of the account
management functions such as create, update, delete, list IAM accounts,
so is not intended to allow versitygw to manage the IAM accounts within
the FreeIPA service.
---------

Co-authored-by: Yuriy Halytskyy <yuriy.halytskyy@gmail.com>
2025-01-08 14:30:23 -08:00
niksis02 847993a514 fix: Changes GetBucketAcl xml response root element from GetBucketAclOutput to AccessControlPolicy 2024-12-14 18:27:12 +04:00
niksis02 94d23cce9a fix: Return XAdminMethodNotSupported error for single IAM methods. 2024-12-11 23:40:54 +04:00
jonaustin09 c6359a7050 feat: Refactoring admin APIs: changes i/o data transfer encoding to xml, implements traditional aws-like error handling, adds admin role checker middleware. Refactoring admin CLI actions to handle aws-like error responses 2024-10-25 11:40:23 -04:00
Ben McClelland b7a2e8a2c3 fix: unexpected errors during upload races
This fixes the cases for racing uploads with the same object names.
Before we were making some bad assumptions about what would cause
an error when trying to link/rename the final object name into
the namespace, but missed the case that another upload for the
same name could be racing with this upload and causing an incorrect
error.

This also changes the order of setting metadata to prevent
accidental setting of metadata for the current upload to another
racing upload.

This also fix auth.CheckObjectAccess() when objects are removed
while this runs.

Fixes #854
2024-10-07 17:24:44 -07:00
jonaustin09 b0aee40f21 fix: Implemented the AmzDate type to handle iso8601 date parsing and validation. Used it to parse/validate the RetainUntilDate prop in PutObjectRetention data input 2024-10-03 13:13:18 -04:00
jonaustin09 7d368be82e feat: Implemented object locking for object versions 2024-09-30 17:26:49 -04:00
jonaustin09 8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
2024-09-18 13:04:34 -07:00
jonaustin09 23fd0d3fdd fix: Fixed PutBucketAcl action error handling, removed the bucket owner check for all the acl options 2024-08-12 15:27:03 -04:00
jonaustin09 34b7fd6ee7 fix: Added the root user data in the iam services records 2024-08-09 16:14:51 -04:00
jonaustin09 23a40d86a2 fix: Removed the bucket ACL owner check for admin and root users 2024-07-17 09:39:00 -04:00
jonaustin09 828eb93bee fix: Added 'Type' property support in bucket ACL Grantee schema 2024-07-16 18:17:16 -04:00