Commit Graph

123 Commits

Author SHA1 Message Date
Ben McClelland
0bf49872cf Merge pull request #1507 from versity/ben/get-object-overrides
feat: add response header overrides for GetObject
2025-09-01 14:17:28 -07:00
Ben McClelland
8cad7fd6d9 feat: add response header overrides for GetObject
GetObject allows overriding response headers with the following
paramters:
response-cache-control
response-content-disposition
response-content-encoding
response-content-language
response-content-type
response-expires

This is only valid for signed (and pre-singed) requests. An error
is returned for anonymous requests if these are set.

More info on the GetObject overrides can be found in the GetObject
API reference.

This also clarifies the naming of the AccessOptions IsPublicBucket
to IsPublicRequest to indicate this is a public access request
and not just accessing a bucket that allows public access.

Fixes #1501
2025-08-30 14:13:20 -07:00
Ben McClelland
58117c011a feat: add get bucket location frontend handlers
GetBucketLocation is being deprecated by AWS, but is still used
by some clients. We don't need any backend handlers for this since
the region is managed by the frontend. All we need is to test for
bucket existence, so we can use HeadBucket for this.

Fixes #1499
2025-08-30 12:29:26 -07:00
tannevaled
fbde51b3ea be able to debug LDAP queries; be consistent between GetUserAccount() and ListUserAccounts() on how to build the search filters; objectClasses were missing in GetUserAccount research filter leading to a bad result for example when a posixgGroup have the same name as a posixUser. 2025-08-29 10:50:08 +02:00
niksis02
843620235b fix: adds full wildcard and any character match for bucket policy actions
Fixes #1488

Adds full wildcard (`*`) and single-character (`?`) support for bucket policy actions, fixes resource detection with wildcards, and includes unit tests for `bucket_policy_actions`, `bucket_policy_effect`, and `bucket_policy_principals`.
2025-08-27 20:44:30 +04:00
niksis02
3d20a63f75 fix: adds Acces-Control-Allow-Headers to cors responses
Fixes #1486

* Adds the `Access-Control-Allow-Headers` response header to CORS responses for both **OPTIONS preflight requests** and any request containing an `Origin` header.
* The `Access-Control-Allow-Headers` response includes only the headers specified in the `Access-Control-Request-Headers` request header, always returned in lowercase.
* Fixes an issue with allow headers comparison in cors evaluation by making it case-insensitive.
* Adds missing unit tests for the **OPTIONS controller**.
2025-08-27 00:31:47 +04:00
Ben McClelland
9772badd43 Merge pull request #1473 from versity/ben/ldap-reconnect
fix: iam ldap reconnect after network disconnects
2025-08-25 13:58:25 -07:00
niksis02
d90944afd1 feat: implementes GetBucketPolicyStatus s3 action
Closes #1454

Adds the implementation of [S3 GetBucketPolicyStatus action](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketPolicyStatus.html). The implementation goes to front-end. Front-End loads the bucket policy and checks if it grants public access to all users.

A bucket policy document `is public` only when `Principal` contains `*`(all users): only when it grants access to `ALL` users.
2025-08-25 21:48:06 +04:00
niksis02
14a2984d59 feat: adds not implemented routes for bucket website actions
Closes #1450

Adds `NotImplemented` routes for bucket website S3 actions:
- `PutBucketWebsite`
- `GetBucketWebsite`
- `DeleteBucketWebsite`
2025-08-22 19:56:51 +04:00
niksis02
0895ada9ed feat: adds not implemented routes for bucket accelerate configuration actions
Closes #1452

Adds `NotImplemented` routes for bucket accelerate configuration S3 actions:
- `PutBucketAccelerateConfiguration`
- `GetBucketAccelerateConfiguration`
2025-08-22 14:45:42 +04:00
niksis02
d784c0a841 feat: adds not implemented routes for bucket notification configuration actions
Closes #1453

Adds `NotImplemented` routes for bucket notification configuration S3 actions:
- `PutBucketNotificationConfiguration`
- `GetBucketNotificationConfiguration`
2025-08-21 20:40:18 +04:00
niksis02
be79fc249d feat: adds not implemented routes for bucket public access block actions
Closes #1451

Adds `NotImplemented` routes for bucket public access block S3 actions:
- `PutPublicAccessBlock`
- `GetPublicAccessBlock`
- `DeletePublicAccessBlock`
2025-08-21 20:10:29 +04:00
niksis02
88f84bfd89 feat: adds not implemented routes for bucket replication actions
Closes #1449

Adds `NotImplemented` routes for bucket replication S3 actions:
- `PutBucketReplication`
- `GetBucketReplication`
- `DeleteBucketReplication`

Adds missing actions in metrics `ActionMap`
2025-08-21 16:44:29 +04:00
niksis02
45a1f7ae7c feat: adds not implemented routes for bucket metrics configuration actions
Closes #1445

Adds `NotImplemented` routes for bucket metrics configuration S3 actions:
- `PutBucketMetricsConfiguration`
- `GetBucketMetricsConfiguration`
- `ListBucketMetricsConfigurations`
- `DeleteBucketMetricsConfiguration`

Adds the missing bucket actions to `supportedActionList` in bucket policy supported actions list.
2025-08-21 16:05:06 +04:00
Ben McClelland
be1708b1df Merge pull request #1476 from versity/sis/bucket-request-payment-actions-not-implemented
feat: adds not implemented routes for bucket request payment actions
2025-08-20 17:10:54 -07:00
Ben McClelland
617ad0fd31 Merge pull request #1475 from versity/sis/bucket-logging-actions-not-implemented
feat: adds not implemented routes for bucket logging actions
2025-08-20 17:10:32 -07:00
Ben McClelland
3e4c31f14a Merge pull request #1474 from versity/sis/bucket-lifecycle-configuration-actions-not-implemented
feat: adds not implemented routes for bucket lifecycle configuration actions
2025-08-20 17:09:59 -07:00
niksis02
6b450a5c11 feat: adds not implemented routes for bucket request payment actions
Closes #1455

Adds `NotImplemented` routes for bucket request payment S3 actions:
- `PutBucketRequestPayment`
- `GetBucketRequestPayment`
2025-08-21 00:54:31 +04:00
niksis02
5f28a7449e feat: adds not implemented routes for bucket logging actions
Closes #1444

Adds `NotImplemented` routes for bucket logging S3 actions:
- `PutBucketLogging`
- `GetBucketLogging`
2025-08-20 21:07:09 +04:00
niksis02
025b0ee3c8 feat: adds not implemented routes for bucket lifecycle configuration actions
Closes #1443

Adds `NotImplemented` routes for bucket lifecycle configuration S3 actions.
- `PutBucketLifecycleConfiguration`
- `GetBucketLifecycleConfiguration`
- `DeleteBucketLifecycle`
2025-08-20 20:48:58 +04:00
niksis02
09031a30e5 feat: bucket cors implementation
Closes #1003

**Changes Introduced:**

1. **S3 Bucket CORS Actions**

   * Implemented the following S3 bucket CORS APIs:

     * `PutBucketCors` – Configure CORS rules for a bucket.
     * `GetBucketCors` – Retrieve the current CORS configuration for a bucket.
     * `DeleteBucketCors` – Remove CORS configuration from a bucket.

2. **CORS Preflight Handling**

   * Added an `OPTIONS` endpoint to handle browser preflight requests.
   * The endpoint evaluates incoming requests against bucket CORS rules and returns the appropriate `Access-Control-*` headers.

3. **CORS Middleware**

   * Implemented middleware that:

     * Checks if a bucket has CORS configured.
     * Detects the `Origin` header in the request.
     * Adds the necessary `Access-Control-*` headers to the response when the request matches the bucket CORS configuration.
2025-08-20 20:45:09 +04:00
Ben McClelland
dafe099d9b fix: iam ldap reconnect after network disconnects
Handle LDAP connection failures by attempting to reconnect.
This should resolve the issue of connections being closed by
the LDAP server after a period of inactivity.

Fixes #1464
2025-08-19 18:17:12 -07:00
niksis02
24b88e20e0 feat: adds not implemented routes for bucket inventory configuration actions
Closes #1440

Adds `NotImplemented` routes for bucket inventory configuration S3 actions:
- `PutBucketInventoryConfiguration`
- `GetBucketInventoryConfiguration`
- `ListBucketInventoryConfigurations`
- `DeleteBucketInventoryConfiguration`
2025-08-19 21:49:38 +04:00
niksis02
cdccdcc4d6 feat: adds not implemented routes for bucket intelligent tiering actions
Closes #1440

Adds `NotImplemented` routes for intelligent tiering S3 actions:
- `PutBucketIntelligentTieringConfiguration`
- `GetBucketIntelligentTieringConfiguration`
- `ListBucketIntelligentTieringConfigurations`
- `DeleteBucketIntelligentTieringConfiguration`
2025-08-19 21:23:05 +04:00
niksis02
ed92ad3daa feat: adds not implemented routes for bucket ecryption actions
Closes #1439

Adds `NotImplemented` routes for bucket encryption S3 actions:

- `PutBucketEncryption`
- `GetBucketEncryption`
- `DeleteBucketEncryption`
2025-08-19 20:30:02 +04:00
niksis02
8db196634b fix: adds not implemented routes for bucket analytics s3 actions.
Fixes #1433
Fixes #1437
Fixes #1438

Adds 4 routes to return `NotImplemented` for bucket analytics `S3` actions:

- `PutBucketAnalyticsConfiguration`
- `GetBucketAnalyticsConfiguration`
- `DeleteBucketAnalyticsConfiguration`
- `ListBucketAnalyticsConfiguration`
2025-08-19 02:14:31 +04:00
niksis02
e5850ff11f feat: adds copy source validation for x-amz-copy-source header.
Fixes #1388
Fixes #1389
Fixes #1390
Fixes #1401

Adds the `x-amz-copy-source` header validation for `CopyObject` and `UploadPartCopy` in front-end.
The error:
```
	ErrInvalidCopySource: {
		Code:           "InvalidArgument",
		Description:    "Copy Source must mention the source bucket and key: sourcebucket/sourcekey.",
		HTTPStatusCode: http.StatusBadRequest,
	},
```
is now deprecated.

The conditional read/write headers validation in `CopyObject` should come with #821 and #822.
2025-07-22 14:40:11 -07:00
niksis02
7f9ab35347 feat: implements unit tests for object PUT controllers 2025-07-22 20:55:22 +04:00
niksis02
866b07b98f feat: implementes unit tests for all the bucket action controllers. 2025-07-22 20:55:22 +04:00
niksis02
5be9e3bd1e feat: a total refactoring of the gateway middlewares by lowering them from server to router level. 2025-07-22 20:55:22 +04:00
Ben McClelland
08ccf821f9 fix: refresh expired iam vault tokens when needed
The IAM vault client stores an access token once authenticated,
but this token will expire after a certain amount of time set
by the server generating the token. Once this token is expired
or revoked, it can no longer be use by the vault client. So
the client should try to refresh the token with any errors
indicating expired or revoked tokens.

Fixes #976
2025-07-17 09:32:40 -07:00
Maksim Loviagin
e39ab6f0ee feat: split the vault mount path into kv and auth 2025-07-15 18:57:44 +00:00
Ben McClelland
f295df2217 fix: add new auth method to update ownership within acl
Add helper util auth.UpdateBucketACLOwner() that sets new
default ACL based on new owner and removes old bucket policy.

The ChangeBucketOwner() remains in the backend.Backend
interface in case there is ever a backend that needs to manage
ownership in some other way than with bucket ACLs. The arguments
are changing to clarify the updated owner. This will break any
plugins implementing the old interface. They should use the new
auth.UpdateBucketACLOwner() or implement the corresponding
change specific for the backend.
2025-07-09 16:16:34 -07:00
Ben McClelland
91b904d10f fix: add retry for iam freeipa http requests
The IPA service connections have been seen to not always work
correctly on the first network connection attempt. Add retry
logic for errors that appear to be transient network issues.
2025-07-07 22:28:58 -07:00
niksis02
458db64e2d feat: implements public bucket access.
This implementation introduces **public buckets**, which are accessible without signature-based authentication.

There are two ways to grant public access to a bucket:

* **Bucket ACLs**
* **Bucket Policies**

Only `Get` and `List` operations are permitted on public buckets. All **write operations** require authentication, regardless of whether public access is granted through an ACL or a policy.

The implementation includes an `AuthorizePublicBucketAccess` middleware, which checks if public access has been granted to the bucket. If so, authentication middlewares are skipped. For unauthenticated requests, appropriate errors are returned based on the specific S3 action.

---

**1. Bucket-Level Operations:**

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test"
    }
  ]
}
```

**2. Object-Level Operations:**

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test/*"
    }
  ]
}
```

**3. Both Bucket and Object-Level Operations:**

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test"
    },
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": "arn:aws:s3:::test/*"
    }
  ]
}
```

---

```sh
aws s3api create-bucket --bucket test --object-ownership BucketOwnerPreferred
aws s3api put-bucket-acl --bucket test --acl public-read
```
2025-07-02 00:11:10 +04:00
Yuriy Halytskyy
925f89465e fix: IPA IAM use http proxy from environment 2025-05-20 09:33:10 -07:00
niksis02
4334f869f2 feat: makes the user role editable in /update-user iam endpoint
Closes #1295

Makes the user `role` mutable in /update-user admin endpoint.
Integrates the changes in the `admin update-user` cli command, by adding the `role` flag for a user role modification.
2025-05-14 23:10:15 +04:00
Ben McClelland
e98f7763d0 fix: use createtemp()/rename() for iam internal files
This cleans up a previous fix to #630 to use a better temp/rename
scheme thats less likely to have bad side effects.

The test for the previous issue still passes these cases, and we
will be less liekly to find a case where the file doesnt exist
or corrpted backup files.
2025-05-03 12:39:06 -07:00
Ben McClelland
eb72905401 Merge pull request #1200 from versity/sis/obj-lock-actions-invalid-body
fix: Changes the error type to MalformedXML for PutObjectRetention and PutObjectLegalHold empty or invalid body
2025-04-08 20:55:15 -07:00
niksis02
20d00f7f6d fix: Changes the error type to MalformedXML for PutObjectRetention and PutObjectLegalHold empty or invalid body
Fixes #1185
Fixes #1191

`PutObjectLegalHold` and `PutObjectRetention` should return `MalformedXML` if the request body is empty or invalid.`
2025-04-08 19:01:00 +04:00
niksis02
2b7a71568c fix: Changes the GetObjectLegalHold result root tag to LegalHold
Fixes #1193

Changes the xml root tag element to `LegalHold` in `GetObjectLegalHold` response.
2025-04-08 18:26:26 +04:00
niksis02
30f319bc92 fix: Hanldes the case when Statement is missing in PutBucketPolicy json document.
If `Statement` field is missing in the json document in `PutBucketPolicy` body, the gateway returns `Missing required field Statement` error description alongside with `MalformedPolicy` error code.
2025-03-31 18:23:33 +04:00
niksis02
90033845ad feat: Implements bucket cors actions in FE to return not implemented.
Implements the bucket cors s3 actions in FE to return `NotImplemented` error.
Actions implemented:
- `PutBucketCors`
- `GetBucketCors`
- `DeleteBucketCors`

`Note`: no logic is implemented for the actions in any backend and no input or output data validation is added.
2025-03-27 20:47:51 +04:00
niksis02
4219d00fd7 fix: Changes the GetUserAccount method behavior in single iam mode to return root user account, if the root user access is provided.
Fixes #977

Changes the `GetUserAccount` method implementation in isma single user mode to return the root account, if the root user account is requested and ErrAdminUserNotFound otherwise. In result the `ChangeBucketOwner` admin api behavior is changed for the root user to be able to change the bucket owner to root in iam single user mode.
2025-03-20 16:23:56 +04:00
niksis02
d82da64942 fix: Fixes the bucket policy json document validation errors for invalid jsons.
Fixes #965

Changes the returned error description to `Policies must be valid JSON and the first byte must be '{'` for invalid bucket policy json documents, which doesn't start with `{`.
The gateway returns `This policy contains invalid Json` error description, if the document starts with `{`, but still isn't valid json.

Implements the `policyErr` string type which implements the `error` interface, to handle the policy json document validation errors, by avoiding staticchecker warnings.
2025-03-20 01:08:47 +04:00
Ben McClelland
22cecea1c0 fix: add err to ldap user/group conversion errors 2025-03-19 08:48:01 -07:00
niksis02
9a02c474b1 fix: Implements a generic wildcard match for bucket policy resources 2025-03-03 20:04:31 +04:00
niksis02
30f3fac4e1 fix: Prioritize explicit deny in bucket policy statements 2025-03-01 01:14:12 +04:00
Ben McClelland
a3338dbd34 fix: return default bucket acl if none exists
We were trying to parse a non existing acl and returning an
internal server error due to invalid json acl data.

If the bucket acl does not exist, return a default acl with the
root account as the owner.

This fixes #1060, but does not address the invalid acl format
from s3cmd reported in #963.
2025-02-12 16:38:04 -08:00
Ben McClelland
fd4bb8ffbc Merge pull request #1053 from versity/ben/fix_ipa_quoting
Potential fix for code scanning alert no. 15: Potentially unsafe quoting
2025-02-03 09:17:19 -08:00