Commit Graph

124 Commits

Author SHA1 Message Date
jonaustin09
768983be34 fix: Added VersioningNotConfigured error in Put/GetBucketVersioning acitons 2024-10-04 20:52:31 -04:00
Ben McClelland
3a9cbfcbd6 fix: azure tests and add azure ci test 2024-09-23 09:10:52 -07:00
jonaustin09
6d4ff09d6f feat: Added integration tests for bucket object versioning. Made a couple of bug fixes in the versioning implementation 2024-09-19 13:29:14 -07:00
jonaustin09
8252ecd452 feat: basic logic implementation of bucket object versioning in posix backend
New posix backend option --versioning-dir will enable storing object versions
in specified directory.
2024-09-18 13:04:34 -07:00
Ben McClelland
923ee5f0db fix: update help description to reference repo 2024-08-28 19:00:46 -07:00
Ben McClelland
810bf01871 feat: change startup banner to versitygw version
This changes the startup banner to report the versitygw version
and build info along with interfaces configured for admin and
s3 services when quiet option not enabled.

Fixes #728
2024-08-28 10:50:12 -07:00
Ben McClelland
296a78ed56 feat: enable ipv6 support for listening socket
Fiber allows for dual stack ipv4/ipv6 by setting Network setting to
fiber.NetworkTCP. The default is fiber.NetworkTCP4 which is ipv4
only because the dual stack is not compatible with prefork. But
we do not use prefork, so it is fine to enable the dual ipv4/ipv6
support.
2024-08-22 13:46:06 -07:00
jonaustin09
34b7fd6ee7 fix: Added the root user data in the iam services records 2024-08-09 16:14:51 -04:00
Ben McClelland
3361391506 Merge pull request #674 from versity/admin-api-access-logs
Admin APIs access logs
2024-07-16 08:47:19 -07:00
jonaustin09
ddd048495a feat: Implemented server access logs with file for Admin APIs 2024-07-15 15:49:03 -04:00
Ben McClelland
f6dd2f947c feat: add option to allow symlinked directories as buckets
This adds the ability to treat symlinks to directories at the top
level gateway directory as buckets the same as normal directories.

This could be a potential security issue allowing traversal into
other filesystems within the system, so is defaulted to off. This
can be enabled when specifically needed for both posix and scoutfs
backend systems.

Fixes #644
2024-07-13 10:21:15 -07:00
jonaustin09
1808335381 feat: Added admin api and CLI command for updating gateway users attributes 2024-06-20 08:13:10 -07:00
jonaustin09
557a8b683a feat: iam service hashicorp vault
Use Vault as an IAM service. This is intended to be managed through
the versitygw admin commands similar to the internal iam service.
This uses the kv-v2 key/value secrets storage, and uses access key
for the key and stores the JSON serialized account data as the value.

This currently only supports roleid/rolesecret or root token
authentication methods to Vault.
2024-06-09 09:10:35 -07:00
Ben McClelland
b7cc7feffa fix: use json.MarshalIndent to format event_config.json
This adds indent fomratting to the generated event_config.json
for easier reading/editing.
2024-05-31 10:22:25 -07:00
Ben McClelland
1d9f272ce1 fix: remove unused project id in account info
The intent was to have a project id that could be set along with
user and group ids for new files/objects in the backend. However,
most filesystems don't actually associate a project with a user,
and instead have the project id inherited from parent directories.

Let's remove the project id for now, and we can always bring it
back if we have a backend that will be able to make use of it.
2024-05-29 21:29:16 -07:00
Ben McClelland
e48d3c7463 Merge pull request #592 from versity/ben/delete_user_msg
fix: correct error message for delete user
2024-05-29 13:37:21 -07:00
Ben McClelland
f4e0d6ae62 fix: correct error message for delete user 2024-05-28 15:59:10 -07:00
Ben McClelland
bdef050231 feat: add dogstats to metrics manager 2024-05-28 15:46:40 -07:00
Ben McClelland
50541e0921 feat: remove unused gauge metrics and add service name option 2024-05-28 15:46:40 -07:00
jonaustin09
be6f9a86cd feat: Integrated metric manager into the gateway 2024-05-28 12:50:50 -07:00
jonaustin09
2a2f9c827c feat: Closes #484. Added support to run the gateway on read only mode 2024-05-06 16:41:39 -04:00
jonaustin09
6f9c6fde37 feat: Added DeleteObjects event support in bucket event notifications 2024-04-25 16:18:02 -04:00
Ben McClelland
4429570388 fix: use xattr.ENOATTR check for posix xattrs
The xattr package has a more universal error type for xattrs
not existing. Use this for better platform compatibility.

This also adds the xattr.XATTR_SUPPORTED check for platform
xattr suport in xattr package.

Fixes #527
2024-04-18 18:20:43 -07:00
Ben McClelland
c81403fe90 feat: add metadata storage abstraction layer
Closes #511. This adds an abstraction layer to the metadata
storage to allow for future non-xattr metadata storage
implementations.
2024-04-15 13:57:31 -07:00
Ben McClelland
5ec2de544c fix: admin change-bucket-owner return status 2024-04-11 16:11:59 -07:00
Ben McClelland
53a50df742 fix: admin change-bucket-owner cert disable verify 2024-04-11 14:44:37 -07:00
Ben McClelland
ffe1fc4ad3 feat: optional disable cert check for admin cli actions
Fixes #499. Allows running admin cli commands against servers
with self signed certs.
2024-04-09 08:37:11 -07:00
Ben McClelland
17b1dbe025 fix: return non 0 exit status for cli admin error
Fixes #505. This returns the body as an error when the http status
for the admin request is non-success.
2024-04-08 17:29:02 -07:00
jonaustin09
dbc0ad4325 feat: Closes #475, Implemented filters for s3 bucket event notifications, created a utility CLI command to create config file 2024-04-04 13:25:01 -07:00
Ben McClelland
4265270e4d Merge pull request #488 from versity/event-notif-webhook
Bucket event notifications with webhook URL
2024-04-02 15:58:02 -07:00
jonaustin09
81d6635fe9 feat: Adeed webhook URL support for bucket event notifications. Made some bug fixing and refactoring in event sender and audit logger interfaces 2024-04-02 15:17:36 -04:00
Ben McClelland
c228bbfd79 feat: add option to change ownership of dir/files to acct settings
When enabled, any new directories or files created through the
gateway will change ownership based on the account uid/gid.

Fixes #238.
2024-03-30 22:26:29 -07:00
Ben McClelland
fcf0f4cf68 Merge pull request #480 from versity/access-control-tests
Access control integration tests
2024-03-28 12:28:53 -07:00
jonaustin09
e6203c5765 feat: Closes #441, Added access control integration tests, fixed some bugs in bucket policy and acl access checking flow 2024-03-28 14:52:56 -04:00
Ben McClelland
5b30db9e48 feat: add optional pprof debug endpoint
Fixes #359. This adds the pprof listening endpoint when configured.
The option requires providing the listening port. Once enabled,
pprof debug utilities are provided at this endpoint.

For example, adding to following option:
--pprof 127.0.0.1:9999
Creates a listener on localhost port 9999. You can then point a
browser to http://localhost:9999/debug/pprof/ to get access
to the debug utilities.

Another useful case is to get goroutine stack traces live with
the following:
curl 'http://localhost:9999/debug/pprof/goroutine?debug=1'
2024-03-27 11:48:11 -07:00
Ben McClelland
b592cfb69d Merge pull request #468 from versity/ben/root_cred_check
fix: require root credentials be set to start gateway
2024-03-19 12:49:42 -07:00
Ben McClelland
6e0a3fbce3 Merge pull request #461 from versity/ben/systemd
feat: add systemd unit support for rpm/deb packaging
2024-03-19 08:27:22 -07:00
Ben McClelland
388f6b1093 fix: return success exit status if shutdown succeeds
Fixes #465
2024-03-18 15:41:25 -07:00
Ben McClelland
1cd86d188f fix: require root credentials be set to start gateway
Fixes #466
2024-03-18 15:32:55 -07:00
Ben McClelland
78665dd74a feat: add systemd unit support for rpm/deb packaging 2024-03-18 11:05:41 -07:00
Ben McClelland
8ad9c4834b chore: cleanup top level repo by moving test related dirs to tests 2024-03-10 09:15:22 -07:00
static-moonlight
dcfb10810e Add additional environment variables for configuration properties
- New environment variables for all parameters which didn't have any
- All new environment variables have the prefix `VGW_`
- Already existing environment variables have been left unchanged
2024-02-29 23:48:06 +01:00
jonaustin09
94051634a5 feat: Added optional health endpoint in the gateway 2024-02-23 15:08:15 -05:00
jonaustin09
3a07a5b685 feat: Added IAM integration tests 2024-02-22 16:07:53 -05:00
jonaustin09
fa54dfeb9f feat: Added userplus role in IAM, who has the same opportunities as the user, but may also create a bucket 2024-02-20 14:24:26 -05:00
Jon Austin
0c056f935b ListObjectsV2 start-after prop (#388)
* fix: Fixes #138, Added StartAfter property in ListObjectsV2 action, added couple of integration tests for ListObjectsV2
2024-02-01 11:04:52 -08:00
jonaustin09
3cce3a5201 feat: Added unit and integration test cases for posix bucket tagging related actions 2024-01-31 10:09:48 -08:00
jonaustin09
0d3a238ceb feat: Implemented logic to add bucket ACL on bucket creation 2024-01-31 09:49:56 -08:00
Ben McClelland
3db43b7206 feat: add azure local env auth
This is the recommended auth from the following:
https://github.com/Azure-Samples/storage-blobs-go-quickstart/blob/master/storage-quickstart.go
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-go?toc=%2Fazure%2Fdeveloper%2Fgo%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fgo%2Fbreadcrumb%2Ftoc.json&tabs=roles-azure-portal#authenticate-to-azure-and-authorize-access-to-blob-data
2024-01-09 22:21:39 -08:00
jonaustin09
e5fc12042b feat: Added sas token authentication for azure backend 2024-01-09 22:03:13 -08:00