mirrors/versitygw
1
0
Fork 0
mirror of https://github.com/versity/versitygw.git synced 2026-04-23 06:00:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
665 Commits 29 Branches 50 Tags
b592cfb69d0dfc3e353593d4ea6531f0e4b2abde
Commit Graph

89 Commits

Author SHA1 Message Date
Ben McClelland
b592cfb69d Merge pull request #468 from versity/ben/root_cred_check 
fix: require root credentials be set to start gateway
 2024-03-19 12:49:42 -07:00
Ben McClelland
6e0a3fbce3 Merge pull request #461 from versity/ben/systemd 
feat: add systemd unit support for rpm/deb packaging
 2024-03-19 08:27:22 -07:00
Ben McClelland
388f6b1093 fix: return success exit status if shutdown succeeds 
Fixes #465
 2024-03-18 15:41:25 -07:00
Ben McClelland
1cd86d188f fix: require root credentials be set to start gateway 
Fixes #466
 2024-03-18 15:32:55 -07:00
Ben McClelland
78665dd74a feat: add systemd unit support for rpm/deb packaging 2024-03-18 11:05:41 -07:00
Ben McClelland
8ad9c4834b chore: cleanup top level repo by moving test related dirs to tests 2024-03-10 09:15:22 -07:00
static-moonlight
dcfb10810e Add additional environment variables for configuration properties 
- New environment variables for all parameters which didn't have any
- All new environment variables have the prefix `VGW_`
- Already existing environment variables have been left unchanged
 2024-02-29 23:48:06 +01:00
jonaustin09
94051634a5 feat: Added optional health endpoint in the gateway 2024-02-23 15:08:15 -05:00
jonaustin09
3a07a5b685 feat: Added IAM integration tests 2024-02-22 16:07:53 -05:00
jonaustin09
fa54dfeb9f feat: Added userplus role in IAM, who has the same opportunities as the user, but may also create a bucket 2024-02-20 14:24:26 -05:00
Jon Austin
0c056f935b ListObjectsV2 start-after prop (#388) 
* fix: Fixes #138, Added StartAfter property in ListObjectsV2 action, added couple of integration tests for ListObjectsV2
 2024-02-01 11:04:52 -08:00
jonaustin09
3cce3a5201 feat: Added unit and integration test cases for posix bucket tagging related actions 2024-01-31 10:09:48 -08:00
jonaustin09
0d3a238ceb feat: Implemented logic to add bucket ACL on bucket creation 2024-01-31 09:49:56 -08:00
Ben McClelland
3db43b7206 feat: add azure local env auth 
This is the recommended auth from the following:
https://github.com/Azure-Samples/storage-blobs-go-quickstart/blob/master/storage-quickstart.go
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-go?toc=%2Fazure%2Fdeveloper%2Fgo%2Ftoc.json&bc=%2Fazure%2Fdeveloper%2Fgo%2Fbreadcrumb%2Ftoc.json&tabs=roles-azure-portal#authenticate-to-azure-and-authorize-access-to-blob-data
 2024-01-09 22:21:39 -08:00
jonaustin09
e5fc12042b feat: Added sas token authentication for azure backend 2024-01-09 22:03:13 -08:00
jonaustin09
8fc16392d1 feat: Dockerized azure backend to run 2 images: one for azurite, one for azure backend 2024-01-08 10:07:50 -08:00
jonaustin09
8cc89fa713 feat: Azure backend implementation 2023-12-29 21:55:32 -08:00
Ben McClelland
3b945f72fc feat: azure blob backend initial pass 2023-12-29 21:54:56 -08:00
Ben McClelland
6146dcff4a Merge pull request #358 from versity/ben/gw_perf 
fix: disable keepalive for better request rates under heavy load
 2023-12-26 10:28:46 -08:00
Ben McClelland
3ba218bd9a Merge pull request #357 from versity/ben/quiet_logging 
feat: add quiet option to silence request log output
 2023-12-26 10:28:30 -08:00
Ben McClelland
60bc9a3fc5 Merge pull request #354 from versity/ben/req_body_limit 
fix: body limit no longer needed for streaming request body
 2023-12-26 10:27:44 -08:00
Ben McClelland
216e50b9fd fix: disable keepalive for better request rates under heavy load 2023-12-23 22:56:49 -08:00
Ben McClelland
d47cbcb39f feat: add quiet option to silence request log output 2023-12-23 16:52:56 -08:00
Ben McClelland
43bfe8a869 fix: body limit no longer needed for streaming request body 2023-12-22 15:24:23 -08:00
Ben McClelland
6e37096b35 fix: s3 backend performance increase with client reuse 
The previous way of initializing the s3 client in each call was
adding a lot of overhead and would tank performance beyond about
20 simultaneous requests.

Since the backend access is through a single account, we can init
and store this client for use from each api call.
 2023-12-22 15:09:44 -08:00
Ben McClelland
3d6e37bbb1 fix: make iam s3 object work similar to internal files 2023-12-18 10:39:20 -08:00
Ben McClelland
ab43240b4e fix: add cli options to enable s3 iam service 2023-12-18 08:29:04 -08:00
jonaustin09
27eb43d089 feat: Closes #290, implemented request body stream reading for PutObject and UploadPart actions. 2023-12-14 18:27:51 -08:00
Jon Austin
e9f01c8cce S3 proxy ListParts max parts issue (#338) 
* feat: implemented the logic to run integration tests separately

* fix: Resolved tests closer bug

* fix: Fixes #329, Fixed ListParts max-parts property issue

* fix: removed max-parts max int value check
 2023-12-13 10:22:34 -08:00
Jon Austin
68073b9b73 Docker proxy fix (#337) 
* feat: implemented the logic to run integration tests separately

* fix: Fixed docker proxy command due to latest changes
 2023-12-12 16:06:55 -08:00
jonaustin09
056c905a65 fix: Closes #323, fixed s3 proxy single user PutBucketAcl issue 2023-12-11 11:05:08 -08:00
jonaustin09
ef5a94420c feat: Created admin CLI actions in s3 proxy, Created iam proxy for proxy server 2023-12-11 11:05:08 -08:00
jonaustin09
c5007a68aa eat: Closes #317, Changed s3 proxy behavior to run in single user mode with CLI provided credentials 2023-12-04 10:06:59 -05:00
Ben McClelland
26a7d567b0 feat: allow integrations tests to be called as unit tests 
This reconfigures the integration tests a bit to be called as
unit tests. This makes debugging failing tests a little easier.
 2023-11-29 09:53:17 -08:00
jonaustin09
f2c02c6362 feat: Added gateway throughput performance test, got upload and download tests separated 2023-11-16 15:56:38 -05:00
jonaustin09
920b4945cd feat: Closes #236, Added 3 optional fields in iam user account, UserID, GroupID, ProjectID 2023-10-12 13:24:34 -04:00
Ben McClelland
f58646b58d feat: s3proxy backend 
This backend redirects incoming requests to another s3 service.
This will use the incoming credentials to setup the client
requests to the external s3 service. So the IAM accounts (or
root account) must match what the external s3 service expects.
 2023-10-11 09:00:02 -07:00
jonaustin09
52674ab0c5 feat: Closes #282, created a new integration test group and test cases for posix specific behaviours testing 2023-10-11 08:17:10 -07:00
Ben McClelland
ae0b270c2c feat: move local iam cache to a more generic cache mechanism 
The local IAM accounts were being cached in memory for improved
performance, but this can be moved up a layer so that the cache
can benefit any configured IAM service.

This adds options to disable and tune TTL for cache. The balance
for the TTL is that a longer life will send requests to the IAM
service less frequently, but could be out of date with the service
accounts for that duration.
 2023-10-09 08:15:56 -07:00
jonaustin09
8f7a1bfc86 feat: Integrated a new option for IAM servcie: store IAM data in LDAP server 2023-10-03 14:02:21 -04:00
Ben McClelland
ae4e382e61 feat: refactor internal iam service 
This moves the internal iam service from the posix backend so
that we can start implementing new iam services right in the auth
module.

The internal iam service has same behavior as before, but now
must be enabled with the --iam-dir cli option.

New single user service is the default when no other iam service
is selected. This just runs the gateway in single user mode with
just the root account.
 2023-10-02 11:12:18 -07:00
Ben McClelland
c9475adb04 fix: remove body limit for admin app 
The BodyLimit is needed for large PUTs in the s3 api server, but
the admin server requests are not expected to exceed the default
4MB limit.
 2023-09-12 16:16:47 -07:00
jonaustin09
5ab38e3dab feat: Closes #232, Added an option to run admin server in a different network, by specifying admin server address/ip 2023-09-12 16:04:31 -07:00
Ben McClelland
c04f6d7f00 Merge pull request #233 from versity/admin/list-buckets 
Issue 217, Admin API and CLI action to list all the buckets and its owners.
 2023-09-12 16:02:29 -07:00
jonaustin09
6ac69b3198 feat: Closes #217, Created an admin API and CLI action to list all the buckets and its owners as a table 2023-09-12 08:29:34 -04:00
Ben McClelland
145c2dd4e3 fix: builds for non 64 bit linux arch 
The scoutfs backend is only supported on 64bit linux.  This corrects
the build constraints to only supported linux arch, and prevents the
incompatible import for unspported arch.

We also need to adjust the body limit setting on 32 bit since this
is an int, and our default limit will overfow on 32 bit.
 2023-09-10 20:49:56 -07:00
Ben McClelland
6ab4090216 fix: add ADMIN_ENDPOINT_URL env var to admin cli 2023-09-07 13:17:54 -07:00
Ben McClelland
ef92f57e7d feat: format admin cli list-users output in table 
Format list-users output in an easier to read table:
% versitygw admin list-users
Account  Role
-------  ----
myadmin  admin
myuser   user
 2023-09-07 08:48:18 -07:00
jonaustin09
7620651a49 fix: Merge conflicts resolved with main 2023-09-06 17:45:23 -04:00
jonaustin09
4c7584c99f feat: Closes #206, Added an admin api endpoint and a CLI action to change buckets owner 2023-09-06 17:41:47 -04:00