Added missing debug logs in the `front-end` and `utility` functions.
Enhanced debug logging with the following improvements:
- Each debug message is now prefixed with [DEBUG] and appears in color.
- The full request URL is printed at the beginning of each debug log block.
- Request/response details are wrapped in framed sections for better readability.
- Headers are displayed in a colored box.
- XML request/response bodies are pretty-printed with indentation and color.
O_TMPFILE can fail if the location we need to link the final
file is not within the same filesystem. This can happen if
there are different filesystem mounts within a bucket or if
using zfs nested datasets within a bucket.
Fixes#1194Fixes#1035
We had put the error handling in for the read only filesystems
when O_TMPFILE is supported, but missed the CreateTemp() fallback
case. This fixes this case to also return the method not allowed
error.
This also adds the error handling for the scoutfs case as well.
Fixes#1195
Fixes#1182
S3 calculates the `CRC64NVME` checksum of an object on object upload(`PutObject`), when no checksum algorithm or precalculated checksum header is provided.
Makes the `CRC64NVME` checksum as default for `PutObject`, when no checksum is provided.
Fixes#1181
`DeleteObjects` should remove non-empty directory objects, which has been uploaded as a separate object.
e.g
Upload -> `foo/bar`
Upload -> `foo/`
Delete -> `foo/`
The last action call should succeed.
The PR introduces changes which removes `ETag` from the directory object attempting to `delete`, which has been uploaded as a separate object.
Closes#819
ListObjects returns object owner data in each object entity in the result, while ListObjectsV2 has fetch-owner query param, which indicates if the objects owner data should be fetched.
Adds these changes in the gateway to add `Owner` data in `ListObjects` and `ListObjectsV2` result. In aws the objects can be owned by different users in the same bucket. In the gateway all the objects are owned by the bucket owner.
Fixes#1021
`foo` and `foo/` object paths were considered as the same in `CopyObject` source and destination object paths in posix.
Implements the `joinPathWithTrailer` function, which calls `filepath.Join` and adds trailing `/` if it existed in the original path. This way the implementation puts separation
between directory and file objects with the same name.
Fixes#998Closes#1125Closes#1126Closes#1127
Implements objects meta properties(Content-Disposition, Content-Language, Content-Encoding, Cache-Control, Expires) and tagging besed on the directives(metadata, tagging) in CopyObject in posix and azure backends. The properties/tagging should be coppied from the source object if "COPY" directive is provided and it should be replaced otherwise.
Changes the object copy principle in azure: instead of using the `CopyFromURL` method from azure sdk, it first loads the object then creates one, to be able to compare and store the meta properties.
This syncs recent updates to posix for scoutfs backend including
the extra metadata such as Content-Disposition,
Content-Language, Cache-Control and Expires. This also fixes the
directory object listings that have a double trailing slash due
to the change in the backend.Walk().
This also simplifies head-object to call the posix on and then
post process for glacier changes. This allows keeping in closer
sync with posix head-object over time.
Closes#1128
Adds `Content-Disposition`, `Content-Language`, `Cache-Control` and `Expires` object meta properties support in posix and azure backends.
Changes the `PutObject` and `CreateMultipartUpload` actions backend input type to custom `s3response` types to be able to store `Expires` as any string.
Fixes#1004Fixes#1122Fixes#1120
Separates `GetObject` and `UploadPartCopy` range parsing/validation.
`GetObject` returns a successful response if acceptRange is invalid.
Adjusts the range upper limit, if it exceeds the actual objects size for `GetObject`.
Corrects the `ContentRange` in the `GetObject` response.
Fixes the `UploadPartCopy` action copy source range parsing/validation.
`UploadPartCopy` returns `InvalidArgument` if the copy source range is not valid.
Instead of an internal server error, we should be returning
method not allowed when trying to upload to a read only
filesystem or make other modifications that are expected to
fail. This will give clearer feedback to the clients
that this is not expected to work.
Fixes#1062
- extract method to find top-level dirs, optionally including links to
dirs
- rework ListBuckets and ListBucketsAndOwners to use FileInfo rather
than DirEntry
Fixes#1081
This adds the option to store metadata for objects and buckets
within a specified directory:
bucket: <sidecar dir>/<bucket>/meta/<attribute>
object: <sidecar dir>/bucket/<object>/meta/<attribute>
Example invocation:
./versitygw -a myaccess -s mysecret posix --sidecar /tmp/sidecar /tmp/gw
The attributes are stored by name within the hidden directory.
The posix backend will return ENOTEMPTY when trying to delete a
directory that is not empty. This normally would run successfully
on object systems. So we need to create another non-standard error
for this case. We mainly just don't want to return InternalError
for this case.
Fixes#946
The fileystem will return ENOTDIR if we try to access a file path
where a parent directory within the path already exists as a file.
In this case we need to return a standard 404 no such key since
the request object does not exist within the filesytem.
Fixes#942
From #919, This provides the *os.File handle to io.Copy() for the
case where the full file range is requested. This prevents hiding
the *os.File type for io.Copy() optimizations.
This still requires the change to valyala/fasthttp#1889 to expose
the net.Conn similarly to enable the linux sendfile optimization.
We had 0755 hard coded for newly created directories before. This
adds a user option to configure what the default mode permissions
should be for newly created directories.
Fixes#878
Ben McClellandandGitHub
niksis02
John BerthelsandBen McClelland
jonaustin09
Ryan HilemanandBen McClelland