This provides an option for #223 without fully resolving it. (I think.)
Essentially, it acts very similar to the `gzip_static` and similar options for nginx, where it will check for the existence of pre-compressed files and serve those instead if the client allows it. I couldn't find a pre-existing way to actually parse the Accept-Encoding header properly (admittedly didn't look very hard) and just implemented one on my own that should be fine.
This should hopefully not have the same DOS vulnerabilities as #302, since it relies on the existing caching system. Compressed versions of files will be cached just like any other files, and that includes cache for missing files as well.
The compressed files will also be accessible directly, and this won't automatically decompress them. So, if you have a `tar.gz` file that you access directly, it will still be downloaded as the gzipped version, although you will now gain the option to download the `.tar` directly and decompress it in transit. (Which doesn't affect the server at all, just the client's way of interpreting it.)
----
One key thing this change also adds is a short-circuit when accessing directories: these always return 404 via the API, although they'd try the cache anyway and go through that route, which was kind of slow. Adding in the additional encodings, it's going to try for .gz, .br, and .zst files in the worst case as well, which feels wrong. So, instead, it just always falls back to the index-check behaviour if the path ends in a slash or is empty. (Which is implicitly just a slash.)
----
For testing, I set up this repo: https://codeberg.org/clarfonthey/testrepo
I ended up realising that LFS wasn't supported by default with `just dev`, so, it ended up working until I made sure the files on the repo *didn't* use LFS.
Assuming you've run `just dev`, you can go directly to this page in the browser here: https://clarfonthey.localhost.mock.directory:4430/testrepo/
And also you can try a few cURL commands:
```shell
curl https://clarfonthey.localhost.mock.directory:4430/testrepo/ --verbose --insecure
curl -H 'Accept-Encoding: gz' https://clarfonthey.localhost.mock.directory:4430/testrepo/ --verbose --insecure | gunzip -
curl -H 'Accept-Encoding: br' https://clarfonthey.localhost.mock.directory:4430/testrepo/ --verbose --insecure | brotli --decompress -
curl -H 'Accept-Encoding: zst' https://clarfonthey.localhost.mock.directory:4430/testrepo/ --verbose --insecure | zstd --decompress -
```
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/387
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: ltdk <usr@ltdk.xyz>
Co-committed-by: ltdk <usr@ltdk.xyz>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| [golangci/golangci-lint](https://github.com/golangci/golangci-lint) | minor | `v1.59.1` -> `v1.60.1` |
---
### Release Notes
<details>
<summary>golangci/golangci-lint (golangci/golangci-lint)</summary>
### [`v1.60.1`](https://github.com/golangci/golangci-lint/blob/HEAD/CHANGELOG.md#v1601)
[Compare Source](https://github.com/golangci/golangci-lint/compare/v1.59.1...v1.60.1)
1. Updated linters
- `errorlint`: from 1.5.2 to 1.6.0
- `exhaustruct`: from 3.2.0 to 3.3.0 (recognize custom error values in return)
- `fatcontext`: from 0.2.2 to 0.4.0 (fix false positives for context stored in structs)
- `gocognit`: from 1.1.2 to 1.1.3
- `gomodguard`: from 1.3.2 to 1.3.3
- `govet` (`printf`): report non-constant format, no args
- `lll`: advertise max line length instead of just reporting failure
- `revive`: from 1.3.7 to 1.3.9 (new rule: `comments-density`)
- `sloglint`: from 0.7.1 to 0.7.2
- `spancheck`: from 0.6.1 to 0.6.2
- `staticcheck`: from 0.4.7 to 0.5.0
- `tenv`: from 1.7.1 to 1.10.0 (remove reports on fuzzing)
- `testifylint`: from 1.3.1 to 1.4.3 (new options: `formatter`, `suite-broken-parallel`, `suite-subtest-run`)
- `tparallel`: from 0.3.1 to 0.3.2
- `usestdlibvars`: from 1.26.0 to 1.27.0 (fix false-positive with number used inside a mathematical operations)
- `wsl`: from 4.2.1 to 4.4.1
- ️⚠️ `unused`: remove `exported-is-used` option
2. Fixes
- SARIF: sanitize level property
- ️⚠️ `typecheck` issues should never be ignored
3. Documentation
- Add link on linter without configuration
- Remove 'trusted by' page
- `wsl` update documentation of the configuration
4. misc.
- 🎉 go1.23 support
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - "before 4am" (UTC).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
Co-authored-by: woodpecker-bot <woodpecker-bot@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/381
Co-authored-by: Dependency bot <renovate-bot@noreply.codeberg.org>
Co-committed-by: Dependency bot <renovate-bot@noreply.codeberg.org>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| golang | minor | `1.22` -> `1.23` |
---
### Configuration
📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - "before 4am" (UTC).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40NDAuNyIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
Co-authored-by: woodpecker-bot <woodpecker-bot@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/380
Co-authored-by: Dependency bot <renovate-bot@noreply.codeberg.org>
Co-committed-by: Dependency bot <renovate-bot@noreply.codeberg.org>
HTTP uses GMT [1,2] rather than UTC as timezone for timestamps. However,
the Last-Modified header used UTC which confused at least wget.
Before, UTC was used:
$ wget --no-check-certificate -S --spider https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
...
Last-Modified: Sun, 11 Sep 2022 08:37:42 UTC
...
Last-modified header invalid -- time-stamp ignored.
...
After, GMT is used:
$ wget --no-check-certificate -S --spider https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
...
Last-Modified: Sun, 11 Sep 2022 08:37:42 GMT
...
(no last-modified-header-invalid warning)
[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified
[2]: https://www.rfc-editor.org/rfc/rfc9110#name-date-time-formatsFixes#364
---
Whatt I noticed is that the If-Modified-Since header isn't accepted (neither with GMT nor with UTC):
```
$ wget --header "If-Modified-Since: Sun, 11 Sep 2022 08:37:42 GMT" --no-check-certificate -S --spider https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
Spider mode enabled. Check if remote file exists.
--2024-07-15 23:31:41-- https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
Resolving cb_pages_tests.localhost.mock.directory (cb_pages_tests.localhost.mock.directory)... 127.0.0.1
Connecting to cb_pages_tests.localhost.mock.directory (cb_pages_tests.localhost.mock.directory)|127.0.0.1|:4430... connected.
WARNING: The certificate of ‘cb_pages_tests.localhost.mock.directory’ is not trusted.
WARNING: The certificate of ‘cb_pages_tests.localhost.mock.directory’ doesn't have a known issuer.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Cache-Control: public, max-age=600
Content-Length: 124635
Content-Type: image/jpeg
Etag: "073af1960852e2a4ef446202c7974768b9881814"
Last-Modified: Sun, 11 Sep 2022 08:37:42 GMT
Referrer-Policy: strict-origin-when-cross-origin
Server: pages-server
Strict-Transport-Security: max-age=63072000; includeSubdomains; preload
Date: Mon, 15 Jul 2024 21:31:42 GMT
Length: 124635 (122K) [image/jpeg]
Remote file exists
```
I would have expected a 304 (Not Modified) rather than a 200 (OK). I assume this is simply not supported and on production 304 is returned by a caching proxy in front of pages-server.
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/365
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Peter Gerber <peter@arbitrary.ch>
Co-committed-by: Peter Gerber <peter@arbitrary.ch>
This PR renames `gitea` in cli args to `forge` and `GITEA` in environment variables to `FORGE` and adds the gitea names as aliases for the forge names.
Also closes#311
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/339
I know get the following
```
docker run --rm pages-server:test
12:40PM ERR A fatal error occurred error="could not create new gitea client: Get \"/api/v1/version\": unsupported protocol scheme \"\""
```
which I am not sure is OK as doing the same with v5.1 results in
```
docker run --platform linux/amd64 --rm -it codeberg.org/codeberg/pages-server:v5.1
ACME client has wrong config: you must set $ACME_ACCEPT_TERMS and $DNS_PROVIDER, unless $ACME_API is set to https://acme.mock.directory
```
The error is the same though what I get when building of 8cba7f9c8a (just before merging the multi-arch PR).
Not sure if this ERR is expected but it should be unrelated to the multiarch approach.
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/337
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: pat-s <patrick.schratz@gmail.com>
Co-committed-by: pat-s <patrick.schratz@gmail.com>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| mstruebing/editorconfig-checker | major | `2.7.2` -> `v3.0.1` |
---
### Configuration
📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - "before 4am" (UTC).
🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNDAuNSIsInVwZGF0ZWRJblZlciI6IjM3LjM0MC41IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
Co-authored-by: woodpecker-bot <woodpecker-bot@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/329
Co-authored-by: Dependency bot <renovate-bot@noreply.codeberg.org>
Co-committed-by: Dependency bot <renovate-bot@noreply.codeberg.org>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| golang | minor | `1.21` -> `1.22` |
---
### Configuration
📅 **Schedule**: Branch creation - "every weekend" (UTC), Automerge - "before 4am" (UTC).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zMzMuMSIsInVwZGF0ZWRJblZlciI6IjM3LjMzMy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJkZXBlbmRlbmNpZXMiXX0=-->
Co-authored-by: woodpecker-bot <woodpecker-bot@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/326
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Dependency bot <renovate-bot@noreply.codeberg.org>
Co-committed-by: Dependency bot <renovate-bot@noreply.codeberg.org>
Julien MalkaandGusted
Dependency bot
crapStoneandcrapStone
ltdkandcrapStone
GnaaarwhalandcrapStone
Peter GerberandcrapStone
fnetXandcrapStone
adagioandcrapStone
pat-sandcrapStone
