Commit Graph

487 Commits

Author SHA1 Message Date
miyuko
a233cdfbb8 Fix S3Backend.SearchAuditLog ignoring search options. 2026-04-27 16:48:36 +01:00
Catherine
4d8e620846 Account for manifest's own size for -size-histogram. 2026-04-27 01:16:16 +02:00
Catherine
e8112c1abe Add a CLI command -audit-expire to purge old audit records.
This is particularly important with the FS backend, where there isn't
necessarily native tooling capable of handling this task correctly
(since not every filesystem supports file "birth times", and since
restoring data from a backup will reset the "birth time" of audit
records to the moment of restoration).
2026-04-26 23:10:22 +00:00
Catherine
b0a674abf4 Fix incorrect start time in AuditID.CompareTime. 2026-04-26 22:59:36 +00:00
Catherine
f001107056 Create audit records as read-only when using FS backend.
There is no reason to ever modify the records.
2026-04-26 22:55:30 +00:00
Catherine
b7170e3077 Create a domain cache for CLI operations.
Fixes a regression (crash) in `-update-site` introduced in commit
  bbdaae7280
.
v0.8.1
2026-04-26 21:05:55 +00:00
whitequark
7f5e02081d Add links to support channels. 2026-04-25 22:52:06 +02:00
Catherine
59cf185143 Only log media type for PUT, PATCH, and POST requests.
There isn't much point in logging `Accept:` for GET requests and it
is very noisy.
2026-04-23 16:42:46 +00:00
Catherine
c5c5306688 [breaking-change] Use a distinct scope for forge DNS allowlist authz.
Before this commit, a `_git-pages-repository.<host>` TXT record would
allow both forge DNS allowlist authorization, as well as normal DNS
allowlist authorization. This means that a site set up to have its
contents updated by a Forgejo Action could have its contents replaced
by the contents of the repository which contains the Forgejo Action,
which will effectively erase the site in most cases. This is a classic
confused deputy scenario.

To fix this, forge DNS allowlist authorization now uses a distinct
`_git-pages-forge-allowlist.<host>` TXT record, removing ambiguity
that allows this scenario to happen.

The issue was introduced in 27a6de792c
and existed in `main` for about a hour, so it is unlikely anybody
has been impacted by this.
v0.8.0
2026-04-23 15:20:32 +00:00
Catherine
27a6de792c Allow using forge authorization with non-wildcard domains.
The new authorization method combines DNS allowlist and existing forge
authorization methods: DNS records are used to determine the allowed
repository URL, and forge authorization is used to check for push
permissions to that URL.
2026-04-22 01:59:37 +00:00
Catherine
2c109a5e1e Factor out common authorization code. NFC
This commit unifies most of the implementation of `AuthorizeDeletion`
and `AuthorizeUpdateFromArchive`, with the latter additionally checking
that the repository URL in the authorization grant follows the limits.

This is done in preparation of adding a second forge authorization
sub-mechanism that can handle non-wildcard domains.
2026-04-22 01:59:37 +00:00
Catherine
d17c645927 Improve forge authorization error message for invalid tokens.
Before:

    - not authorized by forge (wildcard)
      - cannot check repository permissions: GET https://codeberg.org/api/v1/repos/whitequark/whitequark.codeberg.page returned 401 Unauthorized

After:

    - not authorized by forge (wildcard)
      - no access to whitequark/whitequark.codeberg.page or invalid token
2026-04-22 01:59:37 +00:00
Catherine
57e9d05c7f Update default index branch name for codeberg-pages-compat quirk.
The actual Codeberg Pages v2 server uses the Forgejo default branch
for the index repository. The quirk previously used the `main` branch
unconditionally.

This is complex to implement, so per discussion with gusted we have
decided to change the default branch to `pages` so that it has parity
with non-Codeberg-specific behavior.
2026-04-22 00:47:49 +00:00
woodpecker-bot
1e6afe6570 [Renovate] Update docker.io/library/caddy:2.11.2-builder Docker digest to 10ed025 2026-04-20 02:41:04 +02:00
Andrew Cassidy
b3692362d8 Allow loading secrets from an additional configuration file.
Adds the `-secrets` command line flag, which defaults to `$CREDENTIALS_DIRECTORY/secrets.toml` if it exists. The secrets.toml file will be loaded the same way as the main config.toml.

Reviewed-on: https://codeberg.org/git-pages/git-pages/pulls/137
Reviewed-by: Catherine <whitequark@whitequark.org>
Co-authored-by: Andrew Cassidy <drewcassidy@me.com>
Co-committed-by: Andrew Cassidy <drewcassidy@me.com>
2026-04-20 02:40:34 +02:00
David Leadbeater
021c493daa Revert "Revert "[Renovate] Update all dependencies""
This reverts commit 57dc8f8520.
2026-04-18 23:12:49 +10:00
David Leadbeater
b54664258b Update go-git API to v6.0.0-alpha.2 2026-04-18 23:12:18 +10:00
Catherine
57dc8f8520 Revert "[Renovate] Update all dependencies"
This reverts commit 2b35996f62.
2026-04-18 12:08:07 +00:00
woodpecker-bot
2b35996f62 [Renovate] Update all dependencies 2026-04-18 11:59:05 +00:00
Catherine
cf050f505b Improve performance of -trace-garbage. 2026-04-14 05:01:37 +00:00
Catherine
6097a9abb8 Add a Server: header unconditionally.
Previously we wouldn't do it if hostname could not be determined, which
would break git-pages-cli based uploads on those machines.
2026-04-14 03:39:52 +00:00
Catherine
fe329d748d [breaking-change] Drop Fly.io-specific behavior.
Fly.io is led by AI boosterism, and we don't want to encourage that
kind of behavior.
2026-04-14 03:39:52 +00:00
miyuko
bbdaae7280 Add a domain cache to quickly reject non-existent domains. 2026-04-13 13:45:16 +00:00
miyuko
f400f8d246 Enable all S3 features when initializing the store. 2026-04-13 13:13:14 +00:00
bin
86259acf9c nix: fix building on non-linux platforms 2026-04-12 10:56:44 +00:00
woodpecker-bot
af7657a787 [Renovate] Update all dependencies 2026-04-12 01:06:02 +00:00
miyuko
ed24f08d5f Constrain the parallelism of fetching audit log records. 2026-04-11 19:43:13 +00:00
Catherine
d7651941c0 Fetch manifests from S3 in parallel for histogram and tracing.
This is mainly done to speed up histogram collection, as waiting some
minutes defeats the purpose of having a quick overview function.

This commit does speed up GC tracing as well, but not as much because
audit records are still retrieved one at a time. A similar mechanism
could be added in the future there.

Filesystem logic is functionally identical since it was fine already.
2026-04-04 21:10:05 +00:00
Catherine
bcd628fa6b Allow Chmod() in PutBlob() to fail with -EPERM.
This can happen on an NFSv4 filesystem with POSIX permissions disabled.

Fixes #131.
2026-04-04 01:17:32 +00:00
woodpecker-bot
6a3372a36a [Renovate] Update all dependencies 2026-04-04 00:13:10 +00:00
miyuko
8d4ea36dec Re-throw http.ErrAbortHandler from our panic handler.
This aborts the response to the client and doesn't log an error.

httputil.ReverseProxy commonly panics with this error.

This results in different behavior from simply swallowing the panic.
Panicking prevents flushing the response to the client, and in the case
of a panic from httputil.ReverseProxy it results in clients potentially
receiving an empty response instead of what was already written to
http.ResponseWriter. This behavior is the same as if the panic handler
hadn't been installed.
2026-04-03 00:29:45 +00:00
Catherine
6509a8e1d2 Add -size-histogram option for summarizing resource use.
Useful to evaluate who consumes the most storage (or the most size
quota) visually at a glance.
2026-04-01 23:52:24 +00:00
Catherine
6775f4aab5 Fix incorrect frozen domain check for S3 backend. 2026-04-01 22:50:40 +00:00
Catherine
1df1402f6b CI: fetch tags when building release artifacts. 2026-03-31 09:22:03 +00:00
Catherine
8dffd9cf11 CI: fix embedding of version information. 2026-03-31 09:05:37 +00:00
Catherine
5258bf756b Add support for Netlify Basic-Auth: mechanism. 2026-03-29 12:11:56 +00:00
woodpecker-bot
38eb8afd0e [Renovate] Update all dependencies 2026-03-29 12:06:56 +00:00
Catherine
2fdf0b805d Add hardlink support for tar archive upload.
"Why the fuck would anybody want that", you could reasonably ask.
Well, most wouldn't want this. However, if you wanted to use git-pages
to deduplicate your backups, you might find it that some backups
include hardlinks.

"Why the fuck would anybody put their backups in git-pages", you could
even more reasonably ask. Well, almost nobody would! However, tarsnap
doesn't let you download deduplicated data (even though it deduplicates
data in storage), restic can't ingest tarballs, I didn't have
a partition I could format for btrfs, and git-pages performed much
better than alternatives like juicefs.

In the end this is correct and not expensive to do, just very niche.
2026-03-28 17:04:12 +00:00
Catherine
e28d8cf0f2 Fix statistical accounting for incremental uploads. 2026-03-28 16:49:14 +00:00
miyuko
ccabfc22a6 Remove README text about the now-removed Sentry support. 2026-03-28 04:36:06 +00:00
miyuko
005e0fefed Remove the unused sensitiveHTTPHeaders variable. 2026-03-28 04:36:06 +00:00
Catherine
2267ab929c Update go-toml to v2.3.0 and freeze it.
The upstream added AGENTS.md and removed all unsafe code in the same
release. I've manually reviewed the entire v2.2.4..v2.3.0 diff and
found no issues except for one potential problem with
`go-toml/errors.subsliceOffset` that would only appear with a moving
GC. This seems like a strict improvement but we don't want any more
updates.
2026-03-28 00:35:01 +00:00
Catherine
338487c048 [breaking-change] Drop Sentry support.
The upstream added AGENTS.md and I have no time to review what they're
doing with that.
2026-03-28 00:34:57 +00:00
Catherine
b84a533be7 Add version override in Nix flake. v0.7.0 2026-03-27 23:13:42 +00:00
Catherine
678868f7e6 Add a -version flag. 2026-03-27 22:50:55 +00:00
Catherine
1ca67f0590 Add a configurable limit on concurrent blob uploads.
Otherwise uploading a site with over 50,000 files will fail with
the default Go runtime configuration.
2026-03-26 14:52:11 +00:00
Maxim Slipenko
c74ec4ad23 Add configuration file example with default values only.
Reviewed-on: https://codeberg.org/git-pages/git-pages/pulls/127
Co-authored-by: Maxim Slipenko <maxim@slipenko.com>
Co-committed-by: Maxim Slipenko <maxim@slipenko.com>
2026-03-26 12:19:34 +01:00
Catherine
b37ca8cd14 Fix combined partial and incremental updates.
It seems that I forgot to implement incremental update support for
partial updates entirely.
v0.6.2
2026-03-25 05:08:42 +00:00
Catherine
310cc7d438 Fix incorrect explanation in README. 2026-03-25 04:59:25 +00:00
Catherine
ad327b0382 Fix collection of symlinks in tar archives. 2026-03-25 04:55:34 +00:00