74 Commits

Author SHA1 Message Date
Catherine
3840ba3c98 Use TOML output for -print-config instead of JSON.
This is much easier to read, and can be used as a template for
a new configuration.
2025-12-07 05:43:00 +00:00
Catherine
43b6d92492 Split UnfreezeDomain off FreezeDomain. NFC
The code would branch on the value of `freeze` in basically all
implementations and call sites.
2025-12-06 01:40:19 +00:00
Catherine
82aebb70bf Add basic garbage tracer.
This isn't a concurrent GC and it cannot provide a reliable result;
the output is just an estimate.
2025-12-06 01:21:19 +00:00
Catherine
ed2d853cbe Add EnumerateManifests API and -list-manifests option.
The new API replaces the `ListManifests` API.

This also adds `Name` and `Size` to manifest metadata.
2025-12-06 00:10:04 +00:00
Catherine
1e3c39b7f6 Add EnumerateBlobs API and -list-blobs option.
This also adds `Name` to blob metadata.
2025-12-06 00:10:04 +00:00
Catherine
92dc8f7231 Consolidate return values into BlobMetadata. NFC 2025-12-06 00:10:04 +00:00
miyuko
e9edfb8f5c [breaking-change] Read principal's IP address from X-Forwarded-For. 2025-12-06 00:04:42 +00:00
Catherine
6cf49c6093 Stabilize feature serve-h2c.
It has been tested on Grebedoc (Fly.io servers) and found to work
satisfactorily, though without any apparent benefit. It requires client
opt-in and so enabling it at all times is benign.
2025-12-05 03:56:26 +00:00
Catherine
8c29ba3fe7 Implement -audit-server.
To use this function, configure git-pages with e.g.:

    [audit]
    collect = true
    notify-url = "http://localhost:3004/"

and run an audit server with e.g.:

    git-pages -audit-server tcp/:3004 python $(pwd)/process.py

The provided command line is executed after appending two arguments
(audit record ID and event type), and runs in a temporary directory
with the audit record extracted into it. The following files will
be present in this directory:
  * `$1-event.json` (always)
  * `$1-manifest.json` (if type is `CommitManifest`)
  * `$1-archive.tar` (if type is `CommitManifest`)

The script must complete successfully for the event processing to
finish. The notification will keep being re-sent (by the worker) with
exponential backoff until it does.
2025-12-05 03:19:32 +00:00
Catherine
1c017020c1 Update -audit-log to fetch records in parallel.
This makes it *much* faster.
2025-12-04 16:45:29 +00:00
Catherine
886635ce5e Implement -audit-log option.
Also, record the principal of `git-pages -{freeze,unfreeze}-domain`
and `git-pages -update-site` as the CLI administrator.
2025-12-04 15:58:14 +00:00
Catherine
4161013fc0 Implement -audit-read option. 2025-12-04 15:25:28 +00:00
Catherine
8f0712b3ad Improve error handling in CollectTar. 2025-12-04 15:23:15 +00:00
Catherine
92d6796ad9 Return both LastModified and ETag in manifest metadata. NFCI 2025-12-04 03:00:47 +00:00
Catherine
21b82f8e2c [breaking-change] Implement audit record retrieval.
This is only a breaking change if you've enabled the `audit` feature.
All past audit reports should be removed once this commit is deployed,
as both the Protobuf schema and the Snowflake epoch have changed.
2025-12-03 16:43:33 +00:00
Catherine
5b8267ace5 Parse URLs while loading configuration. NFCI
This avoids spreading URL parse error handling code all over
the codebase. It's not even easy to trigger that error!
2025-12-03 06:10:51 +00:00
Catherine
cc5f8f608e Consistently use context in Main().
Without this, some of the slog lines end in `\n` and some do not, which
I find deeply irritating.
2025-12-03 01:32:54 +00:00
Catherine
f089208ca7 [breaking-change] Make fallback handler per-instance, not per-wildcard.
There was never a particularly good reason to tie the fallback handler
to a wildcard domain; most importantly, this prevented it from being
used for custom domains, which is required for migrating custom domains
from Codeberg Pages v2 server.
2025-12-03 00:39:07 +00:00
Catherine
c250922f8d Allow domains to be administratively frozen.
The following script may be used to handle abusive sites:

    cd $(mktemp -d)
    echo "<h1>Gone</h1>" >index.html
    echo "/* /index.html 410" >_redirects
    tar cf site.tar index.html _redirects
    git-pages -update-site $1 site.tar
    git-pages -freeze-domain $1
2025-12-02 23:56:01 +00:00
Catherine
d82ae69625 Simplify SIGINT handling code. NFC 2025-11-23 03:03:33 +00:00
miyuko
cb7802df10 Pass the context to logging functions. 2025-11-22 07:05:07 +00:00
miyuko
b01e67f993 Exit gracefully (run deferred statements in main()) on SIGINT. 2025-11-21 23:34:33 +00:00
Catherine
982c3321e0 Reword some log messages. NFC 2025-11-20 04:11:57 +00:00
Catherine
eaf77565bc Improve configuration reload and clarify scope.
This commit also moves all of the globals into `main.go`.
2025-11-20 04:06:09 +00:00
Catherine
c93d3a0bb5 Reload configuration on SIGHUP (if supported by OS).
On Windows, there is no way to reload configuration at runtime.
2025-11-20 03:43:22 +00:00
Catherine
a924dd5116 Improve CLI usage text. 2025-11-20 03:15:03 +00:00
Catherine
f148792bcd Accept an output argument in -get-blob, -get-manifest, -get-archive. 2025-11-20 03:03:03 +00:00
Catherine
6db850e2c4 Allow downloading entire site via CLI or HTTP.
The HTTP endpoint is `/.git-pages/archive.tar` and it is gated behind
a feature flag `archive-site`. It serially downloads every blob and
writes it to the client in a chunked response, optionally compressed
with gzip or zstd as per `Accept-Encoding:`. It is authorized the same
as `/.git-pages/manifest.json`, for the same reasons.

The CLI operation is `-get-archive <site-name>` and it writes a tar
archive to stdout. This could be useful for an administrator to review
the contents of a site in response to a report.

Both `_headers` and `_redirects` files are present in the output,
reconstituted from the manifest.
2025-11-20 02:09:49 +00:00
Catherine
0e342b11f6 Add Last-Modified: header to /.git-pages/ metadata responses. 2025-11-19 22:37:06 +00:00
miyuko
aa965c5a08 Use s3:GetObject instead of s3:ListObjects for CheckDomain. 2025-10-22 13:45:15 +01:00
Catherine
d1be93919f Make installable with go install. 2025-10-22 05:24:55 +00:00
miyuko
eda3e8a791 Add stale-while-revalidate support to the cache. 2025-10-15 23:53:12 +01:00
Catherine
a7c47a798f Allow running without a configuration TOML file.
This is useful if it's more convenient to configure the server via
environment variables only.
2025-10-02 11:33:21 +00:00
miyuko
d5a4ac4098 Fix URL argument validation when using the -update-site CLI option. 2025-09-30 21:59:41 +01:00
Catherine
396cabd43b Exclude metrics endpoint from observability.
Who watches the watchers? Nobody should!
2025-09-30 02:26:20 +00:00
Catherine
b1c50c10de Thread context argument through the backend interface. NFC 2025-09-29 23:10:33 +00:00
Catherine
217f3a9320 Switch log handling to slog.
This currently doesn't add any structure to the logs, changing just
the handler and output format. It does also add Sentry logging support.

The `log-format` configuration value now accepts values `none`, `text`,
and `json`.
2025-09-29 22:10:41 +00:00
Catherine
c3575a09ca Add Sentry support. 2025-09-29 21:11:10 +00:00
Catherine
dcabcac106 Rename feature h2c to serve-h2c. 2025-09-29 01:47:13 +00:00
Catherine
79bfbb96c0 Accept repository URL in -update-site CLI option. 2025-09-29 01:16:59 +00:00
miyuko
1c7ef99359 Add manifest and blob metrics. 2025-09-27 18:36:17 +01:00
Catherine
94ec490ca5 Add -update-site CLI argument, for administrative updates. 2025-09-26 00:31:14 +00:00
Catherine
fc029ad219 [breaking-change] Remove health endpoint handler and renumber metrics.
The previous commit has eliminated any way to use `ServeHealth()`,
but kept it around to not introduce breaking changes. This one has
no such constraint.
2025-09-23 02:39:11 +00:00
Catherine
38269c66ae Add -get-blob mode, printing contents of a blob. 2025-09-22 21:33:23 +00:00
Catherine
3701aa1800 Simplify. NFC 2025-09-22 19:46:17 +00:00
miyuko
1aef0288e7 Add page operation metrics and expose them in Prometheus text format. 2025-09-22 19:03:59 +01:00
Catherine
6dfc03c3a8 Reimplement ugly automemlimit logging at startup. 2025-09-22 07:02:42 +00:00
Catherine
a159dba0b8 [breaking-change] Redesign environment var configuration overrides.
This is done using reflection to avoid boilerplate and potential desync
of the two configuration interfaces. The `[[wildcards]]` section did
not fit well into the "splat every config key" paradigm, so it is
unmarshalled as a whole from a JSON payload in an environment variable.

This commit also splits up the `Config` type into small per-section
struct types and removes most references to the global `config` in
favor of passing pointers to sections around.

A new option, `-print-config-env-vars`, shows the names and types of
all of the available configuration knobs.
2025-09-22 07:02:42 +00:00
Catherine
d81676fea0 Miscellaneous minor configuration related fixes. 2025-09-22 01:31:41 +00:00
Catherine
bf2922f892 [breaking-change] Add default config values where appropriate. 2025-09-21 23:08:27 +00:00