374 Commits

Author SHA1 Message Date
Catherine
3840ba3c98 Use TOML output for -print-config instead of JSON.
This is much easier to read, and can be used as a template for
a new configuration.
v0.3.1
2025-12-07 05:43:00 +00:00
Catherine
b58fe54c50 Report "dead" redirects as site issues.
Using a non-forced redirect with a URL matching a manifest entry turns
out to be a common and confusing mistake.
v0.3.0
2025-12-07 04:21:00 +00:00
Catherine
d1f55d6776 Style. NFC 2025-12-07 03:41:16 +00:00
woodpecker-bot
9e0267828d chore(deps): update all dependencies 2025-12-07 00:25:53 +00:00
Catherine
cf2c8f6270 Don't observe errors expected during incremental updates. 2025-12-06 23:15:25 +00:00
Catherine
43b6d92492 Split UnfreezeDomain off FreezeDomain. NFC
The code would branch on the value of `freeze` in basically all
implementations and call sites.
2025-12-06 01:40:19 +00:00
Catherine
609e5ca452 Display dead blob count after tracing. 2025-12-06 01:36:52 +00:00
Catherine
82aebb70bf Add basic garbage tracer.
This isn't a concurrent GC and it cannot provide a reliable result;
the output is just an estimate.
2025-12-06 01:21:19 +00:00
woodpecker-bot
7c3b2248c9 chore(deps): update all dependencies 2025-12-06 00:45:24 +00:00
Catherine
9c6f735df0 Fix loss of context in POST handler.
This caused the principal to not be available when creating the new
audit record.
2025-12-06 00:36:46 +00:00
Catherine
ed2d853cbe Add EnumerateManifests API and -list-manifests option.
The new API replaces the `ListManifests` API.

This also adds `Name` and `Size` to manifest metadata.
2025-12-06 00:10:04 +00:00
Catherine
1e3c39b7f6 Add EnumerateBlobs API and -list-blobs option.
This also adds `Name` to blob metadata.
2025-12-06 00:10:04 +00:00
Catherine
92dc8f7231 Consolidate return values into BlobMetadata. NFC 2025-12-06 00:10:04 +00:00
miyuko
e9edfb8f5c [breaking-change] Read principal's IP address from X-Forwarded-For. 2025-12-06 00:04:42 +00:00
miyuko
2cd8b58944 Don't put blobs that only contain hashes when incrementally uploading. 2025-12-05 20:41:12 +00:00
Catherine
1283b4e0eb Set Content-Type: to negotiated content type. 2025-12-05 19:33:06 +00:00
Catherine
7313ab7d13 Fix several content type negotiation issues.
* No `Accept:` header should be the same as `Accept: */*`.
  * For unresolved reference error, `text/plain` should take priority.
2025-12-05 18:56:20 +00:00
Catherine
bd44f65b51 Add handling of Accept: application/vnd.git-pages.unresolved.
This will be used for incremental archive updates.
2025-12-05 18:21:42 +00:00
Catherine
8d58793576 Provide Accept-Encoding: in 406 Not Acceptable responses. 2025-12-05 16:38:31 +00:00
Catherine
6076c17c51 Rename HTTP negotiation items. NFC 2025-12-05 16:37:49 +00:00
Catherine
959715269f Collect unresolved blob references in a dedicated error structure.
This will be used for incremental archive uploads.
2025-12-05 11:31:34 +00:00
Catherine
faa486c779 Collect statistics on blob reuse during archive upload. 2025-12-05 11:20:28 +00:00
Catherine
50d28f3c8b Resolve /git/blobs/ symlinks as blob references to the old manifest.
This will be used for incremental archive uploads.
2025-12-05 10:53:49 +00:00
Catherine
eb6418b9b6 Fill in git_hash for regular files in archive uploads.
This will be used for incremental archive uploads.
2025-12-05 10:53:44 +00:00
Catherine
32c449e380 Use path.Join where applicable. NFC 2025-12-05 05:52:07 +00:00
Catherine
9036915ff9 Cache individual layers of Docker container as it is being built.
Unfortunately this is still not enough to fit into codeberg-medium :(
v0.2.0
2025-12-05 04:45:03 +00:00
Catherine
6cf49c6093 Stabilize feature serve-h2c.
It has been tested on Grebedoc (Fly.io servers) and found to work
satisfactorily, though without any apparent benefit. It requires client
opt-in and so enabling it at all times is benign.
2025-12-05 03:56:26 +00:00
Catherine
da0758b972 Stabilize features patch and archive-site.
The PATCH method has been tested by myself and on Codeberg and found
to work satisfactorily.

Because using PATCH causes the git-pages server to store state that
is not necessarily easily reproducible from any single specific source
(i.e. it stores a composition of many disparate requests), it may be
necessary to back it up. For this, the feature `archive-site` is also
stabilized. It has not seen much use but not providing a backup method
would be a disservice.
2025-12-05 03:55:54 +00:00
Catherine
8eeaf222ca Log beginning of audit event processing.
This helps debugging slow scripts (e.g. using ClamAV).
2025-12-05 03:41:23 +00:00
Catherine
8c29ba3fe7 Implement -audit-server.
To use this function, configure git-pages with e.g.:

    [audit]
    collect = true
    notify-url = "http://localhost:3004/"

and run an audit server with e.g.:

    git-pages -audit-server tcp/:3004 python $(pwd)/process.py

The provided command line is executed after appending two arguments
(audit record ID and event type), and runs in a temporary directory
with the audit record extracted into it. The following files will
be present in this directory:
  * `$1-event.json` (always)
  * `$1-manifest.json` (if type is `CommitManifest`)
  * `$1-archive.tar` (if type is `CommitManifest`)

The script must complete successfully for the event processing to
finish. The notification will keep being re-sent (by the worker) with
exponential backoff until it does.
2025-12-05 03:19:32 +00:00
Catherine
464c40db9c Add Create-Parents: mode to PATCH method.
This acts like `mkdir -p`, making it much less annoying to deploy
e.g. documentation preview generators that use deep paths.

Like before, the site must already exist: we cannot do a CAS on
a non-existent manifest at the moment.
2025-12-04 18:23:44 +00:00
Catherine
93565e4e04 Factor out getWebRoot(). NFCI 2025-12-04 17:51:07 +00:00
Catherine
5f1ce5d334 Fix a bug preventing new manifests from being committed to S3. 2025-12-04 17:50:28 +00:00
Catherine
f59830ae20 Rename PATCH Race-Free: header to Atomic:.
Neither of these names is self-explanatory, and it is better to have
fewer distinct identifiers for the same concept.
2025-12-04 16:51:32 +00:00
Catherine
1c017020c1 Update -audit-log to fetch records in parallel.
This makes it *much* faster.
2025-12-04 16:45:29 +00:00
Catherine
e730b2bcd2 Fix phrasing in README. 2025-12-04 16:18:31 +00:00
Catherine
886635ce5e Implement -audit-log option.
Also, record the principal of `git-pages -{freeze,unfreeze}-domain`
and `git-pages -update-site` as the CLI administrator.
2025-12-04 15:58:14 +00:00
Catherine
4161013fc0 Implement -audit-read option. 2025-12-04 15:25:28 +00:00
Catherine
8f0712b3ad Improve error handling in CollectTar. 2025-12-04 15:23:15 +00:00
Catherine
2ebf4400bf Update introduction in README. 2025-12-04 13:52:37 +00:00
Catherine
1ad5d5590c Optionally include principal's IP address in audit record metadata.
Currently this doesn't handle `X-Forwarded-For` and as such isn't very
useful. It is surprisingly difficult to find a high-quality library for
parsing `X-Forwarded-For` and a solution will have to be found.
2025-12-04 06:17:11 +00:00
Catherine
75489f563e Fix issues with locking on Windows. 2025-12-04 05:09:28 +00:00
Catherine
d5360817f3 Simplify fetch logging. NFC 2025-12-04 03:52:03 +00:00
Catherine
e8c9cf588c Drop the shared manifest lock in the FS backend.
On Linux and macOS, two file descriptors opened by the same process are
treated as if they were different processes for the purpose of locking.
2025-12-04 03:00:47 +00:00
Catherine
f5c48d0759 Use ETag as precondition for partial updates.
Last-Modified does not have enough resolution to be fully reliable;
ETag does. This test now passes on both filesystem and MinIO:

    $ go run ./test/stresspatch -count 100
    ...
    written: 100 of 100

Other S3 implementations haven't been tested.
2025-12-04 03:00:47 +00:00
Catherine
92d6796ad9 Return both LastModified and ETag in manifest metadata. NFCI 2025-12-04 03:00:47 +00:00
Catherine
460ff41cc9 Allow PATCH method to apply partial updates.
Gated behind the `patch` feature.
2025-12-04 03:00:47 +00:00
Catherine
be75cc82a4 Factor out functions to create and fill a manifest. NFCI 2025-12-03 19:36:15 +00:00
Catherine
e99fade242 Make .gz/.zst extraction composable. NFCI 2025-12-03 17:30:55 +00:00
Catherine
21b82f8e2c [breaking-change] Implement audit record retrieval.
This is only a breaking change if you've enabled the `audit` feature.
All past audit reports should be removed once this commit is deployed,
as both the Protobuf schema and the Snowflake epoch have changed.
2025-12-03 16:43:33 +00:00