Commit Graph

376 Commits

Author SHA1 Message Date
miyuko 325d6bedda [breaking-change] Change the format of the SYSLOG_ADDR env variable. 2025-12-07 09:52:15 +00:00
Catherine fc9e6fcf7b [breaking-change] Listen only on localhost by default.
It is expected that in most deployments, a reverse proxy server like
Caddy or Nginx will be connecting to Caddy; listening on any address
by default is a privacy and security concern.
2025-12-07 07:17:54 +00:00
Catherine 3840ba3c98 Use TOML output for -print-config instead of JSON.
This is much easier to read, and can be used as a template for
a new configuration.
v0.3.1
2025-12-07 05:43:00 +00:00
Catherine b58fe54c50 Report "dead" redirects as site issues.
Using a non-forced redirect with a URL matching a manifest entry turns
out to be a common and confusing mistake.
v0.3.0
2025-12-07 04:21:00 +00:00
Catherine d1f55d6776 Style. NFC 2025-12-07 03:41:16 +00:00
woodpecker-bot 9e0267828d chore(deps): update all dependencies 2025-12-07 00:25:53 +00:00
Catherine cf2c8f6270 Don't observe errors expected during incremental updates. 2025-12-06 23:15:25 +00:00
Catherine 43b6d92492 Split UnfreezeDomain off FreezeDomain. NFC
The code would branch on the value of `freeze` in basically all
implementations and call sites.
2025-12-06 01:40:19 +00:00
Catherine 609e5ca452 Display dead blob count after tracing. 2025-12-06 01:36:52 +00:00
Catherine 82aebb70bf Add basic garbage tracer.
This isn't a concurrent GC and it cannot provide a reliable result;
the output is just an estimate.
2025-12-06 01:21:19 +00:00
woodpecker-bot 7c3b2248c9 chore(deps): update all dependencies 2025-12-06 00:45:24 +00:00
Catherine 9c6f735df0 Fix loss of context in POST handler.
This caused the principal to not be available when creating the new
audit record.
2025-12-06 00:36:46 +00:00
Catherine ed2d853cbe Add EnumerateManifests API and -list-manifests option.
The new API replaces the `ListManifests` API.

This also adds `Name` and `Size` to manifest metadata.
2025-12-06 00:10:04 +00:00
Catherine 1e3c39b7f6 Add EnumerateBlobs API and -list-blobs option.
This also adds `Name` to blob metadata.
2025-12-06 00:10:04 +00:00
Catherine 92dc8f7231 Consolidate return values into BlobMetadata. NFC 2025-12-06 00:10:04 +00:00
miyuko e9edfb8f5c [breaking-change] Read principal's IP address from X-Forwarded-For. 2025-12-06 00:04:42 +00:00
miyuko 2cd8b58944 Don't put blobs that only contain hashes when incrementally uploading. 2025-12-05 20:41:12 +00:00
Catherine 1283b4e0eb Set Content-Type: to negotiated content type. 2025-12-05 19:33:06 +00:00
Catherine 7313ab7d13 Fix several content type negotiation issues.
* No `Accept:` header should be the same as `Accept: */*`.
  * For unresolved reference error, `text/plain` should take priority.
2025-12-05 18:56:20 +00:00
Catherine bd44f65b51 Add handling of Accept: application/vnd.git-pages.unresolved.
This will be used for incremental archive updates.
2025-12-05 18:21:42 +00:00
Catherine 8d58793576 Provide Accept-Encoding: in 406 Not Acceptable responses. 2025-12-05 16:38:31 +00:00
Catherine 6076c17c51 Rename HTTP negotiation items. NFC 2025-12-05 16:37:49 +00:00
Catherine 959715269f Collect unresolved blob references in a dedicated error structure.
This will be used for incremental archive uploads.
2025-12-05 11:31:34 +00:00
Catherine faa486c779 Collect statistics on blob reuse during archive upload. 2025-12-05 11:20:28 +00:00
Catherine 50d28f3c8b Resolve /git/blobs/ symlinks as blob references to the old manifest.
This will be used for incremental archive uploads.
2025-12-05 10:53:49 +00:00
Catherine eb6418b9b6 Fill in git_hash for regular files in archive uploads.
This will be used for incremental archive uploads.
2025-12-05 10:53:44 +00:00
Catherine 32c449e380 Use path.Join where applicable. NFC 2025-12-05 05:52:07 +00:00
Catherine 9036915ff9 Cache individual layers of Docker container as it is being built.
Unfortunately this is still not enough to fit into codeberg-medium :(
v0.2.0
2025-12-05 04:45:03 +00:00
Catherine 6cf49c6093 Stabilize feature serve-h2c.
It has been tested on Grebedoc (Fly.io servers) and found to work
satisfactorily, though without any apparent benefit. It requires client
opt-in and so enabling it at all times is benign.
2025-12-05 03:56:26 +00:00
Catherine da0758b972 Stabilize features patch and archive-site.
The PATCH method has been tested by myself and on Codeberg and found
to work satisfactorily.

Because using PATCH causes the git-pages server to store state that
is not necessarily easily reproducible from any single specific source
(i.e. it stores a composition of many disparate requests), it may be
necessary to back it up. For this, the feature `archive-site` is also
stabilized. It has not seen much use but not providing a backup method
would be a disservice.
2025-12-05 03:55:54 +00:00
Catherine 8eeaf222ca Log beginning of audit event processing.
This helps debugging slow scripts (e.g. using ClamAV).
2025-12-05 03:41:23 +00:00
Catherine 8c29ba3fe7 Implement -audit-server.
To use this function, configure git-pages with e.g.:

    [audit]
    collect = true
    notify-url = "http://localhost:3004/"

and run an audit server with e.g.:

    git-pages -audit-server tcp/:3004 python $(pwd)/process.py

The provided command line is executed after appending two arguments
(audit record ID and event type), and runs in a temporary directory
with the audit record extracted into it. The following files will
be present in this directory:
  * `$1-event.json` (always)
  * `$1-manifest.json` (if type is `CommitManifest`)
  * `$1-archive.tar` (if type is `CommitManifest`)

The script must complete successfully for the event processing to
finish. The notification will keep being re-sent (by the worker) with
exponential backoff until it does.
2025-12-05 03:19:32 +00:00
Catherine 464c40db9c Add Create-Parents: mode to PATCH method.
This acts like `mkdir -p`, making it much less annoying to deploy
e.g. documentation preview generators that use deep paths.

Like before, the site must already exist: we cannot do a CAS on
a non-existent manifest at the moment.
2025-12-04 18:23:44 +00:00
Catherine 93565e4e04 Factor out getWebRoot(). NFCI 2025-12-04 17:51:07 +00:00
Catherine 5f1ce5d334 Fix a bug preventing new manifests from being committed to S3. 2025-12-04 17:50:28 +00:00
Catherine f59830ae20 Rename PATCH Race-Free: header to Atomic:.
Neither of these names is self-explanatory, and it is better to have
fewer distinct identifiers for the same concept.
2025-12-04 16:51:32 +00:00
Catherine 1c017020c1 Update -audit-log to fetch records in parallel.
This makes it *much* faster.
2025-12-04 16:45:29 +00:00
Catherine e730b2bcd2 Fix phrasing in README. 2025-12-04 16:18:31 +00:00
Catherine 886635ce5e Implement -audit-log option.
Also, record the principal of `git-pages -{freeze,unfreeze}-domain`
and `git-pages -update-site` as the CLI administrator.
2025-12-04 15:58:14 +00:00
Catherine 4161013fc0 Implement -audit-read option. 2025-12-04 15:25:28 +00:00
Catherine 8f0712b3ad Improve error handling in CollectTar. 2025-12-04 15:23:15 +00:00
Catherine 2ebf4400bf Update introduction in README. 2025-12-04 13:52:37 +00:00
Catherine 1ad5d5590c Optionally include principal's IP address in audit record metadata.
Currently this doesn't handle `X-Forwarded-For` and as such isn't very
useful. It is surprisingly difficult to find a high-quality library for
parsing `X-Forwarded-For` and a solution will have to be found.
2025-12-04 06:17:11 +00:00
Catherine 75489f563e Fix issues with locking on Windows. 2025-12-04 05:09:28 +00:00
Catherine d5360817f3 Simplify fetch logging. NFC 2025-12-04 03:52:03 +00:00
Catherine e8c9cf588c Drop the shared manifest lock in the FS backend.
On Linux and macOS, two file descriptors opened by the same process are
treated as if they were different processes for the purpose of locking.
2025-12-04 03:00:47 +00:00
Catherine f5c48d0759 Use ETag as precondition for partial updates.
Last-Modified does not have enough resolution to be fully reliable;
ETag does. This test now passes on both filesystem and MinIO:

    $ go run ./test/stresspatch -count 100
    ...
    written: 100 of 100

Other S3 implementations haven't been tested.
2025-12-04 03:00:47 +00:00
Catherine 92d6796ad9 Return both LastModified and ETag in manifest metadata. NFCI 2025-12-04 03:00:47 +00:00
Catherine 460ff41cc9 Allow PATCH method to apply partial updates.
Gated behind the `patch` feature.
2025-12-04 03:00:47 +00:00
Catherine be75cc82a4 Factor out functions to create and fill a manifest. NFCI 2025-12-03 19:36:15 +00:00