32 Commits

Author SHA1 Message Date
miyuko
ad92847fa0 Record git repo URL in manifest for archive uploads with forge auth.
Resolves: https://codeberg.org/git-pages/git-pages/issues/165
2026-05-11 03:38:54 +01:00
miyuko
bbdaae7280 Add a domain cache to quickly reject non-existent domains. 2026-04-13 13:45:16 +00:00
Catherine
b37ca8cd14 Fix combined partial and incremental updates.
It seems that I forgot to implement incremental update support for
partial updates entirely.
2026-03-25 05:08:42 +00:00
Catherine
96f210d253 Clear git metadata from PATCH'd manifests. 2025-12-24 02:18:09 +00:00
Catherine
cf2c8f6270 Don't observe errors expected during incremental updates. 2025-12-06 23:15:25 +00:00
Catherine
faa486c779 Collect statistics on blob reuse during archive upload. 2025-12-05 11:20:28 +00:00
Catherine
50d28f3c8b Resolve /git/blobs/ symlinks as blob references to the old manifest.
This will be used for incremental archive uploads.
2025-12-05 10:53:49 +00:00
Catherine
464c40db9c Add Create-Parents: mode to PATCH method.
This acts like `mkdir -p`, making it much less annoying to deploy
e.g. documentation preview generators that use deep paths.

Like before, the site must already exist: we cannot do a CAS on
a non-existent manifest at the moment.
2025-12-04 18:23:44 +00:00
Catherine
4161013fc0 Implement -audit-read option. 2025-12-04 15:25:28 +00:00
Catherine
f5c48d0759 Use ETag as precondition for partial updates.
Last-Modified does not have enough resolution to be fully reliable;
ETag does. This test now passes on both filesystem and MinIO:

    $ go run ./test/stresspatch -count 100
    ...
    written: 100 of 100

Other S3 implementations haven't been tested.
2025-12-04 03:00:47 +00:00
Catherine
92d6796ad9 Return both LastModified and ETag in manifest metadata. NFCI 2025-12-04 03:00:47 +00:00
Catherine
460ff41cc9 Allow PATCH method to apply partial updates.
Gated behind the `patch` feature.
2025-12-04 03:00:47 +00:00
Catherine
e99fade242 Make .gz/.zst extraction composable. NFCI 2025-12-03 17:30:55 +00:00
Catherine
89c57cfadb Use git filters for incremental updates from a git repository.
This commit changes the git fetch algorithm to only retrieve blobs
that aren't included in the previously deployed site manifest, if
git filters are supported by the remote.

It also changes how manifest entry sizes are represented, such that
both decompressed and compressed sizes are stored. This enables
computing accurate (and repeatable) sizes even after incremental
updates.

Co-authored-by: David Leadbeater <dgl@dgl.cx>
2025-12-02 22:23:43 +00:00
miyuko
cb7802df10 Pass the context to logging functions. 2025-11-22 07:05:07 +00:00
Catherine
0e342b11f6 Add Last-Modified: header to /.git-pages/ metadata responses. 2025-11-19 22:37:06 +00:00
miyuko
aa965c5a08 Use s3:GetObject instead of s3:ListObjects for CheckDomain. 2025-10-22 13:45:15 +01:00
Catherine
d1be93919f Make installable with go install. 2025-10-22 05:24:55 +00:00
Catherine
99b87226a1 Move update error observation to a single place. NFC 2025-10-18 21:49:54 +00:00
miyuko
eda3e8a791 Add stale-while-revalidate support to the cache. 2025-10-15 23:53:12 +01:00
miyuko
8bb6d0ff28 Unconditionally sample HTTP requests for tracing that take too long. 2025-10-15 18:26:33 +01:00
Catherine
1a0e594624 Add span based timings measurement and Sentry integration. 2025-09-30 00:56:58 +00:00
Catherine
b1c50c10de Thread context argument through the backend interface. NFC 2025-09-29 23:10:33 +00:00
Catherine
9907a26397 Remove leftover comments. NFC 2025-09-22 19:43:56 +00:00
Catherine
2af2975713 Add tar+gzip and tar+zstd compressed archive support. 2025-09-21 06:25:10 +00:00
Catherine
15b2f1ea39 Allow zip and tar archive uploads PUT request. 2025-09-20 07:16:10 +00:00
Catherine
95814dd3f3 Parse _redirects file and store rules in manifest. 2025-09-20 07:16:10 +00:00
Catherine
d89f03e665 Upgrade protobuf schema to edition 2023. NFCI
Also, some renames for consistency:
- `Manifest.repoURL`→`Manifest.repo_url`
- `Manifest.tree`→`Manifest.files`
2025-09-19 14:12:08 +00:00
Catherine
df6ca018a5 Unpublish site when pushing an empty repository. 2025-09-19 05:41:01 +00:00
Catherine
a0bd7d8650 Implement migration from v1 data layout. 2025-09-17 13:14:10 +00:00
miyuko
31131a6360 Use a context to ensure a time-based deadline for update operations. 2025-09-17 13:14:42 +01:00
Catherine
7fc81d3d97 [breaking-change] Rearchitect for better object store compatibility.
Co-authored-by: bin <flumf@users.noreply.github.com>
2025-09-17 05:59:50 +00:00