Catherine
6db850e2c4
Allow downloading entire site via CLI or HTTP.
...
The HTTP endpoint is `/.git-pages/archive.tar` and it is gated behind
a feature flag `archive-site`. It serially downloads every blob and
writes it to the client in a chunked response, optionally compressed
with gzip or zstd as per `Accept-Encoding:`. It is authorized the same
as `/.git-pages/manifest.json`, for the same reasons.
The CLI operation is `-get-archive <site-name>` and it writes a tar
archive to stdout. This could be useful for an administrator to review
the contents of a site in response to a report.
Both `_headers` and `_redirects` files are present in the output,
reconstituted from the manifest.
2025-11-20 02:09:49 +00:00
Catherine
0e342b11f6
Add Last-Modified: header to /.git-pages/ metadata responses.
2025-11-19 22:37:06 +00:00
Catherine
073435aa2b
Redirect domain.tld/project to domain.tld/project/ when present.
...
This is to match the behavior of GitHub, as well as because it isn't
particularly useful to serve a file from the index repo with the same
path segment as the project name (and quite confusing too).
2025-11-18 22:27:03 +00:00
Catherine
325c283e05
Refactor redirect code. NFC
2025-11-18 22:21:51 +00:00
miyuko
de17426f41
Observe blob fetch errors during GET requests.
2025-11-17 11:09:26 +00:00
Catherine
de40c8263a
Set Update-Result for DELETE requests.
...
Done for uniformity and to make git-pages-cli implementation nicer.
2025-11-16 00:18:29 +00:00
Catherine
3e59fd2734
Rename X-Pages-Update header to Update-Result.
...
Same rationale as in 9d0a3ac6ad
2025-11-15 23:46:20 +00:00
Catherine
9a431b8bbb
Add /.git-pages/health endpoint.
2025-11-15 21:17:30 +00:00
Catherine
3431217a09
Don't respond with a completely blank 404 page.
...
We respond to all other errors with a simple, 1-line explanation that
you could see when using e.g. curl. The one case of "site is found and
the path is a normal path, but it doesn't exist and the 404 page does
not exist either" was unhandled by accident.
2025-11-15 01:42:55 +00:00
Catherine
9d0a3ac6ad
Use Branch: instead of X-Pages-Branch: to set custom branch name.
2025-11-12 17:05:11 +00:00
Catherine
ed77339144
Remove deprecated COOP/COEP assignment based on content type.
2025-11-11 17:56:02 +00:00
Catherine
26b29ec4be
Add Netlify _headers support.
2025-11-11 15:36:14 +00:00
Catherine
f9e142dd51
Observe all storage errors reported by GetManifest.
...
Otherwise users may get jumpscares of "site not found" due to temporary
conditions (network errors to S3 backend included).
2025-11-11 06:10:01 +00:00
Catherine
2db3de01c7
Fix a nil dereference on non-custom 404 pages.
2025-10-27 16:14:35 +00:00
Catherine
91cafac86a
Apply Content-Type from the manifest to non-200 status pages.
2025-10-27 15:25:14 +00:00
Catherine
26b926293b
Serve X-Content-Type-Options: nosniff.
...
Mozilla HTTP Observatory cares about this (5 points), and there isn't
really any reason not to send it at all times.
2025-10-24 09:28:49 +00:00
Catherine
68343a3dff
Turns out a Web Worker is a type of frame (for COEP purposes).
2025-10-24 09:26:54 +00:00
miyuko
ffedc45a14
Don't send COEP/COOP headers for non-HTML resources.
2025-10-22 17:25:10 +01:00
miyuko
d6a7a72e09
Serve compressed content directly if client indicates support.
2025-10-22 16:59:35 +01:00
Catherine
d1be93919f
Make installable with go install.
2025-10-22 05:24:55 +00:00
miyuko
c39e57a857
Fetch manifests in parallel when handling GET requests.
2025-10-22 00:25:21 +01:00
Catherine
25f7ea08c9
Sniff Content-Type during site update.
...
This isn't yet used in the code responding to GET requests because we
do not yet have a migration path for legacy code.
2025-10-21 03:40:29 +00:00
Catherine
99b87226a1
Move update error observation to a single place. NFC
2025-10-18 21:49:54 +00:00
miyuko
fcc6245ce8
Respond to webhook deliveries in under 3 seconds.
2025-10-18 04:38:06 +01:00
Catherine
d54976e756
Report update errors or timeouts.
...
Looking through Sentry history, `update <domain> err:` is an extremely
high SNR signal of something going wrong; from configuration errors on
our side, to people pushing too-large git repositories and it failing.
Either way we should know.
2025-10-17 10:33:41 +00:00
miyuko
eda3e8a791
Add stale-while-revalidate support to the cache.
2025-10-15 23:53:12 +01:00
miyuko
8bb6d0ff28
Unconditionally sample HTTP requests for tracing that take too long.
2025-10-15 18:26:33 +01:00
miyuko
87262e82f0
Swallow DNS allowlist parsing errors if at least one record is valid.
2025-10-15 02:36:14 +01:00
miyuko
443e929dea
Respond within 10 seconds when receiving GitHub webhooks.
2025-10-13 00:52:07 +01:00
Catherine
188c66c434
Use an ad-hoc type to deserialize JSON webhook payload.
...
This is both better structured, and avoids crashes on invalid payloads
that would occur before this commit due to a lack of checking for nil
in the maps.
2025-10-09 14:53:01 +00:00
Catherine
0eede0792d
Add hostname and (if present) $PAGES_REGION to Server:.
...
Indispensable when debugging anycast configurations.
2025-10-05 08:11:52 +00:00
Catherine
1e01a12958
Implement force redirects.
2025-10-02 16:41:23 +00:00
Catherine
157ceb8342
Add an HTTP status code workaround for GitHub.
2025-10-02 12:41:15 +00:00
Catherine
f63940b459
Submit server region and machine ID to tracing.
2025-09-30 02:09:38 +00:00
Catherine
b1c50c10de
Thread context argument through the backend interface. NFC
2025-09-29 23:10:33 +00:00
Catherine
25b1720940
Compress files with Zstandard.
...
This can save as much as 30% of storage space while adding negligible
CPU overhead.
2025-09-29 01:17:25 +00:00
miyuko
1c7ef99359
Add manifest and blob metrics.
2025-09-27 18:36:17 +01:00
Catherine
80aa8e2901
Add Codeberg Pages v2 specific DNS authorization mechanism.
...
Feature-gated behind `codeberg-pages-compat` since it has no generic
use outside of that specific deployment.
2025-09-25 01:18:47 +00:00
Catherine
714e37cce8
Implement partial Netlify _redirects support.
...
This is roughly to match the Codeberg subset:
https://docs.codeberg.org/codeberg-pages/redirects/
2025-09-24 19:11:27 +00:00
Catherine
4c087278cb
Fly.io: switch health check method to [[services.http_checks]].
...
More specifically, remove the dedicated HTTP datapath for health
checks and verify the entire stack, from TLS frontend to S3 backend.
Verifying too little has resulted in a small outage recently when
the pages listener got misconfigured but the health listener happily
accepted connections like normal. This would not happen now that
the health check uses port 443, too.
2025-09-23 02:34:55 +00:00
Catherine
922cc6315a
Use filename, not URL path, for MIME type sniffing.
...
Otherwise, even though `/foo/index.html` and `/foo` are identical
resources in every way, their MIME type could differ.
2025-09-23 00:42:19 +00:00
Catherine
f0b19debdc
Make X-Pages-Update: no-change a reportable event.
2025-09-22 19:48:31 +00:00
miyuko
1aef0288e7
Add page operation metrics and expose them in Prometheus text format.
2025-09-22 19:03:59 +01:00
Catherine
584957a92d
Provide Allow: header when responding with 405 Method Not Allowed.
2025-09-22 17:41:03 +00:00
Catherine
789a5e682e
[breaking-change] Use type-safe representation for time durations.
2025-09-22 17:05:22 +00:00
Catherine
fd7b632b52
Expand Server: header to include machine ID on Fly.io.
2025-09-22 08:27:03 +00:00
Catherine
51606aac98
Replace hardcoded limits with a config file section.
2025-09-21 19:00:36 +00:00
Catherine
21227ce59f
Only send Access-Control-Allow-Origin: in response to a CORS request.
...
This saves a bit of bandwidth. NFC otherwise.
2025-09-21 08:19:08 +00:00
Catherine
382bee9b4e
Don't send Access-Control-Max-Age: in response to GET requests.
...
This header only has meaning in an `OPTIONS` response.
2025-09-21 06:05:17 +00:00
Catherine
b5ab776a23
Use Cache-Control: max-age=60, stale-while-revalidate=3600.
...
This is a way to avoid blocking on network requests in the browser
while ensuring the content is served very fresh.
2025-09-21 05:31:06 +00:00
