Commit Graph

533 Commits

Author SHA1 Message Date
Catherine ec66cdb6b4 Check git blob size against cumulative limit before reading it.
This avoids exhausting RAM when reading e.g. a repository with a single
extremely large file. Note that there is still a risk of exhausting
space in `/tmp`.

V12-Ref: F-77211
2026-05-30 18:10:43 +00:00
Catherine 577bd04d53 Limit how long forge API responses can be.
V12-Ref: F-77222
2026-05-30 18:10:43 +00:00
Catherine bde52a911e Limit cardinality for git_pages_http_request_* metrics.
V12-Ref: F-77243
2026-05-30 18:10:43 +00:00
Catherine 101dba6d03 Fix bits/bytes mixup in bloom filter configuration.
V12-Ref: F-77215
2026-05-30 18:10:43 +00:00
Catherine 32300bc16c Fix goroutine leak in POST webhook handler.
In Go, unsized channels are unbuffered and require the sender and
the receiver to rendezvous, or the sender blocks. If the receiver
never listens the sender cannot know this.

In Rust, unsized channels have an unbounded buffer and sending into
a channel with no receiver returns an error.

I got confused by the difference in semantics.

V12-Ref: F-77239
2026-05-30 18:10:43 +00:00
Catherine 3fe45f0c98 Add a comment on threat model (lack thereof) for Basic-Auth:.
V12-Ref: F-77238, F-77261
2026-05-30 18:10:43 +00:00
Catherine c181e86f48 Fix minor resource leak in notifyAudit.
This would leak a response body on each valid, non-200 status response.

V12-Ref: F-77171
2026-05-30 18:10:43 +00:00
Catherine 00567a5257 Treat empty PAGES_* environment variables as nonexistent.
The behavior for lists is confusing otherwise: it results in `[""]`.

V12-Ref: F-77212
2026-05-30 18:10:43 +00:00
Catherine 16505f6054 [security] Actually check result of appendNewAuditRecord.
Before this commit, if `backend.AppendAuditLog` failed, the request
would proceed anyway. This is contrary to the explicit contract and
design intent.

If Go had `#[must_use]`, this would not have happened :/

V12-Ref: F-77170
2026-05-30 18:10:43 +00:00
Catherine 0cd96a2b0b Fix incorrect derivation of endpoint from fallback URL.
The port would be appended twice if you configure e.g.

    [fallback]
    proxy-to = "https://codeberg.page:443"

V12-Ref: F-77193
2026-05-30 18:10:43 +00:00
Catherine 8883c78250 Pass context to tls.Dialer in tryDialWithSNI.
This cleans up resources that would otherwise be tied up by Caddy
endpoint requests where the originating TLS connection to Caddy has
went away.

V12-Ref: F-77195
2026-05-30 18:10:43 +00:00
Catherine c8dba5dcb5 Fix opaque panic when ingesting an invalid tar file.
Use PUT to upload the following tar file (`unzstd | base64 -d`):

KLUv/QRY7QIAcoQOFLCnDQ0QaaURkYASyN1LJveuZAKkXivfoQMXZ5MhIGJAXHUWHclJufKB
PLvNDSbmD81Htf9W1f/3BgsA/QPwwAuojAHiDA8mpAEqhsJB8IUcTATEusLVn0AbU7ZnkA==

After this commit it should no longer crash the handler.

V12-Ref: F-77219
2026-05-30 18:10:43 +00:00
Catherine 970806ab4a Fix opaque panic on invariant violation in ApplyTarPatch.
To reproduce, use PUT to upload this archive (`unzstd | base64 -d`):

KLUv/QRY7QIAxAJhL2IAMDAwMDY0NDAwMDAwMDEAADAwNzU2MAAgMAB1c3RhcgAwAGEAMzM3
YREA/UEF/EC9Y0AdDJBP8GDCTaDGBxATkAAd3gJoMPAbJANAciACGDTAsXKZngAR/m3nXA==

then issue any PATCH request to that site.

After this commit, the server returns "malformed manifest (not
a directory)" instead of "assignment to entry in nil map".

While ideally incoming manifests should be checked for consistency
regardless of how they're uploaded, in practice this is only a self-DoS
so it's probably not worth fixing.

V12-Ref: F-77244
2026-05-30 18:10:43 +00:00
Catherine 0849735f9e [security] Fix an issue with pull request preview authorization.
Pull request number was compared, but pull request owner and repository
name were not. As a result you could overwrite any preview site with
the matching PR number.

This functionality is feature-gated and there are no known usable
deployments at the moment.

V12-Ref: F-77256
2026-05-30 18:04:13 +00:00
Catherine 43088db596 [breaking-change] Replace -size-histogram with -analyze-storage.
The old function did not even draw a histogram (it was a bar chart),
and would essentially always overcount sizes.

The new function is always accurate and just as useful at a glance.
It provides two modes, `text` (optionally colorized) and `json`.
2026-05-29 08:08:53 +00:00
Catherine 7e72765dd1 Simplify TraceGarbage. NFC 2026-05-29 06:17:16 +00:00
Catherine 6f84e0f0d4 Allow limiting maximum lifetime of preview sites. 2026-05-28 22:20:49 +00:00
Catherine a7063e00ef Implement site expiration.
Requires `feature = ["expiration"]`.
2026-05-29 00:13:44 +02:00
Catherine 9113025646 Fix typo in package name. NFC 2026-05-28 21:49:23 +00:00
Catherine af333e3d15 Stabilize existence-cache feature.
Also, make the logging a bit quieter.
2026-05-28 23:45:50 +02:00
Catherine f2811a4947 log.Printlog.Println. NFC 2026-05-28 23:45:50 +02:00
Catherine 4d9872067d Expose the binary serialization of the manifest.
This action implies a commitment to maintain the binary serialization
in a forward-compatible way (which is really the status quo).
2026-05-28 23:44:01 +02:00
Catherine de3162bba2 Move OS-related modules into a sub-package. NFC
Historically git-pages did not use those, but the codebase is growing
a bit out of hand and it seems like a good place to start.
2026-05-28 15:37:51 +02:00
Catherine 6e232b5229 Add an index of all known features.
This helps avoid incorrect behavior on typos and notifies end users
that a feature has been stabilized and removed. It also helps us avoid
reusing feature names by accident.
2026-05-28 12:01:47 +00:00
Catherine 3efb332351 Implement pull request preview authorization.
Requires `feature = ["preview"]`.
2026-05-25 23:59:39 +00:00
Catherine 24a54f6fce Factor out forge API integration code. NFC 2026-05-25 21:27:59 +00:00
miyuko ddfa41dadc Allow forge auth for any repo in the forge user's namespace. 2026-05-23 02:37:10 +01:00
Catherine 19332e750f Actually skip creating existence cache for FS backend.
Otherwise, if the feature is enabled, the cache would be refreshed
on every query.
2026-05-19 12:22:42 +00:00
Catherine 9af5565659 Make the existence cache more type-safe.
This makes the uncertain nature of a Bloom filter hit explicit at every
call site.
2026-05-19 12:22:39 +00:00
Catherine 0d24e1aa70 Rename "site existence cache" to "existence cache", tidy it up.
This commit includes no behavioral changes, only cosmetic ones:
 * Renames the concept to "existence cache".
 * Makes log messages more concise.
 * Adds written rationale for the module.
 * Renames feature to `existence-cache`.
2026-05-19 12:22:22 +00:00
miyuko c1400d5934 Add site-level granularity to the domain existence cache. 2026-05-19 05:07:06 +01:00
miyuko f096666829 Remove useless check. 2026-05-19 04:28:38 +01:00
Catherine e40456b51c Correctly handle PATCH requests that overwrite the root node. 2026-05-17 06:46:23 +00:00
Catherine b9165ba288 [breaking-change] Reorder -audit-log columns for readability. 2026-05-14 15:47:44 +02:00
Catherine f1e773b749 Make PAGES_INSECURE bypass [limit].allowed-repository-url-prefixes.
This is the intended behavior but it was accidentally broken
in commit 2c109a5e1e.
2026-05-14 13:16:25 +00:00
Catherine 55f87083e5 [security] Fix false positives on Caddy endpoint due to domain cache.
In commit bbdaae7280, a domain cache was
introduced to deal with misbehaving crawlers that forge `Host:` header
and may cause thousands of expensive S3 requests to be submitted.
This domain cache is implemented using a Bloom filter (which can
produce false positives but not false negatives) for S3 backend, and
using a function always returning true (which will be a false positive
in most cases) for the FS backend.

Both of these behaviors are unacceptable for the Caddy endpoint, but
the FS backend case much more so. If you use git-pages with Caddy you
should upgrade to a build that includes this commit as soon as possible
or Let's Encrypt may rate-limit or restrict your account when you get
unlucky with a crawler.
v0.9.0
2026-05-11 10:26:53 +00:00
miyuko a9fc5780b1 Record git repo URL in the principal when forge auth is used.
Resolves: https://codeberg.org/git-pages/git-pages/issues/167
2026-05-11 03:40:46 +01:00
miyuko ad92847fa0 Record git repo URL in manifest for archive uploads with forge auth.
Resolves: https://codeberg.org/git-pages/git-pages/issues/165
2026-05-11 03:38:54 +01:00
Catherine 3311fb639d Fix incorrect example use case of _headers.
git-pages enables CORS automatically and unconditionally, but COOP/COEP
have to be configured manually.
2026-05-05 03:12:51 +00:00
Catherine 93ce4f9671 Bring authorization flow documentation up to date. 2026-05-05 02:56:08 +00:00
Catherine 73e47cd8d5 Significantly improve efficiency of tracing.
I thought I was being smart by using a trie to record blob existence
and sizes. I was not. The trie approach had at least ~5 times less
throughput and consumed entirely unreasonable amounts of RAM.

A hashmap works just fine here.
2026-05-05 01:57:41 +00:00
Catherine dd7268a657 Fix typo. 2026-05-03 12:30:17 +00:00
Catherine edae862551 Surface detached status of audit records in diagnostic output. 2026-05-03 12:01:33 +00:00
miyuko 5808e90e5a Allow detaching all audit records related to a site. 2026-05-03 11:53:01 +00:00
miyuko 684553ba72 Replace pelletier/go-toml with BurntSushi/toml.
Most of the code was borrowed from
https://codeberg.org/whitequark/ircv3-filehost-server.
2026-04-27 18:26:33 +01:00
miyuko 89f672beda Allow detaching audit records from their blobs for garbage collection.
Resolves: https://codeberg.org/git-pages/git-pages/issues/148
2026-04-27 17:29:16 +01:00
miyuko a233cdfbb8 Fix S3Backend.SearchAuditLog ignoring search options. 2026-04-27 16:48:36 +01:00
Catherine 4d8e620846 Account for manifest's own size for -size-histogram. 2026-04-27 01:16:16 +02:00
Catherine e8112c1abe Add a CLI command -audit-expire to purge old audit records.
This is particularly important with the FS backend, where there isn't
necessarily native tooling capable of handling this task correctly
(since not every filesystem supports file "birth times", and since
restoring data from a backup will reset the "birth time" of audit
records to the moment of restoration).
2026-04-26 23:10:22 +00:00
Catherine b0a674abf4 Fix incorrect start time in AuditID.CompareTime. 2026-04-26 22:59:36 +00:00