mirrors/pages-server
1
0
Fork 0
mirror of https://codeberg.org/Codeberg/pages-server.git synced 2026-02-09 04:00:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
222 Commits 7 Branches 36 Tags
feat/redis-caching
Commit Graph

112 Commits

Author SHA1 Message Date
Moritz Marquardt
794e155169 Fix new CI issues 2024-04-18 21:17:27 +02:00
Moritz Marquardt
42a4502206 Run gofumpt 2024-04-18 19:08:47 +02:00
Moritz Marquardt
f6ed4285bc Do not cache incomplete requests 2024-04-18 19:07:34 +02:00
Moritz Marquardt
18d09a163c Use hashicorp's LRU cache for DNS & certificates 
DNS caching is also limited to 30 seconds now instead of 5 minutes
 2024-04-18 19:07:34 +02:00
Moritz Marquardt
7694deec83 Fix missing return on redis parsing issues 
Suggested-By: Gusted (https://codeberg.org/Codeberg/pages-server/pulls/301/files#issuecomment-1732054)
 2024-04-18 19:07:34 +02:00
Moritz Marquardt
5bf538c5ec Format go code 2024-04-18 19:07:34 +02:00
Moritz Marquardt
584ba5c74d Fix cached error when .domains is not readable (fixes https://codeberg.org/Codeberg/Community/issues/1512) 
Co-authored-by: @algernon
 2024-04-18 19:07:34 +02:00
Moritz Marquardt
f6d3147ba1 Fix tests not running 2024-04-18 19:07:34 +02:00
Moritz Marquardt
48e919a7bf Cache empty files & fix #303 (missing content cache) 2024-04-18 19:07:34 +02:00
Moritz Marquardt
46c8daacba Move redis config to CacheConfig struct, add cache prefixes & trace logging 2024-04-18 19:07:34 +02:00
Moritz Marquardt
e1a22d5f4c Make it possible to actually use redis for caching through the config flags 2024-04-18 19:07:34 +02:00
Moritz Marquardt
c4181d1206 Move to []byte for caching and make it compile 2024-04-18 19:07:34 +02:00
Moritz Marquardt
5b6eecc75f Add redis for caching, first try during a train ride so expect it to not be working yet 2024-04-18 19:07:34 +02:00
Jean-Marie 'Histausse' Mineau
b8b9886ee1 fix lint 2024-04-18 19:07:34 +02:00
Jean-Marie 'Histausse' Mineau
c89ce83b6b simplify wildecard logic 2024-04-18 19:07:34 +02:00
Jean-Marie 'Histausse' Mineau
03881382a4 Add option to disable DNS ACME provider (#290) 
This PR add the `$NO_DNS_01` option (disabled by default) that removes the DNS ACME provider, and replaces the wildcard certificate by individual certificates obtained using the TLS ACME provider.

This option allows an instance to work without having to manage access tokens for the DNS provider. On the flip side, this means that a certificate can be requested for each subdomains. To limit the risk of DOS, the existence of the user/org corresponding to a subdomain is checked before requesting a cert, however, this limitation is not enough for an forge with a high number of users/orgs.

Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/290
Reviewed-by: Moritz Marquardt <momar@noreply.codeberg.org>
Co-authored-by: Jean-Marie 'Histausse' Mineau <histausse@protonmail.com>
Co-committed-by: Jean-Marie 'Histausse' Mineau <histausse@protonmail.com>
 2024-04-18 17:05:20 +00:00
Hoernschen
a6e9510c07 FIX blank internal pages (#164) (#292) 
Hello 👋

since it affected my deployment of the pages server I started to look into the problem of the blank pages and think I found a solution for it:

1. There is no check if the file response is empty, neither in cache retrieval nor in writing of a cache. Also the provided method for checking for empty responses had a bug.
2. I identified the redirect response to be the issue here. There is a cache write with the full cache key (e. g. rawContent/user/repo|branch|route/index.html) happening in the handling of the redirect response. But the written body here is empty. In the triggered request from the redirect response the server then finds a cache item to the key and serves the empty body. A quick fix is the check for empty file responses mentioned in 1.
3. The decision to redirect the user comes quite far down in the upstream function. Before that happens a lot of stuff that may not be important since after the redirect response comes a new request anyway. Also, I suspect that this causes the caching problem because there is a request to the forge server and its error handling with some recursions happening before. I propose to move two of the redirects before "Preparing"
4. The recursion in the upstream function makes it difficult to understand what is actually happening. I added some more logging to have an easier time with that.
5. I changed the default behaviour to append a trailing slash to the path to true. In my tested scenarios it happened anyway. This way there is no recursion happening before the redirect.

I am not developing in go frequently and rarely contribute to open source -> so feedback of all kind is appreciated

closes #164

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/292
Reviewed-by: 6543 <6543@obermui.de>
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Hoernschen <julian.hoernschemeyer@mailbox.org>
Co-committed-by: Hoernschen <julian.hoernschemeyer@mailbox.org>
 2024-02-26 22:21:42 +00:00
crapStone
7e80ade24b Add config file and rework cli parsing and passing of config values (#263) 
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/263
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <me@crapstone.dev>
Co-committed-by: crapStone <me@crapstone.dev>
 2024-02-15 16:08:29 +00:00
crapStone
c1fbe861fe rename gitea to forge in html error messages (#287) 
closes #286

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/287
Reviewed-by: Andreas Shimokawa <ashimokawa@noreply.codeberg.org>
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
 2024-02-11 12:43:25 +00:00
Gusted
15916444e1 Fix panic in formatting not found message (#276) 
Fix panic that was found in the logs, `targetOpt` is `nil`:

http2: panic serving 10.0.3.1:[...]: runtime error: invalid memory address or nil pointer dereference
net/http.(*http2serverConn).runHandler.func1()
        /usr/local/go/src/net/http/h2_bundle.go:6104 +0x145
panic({0x19c6820, 0x2d66db0})
        /usr/local/go/src/runtime/panic.go:884 +0x213
codeberg.org/codeberg/pages/server/handler.handleSubDomain({{0x2008c68, 0xc00047df90}, 0x2, {0x0, 0x0}, {0xc0fe3ef800, 0x55, 0x1f4}, {0xc00047dfa0, 0x1, ...}, ...}, ...)
        /woodpecker/src/codeberg.org/Codeberg/pages-server/server/handler/handler_sub_domain.go:59 +0x5e0

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/276
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
 2024-01-18 14:31:46 +00:00
crapStone
fffb8ffcb6 remove use of rawInfoPage redirect (#261) 
closes #244

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/261
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
 2023-11-16 17:33:39 +00:00
crapStone
cbb2ce6d07 add go templating engine for error page and make errors more clear (#260) 
ping #199
closes #213

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/260
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
 2023-11-16 17:11:35 +00:00
crapStone
1e1c67be93 let gitea client send user-agent with version (#258) 
closes #255

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/258
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
 2023-11-15 15:25:14 +00:00
crapStone
a8272f0ce9 Don't send server version to client (#254) 
closes #247

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/254
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <crapstone01@gmail.com>
Co-committed-by: crapStone <crapstone01@gmail.com>
 2023-11-15 01:49:29 +00:00
Moritz Marquardt
56d3e291c4 Security Fix: clean paths correctly to avoid circumvention of BlacklistedPaths 2023-08-27 10:13:15 +02:00
6543
d720d25e42 Use http.NoBody as per linter (#231) 
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/231
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
 2023-07-17 19:44:58 +00:00
Gusted
7f318f89a6 Fix escaped error message (#230) 
- This specific message will [already be generated](https://codeberg.org/Codeberg/pages-server/src/commit/974229681f4cc7f1ed31df9b05eabef2df01380/html/error.go#L44) when `http.StatusMisdirectedRequest` is set as status with [an empty message](974229681f/html/error.go (L25-L28)).
- Resolves https://codeberg.org/Codeberg/pages-server/issues/228

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/230
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
 2023-07-16 22:34:46 +00:00
video-prize-ranch
974229681f Initial redirects implementation (#148) 
Adds basic support for `_redirects` files. It supports a subset of what IPFS supports: https://docs.ipfs.tech/how-to/websites-on-ipfs/redirects-and-custom-404s/

Example:
```
/redirect https://example.com/ 301
/another-redirect /page 301
/302  https://example.com/  302
/app/* /index.html 200
/articles/* /posts/:splat 301
```
301 redirect: https://video-prize-ranch.localhost.mock.directory:4430/redirect
SPA rewrite: https://video-prize-ranch.localhost.mock.directory:4430/app/path/path
Catch-all with splat: https://video-prize-ranch.localhost.mock.directory:4430/articles/path/path

Closes #46

Co-authored-by: video-prize-ranch <cb.8a3w5@simplelogin.co>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/148
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: video-prize-ranch <video-prize-ranch@noreply.codeberg.org>
Co-committed-by: video-prize-ranch <video-prize-ranch@noreply.codeberg.org>
 2023-03-30 21:36:31 +00:00
Crystal
c40dddf471 Fix certificate renewal (#209) 
A database bug in xorm.go prevents the pages-server from saving a
renewed certificate for a domain that already has one in the database.

Co-authored-by: crystal <crystal@noreply.codeberg.org>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/209
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Crystal <crystal@noreply.codeberg.org>
Co-committed-by: Crystal <crystal@noreply.codeberg.org>
 2023-03-20 22:57:26 +00:00
6543
c9050e5722 Handle Relative Symlinks (#205) 
enhance #114

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/205
 2023-03-11 05:07:17 +00:00
deblan
42d5802b9b Allow to define default branches (#125) 
This try to address #115

Co-authored-by: Simon Vieille <simon@deblan.fr>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/125
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: deblan <deblan@noreply.codeberg.org>
Co-committed-by: deblan <deblan@noreply.codeberg.org>
 2023-02-14 03:03:00 +00:00
6543
0adac9a5b1 fix http -> https redirect and add integration tests for it (#184) 
and more logging

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/184
 2023-02-14 02:23:28 +00:00
6543
42b3f8d1b7 use mockery for mock code generation (#185) 
close 	#181

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/185
 2023-02-13 23:13:30 +00:00
6543
9a3d1c36dc Document more flags & make http port customizable (#183) 
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/183
 2023-02-13 20:14:45 +00:00
crystal
46316f9e2f Fix raw domain for branches with custom domains and index.html (#159) 
fix #156
fix #157

Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/159
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crystal <crystal@noreply.codeberg.org>
Co-committed-by: crystal <crystal@noreply.codeberg.org>
 2023-02-11 03:12:42 +00:00
6543
5753f7136d Move acmeClient creation into own file & struct (#179) 
get rid of gobal vars and make make functions with less args :)

tldr: collect funcs and create a own ACME client to manage that stuff

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/179
 2023-02-11 02:29:08 +00:00
6543
fd643d15f0 Drop: pogreb support (#175) 
followup of #173

close #95

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/175
 2023-02-11 02:04:57 +00:00
6543
272c7ca76f Fix xorm regressions by handle wildcard certs correctly (#177) 
close #176

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/177
 2023-02-11 01:26:21 +00:00
6543
d8d119b0b3 Fix Cache Bug (#178) 
error io.EOF is gracefully end of file read.

so we don't need to cancel cache saving

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/178
 2023-02-11 00:31:56 +00:00
6543
7b35a192bf Add cert store option based on sqlite3, mysql & postgres (#173) 
Deprecate **pogreb**!

close #169

Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/173
 2023-02-10 03:00:14 +00:00
foehammer
7fce7cf68b Added Whitespace Trimming TXT DNS Records (#152) 
Solves https://codeberg.org/Codeberg/Community/issues/823 and https://codeberg.org/Codeberg/pages-server/issues/143

Co-authored-by: foehammer127 <foehammer127@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/152
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: foehammer <foehammer@noreply.codeberg.org>
Co-committed-by: foehammer <foehammer@noreply.codeberg.org>
 2023-02-10 01:44:44 +00:00
Gusted
8b1f497bc4 Allow to use certificate even if domain validation fails (#160) 
- Currently if the canonical domain validations fails(either for
legitimate reasons or for bug reasons like the request to Gitea/Forgejo
failing) it will use main domain certificate, which in the case for
custom domains will warrant a security error as the certificate isn't
issued to the custom domain.
- This patch handles this situation more gracefully and instead only
disallow obtaining a certificate if the domain validation fails, so in
the case that a certificate still exists it can still be used even if
the canonical domain validation fails. There's a small side effect,
legitimate users that remove domains from `.domain` will still be able
to use the removed domain(as long as the DNS records exists) as long as
the certificate currently hold by pages-server isn't expired.
- Given the increased usage in custom domains that are resulting in
errors, I think it ways more than the side effect.
- In order to future-proof against future slowdowns of instances, add a retry mechanism to the domain validation function, such that it's more likely to succeed even if the instance is not responding.
- Refactor the code a bit and add some comments.

Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/160
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
 2023-02-10 01:38:15 +00:00
Gusted
513e79832a Use correct log level for CheckCanonicalDomain (#162) 
- Currently any error generated by requesting the `.domains` file of a repository would be logged under the info log level, which isn't the correct log level when we exclude the not found error.
- Use warn log level if the error isn't the not found error.

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/162
Reviewed-by: Otto <otto@codeberg.org>
 2023-01-22 18:52:21 +00:00
crapStone
bd538abd37 Fix wrong redirect on custom domain with path (#154) 
closes #153

Co-authored-by: crapStone <crapstone01@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/154
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: crapStone <crapstone@noreply.codeberg.org>
Co-committed-by: crapStone <crapstone@noreply.codeberg.org>
 2023-01-11 00:00:37 +00:00
Felipe Leopoldo Sologuren Gutiérrez
c286b3b1d0 Added TokenBucket to limit the rate of validation failures (#151) 
Added new TockenBucket named `acmeClientFailLimit` to avoid being banned because of the [Failed validation limit](https://letsencrypt.org/docs/failed-validation-limit/) of Let's Encrypt.

The behaviour is similar to the other limiters blocking the `obtainCert` func ensuring rate under limit.

Co-authored-by: fsologureng <sologuren@estudiohum.cl>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/151
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Felipe Leopoldo Sologuren Gutiérrez <fsologureng@noreply.codeberg.org>
Co-committed-by: Felipe Leopoldo Sologuren Gutiérrez <fsologureng@noreply.codeberg.org>
 2023-01-04 05:26:14 +00:00
Gusted
98d198d419 Safely get certificate's leaf (#150) 
- It's not guaranteed that `tls.X509KeyPair` will set `c.Leaf`.
- This patch fixes this by using a wrapper that parses the leaf
certificate(in bytes) if `c.Leaf` wasn't set.
- Resolves #149

Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/150
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
 2023-01-04 04:51:27 +00:00
jklippel
caeb1a4acb Return a 404 if there is no repository (#141) 
If no repository is found the user expects a 404 status code
instead of a dependency failed status code (as it was before).

Signed-off-by: Jan Klippel <c0d3b3rg@kl1pp3l.de>

Fixes: https://codeberg.org/Codeberg/Community/issues/809

Co-authored-by: Jan Klippel <c0d3b3rg@kl1pp3l.de>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/141
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: jklippel <jklippel@noreply.codeberg.org>
Co-committed-by: jklippel <jklippel@noreply.codeberg.org>
 2022-11-22 21:26:10 +00:00
Gusted
f2f943c0d8 Remove unnecessary conversion (#139) 
- Remove unnecessary type conversion.
- Enforce via CI

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/139
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
 2022-11-15 16:15:11 +01:00
Gusted
3c61a39864 Enable http/2 support (#137) 
As per [the documentation](https://pkg.go.dev/net/http#Serve), it doesn't enable HTTP2 by-default, unless we enable it via the `NextProtos` option.

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/137
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
 2022-11-12 22:25:20 +01:00
6543
4565481643 refactor: finish use default const for defaultPagesBranch and defaultPagesRepo 2022-11-12 21:16:11 +01:00