fix: add keeplive option (CLI and env var)

This fix enables Versity Gateway to serve clients that use the AWS C++ SDK - without enabling keepalive in the fiber connection, clients that use the AWS C++ SDK like Run:ai's model streamer [will wig out from all of the closed connections and fail to function](https://github.com/run-ai/runai-model-streamer/issues/55) when connecting to a Versity GW back end. 

This fix is intentionally side-effect free in that it retains the current default behavior, with the ability to override it via an env var or CLI arg
This commit is contained in:
Rich Siomporas
2025-08-26 21:47:19 -04:00
committed by GitHub
parent 45a4d1892f
commit 6a82213606

View File

@@ -54,6 +54,7 @@ var (
healthPath string
virtualDomain string
debug bool
keepAlive bool
pprof string
quiet bool
readonly bool
@@ -223,6 +224,12 @@ func initFlags() []cli.Flag {
EnvVars: []string{"VGW_PPROF"},
Destination: &pprof,
},
&cli.BoolFlag{
Name: "keep-alive",
Usage: "enable keep-alive connections (for finnicky clients)",
EnvVars: []string{"VGW_KEEP_ALIVE"},
Destination: &keepAlive,
},
&cli.BoolFlag{
Name: "quiet",
Usage: "silence stdout request logging output",
@@ -604,7 +611,7 @@ func runGateway(ctx context.Context, be backend.Backend) error {
AppName: "versitygw",
ServerHeader: "VERSITYGW",
StreamRequestBody: true,
DisableKeepalive: true,
DisableKeepalive: !keepAlive,
Network: fiber.NetworkTCP,
DisableStartupMessage: true,
})