Files
remark42/backend/app/store/image/image_test.go
T
Dmitry VerkhoturovandGitHub 0e20861419 fix(security): reject non-image content-types in image proxy and /picture/ to prevent stored XSS (#2067)
* fix(security): reject non-image content-types in image proxy and /picture/ to prevent stored XSS

The /api/v1/img proxy and /api/v1/picture/{user}/{id} endpoints emitted
http.DetectContentType on the served bytes as the response Content-Type. A
controlled upstream serving Content-Type: image/png with an HTML body passed
the upstream check (only the response header was inspected, not the body),
and the body bytes then sniffed back to text/html — so the proxy served the
attacker's HTML from the remark42 origin. Browsers honoured the declared
text/html and executed the response as a document with access to cookies and
CSRF tokens. Affected from v1.6.0 (April 2020) through v1.15.0; verified live
via published docker images.

Layered defense applied to both handlers:

- rest.SafeImgContentType (in backend/app/rest/) validates sniffed content
  against a strict allowlist: image/png, image/jpeg, image/gif, image/webp,
  image/bmp, image/x-icon. Anything else (HTML, XML, SVG, plain text,
  octet-stream, or any future image type the stdlib sniffer may learn) is
  rejected with no body echo. SVG is implicitly excluded — it sniffs as
  text/xml or text/plain, never image/svg+xml, and SVG can execute scripts
  when navigated to top-level. The previous octet-stream → image/* fallback
  is gone.
- Per-endpoint Content-Security-Policy override sets
  "default-src 'none'; sandbox; frame-ancestors 'none'" on every response
  (success, 304, or error). Sandbox neuters scripts even if Content-Type
  ever regresses. The same policy is also applied to all /api/v1/* via
  apiCSPMiddleware as defense-in-depth.
- Content-Disposition: inline; filename="image" frames the response as a
  file rather than a renderable document.
- /picture/ rejection paths set Cache-Control: no-store so 4xx responses
  are never cached.

The defense headers and the strict ETag matcher are extracted as
rest.SetImageDefenseHeaders and rest.EtagMatches in the shared rest package
(consumed by both proxy/image and api/rest_public — no package cycle).

The /api/v1/img path additionally bumps the ETag to a versioned `"v2:..."`
so revalidating clients (top-level navigation, Ctrl+R, intermediaries) get
a fresh 200 instead of a 304 against poisoned pre-fix cached HTML.

DELIBERATE TRADEOFF: Cache-Control on /api/v1/img success responses remains
max-age=2592000 (30 days), unchanged from before. An aggressive "force
revalidate on every reuse" policy was prototyped during review but reverted
because the perf cost (a server round-trip on every image view, even with
304 saving the body bytes) outweighed the corner-case mitigation. The
realistic exposure of cache carryover is narrow: cache carryover only
affects users who navigated top-level to an attacker URL pre-fix and still
have it in their local cache — the normal <img> embed path cached text/html
but never executed it. Local browser caches that hold pre-fix bytes
continue to serve them until their 30-day TTL expires or are evicted under
memory pressure. The ETag bump reaches all clients that DO revalidate
during the cached lifetime (Ctrl+R, intermediaries, post-expiry use); for
the rest, exposure self-limits via cache expiry. Operators running a
CDN/edge cache in front of remark42 should purge /api/v1/img after deploy.

The /api/v1/img handler short-circuits on a matching current-version
If-None-Match before any store Load or upstream fetch, returning a bodyless
304 with the defense headers set. Safe because the 304 carries no body and
the client's cached bytes came from a prior validated 200; an attacker
fabricating an etag value can only short-circuit fetches for URLs they
themselves crafted. This avoids upstream DoS amplification when clients
revalidate on hot comment pages.

The /api/v1/img route was moved from the "open routes" group (which uses
middleware.NoCache, stripping If-None-Match from incoming requests) to the
"open routes, cached" group alongside /picture/ and /qr/telegram so the
304 revalidation path is no longer broken upstream of the handler.

The /picture/{user}/{id} endpoint does not need the v2 etag prefix. Upload
validates input format via readAndValidateImage and the serve path
re-validates the stored bytes via rest.SafeImgContentType. Bytes within
the resize dimension limits are preserved verbatim, so the browser defense
relies on the response headers (validated Content-Type + nosniff + strict
CSP + Content-Disposition: inline), not on byte normalization.

Global CSP: font-src data: → font-src 'none'. Audit confirmed no @font-face,
no base64 fonts, no icon-font library in the bundle. Drops an unnecessary
attack surface; no behavioural change.

Tests: TestImage_ContentTypeHandling table-tests a real PNG and attack
shapes (HTML claimed as image/png, image/jpeg, image/gif, image/svg+xml,
image/webp; svg with onload; html fragment; polyglot PNG+HTML), proving
the defense holds across arbitrary upstream Content-Type variation.
Polyglot case is intentionally served as image/png — the browser cannot
execute the trailing HTML when the response type is image/png with nosniff.
TestImage_ContentTypeHandling_CacheHit exercises the cache-hit branch with
attacker bytes preloaded into the store. TestImage_PerRequestRevalidation
alternates upstream PNG/HTML across four proxy calls to prove no trust
accumulates between requests. TestImage_RoutesUsingCachedImage asserts
cache-poisoning is caught at serve time. TestImage_EtagVersioned asserts
the v2 prefix invalidates pre-fix etags AND that the revalidation 304
triggers no store Load. TestImage_RevalidationSkipsIO proves the
short-circuit works even with no upstream reachable. TestSafeImgContentType
covers the allowlist directly. TestRest_LoadPictureDefenseHeaders and
TestRest_LoadPictureRejectsNonImage exercise the /picture/ endpoint.
TestRest_apiCSP covers the strict CSP middleware on JSON API + RSS routes;
TestRest_securityHeaders confirms /web/ HTML pages keep the global CSP.

Verified end-to-end against the dev docker image: the original demo URL
(arbitrary HTML claimed as image/png) now returns 415 application/json with
CSP/nosniff/Content-Disposition set, no XSS in the browser.

* fix(security): set Cache-Control: no-store on image-proxy error paths, sync stale route comment

Addresses two review comments on #2067:

1. Cache-Control: max-age=2592000 and Etag were set before the
   load/download/validation block, so 404/400/415 error responses inherited
   the 30-day cache TTL and the versioned etag — a transient failure (or an
   intentionally triggered 415) would be pinned in browser/intermediary
   caches for that TTL, keeping users locked out even after the underlying
   cause was resolved. Now: etag is computed but not set as a header until
   after validation succeeds; error paths route through sendImageProxyError
   which sets Cache-Control: no-store and never sets Etag. The 304
   short-circuit still sets both because that path serves the same validated
   content the client already has cached.

2. The comment at rest.go:282 still described the prototyped
   no-cache/must-revalidate Cache-Control policy that was reverted before
   the PR landed. Updated to match the actual 30-day max-age behavior.

Tests: TestImage_ContentTypeHandling now asserts reject paths carry
Cache-Control: no-store and have no Etag header, and accept paths carry
the max-age=2592000 + v2: etag.
2026-05-20 22:37:25 -05:00

365 lines
14 KiB
Go

package image
import (
"bytes"
"context"
"encoding/base64"
"fmt"
"image"
"io"
"os"
"strconv"
"strings"
"testing"
"testing/synctest"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestService_SaveAndLoad(t *testing.T) {
store := StoreMock{
SaveFunc: func(string, []byte) error {
return nil
},
LoadFunc: func(string) ([]byte, error) {
return nil, nil
},
}
svc := NewService(&store, ServiceParams{MaxSize: 1500, MaxWidth: 32, MaxHeight: 32})
err := svc.SaveWithID("test_id", gopherPNG())
assert.NoError(t, err)
assert.Equal(t, 1, len(store.SaveCalls()))
assert.Equal(t, "test_id", store.SaveCalls()[0].ID)
img, err := svc.Load("test_id")
assert.NoError(t, err)
assert.Nil(t, img)
assert.Equal(t, 1, len(store.LoadCalls()))
assert.Equal(t, "test_id", store.LoadCalls()[0].ID)
}
func TestService_Resize(t *testing.T) {
img, err := readAndValidateImage(gopherPNG(), 1500)
assert.NoError(t, err)
assert.Equal(t, 1462, len(img))
img = resize(img, 32, 32)
assert.Equal(t, 1135, len(img))
}
func TestService_ResizeJpeg(t *testing.T) {
fh, err := os.Open("testdata/circles.jpg")
defer func() { assert.NoError(t, fh.Close()) }()
assert.NoError(t, err)
img, err := readAndValidateImage(fh, 32000)
assert.NoError(t, err)
assert.InDelta(t, 16756, len(img), 100)
img = resize(img, 400, 300)
assert.InDelta(t, 10913, len(img), 100)
}
func TestService_SaveTooLarge(t *testing.T) {
svc := Service{ServiceParams: ServiceParams{ImageAPI: "/blah/"}}
svc.MaxSize = 2000
_, err := svc.Save("user2", io.MultiReader(gopherPNG(), gopherPNG()))
assert.Error(t, err)
assert.Contains(t, err.Error(), "is too large")
err = svc.SaveWithID("test_id", io.MultiReader(gopherPNG(), gopherPNG()))
assert.Error(t, err)
assert.Contains(t, err.Error(), "is too large")
}
func TestService_WrongFormat(t *testing.T) {
svc := Service{ServiceParams: ServiceParams{ImageAPI: "/blah/"}}
_, err := svc.Save("user1", strings.NewReader("blah blah bad image"))
assert.Error(t, err)
}
func TestService_ExtractPictures(t *testing.T) {
svc := Service{ServiceParams: ServiceParams{ImageAPI: "/blah/", ProxyAPI: "/non_existent"}}
html := `blah <img src="/blah/user1/pic1.png"/> foo
<img src="/blah/user2/pic3.png"/> xyz <p>123</p> <img src="/pic3.png"/> <img src="https://i.ibb.co/0cqqqnD/ezgif-5-3b07b6b97610.png" alt="">`
ids := svc.ExtractPictures(html)
require.Equal(t, 2, len(ids), "two images")
assert.Equal(t, "user1/pic1.png", ids[0])
assert.Equal(t, "user2/pic3.png", ids[1])
svc = Service{ServiceParams: ServiceParams{ImageAPI: "https://remark42.radio-t.com/api/v1/picture/", ProxyAPI: "https://remark42.radio-t.com/api/v1/img"}}
html = `<p>TLDR: такое в go пока правильно посчитать трудно. То, что они считают это общее количество go packages в коде.
</p>\n\n<p>Пакеты в го это средство организации кода, они могут быть связанны друг с другом в рамках одной библиотеки (модуля).
Например одна из моих вот так выглядит на libraries.io:</p>\n\n
<p><img src="https://remark42.radio-t.com/api/v1/picture/github_ef0f706a79cc24b17bbbb374cd234a691d034128/bjttt8ahajfmrhsula10.png" alt="bjtr0-201906-08110846-i324c.png"/></p>\n\n<p>
По форме все верно, это все packages, но по сути это все одна библиотека организованная таким образом. При ее импорте, например посредством go mod, она выглядит как один модуль, т.е.
<code>github.com/go-pkgz/auth v0.5.2</code>.</p>\n`
ids = svc.ExtractPictures(html)
require.Equal(t, 1, len(ids), "one image in")
assert.Equal(t, "github_ef0f706a79cc24b17bbbb374cd234a691d034128/bjttt8ahajfmrhsula10.png", ids[0])
// proxied image
html = `<img src="https://remark42.radio-t.com/api/v1/img?src=aHR0cHM6Ly9ob21lcGFnZXMuY2FlLndpc2MuZWR1L35lY2U1MzMvaW1hZ2VzL2JvYXQucG5n" alt="cat.png">`
ids = svc.ExtractPictures(html)
require.Equal(t, 1, len(ids), "one image in")
assert.Equal(t, "cached_images/12318fbd4c55e9d177b8b5ae197bc89c5afd8e07-a41fcb00643f28d700504256ec81cbf2e1aac53e", ids[0])
require.Empty(t, svc.ExtractNonProxiedPictures(html), "no non-proxied images expected to be found")
// bad url
html = `<img src=" https://remark42.radio-t.com/api/v1/img">`
ids = svc.ExtractPictures(html)
require.Empty(t, ids)
// bad src
html = `<img src="https://remark42.radio-t.com/api/v1/img?src=bad">`
ids = svc.ExtractPictures(html)
require.Empty(t, ids)
// good src with bad content
badURL := base64.URLEncoding.EncodeToString([]byte(" http://foo.bar"))
html = fmt.Sprintf(`<img src="https://remark42.radio-t.com/api/v1/img?src=%s">`, badURL)
ids = svc.ExtractPictures(html)
require.Empty(t, ids)
}
func TestService_Cleanup(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
store := StoreMock{
CleanupFunc: func(context.Context, time.Duration) error {
return nil
},
}
svc := NewService(&store, ServiceParams{EditDuration: 20 * time.Millisecond})
// cancel context after 2.1 cleanup TTLs
ctx, cancel := context.WithTimeout(context.Background(), svc.EditDuration/100*15*21)
defer cancel()
svc.Cleanup(ctx)
assert.Equal(t, 2, len(store.CleanupCalls()))
})
}
func TestService_Submit(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
store := StoreMock{
CommitFunc: func(string) error { return nil },
ResetCleanupTimerFunc: func(string) error { return nil },
}
svc := NewService(&store, ServiceParams{ImageAPI: "/blah/", EditDuration: time.Millisecond * 100})
svc.Submit(func() []string { return []string{"id1", "id2", "id3"} })
assert.Equal(t, 3, len(store.ResetCleanupTimerCalls()))
err := svc.Commit(func() []string { return []string{"id4", "id5"} })
assert.NoError(t, err)
svc.Submit(func() []string { return []string{"id6", "id7"} })
assert.Equal(t, 5, len(store.ResetCleanupTimerCalls()))
svc.Submit(nil)
assert.Equal(t, 2, len(store.CommitCalls()))
time.Sleep(time.Millisecond * 175)
assert.Equal(t, 7, len(store.CommitCalls()))
svc.Close(context.TODO())
})
}
func TestService_Close(t *testing.T) {
store := StoreMock{
CommitFunc: func(string) error { return nil },
ResetCleanupTimerFunc: func(string) error { return nil },
}
svc := Service{store: &store, ServiceParams: ServiceParams{ImageAPI: "/blah/", EditDuration: time.Hour * 24}}
svc.Submit(func() []string { return []string{"id1", "id2", "id3"} })
svc.Submit(func() []string { return []string{"id4", "id5"} })
svc.Submit(nil)
assert.Equal(t, 5, len(store.ResetCleanupTimerCalls()))
svc.Close(context.TODO())
assert.Equal(t, 5, len(store.CommitCalls()))
}
func TestService_SubmitDelay(t *testing.T) {
synctest.Test(t, func(t *testing.T) {
store := StoreMock{
CommitFunc: func(string) error { return nil },
ResetCleanupTimerFunc: func(string) error {
return nil
},
}
svc := NewService(&store, ServiceParams{EditDuration: 20 * time.Millisecond})
svc.Submit(func() []string { return []string{"id1", "id2", "id3"} })
time.Sleep(150 * time.Millisecond) // let first batch to pass TTL
svc.Submit(func() []string { return []string{"id4", "id5"} })
svc.Submit(nil)
assert.Equal(t, 5, len(store.ResetCleanupTimerCalls()))
assert.Equal(t, 3, len(store.CommitCalls()))
svc.Close(context.TODO())
assert.Equal(t, 5, len(store.CommitCalls()))
})
}
func TestService_Info(t *testing.T) {
store := StoreMock{InfoFunc: func() (StoreInfo, error) {
return StoreInfo{}, nil
}}
svc := Service{store: &store, ServiceParams: ServiceParams{}}
info, err := svc.Info()
assert.NoError(t, err)
assert.True(t, info.FirstStagingImageTS.IsZero())
assert.Equal(t, 1, len(store.InfoCalls()))
}
func TestService_resize(t *testing.T) {
t.Run("empty data returns nil", func(t *testing.T) {
assert.Nil(t, resize(nil, 100, 100))
assert.Nil(t, resize([]byte{}, 100, 100))
})
t.Run("non-image bytes are refused", func(t *testing.T) {
// previously resize would fall back to the raw bytes on decode failure, letting
// attacker-controlled non-image content reach the store. After hardening, refuse.
assert.Nil(t, resize([]byte("some picture bin data"), -1, -1))
assert.Nil(t, resize([]byte("invalid image content"), 100, 100))
})
cases := []struct {
file string
wr, hr int
}{
{"testdata/circles.png", 400, 300}, // full size: 800x600 px
{"testdata/circles.jpg", 300, 400}, // full size: 600x800 px
}
for _, c := range cases {
img, err := os.ReadFile(c.file)
require.NoError(t, err, "can't open test file %s", c.file)
// no need for resize, image dimensions are smaller than resize limit
resized := resize(img, 800, 800)
assert.NotNil(t, resized, "file %s", c.file)
assert.Equal(t, resized, img)
// resizing to half of width
resized = resize(img, 400, 400)
assert.NotNil(t, resized, "file %s", c.file)
imgRz, format, err := image.Decode(bytes.NewBuffer(resized))
assert.NoError(t, err, "file %s", c.file)
assert.Equal(t, "png", format, "file %s", c.file)
bounds := imgRz.Bounds()
assert.Equal(t, c.wr, bounds.Dx(), "file %s", c.file)
assert.Equal(t, c.hr, bounds.Dy(), "file %s", c.file)
}
}
// TestService_SaveWithIDShortPayload guards readAndValidateImage from panicking
// on a body shorter than 512 bytes — historically it sliced data[:512] without
// a bounds check, which would panic before any decode-bomb defense could fire.
func TestService_SaveWithIDShortPayload(t *testing.T) {
short := []byte("not an image")
svc := Service{ServiceParams: ServiceParams{ImageAPI: "/blah/", MaxSize: 1500, MaxWidth: 32, MaxHeight: 32}}
err := svc.SaveWithID("test_id", bytes.NewReader(short))
require.Error(t, err, "short non-image body must return an error, not panic")
assert.Contains(t, err.Error(), "file format not allowed")
}
// TestService_SaveWithIDWebP confirms that WebP — listed as an allowed format
// in readAndValidateImage — still round-trips through prepareImage now that
// resize() runs image.DecodeConfig. Without registering the WebP decoder, a
// legitimate WebP upload would fail DecodeConfig and prepareImage would error.
func TestService_SaveWithIDWebP(t *testing.T) {
webp, err := os.ReadFile("testdata/pixel.webp")
require.NoError(t, err)
// sanity: the fixture must be a well-formed 1x1 WebP that DecodeConfig accepts.
cfg, format, err := image.DecodeConfig(bytes.NewReader(webp))
require.NoError(t, err)
require.Equal(t, "webp", format)
require.Equal(t, 1, cfg.Width)
require.Equal(t, 1, cfg.Height)
store := StoreMock{SaveFunc: func(string, []byte) error { return nil }}
svc := Service{store: &store, ServiceParams: ServiceParams{MaxSize: 1500}}
err = svc.SaveWithID("webp_id", bytes.NewReader(webp))
require.NoError(t, err, "valid WebP must round-trip through SaveWithID")
assert.Equal(t, 1, len(store.SaveCalls()))
assert.Equal(t, webp, store.SaveCalls()[0].Img, "no-resize path must return bytes verbatim")
}
// TestService_resizeRejectsDecompressionBomb verifies the dimension-cap defense.
// Builds a tiny GIF that declares 65535x65535 (4 gigapixels) in its logical-screen
// header — image.DecodeConfig reads the dimensions, the int64 product overflows
// any 32-bit int wrap, and resize must refuse before image.Decode allocates ~17 GB
// of pixel memory.
func TestService_resizeRejectsDecompressionBomb(t *testing.T) {
// minimal GIF87a header with 65535x65535 logical screen, no global color table.
// Bytes 6-7 are the little-endian width, 8-9 are the little-endian height.
bomb := []byte{
'G', 'I', 'F', '8', '7', 'a',
0xFF, 0xFF,
0xFF, 0xFF,
0x00,
0x00,
0x00,
0x3B,
}
cfg, _, err := image.DecodeConfig(bytes.NewReader(bomb))
require.NoError(t, err, "bomb header must decode at the config level")
assert.Equal(t, 65535, cfg.Width)
assert.Equal(t, 65535, cfg.Height)
assert.Nil(t, resize(bomb, 100, 100), "resize must refuse oversized dimensions before raster decode")
assert.Nil(t, resize(bomb, 0, 0), "even with no-resize limits, oversized dims must be refused")
// integration-level: SaveWithID must reject the same bomb without panicking
// or allocating gigabytes of raster memory.
store := StoreMock{SaveFunc: func(string, []byte) error { return nil }}
svc := Service{store: &store, ServiceParams: ServiceParams{MaxSize: 1500}}
err = svc.SaveWithID("bomb_id", bytes.NewReader(bomb))
require.Error(t, err, "SaveWithID must reject decompression bomb")
assert.Equal(t, 0, len(store.SaveCalls()), "rejected bomb must not be stored")
}
func TestGetProportionalSizes(t *testing.T) {
tbl := []struct {
inpW, inpH int
limitW, limitH int
resW, resH int
}{
{10, 20, 50, 25, 10, 20},
{400, 200, 50, 25, 50, 25},
{100, 100, 50, 25, 25, 25},
{100, 200, 50, 25, 12, 25},
}
for i, tt := range tbl {
t.Run(strconv.Itoa(i), func(t *testing.T) {
resW, resH := getProportionalSizes(tt.inpW, tt.inpH, tt.limitW, tt.limitH)
assert.Equal(t, tt.resW, resW, "width")
assert.Equal(t, tt.resH, resH, "height")
})
}
}
func TestCachedImgID(t *testing.T) {
img, err := CachedImgID(" http://foo.com")
assert.Error(t, err)
assert.Empty(t, img)
imgURL := "http://example.org/img/1.png"
img, err = CachedImgID(imgURL)
assert.NoError(t, err)
assert.Equal(t, "cached_images/"+Sha1Str("example.org")+"-"+Sha1Str(imgURL), img)
}
func TestService_DoubleClose(*testing.T) {
store := StoreMock{}
svc := NewService(&store, ServiceParams{EditDuration: 20 * time.Millisecond})
svc.Close(context.TODO())
// second call should not result in panic
svc.Close(context.TODO())
}
// TestSafeImgContentType now lives in the rest package alongside the SafeImgContentType
// helper itself (see backend/app/rest/image_headers_test.go).