mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-05-22 07:42:16 +00:00
Code Issues Packages Projects Releases Wiki Activity

test/alternator: test response compression of tiny responses

Browse Source 
This patch adds to the existing collection of tests for Alternator
response compression another test with a tiny response being compressed.
This test serves two purposes:

1. It verifies setting alternator_response_compression_threshold_in_bytes
   to a tiny number like 1 really means that tiny responses would be
   compressed.

2. It verifies that our compression code, which has a special code path
   for the small chunk at the end of the compression, works correctly.

The original motivation for writing this test was a false alarm by
Claude Code which claimed that Alternator's response compression code
has a serious, exploitable, memory overrun bug, because it set the
wrong size limit on that last chunk. Claude was wrong, there is no such
bug. We did set an oversized limit on the last chunk (so this patch
fixes this typo), but it didn't matter - because the code used
deflateBound - the guaranteed maximum size of the uncompressed data -
for the buffer's size, so the buffer was unconditionally big enough,
no matter which avail_out limit we passed to delate() it could never
overflow.

The included test passes even before this patch, even with ASAN
enabled to detect memory overflows - no overflow was happening.
It also passes after the typo correction in this patch.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>

Closes scylladb/scylladb#29718
This commit is contained in:
Nadav Har'El
2026-04-30 14:15:40 +03:00
committed by Botond Dénes
parent 85374207ca
commit cd61a44ab8
2 changed files with 24 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
alternator/http_compression.cc
View File

@@ -54,14 +54,14 @@ public:
_output_buf = temporary_buffer<char>(compressed_buffer_size);
}
_zs.next_out = reinterpret_cast<unsigned char*>(_output_buf.get_write());
_zs.avail_out = compressed_buffer_size;
_zs.avail_out = _output_buf.size();
}
int e = deflate(&_zs, mode);
if (e < Z_OK) {
throw api_error::internal("Error during compression of response body");
}
if (e == Z_STREAM_END || _zs.avail_out < compressed_buffer_size / 4) {
_output_buf.trim(compressed_buffer_size - _zs.avail_out);
_output_buf.trim(_output_buf.size() - _zs.avail_out);
co_await _write_func(std::move(_output_buf));
if (e == Z_STREAM_END) {
break;

22
test/alternator/test_compressed_response.py
View File

@@ -135,6 +135,28 @@ def test_compressed_response_large(dynamodb, test_table_s, encoding):
# Not very precise, but check that some compression actually happened
assert response["DecompressedLength"] > 1.2 * response["HTTPContentLength"]
# Test with a very small compressed response (which we can force only on
# Alternator, so this is a scylla_only test). Our implementation has special
# handling for the last 1024-byte chunk of compression, so this test exercises
# this code path. This test also verifies that setting the configuration option
# alternator_response_compression_threshold_in_bytes to 1 really forces
# compression even for small responses.
@pytest.mark.parametrize("encoding", ["gzip", "deflate"])
def test_compressed_response_small(dynamodb, test_table_s, scylla_only, encoding):
p = random_string()
# A tiny value that will be returned compressed by GetItem.
small_data = 'x' * 50
test_table_s.put_item(Item={'p': p, 'x': small_data})
with client_response_decompression(dynamodb):
# Tell Alternator to compress even tiny responses:
with server_response_compression(dynamodb, compression_threshold=1):
with request_custom_headers(dynamodb, {"Accept-Encoding": encoding}):
response = test_table_s.get_item(Key={'p': p})
assert response['Item']['x'] == small_data
assert response['WasCompressed'] == encoding
# Sanity check - confirm the response was really small, as expected
assert response['DecompressedLength'] < 200
# Separate test to check that compression works also for chunked responses.
# To trigger chunked responses in Alternator, we can use the BatchGetItem with response above 100KB.
# DynamoDB probably does not use chunked responses, so we don't require it in that case.