mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-27 11:55:15 +00:00
343c521e28
The BatchGetItem request can return a very large response - according to DynamoDB documentation up to 16 MB, but presently in Alternator, we allow even more (see #5944). The problem is that the existing code prepares the entire response as a large contiguous string, resulting in oversized allocation warnings - and potentially allocation failures. So in this patch we estimate the size of the BatchGetItem response, and if it is "big enough" (currently over 100 KB), we return it with the recently added streaming output support. This streaming output doesn't avoid the extra memory copies unfortunately, but it does avoid a *contiguous* allocation which is the goal of this patch. After this patch, one oversized allocation warning is gone from the test: test/alternator/run test_batch.py::test_batch_get_item_large (a second oversized allocation is still present, but comes from the unrelated BatchWriteItem issue #8183). Fixes #8522 Signed-off-by: Nadav Har'El <nyh@scylladb.com> Message-Id: <20220111170541.637176-1-nyh@scylladb.com>