The default for get_unlimited_query_max_result_size() is 100MB (adjustable through config), whereas query::result_memory_limiter::maximum_result_size is 1MB (hard coded, should be enough for everybody)
This limit is then used by the replica to decide when to break pages and, in case of reversed clustering order reads, when to fail the read when accumulated data crosses the threshold. The latter behavior stems from the fact that reversed reads had to accumulate all the data (read in forward order) before they can reverse it and return the result. Reverse reads thus need a higher limit so that they have a higher chance of succeeding.
Most readers are now supporting reading in reverse natively, and only reversing wrappers (make_reversing_reader()) inserted on top of ka/la sstable readers need to accumulate all the data. In other cases, we could break pages sooner. This should lead to better stability (less memory usage) and performance (lower page build latency, higher read concurrency due to less memory footprint).
Tests: unit(dev)
Closes#9815
* github.com:scylladb/scylla:
storage_proxy: Send page_size in the read_command
gms: add SEPARATE_PAGE_SIZE_AND_SAFETY_LIMIT feature
result_memory_accounter: use new max_result_size::get_page_size in check_local_limit
max_result_size: Add page_size field
4a323772c1