7fd31088f2
SSTables that contain many keys - a common case with small partitions in long lived nodes - can generate filters that are quite large. I have seen stalls over 80ms when reading a filter that was the result of a 6h write load of very small keys after nodetool compact (filter was in the 100s of MB) Similar care should be taken when creating the filter, as if the estimated number of partitions is big, the resulting large_bitset can be quite big as well. If we treat the i_filter.hh and large_bitset.hh interfaces as truly generic, then maybe we should have an in_thread version along with a common version. But the bloom filter is the only user for both and even if that changes in the future, it is still a good idea to run something with a massive loop in a thread. So for simplicity, I am just asserting that we are on a thread to avoid surprises, and inserting preemption points in the loops. Signed-off-by: Glauber Costa <glauber@scylladb.com>
62 lines
1.8 KiB
C++
62 lines
1.8 KiB
C++
/*
|
|
* Copyright (C) 2015 ScyllaDB
|
|
*/
|
|
|
|
/*
|
|
* This file is part of Scylla.
|
|
*
|
|
* Scylla is free software: you can redistribute it and/or modify
|
|
* it under the terms of the GNU Affero General Public License as published by
|
|
* the Free Software Foundation, either version 3 of the License, or
|
|
* (at your option) any later version.
|
|
*
|
|
* Scylla is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU General Public License
|
|
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
|
|
*/
|
|
|
|
#include "large_bitset.hh"
|
|
#include <algorithm>
|
|
#include <seastar/core/align.hh>
|
|
#include <seastar/core/thread.hh>
|
|
#include "seastarx.hh"
|
|
|
|
using namespace seastar;
|
|
|
|
large_bitset::large_bitset(size_t nr_bits) : _nr_bits(nr_bits) {
|
|
assert(thread::running_in_thread());
|
|
|
|
auto nr_blocks = align_up(nr_bits, bits_per_block()) / bits_per_block();
|
|
_storage.reserve(nr_blocks);
|
|
size_t nr_ints = align_up(nr_bits, bits_per_int()) / bits_per_int();
|
|
while (nr_ints) {
|
|
auto now = std::min(ints_per_block(), nr_ints);
|
|
_storage.push_back(std::make_unique<int_type[]>(now));
|
|
std::fill_n(_storage.back().get(), now, 0);
|
|
nr_ints -= now;
|
|
if (need_preempt()) {
|
|
thread::yield();
|
|
}
|
|
}
|
|
}
|
|
|
|
void
|
|
large_bitset::clear() {
|
|
assert(thread::running_in_thread());
|
|
|
|
size_t nr_ints = align_up(_nr_bits, bits_per_int()) / bits_per_int();
|
|
auto bp = _storage.begin();
|
|
while (nr_ints) {
|
|
auto now = std::min(ints_per_block(), nr_ints);
|
|
std::fill_n(bp++->get(), now, 0);
|
|
nr_ints -= now;
|
|
if (need_preempt()) {
|
|
thread::yield();
|
|
}
|
|
}
|
|
}
|