mirrors/scylladb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

compaction: Fix leak of expired sstable in the backlog tracker

Browse Source 
expired sstables are skipped in the compaction setup phase, because they don't
need to be actually compacted, but rather only deleted at the end.
that is causing such sstables to not be removed from the backlog tracker,
meaning that backlog caused by expired sstables will not be removed even after
their deletion, which means shares will be higher than needed, making compaction
potentially more aggressive than it have to.

to fix this bug, let's manually register these sstables into the monitor,
such that they'll be removed from the tracker once compaction completes.

Fixes #6054.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20210216203700.189362-1-raphaelsc@scylladb.com>
This commit is contained in:
Raphael S. Carvalho
2021-02-16 17:37:00 -03:00
committed by Avi Kivity
parent d7f202f900
commit 5206a97915
1 changed files with 11 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
sstables/compaction.cc
View File

@@ -310,6 +310,7 @@ struct compaction_read_monitor_generator final : public read_monitor_generator {
_generated_monitors.emplace_back(std::move(sst), _compaction_manager, _cf);
return _generated_monitors.back();
}
compaction_read_monitor_generator(compaction_manager& cm, column_family& cf)
: _compaction_manager(cm)
, _cf(cf) {}
@@ -574,6 +575,7 @@ private:
// Do not actually compact a sstable that is fully expired and can be safely
// dropped without ressurrecting old data.
if (tombstone_expiration_enabled() && fully_expired.contains(sst)) {
on_skipped_expired_sstable(sst);
continue;
}
@@ -680,6 +682,9 @@ private:
virtual void on_end_of_compaction() {};
// Inform about every expired sstable that was skipped during setup phase
virtual void on_skipped_expired_sstable(shared_sstable sstable) {}
// create a writer based on decorated key.
virtual compaction_writer create_compaction_writer(const dht::decorated_key& dk) = 0;
// stop current writer
@@ -923,6 +928,12 @@ public:
}
replace_remaining_exhausted_sstables();
}
virtual void on_skipped_expired_sstable(shared_sstable sstable) override {
// manually register expired sstable into monitor, as it's not being actually compacted
// this will allow expired sstable to be removed from tracker once compaction completes
_monitor_generator(std::move(sstable));
}
private:
void backlog_tracker_incrementally_adjust_charges(std::vector<shared_sstable> exhausted_sstables) {
//