Fixes#309.
When scanning memtable readers detect is was flushed, which means that
it started to be moved to cache, they fall back to reading from
memtable's sstable.
Eventually what we should do is to combine memtable and cache contents
so that as long as data is not evicted we won't do IO. We do not
support scanning in cache yet though, so there is no point in doing
this now, and it is not trivial.
This fixes a mysterious compilation problem which popped up after
changing header order. It could be that the name "VERSION" picks up
some macro, but I haven't really figured that exactly.
Refs #293
Even more horrible that the shutdown patch. Tests using cql_test_env
are dependant on init.cc functions, but then scylla stopped being shut down
properly, those tests did to -> assert in sharded.hh
Yet another temp patch, simply duplicating the init.cc code for clq_test_env
to ensure we get what we think.
"Refs #293
* Disables all normal service shutdowns.
* Calls "stop()" explicitly for database (which in turn will also flush
commitlog etc). Then just does a hard "_exit".
* Add shutdown() + gate to commitlog to prevent data from being added once
system shutdown is initiated. Will and does cause exceptions in write
paths during shutdown.
This is an explicitly asked for workaround series for the interdependency
issues making shutdown as formulated by at_exit + sharded::stop unreliable
right now.
Proper ways of doing this would be to
a.) Make services actually stop service when asked to (stop())
b.) Do shutdown in two steps; stop() and later delete.
Note: I've left the old "at_exit" calls commented in main.cc/init.cc as a
reminder that this is not a final solution."
Bloom filter loading and saving is slow with single-bit access to the bitmap,
causing latency spikes of ~100ms for 20MB sstables. Larger sstables will be
much worse.
Fix by using the newly introduced large_bitmap bulk load/save methods. With
this, the maximum observed task latency was 16ms.
Fixes#299 (partially at least; larger bitmaps may require more work still).
Single-bit accessors are very slow, especially because we don't support
setting a bit to a value (just set to 1 and clear to 0). This causes
loading and retrieving the contents of a bitmap to be painfully slow.
Fix by providing iterator-based load() and save() methods. The methods
support partial load/save so that access to very large bitmaps can be
split over multiple tasks.
* Issue the "stop" method on DB (flushed CL + tables (partially))
* Do hard exit (_exit) to escape destructors and sanity checks.
This patch is horrible but sort of a workaround for various interdepdency
shutdown issues. Until services can actually be turned off, this might be
a viable option.
Refs #293. I will not call it a fix.
"Refs #293
* Add a commitlog::sync_all_segments, that explicitly forces all pending
disk writes
* Only delete segments from disk IFF they are marked clean. Thus on partial
shutdown or whatnot, even if CL is destroyed (destructor runs) disk files
not yet clean visavi sstables are preserved and replayable
* Do a sync_all_segments first of all in database::stop.
Exactly what to not stop in main I leave up to others discretion, or at least
another patch."
From Pawel:
This series makes compaction remove items that are no longer items:
- expired cells are changed into tombstones
- items covered by higher level tombstones are removed
- expired tombstones are removed if possible
Fixes#70.
Fixes#71.
It's great to have statistics, but assert is too big of a hammer. We don't need
to crash due to the lack of it, and can try our best to continue.
We currently have a problem (described in 265), in which we, for some reason,
fail to read the Statistics file. Throwing an exception will still cause us to
fail to boot, but at least it will be more informative.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
A region being merged can still be in use; but after merging, compaction_lock
and the reclaim counter will no longer work. This can lead to
use-after-compact-without-re-lookup errors.
Fix by making the source region be the same as the target region; they
will share compaction locks and reclaim counters, so lookup avoidance
will still work correctly.
Fixes#286.
Route request to CPU 0. _operation_mode is not replicated to other CPUS.
Without this:
$ curl -X GET --header "Accept: application/json"
"http://127.0.0.1:10000/storage_service/operation_mode"
returns "NORMAL" and "STARTING" randomly.
Only cpu 0 instance of gossip has the correct information, route request
to cpu 0.
Fix a bug where
$ curl -X GET --header "Accept: application/json"
"http://172.31.5.77:10000/storage_service/gossiping"
returns true and false randomly.
