Commit Graph

53948 Commits

Author SHA1 Message Date
Gleb Natapov
327e27b67b storage_proxy: stop query timeout timer when all replies are received
Fixes #285
2015-09-07 15:21:51 +02:00
Gleb Natapov
41f16159b3 storage_proxy: track reference to storage_proxy during mutate/query operations
This patch makes sure that storage_proxy cannot be deleted while
mutate/query operation is in progress.
2015-09-07 14:46:13 +02:00
Gleb Natapov
f51f5c819e messaging: Add unregister function for verbs used by storage proxy 2015-09-07 14:46:13 +02:00
Gleb Natapov
b884aba147 storage_proxy: drop superfluous captures 2015-09-07 14:46:13 +02:00
Gleb Natapov
5af2d18b6f stroage_proxy: change storage_proxy::mutate_locally to use do_with 2015-09-07 14:46:13 +02:00
Takuya ASADA
8fa868d4e9 dist: use mock rpm instead of rpmbuild
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-07 15:41:13 +03:00
Takuya ASADA
45502b7110 dist: Dynamically configure scylla.yaml on EC2 instance
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-07 15:41:13 +03:00
Takuya ASADA
a5dcc39494 dist: fix scylla_run to specify '--network-mode posix' when NETWORK_MODE is posix
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-07 15:40:08 +03:00
Tomasz Grabiec
1f3d7aa78c Bump up seastar submodule head
Changes:

    tests: improve rpc timeout test
    rpc: add unregister_handler
    future: Fix assertion failure in case schedule() throws
    rpc: fix rpc timeout
    build: disable -fsanitize=vptr if it is available and broken
2015-09-07 14:04:53 +02:00
Calle Wilund
380649eb66 Database: Add commitlog flush handler to switch memtables to disk
Initiates flushing of CF:s to sstable on CL disk overflow (flush req)
2015-09-07 13:21:46 +02:00
Calle Wilund
fdb921afb2 Commitlog: Add flushing of segment CF:s on disk overflow
* Do not throw away commitlog segments on disk size overflow. 
  Issue a flush request (i.e. calculate RP we want to free unto, 
  and for all dirty CF:s, do a request).
  "Abstracted" as registerable callback. I.e. DB:s responsibility 
  to actually do something with it.
2015-09-07 13:21:43 +02:00
Calle Wilund
31f2dcb342 Config: change commilog max size on disk to be in sync with scylla.yaml 2015-09-07 13:13:51 +02:00
Calle Wilund
841dd32a8a Commitlog: divide max on-disk-size by num cpus
To try to keep the resulting limit as configured
2015-09-07 13:13:46 +02:00
Asias He
f89a25562c storage_service: Fix is_auto_bootstrap
Get the value from cfg option.
2015-09-07 12:53:58 +03:00
Raphael S. Carvalho
1157e6f119 sstable: use desired buffer size in write_simple
Currently, we use a 128k buffer for creation of data and index
files, but for other components we use a 4k buffer size.
Let's also use a 128k buffer for the other components.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-09-07 12:52:26 +03:00
Tomasz Grabiec
433a298f60 row_cache: Extract comparator construction before the loop 2015-09-07 09:41:36 +02:00
Tomasz Grabiec
bf6062493e tests: Introduce tests/perf_row_cache_update 2015-09-07 09:41:36 +02:00
Tomasz Grabiec
10453c71d2 tests: perf: Make iterations between clock readings in time_it() configurable 2015-09-07 09:41:36 +02:00
Tomasz Grabiec
74603425ac mutation_partition: Introduce r-value version of apply() 2015-09-07 09:41:36 +02:00
Asias He
7cc768a864 gossip: Fix wrong cluster name and partitioner name
Right now, gossip returns hard coded cluster and partitioner name.

  sstring get_cluster_name() {
      // FIXME: DatabaseDescriptor.getClusterName()
      return "my_cluster_name";
  }
  sstring get_partitioner_name() {
      // FIXME: DatabaseDescriptor.getPartitionerName()
      return "my_partitioner_name";
  }

Fix it by setting the correct name from configure option.

With this

   cqlsh 127.0.0.$i -e "SELECT * from system.local;

returns correct cluster_name.

Fixes #291
2015-09-07 09:21:18 +03:00
Tomasz Grabiec
8a140a9ba9 mutation_partition: row: Move implementation to source file 2015-09-06 21:25:44 +02:00
Tomasz Grabiec
bf2b64d3f7 mutation_partition: row: Fix operator==()
Spotted by clion inspections.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
ba35788817 mutation_partition: De-templetize methods
Instead of accepting a column resolver callable, accept a schema and
column_kind or column_selector. Makes the interface easier to use and
enables us to move implementation to .cc file.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
49bf844418 tests: Introduce row_cache_alloc_stress
Tests stability of row_cache operations under low/fragmented memory.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
3b441416fa lsa: Make segment size publicly accessible
Some tests depend on segment size.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
49f094ad5f tests: Add test for row_cache::update() 2015-09-06 21:25:44 +02:00
Tomasz Grabiec
122bd8ea46 row_cache: Restore indentation 2015-09-06 21:25:44 +02:00
Tomasz Grabiec
d1f89b4eab row_cache: Use allocation_section
See #259.

When transferring mutations between memtable and cache, lsa sometimes
runs out of memory. This solves the first two points, keeping reserve
filled up and adjusting the amount of reserve based on execution
history.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
7efcde12aa row_cache: Introduce row_cache::touch()
Useful in tests for ensuring that certain entries survive eviction.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
24a5221280 row_cache: Avoid leaking of partitions when exception is thrown inside update() 2015-09-06 21:25:44 +02:00
Tomasz Grabiec
1d182903cd mutation_partition: Document exception guarantees of apply() 2015-09-06 21:25:44 +02:00
Tomasz Grabiec
c82325a76c lsa: Make region evictor signal forward progress
In some cases region may be in a state where it is not empty and
nothing could be evicted from it. For example when creating the first
entry, reclaimer may get invoked during creation before it gets
linked. We therefore can't rely on emptiness as a stop condition for
reclamation, the evction function shall signal us if it made forward
progress.
2015-09-06 21:25:44 +02:00
Tomasz Grabiec
94f0db933f lsa: Fix typo in the word 'emergency' 2015-09-06 21:24:59 +02:00
Tomasz Grabiec
200562abe7 lsa: Reclaim over-max segments from segment pool reserve 2015-09-06 21:24:59 +02:00
Tomasz Grabiec
d022a1a4a3 lsa: Introduce allocating_section
Related to #259. In some cases we need to allocate memory and hold
reclaim lock at the same time. If that region holds most of the
reclaimable memory, allocations inside that code section may
fail. allocating_section is a work-around of the problem. It learns
how big reserves shold be from past execution of critical section and
tries to ensure proper reserves before entering the section.
2015-09-06 21:24:59 +02:00
Tomasz Grabiec
3caad2294b lsa: Tolerate empty segments when region is destroyed
Some times we may close an empty active segment, if all data in it was
evicted. Normally segments are removed as soon as the last object in
it is freed, but if the segment is already empty when closed, noone is
supposed to call free on it. Such segments would be quickly reclaimed
during compaction, but it's possible that we will destroy the region
before they're reclaimed by compaction. Currently we would fail on an
assertion which checks that there are no segments. This change fixes
the problem by handling empty closed segments when region is
destroyed.
2015-09-06 21:24:59 +02:00
Tomasz Grabiec
c37aa73051 lsa: Drop alignment requirement from segment 2015-09-06 21:24:59 +02:00
Tomasz Grabiec
2c1536b5a7 lsa: Make free() path noexcept
Memory releasing is invoked from destructors so should not throw. As a
consequence it should not allocate memory, so emergency segment pool
was switched from std::deque<> to an alloc-free intrusive stack.
2015-09-06 21:24:59 +02:00
Tomasz Grabiec
773595a1f3 mutation_partition: row: Mark move assignment noexcept 2015-09-06 21:24:59 +02:00
Tomasz Grabiec
f404a238bb allocation_strategy: Make construct() exception-safe 2015-09-06 21:24:59 +02:00
Tomasz Grabiec
1c68045b61 mutation_partition: row: Make copy constructor exception-safe 2015-09-06 21:24:58 +02:00
Tomasz Grabiec
828b7c2f54 mutation_partition: Make copy constructor exception-safe 2015-09-06 21:24:58 +02:00
Tomasz Grabiec
0d7cdab0ff utils: managed_vector: Make copy assignment exception-safe 2015-09-06 21:24:58 +02:00
Tomasz Grabiec
81d81c81f2 utils: managed_vector: Make copy constructor exception-safe
Copying values may throw (std::bad_alloc for example), which will
result in a leak because destructor will not be called when
constructor throws.

May manifest as the following assertion failure:

utils/logalloc.cc:672: virtual logalloc::region_impl::~region_impl(): Assertion `_active->is_empty()' failed.
2015-09-06 21:24:58 +02:00
Tomasz Grabiec
802a9db9b0 Fix spelling of 'definitely_doesnt_exist' 2015-09-06 21:24:58 +02:00
Tomasz Grabiec
c08742e4fc query-result-writer: Remove assert(_finished) guards from destructors
They're not exception-safe. If exception is thrown before finish() is
called, they'll trigger.
2015-09-06 21:24:58 +02:00
Tomasz Grabiec
fa8d530cc2 lsa: Add ability to trace reclaiming latency 2015-09-06 21:24:58 +02:00
Avi Kivity
0ed6e9c439 Merge "Use rpc's timeout support when sending gossip message" from Asias
"Timeout support was added to gossip message by using semaphore's timeout
support, now that rpc has timeout support, switch to it.

Fixes #284"
2015-09-06 15:31:58 +03:00
Asias He
8cff2318dc gossip: Add timeout support for send_echo 2015-09-06 16:35:11 +08:00
Asias He
2a06214306 gossip: Switch to use rpc timeout for send_gossip_digest_syn
Timeout support was added to gossip message by using semaphore's
timeout support, now that rpc has timeout support, switch to it.
2015-09-06 16:34:41 +08:00