Commit Graph

7306 Commits

Author SHA1 Message Date
Calle Wilund
545d3151e2 paging_state implementation
Note: serial format blob is different compared to origin, due to scyllas
different internal architecture. I.e. we query actual rows.
But drivers etc ignore the content of the blob, it is opaque.
2015-11-10 13:12:33 +01:00
Calle Wilund
7e2569c680 result_set: Make "paging_state" a pointer to const member
Paging state is immutable. Constness is nice.
2015-11-10 13:12:33 +01:00
Calle Wilund
4a1a17defc cql3::selection: Move result set building visitor to result_set_builder
Allows its use (and partial override - hint hint) in more place than
one.
2015-11-10 13:12:33 +01:00
Calle Wilund
820ba3540b result_view: Add static helper method to do the dispatch from select
Again, makes it easier to use the same optimization/technique at more than
one location.
2015-11-10 13:12:33 +01:00
Calle Wilund
23b6240dad cql3::selection: Fix some constness correctness 2015-11-10 13:12:33 +01:00
Calle Wilund
369e09459c cql3::result_set: Add non-const metadata getter. 2015-11-10 13:12:33 +01:00
Calle Wilund
0fa543800a data_output: Template "blob" writers (bytes*) to allow for varying "size" type 2015-11-10 13:12:33 +01:00
Calle Wilund
9ee8204993 data_input: Fix missing bounds check 2015-11-10 13:12:33 +01:00
Avi Kivity
fee9688ae8 Merge "Fixes for collections of collections" from Paweł
"These are some fixes for removing items from collections of frozen
collections."
2015-11-10 09:38:13 +02:00
Paweł Dziepak
e494b2c1a0 tests/cql: add tests for collections of collections
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 16:37:50 +01:00
Paweł Dziepak
ee182f39a5 cql3/sets: simplify sets::discarder
Since the introduction of sets::element_discarder sets::discarder is
always given a set, never a single value.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 15:24:25 +01:00
Paweł Dziepak
0b0cef2457 cql3/maps: do not assume that the term type is constants::value
Fixes #516.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 15:18:07 +01:00
Paweł Dziepak
101ee1affd cql3/sets: add element_discarder
Currently sets::discarder is used by both set difference and removal of
a single element operations. To distinguish between them the discarder
checks whether the provided value is a set or something else, this won't
work however if a set of frozen sets is created.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 14:47:09 +01:00
Tomasz Grabiec
488528d1cd range: Fix range::subtract() for some cases
Was not handling wrap-around cases like this properly:

 (8, 3) - (2, 1)
2015-11-09 14:20:31 +02:00
Glauber Costa
8e0ad183b9 dist: AMI: add discard mount option
Crucial on SSDs, inocuous (hopefully) on HDDs. Let's add it to make peace
with our inner selves.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-11-09 13:31:01 +02:00
Asias He
d622fe867e gossip: Pass const ref if possible
It is clear that we will not change the parameter.
2015-11-09 13:01:37 +02:00
Asias He
615e1362da gossip: Add const version for get_heart_beat_state and friends
get_endpoint_state_map
get_application_state_map
2015-11-09 13:01:36 +02:00
Gleb Natapov
d77a2a0f03 do not try to write same memtable to sstable twice if moving it to a cache failed.
Error handling in column_family::try_flush_memtable_to_sstable() is
misplaced. It happens after update_cache(), so writing sstable may
have succeeded, but moving memtable into the cache may have failed.
update_cache() destroys memtable even if it fails, but error handler
is not aware of it (it does not even distinguish whether error happened
during sstable creation or moving into cache) and when it tells caller
to retry it retries with already destroyed memtable. Fix it by ignoring
moving to cache errors.
2015-11-09 11:27:37 +01:00
Avi Kivity
cb93af2ad7 Revert "do not try to write same memtable to sstable twice if moving it to a cache failed."
This reverts commit fff37d15cd.

Says Tomek (and the comment in the code):

"update_cache() must be called before unlinking the memtable because cache + memtable at any time is supposed to be authoritative source of data for contained partitions. If there is a cache hit in cache, sstables won't be checked. If we unlink the memtable before cache is updated, it's possible that a query will miss data which was in that unlinked memtable, if it hits in the cache (with an old value)."
2015-11-09 11:22:12 +02:00
Avi Kivity
975a3eb49f Merge "move node support + decommission fix" from Asias
"Usage example:

$ nodetool -p 7200 move -- "-9223372036854775000"

or

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
  "http://127.0.0.2:10000/storage_service/move?new_token=8000000"

Note: Tomek's range subtract and cql_test_env fix for system_keyspace seriers are included."
2015-11-09 11:08:51 +02:00
Tzach Livyatan
0e9b3e4d37 add build and run from source instructions in README.md
fix #451

Signed-off-by: Tzach Livyatan <tzach@cloudius-systems.com>
2015-11-09 10:05:05 +02:00
Gleb Natapov
fff37d15cd do not try to write same memtable to sstable twice if moving it to a cache failed.
Error handling in column_family::try_flush_memtable_to_sstable() is
misplaced. It happens after update_cache(), so writing sstable may
have succeeded, but moving memtable into the cache may have failed.
update_cache() destroys memtable even if it fails, but error handler
is not aware of it (it does not even distinguish whether error happened
during sstable creation or moving into cache) and when it tells caller
to retry it retries with already destroyed memtable. Fix it by ignoring
moving to cache errors.
2015-11-09 09:56:45 +02:00
Asias He
85b42e362b storage_service: Fix hang of decommission
nodetool decommission hangs forever due to a recursive lock.

decommission()
  with api lock
  shutdown_client_servers()
     with api lock
       stop_rpc_server()
     with api lock
       stop_native_transport()

Fix it by calling helpers for stop_rpc_server and stop_native_transport
without the lock.
2015-11-09 08:43:04 +08:00
Asias He
cb8b0eedfc token_metadata: Fix set_difference in calculate_pending_ranges
std::set_difference requires the container to be sorted.
2015-11-09 08:43:04 +08:00
Asias He
87292d6a16 range_streamer: Simplify unordered_multimap_to_unordered_map
operator[] is own friend, it creates map[x] if x is not in the map.
2015-11-09 08:43:04 +08:00
Asias He
a54989cd65 range_streamer: Fix get_all_ranges_with_strict_sources_for
std::set_difference requires the container to be sorted which is not
true here, use remove_if.

Do not use assert, use throw instead so that we can recover from this
error.
2015-11-09 08:43:04 +08:00
Asias He
145c122cb7 storage_service: Enable commented out in on_restart
on_dead is now available.
2015-11-09 08:43:04 +08:00
Asias He
1ab1a1d8d9 storage_service: Fix bogus log in on_dead 2015-11-09 08:43:04 +08:00
Asias He
9f69b4c969 storage_service: Enable add_moving_endpoint in handle_state_moving 2015-11-09 08:43:04 +08:00
Asias He
c90e9c97f5 token_metadata: Add add_moving_endpoint 2015-11-09 08:43:04 +08:00
Asias He
0fc6e76d1e storage_service: Make fd a reference in remove_node 2015-11-09 08:43:04 +08:00
Asias He
2d55e5e8ff api: Implement storage_service::move
Start node 1 and node 2:

node 1:
$ scylla --initial-token 0

node 2:
$ scylla --initial-token 2000000

To move node 2 to a new token 8000000

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
"http://127.0.0.2:10000/storage_service/move?new_token=8000000"
2015-11-09 08:43:04 +08:00
Asias He
d124df17b4 storage_service: Use get_local_tokens in set_tokens
We are using _bootstrap_tokens since get_local_tokens was not available.
2015-11-09 08:43:04 +08:00
Asias He
e637faefbe storage_service: Implement move
Moves the node on the token ring to a new token.

$ nodetool move <new token>
2015-11-09 08:43:04 +08:00
Asias He
b12d95d955 storage_service: Implement calculate_to_from_streams 2015-11-09 08:43:04 +08:00
Asias He
d166b0f3fa range_streamer: Add get_work_map 2015-11-09 08:43:04 +08:00
Asias He
ada2466e18 token_metadata: Add clone_after_all_settled
Needed by storage_service::range_relocator::calculate_to_from_streams.
2015-11-09 08:43:04 +08:00
Asias He
47cf4dfbc5 storage_service: Implement range_relocator class 2015-11-09 08:43:04 +08:00
Asias He
53d30ed985 storage_service: Implement calculate_stream_and_fetch_ranges
This is needed by calculate_to_from_streams.
2015-11-09 08:42:56 +08:00
Tomasz Grabiec
3c4c83c66f cql_test_env: Initialize system keyspace 2015-11-09 08:42:53 +08:00
Tomasz Grabiec
1c9d8571a8 cql_test_env: Flatten stop() 2015-11-09 08:42:51 +08:00
Tomasz Grabiec
1c1a4c9656 tests: Add test for range::subtract() 2015-11-09 08:42:47 +08:00
Tomasz Grabiec
0f2727b183 range: Implement subtract() 2015-11-09 08:42:45 +08:00
Tomasz Grabiec
48c7f51966 range: Extract bound comparators 2015-11-09 08:42:42 +08:00
Tomasz Grabiec
f37512d1f7 range: Make unwrap() return ranges in-order 2015-11-09 08:42:39 +08:00
Gleb Natapov
9abd724b30 storage_proxy: fix mutation verb error reporting
Currently error code is attached to a future returned by when_all() which
is never is exceptional one, but it may hold exceptional future as a
first element. Move error handling close to where error it tries to
catch is generated instead.
2015-11-08 16:35:41 +02:00
Raphael S. Carvalho
4eb94fbc35 sstables: improve exception handling in compact_sstables
Let's move the code that prints that a compaction succeeded only
after the code that catches exception on either read or write
fibers. Let's also get rid of done and use repeat instead in
the read fiber.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-11-08 11:24:55 +02:00
Takuya ASADA
368ee17d53 dist: prevent error building ubuntu package on EC2
fixes #491

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-06 22:06:55 +02:00
Gleb Natapov
120ae08082 storage_proxy: fix race during write completion
If write timeout and last acknowledgement needed for CL happen simultaneously
_ready can be sent to be exceptional by the timeout handler, but since
removal of the response handler happens in continuation it may be
reordered with last ack processing and there _ready will be set again
which will trigger assert. Fix it by removing the handler immediately,
no need to wait for continuation. It makes code simpler too.
2015-11-05 19:35:18 +02:00
Paweł Dziepak
64f1c2866c lsa: free segment in trim_emergency_reserve_to_max()
_emergency_reserve is an intrusive containers and it doesn't care about
segment lifetime.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-05 18:04:38 +02:00