Commit Graph

7299 Commits

Author SHA1 Message Date
Calle Wilund
9ee8204993 data_input: Fix missing bounds check 2015-11-10 13:12:33 +01:00
Avi Kivity
fee9688ae8 Merge "Fixes for collections of collections" from Paweł
"These are some fixes for removing items from collections of frozen
collections."
2015-11-10 09:38:13 +02:00
Paweł Dziepak
e494b2c1a0 tests/cql: add tests for collections of collections
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 16:37:50 +01:00
Paweł Dziepak
ee182f39a5 cql3/sets: simplify sets::discarder
Since the introduction of sets::element_discarder sets::discarder is
always given a set, never a single value.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 15:24:25 +01:00
Paweł Dziepak
0b0cef2457 cql3/maps: do not assume that the term type is constants::value
Fixes #516.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 15:18:07 +01:00
Paweł Dziepak
101ee1affd cql3/sets: add element_discarder
Currently sets::discarder is used by both set difference and removal of
a single element operations. To distinguish between them the discarder
checks whether the provided value is a set or something else, this won't
work however if a set of frozen sets is created.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-09 14:47:09 +01:00
Tomasz Grabiec
488528d1cd range: Fix range::subtract() for some cases
Was not handling wrap-around cases like this properly:

 (8, 3) - (2, 1)
2015-11-09 14:20:31 +02:00
Glauber Costa
8e0ad183b9 dist: AMI: add discard mount option
Crucial on SSDs, inocuous (hopefully) on HDDs. Let's add it to make peace
with our inner selves.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-11-09 13:31:01 +02:00
Asias He
d622fe867e gossip: Pass const ref if possible
It is clear that we will not change the parameter.
2015-11-09 13:01:37 +02:00
Asias He
615e1362da gossip: Add const version for get_heart_beat_state and friends
get_endpoint_state_map
get_application_state_map
2015-11-09 13:01:36 +02:00
Gleb Natapov
d77a2a0f03 do not try to write same memtable to sstable twice if moving it to a cache failed.
Error handling in column_family::try_flush_memtable_to_sstable() is
misplaced. It happens after update_cache(), so writing sstable may
have succeeded, but moving memtable into the cache may have failed.
update_cache() destroys memtable even if it fails, but error handler
is not aware of it (it does not even distinguish whether error happened
during sstable creation or moving into cache) and when it tells caller
to retry it retries with already destroyed memtable. Fix it by ignoring
moving to cache errors.
2015-11-09 11:27:37 +01:00
Avi Kivity
cb93af2ad7 Revert "do not try to write same memtable to sstable twice if moving it to a cache failed."
This reverts commit fff37d15cd.

Says Tomek (and the comment in the code):

"update_cache() must be called before unlinking the memtable because cache + memtable at any time is supposed to be authoritative source of data for contained partitions. If there is a cache hit in cache, sstables won't be checked. If we unlink the memtable before cache is updated, it's possible that a query will miss data which was in that unlinked memtable, if it hits in the cache (with an old value)."
2015-11-09 11:22:12 +02:00
Avi Kivity
975a3eb49f Merge "move node support + decommission fix" from Asias
"Usage example:

$ nodetool -p 7200 move -- "-9223372036854775000"

or

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
  "http://127.0.0.2:10000/storage_service/move?new_token=8000000"

Note: Tomek's range subtract and cql_test_env fix for system_keyspace seriers are included."
2015-11-09 11:08:51 +02:00
Tzach Livyatan
0e9b3e4d37 add build and run from source instructions in README.md
fix #451

Signed-off-by: Tzach Livyatan <tzach@cloudius-systems.com>
2015-11-09 10:05:05 +02:00
Gleb Natapov
fff37d15cd do not try to write same memtable to sstable twice if moving it to a cache failed.
Error handling in column_family::try_flush_memtable_to_sstable() is
misplaced. It happens after update_cache(), so writing sstable may
have succeeded, but moving memtable into the cache may have failed.
update_cache() destroys memtable even if it fails, but error handler
is not aware of it (it does not even distinguish whether error happened
during sstable creation or moving into cache) and when it tells caller
to retry it retries with already destroyed memtable. Fix it by ignoring
moving to cache errors.
2015-11-09 09:56:45 +02:00
Asias He
85b42e362b storage_service: Fix hang of decommission
nodetool decommission hangs forever due to a recursive lock.

decommission()
  with api lock
  shutdown_client_servers()
     with api lock
       stop_rpc_server()
     with api lock
       stop_native_transport()

Fix it by calling helpers for stop_rpc_server and stop_native_transport
without the lock.
2015-11-09 08:43:04 +08:00
Asias He
cb8b0eedfc token_metadata: Fix set_difference in calculate_pending_ranges
std::set_difference requires the container to be sorted.
2015-11-09 08:43:04 +08:00
Asias He
87292d6a16 range_streamer: Simplify unordered_multimap_to_unordered_map
operator[] is own friend, it creates map[x] if x is not in the map.
2015-11-09 08:43:04 +08:00
Asias He
a54989cd65 range_streamer: Fix get_all_ranges_with_strict_sources_for
std::set_difference requires the container to be sorted which is not
true here, use remove_if.

Do not use assert, use throw instead so that we can recover from this
error.
2015-11-09 08:43:04 +08:00
Asias He
145c122cb7 storage_service: Enable commented out in on_restart
on_dead is now available.
2015-11-09 08:43:04 +08:00
Asias He
1ab1a1d8d9 storage_service: Fix bogus log in on_dead 2015-11-09 08:43:04 +08:00
Asias He
9f69b4c969 storage_service: Enable add_moving_endpoint in handle_state_moving 2015-11-09 08:43:04 +08:00
Asias He
c90e9c97f5 token_metadata: Add add_moving_endpoint 2015-11-09 08:43:04 +08:00
Asias He
0fc6e76d1e storage_service: Make fd a reference in remove_node 2015-11-09 08:43:04 +08:00
Asias He
2d55e5e8ff api: Implement storage_service::move
Start node 1 and node 2:

node 1:
$ scylla --initial-token 0

node 2:
$ scylla --initial-token 2000000

To move node 2 to a new token 8000000

$ curl -X POST --header "Content-Type: application/json" --header "Accept: application/json"
"http://127.0.0.2:10000/storage_service/move?new_token=8000000"
2015-11-09 08:43:04 +08:00
Asias He
d124df17b4 storage_service: Use get_local_tokens in set_tokens
We are using _bootstrap_tokens since get_local_tokens was not available.
2015-11-09 08:43:04 +08:00
Asias He
e637faefbe storage_service: Implement move
Moves the node on the token ring to a new token.

$ nodetool move <new token>
2015-11-09 08:43:04 +08:00
Asias He
b12d95d955 storage_service: Implement calculate_to_from_streams 2015-11-09 08:43:04 +08:00
Asias He
d166b0f3fa range_streamer: Add get_work_map 2015-11-09 08:43:04 +08:00
Asias He
ada2466e18 token_metadata: Add clone_after_all_settled
Needed by storage_service::range_relocator::calculate_to_from_streams.
2015-11-09 08:43:04 +08:00
Asias He
47cf4dfbc5 storage_service: Implement range_relocator class 2015-11-09 08:43:04 +08:00
Asias He
53d30ed985 storage_service: Implement calculate_stream_and_fetch_ranges
This is needed by calculate_to_from_streams.
2015-11-09 08:42:56 +08:00
Tomasz Grabiec
3c4c83c66f cql_test_env: Initialize system keyspace 2015-11-09 08:42:53 +08:00
Tomasz Grabiec
1c9d8571a8 cql_test_env: Flatten stop() 2015-11-09 08:42:51 +08:00
Tomasz Grabiec
1c1a4c9656 tests: Add test for range::subtract() 2015-11-09 08:42:47 +08:00
Tomasz Grabiec
0f2727b183 range: Implement subtract() 2015-11-09 08:42:45 +08:00
Tomasz Grabiec
48c7f51966 range: Extract bound comparators 2015-11-09 08:42:42 +08:00
Tomasz Grabiec
f37512d1f7 range: Make unwrap() return ranges in-order 2015-11-09 08:42:39 +08:00
Gleb Natapov
9abd724b30 storage_proxy: fix mutation verb error reporting
Currently error code is attached to a future returned by when_all() which
is never is exceptional one, but it may hold exceptional future as a
first element. Move error handling close to where error it tries to
catch is generated instead.
2015-11-08 16:35:41 +02:00
Raphael S. Carvalho
4eb94fbc35 sstables: improve exception handling in compact_sstables
Let's move the code that prints that a compaction succeeded only
after the code that catches exception on either read or write
fibers. Let's also get rid of done and use repeat instead in
the read fiber.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-11-08 11:24:55 +02:00
Takuya ASADA
368ee17d53 dist: prevent error building ubuntu package on EC2
fixes #491

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-06 22:06:55 +02:00
Gleb Natapov
120ae08082 storage_proxy: fix race during write completion
If write timeout and last acknowledgement needed for CL happen simultaneously
_ready can be sent to be exceptional by the timeout handler, but since
removal of the response handler happens in continuation it may be
reordered with last ack processing and there _ready will be set again
which will trigger assert. Fix it by removing the handler immediately,
no need to wait for continuation. It makes code simpler too.
2015-11-05 19:35:18 +02:00
Paweł Dziepak
64f1c2866c lsa: free segment in trim_emergency_reserve_to_max()
_emergency_reserve is an intrusive containers and it doesn't care about
segment lifetime.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-11-05 18:04:38 +02:00
Amnon Heiman
5f1cae67ff API: compaction_manager should catch the field reference by value
The get_cm_stats gets a pointer to a field in the stats object. It
should capture it by value or segmentation falut may occure when the
caller gets out of scope.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2015-11-05 14:17:12 +02:00
Avi Kivity
c9f02ce42f Merge seastar upstream
* seastar 05b41b8...5c10d3e (3):
  > reactor: more conservative signal handling
  > output_stream: add write(temporary_buffer) variant
  > input_stream: add read() method
2015-11-05 13:20:26 +02:00
Takuya ASADA
b50a6a1521 dist: add missing dist/ubuntu/changelog.in
Missing file for d2dd6b90a9

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-05 11:09:32 +02:00
Glauber Costa
958abc17a0 sstables: be more descriptive with filename parsing errors
Currently, we don't let the user know even what is the filename that failed.
That information should be included in the message.

Signed-off-by: Glauber Costa <glommer@scylladb.com>
2015-11-05 10:45:59 +02:00
Raphael S. Carvalho
279a8f13ee sstables: compaction: handle failure on read fiber
This assert (in write fiber) would fail if read fiber failed
because the variable done will not be set to true.
The use of assert is very bad, because it prevents scylla
from proceeding, which is possible.
To solve it, let's trigger an exception if done is not true.
We do have code that will wait for both read and write fibers,
and catch exceptions, if any.

Closes #523.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-11-04 16:48:01 +01:00
Tomasz Grabiec
598f4104fd query_processor: Use the correct client_state
Since 4641dfff24, query_state keeps a
copy of client_state, not a reference. Therefore _cl is no longer
updated by queries using _qp. Fix by using the client_state from _qp.

Fixes #525.
2015-11-04 12:43:26 +02:00
Takuya ASADA
ec06fd77a8 dist: split debug symbol to sepalate package on ubuntu
Fixes #524

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-11-04 12:35:35 +02:00