mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-06-06 23:13:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
53,948 Commits 257 Branches 558 Tags
b7fc661fa983f754ed25cd94de5dd8dd1bc9a98c
Commit Graph

53948 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Tomasz Grabiec
ff0308104c sstables: Add data_end_position() on summary page level 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
e9a050da78 sstables: Obtain the key from entries using get_key() rather than casting to bytes_view 
The entry contains not only the key, but other stuff like
position. Why would casting to bytes_view give the view on just the
key and not the whole entry. Better to be explicit.
 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
0b5f908a0b sstables: Make key_view comparable with partition_key_view 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
73ccd51cc5 sstables: Add key_view::tri_compare() 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
6882e6ca50 storage_proxy: Fix range splitting when split point is exlcluded by lower bound 
If we had a range (x; ...] then x is excluded, but token iterator was
initialized with x. The splitting loop would exit prematurely because
it would detect that the token is outside the range.

The fix is to teach ring_range() to recognize this and always give
tokens which are not smaller than the range's lower bound.
 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
c219c3676b storage_proxy: Avoid unnecessary copy of a partition_range 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
abdb36a0f6 storage_proxy: Fix ordering comparator used during merging and reconciliation 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
bad7602299 keys: Introduce ring_order_tri_compare() 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
9b52f5bf2b storage_proxy: Make range splitting never produce a wrap around range 
Origin has no notion of a maximum token so a range without upper bound
is represented as (x; min]. The splitting code is supposed to produce
only non-wrapping ranges, but (x; min] looks like a wrapping range, so
database code which consumes it would have to special-case for it. A
simpler solution is to change the splitting code to never produce a
wrapping range.
 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
f557951554 storage_proxy: Encapsulate access to range tokens 
The wrappers also take care of cases when each bound is undefined, and
return mimimum or maximum token respectively, which fixes undefined
behavior in get_restricted_ranges().

Alternative solution would be to make partition_range use a different
type of range, the one where bounds are always specified. However it's
not worth introducing a new range type just for those few users.
 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
5587d2ce9a storage_proxy: Simplify access to storage_proxy::response_id_type 
storage_proxy::storage_proxy::response_id_type
 -> storage_proxy::response_id_type
 2015-07-22 10:27:48 +02:00
Tomasz Grabiec
0b0ea04958 range: Remove start_value() and end_value() 
It's easy to miss that they may be undefined. start() and end(), which
return optional<bound> const&, make it clear.
 2015-07-22 10:27:47 +02:00
Tomasz Grabiec
4a18693a23 db: Remove dead code 2015-07-22 10:27:47 +02:00
Pekka Enberg
757522c597 db/legacy_schema_tables: Convert convertSchemaToMutations() to C++ 
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
 2015-07-22 11:24:37 +03:00
Pekka Enberg
49d9ac716d message/messaging_service.hh: Add missing do_with.hh include 
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
 2015-07-22 11:24:37 +03:00
Raphael S. Carvalho
89698b0d1c db: dont rethrow exceptions for termination of compaction fiber 
broken_semaphore and seastar::gate_closed_exception exceptions are
used for regular termination of compaction fiber, which otherwise
would live forever. We shouldn't re-throw these exceptions, but
instead only print a log message.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
 2015-07-22 11:23:58 +03:00
Asias He
547d9c347e storage_service: Simplfy do_update_system_peers_table 
Using template helper.
 2015-07-22 11:22:18 +03:00
Avi Kivity
f2370725f5 Merge "Enable logger for gossip and storage_service" from Asias 2015-07-22 11:12:36 +03:00
Avi Kivity
0c4430cffc Merge "streaming: Support both pushing and pulling of data" 
"stream_plan.transfer_ranges() sends data from local to remote node.
stream_plan.request_ranges() asks remote to send data to local.

After streaming, both nodes contains all the keys."
 2015-07-22 10:13:10 +03:00
Paweł Dziepak
fc44658fa7 transport: add varint to type_codec::type_id_to_type 
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
 2015-07-22 08:50:54 +03:00
Asias He
54d482afe4 streaming: Test both pushing and pulling of data 
stream_plan.transfer_ranges() sends data from local to remote node.

stream_plan.request_ranges() asks remote to send data to local.

After streaming, both nodes contains all the keys.

$ cat /tmp/out1|grep "\[Stream"
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] Executing streaming plan for MYPLAN
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] Starting streaming to 127.0.0.2
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] Sending stream init for incoming stream
[Stream #9fd8c3c0-3023-11e5-b450-000000000000 ID#0] Beginning stream session with 127.0.0.2
[Stream #9fd8c3c0-3023-11e5-b450-000000000000 ID#0] Prepare completed.  Receiving 1 files(105553124400080 bytes), sending 1 files(105553124104160 bytes)
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] Session with 127.0.0.2 is complete
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] All sessions completed

$ cat /tmp/out2|grep "\[Stream"
[Stream #9fd8c3c0-3023-11e5-b450-000000000000 ID#0] Creating new streaming plan for MYPLAN
[Stream #9fd8c3c0-3023-11e5-b450-000000000000 ID#0] Received streaming plan for MYPLAN
[Stream #9fd8c3c0-3023-11e5-b450-000000000000 ID#0] Prepare completed.  Receiving 1 files(105553124104160 bytes), sending 1 files(105553124400080 bytes)
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] Session with 127.0.0.1 is complete
[Stream #9fd8c3c0-3023-11e5-b450-000000000000] All sessions completed

Node 1
$ sstable2json tmp/1/ks/*/la-1-big-Data.db | grep key | sort
{"key": "1",
{"key": "2",
{"key": "3",
{"key": "4",
{"key": "5",
{"key": "6",

Node 2
$ sstable2json tmp/2/ks/*/la-1-big-Data.db | grep key | sort
{"key": "1",
{"key": "2",
{"key": "3",
{"key": "4",
{"key": "5",
{"key": "6",
 2015-07-22 11:49:30 +08:00
Asias He
96049a99cf streaming: Add more debug print for stream_session::prepare 2015-07-22 11:49:30 +08:00
Asias He
1c60844727 streaming: Always start_streaming_files upon receiving of PREPARE_MESSAGE reply 2015-07-22 11:49:30 +08:00
Asias He
736c0bc08f streaming: Improve query::range<token> deserialization a bit 
query::range<token> is not serialized ATM. Always use full range for
now.
 2015-07-22 11:49:30 +08:00
Asias He
d39dd0f9d1 streaming: Add operator<< for stream_request 2015-07-22 11:49:30 +08:00
Asias He
d934b2c761 streaming: Fix prepare_message and stream_request deserialization 
vector(size_type count) constructs the container with count
default-inserted instances of T. So, current code will end up with 2*num
elements which is wrong.
 2015-07-22 11:49:30 +08:00
Asias He
f83d0bf7c4 streaming: Fix info printout format 2015-07-22 11:49:30 +08:00
Avi Kivity
6d2278f345 build: make libseastar.a depend on seastar configuration 
Should cause a rebuild if seastar configuration changes, for example due
to --enable-dpdk.
 2015-07-21 19:22:48 +03:00
Avi Kivity
006473b45c tests: improve capture of stdout/stderr 
Use a single pipe for both, instead of a temporary file.
 2015-07-21 18:44:31 +03:00
Avi Kivity
675a1ee8ea Merge "Add varint type" from Paweł 
"This patchset adds support for varint type."
 2015-07-21 18:21:21 +03:00
Asias He
ff457b018a gossip: Enable logger in gossiper class 2015-07-21 22:56:24 +08:00
Asias He
7ee4ae2ff7 storage_service: Use logger.error instead of print 2015-07-21 22:56:24 +08:00
Asias He
94e34cde64 storage_service: Drop ss_debug 2015-07-21 22:56:24 +08:00
Asias He
72a8c27b56 storage_service: Enable logger 2015-07-21 22:56:24 +08:00
Glauber Costa
0ea8a27b47 system_keyspace: stop the local cache. 
Apparently we need to register the stop function explicitly. That was not being
visible before, because the urchin binary was failing on exit before this for
some other reason. Once that was fixed, this one became apparent.

Scylla can correctly shutdown now.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
 2015-07-21 17:21:53 +03:00
Avi Kivity
cbc44d19cb build: fix build with dpdk 
Rip out the seastar-related dpdk configuration, replace with simply
forwarding dpdk configuration to seastar.
 2015-07-21 17:19:01 +03:00
Avi Kivity
4991d505c3 Merge seastar upstream 2015-07-21 17:18:46 +03:00
Paweł Dziepak
e24aa67177 tests/cql3: add varint to test_types 
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
 2015-07-21 14:47:17 +02:00
Paweł Dziepak
27c5bb6ada test/types: add tests for varint type 
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
 2015-07-21 14:46:54 +02:00
Paweł Dziepak
babb2fa261 types: add varint type 
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
 2015-07-21 14:46:34 +02:00
Avi Kivity
2c8a994cba Merge "Update system.peers table" from Asias 2015-07-21 13:03:40 +03:00
Avi Kivity
8ba5d19db5 db: avoid ubsan false-positive in query_state move constructor 
The value is moved before initialization due to a do_with().  It's harmless,
but better to silence the warning.
 2015-07-21 12:19:54 +03:00
Avi Kivity
c84fc789c5 cql3: avoid use-after-free of stack variable in create_keyspace_statement 
Capture it by value.
 2015-07-21 12:19:19 +03:00
Asias He
344d8e95d1 storage_service: Update system.peers table 
Before:

   cqlsh> SELECT * from system.peers ;
   cqlsh:system> SELECT * FROM system.peers ;
    peer | data_center | host_id | preferred_ip | rack | release_version | rpc_address | schema_version | tokens
   ------+-------------+---------+--------------+------+-----------------+-------------+----------------+--------
   (0 rows)

After:

   cqlsh> SELECT * from system.peers ;
    peer      | data_center | host_id                              | preferred_ip | rack  | release_version | rpc_address | schema_version | tokens
   -----------+-------------+--------------------------------------+--------------+-------+-----------------+-------------+----------------+--------
    127.0.0.2 | datacenter1 | 7daa116a-03d0-4623-8084-f213701f3136 |         null | rack1 |      urchin_1_0 |   127.0.0.2 |           null |   null
   (1 rows)
 2015-07-21 17:10:56 +08:00
Asias He
d8a281e811 storage_service: Use broadcast_address as broadcast_rpc_address for now 
Until we can get it from the config system.

It is better than a empty address.
 2015-07-21 17:00:15 +08:00
Asias He
b16eb27c58 gms/versioned_value: Fix rpcaddress 2015-07-21 17:00:15 +08:00
Asias He
ba04f0bc6b gms/versioned_value: Fix network_version 2015-07-21 17:00:15 +08:00
Asias He
b8be013934 messaging_service: Fix undefined reference to current_version 
/usr/include/boost/format/feed_args.hpp:135: undefined reference to
`net::messaging_service::current_version'
 2015-07-21 17:00:15 +08:00
Asias He
268f2fa8a1 db/system_keyspace: Change host_id to uuid_type 2015-07-21 17:00:15 +08:00
Avi Kivity
c1e25e40f0 Merge "Streaming updates" from Asias 
"With this series:

1) We can verify that data from Node A to Node B are streamed correctly.

2) Session completion are handled now.

Node A:
[Stream #08a2d480-2f7b-11e5-ae28-000000000000] Session with 127.0.0.2 is complete
[Stream #08a2d480-2f7b-11e5-ae28-000000000000] All sessions completed

Node B:
[Stream #08a2d480-2f7b-11e5-ae28-000000000000] Session with 127.0.0.1 is complete
[Stream #08a2d480-2f7b-11e5-ae28-000000000000] All sessions completed"
 2015-07-21 11:29:26 +03:00