mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-06-02 21:17:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,716 Commits 257 Branches 556 Tags
82394debe68f450dffd19187a99655f4e433044b
Commit Graph

11716 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Asias He
ad4a096b80 streaming: Get rid of stream_init_message 
Unlike streaming in c*, scylla does not need to open tcp connections in
streaming service for both incoming and outgoing messages, seastar::rpc
does the work. There is no need for a standalone stream_init_message
message in the streaming negotiation stage, we can merge the
stream_init_message into stream_prepare_message.
 2016-01-25 16:24:16 +08:00
Asias He
048965ea02 streaming: Do not print session_index in handle_session_prepared 
session_index is always 0. It will be removed soon.
 2016-01-25 16:24:16 +08:00
Avi Kivity
449b81f5d3 Merge "streaming cleanup" from Asias 
"No mercy to the unused parameters and messages.
This will help the upcoming IDL serialize/deserialize work."
 2016-01-25 10:21:16 +02:00
Avi Kivity
9ebd3f8098 Merge "Move gossip to use IDL" from Asias 
"This changes gossip to use IDL based serialization code."
 2016-01-25 10:18:34 +02:00
Asias He
20496ed9a8 tests: Stop gossip during shutdown in cql_test_env 
Fixes the heap-use-after-free error in build/debug/tests/auth_test

==1415==ERROR: AddressSanitizer: heap-use-after-free on address
0x62200032cfa8 at pc 0x00000350701d bp 0x7fec96df8d40 sp
0x7fec96df8d30
READ of size 8 at 0x62200032cfa8 thread T1
    #0 0x350701c in
_ZZN3gms8gossiper3runEvENKUlOT_E0_clI6futureIJEEEEDaS2_
(/home/penberg/scylla/build/debug/tests/auth_test_g+0x350701c)
    #1 0x35795b1 in apply<gms::gossiper::run()::<lambda(auto:40&&)>,
future<> > /home/penberg/scylla/seastar/core/future.hh:1203
    #2 0x369103d in
_ZZN6futureIJEE12then_wrappedIZN3gms8gossiper3runEvEUlOT_E0_S0_EET0_S5_ENUlS5_E_clI12future_stateIJEEEEDaS5_
(/home/penberg/scylla/build/debug/tests/auth_test_g+0x369103d)
    #3 0x369182a in run /home/penberg/scylla/seastar/core/future.hh:399
    #4 0x435f24 in
reactor::run_tasks(circular_buffer<std::unique_ptr<task,
std::default_delete<task> >, std::allocator<std::unique_ptr<task,
std::default_delete<task> > > >&) core/reactor.cc:1368
    #5 0x43a44f in reactor::run() core/reactor.cc:1672
    #6 0x952e4b in app_template::run_deprecated(int, char**,
std::function<void ()>&&) core/app-template.cc:123
    #7 0x58dc79d in test_runner::start(int,
char**)::{lambda()#1}::operator()()
(/home/penberg/scylla/build/debug/tests/auth_test_g+0x58dc79d)
    #8 0x58e6cd6 in _M_invoke /usr/include/c++/5.3.1/functional:1871
    #9 0x688639 in std::function<void ()>::operator()() const
/usr/include/c++/5.3.1/functional:2271
    #10 0x8d939c in posix_thread::start_routine(void*) core/posix.cc:51
    #11 0x7feca02a4609 in start_thread (/lib64/libpthread.so.0+0x7609)
    #12 0x7fec9ffdea4c in clone (/lib64/libc.so.6+0x102a4c)

0x62200032cfa8 is located 5800 bytes inside of 5808-byte region
[0x62200032b900,0x62200032cfb0)
freed by thread T1 here:
    #0 0x7feca4f76472 in operator delete(void*, unsigned long)
(/lib64/libasan.so.2+0x9a472)
    #1 0x3740772 in gms::gossiper::~gossiper()
(/home/penberg/scylla/build/debug/tests/auth_test_g+0x3740772)
    #2 0x2588ba1 in shared_ptr<gms::gossiper>::~shared_ptr()
seastar/core/shared_ptr.hh:389
    #3 0x4fc908c in
seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1}::~stop()
(/home/penberg/scylla/build/debug/tests/auth_test_g+0x4fc908c)
    #4 0x4ff722a in future<>
future<>::then<seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1}, future<>
>(seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1}&&)::{lambda(seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1})#1}::~then()
(/home/penberg/scylla/build/debug/tests/auth_test_g+0x4ff722a)
    #5 0x509a28c in continuation<future<>
future<>::then<seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1}, future<>
>(seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1}&&)::{lambda(seastar::sharded<gms::gossiper>::stop()::{lambda(unsigned
int)#1}::operator()(unsigned
int)::{lambda()#1}::operator()()::{lambda()#1})#1}>::~continuation()
seastar/core/future.hh:395
    #6 0x509a40d in continuation<future<>
Message-Id: <f8f1c92c1eb88687ab0534f5e7874d53050a5b93.1453446350.git.asias@scylladb.com>
 2016-01-25 08:19:18 +02:00
Asias He
bc4ac5004e streaming: Kill stream_result_future::create_and_register 
The helper is used only once in init_sending_side and in
init_receiving_side we do not use create_and_register to create
stream_result_future. Kill the trivial helper to make the code more
consistent.

In addition, rename variables "future" and "f" to sr (streaming_result).
 2016-01-25 11:38:13 +08:00
Asias He
face74a8f2 streaming: Rename stream_result_future::init to ::init_sending_side 
So we have:

- init_sending_side
  called when the node initiates a stream_session

- init_receiving_side
  called when the node is a receiver of a stream_session initiated by a peer
 2016-01-25 11:38:13 +08:00
Asias He
dc94c5e42e streaming: Rename get_or_create_next_session to get_or_create_session 
There is only one session for each peer in stream_coordinator.
 2016-01-25 11:38:13 +08:00
Asias He
e46d4166f2 streaming: Refactor host_streaming_data 
In scylla, in each stream_coordinator, there will be only one
stream_session for each remote peer. Drop the code supporting multiple
stream_sessions in host_streaming_data.

We now have

   shared_ptr<stream_session> _stream_session

instead of

   std::map<int, shared_ptr<stream_session>> _stream_sessions
 2016-01-25 11:38:13 +08:00
Asias He
8a4b563729 streaming: Drop the get_or_create_session_by_id interafce 
The session index will always be 0 in stream_coordinator. Drop the api for it.
 2016-01-25 11:38:13 +08:00
Asias He
9a346d56b9 streaming: Drop unnecessary parameters in stream_init_message 
- from
  We can get it form the rpc::client_info

- session_index
  There will always be one session in stream_coordinator::host_streaming_data with a peer.

- is_for_outgoing
  In cassandra, it initiates two tcp connections, one for incoming stream and one for outgoing stream.
  logger.debug("[Stream #{}] Sending stream init for incoming stream", session.planId());
  logger.debug("[Stream #{}] Sending stream init for outgoing stream", session.planId());
  In scylla, it only initiates one "connection" for sending, the peer initiates another "connection" for receiving.
  So, is_for_outgoing will also be true in scylla, we can drop it.

- keep_ss_table_level
  In scylla, again, we stream mutations instead of sstable file. It is
  not relevant to us.
 2016-01-25 11:38:13 +08:00
Asias He
1bc5cd1b22 streaming: Drop streaming/messages/session_failed_message 
It is not used.
 2016-01-25 11:38:13 +08:00
Asias He
2a04e8d70e streaming: Drop streaming/messages/incoming_file_message 
It is not used.
 2016-01-25 11:38:13 +08:00
Asias He
26ba21949e streaming: Drop streaming/messages/retry_message 
It is not used.
 2016-01-25 11:38:13 +08:00
Asias He
4b4363b62d streaming: Drop streaming/messages/received_message 
It is not used.
 2016-01-25 11:38:13 +08:00
Asias He
b3e00472ed streaming: Drop streaming/streaming.cc 
It is used in the early stage of development to make sure things compile.
 2016-01-25 11:38:13 +08:00
Asias He
5a0bf10a0b streaming: Drop streaming/messages/complete_message 
It is not used.
 2016-01-25 11:38:13 +08:00
Asias He
bdd6a69af7 streaming: Drop unused parameters 
- int connections_per_host

Scylla does not create connections per stream_session, instead it uses
rpc, thus connections_per_host is not relevant to scylla.

- bool keep_ss_table_level
- int repaired_at

Scylla does not stream sstable files. They are not relevant to scylla.
 2016-01-25 11:38:13 +08:00
Asias He
7b633ad127 gossip: Drop unused serialization code 
- heart_beat_state
 2016-01-25 11:28:29 +08:00
Asias He
4ce08ff251 messaging_service: Add heart_beat_state implementation 2016-01-25 11:28:29 +08:00
Asias He
d7c7994f37 gossip: Drop unused serialization code 
- versioned_value
 2016-01-25 11:28:29 +08:00
Asias He
8098ba10b7 gossip: Drop unused serialization code 
- endpoint_state
 2016-01-25 11:28:29 +08:00
Asias He
ecca969adf messaging_service: Add gossip::endpoint_state implementation 2016-01-25 11:28:29 +08:00
Asias He
2a0b6589dd messaging_service: Add versioned_value implementation 2016-01-25 11:28:29 +08:00
Asias He
6660658742 gossip: Drop unused serialization code 
- gossip_digest_serialization_helper
- gossip_digest
 2016-01-25 11:28:29 +08:00
Asias He
15f2b353b9 messaging_service: Add gossip_digest implementation 2016-01-25 11:28:29 +08:00
Asias He
736d21a912 gossip: Drop unused serialization code 
- gossip_digest_syn
- gossip_digest_ack
- gossip_digest_ack2
 2016-01-25 11:28:29 +08:00
Asias He
d81fc12af3 messaging_service: Add gossip_digest_ack2 implementation 2016-01-25 11:28:29 +08:00
Asias He
e67cecaee1 messaging_service: Add gossip_digest_syn implementation 2016-01-25 11:28:29 +08:00
Asias He
d94b7e49d2 idl: Add gossip_digest_syn 
Added get_partioner and get_cluster_id
 2016-01-25 11:28:28 +08:00
Asias He
60f5891c3f idl: Add gossip_digest_ack2 2016-01-25 11:28:26 +08:00
Takuya ASADA
0f0d1c7aed dist: don't depends on libvirtd, since we are not using it 
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453470970-31036-1-git-send-email-syuu@scylladb.com>
 2016-01-24 17:15:13 +02:00
Avi Kivity
65a140481c Merge " streaming COMPLETE_MESSAGE failure and message retry logic fix" from Asias 
"This series:

- Add more debug info to stream session
- Fail session if we fail to send COMPLETE_MESSAGE
- Handle message retry logic for verbs used by streaming

See commit log for details."
 2016-01-24 16:41:06 +02:00
Avi Kivity
6135e0ae78 Merge "Move read/write mutation path to use IDL" from Gleb 2016-01-24 13:35:04 +02:00
Avi Kivity
b415f87324 Merge "Serializer Deserializer code generation" from Amnon 
"The series do the following:
It adds the code generation
Perform the needed changes in the current classes so each would have getter for
each of its serializable value and a constructor from the serialized values.
It adds a schema definition that cover gossip_diget_ack
It changes the messaging_service to use the generated code.

An overall explanation of the solution with a description of the schema IDL can
be found on the wiki page:

https://github.com/scylladb/scylla/wiki/Serializer-Deserializer-Code-generation
"
 2016-01-24 12:56:42 +02:00
Gleb Natapov
b9b6f703c3 Remove old serializer for frozen_mutation and reconcilable_result 2016-01-24 12:45:41 +02:00
Gleb Natapov
067bdb23cd Move reconcilable_result and frozen_mutation to idl 2016-01-24 12:45:41 +02:00
Gleb Natapov
18dff5ebc8 Move smart pointer serialization helpers to .cc file. 
They are not used outside of the .cc file, so should not be in the
header.
 2016-01-24 12:45:41 +02:00
Gleb Natapov
93da9b2725 Remove redundant vector serialization code. 
IDL serializer has the code to serialize vectors, so use it instead.
 2016-01-24 12:45:41 +02:00
Gleb Natapov
ab6703f9bc Remove old query::result serializer 2016-01-24 12:45:41 +02:00
Gleb Natapov
afc407c6e5 Move query::result to use idl. 2016-01-24 12:45:41 +02:00
Gleb Natapov
be4e68adbf Add bytes_ostream serializer. 2016-01-24 12:45:41 +02:00
Gleb Natapov
043d132ba9 Remove no longer used serializers. 2016-01-24 12:45:41 +02:00
Gleb Natapov
4ae906b204 Add serializer overload for query::partition_range. 
From now on query::partition_range will use generated code.
 2016-01-24 12:45:41 +02:00
Gleb Natapov
2d1b2765e6 Add serializer overload for query::read_command. 
From now on query::read_command will use generated code.
 2016-01-24 12:45:41 +02:00
Gleb Natapov
49ce2b83df Add ring_position constructor needed by serializer. 2016-01-24 12:45:41 +02:00
Gleb Natapov
6cc5b15a9c Fix read_command constructor to not copy parameters. 2016-01-24 12:45:41 +02:00
Gleb Natapov
4384c7fe85 un-nest range::bound class. 
Serializer does not support nested classes yet, so move bound outside.
 2016-01-24 12:45:41 +02:00
Gleb Natapov
7357b1ddfe Move specific_ranges to .hh and un-nest it. 
Serializer requires class to be defined, so it has to be in .h file. It
also does not support nested types yet, so move it outside of containing
class.
 2016-01-24 12:45:41 +02:00
Gleb Natapov
9ae7dc70da Prepare partition_slice to be used by serializer. 
Add missing _specific_ranges getter and setter.
 2016-01-24 12:45:41 +02:00