Commit Graph

16 Commits

Author SHA1 Message Date
Asias He
f9109c33ba streaming: Implement stream_transfer_task completion logic 2015-07-21 16:12:54 +08:00
Asias He
f2960a7cb0 streaming: Send plan_id for STREAM_MUTATION
We need this to find session associated with this frozen_mutation.
2015-07-21 16:12:54 +08:00
Asias He
ccb32ceec5 streaming: Add stream_transfer_task::complete 2015-07-21 16:12:54 +08:00
Asias He
8561315cf2 streaming: de-thread_local-ize logger 2015-07-21 16:12:54 +08:00
Asias He
857fa5ccbb messaging_service: Add wrapper for STREAM_MUTATION verb 2015-07-16 17:19:51 +08:00
Asias He
d720dadf7b streaming: Switch to use logger class 2015-07-14 20:56:28 +08:00
Asias He
e82bdf2995 streaming: Swith to use shared_ptr from std::shared_ptr
Since our shared_ptr works with incomplete types now, switch to it.
2015-07-14 20:41:14 +08:00
Asias He
8fd8f39d63 streaming: Add more debug info for message exchange 2015-07-14 20:41:14 +08:00
Asias He
ca7f5ca5c9 streaming: Set proper dst_cpu_id in shard_id for PREPARE_MESSAGE and STREAM_MUTATION 2015-07-14 20:41:14 +08:00
Asias He
14ae9e66ae streaming: Use shared_ptr to track back to stream_session
I tried our lw_shared_ptr, the compiler complained endless usage of
incomplete type stream_session. I can not include stream_session.hh
everywhere due to circular dependency.

For now, I'm using std::shared_ptr which works fine.
2015-07-14 20:41:14 +08:00
Asias He
b7b0aa3318 streaming: Negotiate core to core connection.
In streaming code, we need core to core connection(the second connection
from B to A). That is when node A initiates a stream to node B, it is
possible that node A will transfer data to node B and vice verse, so we
need two connections. When node A creates a tcp connection (within the
messaging_service) to node B, we have a connection ip_a:core_a to
ip_b:core_b. When node B creates a connection to node B, we can not
guarantee it is ip_b:core_b to ip_a:core_a.

Current messaging_service does not support core to core connection yet,
although we use shard_id{ip, cpu_id} as the destination of the message.

We can solve the issue in upper layer. We can pass extra cpu_id as a
user msg.

Node A sends stream_init_message with my_cpu_id = current_cpu_id

Node B receives stream_init_message, it runs on whatever cpu this
connection goes to, then it sends response back with Node B's
current_cpu_id.

After this, each node knows which cpu_id to send to each other.

TODO: we need to handle the case when peer node reboots with different
number of cpus.
2015-07-09 15:52:28 +08:00
Asias He
3256a21556 streaming: Use frozen_mutation to send mutations
Each outgoing_file_message might contain multiple mutations. Send them
one mutation per RPC call (using frozen_mutation), instead of one big
outgoing_file_message per one RPC call.
2015-07-09 15:52:28 +08:00
Asias He
4718211d4a streaming: Wire up stream_transfer_task::add_transfer_file
Wire up with outgoing_file_message
2015-07-09 15:52:27 +08:00
Asias He
ad3692f666 streaming: Implement stream_session::add_transfer_ranges
Given keyspace names, ranges and column_families names, figure out
mutation_readers to transfer.
2015-07-09 15:52:27 +08:00
Asias He
4c9af76261 streaming: Move add_transfer_file to source file
Reduce dependency to stream_session
2015-06-24 16:13:30 +08:00
Asias He
334b1f81fc streaming: Convert StreamTransferTask.java to C++ 2015-06-18 14:55:07 +08:00