Commit Graph

37 Commits

Author SHA1 Message Date
Asias He
b7b0aa3318 streaming: Negotiate core to core connection.
In streaming code, we need core to core connection(the second connection
from B to A). That is when node A initiates a stream to node B, it is
possible that node A will transfer data to node B and vice verse, so we
need two connections. When node A creates a tcp connection (within the
messaging_service) to node B, we have a connection ip_a:core_a to
ip_b:core_b. When node B creates a connection to node B, we can not
guarantee it is ip_b:core_b to ip_a:core_a.

Current messaging_service does not support core to core connection yet,
although we use shard_id{ip, cpu_id} as the destination of the message.

We can solve the issue in upper layer. We can pass extra cpu_id as a
user msg.

Node A sends stream_init_message with my_cpu_id = current_cpu_id

Node B receives stream_init_message, it runs on whatever cpu this
connection goes to, then it sends response back with Node B's
current_cpu_id.

After this, each node knows which cpu_id to send to each other.

TODO: we need to handle the case when peer node reboots with different
number of cpus.
2015-07-09 15:52:28 +08:00
Asias He
853175fc61 streaming: Implement prepare_message handler
This is a bit different from Origin. We always send back a
prepare_message even if the initializer requested no data from the
follower, to unify the handling.
2015-07-09 15:52:28 +08:00
Asias He
abfcf7d825 streaming: Send prepare_message message 2015-07-09 15:52:28 +08:00
Asias He
7ae860a43a streaming: Enable init_receiving_side 2015-07-09 15:52:28 +08:00
Asias He
f1dc4f21d5 streaming: Send stream_init_message to remote 2015-07-09 15:52:28 +08:00
Asias He
3256a21556 streaming: Use frozen_mutation to send mutations
Each outgoing_file_message might contain multiple mutations. Send them
one mutation per RPC call (using frozen_mutation), instead of one big
outgoing_file_message per one RPC call.
2015-07-09 15:52:28 +08:00
Asias He
1aab3c7bb9 streaming: Drop serialization interface for {outgoing,incoming}_file_message
We will send mutations using the frozen_mutation verb
2015-07-09 15:52:28 +08:00
Asias He
ad3692f666 streaming: Implement stream_session::add_transfer_ranges
Given keyspace names, ranges and column_families names, figure out
mutation_readers to transfer.
2015-07-09 15:52:27 +08:00
Asias He
4b676bfac5 streaming: Init streaming service in main
We need a reference to <distributed> db.
2015-07-09 15:52:27 +08:00
Asias He
dec8bee44d streaming: Add stream_session::description 2015-06-30 17:12:37 +08:00
Asias He
1e5cfa89d1 streaming: Kill a FIXME in stream_session::plan_id 2015-06-30 17:08:45 +08:00
Asias He
92dd0616fa streaming: Wire up stream_result_future in stream_session 2015-06-30 16:39:12 +08:00
Asias He
878611beee streaming: Enable connect_all_stream_sessions 2015-06-30 15:25:53 +08:00
Asias He
fcecca89af streaming: Rename start to init_streaming_service
stream_session::start is used to start a session instead of
initialization on startup.
2015-06-30 15:07:19 +08:00
Asias He
c0cd69517a streaming: Implement stream_session::file_sent 2015-06-26 15:04:33 +08:00
Asias He
6a92924a75 streaming: Add stream_session::on_error 2015-06-26 14:49:24 +08:00
Asias He
1134b99610 streaming: Add stream_session::on_initialization_complete 2015-06-26 14:39:22 +08:00
Asias He
8aa6c8f3a9 streaming: Wire up two more close_session calls 2015-06-26 14:25:52 +08:00
Asias He
f7542e7cf0 streaming: Implement stream_session::add_transfer_files 2015-06-26 14:24:10 +08:00
Asias He
9cbed3e4e1 streaming: Make stream_session::_handlers static
It is used to register the streaming message handler once. We don't need
it in every stream_session instance. The side effect is making
stream_session copyable, which is needed in get_all_stream_sessions.
2015-06-26 09:41:11 +08:00
Asias He
89c3cab526 streaming: Add stream_session::close_session 2015-06-26 08:31:28 +08:00
Asias He
1a34df6188 streaming: Add stream_session::task_completed 2015-06-26 08:31:28 +08:00
Asias He
af836e4670 streaming: Add stream_session::get_session_info 2015-06-26 08:31:28 +08:00
Asias He
f2dc73b34d streaming: Add stream_session::session_failed 2015-06-26 08:31:28 +08:00
Asias He
50de45dff7 streaming: Add stream_session::complete 2015-06-26 08:31:28 +08:00
Asias He
e8d9e7f3c5 streaming: Add stream_session::retry 2015-06-26 08:31:28 +08:00
Asias He
c5729ab0c3 streaming: Add stream_session::received 2015-06-26 08:31:28 +08:00
Asias He
2097cac719 streaming: Add stream_session::progress 2015-06-26 08:31:28 +08:00
Asias He
f113b9a199 streaming: Add stream_session::receive 2015-06-26 08:31:28 +08:00
Asias He
706ca47574 streaming: Add stream_session::prepare 2015-06-26 08:31:28 +08:00
Asias He
a9556768f3 streaming: Add SESSION_FAILED_MESSAGE handler 2015-06-17 16:55:11 +08:00
Asias He
7b84914562 streaming: Add COMPLETE_MESSAGE handler 2015-06-17 16:45:00 +08:00
Asias He
a9a115139a streaming: Add RETRY_MESSAGE handler 2015-06-17 16:29:43 +08:00
Asias He
11df6fafc3 streaming: Add OUTGOING_FILE_MESSAGE handler 2015-06-17 16:08:38 +08:00
Asias He
2abf015d82 streaming: Add PREPARE_MESSAGE handler 2015-06-17 16:08:38 +08:00
Asias He
35e5bb7c72 streaming: Add STREAM_INIT_MESSAGE handler 2015-06-17 16:08:37 +08:00
Asias He
a6950a6d98 streaming: Add init_messaging_service_handler
We will add handlers for streaming related verbs soon.
2015-06-17 16:08:37 +08:00