Asias He
b7b0aa3318
streaming: Negotiate core to core connection.
...
In streaming code, we need core to core connection(the second connection
from B to A). That is when node A initiates a stream to node B, it is
possible that node A will transfer data to node B and vice verse, so we
need two connections. When node A creates a tcp connection (within the
messaging_service) to node B, we have a connection ip_a:core_a to
ip_b:core_b. When node B creates a connection to node B, we can not
guarantee it is ip_b:core_b to ip_a:core_a.
Current messaging_service does not support core to core connection yet,
although we use shard_id{ip, cpu_id} as the destination of the message.
We can solve the issue in upper layer. We can pass extra cpu_id as a
user msg.
Node A sends stream_init_message with my_cpu_id = current_cpu_id
Node B receives stream_init_message, it runs on whatever cpu this
connection goes to, then it sends response back with Node B's
current_cpu_id.
After this, each node knows which cpu_id to send to each other.
TODO: we need to handle the case when peer node reboots with different
number of cpus.
2015-07-09 15:52:28 +08:00
Asias He
853175fc61
streaming: Implement prepare_message handler
...
This is a bit different from Origin. We always send back a
prepare_message even if the initializer requested no data from the
follower, to unify the handling.
2015-07-09 15:52:28 +08:00
Asias He
abfcf7d825
streaming: Send prepare_message message
2015-07-09 15:52:28 +08:00
Asias He
7ae860a43a
streaming: Enable init_receiving_side
2015-07-09 15:52:28 +08:00
Asias He
f1dc4f21d5
streaming: Send stream_init_message to remote
2015-07-09 15:52:28 +08:00
Asias He
3256a21556
streaming: Use frozen_mutation to send mutations
...
Each outgoing_file_message might contain multiple mutations. Send them
one mutation per RPC call (using frozen_mutation), instead of one big
outgoing_file_message per one RPC call.
2015-07-09 15:52:28 +08:00
Asias He
1dd80bac65
streaming: Make get_local_db() and ms() public
2015-07-09 15:52:28 +08:00
Asias He
1aab3c7bb9
streaming: Drop serialization interface for {outgoing,incoming}_file_message
...
We will send mutations using the frozen_mutation verb
2015-07-09 15:52:28 +08:00
Asias He
4718211d4a
streaming: Wire up stream_transfer_task::add_transfer_file
...
Wire up with outgoing_file_message
2015-07-09 15:52:27 +08:00
Asias He
ad3692f666
streaming: Implement stream_session::add_transfer_ranges
...
Given keyspace names, ranges and column_families names, figure out
mutation_readers to transfer.
2015-07-09 15:52:27 +08:00
Asias He
3d42a9f9ad
streaming: Introduce stream_detail
...
It is used to hold mutations which we will send over network.
2015-07-09 15:52:27 +08:00
Asias He
4b676bfac5
streaming: Init streaming service in main
...
We need a reference to <distributed> db.
2015-07-09 15:52:27 +08:00
Asias He
dec8bee44d
streaming: Add stream_session::description
2015-06-30 17:12:37 +08:00
Asias He
1e5cfa89d1
streaming: Kill a FIXME in stream_session::plan_id
2015-06-30 17:08:45 +08:00
Asias He
7ec2ee6b86
streaming: Add stream_plan::listeners
2015-06-30 16:55:30 +08:00
Asias He
6f0994349a
streaming: Add stream_plan::execute
2015-06-30 16:47:25 +08:00
Asias He
92dd0616fa
streaming: Wire up stream_result_future in stream_session
2015-06-30 16:39:12 +08:00
Asias He
212a70f42f
streaming: Add constructor for stream_result_future
2015-06-30 16:08:53 +08:00
Asias He
60449c4b59
streaming: Convert StreamResultFuture to C++
2015-06-30 16:00:56 +08:00
Asias He
ee1f4f022b
streaming: Import StreamResultFuture.java
2015-06-30 15:48:21 +08:00
Asias He
abf24b3bfa
streaming: Add flush_before_transfer to stream_plan
2015-06-30 15:38:18 +08:00
Asias He
878611beee
streaming: Enable connect_all_stream_sessions
2015-06-30 15:25:53 +08:00
Asias He
fcecca89af
streaming: Rename start to init_streaming_service
...
stream_session::start is used to start a session instead of
initialization on startup.
2015-06-30 15:07:19 +08:00
Asias He
e8156ca552
streaming: Implement serialize interface for file_message_header
2015-06-30 12:19:01 +08:00
Asias He
b35d29f91f
streaming: Implement serialize interface for prepare_message
2015-06-30 11:43:29 +08:00
Asias He
3beb3e749d
streaming: Implement serialize interface for stream_summary
2015-06-30 11:14:21 +08:00
Asias He
72394f3739
streaming: Implement serialize interface for stream_request
2015-06-30 11:07:04 +08:00
Asias He
237fedb1fc
streaming: Implement serialize interface for received_message
2015-06-30 10:37:49 +08:00
Asias He
85ecd6b2f2
streaming: Implement serialize interface for retry_message
2015-06-30 10:33:13 +08:00
Asias He
8c67244a13
streaming: Implement serialize interface for stream_init_message
2015-06-30 10:33:13 +08:00
Asias He
c0cd69517a
streaming: Implement stream_session::file_sent
2015-06-26 15:04:33 +08:00
Asias He
6a92924a75
streaming: Add stream_session::on_error
2015-06-26 14:49:24 +08:00
Asias He
1134b99610
streaming: Add stream_session::on_initialization_complete
2015-06-26 14:39:22 +08:00
Asias He
8aa6c8f3a9
streaming: Wire up two more close_session calls
2015-06-26 14:25:52 +08:00
Asias He
f7542e7cf0
streaming: Implement stream_session::add_transfer_files
2015-06-26 14:24:10 +08:00
Asias He
66a9f92df9
streaming: Move stream_coordinator code to source file
2015-06-26 10:07:57 +08:00
Asias He
78f5aaf265
streaming: Add stream_coordinator::get_peers
2015-06-26 09:45:32 +08:00
Asias He
774f677097
streaming: Add stream_coordinator::is_receiving
2015-06-26 09:43:01 +08:00
Asias He
c1d6b73838
streaming: Add stream_coordinator::get_all_stream_sessions
2015-06-26 09:41:22 +08:00
Asias He
9cbed3e4e1
streaming: Make stream_session::_handlers static
...
It is used to register the streaming message handler once. We don't need
it in every stream_session instance. The side effect is making
stream_session copyable, which is needed in get_all_stream_sessions.
2015-06-26 09:41:11 +08:00
Asias He
ec8a0e2566
streaming: Add add_session_info and get_all_session_info
2015-06-26 09:21:21 +08:00
Asias He
a16623dae2
streaming: Add stream_coordinator::update_progress
2015-06-26 08:51:57 +08:00
Asias He
fa1f4707af
streaming: Implement get_or_create_session_by_id
2015-06-26 08:31:28 +08:00
Asias He
794c65e58c
streaming: Complete transfer_ranges and request_ranges in stream_plan
2015-06-26 08:31:28 +08:00
Asias He
a4235ebc13
streaming: Implement transfer_files
2015-06-26 08:31:28 +08:00
Asias He
89c3cab526
streaming: Add stream_session::close_session
2015-06-26 08:31:28 +08:00
Asias He
1a34df6188
streaming: Add stream_session::task_completed
2015-06-26 08:31:28 +08:00
Asias He
af836e4670
streaming: Add stream_session::get_session_info
2015-06-26 08:31:28 +08:00
Asias He
f2dc73b34d
streaming: Add stream_session::session_failed
2015-06-26 08:31:28 +08:00
Asias He
50de45dff7
streaming: Add stream_session::complete
2015-06-26 08:31:28 +08:00
