Commit Graph

44 Commits

Author SHA1 Message Date
Asias He
d720dadf7b streaming: Switch to use logger class 2015-07-14 20:56:28 +08:00
Asias He
e82bdf2995 streaming: Swith to use shared_ptr from std::shared_ptr
Since our shared_ptr works with incomplete types now, switch to it.
2015-07-14 20:41:14 +08:00
Asias He
38ee079916 streaming: Add test helper function
This is a very preliminary test to make sure negotiating between two
nodes is ok.
2015-07-14 20:41:14 +08:00
Asias He
01aa42ddca streaming: Add streaming_debug
Add debug print for message exchange
2015-07-14 20:41:14 +08:00
Asias He
85d9204d0e streaming: Drop connection_handler
stream_session::stream_session(inet_address peer_, inet_address connecting_,
    int index_, bool keep_ss_table_level_)
    : peer(peer_)
    , connecting(connecting_)
    , conn_handler(shared_from_this())

Calling shared_from_this() inside stream_session's constructor is
problematic. I got

   Exiting on unhandled exception of type 'std::bad_weak_ptr': bad_weak_ptr

exceptions, with

   auto session = std::make_shared<stream_session>(peer, connecting, size, _keep_ss_table_level)

Also, the logic in connection_handler is not very useful for us. The
sending and receiving of messages are handled using messaging_service.
There is no need to add another layer.
2015-07-14 20:41:14 +08:00
Asias He
14ae9e66ae streaming: Use shared_ptr to track back to stream_session
I tried our lw_shared_ptr, the compiler complained endless usage of
incomplete type stream_session. I can not include stream_session.hh
everywhere due to circular dependency.

For now, I'm using std::shared_ptr which works fine.
2015-07-14 20:41:14 +08:00
Asias He
5ffb946dac streaming: futurize stream_session::on_initialization_complete 2015-07-14 20:41:14 +08:00
Asias He
b7b0aa3318 streaming: Negotiate core to core connection.
In streaming code, we need core to core connection(the second connection
from B to A). That is when node A initiates a stream to node B, it is
possible that node A will transfer data to node B and vice verse, so we
need two connections. When node A creates a tcp connection (within the
messaging_service) to node B, we have a connection ip_a:core_a to
ip_b:core_b. When node B creates a connection to node B, we can not
guarantee it is ip_b:core_b to ip_a:core_a.

Current messaging_service does not support core to core connection yet,
although we use shard_id{ip, cpu_id} as the destination of the message.

We can solve the issue in upper layer. We can pass extra cpu_id as a
user msg.

Node A sends stream_init_message with my_cpu_id = current_cpu_id

Node B receives stream_init_message, it runs on whatever cpu this
connection goes to, then it sends response back with Node B's
current_cpu_id.

After this, each node knows which cpu_id to send to each other.

TODO: we need to handle the case when peer node reboots with different
number of cpus.
2015-07-09 15:52:28 +08:00
Asias He
853175fc61 streaming: Implement prepare_message handler
This is a bit different from Origin. We always send back a
prepare_message even if the initializer requested no data from the
follower, to unify the handling.
2015-07-09 15:52:28 +08:00
Asias He
1dd80bac65 streaming: Make get_local_db() and ms() public 2015-07-09 15:52:28 +08:00
Asias He
ad3692f666 streaming: Implement stream_session::add_transfer_ranges
Given keyspace names, ranges and column_families names, figure out
mutation_readers to transfer.
2015-07-09 15:52:27 +08:00
Asias He
4b676bfac5 streaming: Init streaming service in main
We need a reference to <distributed> db.
2015-07-09 15:52:27 +08:00
Asias He
dec8bee44d streaming: Add stream_session::description 2015-06-30 17:12:37 +08:00
Asias He
1e5cfa89d1 streaming: Kill a FIXME in stream_session::plan_id 2015-06-30 17:08:45 +08:00
Asias He
92dd0616fa streaming: Wire up stream_result_future in stream_session 2015-06-30 16:39:12 +08:00
Asias He
878611beee streaming: Enable connect_all_stream_sessions 2015-06-30 15:25:53 +08:00
Asias He
fcecca89af streaming: Rename start to init_streaming_service
stream_session::start is used to start a session instead of
initialization on startup.
2015-06-30 15:07:19 +08:00
Asias He
c0cd69517a streaming: Implement stream_session::file_sent 2015-06-26 15:04:33 +08:00
Asias He
6a92924a75 streaming: Add stream_session::on_error 2015-06-26 14:49:24 +08:00
Asias He
1134b99610 streaming: Add stream_session::on_initialization_complete 2015-06-26 14:39:22 +08:00
Asias He
f7542e7cf0 streaming: Implement stream_session::add_transfer_files 2015-06-26 14:24:10 +08:00
Asias He
9cbed3e4e1 streaming: Make stream_session::_handlers static
It is used to register the streaming message handler once. We don't need
it in every stream_session instance. The side effect is making
stream_session copyable, which is needed in get_all_stream_sessions.
2015-06-26 09:41:11 +08:00
Asias He
a4235ebc13 streaming: Implement transfer_files 2015-06-26 08:31:28 +08:00
Asias He
89c3cab526 streaming: Add stream_session::close_session 2015-06-26 08:31:28 +08:00
Asias He
1a34df6188 streaming: Add stream_session::task_completed 2015-06-26 08:31:28 +08:00
Asias He
af836e4670 streaming: Add stream_session::get_session_info 2015-06-26 08:31:28 +08:00
Asias He
f2dc73b34d streaming: Add stream_session::session_failed 2015-06-26 08:31:28 +08:00
Asias He
50de45dff7 streaming: Add stream_session::complete 2015-06-26 08:31:28 +08:00
Asias He
e8d9e7f3c5 streaming: Add stream_session::retry 2015-06-26 08:31:28 +08:00
Asias He
c5729ab0c3 streaming: Add stream_session::received 2015-06-26 08:31:28 +08:00
Asias He
2097cac719 streaming: Add stream_session::progress 2015-06-26 08:31:28 +08:00
Asias He
f113b9a199 streaming: Add stream_session::receive 2015-06-26 08:31:28 +08:00
Asias He
706ca47574 streaming: Add stream_session::prepare 2015-06-26 08:31:28 +08:00
Asias He
75f188276d streaming: Add stream_session::add_stream_request 2015-06-26 08:31:28 +08:00
Asias He
a936ff0ac3 streaming: Add class ss_table_streaming_sections 2015-06-24 16:13:30 +08:00
Asias He
8c3294d9d7 streaming: Add constructor for stream_session 2015-06-24 16:13:30 +08:00
Asias He
4a7ef682ab streaming: Use stream_session_state
In session_info, stream_coordinator and stream_session
2015-06-24 16:13:30 +08:00
Asias He
934c17fa6b streaming: Add set_state and get_state to stream_session()
Note: Origin uses name sate() for the function name. We can not since it
conflicts with the name of enum class stream_session::state.
2015-06-24 16:13:30 +08:00
Asias He
8ecacca2a9 streaming: Add keep_ss_table_level to stream_session 2015-06-24 14:18:11 +08:00
Asias He
739cf1b3ef streaming: Add is_success() and plan_id() to stream_session 2015-06-19 15:07:06 +08:00
Asias He
5ed8f689f7 streaming: Add gossip callback 2015-06-18 23:42:50 +08:00
Asias He
3f97d610f3 streaming: Add stream_session::state 2015-06-18 23:02:31 +08:00
Asias He
a6950a6d98 streaming: Add init_messaging_service_handler
We will add handlers for streaming related verbs soon.
2015-06-17 16:08:37 +08:00
Asias He
ea0b6931b0 streaming: Convert StreamSession.java to C++ 2015-06-17 16:08:37 +08:00