mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-04-21 00:50:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,960 Commits 235 Branches 538 Tags
94e34cde64d7604310aaa1ff42bea67fe053ebcd
Commit Graph

206 Commits

Author SHA1 Message Date
Asias He
19f46fdbe1 streaming: Remove redundant debug log info 2015-07-21 16:12:54 +08:00
Asias He
93f17024a8 streaming: Fix a logger printout 
Before:

[Stream #136c5310-2f7a-11e5-87df-000000000000 ID#0]
[Stream #136c5310-2f7a-11e5-87df-000000000000, ID#0]

After:

[Stream #136c5310-2f7a-11e5-87df-000000000000 ID#0]
[Stream #136c5310-2f7a-11e5-87df-000000000000 ID#0]
 2015-07-21 16:12:54 +08:00
Asias He
0e5fa35bd2 streaming: Register to stream_manager in create_and_register 2015-07-21 16:12:54 +08:00
Asias He
fc718dc87d streaming: Set up dst_cpu_id in PREPARE_MESSAGE hanlder 2015-07-21 16:12:54 +08:00
Asias He
ad86f5ea6e streaming: Fix get_or_create_host_data 
It is always creating a new host_streaming_data which is incorrect.
 2015-07-21 16:12:54 +08:00
Asias He
a010829f0c streaming: Add src_cpu_id parameter for PREPARE_MESSAGE verb 
We need it to setup dst_cpu_id for the session of the follower.
 2015-07-21 16:12:54 +08:00
Asias He
6712e9404e streaming: Implement session completion logic 2015-07-21 16:12:54 +08:00
Asias He
f9109c33ba streaming: Implement stream_transfer_task completion logic 2015-07-21 16:12:54 +08:00
Asias He
f2960a7cb0 streaming: Send plan_id for STREAM_MUTATION 
We need this to find session associated with this frozen_mutation.
 2015-07-21 16:12:54 +08:00
Asias He
ccb32ceec5 streaming: Add stream_transfer_task::complete 2015-07-21 16:12:54 +08:00
Asias He
34e33a9afe streaming: Wire up handle_session_complete and friends 2015-07-21 16:12:54 +08:00
Asias He
1f5feee5f2 streaming: Implement stream_result_future::handle_progress 2015-07-21 16:12:54 +08:00
Asias He
d74e414d09 streaming: Implement stream_result_future::handle_session_complete 2015-07-21 16:12:54 +08:00
Asias He
5e6f84cba8 streaming: Implement stream_result_future::handle_session_prepared 
Instead of playing the game of casting between stream_event and derived
class. We overload handle_stream_event with derived stream_event class.

virtual void handle_stream_event(session_complete_event event) {}
virtual void handle_stream_event(progress_event event) {}
virtual void handle_stream_event(session_prepared_event event) {}

Also, make the virtual function non pure virtual, so user can override
the interested event only without defining all of the three.
 2015-07-21 16:12:54 +08:00
Asias He
5ff5abbe2a streaming: handle_stream_event should be public 2015-07-21 16:12:54 +08:00
Asias He
d574e6e3c8 streaming: Rename getTotalFilesToSend to get_total_files_to_send 2015-07-21 16:12:54 +08:00
Asias He
f20b281e1c streaming: Update comments in stream_session::prepare 
To make things a bit clear. Note: In Origin, prepare() can be called by
both initiator and follower.
 2015-07-21 16:12:54 +08:00
Asias He
9b8d542b35 streaming: Improve handler of prepare_message from follower 
Upon receiving of the prepare_message message from the follower,
msg.requests can only be empty. Don't bother look at the requests.
 2015-07-21 16:12:54 +08:00
Asias He
d9b7cb4142 streaming: Enable logger in stream_result_future 2015-07-21 16:12:54 +08:00
Asias He
2e8b34dd9c streaming: Enable logger in stream_coordinator 
Move connect_all_stream_sessions to source first.
 2015-07-21 16:12:54 +08:00
Asias He
2dac3306a0 streaming: Enable more loggers in stream_session 2015-07-21 16:12:54 +08:00
Asias He
8561315cf2 streaming: de-thread_local-ize logger 2015-07-21 16:12:54 +08:00
Asias He
521df14240 streaming: Rename sstable_details to stream_details 2015-07-21 16:12:54 +08:00
Asias He
b53b8dfec8 streaming: Kill stream_reader 
It is used by IncomingFileMessage's deserialize to read from network and
write int sstable. In urchin, we use messaging service, incoming
mutation is handler within the STREAM_MUTATION handler. No need for
stream_reader.
 2015-07-21 16:12:54 +08:00
Asias He
8acf335f15 streaming: Kill stream_writer 
In Origin, it is used by OutgoingFileMessage's serialize function to
write given section of the SSTable to network. In urchin, we send
mutaion directly in stream_transfer_task::start(). We can kill
stream_write now.
 2015-07-21 16:12:54 +08:00
Asias He
7e365cb440 streaming: Handle cf_id does not exist case in STREAM_MUTATION handler 2015-07-21 16:12:54 +08:00
Asias He
69ce554a0f streaming: Init session when follower receives a PREPARE_MESSAGE 2015-07-21 16:12:54 +08:00
Asias He
9794fa1f97 streaming: Improve the test 
Instead of streaming system.local table, create and stream user created
table.
 2015-07-21 16:12:54 +08:00
Asias He
fa2aee57ac utils: Move util/serialization.hh to utils/serialization.hh 
Now we will not have the ugly utils and util directories, only utils.
 2015-07-21 16:12:54 +08:00
Asias He
292ddd68c2 streaming: Delay adding wrappers for some stream verbs 
It might be changed or be dropped since the upcoming change of streaming
code. It is not used at the moment.
 2015-07-16 17:22:53 +08:00
Asias He
857fa5ccbb messaging_service: Add wrapper for STREAM_MUTATION verb 2015-07-16 17:19:51 +08:00
Asias He
1f6a76f2a0 messaging_service: Add wrapper for PREPARE_MESSAGE verb 2015-07-16 17:19:51 +08:00
Glauber Costa
9c464aff9b database: clean up various APIs 
In much of our column_families APIs, we need to pass a pointer to the database.
The only reason we do that, is so we can properly handle the commit log entries
after we seal the current memtables into sstables.

Now that we store a pointer to the commit log in the CF itself at the time it
is created, we no longer have to do it. As a result, the APIs are a lot
cleaner, with no gratuitous parameters.

My motivation for this was the flush method, but as a result, apply() also gets
cleaner.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
 2015-07-15 10:24:20 -04:00
Asias He
6191fcb62e messaging_service: Add wrapper for STREAM_INIT_MESSAGE verb 
This patch serves as an example of how we can add wrappers for
ms.send_message and ms.register_handler.

When we convert all the users of them, we can make messaging_service.hh
do not include rpc.hh.
 2015-07-15 14:17:04 +03:00
Asias He
0313be256c streaming: Handle non-exist cf_name in get_column_family_stores 2015-07-14 20:56:37 +08:00
Asias He
d720dadf7b streaming: Switch to use logger class 2015-07-14 20:56:28 +08:00
Asias He
e82bdf2995 streaming: Swith to use shared_ptr from std::shared_ptr 
Since our shared_ptr works with incomplete types now, switch to it.
 2015-07-14 20:41:14 +08:00
Asias He
8fd8f39d63 streaming: Add more debug info for message exchange 2015-07-14 20:41:14 +08:00
Asias He
ca7f5ca5c9 streaming: Set proper dst_cpu_id in shard_id for PREPARE_MESSAGE and STREAM_MUTATION 2015-07-14 20:41:14 +08:00
Asias He
845de56614 streaming: Fix handler of PREPARE_MESSAGE 
With this patch, the receiver of PREPARE_MESSAGE can create a session
correctly now.
 2015-07-14 20:41:14 +08:00
Asias He
0282d4d6d0 streaming: Start stream_manager on startup 2015-07-14 20:41:14 +08:00
Asias He
8734213890 streaming: Lookup session 2015-07-14 20:41:14 +08:00
Asias He
fd04337e80 streaming: Wire up register_receiving init_receiving_side 
Now stream_result_future can create a stream_coordinator if not
provided.

So
 - On sending side, stream_coordinator is created by stream_plan
 - On receiving side, stream_coordinator is created by stream_result_future
 2015-07-14 20:41:14 +08:00
Asias He
d1720ffed1 streaming: Hold a shared_ptr inside stream_plan 2015-07-14 20:41:14 +08:00
Asias He
1828019d60 streaming: Convert StreamManager.java to C++ 2015-07-14 20:41:14 +08:00
Asias He
692c5b4c40 streaming: Import StreamManager.java 2015-07-14 20:41:14 +08:00
Asias He
f1cab93212 streaming: Return empty prepare_message for now 
We can not find a session on the receiver side currently. Return a empty
prepare_message for now.
 2015-07-14 20:41:14 +08:00
Asias He
38ee079916 streaming: Add test helper function 
This is a very preliminary test to make sure negotiating between two
nodes is ok.
 2015-07-14 20:41:14 +08:00
Asias He
01aa42ddca streaming: Add streaming_debug 
Add debug print for message exchange
 2015-07-14 20:41:14 +08:00
Asias He
285d1907ef streaming: Fix Assertion _instances.empty() failed 
seastar: ./core/sharded.hh:227: seastar::sharded<Service>::~sharded()
[with Service = streaming::stream_session::handler]: Assertion
`_instances.empty()' failed.
 2015-07-14 20:41:14 +08:00