Commit Graph

28 Commits

Author SHA1 Message Date
Asias He
49229964d0 streaming: Add streaming plan name when session is failed
Before:
[shard 0] stream_session - [Stream #fc1b66e0-b75b-11e6-b295-000000000000]
Stream failed, peers={127.0.0.1, 127.0.0.2}

After:
[shard 0] stream_session - [Stream #fc1b66e0-b75b-11e6-b295-000000000000]
Stream failed for streaming plan repair-in-29, peers={127.0.0.1, 127.0.0.2}
2016-12-05 08:20:18 +08:00
Asias He
1c47e26913 streaming: Add streaming plan name when all sessions are completed
Before:
[shard 0] stream_session - [Stream #e050b710-b758-11e6-9321-000000000000]
All sessions completed, peers={127.0.0.2}

After:
[shard 0] stream_session - [Stream #e050b710-b758-11e6-9321-000000000000]
All sessions completed for streaming plan repair-in-32, peers={127.0.0.2}
2016-12-05 08:20:18 +08:00
Asias He
984f427cb5 streaming: Log streaming bandwidth
It looks like:

[Stream #f3907fd0-a557-11e6-a583-000000000000] Session with 127.0.0.1 is complete, state=COMPLETE
[Stream #f3907fd0-a557-11e6-a583-000000000000] Session with 127.0.0.2 is complete, state=COMPLETE
[Stream #f3907fd0-a557-11e6-a583-000000000000] Session with 127.0.0.3 is complete, state=COMPLETE
[Stream #f3907fd0-a557-11e6-a583-000000000000] bytes_sent = 393284364, bytes_received = 0, tx_bandwidth = 17.048 MiB/s, rx_bandwidth = 0.000 MiB/s
[Stream #f3907fd0-a557-11e6-a583-000000000000] All sessions completed, peers={127.0.0.1, 127.0.0.2, 127.0.0.3}

Fixes #1826
2016-12-05 08:20:18 +08:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Asias He
8060b97d67 streaming: Log number of bytes sent and recevied when stream_plan completes
It is useful for test code to verify number of bytes sent/received.

It looks like below in the log.

/tmp/out1:INFO  [shard 0] stream_session - \
[Stream #1f3e23f0-db9e-11e5-9cfb-000000000000] bytes_sent = 0, bytes_received = 15760704

/tmp/out2:INFO  [shard 0] stream_session - \
[Stream #1f3e23f0-db9e-11e5-9cfb-000000000000] bytes_sent = 0, bytes_received = 18203964

/tmp/out3:INFO  [shard 0] stream_session - \
[Stream #1f3e23f0-db9e-11e5-9cfb-000000000000] bytes_sent = 33964668, bytes_received = 0
2016-02-26 17:38:37 +08:00
Asias He
cb92fe75e6 streaming: Introduce get_session helper
To simplify streaming verb handler.

- Use get_session instead of open coded logic to get get_coordinator and
  stream_session in all the verb handlers

- Use throw instead of assert for error handling

- init_receiving_side now returns a shared_ptr<stream_result_future>
2016-01-29 16:31:07 +08:00
Asias He
aa69d5ffb2 streaming: Drop update_progress in stream_coordinator
Since we have session_info inside stream_session now, we can call
update_progress directly in stream_session.
2016-01-29 16:31:07 +08:00
Asias He
46bec5980b streaming: Put session_info inside stream_session
It is 1:1 mapping between session_info and stream_session. Putting
session_info inside stream_session, we can get rid of the
stream_coordinator::host_streaming_data class.
2016-01-29 16:31:07 +08:00
Asias He
048965ea02 streaming: Do not print session_index in handle_session_prepared
session_index is always 0. It will be removed soon.
2016-01-25 16:24:16 +08:00
Asias He
bc4ac5004e streaming: Kill stream_result_future::create_and_register
The helper is used only once in init_sending_side and in
init_receiving_side we do not use create_and_register to create
stream_result_future. Kill the trivial helper to make the code more
consistent.

In addition, rename variables "future" and "f" to sr (streaming_result).
2016-01-25 11:38:13 +08:00
Asias He
face74a8f2 streaming: Rename stream_result_future::init to ::init_sending_side
So we have:

- init_sending_side
  called when the node initiates a stream_session

- init_receiving_side
  called when the node is a receiver of a stream_session initiated by a peer
2016-01-25 11:38:13 +08:00
Asias He
9a346d56b9 streaming: Drop unnecessary parameters in stream_init_message
- from
  We can get it form the rpc::client_info

- session_index
  There will always be one session in stream_coordinator::host_streaming_data with a peer.

- is_for_outgoing
  In cassandra, it initiates two tcp connections, one for incoming stream and one for outgoing stream.
  logger.debug("[Stream #{}] Sending stream init for incoming stream", session.planId());
  logger.debug("[Stream #{}] Sending stream init for outgoing stream", session.planId());
  In scylla, it only initiates one "connection" for sending, the peer initiates another "connection" for receiving.
  So, is_for_outgoing will also be true in scylla, we can drop it.

- keep_ss_table_level
  In scylla, again, we stream mutations instead of sstable file. It is
  not relevant to us.
2016-01-25 11:38:13 +08:00
Asias He
bdd6a69af7 streaming: Drop unused parameters
- int connections_per_host

Scylla does not create connections per stream_session, instead it uses
rpc, thus connections_per_host is not relevant to scylla.

- bool keep_ss_table_level
- int repaired_at

Scylla does not stream sstable files. They are not relevant to scylla.
2016-01-25 11:38:13 +08:00
Asias He
88e99e89d6 streaming: Add more debug info
- Add debug for the peer address info
- Add debug in stream_transfer_task and stream_receive_task
- Add debug when cancel the keep_alive timer
- Add debug for has_active_sessions in stream_result_future::maybe_complete
2016-01-22 07:43:16 +08:00
Asias He
13934140f6 streaming: Remove bogus file size info for prepare completed
Scylla does not streaming sstable files directly for streaming. The file
size info is incorrect, let's get rid of it.
2015-11-10 15:39:34 +08:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Asias He
804564fe76 streaming: Remove completed stream in stream_manager 2015-07-31 16:27:55 +08:00
Asias He
1749305165 streaming: Fix stream_result_future::create_and_register
It is used to create stream initiated by us.
2015-07-31 16:27:55 +08:00
Asias He
59cae82470 streaming: Make stream_plan::execute return a future
Returns a ready future if the stream_plan completes successfully, or a
failed future otherwise.
2015-07-31 16:27:55 +08:00
Asias He
3a5af0a7fb streaming: Fix stream_coordinator::is_receiving
Unlike Origin, We can not use _connections_per_host == 0 to indicate we
are a follower since we set _connections_per_host to 1 for follower too.
2015-07-31 16:27:55 +08:00
Asias He
93f17024a8 streaming: Fix a logger printout
Before:

[Stream #136c5310-2f7a-11e5-87df-000000000000 ID#0]
[Stream #136c5310-2f7a-11e5-87df-000000000000, ID#0]

After:

[Stream #136c5310-2f7a-11e5-87df-000000000000 ID#0]
[Stream #136c5310-2f7a-11e5-87df-000000000000 ID#0]
2015-07-21 16:12:54 +08:00
Asias He
0e5fa35bd2 streaming: Register to stream_manager in create_and_register 2015-07-21 16:12:54 +08:00
Asias He
1f5feee5f2 streaming: Implement stream_result_future::handle_progress 2015-07-21 16:12:54 +08:00
Asias He
d74e414d09 streaming: Implement stream_result_future::handle_session_complete 2015-07-21 16:12:54 +08:00
Asias He
5e6f84cba8 streaming: Implement stream_result_future::handle_session_prepared
Instead of playing the game of casting between stream_event and derived
class. We overload handle_stream_event with derived stream_event class.

virtual void handle_stream_event(session_complete_event event) {}
virtual void handle_stream_event(progress_event event) {}
virtual void handle_stream_event(session_prepared_event event) {}

Also, make the virtual function non pure virtual, so user can override
the interested event only without defining all of the three.
2015-07-21 16:12:54 +08:00
Asias He
d9b7cb4142 streaming: Enable logger in stream_result_future 2015-07-21 16:12:54 +08:00
Asias He
fd04337e80 streaming: Wire up register_receiving init_receiving_side
Now stream_result_future can create a stream_coordinator if not
provided.

So
 - On sending side, stream_coordinator is created by stream_plan
 - On receiving side, stream_coordinator is created by stream_result_future
2015-07-14 20:41:14 +08:00
Asias He
1828019d60 streaming: Convert StreamManager.java to C++ 2015-07-14 20:41:14 +08:00