Commit Graph

189 Commits

Author SHA1 Message Date
Vlad Zolotarov
1489b28ffd cql_server::connection::process_prepare(): don't std::move() a shared_ptr captured by reference in value_of() lambda
A seastar::value_of() lambda used in a trace point was doing the unthinkable:
it called std::move() on a value captured by reference. Not only it compiled(!!!)
but it also actually std::move()ed the shared_ptr before it was used in a make_result()
which naturally caused a SIGSEG crash.

Fixes #1491

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1469193763-27631-1-git-send-email-vladz@cloudius-systems.com>
(cherry picked from commit 9423c13419)
2016-07-22 16:33:17 +03:00
Vlad Zolotarov
b0a39f210d transport: CQL tracing: QUERY instrumentation: align the session creation parameters with origin
- Don't put the query name as a 'request' but rather save it as one of entries in a
     'params' map.
   - Save some additional query parameters in 'params'.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
962bddf8fe transport: CQL tracing: instrument a BATCH command
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
d21eaabcfe transport: CQL tracing: instrument EXECUTE command
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
147dd72517 transport: CQL tracing: instrument a PREPARE command
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
be88074f47 service::query_state: get rid of begin_tracing()
Use tracing::begin() directly.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
952dc8a3d4 query_state: add get_trace_state() method
Adding this method allows to use tracing helper functions
and remove the no longer needed accessors in the query_state.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
0689843e79 tracing::trace_state: add method to set the session's "params" map entries
Sometimes we want to be able to set "params" map after we
started a tracing session, e.g. when the parameters values,
like a consistency level parsed from the "options" part of a binary frame,
are available only after some heavy part of a flow we would like
to trace.

This patch includes the following changes:

   - No longer pass a map to the begin().
   - Limit the parameters to the known set.
   - Define a method to set each such parameter and save its
     value till the final sstring->sstring map is created.
   - Construct the final sstring->sstring map in the destructor of the trace_state
     object in order to defer all the formatting to be after the traced flow.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:58 +03:00
Vlad Zolotarov
a5022a09a4 tracing: use 'write' instead of 'flush' and 'store' for consistency with seastar's API
In names of functions and variables:
s/flush_/write_/
s/store_/write_/

In a i_tracing_backend_helper:
s/flush()/kick()/

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-07-19 18:21:57 +03:00
Avi Kivity
f126efd7f2 transport: encode user-defined type metadata
Right now we fall back to tuples, which confuses the client.

Fixes #1443.

Reviewed-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <1468167120-1945-1-git-send-email-avi@scylladb.com>
2016-07-11 08:51:17 +03:00
Duarte Nunes
c8afb4cc46 query_processor: Support thrift prepared statements
This patch adds support for thrift prepared statements. It specializes
the result_message::prepared into two types:
result_message::prepared::cql and result_message::prepared::thrift, as
their identifiers have different types.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-06-27 15:39:02 +02:00
Vlad Zolotarov
905190ac06 tracing: add support for probabilistic tracing
Add a support for defining a probability (a value in a [0,1] range)
for tracing the next CQL request.

Traces for requests that are chosen to be traced due to this feature
are not going to flushed immediately.

Use std::subtract_with_carry_engine (implements the "lagged Fibonacci" algorithm)
random number engine for fastest generation of random integer values.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-06-06 15:41:01 +03:00
Vlad Zolotarov
099ff0d2d5 transport: instrument a QUERY with tracing
- Store a trace state inside a client_state.
   - Start tracing in a cql_server::connection::process_query().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-06-01 20:14:29 +03:00
Vlad Zolotarov
f994e0a8d0 transport/server: add support for sending a tracing session ID in a CQL response
- Add a tracing ID (UUID) optional field to cql_server::response.
   - If _tracing_id is set make_frame() would insert a tracing ID
     in the response message. According to CQL spec it should be the
     first thing in the response "body" and the TRACING bit (0x02) should be
     set in the "flags" field.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-06-01 20:13:53 +03:00
Vlad Zolotarov
9e61a3498d cql_server::response: rework make_frame()
Use a template function to avoid code duplication.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-06-01 20:13:53 +03:00
Vlad Zolotarov
8bf34fca02 service::client_state: store a client address
When client_state is created with an external_tag - store
a client address in the client state.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2016-06-01 20:13:53 +03:00
Avi Kivity
0135b4d5cd cql3: constify metadata users
Metadata usually doesn't change after it is created; make that visible in
the code, allowing further optimizations to be applied later.
Message-Id: <1464334638-7971-3-git-send-email-avi@scylladb.com>
2016-05-31 09:12:11 +03:00
Avi Kivity
6728454591 cql3: rationalize extract_result_metadata()
Rather than dynamic_cast<>ing the statement to see whether it is a
select statement, add a virtual function to cql_statement to get the
result metadata.

This is faster and easier to follow.
Message-Id: <1464334638-7971-2-git-send-email-avi@scylladb.com>
2016-05-31 09:12:02 +03:00
Avi Kivity
caf8d4f0e6 cql3: separate parsed_statement and parsed_statment::prepared
cql3::statements::parsed_statement
    -> cql3::statements::raw::parsed_statement
  cql3::statements::parsed_statement::prepared
    -> cql3::statements::prepared_statement
Message-Id: <1464609556-3756-2-git-send-email-avi@scylladb.com>
2016-05-31 09:09:10 +03:00
Pekka Enberg
d7d8c76fe5 transport/server: Add CQL frame LZ4 compression support
The default CQL frame compression algorithm in Cassandra is LZ4. Add
support for decompressing incoming frames and compressing outgoing
frames with LZ4 if the CQL driver asks for that.

Fixes #416

Message-Id: <1464086807-11325-1-git-send-email-penberg@scylladb.com>
2016-05-24 15:03:33 +03:00
Pekka Enberg
9c450f673c cql3: Clean up prepared_metadata class
Return vectors by const reference in prepared_metadata class and add a
FIXME to result_message class.

Message-Id: <1463425756-20225-1-git-send-email-penberg@scylladb.com>
2016-05-17 10:02:14 +03:00
Pekka Enberg
868ff5107c cql3: Introduce prepared_metadata class
Introduce a new prepared_metadata class that holds prepared statement
metadata and implement CQL binary protocol serialization that works for
all versions.
2016-05-16 18:06:01 +03:00
Calle Wilund
917bf850fa transport::server: Do not treat accept exception as fatal
1.) It most likely is not, i.e. either tcp or more likely, ssl
    negotiation failure. In any case, we can still try next
    connection.
2.) Not retrying will cause us to "leak" the accept, and then hang
    on shutdown.

Also, promote logging message on accept exception to "warn", since
dtest(s?) depend on seeing log output.

Message-Id: <1462283265-27051-4-git-send-email-calle@scylladb.com>
2016-05-09 14:13:07 +03:00
Calle Wilund
437ebe7128 cql_server: Use credentials_builder to init tls
Slightly cleaner, and shard-safe tls init.

Message-Id: <1462283265-27051-3-git-send-email-calle@scylladb.com>
2016-05-09 14:12:59 +03:00
Paweł Dziepak
b00a3a76cc transport: ignore errors during connection shutdown
If the other end of the connection has already disconnected the shutdown
will fail with ENOTCONN. The resulting exception is going to propagate
through the continuation chain that is supposed to shut the cql server
down preventing it from properly waiting for all outstanding
continuations.

The solution is to just ignore any errors that shutdown() may return.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-04-11 23:54:47 +01:00
Paweł Dziepak
9cd3da496e transport: retry do_accept() in case of bad_alloc
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-04-11 23:52:13 +01:00
Calle Wilund
7ebac35779 client_state: break up setting login/validation
transport::server uses client_state in a move-temporary-around
fashion. Having a setter that does continuation-bound validation
makes this messier. Break them up to separate "this" placement
from the actual validation continuation logic
2016-04-11 09:10:41 +00:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Piotr Jastrzebski
613e7d8618 Add more info to wrong RPC address error
If listening on RPC address failed then report
IP address and port in the error message.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
Message-Id: <c4db3527df2ce6dccb3b619584ee3fcb1e70ffd1.1459512258.git.piotr@scylladb.com>
2016-04-03 12:57:19 +03:00
Gleb Natapov
c6157dd99e enable rpc_keepalive parameter
Fixes #1044

Message-Id: <20160315104609.GV6117@scylladb.com>
2016-03-15 12:51:12 +02:00
Pekka Enberg
dfcc48d82a transport: Add result metadata to PREPARED message
The gocql driver assumes that there's a result metadata section in the
PREPARED message. Technically, Scylla is not at fault here as the CQL
specification explicitly states in Section 4.2.5.4. ("Prepared") that the
section may be empty:

   - <result_metadata> is defined exactly as <metadata> but correspond to the
      metadata for the resultSet that execute this query will yield. Note that
      <result_metadata> may be empty (have the No_metadata flag and 0 columns, See
      section 4.2.5.2) and will be for any query that is not a Select. There is
      in fact never a guarantee that this will non-empty so client should protect
      themselves accordingly. The presence of this information is an

However, Cassandra always populates the section so lets do that as well.

Fixes #912.

Message-Id: <1456317082-31688-1-git-send-email-penberg@scylladb.com>
2016-02-24 14:43:24 +02:00
Pekka Enberg
a15cbf0968 transport: Remove read_unsigned_short() variant
As explained in commit 0ff0c55 ("transport: server: 'short' should be
unsigned"), "short" type is always unsigned in the CQL binary protocol.
Therefore, drop the read_unsigned_short() variant altogether and just
use read_short() everywhere.

Message-Id: <1456133171-1433-1-git-send-email-penberg@scylladb.com>
2016-02-22 11:39:33 +02:00
Tomasz Grabiec
eedc1548e7 transport: server: Fix typo
Spotted-by: Vitaly Davidovich <vitalyd@gmail.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@scylladb.com>
2016-02-17 15:55:10 +01:00
Tomasz Grabiec
6e7bac14b3 transport: server: Throw instead of abort on bounds check failures
Instead of crashing the server we will respond with a "Server Error"
to the requestor.

Fixes #809.
2016-02-17 13:12:11 +01:00
Tomasz Grabiec
0ff0c5555a transport: server: 'short' should be unsigned
According to CQL binary protocol v3 [1], "short" fields are unsigned:

   [short]        A 2 bytes unsigned integer

[1] https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=doc/native_protocol_v3.spec

C* code agrees as well.

Fixes #807.
2016-02-17 13:12:11 +01:00
Tomasz Grabiec
9375b7df7b transport: server: Size check should allow max value 2016-02-17 13:12:11 +01:00
Tomasz Grabiec
6709c0ac15 cql_serialization_format: Make it CQL protocol version aware
We want to serialize it as a single number, the CQL binary protocol
version to which it corresponds, so it needs to be aware of the
version number.
2016-02-15 17:05:55 +01:00
Tomasz Grabiec
9d11968ad8 Rename serialization_format to cql_serialization_format 2016-02-15 16:53:56 +01:00
Paweł Dziepak
c90ec731c8 transport: do not close gate at connection shutdown
connection::_pending_requests_gate is responsible for keeping connection
objects alive as long as there are outstanding requests and is closed
in connection::proccess() when needed. Closing it in connection::shutdown()
as well may cause the gate to be closed twice what is a bug.

Fixes #690.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1454596390-23239-1-git-send-email-pdziepak@scylladb.com>
2016-02-07 20:07:23 +02:00
Calle Wilund
a00ff015f4 transport::server: read cqlv2 batch options correctly
Fixes #563.
Refs #584

CQLv2 encodes batch query_options in v1 format, not v2+.
CQLv1 otoh has no batch support at all.
Make read_options use explicit version format if needed.

v2: Ensure we preserve cql protocol version in query_opts
Message-Id: <1454514510-21706-1-git-send-email-calle@scylladb.com>
2016-02-03 16:55:07 +01:00
Asias He
e10580f474 cql_server: Fix connection shutdown
_fd is of type connected_socket. shutdown_input() and shutdown_output()
return future<>. Do not ignore the future.

Message-Id: <786eee890541a18d3501ecd52415f2900c545157.1452835922.git.asias@scylladb.com>
2016-01-15 11:37:30 +02:00
Calle Wilund
1d811f1e8f transport::server: Add authentication support
If system autheticator object requires authentication, issue
a challenge to client, and process response.
2016-01-13 15:35:36 +00:00
Tomasz Grabiec
8deb3f18d3 query_processor: Invalidate prepared statements when columns change
Replicates https://issues.apache.org/jira/browse/CASSANDRA-7910 :

"Prepare a statement with a wildcard in the select clause.
2. Alter the table - add a column
3. execute the prepared statement
Expected result - get all the columns including the new column
Actual result - get the columns except the new column"
2016-01-11 10:34:55 +01:00
Avi Kivity
c559008915 transport: protect against excessive memory consumption
If requests are delayed downstream from the cql server, and the client is
able to generate unrelated requests without limit, then the transient memory
consumed by the requests will overflow the shard's capacity.

Fix by adding a semaphore to cap the amount of transient memory occupied by
requests.

Fixes #674.
2016-01-04 12:11:00 +01:00
Asias He
7c9f9f068f cql_server: Do not ignore future in stop
Now connection.shutdown() returns future, we can not ignore it.
Plus, add shutdown process info for debug.

Message-Id: <b2d100bf9c817d7a230c6cd720944ba4fae416e2.1451894645.git.asias@scylladb.com>
2016-01-04 10:17:44 +02:00
Calle Wilund
51d3990261 cql_server: Allow using SSL socket
Optional credentials argument determine if SSL or normal
server socket is created.

Note: This does not follow the pattern of "socket as argument", simply
because this is a distributed object, so only trivial or immutable
objects should be passed to it.
2015-12-28 10:13:48 +00:00
Paweł Dziepak
31672906d3 transport: wait for outstanding requests to end during shutdown
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-17 14:06:41 +01:00
Paweł Dziepak
28e6edf927 transport: ignore future when stopping the server
When the server is shutting down a flag _stopping is set and listeners
are aborted using abort_accept(), which causes accept() calls to return
failed futures. However, accept handler just checks that the flag
_stopping is set and returns which causes a failed future to be
destroyed and a warning is printed.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-17 14:06:40 +01:00
Pekka Enberg
a26ffefd53 transport/server: Remove CQL text type from encoding
The text data type is no longer present in CQL binary protocol v3 and
later. We don't need it for encoding earlier versions either because
it's an alias for varchar which is present in all CQL binary protocol
versions.

Fixes #526.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-11-27 09:13:56 +01:00
Asias He
ed9cd23a2d transport: Fix duplicate up/down messages sent to native clients
This patch plus pekka's previous commit 3c72ea9f96

   "gms: Fix gossiper::handle_major_state_change() restart logic"

fix CASSANDRA-7816.

Backported from:

   def4835 Add missing follow on fix for 7816 only applied to
           cassandra-2.1 branch in 763130bdbde2f4cec2e8973bcd5203caf51cc89f
   763130b Followup commit for 7816
   2199a87 Fix duplicate up/down messages sent to native clients

Tested by:
   pushed_notifications_test.py:TestPushedNotifications.restart_node_test
2015-11-27 15:31:56 +08:00