Commit Graph

32 Commits

Author SHA1 Message Date
Calle Wilund
00de63c920 cql3::query_processor: Add processing helpers for internal usage
syntactical sugar + "process" for internal, similar to 
execute_internal, but allowing querying the whole cluster, and optional
statement caching.
2016-01-13 15:35:21 +00:00
Tomasz Grabiec
036eec295f query_processor: Invalidate statements synchronously
We want the statements to be removed before we ack the schema change,
otherwise it will race with all future operations.

Since the subscriber will be invoked on each shard, there is no need
to broadcast to all shards, we can just handle current shard.
2016-01-11 10:34:55 +01:00
Tomasz Grabiec
8deb3f18d3 query_processor: Invalidate prepared statements when columns change
Replicates https://issues.apache.org/jira/browse/CASSANDRA-7910 :

"Prepare a statement with a wildcard in the select clause.
2. Alter the table - add a column
3. execute the prepared statement
Expected result - get all the columns including the new column
Actual result - get the columns except the new column"
2016-01-11 10:34:55 +01:00
Tomasz Grabiec
0768deba74 query_processor: Add trace-level logging of processed statements 2016-01-08 21:10:25 +01:00
Pekka Enberg
e56bf8933f Improve not implemented errors
Print out the function name where we're throwing the exception from to
make it easier to debug such exceptions.
2015-12-18 10:51:37 +01:00
Tomasz Grabiec
e8d49a106c query_processor: Add trace-level logging of queries 2015-12-16 18:06:54 +01:00
Tomasz Grabiec
598f4104fd query_processor: Use the correct client_state
Since 4641dfff24, query_state keeps a
copy of client_state, not a reference. Therefore _cl is no longer
updated by queries using _qp. Fix by using the client_state from _qp.

Fixes #525.
2015-11-04 12:43:26 +02:00
Avi Kivity
2c3591cbd9 data_value de-any-fication
We use boost::any to convert to and from database values (stored in
serlialized form) and native C++ values.  boost::any captures information
about the data type (how to copy/move/delete etc.) and stores it inside
the boost::any instance.  We later retrieve the real value using
boost::any_cast.

However, data_value (which has a boost::any member) already has type
information as a data_type instance.  By teaching data_type intances about
the corresponding native type, we can elimiante the use of boost::any.

While boost::any is evil and eliminating it improves efficiency somewhat,
the real goal is growing native type support in data_type.  We will use that
later to store native types in the cache, enabling O(log n) access to
collections, O(1) access to tuples, and more efficient large blob support.
2015-10-30 17:38:51 +01:00
Pekka Enberg
e77fbfe709 cql3/query_processor: Wire up migration subscriber
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-15 09:18:53 +03:00
Pekka Enberg
f659e7bc10 cql3: Convert MigrationSubscriber to C++
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-15 09:18:52 +03:00
Pekka Enberg
3a30486d30 cql3/query_processor: Implement invalidate_prepared_statement()
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-15 09:18:52 +03:00
Pekka Enberg
c80deeb6fe cql3/query_processor: Add get_query_processor() helper
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-10-15 09:18:52 +03:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Calle Wilund
dc7a8faa0f query_processor: Add process_batch
More or less identical to Origins version.
2015-09-15 11:20:14 +02:00
Paweł Dziepak
c0547160f4 cql3: make ref to client_state const where possible
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-10 17:24:20 +03:00
Pekka Enberg
5b9901d693 cql3/query_options: Encapsulate underlying values
Encapsulate the '_values' vector to make it easier to switch the
underlying type from bytes_opt to bytes_view_opt.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-17 09:48:58 +03:00
Paweł Dziepak
5f2fd9b4dd cql3: cassandra_exceptions shouldn't be logged as errors
Only unexpected server errors should be logged, exception classes
deriving from cassandra_exception do not count as those.

Fixes #60.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-08-03 11:09:50 +03:00
Avi Kivity
99a15de9e5 logger: de-thread_local-ize logger
The logger class constructor registers itself with the logger registry,
in order to enable dynamically setting log levels.  However, since
thread_local variables may be (and are) initialized at the time of first
use, when the program starts up no loggers are registered.

Fix by making loggers global, not thread_local.  This requires that the
registry use locking to prevent registration happening on different threads
from corrupting the registry.

Note that technically global variables can also be initialized at the
point of first use, and there is no portable way for classes to self-register.
However this is the best we can do.
2015-07-14 17:18:11 +03:00
Glauber Costa
cdeab4a3d3 cql3: provide a mechanism to get a timestamp higher than any
It is common for some operations, like system table updates, to try and guarantee
some particular ordering of operations.

The way Origin does it, is by simply incrementing one to the current timestamp.
Our calls, however, are being dispatched through our internal query processor, which
has a builtin client_state.

Our client_state has a mechanism to guarantee monotonicity, by adding 1 if needed
to operations that happen subsequentially. By using a clock that is not wired up
to this mechanism, we can't really guarantee that if other operations happened to
get in between.

If we expose this mechanism through the query_processor, we will be able to guarantee
that.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Calle Wilund
84b5ba5270 query_processor bug fix: execute_internal did not keep options alive
... across the whole call chain. Should fix test failure.
2015-07-08 11:03:57 +03:00
Glauber Costa
586ccd43b0 query processor: relay internal queries
If the client state is marked as internal, go for the internal version
of the queries instead.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:22 -04:00
Calle Wilund
3253474eda query_processor : implement execute_internal
For internal CQL queries dispatched almost like external
2015-07-06 08:21:16 +02:00
Paweł Dziepak
67551832c0 cql3: implement displayRecogintionError for lexer and parser
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-06-23 16:17:45 +02:00
Calle Wilund
1631ce132e Add "storage_proxy&" argument to cql_statement::validate
To make db, schemas etc reachable
2015-06-03 10:13:52 +02:00
Avi Kivity
30b40bf7b1 db: make bytes even more distinct from sstring
bytes and sstring are distinct types, since their internal buffers are of
different length, but bytes_view is an alias of sstring_view, which makes
it possible of objects of different types to leak across the abstraction
boundary.

Fix this by making bytes a basic_sstring<int8_t, ...> instead of using char.
int8_t is a 'signed char', which is a distinct type from char, so now
bytes_view is a distinct type from sstring_view.

uint8_t would have been an even better choice, but that diverges from Origin
and would have required an audit.
2015-04-07 10:56:19 +03:00
Pekka Enberg
9d17a4e6da transport: Use constant for CQL version
There's a CQL version constant in query processor. Use it when
advertising CQL version to clients.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-20 11:21:18 +02:00
Pekka Enberg
d04c41d8a8 transport: CQL prepared statements
This patch adds initial support for PREPARE and EXECUTE requests which
are used by the CQL binary protocol for prepared statements. The use of
prepared statement gives a nice 2.5x single core performance boost for
Urchin:

  $ ./build/release/seastar --data data --smp 1

  $ ./tools/bin/cassandra-stress write -mode cql3 simplenative -rate threads=32

  Results:
  op rate                   : 31728
  partition rate            : 31728
  row rate                  : 31728
  latency mean              : 1.0
  latency median            : 0.9
  latency 95th percentile   : 1.8
  latency 99th percentile   : 1.8
  latency 99.9th percentile : 5.6
  latency max               : 181.7
  Total operation time      : 00:00:30
  END

  $ ./tools/bin/cassandra-stress write -mode cql3 simplenative prepared -rate threads=32

  Results:
  op rate                   : 75033
  partition rate            : 75033
  row rate                  : 75033
  latency mean              : 0.4
  latency median            : 0.4
  latency 95th percentile   : 0.7
  latency 99th percentile   : 0.8
  latency 99.9th percentile : 3.4
  latency max               : 205.0
  Total operation time      : 00:00:30
  END

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-19 15:54:46 +02:00
Pekka Enberg
b40661c330 cql3: Use shared_ptr for prepared statements
Query processor needs to store prepared statements as part of a client
session for PREPARE and EXECUTE requests. Switch from unique_ptr to
shared_ptr in preparation for that.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-18 10:49:41 +02:00
Tomasz Grabiec
609e893055 unimplemented: Separate subject from behavior
You can now do:

  fail(unimplemented::cause::PAGING);

and:

  warn(unimplemented::cause::PAGING);
2015-02-27 10:48:56 +01:00
Tomasz Grabiec
ec262ab3e9 cql3: Fix use-after-free on statement
Reported by Pekka:

READ of size 8 at 0x611000062ee8 thread T6
    #0 0x64d20f in std::unique_ptr<cql3::attributes, std::default_delete<cql3::attributes> >::get() const /usr/include/c++/4.9.2/bits/unique_ptr.h:305
    #1 0x641532 in std::unique_ptr<cql3::attributes, std::default_delete<cql3::attributes> >::operator->() const /usr/include/c++/4.9.2/bits/unique_ptr.h:299
    #2 0x63af5b in cql3::statements::modification_statement::get_time_to_live(cql3::query_options const&) const cql3/statements/modification_statement.hh:190
    #3 0x61e072 in _ZZN4cql310statements22modification_statement22make_update_parametersE13lw_shared_ptrISt6vectorI13basic_sstringIcjLj31EESaIS5_EEES2_IS3_INSt12experimental8optionalIS5_EESaISB_EEERKNS_13query_optionsEblENKUlT_E_clINSA_ISt13unordered_mapIS5_St3mapIjN5boost3anyESt4lessIjESaISt4pairIKjSO_EEE15serialized_hash16serialized_equalSaISR_IKS5_SV_EEEEEEEDaSI_ (/home/penberg/urchin/build/debug/seastar+0x61e072)
    #4 0x625ce1 in apply core/apply.hh:34
    #5 0x625df4 in apply<cql3::statements::modification_statement::make_update_parameters(lw_shared_ptr<std::vector<basic_sstring<char, unsigned int, 31u> > >, lw_shared_ptr<std::vector<std::experimental::optional<basic_sstring<char, unsigned int, 31u> > > >, const cql3::query_options&, bool, int64_t)::<lambda(auto:18)>, std::experimental::optional<std::unordered_map<basic_sstring<char, unsigned int, 31u>, std::map<unsigned int, boost::any, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, boost::any> > >, serialized_hash, serialized_equal, std::allocator<std::pair<const basic_sstring<char, unsigned int, 31u>, std::map<unsigned int, boost::any, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, boost::any> > > > > > > > core/apply.hh:41
    #6 0x628354 in _ZZN6futureIINSt12experimental8optionalISt13unordered_mapI13basic_sstringIcjLj31EESt3mapIjN5boost3anyESt4lessIjESaISt4pairIKjS7_EEE15serialized_hash16serialized_equalSaISA_IKS4_SE_EEEEEEE4thenIZN4cql310statements22modification_statement22make_update_parametersE13lw_shared_ptrISt6vectorIS4_SaIS4_EEESR_ISS_INS1_IS4_EESaISW_EEERKNSO_13query_optionsEblEUlT_E_EENSt9result_ofIFT_OSL_EE4typeEOS16_NSt9enable_ifIXsr9is_futureIIS1A_EE5valueEPvE4typeEENUlRS13_E_clI12future_stateIISL_EEEEDaRS16_ (/home/penberg/urchin/build/debug/seastar+0x628354)
    #7 0x628503 in _ZZN6futureIINSt12experimental8optionalISt13unordered_mapI13basic_sstringIcjLj31EESt3mapIjN5boost3anyESt4lessIjESaISt4pairIKjS7_EEE15serialized_hash16serialized_equalSaISA_IKS4_SE_EEEEEEE8scheduleIZNSM_4thenIZN4cql310statements22modification_statement22make_update_parametersE13lw_shared_ptrISt6vectorIS4_SaIS4_EEESS_IST_INS1_IS4_EESaISX_EEERKNSP_13query_optionsEblEUlT_E_EENSt9result_ofIFT_OSL_EE4typeEOS17_NSt9enable_ifIXsr9is_futureIIS1B_EE5valueEPvE4typeEEUlRS14_E_EEvS1C_EN21task_with_ready_state3runEv (/home/penberg/urchin/build/debug/seastar+0x628503)
    #8 0x7a6600 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&, unsigned long) core/reactor.cc:691
    #9 0x7a8ccc in reactor::run() core/reactor.cc:785
    #10 0x7b0bef in smp::configure(boost::program_options::variables_map)::{lambda()#1}::operator()() const (/home/penberg/urchin/build/debug/seastar+0x7b0bef)
    #11 0x84011d in _M_invoke /usr/include/c++/4.9.2/functional:2039
    #12 0x860c97 in std::function<void ()>::operator()() const /usr/include/c++/4.9.2/functional:2439
    #13 0x8e45fa in posix_thread::start_routine(void*) core/posix.cc:50
    #14 0x7f25abb05529 in start_thread (/lib64/libpthread.so.0+0x7529)
    #15 0x7f25ab84179c in __clone (/lib64/libc.so.6+0x10079c)

0x611000062ee8 is located 40 bytes inside of 216-byte region [0x611000062ec0,0x611000062f98)
freed by thread T6 here:
    #0 0x7f25af4bf64f in operator delete(void*) (/lib64/libasan.so.1+0x5864f)
    #1 0x684dbf in shared_ptr_count_for<cql3::statements::update_statement>::~shared_ptr_count_for() core/shared_ptr.hh:261
    #2 0x63de36 in shared_ptr<cql3::cql_statement>::~shared_ptr() core/shared_ptr.hh:326
    #3 0x772711 in cql3::statements::parsed_statement::prepared::~prepared() cql3/statements/parsed_statement.hh:60
    #4 0x77591d in std::default_delete<cql3::statements::parsed_statement::prepared>::operator()(cql3::statements::parsed_statement::prepared*) const /usr/include/c++/4.9.2/bits/unique_ptr.h:76
    #5 0x772cef in std::unique_ptr<cql3::statements::parsed_statement::prepared, std::default_delete<cql3::statements::parsed_statement::prepared> >::~unique_ptr() /usr/include/c++/4.9.2/bits/unique_ptr.h:236
    #6 0x76c8ea in cql3::query_processor::process(std::experimental::basic_string_view<char, std::char_traits<char> > const&, service::query_state&, cql3::query_options&) cql3/query_processor.cc:51
    #7 0x554e9c in cql_server::connection::process_query(unsigned short, temporary_buffer<char>) transport/server.cc:408
    #8 0x553a7a in cql_server::connection::process_request()::{lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)#1}::operator()(future<std::experimental::optional<cql_binary_frame_v3> >&&) const::{lambda(temporary_buffer<char>)#1}::operator()(temporary_buffer) const (/home/penberg/urchin/build/debug/seastar+0x553a7a)
    #9 0x55df9e in apply core/apply.hh:34
    #10 0x55e0b1 in apply<cql_server::connection::process_request()::<lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)>::<lambda(temporary_buffer<char>)>, temporary_buffer<char> > core/apply.hh:41
    #11 0x55e43b in then<cql_server::connection::process_request()::<lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)>::<lambda(temporary_buffer<char>)> > core/future.hh:480
    #12 0x5542c1 in cql_server::connection::process_request()::{lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)#1}::operator()(future<std::experimental::optional<cql_binary_frame_v3> >&&) const (/home/penberg/urchin/build/debug/seastar+0x5542c1)
    #13 0x564846 in _ZZNO6futureIINSt12experimental8optionalI19cql_binary_frame_v3EEEE12then_wrappedIZN10cql_server10connection15process_requestEvEUlOS4_E_EENSt9result_ofIFT_S4_EE4typeEOSB_ENUlRT_E_clI12future_stateIIS3_EEEEDaRSB_ (/home/penberg/urchin/build/debug/seastar+0x564846)
    #14 0x5649c7 in _ZZN7promiseIINSt12experimental8optionalI19cql_binary_frame_v3EEEE8scheduleIZNO6futureIIS3_EE12then_wrappedIZN10cql_server10connection15process_requestEvEUlOS7_E_EENSt9result_ofIFT_S7_EE4typeEOSE_EUlRT_E_EEvSI_EN15task_with_state3runEv (/home/penberg/urchin/build/debug/seastar+0x5649c7)
    #15 0x7a6600 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&, unsigned long) core/reactor.cc:691
    #16 0x7a8ccc in reactor::run() core/reactor.cc:785
    #17 0x7b0bef in smp::configure(boost::program_options::variables_map)::{lambda()#1}::operator()() const (/home/penberg/urchin/build/debug/seastar+0x7b0bef)
    #18 0x84011d in _M_invoke /usr/include/c++/4.9.2/functional:2039
    #19 0x860c97 in std::function<void ()>::operator()() const /usr/include/c++/4.9.2/functional:2439
    #20 0x8e45fa in posix_thread::start_routine(void*) core/posix.cc:50
    #21 0x7f25abb05529 in start_thread (/lib64/libpthread.so.0+0x7529)

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
2015-02-25 12:48:54 +01:00
Avi Kivity
70381a6da5 db: distribute database object
s/database/distributed<database>/ everywhere.

Use simple distribution rules: writes are broadcast, reads are local.
This causes tremendous data duplication, but will change soon.
2015-02-19 17:53:13 +02:00
Tomasz Grabiec
5e742cc13d cql3: Convert QueryProcessor 2015-02-12 19:40:58 +01:00