Commit Graph

15 Commits

Author SHA1 Message Date
Avi Kivity
99a15de9e5 logger: de-thread_local-ize logger
The logger class constructor registers itself with the logger registry,
in order to enable dynamically setting log levels.  However, since
thread_local variables may be (and are) initialized at the time of first
use, when the program starts up no loggers are registered.

Fix by making loggers global, not thread_local.  This requires that the
registry use locking to prevent registration happening on different threads
from corrupting the registry.

Note that technically global variables can also be initialized at the
point of first use, and there is no portable way for classes to self-register.
However this is the best we can do.
2015-07-14 17:18:11 +03:00
Glauber Costa
cdeab4a3d3 cql3: provide a mechanism to get a timestamp higher than any
It is common for some operations, like system table updates, to try and guarantee
some particular ordering of operations.

The way Origin does it, is by simply incrementing one to the current timestamp.
Our calls, however, are being dispatched through our internal query processor, which
has a builtin client_state.

Our client_state has a mechanism to guarantee monotonicity, by adding 1 if needed
to operations that happen subsequentially. By using a clock that is not wired up
to this mechanism, we can't really guarantee that if other operations happened to
get in between.

If we expose this mechanism through the query_processor, we will be able to guarantee
that.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-09 00:12:08 -04:00
Calle Wilund
84b5ba5270 query_processor bug fix: execute_internal did not keep options alive
... across the whole call chain. Should fix test failure.
2015-07-08 11:03:57 +03:00
Glauber Costa
586ccd43b0 query processor: relay internal queries
If the client state is marked as internal, go for the internal version
of the queries instead.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-07 11:38:22 -04:00
Calle Wilund
3253474eda query_processor : implement execute_internal
For internal CQL queries dispatched almost like external
2015-07-06 08:21:16 +02:00
Paweł Dziepak
67551832c0 cql3: implement displayRecogintionError for lexer and parser
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-06-23 16:17:45 +02:00
Calle Wilund
1631ce132e Add "storage_proxy&" argument to cql_statement::validate
To make db, schemas etc reachable
2015-06-03 10:13:52 +02:00
Avi Kivity
30b40bf7b1 db: make bytes even more distinct from sstring
bytes and sstring are distinct types, since their internal buffers are of
different length, but bytes_view is an alias of sstring_view, which makes
it possible of objects of different types to leak across the abstraction
boundary.

Fix this by making bytes a basic_sstring<int8_t, ...> instead of using char.
int8_t is a 'signed char', which is a distinct type from char, so now
bytes_view is a distinct type from sstring_view.

uint8_t would have been an even better choice, but that diverges from Origin
and would have required an audit.
2015-04-07 10:56:19 +03:00
Pekka Enberg
9d17a4e6da transport: Use constant for CQL version
There's a CQL version constant in query processor. Use it when
advertising CQL version to clients.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-20 11:21:18 +02:00
Pekka Enberg
d04c41d8a8 transport: CQL prepared statements
This patch adds initial support for PREPARE and EXECUTE requests which
are used by the CQL binary protocol for prepared statements. The use of
prepared statement gives a nice 2.5x single core performance boost for
Urchin:

  $ ./build/release/seastar --data data --smp 1

  $ ./tools/bin/cassandra-stress write -mode cql3 simplenative -rate threads=32

  Results:
  op rate                   : 31728
  partition rate            : 31728
  row rate                  : 31728
  latency mean              : 1.0
  latency median            : 0.9
  latency 95th percentile   : 1.8
  latency 99th percentile   : 1.8
  latency 99.9th percentile : 5.6
  latency max               : 181.7
  Total operation time      : 00:00:30
  END

  $ ./tools/bin/cassandra-stress write -mode cql3 simplenative prepared -rate threads=32

  Results:
  op rate                   : 75033
  partition rate            : 75033
  row rate                  : 75033
  latency mean              : 0.4
  latency median            : 0.4
  latency 95th percentile   : 0.7
  latency 99th percentile   : 0.8
  latency 99.9th percentile : 3.4
  latency max               : 205.0
  Total operation time      : 00:00:30
  END

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-19 15:54:46 +02:00
Pekka Enberg
b40661c330 cql3: Use shared_ptr for prepared statements
Query processor needs to store prepared statements as part of a client
session for PREPARE and EXECUTE requests. Switch from unique_ptr to
shared_ptr in preparation for that.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-03-18 10:49:41 +02:00
Tomasz Grabiec
609e893055 unimplemented: Separate subject from behavior
You can now do:

  fail(unimplemented::cause::PAGING);

and:

  warn(unimplemented::cause::PAGING);
2015-02-27 10:48:56 +01:00
Tomasz Grabiec
ec262ab3e9 cql3: Fix use-after-free on statement
Reported by Pekka:

READ of size 8 at 0x611000062ee8 thread T6
    #0 0x64d20f in std::unique_ptr<cql3::attributes, std::default_delete<cql3::attributes> >::get() const /usr/include/c++/4.9.2/bits/unique_ptr.h:305
    #1 0x641532 in std::unique_ptr<cql3::attributes, std::default_delete<cql3::attributes> >::operator->() const /usr/include/c++/4.9.2/bits/unique_ptr.h:299
    #2 0x63af5b in cql3::statements::modification_statement::get_time_to_live(cql3::query_options const&) const cql3/statements/modification_statement.hh:190
    #3 0x61e072 in _ZZN4cql310statements22modification_statement22make_update_parametersE13lw_shared_ptrISt6vectorI13basic_sstringIcjLj31EESaIS5_EEES2_IS3_INSt12experimental8optionalIS5_EESaISB_EEERKNS_13query_optionsEblENKUlT_E_clINSA_ISt13unordered_mapIS5_St3mapIjN5boost3anyESt4lessIjESaISt4pairIKjSO_EEE15serialized_hash16serialized_equalSaISR_IKS5_SV_EEEEEEEDaSI_ (/home/penberg/urchin/build/debug/seastar+0x61e072)
    #4 0x625ce1 in apply core/apply.hh:34
    #5 0x625df4 in apply<cql3::statements::modification_statement::make_update_parameters(lw_shared_ptr<std::vector<basic_sstring<char, unsigned int, 31u> > >, lw_shared_ptr<std::vector<std::experimental::optional<basic_sstring<char, unsigned int, 31u> > > >, const cql3::query_options&, bool, int64_t)::<lambda(auto:18)>, std::experimental::optional<std::unordered_map<basic_sstring<char, unsigned int, 31u>, std::map<unsigned int, boost::any, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, boost::any> > >, serialized_hash, serialized_equal, std::allocator<std::pair<const basic_sstring<char, unsigned int, 31u>, std::map<unsigned int, boost::any, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, boost::any> > > > > > > > core/apply.hh:41
    #6 0x628354 in _ZZN6futureIINSt12experimental8optionalISt13unordered_mapI13basic_sstringIcjLj31EESt3mapIjN5boost3anyESt4lessIjESaISt4pairIKjS7_EEE15serialized_hash16serialized_equalSaISA_IKS4_SE_EEEEEEE4thenIZN4cql310statements22modification_statement22make_update_parametersE13lw_shared_ptrISt6vectorIS4_SaIS4_EEESR_ISS_INS1_IS4_EESaISW_EEERKNSO_13query_optionsEblEUlT_E_EENSt9result_ofIFT_OSL_EE4typeEOS16_NSt9enable_ifIXsr9is_futureIIS1A_EE5valueEPvE4typeEENUlRS13_E_clI12future_stateIISL_EEEEDaRS16_ (/home/penberg/urchin/build/debug/seastar+0x628354)
    #7 0x628503 in _ZZN6futureIINSt12experimental8optionalISt13unordered_mapI13basic_sstringIcjLj31EESt3mapIjN5boost3anyESt4lessIjESaISt4pairIKjS7_EEE15serialized_hash16serialized_equalSaISA_IKS4_SE_EEEEEEE8scheduleIZNSM_4thenIZN4cql310statements22modification_statement22make_update_parametersE13lw_shared_ptrISt6vectorIS4_SaIS4_EEESS_IST_INS1_IS4_EESaISX_EEERKNSP_13query_optionsEblEUlT_E_EENSt9result_ofIFT_OSL_EE4typeEOS17_NSt9enable_ifIXsr9is_futureIIS1B_EE5valueEPvE4typeEEUlRS14_E_EEvS1C_EN21task_with_ready_state3runEv (/home/penberg/urchin/build/debug/seastar+0x628503)
    #8 0x7a6600 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&, unsigned long) core/reactor.cc:691
    #9 0x7a8ccc in reactor::run() core/reactor.cc:785
    #10 0x7b0bef in smp::configure(boost::program_options::variables_map)::{lambda()#1}::operator()() const (/home/penberg/urchin/build/debug/seastar+0x7b0bef)
    #11 0x84011d in _M_invoke /usr/include/c++/4.9.2/functional:2039
    #12 0x860c97 in std::function<void ()>::operator()() const /usr/include/c++/4.9.2/functional:2439
    #13 0x8e45fa in posix_thread::start_routine(void*) core/posix.cc:50
    #14 0x7f25abb05529 in start_thread (/lib64/libpthread.so.0+0x7529)
    #15 0x7f25ab84179c in __clone (/lib64/libc.so.6+0x10079c)

0x611000062ee8 is located 40 bytes inside of 216-byte region [0x611000062ec0,0x611000062f98)
freed by thread T6 here:
    #0 0x7f25af4bf64f in operator delete(void*) (/lib64/libasan.so.1+0x5864f)
    #1 0x684dbf in shared_ptr_count_for<cql3::statements::update_statement>::~shared_ptr_count_for() core/shared_ptr.hh:261
    #2 0x63de36 in shared_ptr<cql3::cql_statement>::~shared_ptr() core/shared_ptr.hh:326
    #3 0x772711 in cql3::statements::parsed_statement::prepared::~prepared() cql3/statements/parsed_statement.hh:60
    #4 0x77591d in std::default_delete<cql3::statements::parsed_statement::prepared>::operator()(cql3::statements::parsed_statement::prepared*) const /usr/include/c++/4.9.2/bits/unique_ptr.h:76
    #5 0x772cef in std::unique_ptr<cql3::statements::parsed_statement::prepared, std::default_delete<cql3::statements::parsed_statement::prepared> >::~unique_ptr() /usr/include/c++/4.9.2/bits/unique_ptr.h:236
    #6 0x76c8ea in cql3::query_processor::process(std::experimental::basic_string_view<char, std::char_traits<char> > const&, service::query_state&, cql3::query_options&) cql3/query_processor.cc:51
    #7 0x554e9c in cql_server::connection::process_query(unsigned short, temporary_buffer<char>) transport/server.cc:408
    #8 0x553a7a in cql_server::connection::process_request()::{lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)#1}::operator()(future<std::experimental::optional<cql_binary_frame_v3> >&&) const::{lambda(temporary_buffer<char>)#1}::operator()(temporary_buffer) const (/home/penberg/urchin/build/debug/seastar+0x553a7a)
    #9 0x55df9e in apply core/apply.hh:34
    #10 0x55e0b1 in apply<cql_server::connection::process_request()::<lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)>::<lambda(temporary_buffer<char>)>, temporary_buffer<char> > core/apply.hh:41
    #11 0x55e43b in then<cql_server::connection::process_request()::<lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)>::<lambda(temporary_buffer<char>)> > core/future.hh:480
    #12 0x5542c1 in cql_server::connection::process_request()::{lambda(future<std::experimental::optional<cql_binary_frame_v3> >&&)#1}::operator()(future<std::experimental::optional<cql_binary_frame_v3> >&&) const (/home/penberg/urchin/build/debug/seastar+0x5542c1)
    #13 0x564846 in _ZZNO6futureIINSt12experimental8optionalI19cql_binary_frame_v3EEEE12then_wrappedIZN10cql_server10connection15process_requestEvEUlOS4_E_EENSt9result_ofIFT_S4_EE4typeEOSB_ENUlRT_E_clI12future_stateIIS3_EEEEDaRSB_ (/home/penberg/urchin/build/debug/seastar+0x564846)
    #14 0x5649c7 in _ZZN7promiseIINSt12experimental8optionalI19cql_binary_frame_v3EEEE8scheduleIZNO6futureIIS3_EE12then_wrappedIZN10cql_server10connection15process_requestEvEUlOS7_E_EENSt9result_ofIFT_S7_EE4typeEOSE_EUlRT_E_EEvSI_EN15task_with_state3runEv (/home/penberg/urchin/build/debug/seastar+0x5649c7)
    #15 0x7a6600 in reactor::run_tasks(circular_buffer<std::unique_ptr<task, std::default_delete<task> >, std::allocator<std::unique_ptr<task, std::default_delete<task> > > >&, unsigned long) core/reactor.cc:691
    #16 0x7a8ccc in reactor::run() core/reactor.cc:785
    #17 0x7b0bef in smp::configure(boost::program_options::variables_map)::{lambda()#1}::operator()() const (/home/penberg/urchin/build/debug/seastar+0x7b0bef)
    #18 0x84011d in _M_invoke /usr/include/c++/4.9.2/functional:2039
    #19 0x860c97 in std::function<void ()>::operator()() const /usr/include/c++/4.9.2/functional:2439
    #20 0x8e45fa in posix_thread::start_routine(void*) core/posix.cc:50
    #21 0x7f25abb05529 in start_thread (/lib64/libpthread.so.0+0x7529)

Signed-off-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
2015-02-25 12:48:54 +01:00
Avi Kivity
70381a6da5 db: distribute database object
s/database/distributed<database>/ everywhere.

Use simple distribution rules: writes are broadcast, reads are local.
This causes tremendous data duplication, but will change soon.
2015-02-19 17:53:13 +02:00
Tomasz Grabiec
5e742cc13d cql3: Convert QueryProcessor 2015-02-12 19:40:58 +01:00