Commit Graph

53948 Commits

Author SHA1 Message Date
Pekka Enberg
cd35617855 database: Use keyspace_metadata for creation functions
Use the keyspace_metadata type for keyspace creation functions. This is
needed to be able to have a mapping from keyspace name to keyspace
metadata for various call-sites.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 15:27:47 +03:00
Pekka Enberg
f0ad71f9f1 thrift: Fix keyspace metadata init in system_add_keyspace()
Pass replication strategy options to the create_replication_strategy()
function.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 15:16:14 +03:00
Avi Kivity
db04bba208 db: futurize the single partition query path
Prepare for disk reads.
2015-05-19 15:13:09 +03:00
Avi Kivity
738be63b28 db: define column_family move constructor in .cc
Allows using it from files that do not include sstable.hh.
2015-05-19 15:13:09 +03:00
Avi Kivity
72d721ea56 db: futurize column_family::query() outer loop
In preparation for reading from sstables, allow the outer loop of
column_family::query() (iterating over partition ranges) do defer.
2015-05-19 15:13:06 +03:00
Avi Kivity
d6754823b8 db: conform to sstable naming convention wrt generation 2015-05-19 15:01:29 +03:00
Avi Kivity
05f7c6abd5 sstable: rename convert_row() to read_row()
More reader-friendly.
2015-05-19 15:01:29 +03:00
Amnon Heiman
b795acdf63 Adds the database to the API context
The API needs the database distribute object to get information from it.
This adds a database reference the API context so it would be
available in the API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-19 13:27:57 +03:00
Avi Kivity
104557b3a0 Merge "Keyspace metadata cleanup"
Pekka says:

"We are going to keep the ks_meta_data class around and use it in core
code like the migration manager. Therefore, clean up the class and move
it to the database.hh where user_types_metadata also is defined in. As a
bonus, this also fixes the circular dependency between ks_meta_data.hh
and database.hh."
2015-05-19 11:39:03 +03:00
Avi Kivity
06a9eb2f31 db: fix bad print() in seal_active_memtable()
Noticed by Raphael.
2015-05-19 11:29:52 +03:00
Pekka Enberg
8380df84b4 database: Rename ks_meta_data to keyspace_metadata
Follow the naming convention set by user_types_metadata and rename
ks_meta_data to keyspace_metadata.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:24:06 +03:00
Pekka Enberg
7a84b53d61 database: Use lw_shared_ptr for user types metadata
Use lw_shared_ptr for user types metadata member in ks_meta_data.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:17:55 +03:00
Pekka Enberg
f1fc575401 Clean up ks_meta_data construction
Simplify ks_meta_data construction in few places by using the default
arguments.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:16:11 +03:00
Pekka Enberg
a225439fdb database: Inline ks_meta_data implementation
The implementation part of ks_meta_data is just few lines of code.
Inline that to the database.hh header file.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:07:14 +03:00
Pekka Enberg
032af4d53b database: Move ks_meta_data definition to database.hh
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 11:03:28 +03:00
Pekka Enberg
1931165021 config: Simplify ks_meta_data::new_keyspace()
The constructor has a default argument for user types metadata so use
it.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 10:52:42 +03:00
Pekka Enberg
e80fe254d2 config: Encapsulate ks_meta_data member variables
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 10:51:39 +03:00
Pekka Enberg
a16b915a8e config: Unify ks_meta_data constructors
Use default arguments to unify the ks_meta_data constructors.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 10:33:56 +03:00
Pekka Enberg
491d5e648b config: Unify ks_meta_data::new_keyspace() functions
Use default argument to unify the two new_keyspace() function variants.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 10:28:23 +03:00
Pekka Enberg
b7ddd2c4b6 config: Remove ifdef'd code from ks_meta_data class
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-19 10:26:26 +03:00
Amnon Heiman
c552fdb8b3 json2code Enhence the enum_wrapper
The code generation generate an enum_wrapper for the enum that defined
in the swagger definition file.

This patch adds more functionality to the enum_wrapper:
It is now possible to cast the enum to other enum if wrapped enum is a
subset of the target enum, this is a reverse functionality of
constructing an enum from another enum.

The wrapper now support comparison operator, begin, end method and the
++ operator, with that the boost::irange is used to return a range
object so the the enum values can be iterate using for range loop.

This is an example of the added methods, After code generation:
    template<class T>
    operator T() const {
      switch(v) {
      case my_object_enum_var::VAL1: return T::VAL1;
      case my_object_enum_var::VAL2: return T::VAL2;
      case my_object_enum_var::VAL3: return T::VAL3;
      default: return T::VAL1;
      }
    }
    typedef typename std::underlying_type<my_object_enum_var>::type
pos_type;
    enum_var_wrapper& operator++() {
      v = static_cast<my_object_enum_var>(static_cast<pos_type>(v) + 1);
      return *this;
    }
    enum_var_wrapper& operator++(int) {
      return ++(*this);
    }
    bool operator==(const enum_var_wrapper& c) const {
      return v == c.v;
    }
    bool operator!=(const enum_var_wrapper& c) const {
      return v != c.v;
    }
    bool operator<=(const enum_var_wrapper& c) const {
      return static_cast<pos_type>(v) <= static_cast<pos_type>(c.v);
    }
    static enum_var_wrapper begin() {
      return enum_var_wrapper(my_object_enum_var::VAL1);
    }
    static enum_var_wrapper end() {
      return enum_var_wrapper(my_object_enum_var::NUM_ITEMS);
    }
    static boost::integer_range<enum_var_wrapper> all_items() {
      return boost::irange(begin(), end());
   }

This generated code code be found after compilation at:
./build/release/gen/apps/httpd/demo.json.hh

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-19 10:08:12 +03:00
Avi Kivity
886928962b db: fix query of multiple pkeys ("pkey IN (k1, k2, k3)") failing on missing key
Need to skip on missing partition, not stop the whole thing.
2015-05-18 17:55:28 +02:00
Avi Kivity
e31648c7fd Merge branch 'master' of github.com:cloudius-systems/urchin into db
Conflicts:
	thrift/handler.cc
2015-05-18 17:10:58 +03:00
Avi Kivity
01e1239608 thrift: rename CassandraAsyncHandler to match coding style 2015-05-18 16:31:44 +03:00
Avi Kivity
af15b24902 thrift: fix system_add_keyspace() indentation 2015-05-18 16:20:25 +03:00
Avi Kivity
9b43fe252a Merge branch 'penberg/user-types-cleanup' of github.com:cloudius-systems/seastar-dev into db
Pekka says:

"There's a user_types_metadata type defined in database.hh. Use that and
remove the left-over ut_meta_data from initial CQL translations."

Conflicts:
	thrift/handler.cc
2015-05-18 16:07:19 +03:00
Avi Kivity
51e19fc090 memtable.h: add copyright statement 2015-05-18 15:59:23 +03:00
Pekka Enberg
9ee7e21438 Switch to user_types_metadata
There's a user_types_metadata type in database.hh. Use it and drop the
left-over ut_meta_data from Origin.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-18 15:54:44 +03:00
Avi Kivity
07d7f410f3 Merge branch 'memtable' into db
Conflicts:
	database.hh
	memtable changes moved to memtable.hh
2015-05-18 15:50:24 +03:00
Pekka Enberg
c1aed8a712 config: Move ks_meta_data implementation to .cc file
Move ks_meta_data code out of line in preparation for switching to the
user_types_metadata type that is defined in database.hh.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-05-18 15:47:29 +03:00
Avi Kivity
875148dae6 db: create keyspace/column_family directory structure
This is slightly awkwards, since the directory structure is not sharded.
This requires some processing to occur outside the shard, while the rest
is sharded.
2015-05-18 15:34:41 +03:00
Avi Kivity
20775b9d5c db: store a column_family's memtables in a list instead of a vector
A vector can cause memtables to be move()d around, which breaks any
code that captures a memtable's this pointer.

Fix by using a linked list.
2015-05-18 15:34:25 +03:00
Avi Kivity
394e0d3a8c db: make database::add_keyspace() return void
Returning a reference to the keyspace is dangerous in that the keyspace can
be moved away, when we start futurizing the add_keyspace() process.  Make
it return void and look up the keyspace at the point of use.
2015-05-18 15:34:25 +03:00
Avi Kivity
d8fed7e211 db: add simple memtable sealing policy
Need to be replaced with something better, but we lack the infrastructure so
far (region memory allocator).
2015-05-18 15:34:25 +03:00
Avi Kivity
1cf95e4bd8 Merge branch 'thread' of github.com:cloudius-systems/seastar-dev
This patchset adds thread support to seastar.  Threads can use the normal
seastar API, and may also "block" by calling future<>::get() on an
unavailable future.  Synchronous and asynchronous code may be intermixed.

Threads may not issue blocking operating system calls.
2015-05-18 15:27:22 +03:00
Amnon Heiman
03c4a6d050 rpc: add counters to the rpc client
This adds a counters to net protocol client.
The client would holds counter for the successfully completed, the
exception recieved and as the sent queue is based on the future queue
a counter for pending to sent messages.

The number of messages that waiting for reply is retrieved from the size
of the outstanding table.

The counters are updated by the helper send function as part of the
messages sending flow.

The counters are part of the stat structure and the client has an
internal getter to be able to change their values and a getter that
returns a copy of the counters.
2015-05-18 15:26:46 +03:00
Avi Kivity
0eb842dc5b db: write memtable after sealing it
Still missing handling after write completes.
2015-05-18 15:00:33 +03:00
Avi Kivity
ca49d73f97 db: allow configuring a column family to be memory-only
Useful for tests.
2015-05-18 15:00:33 +03:00
Avi Kivity
dda5cbfd0d db: make column_family and keyspace configurable
Currently used for the data directory.
2015-05-18 15:00:31 +03:00
Avi Kivity
7842113cb6 db: prune some unused column_familiy methods
Made redundant by switching tests to using memtable directly.
2015-05-18 14:59:02 +03:00
Gleb Natapov
3838d82c1f merge_schema copies storage_proxy unintentionally 2015-05-18 10:47:36 +02:00
Avi Kivity
d9d16a37b3 db: avoid storing frozen_mutation serialized size in commit log
The commit log already stores the record size, so we can get away with just
storing the data.

Improves 7630fe332d.
2015-05-18 09:03:21 +02:00
Avi Kivity
fd4f36e499 thrift: add missing #include
Popped up under gcc 5.1.
2015-05-17 23:37:28 +03:00
Avi Kivity
784e03aa31 thread: use setjmp/longjmp for context switches
swapcontext() is expensive as it invokes system calls.  Replace it with
setjmp()/longjmp().  We still use setcontext() initially, since that's
the most reasonable portable method of setting up a stack.

Context switch time (measured by thread_context_switch) is reduced to
120ns (from 450ns), with inefficiencies in the test itself and in future<>
dominating.
2015-05-17 17:22:48 +03:00
Avi Kivity
6223e88653 tests: add thread context switch timing test 2015-05-17 15:57:11 +03:00
Avi Kivity
7a309c07f5 tests: add basic thread test 2015-05-17 15:57:11 +03:00
Avi Kivity
cd0ae463b3 core: thread support
Add a thread class that can be used to launch a blockable thread of
execution.  Within a thread, future<>::get() can be called on an
unavailable future, in which case it blocks until the future is made ready.
2015-05-17 15:57:11 +03:00
Avi Kivity
99f0bc784e tests: remove old sstables before creating new in sstable_test 2015-05-17 12:54:53 +03:00
Glauber Costa
64d1538b7c schema: add filter false positive chance
Add methods to get and manipulate the current bloom filter false positive
chance - this is a per-cf field that lives in CFMetaData for Origin, so we'll
have it in the schema.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-17 12:38:36 +03:00
Glauber Costa
2174285c31 db: move memtable definition to its own file
Following what happened to others: we can now include memtable.hh
without including database.hh

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-17 12:38:32 +03:00