There is another field I missed, index_interval. It is not actually used for
2.1.8 - so that's why it is easy to stop, but it at least exists.
2.1.8 already has "min_index_interval" and "max_index_interval". If we see a
table that contains index_interval, that will become "min_index_interval".
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
They are multi-cell in Origin. This has nothing to do with 2.2 vs 2.1,
and it is just a plain bug.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
We are currently assigning non-partition keys the index 0. That is not what
happens in Origin:
cqlsh> create table ks.twoclust \
(ks int, cl1 int, cl2 int, r1 text, r2 text, primary key (ks, cl1, cl2));
cqlsh> select columnfamily_name, column_name, component_index \
from system.schema_columns where keyspace_name='ks';
columnfamily_name | column_name | component_index
-------------------+-------------+-----------------
twoclust | cl1 | 0
twoclust | cl2 | 1
twoclust | ks | null
twoclust | r1 | 2
twoclust | r2 | 2
This is happening because we use column.position(), which has no knowledge of
the clustering keys at all. We should instead pass that by the schema, which
will then do the right thing.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
We will invoke the schema builder from schema_tables.cc, and at that point, the
information about compact storage no longer exists anywhere. If we just call it
like this, it will be the same as calling it with compact_storage::no, which
will trigger a (wrong) recomputation for compact_storage::yes CFs
The best way to solve that, is make the compact_storage parameter mandatory
every time we create a new table - instead of defaulting to no. This will
ensure that the correct dense and compound calculation are always done when
calling the builder with a parameter, and not done at all when we call it
without a parameter.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This table exists in 2.1.8, and although it is dropped in 2.2, we
should at least list its schema.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2.1.8 tables have 3 more fields in their system tables, that 2.2 don't.
Since we aim at 2.1 compatibility, we have to include them.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
They do not exist in 2.2, and don't serve a huge purpose. But we will
need them for compatibility with 2.1
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Let's leave their schema in here, since it's ready and we may need them in the
future. But since they are not present in 2.1.8, we will remove them from the
schema list.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
ASan does not like commit 05c23c7f73
("database: Add create_keyspace_on_all() helper"):
==8112==WARNING: AddressSanitizer failed to allocate 0x7f88b84fc690 bytes
==8112==AddressSanitizer's allocator is terminating the process instead of returning 0
==8112==If you don't like this behavior set allocator_may_return_null=1
==8112==Sanitizer CHECK failed: ../../../../libsanitizer/sanitizer_common/sanitizer_allocator.cc:147 ((0)) != (0) (0, 0)
I was not able to determine the source of the bug. Make ASan happy by
reverting the code movement and using the "cpu zero" trick we use for
table creation.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
The code is merge_tables() is a twisted maze of tricks that is hard to
restructure so that event notification can be done cleanly like with
keyspaces.
The problem there is that we need to run bunch of database operations
for the merging that really need to happen on all the shards. To fix
the issue, lets cheat a little and simply only run CQL event
notification on cpu zero.
This seems to fix cluster schema propagation issues in urchin-dtest. I
can now run TestSimpleCluster.simple_create_insert_select_test without
any additional delays inserted into the test code.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
We already have all_tables() function converted and there's really no
use for compile() unless we switch to using CQL to create the schema
tables.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
There's nothing legacy about it so rename legacy_schema_tables to
schema_tables. The naming comes from a Cassandra 3.x development branch
which is not relevant for us in the near future.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
"This series implements initial support for CQL events. We introduce
migration_listener hook in migration manager as well as event notifier
in the CQL server that's built on top of it to send out the events via
CQL binary protocol. We also wire up create keyspace events to the
system so subscribed clients are notified when a new keyspace is
created.
There's still more work to be done to support all the events. That
requires some work to restructure existing code so it's better to merge
this initial series now and avoid future code conflicts."
Add a create_keyspace_on_all() helper which is needed for sending just
one event notification per created keyspace, not one per shard.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
We should pass inet_address.addr().
With this, tokens in system.peers are updated correctly.
(1 rows)
cqlsh> SELECT tokens from system.peers;
tokens
------------------------------------------------------------------------
{'-5463187748725106974', '8051017138680641610', '8833112506891013468'}
(1 rows)
I got this error If I pass inet_address to it.
boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::bad_any_cast>
> (boost::bad_any_cast: failed conversion using boost::any_cast)
Several of Scylla's options (in db/config.hh) have the type "string_map"
(unordered_map<sstring, sstring>). The intent was such options could get
multiple "key=value" settings. However, this never actually worked correctly,
and had two bugs:
1. Any option name with a space in it would fail, for example:
$ scylla --logger-log-level 'BatchLog Manager=info'
error: the argument ('BatchLog Manager=info') for option '--level'
is invalid
2. Trying to set multiple entries in the map did *not* work. For example,
$ scylla --logger-log-level a=info --logger-log-level b=info
error: option '--level' cannot be specified more than once
The problem is that boost::program_options does not actually understand
unordered_map<sstring, sstring>: It doesn't know it is a container (it
only recognizes std::vector) so it doesn't allow multiple options, and
it doesn't know how to convert a string to it, so it uses boost::lexical_cast
which for strings, cuts the string at a space...
The solution is to write a custom "validate()" function overload, which
boost::program_options uses to validate (and consume) options into object
types it doesn't understand by default. Getting this function in the right
place in the code was a difficult exercise, but here it is, a working
implementation :-) And it fixes the above two bugs.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Abillity to enable/disable specific sub-modules - this settings do not
affect system tables which are allways persisted,cached and written to
commitlog
enable-in-memory-data-store marks if tables will be written/read to/from
disk
enable-commitllog marks if tables will be written to commitlog
enable-cache marks if tables will be written/read to/from cache
Please note in-memory-data-store does not change the read path so "old"
sstables are still read and cache may be used to cache their data
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
Because we don't support alter table, we won't have anyone actually manipulating this.
But we will set up so it appears properly as an empty map in queries.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
When added to the schema and handled by legacy_schema_tables.cc, will then
appear correctly in the respective system tables.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
The at_exit() callback needs to return a future. In one place we forgot,
and now that at_exit() takes an std::function<>, this is verified at
compilation time and fails compilation.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Move unavailable_exception to exceptions.hh where other CQL transport
level exceptions are defined in.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Move 'consistency_level' enumeration to a separate header file to fix
dependency issues that arise when we move 'unavailable_exception' to
exceptions.hh.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Cleanup consistency_level.hh by removing untranslated code that's been
sitting in the tree for a while.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
"This is my current proposal for Compact Storage tables - plus
the needed infrastructure.
Getting rid of the CellName abstraction allows us to simplify
things by quite a lot: now all we need is to mark whether or
not a table is composite, and provide functions to play the
role of the comparator when dealing with the strings."
Transport layer expects to get error code in an exception of type
exceptions::cassandra_exception. Fix code to use it as a base for
all user visible exceptions and put correct error code there.
The column type of "schema_version" is set to UTF8 which results in the
following value conversion errors:
storage_service: fail to update schema_version for 127.0.0.2: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::bad_any_cast> > (boost::bad_any_cast: failed conversion using boost::any_cast)
Change the column type to UUID like in Origin. Fixes#35.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
We were currently using the regular column name as the comparator. That is
only correct in some specific cases, in particular, of a non-compound cell name,
that has no collections.
Now that we have the cell_name.hh infrastructure, we can use it.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Some of the fields in the schema will not not be passed directly, but
calculated by the schema_builder. This is because they are not simple values of
the form a = x, but slightly more complex and have to be derived from other
properties (for instance, the amount of clustering keys).
For those values, a default value does not really make sense. Take for instance
the case of the comparator: there is always a comparator, and there is not any
comparator that can serve as a default. Which comparator to use depend on
whether or not the table has collections, whether or not it has clustering
keys, and whether or not it is marked as compact storage.
Making all tables go through the builder is a guarantee that all of them
will have their values set.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Origin has another column_kind, that we lack: compact_value. This kind is
used to identify regular columns of dense tables.
Take for instance, the following table:
CREATE TABLE ks2.compact (
ks text,
cl1 text,
cl2 text,
PRIMARY KEY (ks, cl1)
) WITH COMPACT STORAGE
cqlsh> select keyspace_name, columnfamily_name, column_name, type from system.schema_columns \
where keyspace_name='ks2' and columnfamily_name='compact';
keyspace_name | columnfamily_name | column_name | type
---------------+-------------------+-------------+----------------
ks2 | compact | cl1 | clustering_key
ks2 | compact | cl2 | compact_value
ks2 | compact | ks | partition_key
We will treat those columns as regular columns for most purposes. Because of
that, we don't need to separate them from the regular columns when we sort
initially, for instance. All we have to do is change its type.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This is how it happens for Origin. Take for instance the following CF:
CREATE TABLE ks2.noregular_cs2 (
ks text,
cl1 text,
cl2 text,
PRIMARY KEY (ks, cl1, cl2)
) WITH COMPACT STORAGE;
cqlsh> select keyspace_name, columnfamily_name, column_name from system.schema_columns \
where keyspace_name='ks2' and columnfamily_name='noregular_cs2';
keyspace_name | columnfamily_name | column_name
---------------+-------------------+-------------
ks2 | noregular_cs2 | <===== added this.
ks2 | noregular_cs2 | cl1
ks2 | noregular_cs2 | cl2
ks2 | noregular_cs2 | ks
In order to achieve that, we need to relax the test in db/legacy_schema_tables.cc.
It will throw in case it finds an empty name.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
