"Defines origin v3-format for system/schema tables, and use them for
schema storage/retrival.
Includes a legacy_schema_migrator implementation/port from origin. Note
that since we don't support features like triggers, functions and
aggregates, it will bail if encountering such a feature used.
Note also that this patch set does not convert the "hints" and
"backlog" tables, even though these have changed in v3 as well.
That will be a separate patch set.
Tested against dtests. Note that patches for dtest + ccm
will follow."
* 'calle/systemtables' of github.com:cloudius-systems/seastar-dev: (36 commits)
legacy_schema_migrator: Actually truncate legacy schema tables on finish
database: Extract "remove" from "drop_columnfamily"
v3 schema test fixes
thrift: Update CQL mapping of static CFs
schema_tables: Use v3 schema tables and formats
type_parser: Origin expects empty string -> bytes_type
cf_prop_defs: Add crc_check_chance as recognized (even if we don't use)
types_test: v3 style schemas enforce explicit "frozen" in tupes/ut:s
cql3_type: v3 to_string
cql_types: Introduce cql3_type::empty and associate with empty data_type
schema: rename column accessors to be in line with origin
schema: Add "is_static_compact_table"
schema_builder: Add helper to generate unique column names akin origin
schema: Add utility functions for static columns
schema: Use heterogeneous comparator for columns bounds
cql3_type_parser: Resolve from cql3 names/expressions
cql3_type: Add "prepare_interal" and "references_user_type"
cql3::cql3_type: Add prepare_internal path using only "local" holders
cql3_type: Add virtual destructor.
database/main: encapsulate system CF dir touching
...
* seastar 4a3118c...45b718b (7):
> tests: make connect_test use a random port
> log: Introduce log.info0
> configure.py: link to DPDK PMD drivers which are already built on build/dpdk and enabled by default on DPDK config
> Update fmt submodule
> perftune: fix perftune.py IndexError when NIC uses less IRQs than requested.
> build: Add more required build dependencies to the Dockerfile
> Prometheus: Reserve in protobuf object before iterating
"Tested with:
- test.py --mode relase
- debug/test-serialization
- c-s with both debug and relase compiled scylla with authentication enabled:
cassandra-stress write n=10000 no-warmup -rate threads=10 -mode native unprepared cql3 user='cassandra' password='cassandra'
Tested with:
- test.py --mode relase
- debug/test-serialization
- c-s with both debug and relase compiled scylla with authentication enabled:
cassandra-stress write n=10000 no-warmup -rate threads=10 -mode native unprepared cql3 user='cassandra' password='cassandra'"
* 'compress_tracing_session_id-v6' of github.com:cloudius-systems/seastar-dev:
cql_server::response: rework the tracing session ID insertion
utils::UUID: align the UUID serialization API with the similar API of other classes in the project
utils: serialization: unify the variety of serialize_XXX(...)
cql_server::response: rework the compress(...) method
cql_server::response: store the frame flags inside the class
"This series fixes a set of regressions introduced by
f7bc88734a, resulting in two failed
tests:
testDenseNonCompositeTable(org.apache.cassandra.cql3.validation.operations.CreateTest)
and
testStaticColumnsWithDistinct(org.apache.cassandra.cql3.validation.entities.StaticColumnsTest)"
* 'cql-fixes/v1' of github.com:duarten/scylla:
update_statement: Reject empty values for dense clustering key
modification_statement: Fix detection of clustering keys
cql3/restrictions/statement_restrictions: Consider statement type
cql3/statements/modification_statement: Extract statement_type
Insert the tracing session ID into the response body in the cql_server::response constructor.
Fixes#2356
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
The standard serialization API (e.g. in data_value) includes the following methods:
size_t serialized_size() const;
void serialize(bytes::iterator& it) const;
bytes serialize() const;
Align the utils::UUID API with the pattern above.
The only addition is that we are going to make an output iterator parameter of a second method above
a template so that we may serialize into different output sources.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Use the same templated implementation for all different serialize_XXX(...).
The chosen implementation is based on the std::copy_n(char*, size, OutputIterator),
which is heavily optimized and will be using memcpy/memmove where possible.
This patch also removes the not needed specializations that accept signed integer
values since we were casting them to unsigned value anyway.
The std::ostream based specifications are also removed since they are not used
anywhere except for a test-serialization.cc and adjusting the ostream to the iterator
is a single-liner.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Cleanup the compress(...) method interface:
- Encapsulate the technical details inside the method:
- Re-write the _body inside the method instead of returning it.
- Set the response::_flags inside the method.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
It makes a lot more sense to keep the flags mask inside the response and update it each time
the corresponding feature is set instead of holding the separate components like tracing state
pointer.
This patch adds this ability to set the flags.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
The code that removes each sstable runs in a thread. Parallel
removing of a lot of sstables may start a lot of threads each of which
is taking 128k for its stack. There is no much benefit in running
deletion in parallel anyway, so fix it by deleting sstables sequentially.
Fixes#2384
Message-Id: <20170516103018.GQ3874@scylladb.com>
Newer setuptools parse_version() don't like dashed version strings,
so we should trim it to avoid false negative version_compare() checks.
Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20170511162646.22129-1-ultrabug@gentoo.org>
Prevent the accidental dropping of system_auth and system_traces objects (keyspaces and tables)
but allow their modification (including tables).
We need to be able to modify keyspases in order to set/modify the replication strategy and its parameters.
We need to be able to ALTER the tables in order to allow rolling upgrades when some of the tables has changed.
Fixes#2346Fixes#2338
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1494363335-20424-1-git-send-email-vladz@scylladb.com>
_underlying is created with _range, which is captured by
reference. But range_and_underlyig_reader is moved after being
constructed by do_with(), so _range reference is invalidated.
Fixes#2377.
Message-Id: <1494492025-18091-1-git-send-email-tgrabiec@scylladb.com>
Now that update_statement uses statement_restrictions, we need our
validation logic to take the statement type into account, in
particular to deal with insertion statements which only set static
columns but specify clustering values.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
This patch extracts the statement_type into its own file. The type
will be later passed to statement_restrictions for validation
purposes.
Further along, we could add methods to it that currently live in other
statements so we can move more validation into statement_restrictions.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
This patch updates the mapping of static CFs so that their CQL
representation is a non-compound, non-dense schema with static
columns, instead of regular ones. This matches the representation os
static CFs in Cassandra 3.x.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
More pointedly: Expose columns as is (currently
all_columns_in_select_order), expose name->column mapping more
appropriately named.
Renaming like this is not strictly neccesary, but there is a point to
trying to keep nomenclature similar-ish with origin, esp. when select
order column need to become filtered (spoiler alert).
Cassandra 3 uses cql names for column/field types, thus
we need to parse these out-of-line, and resolve more akin
to the cql parser.
Also wrap building user types similarly to origin, using
a "builder" wrapper, and usage graph resolving.
Regular single key query will never reconcile with CL=ONE since there
will be no digest mismatch, but range queries do not have digest stage,
so always goes through reconcile code. For CL=ONE there will be only one
result though, so no need to run complicated reconciliation logic and the
only result can be returned directly.
Message-Id: <20170509100334.GQ28272@scylladb.com>
The "_specs" array contains column specifications that have the bind
marker name if there is one. That results in
get_partition_key_bind_indices() not being able to look up a column
definition for such columns. Fix the issue by keeping track of the
actual column specifications passed to add() like Cassandra does.
Fixes#2369
Message-Id: <1494397358-24795-1-git-send-email-penberg@scylladb.com>
