close() operation is like a destructor, it cannot fail. It just
reports errors, but close itself succeeds. So we should proceed with
the closing even if it fails.
Message-Id: <1484245886-7269-1-git-send-email-tgrabiec@scylladb.com>
As Tomek pointed out, previous code, regardless of version mismatch, of generating
comparator description string was not correct (as in: in sync with origin).
This modifies it to look at
1.) Actual clustring size
2.) Compound-ness
3.) Dense-ness
to determine whether we should generate a compound desc, and whether it
should contain a trailing utf8-desc type.
v2: Simplify non-dense base column addition and ensure it handles
thrift non-utf8 (as per comments from tomek)
Message-Id: <1484670171-18362-1-git-send-email-calle@scylladb.com>
As the metrics migration progressed, some include to scollectd.hh left
behind.
Because of the nature of the scollecd implementation those include
brings alot of code with them to the header files and eventually to many
source file.
This patch remove those include and add a missing include to
storage_proxy.cc.
The reason the compiler didn't complain is an indication to the
problematic nature of those include in the first place.
Before this patch, change in metrics.hh would cause 169 files to
compile, after this change 17.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1484667536-2185-1-git-send-email-amnon@scylladb.com>
Fixes#2019
According to the Java driver and cassandra, all versions < 3
include the PK in the comparator descriptor string.
This broke for us when bumping the cassandra version 2.1 -> 2.2
Message-Id: <1484657580-14411-1-git-send-email-calle@scylladb.com>
This patch ensures that the host only announces and registers the
MATERIALIZED_VIEWS feature if it was started with the experimental
flag.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170116123412.21365-1-duarte@scylladb.com>
After we call unlink_leftmost_without_rebalance(), we must unlink all
elements before mutatation is destroyed. We did this properly from
~reader, but it would not be called if reader construction failed,
which it may.
Message-Id: <1484572581-6537-1-git-send-email-tgrabiec@scylladb.com>
Since ce083308a1
"random_mutation_generator: Generate RTs by default" random mutation
generator produces range tombstones. However, so far the tests were run
with all features disabled (because of incomplete initialization of all
services) which meant that RANGE_TOMBSTONE feature was not enabled and
the code couldn't handle range tombstones that weren't just prefixes.
This patch solves the problem by forcing all features to be enabled when
tests are run.
Message-Id: <20170116103324.22956-1-pdziepak@scylladb.com>
It still has problems:
- while resharding a very large leveled compaction strategy table, a huge
amount of tiny sstables are generated, overwhelming the file descriptor
limits
- there is a large impact on read latency while resharding is going on
(cherry picked from commit cf27d44412)
(forward-ported from branch-1.6)
Since scylla-housekeeping running as scylla user, it doesn't have a permission
to create a file on /etc/scylla.d.
So introduce /var/lib/scylla-housekeeping which owns by scylla user, place uuid
file on the directory.
Fixes#2009
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1484235946-12463-1-git-send-email-syuu@scylladb.com>
query_mutations_locally() takes one_or_two_partition_ranges by
reference and requires, indirectly, that it is kept alive until
operation resolves. However, we were passing expiring value to it, the
result of unwrap().
Fixes dtest failure in consistent_bootstrap_test.py:TestBootstrapConsistency.consistent_reads_after_bootstrap_test
Another potential problem was that we were dereferencing "s" in the same
expression which move-constructs an argument out of it.
Message-Id: <1484222759-4967-1-git-send-email-tgrabiec@scylladb.com>
On DPDK 16.11 dpdk_nic_bind.py is renamed to dpdk-devbind.py, so we are
getting "file not found" both on packaging and scripts, fixed that.
Also fixed inconsistent packaging.
Since Seastar copied dpdk_nic_bind.py to its scripts/ directory, there're two
different versions of the script, .rpm/.deb packaging different one:
dist/redhat: seastar/dpdk/tools/dpdk_nic_bind.py
dist/ubuntu: seastar/scripts/dpdk_nic_bind.py
That's won't work because we sharing setup scripts between two
distributions, so I changed dist/ubuntu package to use DPDK one.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1484191955-28006-1-git-send-email-syuu@scylladb.com>
Explicitly generate tables' IDs of tables from the system_traces KS using
generate_legacy_id() in order to ensure all Nodes create these tables with
the same IDs.
This is going to prevent hitting issue #420.
Fixes#1976
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1484153725-31030-1-git-send-email-vladz@scylladb.com>
This patch changes the random_mutation_generator so it generates range
tombstones by default. This fixes a bug where reversibly applying
range tombstones wasn't being tested.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170110164822.28747-1-duarte@scylladb.com>
boost::intrusive::value_traits_pointers was introduced in boost 1.56, while
we also support boost 1.55. Replace with an equivalent expression.
(with additions by Asias)
Message-Id: <20170110084700.19994-1-avi@scylladb.com>
Commit f0c28e1 ("db/schema_tables: Add schema_functions and
schema_aggregates tables") forgot to add the newly added tables to the
db::schema_tables::ALL list, which is used for authorization checks, for
example.
Fixes the following auth_test.py dtest failures:
('Unable to connect to any servers', {'127.0.0.1': Unauthorized('Error from server: code=2100 [Unauthorized] message="User cathy has no SELECT permission on <table system.schema_functions> or any of its parents"',)})
Message-Id: <1484045277-4997-1-git-send-email-penberg@scylladb.com>
"Intended to reduce memory usage when resharding by sharing sstable
components among shards. File descriptors are also shared from now
on, meaning that a much smaller number of file descriptors will be
used during resharding.
Fixes #1951."
branch 'excessive_memory_usage_v4' of github.com:raphaelsc/scylla
* 'excessive_memory_usage_v4' of github.com:raphaelsc/scylla:
db: avoid excessive memory usage during resharding
checked_file_impl: add support to dup
sstables: group sstable components that can be shared among shards
sstables: rename sstable member
After resharding, sstables may be owned by all shards, which
means that file descriptors and memory usage for metadata will
increase by a factor equal to number of shards. That can easily
lead to OOM.
SSTable components are immutable, so they can be stored in one
shard and shared with others that need it. We use the following
formula to decide which shard will open the sstable and share
it with the others: (generation % smp::count), which is the
inverse of how we calculate generation for new sstables.
So if no resharding is performed, everything is shard-local.
With this approach, resource usage due to loaded sstables will
be evenly distributed among shards.
For this approach to work, we now only populate keyspaces from
shard 0. It's now the sole responsible for iterating through
column family dirs. In addition, most of population functions
are now free and take distributed database object as parameter.
Fixes#1951.
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
"This patch series adds support for CQL 3.3.1. The changes to CQL are listed
here:
https://github.com/apache/cassandra/blob/cassandra-2.2/doc/cql3/CQL.textile#changes
The following CQL features are already supported by Scylla:
- TRUNCATE TABLE alias
- Double-dollar string literals
- Aggregate functions: MIN, MAX, SUM, and AVG
This series adds the following CQL features:
- New data types: tinyint, smallint, date, and time
- CQL binary protocol v4 (required by the new data types)
- Advertise Cassandra 2.2.8 version from Scylla so that drivers correctly
detect the presence of CQL 3.3.1
The following CQL features are not supported by Scylla:
- Role-based access control (issue #1941)
- JSON data type
- User-defined functions (UDFs)
- User-defined aggregates (UDAs)
The following CQL binary protocol v4 changes are not implemented by this
series:
- Read_failure and Write_failure error codes are not implemented.
They error codes not used by the smart drivers but as they are
propagated to application code, we eventually need to wire them up
to our storage proxy implementation.
- Function_failure error code is only used by user-defined functions
and the fromJson function, which are not implemented by Scylla.
Fixes #1284."
* 'penberg/cql-3.3.1/v5' of github.com:cloudius-systems/seastar-dev:
version: Bump Cassandra version to 2.2.8
db/schema_tables: Add schema_functions and schema_aggregates tables
tests/type_tests: TIME type test cases
tests/cql_query_test: TIME type test cases
cql3: TIME data type support
tests/type_tests: DATE type test cases
tests/cql_query_test: DATE type test cases
cql3: DATE type support
date.h: 64-bit year and days representation
licenses: Add utils/date.h license
utils/date.h: Import date and time library sources
tests/type_tests: TINYINT and SMALLINT type test cases
tests/cql_query_test: TINYINT and SMALLINT type test cases
cql3: TINYINT and SMALLINT data type support
types: Fix integer_type_impl::parse_int() for bytes
mutation_result_merger::get() assumes that the merged result may be a
short read if at least one of the partial results is a short read (in
other words, if none of the partial results is a short read, then the
merged result is also not a short read). However this is not true;
because we update the memory accounter incrementally, we may stop
scanning early. All the partial results are full; but we did not scan
the entire range.
Fix by changing the short_read variable initialization from `no`
(which assumes we'll encounter a short read indication when processing
one of the batches) to `this->short_read()`, which also takes into
account the memory accounter.
Fixes#2001.
Message-Id: <20170108111315.17877-1-avi@scylladb.com>
