Commit Graph

11716 Commits

Author SHA1 Message Date
Takuya ASADA
5270da1eef dist: prevent use abrt with scylla
scylla should use with systemd-coredump, not abrt.
Fixes #762

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452713081-32492-1-git-send-email-syuu@scylladb.com>
2016-01-18 10:47:11 +02:00
Pekka Enberg
7d3a3bd201 Merge "column family cleanup support" from Raphael
"This patch is intended to add support to column family cleanup, which will
 make 'nodetool cleanup' possible.

 Why is this feature needed? Remove irrelevant data from a node that loses part
 of its token range to a newly added node."
2016-01-18 10:15:05 +02:00
Pekka Enberg
6cc02242f6 Merge "Multi schema support in commit log" from Paweł
"This series adds support for multiple schema versions to the commit log.
 All segments contain column mappings of all schema versions used by the
 mutations contained in the segment, which are necessary in order to be
 able to read frozen mutations and upgrade them to the current schema
 version."
2016-01-18 10:11:26 +02:00
Avi Kivity
d5050e4c6a storage_proxy: make MUTATION and MUTATION_DONE verbs sychronous at the server side
While MUTATION and MUTATION_DONE are asynchronous by nature (when a MUTATION
completes, it sends a MUTATION_DONE message instead of responding
synchronously), we still want them to be synchronous at the server side
wrt. the RPC server itself.  This is because RPC accounts for resources
consumed by the handler only while the handler is executing; if we return
immediately, and let the code execute asynchronously, RPC believes no
resources are consumed and can instantiate more handlers than the shard
has resources for.

Fix by changing the return type of the handlers to future<no_wait_type>
(from a plain no_wait_type), and making that future complete when local
processing is over.

Ref #596.
Message-Id: <1453048967-5286-1-git-send-email-avi@scylladb.com>
2016-01-18 09:59:34 +02:00
Nadav Har'El
d97cbbbe43 repair: forbid repair with "-dc" not including the current host
Theoretically, one could want to repair a single host *and* all the hosts
in one or more other data centers which don't include this host. However,
Cassandra's "nodetool repair" explicitly does not allow this, and fails if
given a list of data centers (via the "-dc" option) which doesn't include
the host starting the repair. So we need to behave like "nodetool repair"
and fail in this case too.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <1453037016-25775-1-git-send-email-nyh@scylladb.com>
2016-01-18 09:54:16 +02:00
Paweł Dziepak
fa7bef72d4 tests/cql3: add tests for ALTER TABLE validation
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:50 +01:00
Paweł Dziepak
b7e58db7ec tests: allow any future in assert_that_failed()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:44 +01:00
Paweł Dziepak
00f7a873a5 cql3: forbid re-adding collection with incompatible type
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:38 +01:00
Paweł Dziepak
4927ff95da schema: read collections from comparator
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:33 +01:00
Paweł Dziepak
725129deb7 type_parser: accept sstring_view
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:27 +01:00
Paweł Dziepak
6372a22064 schema: use _raw._collections to generate comparator name
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:35:03 +01:00
Paweł Dziepak
84840c1c98 schema: keep track of removed collections
Cassandra disallows adding a column with the same name as a collection
that existed in the past in that table if the types aren't compatible.
To enforce that Scylla needs to keep track of all collections that ever
existed in the column family.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-18 08:34:29 +01:00
Avi Kivity
c4cf4e0bcd Merge seastar upstream
* seastar a8183c1...e93cd9d (2):
  > rpc: make sure we serialize on _resources_available sempahore
  > rpc: fix support for handlers returning future<no_wait_type>
2016-01-17 18:36:22 +02:00
Avi Kivity
249dbc1d8e Merge seastar upstream
* seastar 6f9453d...a8183c1 (2):
  > rpc: fix server losing handler
  > Merge "Fair I/O Queue" from Glauber
2016-01-17 14:21:53 +02:00
Takuya ASADA
01309c0dd8 dist: add missing dependency (xfslibs-dev) for Ubuntu
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-01-16 15:39:09 +09:00
Takuya ASADA
9ad3365353 dist: use gdebi to resolve install-time dependencies
Since we switched to use mk-build-deps, it only resolves build-time dependencies.
We also need to install install-time dependencies.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-01-16 15:39:09 +09:00
Takuya ASADA
705285cf27 dist: resolve build time dependency by mk-build-deps command, do not install them manually
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-01-16 15:39:09 +09:00
Takuya ASADA
90be81f9ba dist: add missing build time dependency for thrift package on Ubuntu
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-01-16 15:39:09 +09:00
Tomasz Grabiec
d332fcaefc row_cache: Restore indentation 2016-01-15 15:33:17 +01:00
Tomasz Grabiec
6b3cd35109 Merge branch 'pdziepak/multi-schema-sstables/v1'
From Paweł:

This series add support for reading sstables using different schema than
the one that was used to write them.
2016-01-15 14:23:18 +01:00
Paweł Dziepak
dbf23fdff5 tests/sstable: add test for multi schema
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-15 13:12:40 +01:00
Paweł Dziepak
cfc0a132a9 sstable: handle multi-cell vs atomic incompatibilities
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-15 13:12:40 +01:00
Paweł Dziepak
581271a243 sstables: ignore data belonging to dropped columns
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-01-15 13:12:40 +01:00
Asias He
e10580f474 cql_server: Fix connection shutdown
_fd is of type connected_socket. shutdown_input() and shutdown_output()
return future<>. Do not ignore the future.

Message-Id: <786eee890541a18d3501ecd52415f2900c545157.1452835922.git.asias@scylladb.com>
2016-01-15 11:37:30 +02:00
Tomasz Grabiec
ccd609185f sstables: Add ability to wait for async sstable cleanup tasks
This patch adds a function which waits for the background cleanup work
which is started from sstable destructors.

We wait for those cleanups on reactor exit so that unit tests don't
leak. This fixes erratic ASAN complaint about memory leak when running
schema_change_test in debug mode:

    Indirect leak of 64 byte(s) in 1 object(s) allocated from:
         0x7fab24413912 in operator new(unsigned long) (/lib64/libasan.so.2+0x99912)
         0x1776aeb in make_unique<continuation<future<T>::then_wrapped(Func&&) [with Func = future<T>::handle_exception(Func&&) [with Func = sstables::sstable::~sstable()::<lambda(auto:52)>; T = {}]::<lambda(auto:5&&)>; Result = future<>; T = {}]::<lambda(auto:2&&)> >, future<T>::then_wrapped(Func&&) [with Func = future<T>::handle_exception(Func&&) [with Func = sstables::sstable::~sstable()::<lambda(auto:52)>; T = {}]::<lambda(auto:5&&)>; Result = future<>; T = {}]::<lambda(auto:2&&)> > /usr/include/c++/5.1.1/bits/unique_ptr.h:765
         0x1752b69 in schedule<future<T>::then_wrapped(Func&&) [with Func = future<T>::handle_exception(Func&&) [with Func = sstables::sstable::~sstable()::<lambda(auto:52)>; T = {}]::<lambda(auto:5&&)>; Result = future<>; T = {}]::<lambda(auto:2&&)> > /home/tgrabiec/src/scylla2/seastar/core/future.hh:513
        0x1711365 in schedule<future<T>::then_wrapped(Func&&) [with Func = future<T>::handle_exception(Func&&) [with Func = sstables::sstable::~sstable()::<lambda(auto:52)>; T = {}]::<lambda(auto:5&&)>; Result = future<>; T = {}]::<lambda(auto:2&&)> > /home/tgrabiec/src/scylla2/seastar/core/future.hh:690
        0x16d0474 in then_wrapped<future<T>::handle_exception(Func&&) [with Func = sstables::sstable::~sstable()::<lambda(auto:52)>; T = {}]::<lambda(auto:5&&)>, future<> > /home/tgrabiec/src/scylla2/seastar/core/future.hh:880
        0x1696e9c in handle_exception<sstables::sstable::~sstable()::<lambda(auto:52)> > /home/tgrabiec/src/scylla2/seastar/core/future.hh:1012
        0x1638ba8 in sstables::sstable::~sstable() sstables/sstables.cc:1619

The leak is about allocations related to close() syscall tasks invoked
from sstable destructor, which were not waited for.

Message-Id: <1452783887-25244-1-git-send-email-tgrabiec@scylladb.com>
2016-01-15 11:32:15 +02:00
Calle Wilund
e935c9cd34 select_statement: Make sure all aggregate queries use paging
Mainly to make sure we respect row limits. Since normal result
generation does not for aggregates.

Fixes #752 

Message-Id: <1452681048-30171-2-git-send-email-calle@scylladb.com>
2016-01-14 19:03:37 +02:00
Calle Wilund
1dc5937f40 query_pagers: fix log message in requires_paging
message would state that all queries required paging, even when
returning the opposite to caller

Message-Id: <1452681048-30171-1-git-send-email-calle@scylladb.com>
2016-01-14 19:03:16 +02:00
Asias He
cc3073b42d gossip: cleanup application_state
Drop the unused one.

Message-Id: <4cc45164d55742951b618d2c7b1e8bdb997f005a.1452771260.git.asias@scylladb.com>
2016-01-14 19:01:51 +02:00
Avi Kivity
d47a58cc32 README: add libxml2 and libpciaccess packages to list or required packages
Needed for link stage.
2016-01-14 17:47:48 +02:00
Avi Kivity
cf7e6cede2 README: add hwloc and numactl to install recommendations 2016-01-14 17:30:43 +02:00
Tomasz Grabiec
b7976f3b82 config: Set default logging level to info
Commit d7b403db1f changed the default in
logging::logger. It affected tests but not scylla binary, where it's
being overwritten in main.cc.
Message-Id: <1452777008-21708-1-git-send-email-tgrabiec@scylladb.com>
2016-01-14 15:11:58 +02:00
Pekka Enberg
9306f4eb22 Merge "Disable ALTER TABLE statement unless --experimental=on" from Tomek 2016-01-14 14:30:20 +02:00
Avi Kivity
cf8ab65fbc Merge seastar upstream
* seastar 43e64c2...6f9453d (2):
  > Merge "rpc resource accounting"
  > core: Introduce smp::invoke_on_all()
2016-01-14 14:28:27 +02:00
Asias He
826b6ed877 gossip: Print node status in handle_major_state_change
Message-Id: <1452768680-32355-1-git-send-email-asias@scylladb.com>
2016-01-14 14:22:37 +02:00
Asias He
e7a899f5f3 gossip: Enable debug msg for convcit
Kill one FIXME in convict

Message-Id: <1452768680-32355-2-git-send-email-asias@scylladb.com>
2016-01-14 14:22:36 +02:00
Tomasz Grabiec
054f1df0a5 cql3: Disable ALTER TABLE unless experimental features are on 2016-01-14 13:21:13 +01:00
Tomasz Grabiec
1fd03ea1d2 tests: cql_test_env: Enable experimental features 2016-01-14 13:21:13 +01:00
Tomasz Grabiec
a13aaa62df config: Add 'experimental' switch 2016-01-14 13:21:13 +01:00
Gleb Natapov
647a09cd7b storage_proxy: improve mutation timeout logging
Message-Id: <20160114105359.GY6705@scylladb.com>
2016-01-14 12:00:35 +01:00
Pekka Enberg
733584c44d main: Start the API service as the last step
This reverts commit f0d68e4 ("main: start the http server in the first
step"). The service layer is not ready to serve clients before it's
fully up and running which causes early startup crashes everywhere.
Message-Id: <1452768015-22763-1-git-send-email-penberg@scylladb.com>
2016-01-14 12:55:50 +02:00
Tomasz Grabiec
1daaf909d7 Merge branch 'tgrabiec/row_cache_invalidate_fix'
Fixes for wrap-around range handling in row_cache.
2016-01-14 11:38:26 +01:00
Takuya ASADA
7479cde28b dist: extend root disk size to 10GB
Since default root disk size is too small for our purpose, it's better to extend.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452762325-5620-1-git-send-email-syuu@scylladb.com>
2016-01-14 11:29:00 +02:00
Pekka Enberg
90123197e1 service/client_state: Use anonymous user when authentication is disabled
If authentication is disabled, nobody calls login() to set the current
user. There's untranslated code in client_state constructor to do just
that.

Fixes "You have not logged in" errors when USE statement is executed
with authentication disabled.
Message-Id: <1452759946-13998-1-git-send-email-penberg@scylladb.com>
2016-01-14 09:29:33 +01:00
Avi Kivity
4143cf6385 Merge "Initial authenticator support" from Calle
"Add implementation of cassandra password authenticator, and user
password checking to CQL connections.

User/pwd are stored in system_auth table. Passwords are hashed
using glibc 'crypt_r'.

The latter is worth noting, as this is a difference compared to origin;
Origin uses Java bcrypt library for salt/hash, i.e. blowfish hashing.
Most glibc variants do _not_ have support for blowfish. To be 100%
compatible with imported origin tables we might need to add
bcrypt/blowfish sources into scylla (no packaged libs available afaict)

The code currently first attempts to use blowfish, if we happen to run
centos or Openwall, which has it compiled in. Otherwise we will fall
back to sha512, sha256 or even md5 depending on lib support.

To use:
* scylla.conf: authenticator=PasswordAuthenticator
* cqlsh -u cassandra -p cassandra

Not implemented (yet):
* "Authorizer", thus no KS/CF access checking
* CQL create/alter/delete user (create_user_statement etc). I.e. there is
  only a single user name; default "cassandra:cassandra" user/pwd combo"
2016-01-13 19:13:05 +02:00
Takuya ASADA
0511b02f90 dist: run scylla_prepare, scylla_stop on sudo
Since we changed uid on scylla-server.service to scylla, we need sudo for these scripts.

Fixes #783

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452704598-5292-1-git-send-email-syuu@scylladb.com>
2016-01-13 19:06:33 +02:00
Tomasz Grabiec
6b059fd828 row_cache: Guard against wrap-around range in make_reader() 2016-01-13 17:50:55 +01:00
Tomasz Grabiec
7fb0bc4e15 row_cache: Take the reclaim lock in invalidate()
It's needed to keep the iterators valid in case eviciton is triggered
somehwere in between. It probably isn't because destructors should not
allocate, but better be safe.
2016-01-13 17:50:55 +01:00
Tomasz Grabiec
5e05f63ee7 tests: Add more tests for row_cache::invalidate()
Regs #785.
2016-01-13 17:50:55 +01:00
Tomasz Grabiec
50cc0c162e row_cache: Make invalidate() handle wrap-around ranges
Currently for wrap around the "begin" iterator would not meet with the
"end" iterator, invoking undefined behavior in erase_and_dispose()
which results in a crash.

Fixes #785
2016-01-13 17:50:55 +01:00
Calle Wilund
8192384338 auth_test: Unit tests for auth objects 2016-01-13 15:37:39 +00:00