Commit Graph

631 Commits

Author SHA1 Message Date
Takuya ASADA
ec2ef467c8 configure.py: configure.py: add --static-thrift option to link libthrift statically
This is needed for Ubuntu packaging, to drop dependency to libthrift0 on installation time.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461594460-2403-1-git-send-email-syuu@scylladb.com>
2016-04-25 17:44:44 +03:00
Duarte Nunes
fbf70e9bed udt: Add alter type statement
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-04-20 18:07:07 +02:00
Duarte Nunes
809b45e160 udt: Add drop type statement
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-04-20 18:07:02 +02:00
Pekka Enberg
4ed702f0da Merge "Authorizer support" from Calle
"Conversion/implementation of "authorizer" code from origin, handling
 permissions management for users/resources.

 Default implementation keeps mapping of <user.resource>->{permissions}
 in a table, contents of which is cached for slightly quicker checks.

 Adds access control to all (existing) cql statements.
 Adds access management support to the CQL impl. (GRANT/REVOKE/LIST)

 Verified manually and with dtest auth_test.py. Note that several of these
 still fail due to (unrelated) unimplemented features, like index, types
 etc.

 Fixes #1138"
2016-04-19 15:00:38 +03:00
Calle Wilund
14cc47d8b9 cql3::statements::revoke_statement: Initial conversion 2016-04-19 11:49:06 +00:00
Calle Wilund
4e1ef3c1bc cql3::statements::grant_statement: Initial conversion 2016-04-19 11:49:05 +00:00
Calle Wilund
04c37def3a cql3::statements::list_permissions_statement: Initial conversion 2016-04-19 11:49:05 +00:00
Calle Wilund
fe23447f6f cql3::statements::permission_altering_statement: Inital conversion
Alter permission base typ
2016-04-19 11:49:05 +00:00
Calle Wilund
add2111c0a cql3::statements::authorizarion_statement: Initial conversion
Auth cql base type
2016-04-19 11:49:05 +00:00
Calle Wilund
1f0bbf2d9a auth::authorizer: Initial conversion
Main authorization endpoint. Default (and only) real authorizer
keeps a mapping resource -> permission sets in system table
2016-04-19 11:49:04 +00:00
Raphael S. Carvalho
29db5f5e1f sstables: move compaction strategy code to a new source file
Moving compaction strategy code from sstables/compaction.cc to
sstables/compaction_strategy.cc
That improves readability. Strategy code should be separated
from the generic compaction code.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <5af6fc8f7321351a071fc0ce03c80ffea21f8396.1460761821.git.raphaelsc@scylladb.com>
2016-04-19 08:45:43 +03:00
Calle Wilund
b8bd77e621 cql3::list_users_statement: Initial conversion 2016-04-11 09:10:41 +00:00
Calle Wilund
adaf21403b cql3::drop_user_statement: Initial conversion 2016-04-11 09:10:41 +00:00
Calle Wilund
8732b3eed7 cql3::alter_user_statement: Initial conversion 2016-04-11 09:10:41 +00:00
Calle Wilund
da89189308 cql3::create_user_statement: Initial conversion 2016-04-11 09:10:41 +00:00
Calle Wilund
57f5bb854f cql3::authentication_statement: cql auth base class 2016-04-11 09:10:41 +00:00
Calle Wilund
cef52d1653 cql3::user_options: Add options wrapper type 2016-04-11 09:10:41 +00:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Pekka Enberg
4892a6ded9 build: Invoke Seastar build only once
Make sure we invoke the Seastar ninja build only once from our own build
process so that we don't have multiple ninjas racing with each other.

Refs #1061.

Message-Id: <1458563076-29502-1-git-send-email-penberg@scylladb.com>
2016-03-21 16:22:11 +02:00
Benoît Canet
1fb9a48ac5 exception: Optionally shutdown communication on I/O errors.
I/O errors cannot be fixed by Scylla the only solution
is to shutdown the database communications.

Signed-off-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1458154098-9977-1-git-send-email-benoit@scylladb.com>
2016-03-17 15:02:52 +02:00
Asias He
9f64c36a08 storage_service: Fix pending_range_calculator_service
Since calculate_pending_ranges will modify token_metadata, we need to
replicate to other shards. With this patch, when we call
calculate_pending_ranges, token_metadata will be replciated to other
non-zero shards.

In addition, it is not useful as a standalone class. We can merge it
into the storage_service. Kill one singleton class.

Fixes #1033
Refs #962
Message-Id: <fb5b26311cafa4d315eb9e72d823c5ade2ab4bda.1457943074.git.asias@scylladb.com>
2016-03-14 10:14:22 +02:00
Pekka Enberg
97bef4fb7c build: Fix http/http_response_parser.hh dependency
Make sure http_response.hh that is pulled by locator/ec2_snitch.hh is
built. The commit is similar to what commit 6ccf8f8 ("build: make sure
to ask seastar to build http/request_parser.hh, and depend on it") did
for request_parser.hh.

Fixes the following build error on CentOS:

  In file included from ./locator/ec2_multi_region_snitch.hh:41:0,
                   from locator/ec2_multi_region_snitch.cc:39:
  ./locator/ec2_snitch.hh:24:40: fatal error: http/http_response_parser.hh: No such file or directory

Spotted by Shlomi.
Message-Id: <1457612266-315-1-git-send-email-penberg@scylladb.com>
2016-03-10 14:46:41 +01:00
Pekka Enberg
2566f8dc18 configure: Remove 'scylla_libs' variable
It's not actually used by anyone so drop it.
Message-Id: <1457531753-27891-2-git-send-email-penberg@scylladb.com>
2016-03-09 14:56:54 +01:00
Pekka Enberg
9bfb6a0c5b configure: Add boost date_time library as a dependency
It's needed to fix the debug build.
Message-Id: <1457531753-27891-1-git-send-email-penberg@scylladb.com>
2016-03-09 14:56:51 +01:00
Takuya ASADA
da56325f69 configure.py: add support --static-stdc++ for seastar binaries (iotune)
Ubuntu 14.04LTS package is broken now because iotune does not statically linked against libstdc++, so this patch fixed it.
Requires seastar patch to add --static-stdc++ on configure.py.

Fixes #982

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1456995050-22007-1-git-send-email-syuu@scylladb.com>
2016-03-03 12:18:47 +02:00
Avi Kivity
bec30ccf25 build: add order-only dependency between building antlr .o and IDL headers
This ensures that if an antlr generated .cpp file depends on an
IDL-generated .hh file, then that .hh is generated before the .o is
built.
2016-03-03 09:52:25 +02:00
Paweł Dziepak
d50594351b db: remove old-style serializers
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-03-02 09:09:30 +00:00
Tomasz Grabiec
6cec131432 query: Switch to IDL-generated views and writers
The query result footprint for cassandra-stress mutation as reported
by tests/memory-footprint increased by 18% from 285 B to 337 B.

perf_simple_query shows slight regression in throughput (-8%):

  build/release/tests/perf/perf_simple_query -c4 -m1G --partitions 100000

Before: ~433k tps
After:  ~400k tps
2016-02-26 12:26:13 +01:00
Tomasz Grabiec
f72fd9eefd Merge branch 'pdziepak/canonical-mutation-idl/v1' from sesastar-dev.git 2016-02-23 17:02:43 +01:00
Pekka Enberg
4ff1692248 cql3: Make 'CREATE TYPE' error message human readable
We don't support the 'CREATE TYPE' statement for now. The user-visible
error message, however, is unreadable because our CQL parser doesn't
even recognize the statement.

  cqlsh:ks1> CREATE TYPE config (url text);
  SyntaxException: <ErrorMessage code=2000 [Syntax error in CQL query] message=" : cannot match to any predicted input...

Implement just enough of 'CREATE TYPE' parsing to be able to report a
human readable error message if someone tries to execute such
statements:

  cqlsh:ks1> CREATE TYPE config (url text);
  ServerError: <ErrorMessage code=0000 [Server error] message="User-defined types are not supported yet">
Message-Id: <1456148719-9473-2-git-send-email-penberg@scylladb.com>
2016-02-22 14:50:25 +01:00
Vlad Zolotarov
cc30956c56 locator: added EverywhereReplicationStrategy
This strategy would ignore an RF configuration and would
always try to replicate on all cluster nodes.

This means that its get_replication_factor()  would return a
number of currently "known" nodes in the cluster and
if a cluster is currently bootstrapping this value obviously may
change in time for the same key. Therefore using this strategy
should be done with caution.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Message-Id: <1456074333-15014-3-git-send-email-vladz@cloudius-systems.com>
2016-02-21 19:29:29 +02:00
Takuya ASADA
7a6f9c9bb4 dist: use /usr/bin/python3.4 for idl-compiler.py on CentOS
Fixes #923

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1455826201-11333-1-git-send-email-syuu@scylladb.com>
2016-02-20 19:15:21 +02:00
Paweł Dziepak
89b75a02d4 commitlog: use IDL-based serialization for entries
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 23:11:59 +00:00
Paweł Dziepak
f548c75200 commitlog: move implementation to *.cc file
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 23:11:59 +00:00
Paweł Dziepak
597ed15dfd tests: add idl unit test
Test auto-generated and writer-based serialization as well as
deserialization of simple compound type, vectors and variants.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:19:30 +00:00
Amnon Heiman
38cd55e9cf Adding the mutation idl
This adds the mutation definition idl and add it to the compilation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 18:42:09 +02:00
Tomasz Grabiec
a921479e71 Merge tag '807-v3' from https://github.com/avikivity/scylla
From Avi:

This patchset introduces a linearization context for managed_bytes objects.

Within this context, any scattered managed_bytes (found only in lsa regions,
so limited to memtable and cache) are auto-linearized for the lifetime of
the context.   This ensures that key and value lookups can use fast
contiguous iterators instead of using slow discontiguous iterators (or
crashing, as is the case now).
2016-02-16 14:29:48 +01:00
Avi Kivity
47ea1237ed build: build seastar's iotune
Target name is build/{mode}/iotune.
2016-02-16 12:13:29 +02:00
Avi Kivity
1f752446d2 Merge "Truncation format & fixes" from Calle
"Fixes #884
Fixes #895

Also at seastar-dev: calle/truncate_more

1.) Change truncation records to be stored with IDL serialization
2.) Fix db::serializers encoding of replay_position
3.) Detect attempted reading of Origin truncation records, and instead
    of crashing, ignore and warn.
4.) Change truncation time stamps to be generated per-shard, _after_
    CF flush is done, otherwise data in memtables at flush would be
    retained/replayed on next start. Retain the highest time stamp
    generated.

Note for (3): This patch set does _not_ clear out origin records
automatically. This because I feel that is a somewhat drastic and
irreversible thing to do. If we want to avail the user of a means
to get rid of the (3) warning, we should probably tell him to either
use cqlsh, or add an API call for this, so he can do it explicitly.
"
2016-02-15 11:39:56 +02:00
Avi Kivity
3c4f67f3e6 build: require boost > 1.55
See #898.

Add checks both for boost being installed, and for the correct version.
Message-Id: <1455193574-24959-1-git-send-email-avi@scylladb.com>
2016-02-11 15:15:49 +02:00
Avi Kivity
af8ef54d5a managed_bytes: introduce with_linearized_managed_bytes()
A large managed_bytes blob can be scattered in lsa memory.  Usually this is
fine, but someone we want to examine it in place without copying it out, but
using contiguous iterators for efficiency.

For this use case, introduce with_linearized_managed_bytes(Func),
which runs a function in a "linearization context".  Within the linearization
context, reads of managed_bytes object will see temporarily linearized copies
instead of scattered data.
2016-02-09 19:55:13 +02:00
Calle Wilund
dff89fffcd IDL: Add idl definitions for replay_position and truncation_record 2016-02-09 13:55:33 +00:00
Avi Kivity
8b0a26f06d build: support for alternative versions of libsystemd pkgconfig
While pkgconfig is supposed to be a distribution and version neutral way
of detecting packages, it doesn't always work this way.  The sd_notify()
manual page documents that sd_notify is available via the libsystemd
package, but on centos 7.0 it is only available via the libsystemd-daemon
package (on centos 7.1+ it works as expected).

Fix by allowing for alternate version of package names, testing each one
until a match is found.

Fixes #879.

Message-Id: <1454858862-5239-1-git-send-email-avi@scylladb.com>
2016-02-07 17:36:57 +02:00
Gleb Natapov
e6f7b12b51 Move partition_checksum to use idl 2016-02-02 12:15:49 +02:00
Gleb Natapov
60e3637efc Move frozen_schema to idl 2016-02-02 12:15:49 +02:00
Gleb Natapov
b065e2003f Move paging_state to use idl 2016-01-27 18:39:43 +02:00
Amnon Heiman
7b53b99968 idl-compiler: split the idl list
Not all the idls are used by the messaging service, this patch removes
the auto-generated single include file that holds all the files and
replaes it with individual include of the generated fiels.
The patch does the following:
* It removes from the auto-generated inc file and clean the configure.py
  from it.
* It places an explicit include for each generated file in
  messaging_serivce.
* It add dependency of the generated code in the idl-compiler, so a
change in the compiler will trigger recreation of the generated files.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1453900241-13053-1-git-send-email-amnon@scylladb.com>
2016-01-27 15:23:00 +02:00
Avi Kivity
a53788d61d Merge "More streaming cleanup and fix" from Asias
"- Drop compression_info/stream_message
- Cleanup outgoing_file_message/prepare_message
- Fix stream manager API (more to come)"
2016-01-26 13:17:58 +02:00
Asias He
750573ca0c configure: Fix idl indentation 2016-01-26 15:04:45 +08:00
Asias He
2e69d50c0c streaming: Cleanup prepare_message
- Drop empty prepare_message.cc
- Drop #if 0'ed code
2016-01-26 13:14:04 +08:00