Commit Graph

8567 Commits

Author SHA1 Message Date
Paweł Dziepak
28fa2a6493 idl-compiler: add serialization callback interface
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:50:29 +00:00
Paweł Dziepak
186061adef mutation_partition: switch serialization to IDL-based one
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:49:08 +00:00
Paweł Dziepak
ccd29bf7a7 frozen_mutation: switch to bytes_ostream
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:54 +00:00
Paweł Dziepak
5127321866 column_mapping: add column_at()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:54 +00:00
Paweł Dziepak
e332f95960 types: make serialize_mutation_form() static
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:42 +00:00
Paweł Dziepak
7586e47004 idl-compiler: avoid copy of basic types
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:42 +00:00
Paweł Dziepak
2ea735f5ed idl-compiler: accept both bytes and bytes_view
bytes can always be trivially converted to bytes_view. Conversion in the
other direction requires a copy.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:42 +00:00
Paweł Dziepak
18b1c66287 idl-compiler: allow auto-generated serializers in writers
This patch allows using both auto-generated serializers or writer-based
serialization for non-stub [[writable]] types.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:21 +00:00
Paweł Dziepak
af2241686f idl-compiler: add reindent() function
This helps having C++ code properly indented both in the compiler source
code and in the auto-generated files.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:20:09 +00:00
Paweł Dziepak
597ed15dfd tests: add idl unit test
Test auto-generated and writer-based serialization as well as
deserialization of simple compound type, vectors and variants.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:19:30 +00:00
Paweł Dziepak
7d1a66d3a0 idl-compiler: move writers and view to *.impl.hh
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:17:00 +00:00
Paweł Dziepak
f1f14631f4 add set_size() overload for bytes_ostream()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:16:55 +00:00
Paweł Dziepak
340d0cccbc serializer: fix duration deserializer
Deserializer is supposed to update input stream.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:11:57 +00:00
Avi Kivity
6330743775 Merge seastar upstream
* seastar 8679033...cf1716f (1):
  > iotune: qualify filesystems for aio
2016-02-18 17:16:46 +02:00
Nadav Har'El
f9ee74f56f repair: options for repairing only a subrange
To implement nodetool's "--start-token"/"--end-token" feature, we need
to be able to repair only *part* of the ranges held by this node.
Our REST API already had a "ranges" option where the tool can list the
specific ranges to repair, but using this interface in the JMX
implementation is inconvenient, because it requires the *Java* code
to be able to intersect the given start/end token range with the actual
ranges held by the repaired node.

A more reasonable approach, which this patch uses, is to add new
"startToken"/"endToken" options to the repair's REST API. What these
options do is is to find the node's token ranges as usual, and only
then *intersect* them with the user-specified token range. The JMX
implementation becomes much simpler (in a separate patch for scylla-jmx)
and the real work is done in the C++ code, where it belongs, not in
Java code.

With the additional scylla-jmx patch to use the new REST API options
provided here, this fixes #917.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <1455807739-25581-1-git-send-email-nyh@scylladb.com>
2016-02-18 17:13:56 +02:00
Raphael S. Carvalho
a53cfc8127 compaction manager: add support to wait for termination of cleanup
'nodetool cleanup' must wait for termination of cleanup, however,
cleanup is handled asynchronously. To solve that, a mechanism is
added here to wait for termination of a cleanup. This mechanism is
about using promise to notificate waiter of cleanup completion.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <6dc0a39170f3f51487fb8858eb443573548d8bce.1455655016.git.raphaelsc@scylladb.com>
2016-02-18 17:01:18 +02:00
Paweł Dziepak
763f6e1dc0 storage_service: don't drain twice
If drain was explicitly requested by user there is no need to do that
again during shutdown.

Fixes segmentation fault when shutting down already drained Scylla.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1455794465-16670-1-git-send-email-pdziepak@scylladb.com>
2016-02-18 14:58:02 +02:00
Avi Kivity
1b49c0ce19 dist: Restart scylla on abnormal termination
Restarting the service can recover from a transient failure.

Fixes #904 (on systemd systems only)
Message-Id: <1455441103-11963-1-git-send-email-avi@scylladb.com>
2016-02-17 21:30:40 +01:00
Avi Kivity
62e96de48a Merge "Adding writers and visitor to IDL" from Amnon
"writers are used to stream objects programmatically and not from objects.
visitors views are used to retrieve information from serialized objects without
deserialize them entirely, but to skip to the position in the buffer with the
relevant information and deserialize only it."
2016-02-17 22:06:46 +02:00
Amnon Heiman
fbc6770837 idl-compiler: Verify member type
This patch adds static assert to the generated code that verify that a
declare type in the idl matches the parameter type.

Accepted type ignores reference and const when comparison.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 21:44:53 +02:00
Amnon Heiman
38cd55e9cf Adding the mutation idl
This adds the mutation definition idl and add it to the compilation.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 18:42:09 +02:00
Amnon Heiman
714d4927d6 idl-compiler: add writers and view to classes.
This patch adds a writer object to classes in the idl.

It adds attribute support to classes. Writer will be created for classes
that are marked as writable.

For the writers, the code generator creates two kind of struct.
states, that holds the write state (manly the place holders for all
current objects, and vectors) and nodes that represent the current
position in the writing state machine.

To write an object create a writer:
For example creating a writer for mutation, if out is a bytes_ostream

writer_of_mutation w(out);

Views are used to read from buffer without deserialize an entire
object.
This patch adds view creation to the idl-compiler. For each view a
read_size function is created that will be used when skipping through
buffers.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 18:42:09 +02:00
Amnon Heiman
33d5c95b90 serialization_visitors: Add skip template
The skip template function is used when skipping data types.
By default is uses a deserializer to calculate the size.

A specific implementation save unneeded deserialization. For fix sized
object the skip function would become an expression allowing the
compiler to drop the function altogether.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 18:42:09 +02:00
Amnon Heiman
64c097422d Adding the serialization_visitors.hh file
The serialization_visitors.hh contains helper classes for the readers and writers
visitors class.

place_holder is a wrapper around bytes_stream place holder.
frame is used to store size in bytes.
empty_frame is used with final object (that do not store their size)
from the code that uses it, it looks the same, but in practice it does
not store any data.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 18:42:08 +02:00
Amnon Heiman
ca72d637f9 bytes_ostream: Allow place holder return a stream
Reader and writer can use the bytes_ostream as a raw bytes stream,
handling the bytes encoding and streaming on their own.

To fully support this functionality, place holder should support it as
well.

This patch adds a get_stream method that return a simple_output_stream
writer can use it using their own serialization function.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 18:42:04 +02:00
Tomasz Grabiec
eedc1548e7 transport: server: Fix typo
Spotted-by: Vitaly Davidovich <vitalyd@gmail.com>
Signed-off-by: Tomasz Grabiec <tgrabiec@scylladb.com>
2016-02-17 15:55:10 +01:00
Avi Kivity
0363a0efe6 Merge "Fix limits handling in CQL server" from Tomasz
"Fixes the following issues:
 #807 Wrong maximum key length
 #809 Scylla assert on returning result when max column size is overflow"
2016-02-17 15:06:51 +02:00
Tomasz Grabiec
6e7bac14b3 transport: server: Throw instead of abort on bounds check failures
Instead of crashing the server we will respond with a "Server Error"
to the requestor.

Fixes #809.
2016-02-17 13:12:11 +01:00
Tomasz Grabiec
0ff0c5555a transport: server: 'short' should be unsigned
According to CQL binary protocol v3 [1], "short" fields are unsigned:

   [short]        A 2 bytes unsigned integer

[1] https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=doc/native_protocol_v3.spec

C* code agrees as well.

Fixes #807.
2016-02-17 13:12:11 +01:00
Tomasz Grabiec
9375b7df7b transport: server: Size check should allow max value 2016-02-17 13:12:11 +01:00
Tomasz Grabiec
48e9d67525 compound: Extract size_type alias 2016-02-17 13:12:11 +01:00
Amnon Heiman
719b8e1e4d serializer: Add boost::variant, chrono::time_point and unknown variant
This patch adds a stub support for boost::variant. Currently variant are
not serialized, this is added just so non stub classes will be able to
compile.

deserialize for chrono::time_point and deserializer for chrono::duration

unknown variant:
Planning for situations where variant could be expanded, there may be
situation that a variant will return an unknown value.

In those cases the data and index will be paseed to the reader, that
can decide what to do with it.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-02-17 11:43:50 +02:00
Avi Kivity
69183be6f0 Update scylla-ami submodule
* dist/ami/files/scylla-ami b3b85be...398b1aa (3):
  > Import AMI initialization code from scylla-server repo
  > Use long options on scylla_raid_setup and scylla_sysconfig_setup
  > Wait more longer to finishing AMI setup
2016-02-17 10:45:50 +02:00
Takuya ASADA
e640b42081 dist: On Ubuntu, log coredump before creating the core
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1455664279-32157-1-git-send-email-syuu@scylladb.com>
2016-02-17 10:43:56 +02:00
Avi Kivity
00f40881d6 Merge "Interactive scylla_setup and refactoring setup scripts" from Takuya 2016-02-17 10:42:26 +02:00
Takuya ASADA
553c2ca523 dist: call scylla_install_ami directly from scylla.json
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
2d429e602a dist: interactive scylla_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
b9ae4ff272 dist: delete build_ami_local.sh, merge it to build_ami.sh
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
6b93505952 dist: long options for build_rpm.sh
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
d754bbe122 dist: add --unstable on build_ami.sh
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
70f397911b dist: long options for build_ami.sh
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
a860e4baae dist: remove AMI initialization code from scylla_setup, move to scylla-ami
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
08038e3f42 dist: long options for scylla_install_pkg
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
d7a03676e3 dist: don't ignore posix_net_conf.sh error
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
684447d3ab dist: long options for scylla_sysconfig_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
7861871c1b dist: long options for scylla_raid_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
889c4706fc dist: long options for scylla_ntp_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
2804e394a6 dist: long options for scylla_coredump_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
c12d95afe0 dist: add options to skip running setups on scylla_setup, support long options
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:05 +09:00
Takuya ASADA
dc9012d5a4 dist: move selinux setup code to scylla_selinux_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:33:01 +09:00