Commit Graph

7631 Commits

Author SHA1 Message Date
Raphael S. Carvalho
ac1a67c8bc sstables: move compaction_stats to header file
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:49:45 -02:00
Avi Kivity
a2eac711cf Merge "compaction history support" from Raphael
"This patchset will make Scylla update the system table
COMPACTION_HISTORY whenever a compaction job finishes.
Functions were added to both update and retrieve the
content of this system table. Compaction history API
is also enabled in this series."
2015-12-15 13:22:14 +02:00
Raphael S. Carvalho
87fbe29cf9 api: add support to compaction history
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-15 09:00:21 -02:00
Takuya ASADA
9c5afb8e58 dist: add scylla-gdb.py on scylla-server-debuginfo rpm package
It will place at /usr/src/debug/scylla-server-development/scylla-gdb.py
Fixes #604

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-15 12:13:17 +02:00
Raphael S. Carvalho
a2fb0ec9a3 sstables: update compaction history at the end of compaction
When compaction job finishes, call function to update the system
table COMPACTION_HISTORY. That's also needed for the compaction
history API.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-14 14:20:03 -02:00
Raphael S. Carvalho
433ed60ca3 db: add method to get compaction history
This method is intended to return content of the system table
COMPACTION_HISTORY as a vector of compaction_history_entry.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-14 14:19:04 -02:00
Raphael S. Carvalho
f3beacac28 db: add method to update the system table COMPACTION_HISTORY
It's supposed to be called at the end of compaction.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
2015-12-14 13:47:10 -02:00
Tomasz Grabiec
0865ecde17 storage_proxy: Fix range splitting
There is a check whose intent was to detect wrap around during walk of
the ring tokens by comparing the split point with minimum token, which
is supposed to be inserted by the ring iterator. It assumed that when
we encounter it, the range is a wrap around. It doesn't hold when
minimum token is part of the token metadata or set of tokens is empty.

In such case, a full range would be split into 3 overlapping full
ranges. The fix is to drop the assumption and instead ensure that
ranges do not wrap around by unwrapping them if necessary.

Fixes #655.
2015-12-14 16:05:54 +02:00
Takuya ASADA
3b7693feda dist: add package dependency to gnutls library
Now Seastar depends to gnutls, we need to add it on .rpm/.deb package dependency.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2015-12-14 13:28:28 +02:00
Pekka Enberg
ba09c545fc dist/docker: Enable SMP support
Now that Scylla has a sleep mode, we can enable SMP support again.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-12-14 13:23:30 +02:00
Avi Kivity
fd14cb3743 mutation_partition: fix leak in move assignment operator
The default move assignment operator calls boost::intrusive::set's move
assignment operator, which leaks, because it does not believe it owns
the data.

Fix by providing a custom implementation.
2015-12-14 10:33:19 +01:00
Asias He
9781e0d34d storage_service: Make bootstrapping/leaving/moving log more consistent
It is useful for test code to grep the log.
2015-12-11 13:57:40 +02:00
Tomasz Grabiec
1991fd5ca2 Merge branch 'pdziepak/fix-clustering-key-comparison/v2' fom seastar-dev.git
From Paweł:

This series fixes comparison of byte order comparable clustering keys.

Fixes #645.
2015-12-11 12:51:02 +01:00
Paweł Dziepak
3a73496817 tests/cql: add test for ordering clustering keys
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 12:05:25 +01:00
Paweł Dziepak
8cab343895 compound: fix compare() of prefixable types
All components of prefixable compound type are preceeded by their
length what makes them not byte order comparable.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 12:04:31 +01:00
Paweł Dziepak
8fd4b9f911 schema: remove _clustering_key_prefix_type
All clustering keys are now prefixable.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 10:47:24 +01:00
Paweł Dziepak
bb9a71f70c thrift: let class_from_compound_type() accept prefixable types
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-11 10:45:56 +01:00
Pekka Enberg
0d8a02453e types: Fix frozen collection type names
Frozen collection type names must be wrapped in FrozenType so that we
are able to store the types correctly in system tables.

This fixes #646 and fixes #580.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-12-11 10:41:11 +01:00
Pekka Enberg
63bdeb65f2 cql3: Implement maps::literal::test_assignment() function
The test_assignement() function is invoked via the Cassandra unit tests
so we might as well implement it.

Signed-off-by: Pekka Enberg <penberg@scylladb.com>
2015-12-11 09:35:13 +01:00
Asias He
57ee9676c2 storage_service: Fix default ring_delay time
It is 30 seconds instead of 5 seconds by default. To align with c*.

Pleas note, after this a node will takes at least 30 seconds to complete
a bootstrap.
2015-12-11 09:05:19 +02:00
Avi Kivity
b3cd672d97 Merge seastar upstream
* seastar ad07a2e...5b9e3da (2):
  > Merge "rpc cleanups and improvements" from Gleb
  > shared_future: Add missing include
2015-12-10 18:11:59 +02:00
Paweł Dziepak
9d482532f4 tests/lsa: reduce the size of large allocation
Originally, large allocation test case attempted to allocate an object
as big as halft of the space used by the lsa. That failed when the test
was executed with lower amount of memory available mainly due to the
memory fragmentation caused by previous test cases.

This patches reduces the size of the large allocation to 3/8 of the
total space used by the lsa which is still a lot but seems to make the
test pass even with as little memory as 64MB per shard.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 13:16:43 +01:00
Avi Kivity
d425aacaeb release: copy version string into heap
If we get a core dump from a user, it is important to be able to
identify its version.  Copy the release string into the heap (which is
copied into the code dump), so we can search for it using the "strings"
or "ident" commands.

Reviewed-by: Nadav Har'El <nyh@scylladb.com>
2015-12-10 13:12:40 +02:00
Lucas Meneghel Rodrigues
2167173251 utils/logalloc.cc - Declare member minimum_size from segment_zone struct
This fixes compile error:

In function `logalloc::segment_zone::segment_zone()':
/home/lmr/Code/scylla/utils/logalloc.cc:412: undefined reference to `logalloc::segment_zone::minimum_size'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
2015-12-10 12:54:34 +02:00
Asias He
b7d10b710e streaming: Propagate fail to send PREPARE_DONE_MESSAGE exception
Otherwise the stream_plan will not be marked as failed state.
2015-12-10 12:38:00 +02:00
Paweł Dziepak
ec453c5037 managed_bytes: fix potentially unaligned accesses
blob_storage defined with attribute packed which makes its alignment
requirement equal 1. This means that its members may be unaligned.
GCC is obviously aware of that and will generate appropriate code
(and not generate ubsan checks). However, there are few places where
members of blob_storage are accessed via pointers, these have to be
wrapped by unaligned_cast<> to let the compiler know that the location
pointed to may be not aligned properly.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 11:59:54 +02:00
Tomasz Grabiec
43498b3158 Merge branch 'pdziepak/fix-partial-clustering-keys/v1' from seastar-dev.git
Form Paweł:

This series fixes support for clustering keys which trailing components
are null. The solution is to use clustering_key_prefix instead of
clustering_key everywhere.

Fixes #515.
2015-12-10 10:43:12 +01:00
Paweł Dziepak
66ff1421f0 tests/cql: add test for clustering keys with empty components
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:47:07 +01:00
Paweł Dziepak
64f50a4f40 db: make clustering_key a prefix
Schemas using compact storage can have clustering keys with the trailing
components not set and effectively being a clustering key prefixes
instead of full clustering keys.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:47 +01:00
Paweł Dziepak
77c7ed6cc5 keys: add prefix_equality_less_compare for prefixes
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
220a3b23c0 keys: allow creating partial views of prefixes
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
3c16ab080a sstables: do not assume clustering_key has the proper format
In case of non-compound dense tables the column name is just the value
of the clustering key (which has only one component). Current code just
casts clustering_key to bytes_view which works because there is no
additional metadata in single element clustering keys.
However, that may change when the internal representation of clustering
key is changed so explicitly extract the proper component.

This change will become necessary when clustering_key is replaced by
clustering_key_prefix.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
5f1e9fd88f mutation_partition: remove unused find_entry()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Paweł Dziepak
3287022000 cql3: do not assume that clustering key is full
In case of schemas that use compact storage it is possible that trailing
components of clustering keys are not set.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-10 05:46:26 +01:00
Avi Kivity
167addbfe1 main: remove issue #417 (poll mode) warning
Fixed.
2015-12-09 19:00:32 +02:00
Avi Kivity
a352d63bf9 Merge seastar upstream
* seastar c5e595b...ad07a2e (1):
  > reactor: add command line option to disable sleep mode

Fixes #417
2015-12-09 19:00:20 +02:00
Glauber Costa
3c988e8240 perf_sstable: use current scylla default directory
When this tool was written, we were still using /var/lib/cassandra as a default
location. We should update it.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
2015-12-09 17:46:31 +02:00
Avi Kivity
01c3670def Merge seastar upstream
* seastar 5dc22fa...c5e595b (3):
  > memory: be less strict about NUMA bindings
  > reactor: let the resource code specify the default memory reserve
  > resource: reserve even more memory when hwloc is compiled in

Fixes #642
2015-12-09 16:47:47 +02:00
Asias He
66938ac129 streaming: Add retransmit logic for streaming verbs
Retransmit streaming related verbs and give up in 5 minutes.

Tested with:

  lein test :only cassandra.batch-test/batch-halves-decommission

Fixes #568.
2015-12-09 15:12:36 +02:00
Avi Kivity
14794af260 Merge seastar upstream
* seastar 9f9182e...5dc22fa (1):
  > future: add repeat_until_value(): repeat an action until it returns a value
2015-12-09 15:11:59 +02:00
Avi Kivity
213700e42f Merge seastar upstream
* seastar d40453b...9f9182e (5):
  > Merge "Sleep mode support"
  > future: add futurize<T>::from_tuple(tuple<T>)
  > tls: Add missing destructor for dh_params::impl, fixes ASAN error
  > tls/socket fix: Add missing noexcept to constructor/move
  > Merge "Initial SSL/TLS socket support" from Calle
2015-12-09 11:01:13 +02:00
Avi Kivity
204610ac61 Merge "Make LSA more large-allocation-friendly" from Paweł
"This series attempts to make LSA more friendly for large (i.e. bigger
than LSA segment) allocations. It is achieved by introducing segment
zones – large, contiguous areas of segments and using them to allocate
segments instead of calling malloc() directly.
Zones can be shrunk when needed to reclaim memory and segments can be
migrated either to reduce number of zone or to defragment one in order
to be able to shrink it. LSA tries to keep all segments at the lower
addresses and reclaims memory starting from the zones in the highest
parts of the address space."
2015-12-09 10:49:23 +02:00
Avi Kivity
883074e936 Merge "Fix replace_node support" from Asias
Also:

[PATCH scylla v1 0/7] gossip mark node down fix + cleanup
[PATCH scylla v1 0/2] Refuse decommissioned node to rejoin
[PATCH scylla] storage_service: Fix added node not showing up in nodetool in status joining
2015-12-09 10:42:52 +02:00
Paweł Dziepak
8ba66bb75d managed_bytes: fix copy size in move constructor
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2015-12-09 10:38:28 +02:00
Asias He
b63d49c773 storage_service: Log removing replaced endpoint from system.peers
This info is important when replacing a node. Useful for debugging.
2015-12-09 12:30:52 +08:00
Asias He
d26c7e671d storage_service: Enable commented out code in handle_state_normal
Add current_owner to endpoints_to_remove if endpoint and current_owner
have the same token and endpoint is newer than current_owner.
2015-12-09 12:30:52 +08:00
Asias He
3793bb7be1 token_metadata: Add get_endpoint_to_token_map_for_reading 2015-12-09 12:30:52 +08:00
Asias He
1cc7887ffb token_metadata: Do nothing if tokens is empty.
When replacing a node, we might ignore the tokens so that the tokens is
empty. In this case, we will have

   std::unordered_map<inet_address, std::unordered_set<token>> = {ip, {}}

passed to token_metadata::update_normal_tokens(std::unordered_map<inet_address,
std::unordered_set<token>>& endpoint_tokens)

and hit the assert

   assert(!tokens.empty());
2015-12-09 12:30:52 +08:00
Asias He
e79c85964f system_keyspace: Flush system.peers in remove_endpoint
1) Start node 1, node 2, node 3
2) Stop  node 3
3) Start node 4 to replace node 3
4) Kill  node 4 (removal of node 3 in system.peers is not flushed to disk)
5) Start node 4 (will load node 3's token and host_id info in bootup)

This makes

   "Token .* changing ownership from 127.0.0.3 to 127.0.0.4"

messages printed again in step 5) which are not expected, which fails the dtest

   FAIL: replace_first_boot_test (replace_address_test.TestReplaceAddress)
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "scylla-dtest/replace_address_test.py",
   line 220, in replace_first_boot_test
       self.assertEqual(len(movedTokensList), numNodes)
   AssertionError: 512 != 256
2015-12-09 12:30:52 +08:00
Asias He
110a18987e token_metadata: Print Token changing ownership from
Needed by test.
2015-12-09 12:30:52 +08:00