Commit Graph

6219 Commits

Author SHA1 Message Date
Paweł Dziepak
ddec2b4d09 batchlog_manager: pass mutations by const ref
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:29 +02:00
Paweł Dziepak
8188896eb7 schema_tables: add missing mutable
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:25 +02:00
Paweł Dziepak
2d0112b11f migration_manager: add missing mutable
If lambda is not marked as mutable objects from its capture list cannot
be moved.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:22 +02:00
Paweł Dziepak
39e5f92433 cql3: modification_statement: avoid copying mutations
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:30:18 +02:00
Paweł Dziepak
11efd5c639 mutation: store mutation data externally
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:29:53 +02:00
Paweł Dziepak
868f5d91df mutation: avoid moving mutation_partition
By passing mutation_partition oither by const ref or rref instead of
by value one move can be avoided if copying is necessary.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-09-03 10:29:32 +02:00
Raphael S. Carvalho
6fe853fe7b sstables: fix possible use-after-free
buf is a stack variable, so it may be destroyed by the time it's
used by output_stream::write().

Spotted while auditing the code.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-09-02 09:26:06 +03:00
Glauber Costa
b1c59ab995 sstable_mutation_test: test condition related to #188
This patch tests that collection within a mutation behave properly.
That is what lead to #188.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-09-02 06:01:39 +03:00
Glauber Costa
0fc2995b54 database: initialize sst field
The reader has a field for the sstable, but we are not initializing it, so it
can be destroyed before we finish our job. It seems to work here, but transposing
this code to the test case crashed it. So this means at some point we will crash
here as well.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-09-02 06:01:38 +03:00
Glauber Costa
268a308d3f sstables::mutation_reader: do not spill previous collection
The sstable mutation_reader knows how to handle the case of multiple
mutations: it wraps the mutation into an optional element, and when a
new mutation comes along, it just moves that to the caller.

However, it fails to do the same for collections. We have the same collection
tracking structure, and as long as we are trying to read a single row, all
works fine.

When we are reading more than a row, though, things change: because we have
stale data for the previous collection, the new one will end up including this.

In the case in which we tested, we had an sstable that contained mutations for
all columns in a row, and another row containing all columns but one, and that
one was precisely a collection. What we ended up seeing, is that the new row
would have that collection nevertheless, with the former row's collection
value.

This patch applies the same method as we use for tracking mutations, to tracking
collections. And with this, it all works.

Fixes #188

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-09-02 03:49:57 +03:00
Tomasz Grabiec
57d41e2155 tests: Don't run lsa_async_eviction_test on very large batch size
Since commit 870e9e5729 eviction no
longer happens during allocation from a region and batch size of 30000
became too large to handle. 3000 is low enough and that test case was
left.
2015-09-01 20:11:17 +03:00
Avi Kivity
ef582753c0 Merge "Lock files for CL+Data dirs" from Calle
"Acquires and maintains lock files during execution.
Lock files are deleted on "clean" exit, and re-taken on uncontended
startup.

Fixes #34 "
2015-09-01 19:18:46 +03:00
Calle Wilund
2fa699896b Main: Use lock files in data dir + CL dir to ensure single instance.
Acquires and maintains lock files during execution. 
Lock files are deleted on "clean" exit, and re-taken on uncontended 
startup.

Fixes #34
2015-09-01 17:50:18 +02:00
Calle Wilund
e3abde50f9 Add lock file helper class "file_lock"
Using posix "lockf" per-fd region locking
2015-09-01 17:50:18 +02:00
Tomasz Grabiec
870e9e5729 lsa: Replace compaction_lock with broader reclaim_lock
Disabling compaction of a region is currently done in order to keep
the references valid. But disabling only compaction is not enough, we
also need to disable eviction, as it also invalidates
references. Rather than introducing another type of lock, compaction
and eviction are controlled together, generalized as "reclaiming"
(hence the reclaim_lock).
2015-09-01 17:29:04 +03:00
Tomasz Grabiec
48569651ea lsa: Fix calculation of bytes.non_lsa_used_space 2015-09-01 17:29:03 +03:00
Avi Kivity
ab2a24486e Merge seastar upstream
* seastar 7a8b261...3e4a237 (1):
  > Merge "net::dpdk: add TCP client support on top of Intel's 40Gbps NICs" from Vlad
2015-09-01 15:23:46 +03:00
Calle Wilund
d95101664d Commitlog: Don't throw exceptions on unrecognized files in CL dir 2015-09-01 14:23:03 +02:00
Avi Kivity
6d1dd76c98 Merge "Scylla RPM fixes" from Pekka
"Fix various issues in the Scylla RPM spec file and support scripts:

  - Include scylla.conf in the RPM

  - Fix tarball generation on Fedora 21

  - Fix scylla user home directory

  - Simplify DPDK build

Scylla RPM now builds and runs on Fedora 21 with DPDK manually disabled
as I don't have matching kernel headers installed on my machine.

Fixes #245."
2015-09-01 15:18:16 +03:00
Tomasz Grabiec
3115a1aaa0 tests: logalloc_test: Disable test_compaction_lock with default allocator
It relies on the fact that the process has a fixed amount of memory
assigned and std::bad_alloc is thrown in a timely manner when it fills
up, which is the case for seastar's allocator, but not with the
default allocator. With the latter the OOM killer kills the process.
2015-09-01 15:17:43 +03:00
Avi Kivity
cc6fa97b35 Revert "transport: improve exception handling on connection close"
This reverts commit 0558a56ab5.  As Vlad points
out, it is bogus.  Real fix in 5ddbaf8612.
2015-09-01 15:16:18 +03:00
Avi Kivity
6ae786ee3f Merge seastar upstream
* seastar 1b3b801...7a8b261 (2):
  > future: assert if promise is inadvertently reused after _state is already destroyed
  > build: don't build dpdk kni modules
2015-09-01 15:15:20 +03:00
Avi Kivity
5ddbaf8612 Merge seastar upstream
* seastar d59c71b...1b3b801 (2):
  > parallel_for_each: fix exception safety
  > xen: fix build
2015-09-01 15:10:25 +03:00
Pekka Enberg
94b80f4dd2 dist/redhat/scylla-server.spec: Simplify DPDK build
Seastar builds DPDK by itself so there's no need to invoke make
ourselves. This ensures that the DPDK configuration we build is correct.

Spotted by Avi.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:41:30 +03:00
Pekka Enberg
eab436b356 dist/redhat/scylla-server.spec: Only build scylla executable
There's no need to build tests as they're not shipped. Speed up RPM
generation by limiting build to the "scylla" executable.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
bcb8c9fe34 dist/redhat: Specify options file absolute path for scripts/scylla_run
Fix Scylla service startup issues on Fedora 21 by specifying an absolute
path for options file.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
e32b5cbafd dist/redhat/scylla-server.spec: Package scylla.yaml
Fix server startup by including scylla.yaml in the generated RPM.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
027cd5860d dist/redhat/scylla-server.spec: Fix "scylla" user home directory
Fix typo in macro usage.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
7f4f271855 dist/redhat/scylla-server.spec: Fix group name typo
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
adf452280f dist/redhat/build_rpm.sh: Force submodules to be initialized and updated
Make sure submodules are initialized and updated when creating a source
tarball.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
3f9fa55688 dist/redhat/build_rpm.sh: Use git-archive-all for creating tarball
Use the newly added git-archive-all helper script for creating the
source tarball. This speeds up RPM building and makes things work on
Fedora 21 that doesn't support the "--exclude-vcs-ignores" command line
option for tar.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Pekka Enberg
f87add31a7 scripts: Add git-archive-all script
We'll use it to create a tarball of the source tree for RPM build.

The script is imported from:

  https://github.com/Kentzo/git-archive-all

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 14:38:24 +03:00
Avi Kivity
a69994051a Merge "Synchronous memory reclamation from LSA" from Tomasz 2015-09-01 12:30:34 +03:00
Avi Kivity
eae8a60930 Merge "storage_service api update" from Asias
"This series wires up more storage_service APIs."
2015-09-01 11:14:13 +03:00
Calle Wilund
1814f89730 Commitlog: Add some more metrics + accessors for json API
Fixes #99

Adding missing commitlog metrics to the rest API.

v2: Mis-send (clumsy fingers)
v3: Use map_reduce0 + subroutine for nicer code
v4: rebased on current master
v5: rebased yet again.

Since the _second_ file in this previous patch set was commited, and is
dependent on this very change below to even compile, some expediency might be
warranted.
2015-09-01 10:15:33 +03:00
Takuya ASADA
16ef2ddc15 dist: add support building AMI
Support build AMI with RAID support.

Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
2015-09-01 10:15:33 +03:00
Avi Kivity
d4060122cf Merge seastar upstream
* seastar 68fee6c...d59c71b (1):
  > Merge dpdk local patches
2015-09-01 09:45:25 +03:00
Pekka Enberg
47b84d9660 locator: Fix minor formatting issues
Fix minor formatting issues introduced in commit
b8211c436b
("locator/abstract_replication_strategy: Add validate_options() helper").

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 09:45:15 +03:00
Calle Wilund
09da053bee Commitlog API: Implement some more metrics accessors
Fixes #99
v2/3: Use map_reduce0 + some abstraction.
2015-09-01 09:42:27 +03:00
Asias He
9de78375f6 api/storage_service: Add get_exceptions 2015-09-01 14:31:44 +08:00
Asias He
325a9c3f09 api/storage_service: Add rebuild 2015-09-01 14:31:44 +08:00
Asias He
4ad352e290 api/storage_service: Add get_load_map 2015-09-01 14:31:44 +08:00
Asias He
5cfdb3cc7b api/storage_service: Add get_load 2015-09-01 14:31:44 +08:00
Asias He
ff72300ddd api/storage_service: Add remove_node 2015-09-01 14:31:44 +08:00
Asias He
c97120aba1 api/storage_service: Add decommission 2015-09-01 14:31:44 +08:00
Asias He
118b370ea8 api/storage_service: Add is_native_transport_running 2015-09-01 14:31:44 +08:00
Asias He
24149e04ce api/storage_service: Add stop_native_transport 2015-09-01 14:31:44 +08:00
Asias He
3889e0568d api/storage_service: Add start_native_transport 2015-09-01 14:31:44 +08:00
Asias He
d0dcaf16fb api/storage_service: Add is_rpc_server_running 2015-09-01 14:31:44 +08:00
Asias He
c2bb9a6cbc api/storage_service: Add start_rpc_server 2015-09-01 14:31:44 +08:00