Commit Graph

13112 Commits

Author SHA1 Message Date
Tomasz Grabiec
fa2c26342c row_cache: Handle eviction in partition reader 2017-09-13 17:38:08 +02:00
Tomasz Grabiec
99aa3d1964 tests: row_cache_test: Don't assume mvcc snapshots are not evictable
The test was not updating the underlying mutation source but still
expecting to get the right data after calling invalidate().  If
snapshots are evictable, that's not guaranteed. Apply to underlying as
well, so data is read from underlying if necessary.
2017-09-13 17:38:08 +02:00
Tomasz Grabiec
adb159d51b row_cache: Reuse allocation_strategy::invalidate_references()
Modification count in the tracker is redundant, we can rely on
allocator's invalidation counter.
2017-09-13 17:38:08 +02:00
Tomasz Grabiec
27a3b4bca9 row_cache: Don't invalidate references on insertion
modification_count is currently only used to detect invalidation of
references, intended to be incremented on erasure.

Insertion into intrusive set doesn't invalidate references, so no
need to increment the counter.
2017-09-13 17:38:08 +02:00
Tomasz Grabiec
87be474c19 lsa: Move reclaim counter concept to allocation_strategy level
So that generic code can detect invalidation of references. Also, to
allow reusing the same mechanism for signalling external reference
invalidation.
2017-09-13 17:38:08 +02:00
Tomasz Grabiec
4053c801e2 mvcc: Ensure partition_snapshot always destroys versions using proper allocator
partition_snapshot is managed by lw_shared_ptr. Currently it is
assumed that before it dies, maybe_merge_versions() is called on it,
which destroyes it in the right allocator context. It's not very
safe. This patch improves safety by using the right allocator in
snapshot's destructor.
2017-09-13 17:38:08 +02:00
Tomasz Grabiec
cda86abdbc mvcc: Encapsulate reference stability check in partition_snapshot 2017-09-13 17:38:08 +02:00
Tomasz Grabiec
2df6f356b1 mvcc: Store LSA region reference in partition_snapshot
Will be useful for improving encapsulation.
2017-09-13 17:38:08 +02:00
Tomasz Grabiec
4c920c9891 tests: cql_test_env: Use cancel_prior_atomic_deletions()
This fixes a failure in view_schema_test, which starts many instances
of single_node_cql_env. cancel_atomic_deletions() causes later
deletions to fail, which causes some of the test cases to fail.
Message-Id: <1505311250-3118-2-git-send-email-tgrabiec@scylladb.com>
2017-09-13 17:11:34 +03:00
Tomasz Grabiec
dc0860ac70 sstables: Introduce cancel_prior_atomic_deletions()
Like cancel_atomic_deletions() but doesn't fail later deletions.
Message-Id: <1505311250-3118-1-git-send-email-tgrabiec@scylladb.com>
2017-09-13 17:11:33 +03:00
Tomasz Grabiec
8a425cedc6 tests: cql_test_env: Cancel pending sstable deletions on shutdown
Fixes a hang on shutdown with --smp 2 in perf_fast_forward. The hang
is in sstables::await_background_jobs_on_all_shards(), which is
waiting on sstable deletions. Not all shards agree to delete certain
sstables, because e.g. not all shards decide to compact them
yet. Cancel those deletes after database is stopped on all shards,
like we do in main.cc

Fixes #2796.

Message-Id: <1505292239-26032-1-git-send-email-tgrabiec@scylladb.com>
2017-09-13 11:56:48 +03:00
Asias He
c84dcabb8f gossip: Use boost::copy_range in apply_state_locally
boost::copy_range is better because the vector is allocated with the
correct size instead of growing when the inserter is called.

[avi: also crashes less]

Message-Id: <b19ca92d56ad070fca1e848daa67c00c024e3a4d.1505291199.git.asias@scylladb.com>
2017-09-13 11:33:15 +03:00
Tomasz Grabiec
b3a8ba5af6 gdb: Introduce "scylla find" command
Finds live objects on seastar heap of current shard which contain
given value. Prints results in 'scylla ptr' format.

Example:

  (gdb) scylla find 0x600005321900
  thread 1, small (size <= 512), live (0x6000000f3800 +48)
  thread 1, small (size <= 56), live (0x6000008a1230 +32)

Message-Id: <1505284614-19577-1-git-send-email-tgrabiec@scylladb.com>
2017-09-13 11:22:23 +03:00
Asias He
fa9d47c7f3 gossip: Fix a log message typo in compare_endpoint_startup
Message-Id: <c4958950e1108082b63e08ab81ee2177edc9b232.1505286843.git.asias@scylladb.com>
2017-09-13 09:54:56 +02:00
Glauber Costa
ecad1be161 compaction_strategy: add missing header
compaction_strategy.hh throws an exception, but it doesn't add the
exception header. It is working in-tree because of inclusion order,
but it broke one of my yet-out-of-tree changes.

In any case, it is best to add the headers we will need to the files,
and that is what this patch does.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20170912233326.26114-1-glauber@scylladb.com>
2017-09-13 08:40:15 +02:00
Raphael S. Carvalho
ef18b1162b sstables/compaction_manager: rename and better explain reshard function
submit doesn't properly describe the function and also improve explanation
of the relationship between function itself and its job parameter.

Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20170912032034.23043-1-raphaelsc@scylladb.com>
2017-09-12 12:25:17 +03:00
Avi Kivity
1bd207a306 sstables: merge filter.cc into sstables.cc
filter.cc has just two smallish functions, which are part of the sstable
class. Move them to sstables.cc where the rest of the class members are defined.
Message-Id: <20170912080541.7836-1-avi@scylladb.com>
2017-09-12 10:06:52 +02:00
Tomasz Grabiec
423142ec81 tests: row_cache_test: Fix abort in debug mode
The test used apply() variant which assumed that it was invoked in a
seastar thread, which is no longer the case after commit d22fdf4. Fix
by copying outisde cache update, and use non-deferring apply() variant
for cache update.

Message-Id: <1505200142-3650-1-git-send-email-tgrabiec@scylladb.com>
2017-09-12 10:57:36 +03:00
Tomasz Grabiec
3f527e028d Merge "Reduce dependencies on sstables.hh" from Avi
This patchset reduces includes of sstables.hh, reducing compile time
by both reducing the amount of code compiled, and the amount of
needless recompiles caused by false dependencies.  It does so by
replacing lw_shared_ptr<sstable>, which requires a complete class,
with a new custom type shared_sstable, which allows an incomplete
sstable class definition.

* https://github.com/avikivity/scylla deps2/v2.1
  database: change truncate() to flush while compaction is disabled
  database: make run_with_compaction_disabled() a non-template
  database: add indirection to compaction_manager instance
  database: remove dependency on compaction.hh and compaction_manager.hh
  size_estimates_virtual_reader.hh: add missing include
  system_keyspace: add missing include
  main: add missing include
  storage_service: add missing include
  repair: add missing include
  compaction.hh: add missig include and forward declaration
  compaction_manager: add missing include
  shared_index_lists.hh: add missing include
  perf_fast_forward: add missing include
  sstable_mutation_test: add missing include
  sstables: extract version and format enum into a separate header file
  database.hh: add missing forward declaration for
    foreign_sstable_open_info
  cql_test_env: add forward declaration
  database: make column_family::disable_sstable_write() out-of-line
  sstables: introduce make_sstable() as a shortcut for
    make_lw_shared<sstable>
  treewide: use shared_sstable, make_sstable in place of
    lw_shared_ptr<sstable>
  sstables: use support for lw_shared_ptr with incomplete type for
    shared_sstable
  sstables: reduce dependencies
  streaming: remove unneeded includes
2017-09-12 09:56:46 +02:00
Tomasz Grabiec
ee1e7732a6 database: Create tables with continuous cache
When table is created, it doesn't contain any data, so we can mark the whole
data range as continuous in cache. This way reads will immediately hit, and
flushes will populate. If sstables are later attached, the attaching process
is supposed to invalidate affected ranges (and it does).

Fixes #2536.

Message-Id: <1505200269-4031-1-git-send-email-tgrabiec@scylladb.com>
2017-09-12 10:53:07 +03:00
Avi Kivity
85a6a2b3cb streaming: remove unneeded includes 2017-09-12 10:43:39 +03:00
Avi Kivity
578bf55371 sstables: reduce dependencies
Use shared_sstable.hh instead of sstables.hh.
2017-09-12 10:43:36 +03:00
Avi Kivity
07feaf9c4c sstables: use support for lw_shared_ptr with incomplete type for shared_sstable
Use the lw_shared_ptr deleter support to define shared_sstable without
pulling the definition of class sstable, reducing compile time and
dependencies if only shared_sstable is needed.
2017-09-12 10:43:05 +03:00
Avi Kivity
f7023501d6 treewide: use shared_sstable, make_sstable in place of lw_shared_ptr<sstable>
Since shared_sstable is going to be its own type soon, we can't use the old alias.
2017-09-12 10:43:05 +03:00
Avi Kivity
1a3cdffbc1 sstables: introduce make_sstable() as a shortcut for make_lw_shared<sstable>
shared_sstable will soon not be an alias for lw_shared_ptr<sstable>, so we need
another factory function.
2017-09-12 10:43:05 +03:00
Avi Kivity
88b91c84a1 database: make column_family::disable_sstable_write() out-of-line
Reduces dependencies.
2017-09-12 10:43:05 +03:00
Avi Kivity
02028df9b1 cql_test_env: add forward declaration
Not worthwhile to add a new #include for this.
2017-09-12 10:43:05 +03:00
Avi Kivity
02e5bf1c20 database.hh: add missing forward declaration for foreign_sstable_open_info
Supplied by an incidental include now, but it will be gone soon.
2017-09-12 10:43:05 +03:00
Avi Kivity
c4bafd912c sstables: extract version and format enum into a separate header file
This allows removing a dependency on sstables.hh later on.
2017-09-12 10:43:05 +03:00
Avi Kivity
5ebb15b9d4 sstable_mutation_test: add missing include 2017-09-12 10:43:05 +03:00
Avi Kivity
fdab47ab32 perf_fast_forward: add missing include 2017-09-12 10:43:05 +03:00
Avi Kivity
ca2d0b4efb shared_index_lists.hh: add missing include 2017-09-12 10:43:05 +03:00
Avi Kivity
eb62b2c00d compaction_manager: add missing include 2017-09-12 10:43:05 +03:00
Avi Kivity
0efa444a56 compaction.hh: add missing includes 2017-09-12 10:42:45 +03:00
Avi Kivity
7ca029c8f1 database_fwd.hh: add column_family forward declaration 2017-09-12 10:41:28 +03:00
Avi Kivity
4751402709 build: disable -fsanitize-address-use-after-scope on CqlParser.o
The parser generator somehow confuses the use-after-scope sanitizer, causing it
to use large amounts of stack space. Disable that sanitizer on that file.
Message-Id: <20170905110628.18047-1-avi@scylladb.com>
2017-09-11 19:42:26 +02:00
Avi Kivity
43a72254ff repair: add missing include 2017-09-11 20:09:45 +03:00
Avi Kivity
aebab377d9 storage_service: add missing include 2017-09-11 20:09:45 +03:00
Avi Kivity
a3b8089bd4 main: add missing include 2017-09-11 20:09:45 +03:00
Avi Kivity
0aaefe665b system_keyspace: add missing include 2017-09-11 20:09:45 +03:00
Avi Kivity
d3cde2e2be size_estimates_virtual_reader.hh: add missing include 2017-09-11 20:09:45 +03:00
Avi Kivity
9b540eccb0 database: remove dependency on compaction.hh and compaction_manager.hh 2017-09-11 20:09:45 +03:00
Avi Kivity
f9c8c1ddc2 database: add indirection to compaction_manager instance
Allows making it forward-declared later on, reducing dependencies.
2017-09-11 20:09:45 +03:00
Avi Kivity
9d0aaa941a database: make run_with_compaction_disabled() a non-template
Allows reducing dependencies down the line, and un-templating
non-performance-critical functions is a good thing.
2017-09-11 20:09:45 +03:00
Avi Kivity
6b5514a3df database: change truncate() to flush while compaction is disabled
In preparation to make run_with_compaction_disabled() a non-template,
we want to remove any non-copyable captures (so the function can be
an std::function, which requires copyability). Move the flush within
the compaction disabled region. This changes the behavior, but it shouldn't
matter.
2017-09-11 20:09:45 +03:00
Avi Kivity
14fd4168dc Merge seastar upstream
* seastar 31b925d...92fdce2 (3):
  > shared_ptr: allow incomplete classes in lw_shared_ptr<>
  > Update DPDK to 17.05
  > future: pass func as mutable to lambda arg of handle_exception[_type]
2017-09-11 20:09:04 +03:00
Tomasz Grabiec
95b3eaac97 debug: Allow running scylla_row_cache_report.stp script against a running process
Message-Id: <1504776359-16424-1-git-send-email-tgrabiec@scylladb.com>
2017-09-11 14:17:30 +03:00
Avi Kivity
fe019ad84d Merge "Refuse to load non-Scylla counter sstables" from Paweł
"These patches make Scylla refuse to load counter sstables that may
contain unsupported counter shards. They are recognised by the lack of
the Scylla component.

Fixes #2766."

* tag 'reject-non-scylla-counter-sstables/v1' of https://github.com/pdziepak/scylla:
  db: reject non-Scylla counter sstables in flush_upload_dir
  db: disallow loading non-Scylla counter sstables
  sstable: add has_scylla_component()
2017-09-11 13:28:44 +03:00
Tzach Livyatan
83eab5c8d7 Remove comment about Too high number of concurrent compactions from scylla_compaction_manager_compactions help
It should never happen and its not clear what too high stands for

Signed-off-by: Tzach Livyatan <tzach@scylladb.com>
Message-Id: <20170911085645.21222-1-tzach@scylladb.com>
2017-09-11 13:27:35 +03:00
Gleb Natapov
d0d8bdf615 storage_proxy: remove unused parameter from get_restricted_ranges() function
Message-Id: <20170911084653.GH24167@scylladb.com>
2017-09-11 11:58:44 +02:00