If schema merging completes at lower rate than incoming pull requests,
then merge processes will accumulate and needlessly request and hold schema mutations.
In rare cases, when there are constant schema changes, they may even
overflow memory. This was seen in dtest:
concurrent_schema_changes_test.py:TestConcurrentSchemaChanges.create_lots_of_schema_churn_test
Allowing only one active and one queued pull request per remote
endpoint is enough.
When we install scylla metapackage with version (ex: scylla-1.7.1),
it just always install newest scylla-server/-jmx/-tools on the repo,
instead of installing specified version of packages.
To install same version packages with the metapackage, limited dependencies to
current package version.
Fixes#2642
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20170726193321.7399-1-syuu@scylladb.com>
Commitlog replay fails when upgrade from <1.7.3 to 2.0, we need to refuse
updating package if current scylla < 1.7.3 && commitlog remains.
Note: We have the problem on scylla-server package, but to prevent
scylla-conf package upgrade, %pretrans should be define on scylla-conf.
Fixes#2551
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20170727110730.613-1-syuu@scylladb.com>
database::make_sstable_reader() creates a reader which will need to
obtain a semaphore permit when invoked. Therefore, each read may
create at most one such reader in order to be guaranteed to make
progress. If the reader tries to create another reader, that may
deadlock (or for non-system tables, timeout), if enough number of such
readers tries to do the same thing at the same time.
Avoid the problem by dropping previous reader before creating a new
one.
Refs #2644.
Message-Id: <1501152454-4866-1-git-send-email-tgrabiec@scylladb.com>
Initialize the system_auth and system_traces keyspaces and their tables after
the Node joins the token ring because as a part of system_auth initialization
there are going to be issues SELECT and possible INSERT CQL statements.
This patch effectively reverts the d3b8b67 patch and brings the initialization order
to how it was before that patch.
Fixes#2273
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1500417217-16677-1-git-send-email-vladz@scylladb.com>
These patches contain some minor fixes for performance regression reported
by perf_fast_forward after partial cache was merged. The solution is still
far from perfect, there is one case that still has 30% degradation, but
there is some improvement so there is no reason to hold these changes back.
Refs #2582.
Some numbers:
before - before cache changes were merged
(555621b537)
cache - at the commit that introduced the partial cache
(9b21a9bfb6)
after - recent master + this series
(based on e988121dbb)
Differences are shown relative to "before".
Testing effectiveness of caching of large partition, single-key slicing reads:
Large partitions, range [0, 500000], populating cache
before cache after
1636840 1013688 1234606
-38% -25%
Large partitions, range [0, 500000], reading from cache
before cache after
2012615 3076812 3035423
+53% +51%
Testing scanning small partitions with skips.
reading small partitions (skip 0)
before cache after
227060 165261 200639
-27% -11%
skipping small partitions (skip 1)
before cache after
29813 27312 38210
-8% +28%
Testing slicing small partitions:
slicing small partitions (offset 0, read 4096)
before cache after
195282 149695 180497
-23% -8%
* https://github.com/pdziepak/scylla.git perf_fast_forward-regression/v3:
sstables: make sure that fill_buffer() actually fills buffer
mutation_merger: improve handling of non-deferring fill_buffer()s
partition_snapshot_row_cursor: avoid apply() in single-version cases
sstables: introduce decorated_key_view
ring_position_comparator: accept sstables::decorated_key_view
sstable: keep a pre-computed token in summary_entry
sstables: cache token in index entries
index_reader: advance_and_check_if_present() use index_comparator
ring_position_comparator: drop unused overloads
cache_streamed_mutation: avoid moving clustering_row
streamed_mutation: introduce consume_mutation_fragments_until()
cache_streamed_mutation: use consumer based read_context reader
rows_entry: make position() inlineable
mutation_fragment: make destructor always_inline
keys: introduce compound_wrapper::from_exploded_view()
sstables: avoid copying key components
compound_compat: explode: reserve some elements in a vector
cache: short-circut static row logic if there are no static columns
cache: use equality comparators instead of tri_compare
sstables: avoid indirect calls to abstract_type::is_multi_cell()
loading_cache invokes a timer that may issue asynchronous operations
(queries) that would end with writing into the internal fields.
We have to ensure that these operations are over before we can destroy
the loading_cache object.
Fixes#2624
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1501096256-10949-1-git-send-email-vladz@scylladb.com>
We need to consider the _live_endpoints size. The nr_live_nodes should
not be larger than _live_endpoints size, otherwise the loop to collect
the live node can run forever.
It is a regression introduced in commit 437899909d
(gossip: Talk to more live nodes in each gossip round).
Fixes#2637
Message-Id: <863ec3890647038ae1dfcffc73dde0163e29db20.1501026478.git.asias@scylladb.com>
Equality comparator may be much cheaper than the fully fledged
trichotomic comparator, especially if the component types are byte order
equal but not byte order comparable.
When we are exploding a compound key we know already that there is more
than one component, but we have no easy way of determining how many of
them are going to be there. Let's reserve space for a few elements so
that we avoid an excessive number of reallocations in case of
medium-sized keys.
mutation_fragment destructor was already made inline-friendly by moving
most of the logic to a separate function. However, the compiler still is
quite reluctant to inline it in certain cases, so let's give it a
stronger hint.
consume_mutation_fragments_until() is a consumer based interface that
avoids indirect calls and continuation overhead present in the naive
streamed_mutation::operator() approach.
clustering_row can stores quite a lot of data internally which makes its
move constructor not exactly cheap.
If possible it is better to move mutation_fragment around as it keeps
everything externally. This also avoids some cases when clustering row
would be extracted from mutation_fragment only to be made to create
another mutation_fragment later.
When a sstable reader is fast forwarded some index entries may be read
(and compared) multiple times. This patch makes sure that once a token
is computed we keep it around and reuse if the entry is accessed again.
Each sstable index lookup involves a binary search in the summary and
each time a partition key of summary entry is compared with anything its
token needs to be calculated.
Since we keep summary in the memory all the time it is better to also
keep the tokens around.
ring_position_comparator has overloads for comparing ring_positions as
well as sstables::key_view. In the case of the latter it needs to
compute the token of the key. However, the sstable layer could cache
some tokens so let's allow the comparator callers to provide it
directly.
It is possible that a call to fill_buffer() will return an immediately
ready future. This patch avoids uncontrolled recursion in case when all
merged streamed mutation do not defer ini fill_buffer() and also
optimises for non-deferring case by avoiding some of the logic.
streamed_mutation::impl::fill_buffer() is supposed to either push
mutation fragments to the buffer or set EOS flag. However, it was
possible that mp_row_consumer would return proceed::no if a skip was
needed without satisfying any of these conditions.
"This patch series addresses some feedback from the preliminary
HACKING.md, adds some new content, and updates the README file with
some quick-start information."
* 'jhk/better_hacking/v3' of github.com:hakuch/scylla:
README.md: Add quick-start section and defer to `HACKING.md`
HACKING.md: `CMakeLists.txt` for analysis works for other IDEs too
HACKING.md: Add details and examples for unit tests
HACKING.md: Add section for project dependencies
HACKING.md: Describe releases and tags
HACKING.md: Re-work "building" section, including memory needs
HACKING.md: Update ccache recommendations
HACKING.md: Update "Contributing" URL
If a node is notified of a schema change where the schema's dropped
columns have changes, that node will miss the changes to the dropped
columns. A scenario where this can happen is where a column c is
dropped, then added as a different typed, and then dropped again, with
a node n having seen the first drop and being notified of the
subsequent add and drop.
Fixes#2616
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20170725170622.4380-1-duarte@scylladb.com>
