"_free_segments_in_zones is not adjusted by
segment_pool::reclaim_segments() for empty zones on reclaim under some
conditions. For instance when some zone becomes empty due to regular
free() and then reclaiming is called from the std allocator, and it is
satisfied from a zone after the one which is empty. This would result
in free memory in such zone to appear as being leaked due to corrupted
free segment count, which may cause a later reclaim to fail. This
could result in bad_allocs.
The fix is to always collect such zones.
Fixes#3129
Refs #3119
Refs #3120"
* 'tgrabiec/fix-free_segments_in_zones-leak' of github.com:scylladb/seastar-dev:
tests: lsa: Test _free_segments_in_zones is kept correct on reclaim
lsa: Expose max_zone_segments for tests
lsa: Expose tracker::non_lsa_used_space()
lsa: Fix memory leak on zone reclaim
(cherry picked from commit 4ad212dc01)
The call chain is:
storage_service::on_change() -> storage_service::handle_state_removing()
-> storage_service::restore_replica_count() -> streamer->stream_async()
Listeners run as part of gossip message processing, which is serialized.
This means we won't be processing any gossip messages until streaming
completes.
In fact, there is no need to wait for restore_replica_count to complete
which can take a long time, since when it completes, this node will send
notification to tell the removal_coordinator that the restore process is
finished on this node. This node will be removed from _replicating_nodes
on the removal_coordinator.
Tested with update_cluster_layout_tests.py
Fixes#2886
Message-Id: <8b4fe637dfea6c56167ddde3ca86fefb8438ce96.1516088237.git.asias@scylladb.com>
(cherry picked from commit 5107b6ad16)
end_bound was not updated in one of the cases in which end and
end_kind was changed, as a result later merging decision using
end_bound were incorrect. end_bound was using the new key, but the old
end_kind.
Fixes#3083.
Message-Id: <1513772083-5257-1-git-send-email-tgrabiec@scylladb.com>
(cherry picked from commit dfe48bbbc7)
Don't enforce the outgoing connections from the 'listen_address'
interface only.
If 'local_address' is given to connect() it will enforce it to use a
particular interface to connect from, even if the destination address
should be accessed from a different interface. If we don't specify the
'local_address' the source interface will be chosen according to the
routing configuration.
Fixes#3066
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1513372688-21595-1-git-send-email-vladz@scylladb.com>
(cherry picked from commit be6f8be9cb)
Currently scylla-housekeeping-daily.service/-restart.service hardcoded
"--repo-files '/etc/yum.repos.d/scylla*.repo'" to specify CentOS .repo file,
but we use same .service for Ubuntu/Debian.
It doesn't work correctly, we need to specify .list file for Debian variants.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1513385159-15736-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit c2e87f4677)
We have had an issue recently where failed SSTable writes left the
generated SSTables dangling in a potentially invalid state. If the write
had, for instance, started and generated tmp TOCs but not finished,
those files would be left for dead.
We had fixed this in commit b7e1575ad4,
but streaming memtables still have the same isse.
Note that we can't fix this in the common function
write_memtable_to_sstable because different flushers have different
retry policies.
Fixes#3062
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20171213011741.8156-1-glauber@scylladb.com>
(cherry picked from commit 1aabbc75ab)
"Fixes cache reader to not skip over data in some cases involving overlapping
range tombstones in different partition versions and discontinuous cache.
Introduced in 2.0
Fixes #3053."
* tag 'tgrabiec/fix-range-tombstone-slicing-v2' of github.com:scylladb/seastar-dev:
tests: row_cache: Add reproducer for issue #3053
tests: mvcc: Add test for partition_snapshot::range_tombstones()
mvcc: Optimize partition_snapshot::range_tombstones() for single version case
mvcc: Fix partition_snapshot::range_tombstones()
tests: random_mutation_generator: Do not emit dummy entries at clustering row positions
(cherry picked from commit 051cbbc9af)
(cherry picked from commit be5127388d)
[tgrabiec: dropped mvcc_test change, because the file does not exist here]
The exception handling code inspects server state, which could be
destroyed before the handle_exception() task runs since it runs after
exiting the gate. Move the exception handling inside the gate and
avoid scheduling another accept if the server has been stopped.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20171116122921.21273-1-duarte@scylladb.com>
(cherry picked from commit 34a0b85982)
thrift/server.cc:237:6: required from here
thrift/server.cc:236:9: error: cannot call member function ‘void thrift_server::maybe_retry_accept(int, bool, std::__exception_ptr::exception_ptr)’ without object
maybe_retry_accept(which, keepalive, std::move(ex));
gcc version: gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1)
Signed-off-by: Raphael S. Carvalho <raphaelsc@scylladb.com>
Message-Id: <20171113184537.10472-1-raphaelsc@scylladb.com>
(cherry picked from commit 564046a135)
"Ensure stop() waits for the accept loop to complete to avoid crashes
during shutdown."
* 'thrift-server-stop/v4' of https://github.com/duarten/scylla:
thrift/server: Restore code format
thrift/server: Stopping the server waits for connection shutdown
thrift/server: Abort listeners on stop()
thrift/server: Avoid manual memory management
thrift/server: Add move ctor for connection
thrift/server: Extract retry logic
thrift/server: Retry with backoff for some error types
thrift/server: Retry accept in case of error
(cherry picked from commit 061f6830fa)
The parser generator somehow confuses the use-after-scope sanitizer, causing it
to use large amounts of stack space. Disable that sanitizer on that file.
Message-Id: <20170905110628.18047-1-avi@scylladb.com>
(cherry picked from commit 4751402709)
The state machines generated by antlr allocate many local variables per function.
In release mode, the stack space occupied by the variables is reused, but in debug
build, it is not, due to Address Sanitizer setting -fstack-reuse=none. This causes
a single function to take above 100k of stack space.
Fix by hacking the generated code to use just one variable.
Fixes#2546
Message-Id: <20170704135824.13225-1-avi@scylladb.com>
(cherry picked from commit a6d9cf09a7)
The universal reference was introduced so we could bind an rvalue to
the argument, but it would have sufficed to make the argument a const
reference. This is also more consistent with the function's other
overload.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20171129132758.19654-1-duarte@scylladb.com>
(cherry picked from commit cda3ddd146)
(cherry picked from commit 106c69ad45)
Fix two issues with serializing non-compound range tombstones as
compound: convert a non-compound clustering element to compound and
actually advertise the issue to other nodes.
* git@github.com:duarten/scylla.git rt-compact-fixes/v1:
compound_compact: Allow rvalues in size()
sstables/sstables: Convert non-compound clustering element to compound
tests/sstable_mutation_test: Verify we can write/read non-correct RTs
service/storage_service: Export non-compound RT feature
(cherry picked from commit e9cce59b85)
(cherry picked from commit 740fcc73b8)
Undid changes to size_estimates_virtual_reader
Changed test from memtable::make_flat_reader to memtable::make_reader
These tests now require having the storage service initialize, which
is needed to decide whether correct non-compound range tombstones
should be emitted or not.
Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20171126152921.5199-1-duarte@scylladb.com>
(cherry picked from commit 922f095f22)
(cherry picked from commit 8567723a7b)
This series mainly fixes issues with the serialization of promoted
index entries for non-compound schemas and with the serialization of
range tombstones, also for non-compound schemas.
We lift the correct cell name writing code into its own function,
and direct all users to it. We also ensure backward compatibility with
incorrectly generated promoted indexes and range tombstones.
Fixes#2995Fixes#2986Fixes#2979Fixes#2992Fixes#2993
* git@github.com:duarten/scylla.git promoted-index-serialization/v3:
sstables/sstables: Unify column name writers
sstables/sstables: Don't write index entry for a missing row maker
sstables/sstables: Reuse write_range_tombstone() for row tombstones
sstables/sstables: Lift index writing for row tombstones
sstables/sstables: Leverage index code upon range tombstone consume
sstables/sstables: Move out tombstone check in write_range_tombstone()
sstables/sstables: A schema with static columns is always compound
sstables/sstables: Lift column name writing logic
sstables/sstables: Use schema-aware write_column_name() for
collections
sstables/sstables: Use schema-aware write_column_name() for row marker
sstables/sstables: Use schema-aware write_column_name() for static row
sstables/sstables: Writing promoted index entry leverages
column_name_writer
sstables/sstables: Add supported feature list to sstables
sstables/sstables: Don't use incorrectly serialized promoted index
cql3/single_column_primary_key_restrictions: Implement is_inclusive()
cql3/delete_statement: Constrain range deletions for non-compound
schemas
tests/cql_query_test: Verify range deletion constraints
sstables/sstables: Correctly deserialize range tombstones
service/storage_service: Add feature for correct non-compound RTs
tests/sstable_*: Start the storage service for some cases
sstables/sstable_writer: Prepare to control range tombstone
serialization
sstables/sstables: Correctly serialize range tombstones
tests/sstable_assertions: Fix monotonicity check for promoted indexes
tests/sstable_assertions: Assert a promoted index is empty
tests/sstable_mutation_test: Verify promoted index serializes
correctly
tests/sstable_mutation_test: Verify promoted index repeats tombstones
tests/sstable_mutation_test: Ensure range tombstone serializes
correctly
tests/sstable_datafile_test: Add test for incorrect promoted index
tests/sstable_datafile_test: Verify reading of incorrect range
tombstones
sstables/sstable: Rename schema-oblivious write_column_name() function
sstables/sstables: No promoted index without clustering keys
tests/sstable_mutation_test: Verify promoted index is not generated
sstables/sstables: Optimize column name writing and indexing
compound_compat: Don't assume compoundness
(cherry picked from commit bd1efbc25c)
Also added sstables::make_sstable() to preserve source compatibility in tests.
_current_pi_idx was not reset from advance_to_next_partition(), which
is used when we skip to the next partition before fully consuming
it. As a result, if we try to skip to a clustering position which is
before the index block used by the last skip in the previous
partition, we would not skip assuming that the new position is in the
current block. This may result in more data being read from the
sstable than necessary.
Fixes#2984
Message-Id: <1510915793-20159-1-git-send-email-tgrabiec@scylladb.com>
(cherry picked from commit 2113299b61)
Broken in f570e41d18.
Not replicating this may cause coordinator to treat a node which is
down as alive, or vice verse.
Fixes regression in dtest:
consistency_test.py:TestAvailability.test_simple_strategy
which was expected to get "unavailable" exception but it was getting a
timeout.
Message-Id: <1510666967-1288-1-git-send-email-tgrabiec@scylladb.com>
(cherry picked from commit 7323fe76db)
Fixes#2938.
* 'tgrabiec/fix-range-tombstone-list-exception-safety-v1' of github.com:scylladb/seastar-dev:
tests: range_tombstone_list: Add test for exception safety of apply()
tests: Introduce range_tombstone_list assertions
cache: Make range tombstone merging exception-safe
range_tombstone_list: Introduce apply_monotonically()
range_tombstone_list: Make reverter::erase() exception-safe
range_tombstone_list: Fix memory leaks in case of bad_alloc
mutation_partition: Fix abort in case range tombstone copying fails
managed_bytes: Declare copy constructor as allocation point
Integrate with allocation failure injection framework
(cherry picked from commit 5a4b46f555)
"This series backports both the cleanup series from Duarte and large cluster
fixes series from Tomek."
* tag 'gossip-2.0-backport-v2' of github.com:scylladb/seastar-dev:
Merge 'Solves problems related to gossip which can be observed in a large cluster' from Tomasz
utils: introduce loading_shared_values
tests/serialized_action: add missing forced defers
utils: Introduce serialized_action
Merge "gms/gossiper: Multiple cleanups" from Duarte
storage_service: Do not use c_str() in the logger
gms/gossiper: Introduce copy-less endpoint_state::get_application_state_ptr()
Merge "gossiper: Optimize endpoint_state lookup" from Duarte
gms: Add is_shutdown helper for endpoint_state class
gossip: Better check for gossip stabilization on startup
Merge "Fix miss opportunity to update gossiper features" from Asias
gossip: Fix a log message typo in compare_endpoint_startup
gossip: Do not use c_str() in the logger
Revert "gossip: Make bootstrap more robust"
gossip: Switch to seastar::lowres_system_clock
gossip: Use unordered_map for _unreachable_endpoints and _shadow_unreachable_endpoints
gossip: Introduce the shadow_round_ms option
"The main problem fixed is slow processing of application state changes.
This may lead to a bootstrapping node not having up to date view on the
ring, and serve incorrect data.
Fixes #2855."
* tag 'tgrabiec/gossip-performance-v3' of github.com:scylladb/seastar-dev:
gms/gossiper: Remove periodic replication of endpoint state map
gossiper: Check for features in the change listener
gms/gossiper: Replicate changes incrementally to other shards
gms/gossiper: Document validity of endpoint_state properties
storage_service: Update token_metadata after changing endpoint_state
gms/gossiper: Process endpoints in parallel
gms/gossiper: Serialize state changes and notifications for given node
utils/loading_shared_values: Allow Loader to return non-future result
gms/gossiper: Encapsulate lookup of endpoint_state
storage_service: Batch token metadata and endpoint state replication
utils/serialized_action: Introduce trigger_later()
gossiper: Add and improve logging
gms/gossiper: Don't fire change listeners when there is no change
gms/gossiper: Allow parallel apply_state_locally()
gms/gossiper: Avoid copies in endpoint_state::add_application_state()
gms/failure_detector: Ignore short update intervals
(cherry picked from commit 044b8deae4)
This class implements an key-value container that is populated
using the provided asynchronous callback.
The value is loaded when there are active references to the value for the given key.
Container ensures that only one entry is loaded per key at any given time.
The returned value is a lw_shared_ptr to the actual value.
The value for a specific key is immediately evicted when there are no
more references to it.
The container is based on the boost::intrusive::unordered_set and is rehashed (grown) if needed
every time a new value is added (asynchronously loaded).
The container has a rehash() method that would grow or shrink the container as needed
in order to get the load factor into the [0.25, 0.75] range.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
(cherry picked from commit ec3fed5c4d)
serialized_action_tests depends on the fact that first part of the
serialized_action is executed at cetrtain points (in which it reads a
global variable that is later updated by the main thread).
This worked well in the release mode before ready continuations were
inlined and run immediately, but not in the debug mode since inlining
was not happening and the main seastar::thread was missing some yield
points.
Message-Id: <20170731103013.26542-1-pdziepak@scylladb.com>
(cherry picked from commit e970630272)
"Based on the functions get_endpoint_state_for_endpoint_ptr(),
get_application_state_ptr() and
endpoint_state::get_application_state_ptr(), this series
cleanups miscelaneous functions related to the gossiper.
It not only removes duplicated code, but also omits many copies.
All pointer usages have been audited for safety."
Acked-by: Asias He <asias@scylladb.com>
Acked-by: Tomasz Grabiec <tgrabiec@scylladb.com>
* 'gossiper-cleanup/v2' of github.com:duarten/scylla: (27 commits)
gms/endpoint_state: Remove get_application_state()
service/storage_service: Avoid copies in prepare_replacement_info()
service/storage_service: Cleanup get_application_state_value()
service/storage_service: Cleanup handle_state_removing()
service/storage_service: Cleanup get_rpc_address()
locator/reconnectable_snitch_helper: Avoid versioned_value copies
locator/production_snitch_base: Cleanup get_endpoint_info()
service/migration_manager: Avoid copies in is_ready_for_bootstrap()
service/migration_manager: Cleanup has_compatible_schema_tables_version()
service/migration_manager: Fix usages of get_application_state()
cache_hit_rate: Avoid copies in get_hit_rate()
gms/endpoint_state: Avoid copies in is_shutdown()
service/load_broadcaster: Avoid copy in on_join()
gms/gossiper: Cleanup get_supported_features()
gms/gossiper: Cleanup get_gossip_status()
gms/gossiper: Cleanup seen_any_seed()
gms/gossiper: Cleanup get_host_id()
gms/gossiper: Removed dead uses_vnodes() function
gms/gossiper: Cleanup uses_host_id()
gms/gossiper: Add get_application_state_ptr()
...
(cherry picked from commit 1701fc2e50)
"gossiper::get_endpoint_state_for_endpoint() returns a copy of
endpoint_state, which we've seen can be very expensive. This
series introduces a function that returns a pointer and avoids
the copy.
Fixes#764"
* 'endpoint-state/v2' of https://github.com/duarten/scylla:
gossiper: Avoid endpoint_state copies
endpoint_state: const-qualify functions
storage_service: Remove duplicate endpoint state check
(cherry picked from commit 4ad3900d8d)
The gossiper checks if features should be enabled from its timer
callback when it detects that endpoint_state_map changed, that is
different than shadow_endpoint_state_map.
shadow_endpoint_state_map is also assigned from endpoint_state_map in
storage_service::replicate_tm_and_ep_map(), called from
storage_service::on_change()
Call gossiper:maybe_enable_features() in replicate_tm_and_ep_map so
that we won't miss gossip feature update.
Fixes#2824
* git@github.com:scylladb/seastar-dev asias/gossip_miss_feature_update_v1:
gossip: Move the _features_condvar signal code to
maybe_enable_features
gossip: Make maybe_enable_features public
storage_service: Check gossip feature update in
replicate_tm_and_ep_map
(cherry picked from commit 02d41864af)
This reverts commit b56ba02335.
After commit 8fa35d6ddf (messaging_service: Get rid of timeout and retry
logic for streaming verb), streaming verb in rpc does not check if a
node is in gossip memebership since all the retry logic is removed.
Remove the extra wait before removing the joining node from gossip
membership.
Message-Id: <a416a735bb8aad533bbee190e3324e6b16799415.1504063598.git.asias@scylladb.com>
(cherry picked from commit cc18da5640)
