mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-06-03 21:47:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
14,402 Commits 257 Branches 558 Tags
b91c3fd47e62e6daa1681c8a61c60add3e8a0a2a
Commit Graph

14402 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Vladimir Krivopalov
b91c3fd47e Use advance_past for single partition upper bound. 
Instead of advancing to the next partition, try first find the more
precise position using promoted index blocks.
advance_past() only seeks within currently available PI blocks (or reads
the first batch, if never read before) and uses the position if found,
otherwise resorts to advance_to_next_partition()

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:57:45 -08:00
Vladimir Krivopalov
6f8c6a0933 Remove obsolete types and methods. 
These types and methods are no longer in use since the index_reader is
now consuming promoted index incrementally.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:57:35 -08:00
Vladimir Krivopalov
0a7a56edd5 Simplify continuous_data_consumer::consume_input() interface. 
Remove redundant input parameter as continuous_data_consumer derivatives
would only use themselves as a context. So take it internally and make
the function regular (non-template) and having no parameters.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:57:26 -08:00
Vladimir Krivopalov
7e15e436de Parse promoted index entries lazily upon request rather than immediately. 
Now promoted index is converted into an input_stream and skipped over
instead of being consumed immediately and stored as a single buffer.
The only part that is read right away is the deletion time as it is
likely to be there in the already read buffer and reading it should both
be cheap and prevent from reading the whole promoted index if only
deletion time mark is needed.

When accessed, promoted index is parsed in chunks, buffer by buffer, to
limit memory consumption.

Fixes #2981

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:57:15 -08:00
Vladimir Krivopalov
9fdf4b24b5 Add helper input streams: buffer_input_stream and prepended_input_stream. 
buffer_input_stream is a simple input_stream wrapping a single
temporary_buffer.

prepended_input_stream suits for the case when some data has been read
into a buffer and the rest is still in a stream. It accepts a buffer and
a data_source and first reads from the buffer and then, when it ends,
proceeds reading from the data_source.

Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:57:04 -08:00
Vladimir Krivopalov
5dca3100ed Support skipping over bytes from input stream in parsers based on continuous_data_consumer 
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:56:55 -08:00
Vladimir Krivopalov
ebdcffab1a Add performance tests for large partition slicing using clustering keys. 
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
 2018-01-29 11:56:35 -08:00
Takuya ASADA
5f835be3aa dist/common/scripts/scylla_io_setup: check data_file_directories existance before running iotune 
Currently we don't check data_file_directories existance before running iotune,
therefore it's shows unclear error message.
To make the message better, check the directory existance on scylla_io_setup.

Fixes #3137

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517200647-6347-1-git-send-email-syuu@scylladb.com>
 2018-01-29 18:11:12 +02:00
Avi Kivity
3ce5ad3c7c Merge seastar upstream 
* seastar d03896d...770c450 (10):
  > tls_test: Fix echo test not setting server trust store
  > tls: Do not restrict re-handshake to client
  > tls: Actually verify client certificate if requested
  > rwlock: add method for determining if an rwlock is locked
  > metrics: Add missing `break` to metric_value::operator+()
  > memory: fix error injector throwing from noexcept memory allocator functions
  > systemwide_memory_barrier: don't use mprotect() on ARM
  > sharded: Add const version of sharded::local()
  > Add const overloads of front() and back() to the circular_buffer.
  > Remove unused lambda captures

Fixes #3072
 2018-01-29 15:28:44 +02:00
Botond Dénes
12b1520415 exponential_backoff_retry::do_until_value(): restore indentation 
Deferred from previous patch.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <a10053f6c0ed8a24a74e51f1df4e9a5acf59922d.1517222195.git.bdenes@scylladb.com>
 2018-01-29 10:50:01 +00:00
Botond Dénes
e0c082616a exponential_backoff_retry::do_until_value(): fix use-after-move 
The exponential_backoff_retry instance is captured by move and is then
indirectly moved again as repeat_until_value() moves the lambda its
passed into its internal state. This caused problems as internal
lambdas store references to the instance and these references go stale
after the move.
To fix this keep hold of the existential_backoff_retry instance in an
enclosing do_with() to make it safe for internal lambdas to reference
it.

Indentation will be fixed by the next patch.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <adc49d25a6176756d60e092f3713c0c897732382.1517222195.git.bdenes@scylladb.com>
 2018-01-29 10:50:01 +00:00
Duarte Nunes
bfe5a8e96f utils/managed_vector: Return reference to emplaced element 
We are in 2018, after all.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20180126105417.54285-1-duarte@scylladb.com>
 2018-01-26 13:49:56 +01:00
Duarte Nunes
269a4aec23 test.py: Rename streamed_mutation_test 
96c97ad1db changed the name of the test,
but didn't update the test.py file.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
 2018-01-26 01:04:23 +01:00
Tomasz Grabiec
1219120c00 Merge cleanup of non-flat mutation readers from Piotr 
Removes uses of obsolete mutation_reader and streamed_mutation.
Superseded by flat_mutation_reader.

* seastar-dev.git haaawk/cleanup:
  Rename streamed_mutation* files to mutation_fragment*
  Delete unused streamed_mutation
  Delete unused  consume_all(streamed_mutation&)
  Delete unused fill_buffer_from<streamed_mutation>
  Delete unused do_consume_streamed_mutation_flattened
  streamed_mutation: delete operator<<
  streamed_mutation: delete unused make_forwardable
  Delete unused streamed_mutation_opt
  Delete unused check_order_of_fragments
  Delete unused streamed_mutation_from_mutation
  Move test_abandoned_flat_mutation_reader_from_mutation to
  Change test_abandoned_streamed_mutation_from_mutation
  test_mutation_merger_conforms_to_mutation_source: use flat reader
  Delete unused consume(streamed_mutation&)
  Delete unused mutation_from_streamed_mutation(streamed_mutation_opt)
  Delete unused mutation_from_streamed_mutation(streamed_mutation&)
  Delete test_mutation_from_streamed_mutation_from_mutation
  Delete unused freeze(streamed_mutation)
  Delete test_freezing_streamed_mutations
  streamed_mutation: delete unused transform
  test_schema_upgrader_is_equivalent_with_mutation_upgrade: use flat reader
  streamed_mutation: delete unused consume_mutation_fragments_until
  Delete unused merge_mutations
  Delete test_mutation_merger
  Delete unused make_empty_streamed_mutation
  Delete unused streamed_mutation_from_forwarding_streamed_mutation
  Delete unused streamed_mutation_assertions
  Turn test_streamed_mutation_fragments_have_monotonic_positions
  Delete run_conversion_to_mutation_reader_tests
  Delete unused assert_that(streamed_mutation_opt)
  Delete unused assert_that(streamed_mutation)
  Delete unused mutation_reader
  perf_fast_forward: delete unused consume_all
  Delete unused consume(mutation_reader&, Consumer)
  Remove unused mutation_reader_assertions
  Remove unused query_state::reader
  Delete unused make_reader_returning
  Delete unused make_reader_returning_many
  Delete unused make_empty_reader
  Delete unused mutation_reader_from_flat_mutation_reader
  Delete unused flat_mutation_reader_from_mutation_reader
  Delete tests for mutation readers converters
  dummy_incremental_selector: use flat reader
  Delete unused streamed_mutation_from_flat_mutation_reader
  perf_fast_forward: use flat reader in test_forwarding_with_restriction
  perf_fast_forward: use flat reader in slice_partitions
  perf_fast_forward: use flat reader in slice_rows_single_key
  perf_fast_forward: use flat reader in test_reading_all
  perf_fast_forward: use flat reader in slice_rows
  perf_fast_forward: add consume_all_with_next_partition
  perf_fast_forward: use flat reader in scan_with_stride_partitions
  perf_fast_forward: use flat reader in scan_rows_with_stride
  perf_fast_forward: add assert_partition_start
  perf_fast_forward: add consume_all(flat_mutation_reader&)
  partition_checksum::compute_legacy: use only flat reader
  row_cache: rename make_flat_reader to make_reader
  row_cache: Delete unused make_reader
  test_mvcc: use flat reader
  test_cache_population_and_clear_race: use flat reader
  test_cache_population_and_update_race: use flat reader
  test_continuity_flag_and_invalidate_race: use flat reader
  test_update_failure: use flat reader
  row_cache_test: use flat reader in verify_has
  row_cache_test: use flat reader in has_key
  test_sliced_read_row_presence: use flat reader
  test_lru: use flat reader
  test_update_invalidating: use flat reader
  test_scan_with_partial_partitions: use flat reader
  test_cache_populates_partition_tombstone: use flat reader
  test_tombstone_merging_in_partial_partition: use flat reader
  consume_all,populate_range: use flat reader
  test_readers_get_all_data_after_eviction: use flat reader
  test_tombstones_are_not_missed_when_range_is_invalidated: use flat reader
  test_exception_safety_of_reads: use flat reader
  test_exception_safety_of_transitioning_from_underlying_read_to_read_from_cache: use flat reader
  test_exception_safety_of_partition_scan: use flat reader
  test_concurrent_population_before_latest_version_iterator: use flat reader
  test_concurrent_populating_partition_range_reads: use flat reader
  test_random_row_population: use flat reader
  test_continuity_is_populated_when_read_overlaps_with_older_version: use flat reader
  test_continuity_population_with_multicolumn_clustering_key: use flat reader
  test_continuity_is_populated_for_single_row_reads: use flat reader
  flat_mutation_reader_assertions: add produces_compacted
  test_concurrent_setting_of_continuity_on_read_upper_bound: use flat reader
  test_reading_from_random_partial_partition: use flat reader
  test_tombstone_merging_of_overlapping_tombstones_in_many_versions: use flat reader
  test_concurrent_reads_and_eviction: use flat reader
  test_eviction: use flat reader
  test_random_partition_population: use flat reader
  test_single_key_queries_after_population_in_reverse_order: use flat reader
  test_query_of_incomplete_range_goes_to_underlying: use flat reader
  test_cache_delegates_to_underlying_only_once_with_single_partition: use flat reader
  test_cache_uses_continuity_info_for_single_partition_query: use flat reader
  test_cache_delegates_to_underlying_only_once_empty_single_partition_query: use flat reader
  test_cache_delegates_to_underlying_only_once_empty_full_range: use flat reader
  test_cache_works_after_clearing: use flat reader
  test_cache_delegates_to_underlying: use flat reader
  cache_flat_mutation_reader_test: use flat reader
  row_cache_alloc_stress: use flat reader
 2018-01-24 21:54:08 +01:00
Piotr Jastrzebski
1f9df7aade Fix master 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 21:00:51 +01:00
Piotr Jastrzebski
96c97ad1db Rename streamed_mutation* files to mutation_fragment* 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
d590a063c6 Delete unused streamed_mutation 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
6f468802f4 Delete unused consume_all(streamed_mutation&) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
970a863950 Delete unused fill_buffer_from<streamed_mutation> 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
28c36d8884 Delete unused do_consume_streamed_mutation_flattened 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
6c6068f1da streamed_mutation: delete operator<< 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
f907073bde streamed_mutation: delete unused make_forwardable 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
a346b32584 Delete unused streamed_mutation_opt 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
7161781586 Delete unused check_order_of_fragments 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
41b23a619e Delete unused streamed_mutation_from_mutation 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
795102a0f8 Move test_abandoned_flat_mutation_reader_from_mutation to 
flat_mutation_reader_test.cc

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
6b78956563 Change test_abandoned_streamed_mutation_from_mutation 
to test_abandoned_flat_mutation_reader_from_mutation

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
9e06711805 test_mutation_merger_conforms_to_mutation_source: use flat reader 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
916a9c339c Delete unused consume(streamed_mutation&) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
d9cbb9fedc Delete unused mutation_from_streamed_mutation(streamed_mutation_opt) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
759271f866 Delete unused mutation_from_streamed_mutation(streamed_mutation&) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
a39ddc8cf6 Delete test_mutation_from_streamed_mutation_from_mutation 
It tests mutation_from_streamed_mutation that is no longer
used and will be removed in the next patch.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
a1cf4b4cae Delete unused freeze(streamed_mutation) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
0f78e9c24a Delete test_freezing_streamed_mutations 
It tests freeze(streamed_mutation) which is no longer used
and will be removed in the next patch.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
05ae4f5d15 streamed_mutation: delete unused transform 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:49 +01:00
Piotr Jastrzebski
1c12884fba test_schema_upgrader_is_equivalent_with_mutation_upgrade: use flat reader 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
eec6c2efb5 streamed_mutation: delete unused consume_mutation_fragments_until 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
ca905d38b1 Delete unused merge_mutations 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
8abbabef30 Delete test_mutation_merger 
merge_mutations is no longer used and will be removed
by the next patch.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
b82f00fafb Delete unused make_empty_streamed_mutation 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
fb42022f03 Delete unused streamed_mutation_from_forwarding_streamed_mutation 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
5959337234 Delete unused streamed_mutation_assertions 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
8bdc74c9e2 Turn test_streamed_mutation_fragments_have_monotonic_positions 
into test_mutation_reader_fragments_have_monotonic_positions

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
a546cfd0d5 Delete run_conversion_to_mutation_reader_tests 
It's no longer needed because converters are no longer used.

Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
05ed42c08d Delete unused assert_that(streamed_mutation_opt) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
912a38d60b Delete unused assert_that(streamed_mutation) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
61f0ac257f Delete unused mutation_reader 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
a944a1f7f1 perf_fast_forward: delete unused consume_all 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
9ce48bc5fc Delete unused consume(mutation_reader&, Consumer) 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00
Piotr Jastrzebski
7729bc5e7b Remove unused mutation_reader_assertions 
Signed-off-by: Piotr Jastrzebski <piotr@scylladb.com>
 2018-01-24 20:56:48 +01:00