mirrors/scylladb
1
0
Fork 0
mirror of https://github.com/scylladb/scylladb.git synced 2026-05-31 12:06:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,927 Commits 255 Branches 556 Tags
fee0b7fa480c08f4fac0d17420d35269fb685a5a
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Duarte Nunes fee0b7fa48 query_result_merger: Limit rows 
This patch makes the row limit enforced by the storage_proxy layer.
It adds a row limit to the query_result_merger, useful when merging
results for concurrent queries.

More importantly, it provides guarantees that upper layers may be
relying on implicitly (e.g., the paging code).

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
2016-12-15 11:00:36 +00:00
.github
Make github issue template less shouty
2016-06-01 10:45:04 +03:00
api
API: cache_capacity should use uint for summing
2016-11-16 13:55:46 +01:00
auth
cql3::query_processor: add cql_stats
2016-11-03 11:48:57 -04:00
conf
get rid of max_memtable_size
2016-11-16 21:20:24 -05:00
cql3
query_result_merger: Limit rows
2016-12-15 11:00:36 +00:00
db
Merge "rework flush criteria" from Glauber
2016-12-14 11:24:10 +02:00
dht
dht: Introduce split_ranges_to_shards
2016-12-12 09:04:21 +08:00
dist
dist/common/scripts/scylla_setup: don't abort scylla_setup when each setup script failed
2016-12-14 13:31:50 +02:00
docs
docs: tracing.md: initial commit
2016-10-10 16:12:02 +03:00
exceptions
exceptions: mark function called before construction static
2016-10-03 16:29:02 +03:00
gms
gossip: Log feature enabled message on shard zero only
2016-12-15 16:33:11 +08:00
idl
reconcilable_result: properly propagate short_read flag
2016-12-14 14:10:02 +00:00
interface
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
io
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
licenses
…
locator
token_metadata: Introduce interval_to_range helper
2016-12-12 11:09:26 +08:00
message
storage_proxy: avoid calculating digest when only one replica is contacted
2016-11-17 13:04:30 +02:00
repair
repair: Fix a typo in the log
2016-12-12 11:09:26 +08:00
scripts
scripts/scylla_install_pkg: support Debian
2016-12-06 12:06:30 +02:00
seastar @ 0b98024073
Merge seastar upstream
2016-12-14 15:01:13 +00:00
service
query_result_merger: Limit rows
2016-12-15 11:00:36 +00:00
sstables
compaction_manager: fix shutdown sequence
2016-12-14 09:26:24 +01:00
streaming
streaming: Use interval_set to sort and merge ranges
2016-12-12 11:09:26 +08:00
swagger-ui @ 1b212bbe71
…
tests
tests: Remove unintentional enablement of trace-level logging
2016-12-14 10:58:07 +01:00
thrift
Merge " Virtualize size_estimates system table" from Duarte
2016-11-28 10:12:59 +02:00
tools/scyllatop
scyllatop: dump all output to stdout instead of running a fancy console interface
2016-08-31 08:31:36 +03:00
tracing
tracing: introduce the tracing::global_trace_state_ptr class
2016-10-02 11:31:37 +03:00
transport
lz4: Conditionally use LZ4_compress_default()
2016-11-24 11:25:03 +02:00
utils
utils: Put crc32 under utils namespace
2016-12-05 11:48:29 +02:00
.gitattributes
…
.gitignore
ScyllaTop: top-like tool to see live scylla metrics
2016-02-23 12:32:47 +02:00
.gitmodules
…
.gitorderfile
…
atomic_cell_hash.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
atomic_cell_or_collection.hh
rename memory_usage() to external_memory_usage() where applicable
2016-11-18 11:25:36 +00:00
atomic_cell.hh
atomic_cell: add overloads taking const bytes&
2016-08-22 09:31:33 +01:00
bytes_ostream.hh
bytes_ostream: make max_chunk_size() an inline function
2016-10-17 11:49:33 +03:00
bytes.cc
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
bytes.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
caching_options.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
canonical_mutation.cc
serializer: Introduce as_input_stream(bytes_view)
2016-02-26 12:26:13 +01:00
canonical_mutation.hh
remove db/serializer.hh includes
2016-03-02 09:07:09 +00:00
cartesian_product.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
checked-file-impl.hh
db: make it possible to use custom error handler with io checker
2016-10-27 15:54:21 -02:00
clustering_bounds_comparator.hh
bounds_view: Create from nonwrapping_range
2016-08-22 17:52:36 +02:00
clustering_key_filter.hh
Remove clustering_key_filtering_context.
2016-08-30 20:31:55 +02:00
combine.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
compaction_strategy.hh
lcs: fix broken token range distribution at higher levels
2016-09-30 14:09:16 -03:00
compatible_ring_position.hh
compatible_ring_position: add function to return token
2016-12-08 14:25:29 -02:00
compound_compat.hh
composite: Use operator[] instead of at()
2016-07-28 11:11:54 +02:00
compound.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
compress.hh
cql3: Disable compression on empty properties
2016-11-09 10:03:59 +02:00
configure.py
size_estimates_virtual_reader: Add unit test
2016-11-21 11:15:05 +00:00
CONTRIBUTING.md
CONTRIBUTING.md: add sections for help and issues
2016-11-18 22:21:10 +02:00
converting_mutation_partition_applier.hh
schema: Dense schemas are correctly upgrades
2016-08-03 13:39:01 +02:00
cql_serialization_format.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
database_fwd.hh
database: keep a pointer to the memtable list in a memtable
2016-11-21 18:18:27 +02:00
database.cc
column_family: Use counters in query::result::builder
2016-12-15 10:27:46 +00:00
database.hh
db: add metrics for short reads and memory used for results
2016-12-14 14:28:36 +00:00
db_clock.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
debug.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
digest_algorithm.hh
storage_proxy: avoid calculating digest when only one replica is contacted
2016-11-17 13:04:30 +02:00
disk-error-handler.cc
db: make it possible to use custom error handler with io checker
2016-10-27 15:54:21 -02:00
disk-error-handler.hh
db: make it possible to use custom error handler with io checker
2016-10-27 15:54:21 -02:00
dns.cc
dns: Move gethostbyname to source file
2015-10-26 15:59:58 +02:00
dns.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
Doxyfile
…
enum_set.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
frozen_mutation.cc
frozen_mutation: use memory_usage()
2016-11-18 11:25:36 +00:00
frozen_mutation.hh
frozen_mutation: avoid buffer linearization and copy
2016-08-22 09:31:33 +01:00
frozen_schema.cc
serializer: Introduce as_input_stream(bytes_view)
2016-02-26 12:26:13 +01:00
frozen_schema.hh
remove db/serializer.hh includes
2016-03-02 09:07:09 +00:00
gc_clock.cc
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
gc_clock.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
hashing_partition_visitor.hh
mutations: Row tombstones are now a set of ranges
2016-06-02 16:21:59 +02:00
hashing.hh
Introduce hashing helpers
2016-01-08 21:10:25 +01:00
idl-compiler.py
Merge seastar upstream
2016-09-15 12:12:16 +03:00
IDL.md
Add an IDL definition file
2016-01-24 12:29:21 +02:00
init.cc
implement listen_on_broadcast_address option
2016-09-26 08:49:54 +03:00
init.hh
implement listen_on_broadcast_address option
2016-09-26 08:49:54 +03:00
json.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
keys.cc
partition_key_view: Implement operator<<
2016-09-30 10:54:54 +02:00
keys.hh
keys: add memory_usage()
2016-11-18 11:25:36 +00:00
LICENSE.AGPL
…
log.hh
Merge seastar upstream
2016-06-01 18:28:42 +03:00
main.cc
commitlog: Fix replay to not delete dirty segments
2016-12-07 15:54:47 +02:00
map_difference.hh
map_difference: Allow on unordered_map
2016-04-20 09:54:06 +02:00
md5_hasher.hh
md5_hasher: add finalize_array()
2016-03-11 18:27:13 +00:00
memtable.cc
database: rework dirty memory hierarchy
2016-12-13 14:07:53 -05:00
memtable.hh
db: Clear memtable after flush when cache is disabled
2016-12-05 12:59:09 +01:00
mutation_compactor.hh
mutation_compactor: honour stop_iteration from consumers
2016-12-14 14:10:02 +00:00
mutation_partition_applier.hh
mutations: Row tombstones are now a set of ranges
2016-06-02 16:21:59 +02:00
mutation_partition_serializer.cc
frozen_mutation: freeze streamed_mutations
2016-06-20 21:29:52 +01:00
mutation_partition_serializer.hh
frozen_mutation: freeze streamed_mutations
2016-06-20 21:29:52 +01:00
mutation_partition_view.cc
mutation_partition_view: Avoid unnecessary copy into temporary
2016-06-14 17:02:17 +03:00
mutation_partition_view.hh
idl: switch to utils::input_stream
2016-08-22 09:31:33 +01:00
mutation_partition_visitor.hh
mutations: Row tombstones are now a set of ranges
2016-06-02 16:21:59 +02:00
mutation_partition.cc
column_family: Use counters in query::result::builder
2016-12-15 10:27:46 +00:00
mutation_partition.hh
rename memory_usage() to external_memory_usage() where applicable
2016-11-18 11:25:36 +00:00
mutation_query.cc
mutation_query: to_data_query_result enforces row limit
2016-12-15 10:56:40 +00:00
mutation_query.hh
mutation_query: to_data_query_result enforces row limit
2016-12-15 10:56:40 +00:00
mutation_reader.cc
mutation_reader: forward fast_forward_to() calls
2016-10-19 15:29:08 +01:00
mutation_reader.hh
mutation_reader: forward fast_forward_to() calls
2016-10-19 15:29:08 +01:00
mutation.cc
reconcilable_result: keep result_memory_tracker object
2016-12-14 14:10:02 +00:00
mutation.hh
mutation: Define + operation
2016-10-18 11:16:08 +02:00
noexcept_traits.hh
Introduce noexcept_traits
2015-12-07 09:50:27 +01:00
NOTICE.txt
…
nway_merger.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
ORIGIN
Update ORIGIN for gossip and storage_service
2015-12-01 19:45:04 +08:00
partition_builder.hh
mutations: Row tombstones are now a set of ranges
2016-06-02 16:21:59 +02:00
partition_range_compat.hh
Convert ring_position and token ranges to be nonwrapping
2016-11-02 21:04:11 +02:00
partition_slice_builder.cc
Merge "Fix query digest mismatch" from Tomasz
2016-04-08 12:13:29 +03:00
partition_slice_builder.hh
Merge "Fix query digest mismatch" from Tomasz
2016-04-08 12:13:29 +03:00
partition_version.cc
partition_version: Fix corruption of partition_version list
2016-10-18 09:25:38 +01:00
partition_version.hh
partition_version: fix const correctness in rows_entry_compare
2016-11-28 10:55:12 +00:00
query_result_merger.hh
query_result_merger: Limit rows
2016-12-15 11:00:36 +00:00
query-request.hh
query: allow short reads
2016-12-14 14:10:01 +00:00
query-result-reader.hh
storage_proxy: avoid calculating digest when only one replica is contacted
2016-11-17 13:04:30 +02:00
query-result-set.cc
query: use result_view::consume() where appropriate
2016-08-22 09:31:33 +01:00
query-result-set.hh
Merge "Fix query digest mismatch" from Tomasz
2016-04-08 12:13:29 +03:00
query-result-writer.hh
query::result: Add partition count
2016-12-15 10:27:46 +00:00
query-result.hh
storage_proxy: Don't fetch superfluous partitions
2016-12-15 10:27:46 +00:00
query.cc
query_result_merger: Limit rows
2016-12-15 11:00:36 +00:00
range_tombstone_list.cc
range_tombstone_list: use non-template lambda for cloning tombstones
2016-06-27 10:48:59 +02:00
range_tombstone_list.hh
range_tombstone_list: add non-const begin() and end()
2016-07-13 09:50:07 +01:00
range_tombstone_to_prefix_tombstone_converter.hh
range_tombstone: check for adjacent instead of equal bounds
2016-06-20 21:29:48 +01:00
range_tombstone.cc
Clean up position_in_partition.
2016-10-25 15:13:20 +01:00
range_tombstone.hh
streamed_mutation: add memory_usage() to mutation fragment types
2016-11-18 11:25:36 +00:00
range.hh
nonwrapping_range: Add slice() function
2016-11-21 11:15:04 +00:00
README-DPDK.md
…
README.md
README: Guidelines for contributing
2016-11-16 12:50:02 +02:00
release.cc
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
release.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
reversibly_mergeable.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
row_cache.cc
memtable: Track flushed memory in memtable object
2016-12-05 12:59:09 +01:00
row_cache.hh
Add separate LRU for wide partitions.
2016-11-15 16:19:13 +01:00
schema_builder.hh
schema_builder: Ensure dense tables have compact col
2016-08-02 14:49:13 +02:00
schema_mutations.cc
schema_mutations: drop old serializers
2016-02-19 23:12:00 +00:00
schema_mutations.hh
schema_mutations: drop old serializers
2016-02-19 23:12:00 +00:00
schema_registry.cc
schema_registry: Fix possible hang in maybe_sync() if syncer doesn't defer
2016-05-11 17:31:22 +02:00
schema_registry.hh
schema_registry: Track synced state of schema
2016-01-11 10:34:52 +01:00
schema.cc
schema: Implement operator<< for column_mapping
2016-12-13 12:20:46 +02:00
schema.hh
schema: Implement operator<< for column_mapping
2016-12-13 12:20:46 +02:00
scylla-blocktune
blocktune: fix syntax error in exception handling
2016-10-23 16:40:00 +03:00
scylla-gdb.py
scylla-gdb: Fix lookup of symbols in 'scylla ptr'
2016-12-01 12:33:29 +02:00
scylla-housekeeping
scylla-housekeeping support uuid file
2016-12-11 16:00:34 +02:00
SCYLLA-VERSION-GEN
dist: renumber development version as 666.development
2016-01-12 12:26:08 +02:00
serialization_visitors.hh
serializer: Move skip() to serializer.hh
2016-07-25 17:35:42 +02:00
serializer_impl.hh
serializer_impl: add serializer for bool_class<Tag>
2016-12-14 14:10:01 +00:00
serializer.hh
Merge seastar upstream
2016-09-28 17:34:16 +03:00
sstable_mutation_readers.hh
sstables: enable fast forwarding for range readers
2016-10-19 15:29:08 +01:00
stdx.hh
Introduce stdx.hh header file
2016-08-17 11:19:49 +01:00
streamed_mutation.cc
Clean up position_in_partition.
2016-10-25 15:13:20 +01:00
streamed_mutation.hh
streamed_mutation: add memory_usage() to mutation fragment types
2016-11-18 11:25:36 +00:00
test.py
size_estimates_virtual_reader: Add unit test
2016-11-21 11:15:05 +00:00
timestamp.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
to_string.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00
tombstone.hh
db: Avoiding checking bloom filters during compaction
2016-07-10 09:54:20 +02:00
types.cc
types: fix uuid_type_impl::less
2016-09-16 11:06:55 +01:00
types.hh
db: Fix commitlog replay to not drop cell mutations with older schema
2016-12-13 12:19:32 +02:00
unimplemented.cc
unimplemented: Add materialized views
2016-10-18 01:18:52 +00:00
unimplemented.hh
unimplemented: Add materialized views
2016-10-18 01:18:52 +00:00
validation.cc
validation: Add KS validation + convinence methods
2016-04-19 11:49:05 +00:00
validation.hh
validation: Add KS validation + convinence methods
2016-04-19 11:49:05 +00:00
version.hh
Fix pre-ScyllaDB copyright statements
2016-04-08 08:12:47 +03:00

README.md

Scylla

Building Scylla

In addition to required packages by Seastar, the following packages are required by Scylla.

Submodules

Scylla uses submodules, so make sure you pull the submodules first by doing:

git submodule init
git submodule update --init --recursive

Building and Running Scylla on Fedora

  • Installing required packages:
sudo dnf install yaml-cpp-devel lz4-devel zlib-devel snappy-devel jsoncpp-devel thrift-devel antlr3-tool antlr3-C++-devel libasan libubsan gcc-c++ gnutls-devel ninja-build ragel libaio-devel cryptopp-devel xfsprogs-devel numactl-devel hwloc-devel libpciaccess-devel libxml2-devel python3-pyparsing lksctp-tools-devel protobuf-devel protobuf-compiler systemd-devel libunwind-devel
  • Build Scylla
./configure.py --mode=release --with=scylla --disable-xen
ninja-build build/release/scylla -j2 # you can use more cpus if you have tons of RAM
  • Run Scylla
./build/release/scylla
  • run Scylla with one CPU and ./tmp as data directory
./build/release/scylla --datadir tmp --commitlog-directory tmp --smp 1
  • For more run options:
./build/release/scylla --help

Building Fedora RPM

As a pre-requisite, you need to install Mock on your machine:

# Install mock:
sudo yum install mock

# Add user to the "mock" group:
usermod -a -G mock $USER && newgrp mock

Then, to build an RPM, run:

./dist/redhat/build_rpm.sh

The built RPM is stored in /var/lib/mock/<configuration>/result directory. For example, on Fedora 21 mock reports the following:

INFO: Done(scylla-server-0.00-1.fc21.src.rpm) Config(default) 20 minutes 7 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-21-x86_64/result

Building Fedora-based Docker image

Build a Docker image with:

cd dist/docker
docker build -t <image-name> .

Run the image with:

docker run -p $(hostname -i):9042:9042 -i -t <image name>

Contributing to Scylla

Guidelines for contributing

Reference in New Issue View Git Blame Copy Permalink
Description
No description provided
Readme 453 MiB
Languages
C++ 72.3%
Python 26.5%
CMake 0.3%
GAP 0.3%
Shell 0.3%