Commit Graph

27 Commits

Author SHA1 Message Date
Paweł Dziepak
9f1ebd4f7c idl/mutation: add counter serialisation logic 2017-02-02 10:35:14 +00:00
Paweł Dziepak
38ee69dee0 idl: allow writers to use any output stream
Original IDL generated code was hardcoded to always use bytes_ostream.
This patch makes the output stream a template parameter so that any
valid output stream can be used.
Unfortunately, making IDL writers generic requires updates in the code
that uses them, this is fixed in C++17 which would be able to deduce the
parameter in most cases.
2016-12-22 13:35:04 +01:00
Paweł Dziepak
249e0ab087 frozen_mutation: use memory_usage()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-11-18 11:25:36 +00:00
Paweł Dziepak
ef57b9a26f rename memory_usage() to external_memory_usage() where applicable
Renaming the function to external_memory_usage() makes it clear that
sizeof(T) is not included, something that was a source of confusion in
the past.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-11-18 11:25:36 +00:00
Paweł Dziepak
ea3ac0a270 frozen_mutation: reduce chunk count in constructor
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
1daf4c73a3 frozen_mutation: avoid buffer linearization and copy
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
3707d7fec3 frozen_mutation: use bytes_ostream internally
Unlike bytes, bytes_ostream supports fragmented buffers, thus reducing
the pressure on the memory allocator caused by large frozen partitions.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
c0425b63ff frozen_mutation: add mutation_view()
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-08-22 09:31:33 +01:00
Paweł Dziepak
93cc4454a6 streamed_mutation: emit range_tombstones directly
Originally, streamed_mutations guaranteed that emitted tombstones are
disjoint. In order to achieve that two separate objects were produced
for each range tombstone: range_tombstone_begin and range_tombstone_end.

Unfortunately, this forced sstable writer to accumulate all clustering
rows between range_tombstone_begin and range_tombstone_end.

However, since there is no need to write disjoint tombstones to sstables
(see #1153 "Write range tombstones to sstables like Cassandra does") it
is also not necessary for streamed_mutations to produce disjoint range
tombstones.

This patch changes that by making streamed_mutation produce
range_tombstone objects directly.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-13 09:51:18 +01:00
Paweł Dziepak
19629e95e2 frozen_mutation: add fragment_add_freeze()
fragment_and_freeze() produces a stream of frozen mutations from a
single streamed_mutation.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-07-07 12:18:30 +01:00
Paweł Dziepak
b70bf086b7 frozen_mutation: handle reversed streams properly
Freezing streamed_mutations assumed that mutation fragments are streamed
in the order they appear in the frozen mutation. That is not true for
reversed streams.

Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
Message-Id: <1467277069-18702-1-git-send-email-pdziepak@scylladb.com>
2016-06-30 11:26:45 +02:00
Paweł Dziepak
6e68f0931e frozen_mutation: freeze streamed_mutations
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-06-20 21:29:52 +01:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Paweł Dziepak
bdc23ae5b5 remove db/serializer.hh includes
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-03-02 09:07:09 +00:00
Tomasz Grabiec
697d9bfa56 serializer: Introduce as_input_stream(bytes_view) 2016-02-26 12:26:13 +01:00
Paweł Dziepak
4f3ee7abbc frozen_mutation: use IDL-based serialization
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:51:17 +00:00
Paweł Dziepak
ccd29bf7a7 frozen_mutation: switch to bytes_ostream
Signed-off-by: Paweł Dziepak <pdziepak@scylladb.com>
2016-02-19 21:47:54 +00:00
Gleb Natapov
b9b6f703c3 Remove old serializer for frozen_mutation and reconcilable_result 2016-01-24 12:45:41 +02:00
Tomasz Grabiec
f9ae1ed1c6 frozen_mutation: Add schema_version field 2016-01-11 10:34:51 +01:00
Tomasz Grabiec
13974234a4 db/serializer: Spread serializers to relax header dependencies 2016-01-08 21:10:26 +01:00
Tomasz Grabiec
09188bccde mutation_query: Make reconcilable_result printable 2015-12-16 18:06:54 +01:00
Tomasz Grabiec
305c2b0880 frozen_mutation: Introduce decorated_key() helper
Requested by Asias for use in streaming code.
2015-12-15 15:16:04 +02:00
Tomasz Grabiec
f0cfa61968 Relax header dependencies 2015-12-03 09:10:02 +01:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Tomasz Grabiec
f537c0f0ac mutation_partition_serializer: Refactor the interface
write() will now write framing information. It will be needed when
serializing many mutations. The old version was left as
write_without_framing().

This change also makes the serializer interface closer to that of
db::serializer<>. To write an object, the serializer instance is
instantiated with the object and remembers the size, so that size
calculation doesn't have to be repeated again for write().
2015-07-04 13:46:31 +02:00
Gleb Natapov
a4c6b4a53f drop frozen_mutation::from_bytes() in favor of explicit constructor 2015-06-15 12:51:09 +03:00
Tomasz Grabiec
4ab66de0ae db: Introduce frozen_mutation
The immediate motivation for introducing frozen_mutation is inability
to deserialize current "mutation" object, which needs schema reference
at the time it's constructed. It needs schema to initialize its
internal maps with proper key comparators, which depend on schema.

frozen_mutation is an immutable, compact form of a mutation. It
doesn't use complex in-memory strucutres, data is stored in a linear
buffer. In case of frozen_mutation schema needs to be supplied only at
the time mutation partition is visited. Therefore it can be trivially
deserialized without schema.
2015-05-08 09:19:01 +02:00