Commit Graph

278 Commits

Author SHA1 Message Date
Botond Dénes
2e09a0317c types, compound: pass std::current_exception() to on_internal_error()
So that  nested exceptions are not lost. Also, marshal exceptions, the
ones we have in these places, already have a backtrace, so might as well
use that, instead of creating a new one, loosing unwound frames.

Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20200507091405.244544-1-bdenes@scylladb.com>
2020-05-07 11:25:25 +02:00
Pavel Solodovnikov
f6e765b70f cql3: pass column_specification via lw_shared_ptr
`column_specification` class is marked as "final": it's safe
to use non-polymorphic pointer "lw_shared_ptr" instead of a
more generic "shared_ptr".

tests: unit(dev, debug)

Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
Message-Id: <20200427084016.26068-1-pa.solodovnikov@scylladb.com>
2020-04-27 12:47:42 +03:00
Rafael Ávila de Espíndola
fbcf741c2d cql functions: Use switch to find the cast function to use
This produces more compact code and avoids the anti-pattern of
building a map with statically known values. If the values are given
to GCC via a switch statement it can do a much better job at compile
time than libstdc++ can at runtime.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20200422224905.198794-1-espindola@scylladb.com>
2020-04-23 11:46:09 +03:00
Botond Dénes
9e1d6ada0f types: compare(): cover more paths with on_internal_error()
Currently we call `on_internal_error()` if `tri_compare()` throws
`marshal_exception`. Some compare paths however might go around
`tri_compare()` and call `abstract_type::compare()` directly. Move the
check there to cover these cases too.

Tests: dev
Signed-off-by: Botond Dénes <bdenes@scylladb.com>
Message-Id: <20200403162530.1175801-1-bdenes@scylladb.com>
2020-04-03 18:35:30 +02:00
Rafael Ávila de Espíndola
80d969ce31 everywhere: Use uninitialized_string instead of sstring::initialized_later
This is just a trivial wrapper over initialized_later when using
sstring, but also works when std::string is used.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2020-03-10 13:17:49 -07:00
Avi Kivity
5dee627f73 types: forward-declare multiprecision_int
This reduces the number of translation units that depend on
boost/multiprecision from 354 to 30, and reduces the size of
database.i (as an example) from 406160 to 382933 (smaller
files will benefit more, relatively).

Ref #1
2020-03-04 13:28:16 +02:00
Avi Kivity
3c772757c0 treewide: use utils::multiprecision_int for varint implementation
The goal is to forward-declare utils::multiprecision_int, something
beyond my capabilities for boost::multiprecision::cpp_int, to reduce
compile time bloat.

The patch is mostly search-and-replace, with a few casts added to
disambiguate conversions the compiler had trouble with.
2020-03-04 13:28:16 +02:00
Rafael Ávila de Espíndola
93de9597bf types: Add more data_value constructors
With this we can construct a data_value from any string type. This
also avoids a few sstring copies.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2020-02-28 08:36:27 -08:00
Benny Halevy
b31867eafa types: tri_compare: turn marshal_exception to on_internal_error
We see this exception on gemini testing with large number of pk, ck, columns, for example:
  2020-02-19T17:52:54+00:00  gemini-8h-large-num-columns-GeminiL-db-node-f2d6a8e0-3 !ERR     | scylla: [shard 0] storage_proxy - Exception when communicating with 10.0.207.169: std::runtime_error (marshaling error: read_simple_exactly - size mismatch (expected 4, got 1) Backtrace:   0x2c4f08d#012  0x9fcd3e#012  0x444b28#012  0x4d8fe5#012  0xa78e8b#012  0xeab269#012  0xc27a67#012  0xc28239#012  0xc600e3#012  0xadebf3#012  0xae14c1#012  0x29ff291#012  0x29ff49f#012  0x2a3fc65#012  0x29a5d6f#012  0x29a6e9e#012  0x72a4e3#012  /opt/scylladb/libreloc/libc.so.6+0x271a2#012  0x77548d#012)

Decoded backtrace:
  seastar::current_backtrace() at crtstuff.c:?
  seastar::internal::backtraced<marshal_exception>::backtraced<seastar::basic_sstring<char, unsigned int, 15u, true> >(seastar::basic_sstring<char, unsigned int, 15u, true>&&) at crtstuff.c:?
  void seastar::throw_with_backtrace<marshal_exception, seastar::basic_sstring<char, unsigned int, 15u, true> >(seastar::basic_sstring<char, unsigned int, 15u, true>&&) at crtstuff.c:?
  abstract_type::compare(std::basic_string_view<signed char, std::char_traits<signed char> >, std::basic_string_view<signed char, std::char_traits<signed char> >) const [clone .cold] at types.cc:?
  bound_view::tri_compare::operator()(clustering_key_prefix const&, int, clustering_key_prefix const&, int) const at crtstuff.c:?
  sstables::sstable_mutation_reader<sstables::data_consume_rows_context_m, sstables::mp_row_consumer_m>::fast_forward_to(position_range, std::chrono::time_point<seastar::lowres_clock, std::chrono::duration<long, std::ratio<1l, 1000l> > >) at crtstuff.c:?
  mutation_reader_merger::fast_forward_to(position_range, std::chrono::time_point<seastar::lowres_clock, std::chrono::duration<long, std::ratio<1l, 1000l> > >) at crtstuff.c:?
  combined_mutation_reader::fast_forward_to(position_range, std::chrono::time_point<seastar::lowres_clock, std::chrono::duration<long, std::ratio<1l, 1000l> > >) at crtstuff.c:?
  restricting_mutation_reader::fast_forward_to(position_range, std::chrono::time_point<seastar::lowres_clock, std::chrono::duration<long, std::ratio<1l, 1000l> > >) at crtstuff.c:?
  cache::cache_flat_mutation_reader::do_fill_buffer(std::chrono::time_point<seastar::lowres_clock, std::chrono::duration<long, std::ratio<1l, 1000l> > >) at crtstuff.c:?

This patch should help us get a core dump if this happens again.

Ref #5856

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20200227131939.388770-1-bhalevy@scylladb.com>
2020-02-28 07:57:13 +02:00
Rafael Ávila de Espíndola
bd93a0af52 types: Return bytes_opt from data_value::serialize
Since a data_value can contain a null value, returning bytes from
serialize() was losing information as it was mapping null to empty.

This also introduces a serialize_nonnull that still returns bytes, but
results in an internal error if called with a null value.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2020-01-29 14:04:59 -08:00
Rafael Ávila de Espíndola
02e8e8d6b3 types: Move operator== for data_value out-of-line
Most of the work is done by decompose and compare which are
out-of-line anyway.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2020-01-29 13:24:10 -08:00
Rafael Ávila de Espíndola
cfb079b2c9 types: Refactor duplicated value_cast implementation
The two implementations of value_cast were almost identical.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20191120181213.111758-3-espindola@scylladb.com>
2019-11-24 11:35:29 +02:00
Rafael Ávila de Espíndola
5417c5356b types: Move get_castas_fctn to cql3
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20191120181213.111758-9-espindola@scylladb.com>
2019-11-21 12:08:50 +02:00
Rafael Ávila de Espíndola
786b1ec364 types: Move json code to its own file
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20191120181213.111758-7-espindola@scylladb.com>
2019-11-21 12:08:49 +02:00
Rafael Ávila de Espíndola
ed65e2c848 types: Move cql3_kind to the cql3 directory
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20191120181213.111758-5-espindola@scylladb.com>
2019-11-21 12:08:47 +02:00
Rafael Ávila de Espíndola
bd560e5520 types: Fix dynamic types of some data_value objects
I found these mismatched types while converting some member functions
to standalone functions, since they have to use the public API that
has more type checks.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
Message-Id: <20191120181213.111758-4-espindola@scylladb.com>
2019-11-21 12:08:46 +02:00
Avi Kivity
1fe062aed4 Merge "Add basic UDF support" from Rafael
"

This patch series adds only UDF support, UDA will be in the next patch series.

With this all CQL types are mapped to Lua. Right now we setup a new
lua state and copy the values for each argument and return. This will
be optimized once profiled.

We require --experimental to enable UDF in case there is some change
to the table format.
"

* 'espindola/udf-only-v4' of https://github.com/espindola/scylla: (65 commits)
  Lua: Document the conversions between Lua and CQL
  Lua: Implement decimal subtraction
  Lua: Implement decimal addition
  Lua: Implement support for returning decimal
  Lua: Implement decimal to string conversion
  Lua: Implement decimal to floating point conversion
  Lua: Implement support for decimal arguments
  Lua: Implement support for returning varint
  Lua: Implement support for returning duration
  Lua: Implement support for duration arguments
  Lua: Implement support for returning inet
  Lua: Implement support for inet arguments
  Lua: Implement support for returning time
  Lua: Implement support for time arguments
  Lua: Implement support for returning timeuuid
  Lua: Implement support for returning uuid
  Lua: Implement support for uuid and timeuuid arguments
  Lua: Implement support for returning date
  Lua: Implement support for date arguments
  Lua: Implement support for returning timestamp
  ...
2019-11-17 16:38:19 +02:00
Piotr Dulikowski
6d78d7cc69 cql3/functions: add missing min/max/count for ascii
Adds missing overloads of functions `count`, `min`, `max` for
type `ascii`. Now they work:

cqlsh> CREATE KEYSPACE ks WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
cqlsh> USE ks;
cqlsh:ks> CREATE TABLE test_ascii (id int PRIMARY KEY, value ascii);
cqlsh:ks> INSERT INTO test_ascii (id, value) VALUES (0, 'abcd');
cqlsh:ks> INSERT INTO test_ascii (id, value) VALUES (1, 'efgh');
cqlsh:ks> INSERT INTO test_ascii (id, value) VALUES (2, 'ijkl');
cqlsh:ks> SELECT * FROM test_ascii;

 id | value
----+-------
  1 |  efgh
  0 |  abcd
  2 |  ijkl

(3 rows)
cqlsh:ks> SELECT count(value) FROM test_ascii;

 system.count(value)
---------------------
                   3

(1 rows)
cqlsh:ks> SELECT min(value) FROM test_ascii;

 system.min(value)
-------------------
              abcd

(1 rows)
cqlsh:ks> SELECT max(value) FROM test_ascii;

 system.max(value)
-------------------
              ijkl

(1 rows)

Tests:
- unit(release)
- cql_group_functions_tests.py (with added check for `ascii` type)

Fixes #5147.
2019-11-12 13:01:34 +01:00
Rafael Ávila de Espíndola
5cef5a1b38 types: Add a friend visitor over data_value
This is a simple wrapper that allows code that is not in the types
hierarchy to visit a data_value.

Will be used by UDF.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-11-07 08:19:52 -08:00
Kamil Braun
abe6c2d3d2 types: introduce to_bytes_opt_vec function.
It converts a vector<bytes_view_opt> to a vector<bytes_opt>.
Used in a bunch of places.
2019-10-25 12:04:44 +02:00
Kamil Braun
a8c7670722 types: add multi_cell field to user_type_impl.
is_value_compatible_with_internal and update_user_type were generalized
to the non-frozen case.

For now, all user_type_impls in the code are non-multi-cell (frozen).
This will be changed in future commits.
2019-10-25 12:04:44 +02:00
Konstantin Osipov
a2b629c3a1 lwt: boost update_parameters to serve as a CAS result set
In modification_statement/batch_statement, we need to prefetch data to
1) apply list operations
2) evaluate CAS conditions
3) return CAS result set.

Boost update_parameters::prefetch_data to serve as a single result set
for all of the above. In case of a batch, store multiple rows for
multiple clustering keys involved in the batch.

Use an ordered set for columns and rows to make sure 3) CAS result set
is returned to the client in an ordered manner.

Deserialize the primary key and add it to result set rows since
it is returned to the client as part of CAS result set.

Index columns using ordinal_id - this allows having a single
set for all columns and makes columns easy to look up.

Remove an extra memcpy to build view objects when looking
up a cell by primary key, use partition_key/clustering_key
objects for lookup.
2019-10-16 15:56:50 +03:00
Piotr Sarna
10f90d0e25 types: remove deprecated comment
The comment does not apply anymore, as this definition is no more
in database.hh.
Message-Id: <a0b6ff851e1e3bcb5fcd402fbf363be7af0219af.1569580556.git.sarna@scylladb.com>
2019-09-27 19:32:17 +02:00
Rafael Ávila de Espíndola
01cd21c04d types: Remove timestamp_native_type
Now that we know that anything expecting a date_type has been
converted to date_type_native_type, switch to using
db_clock::time_point when we want a timestamp_type.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-09-07 10:07:33 -07:00
Rafael Ávila de Espíndola
df6c2d1230 types: Don't specialize data_type_for for db_clock::time_point
This also moves every user to date_type_native_type. A followup patch
will convert to timestamp_type when appropriate.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-09-07 10:07:33 -07:00
Rafael Ávila de Espíndola
e09fa2dcff types: Make it harder to create date_type
date_type was replaced with timestamp_type, but it was very easy to
create a date_type instead of a timestamp_type by accident.

This patch changes the code so that a date_type is no longer
implicitly used when constructing a data_value. All existing code that
was depending on this is converted to explicitly using
date_type_native_type. A followup patch will convert to timestamp_type
when appropriate.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-09-07 10:07:33 -07:00
Rafael Ávila de Espíndola
e0065b414e types: Avoid shared_ptr copies
They are somewhat expensive (in code size at least) and not needed
everywhere.

Inside the getter the variables are 'const data_type&', so we can
return that. Everything still works when a copy is needed, but in code
that just wants to check a property we avoid the copy.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-09-03 07:43:35 -07:00
Rafael Ávila de Espíndola
1cb7127bf3 types: Make abstract_type::serialize a static helper
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
b175657ee7 types: Devirtualize abstract_type::validate
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
bf96f1111c types: Make abstract_type::serialized_size a static helper
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:41 -07:00
Rafael Ávila de Espíndola
6831e05471 types: Move functions that use abstract_type::serialized_size out of line
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
047e34a31d types: Remove serialize_value
It is no longer needed.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
1e0663c56c types: Devirtualize abstract_type::from_string
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
68b26047cc types: Devirtualize abstract_type::serialize
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
18da5f9001 types: Devirtualize abstract_type::from_json_object
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
b987b2dcbe types: Devirtualize abstract_type::to_json_string
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
968365b7e3 types: Devirtualize abstract_type::serialized_size
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
793bc50d69 types: Delete abstract_type::validate_collection_member
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
37686964f0 types: Devirtualize abstract_type::hash
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
396f5c7656 types: Devirtualize abstract_type::native_typeid
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
492043a77d types: Devirtualize abstract_type::native_value_delete
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
4d849d7742 types: Devirtualize abstract_type::native_value_clone
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
ba887b7e56 types: Delete abstract_type::native_value_destroy
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
5c0e78d70c types: Delete abstract_type::native_value_move
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
2bc6471a1e types: Delete abstract_type::native_value_copy
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
33394dfdc1 types: Delete abstract_type::native_value_size
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
c22ca2f9c9 types: Delete abstract_type::native_value_alignment
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
37c0f5b985 types: Devirtualize get_string
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
f633f70616 types: Devirtualize abstract_type::is_value_compatible_with_internal
It now is a static helper.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
19c9a033d9 types: Devirtualize abstract_type::is_compatible_with
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00