Commit Graph

29 Commits

Author SHA1 Message Date
Benny Halevy
b3d91cbf65 utils: hashing: use simple_xx_hasher
Use simple_xx_hasher for bytes_view and effective_replication_map::factory_key
appending hashers instead of their custom, yet identical implementations.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2023-04-24 14:07:25 +03:00
Kefu Chai
99bf8bc0f4 bytes, gms: s/format_to/fmt::format_to/
to disambiguate `fmt::format_to()` from `std::format_to()`. turns out,
we have `using namespace std` somewhere in the source tree, and with
libstdc++ shipped by GCC-13, we have `std::format_to()`, so without
exactly which one to use, compiler complains like

```
/optimized_clang/stage-1-X86/build/bin/clang++ -MD -MT build/dev/mutation/mutation.o -MF build/dev/mutation/mutation.o.d -I/optimized_clang/scylla-X86/seastar/include -I/optimized_clang/scylla-X86/build/dev/seastar/gen/include -U_FORTIFY_SOURCE -DSEASTAR_SSTRING -Werror=unused-result -fstack-clash-protection -DSEASTAR_API_LEVEL=6 -DSEASTAR_BUILD_SHARED_LIBS -DSEASTAR_ENABLE_ALLOC_FAILURE_INJECTION -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -DSEASTAR_TYPE_ERASE_MORE -DFMT_SHARED -I/usr/include/p11-kit-1   -ffile-prefix-map=/optimized_clang/scylla-X86=. -march=westmere -DDEVEL -DSEASTAR_ENABLE_ALLOC_FAILURE_INJECTION -DSCYLLA_ENABLE_ERROR_INJECTION -O2 -DSCYLLA_BUILD_MODE=dev -iquote. -iquote build/dev/gen --std=gnu++20  -ffile-prefix-map=/optimized_clang/scylla-X86=. -march=westmere  -DBOOST_TEST_DYN_LINK   -DNOMINMAX -DNOMINMAX -fvisibility=hidden  -Wall -Werror -Wno-mismatched-tags -Wno-tautological-compare -Wno-parentheses-equality -Wno-c++11-narrowing -Wno-missing-braces -Wno-ignored-attributes -Wno-overloaded-virtual -Wno-unused-command-line-argument -Wno-unsupported-friend -Wno-delete-non-abstract-non-virtual-dtor -Wno-braced-scalar-init -Wno-implicit-int-float-conversion -Wno-delete-abstract-non-virtual-dtor -Wno-psabi -Wno-narrowing -Wno-nonnull -Wno-uninitialized -Wno-error=deprecated-declarations -DXXH_PRIVATE_API -DSEASTAR_TESTING_MAIN -DFMT_DEPRECATED_OSTREAM  -c -o build/dev/mutation/mutation.o mutation/mutation.cc
In file included from mutation/mutation.cc:9:
In file included from mutation/mutation.hh:13:
In file included from mutation/mutation_partition.hh:21:
In file included from ./schema/schema_fwd.hh:13:
In file included from ./utils/UUID.hh:22:
./bytes.hh:116:21: error: call to 'format_to' is ambiguous
                    format_to(out, "{}{:02x}", _delimiter, std::byte(v[i]));
                    ^~~~~~~~~
./bytes.hh:134:43: note: in instantiation of function template specialization 'fmt::formatter<fmt_hex>::format<fmt::basic_format_context<fmt::appender, char>>' requested here
        return fmt::formatter<::fmt_hex>::format(::fmt_hex(bytes_view(s)), ctx);
                                          ^
/usr/include/fmt/core.h:813:64: note: in instantiation of function template specialization 'fmt::formatter<seastar::basic_sstring<signed char, unsigned int, 31, false>>::format<fmt::basic_format_context<fmt::appender, char>>' requested here
    -> decltype(typename Context::template formatter_type<T>().format(
                                                               ^
/usr/include/fmt/core.h:824:10: note: while substituting deduced template arguments into function template 'has_const_formatter_impl' [with Context = fmt::basic_format_context<fmt::appender, char>, T = seastar::basic_sstring<signed char, unsigned int, 31, false>]
  return has_const_formatter_impl<Context>(static_cast<T*>(nullptr));
```

to address this FTBFS, let's be more explicit by adding "fmt::" to
specify which `format_to()` to use.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes #13361
2023-03-29 14:47:28 +03:00
Kefu Chai
36dc2e3f28 bytes: specialize fmt::formatter<bytes>
this is a part of a series to migrating from `operator<<(ostream&, ..)`
based formatting to fmtlib based formatting. the goal here is to enable
fmtlib to print bytes with the help of fmt::ostream.

Refs #13245

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-03-27 20:06:45 +08:00
Kefu Chai
2f9dfba800 bytes: specialize fmt::formatter<fmt_hex>
this is a part of a series to migrating from `operator<<(ostream&, ..)`
based formatting to fmtlib based formatting. the goal here is to enable
fmtlib to print bytes_view with the help of fmt::ostream. because fmtlib
has its own specialization for fmt::formatter<std::basic_string_view<T>>,
we cannot just create a full specialization for std::basic_string_view<int8_t>,
otherwise fmtlib would complain that

> Mixing character types is disallowed.

so we workaround this using a delegate of fmt_hex.

Refs #13245

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-03-27 20:06:45 +08:00
Kefu Chai
ed347c5051 bytes: mark fmt_hex::v const
as fmt_hex is a helper class for formatting the underlying `bytes_view`,
it does not mutate it, so mark the member variable const and mark
the parameter in its constructor const. this change also helps us to
use fmt_hex in the use case where the const semantics is expected.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-03-27 16:49:07 +08:00
Avi Kivity
e2f6e0b848 utils: move hashing related files to utils/ module
Closes #12884
2023-02-17 07:19:52 +02:00
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Avi Kivity
1b64b1a628 bytes: compare_unsigned(): change to std::strong_ordering
Note that the previous implementation was broken for blobs
larger than 4GB. Luckily that can't happen.

Ref #1449.
2021-07-28 13:21:01 +03:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Michał Chojnowski
75898ee44e bytes: implement std::hash using appending_hash
This is a preparation for the upcoming introduction of managed_bytes_view,
intended as a fragmented replacement for bytes_view.
To ease the transition, we want both types to give equal hashes for equal
contents.
2021-01-08 13:17:46 +01:00
Avi Kivity
b59a21967c bytes: add to_bytes(bytes)
Converting from bytes to bytes is nonsensical, but it helps
when transitioning to other types (managed_bytes/managed_bytes_view),
and these types will have to_bytes() conversions.
2020-12-20 15:14:44 +01:00
Avi Kivity
30f2b3ba2f bytes: define contructor for fmt_hex
clang 10 does not implement p0960r3, so we must define a
constructor for fmt_hex.
2020-09-21 16:32:53 +03:00
Piotr Grabowski
ffd8c8c505 utf8: Print invalid UTF-8 character position
Add new validate_with_error_position function
which returns -1 if data is a valid UTF-8 string
or otherwise a byte position of first invalid
character. The position is added to exception
messages of all UTF-8 parsing errors in Scylla.

validate_with_error_position is done in two
passes in order to preserve the same performance
in common case when the string is valid.
2020-09-07 18:11:21 +03:00
Pavel Emelyanov
50e3a30dae mp_row_consumer: Provide hex-formatting wrapper for bytes_view
By default {fmt} doesn't know how to format this type (although it's a
basic_string_view instantiated), and even providing formatter/operator<<
does not help -- it anyway hits an earlier assertion in args mapper about
the disallowance of character types mixing.

The hex-wrapper with own operator<< solves the problem.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
2020-08-26 20:44:11 +03:00
Benny Halevy
ec77777bda bytes: compare_unsigned: do not pass nullptr to memcmp
If any of the compared bytes_view's is empty
consider the empty prefix is same and proceed to compare
the size of the suffix.

A similar issue exists in legacy_compound_view::tri_comparator::operator().
It too must not pass nullptr to memcmp if any of the compared byte_view's
is empty.

Fixes #6797
Refs #6814

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>

Test: unit(dev)
Branches: all
Message-Id: <20200709123453.955569-1-bhalevy@scylladb.com>
2020-07-09 17:54:46 +03:00
Duarte Nunes
fa2b0384d2 Replace std::experimental types with C++17 std version.
Replace stdx::optional and stdx::string_view with the C++ std
counterparts.

Some instances of boost::variant were also replaced with std::variant,
namely those that called seastar::visit.

Scylla now requires GCC 8 to compile.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20190108111141.5369-1-duarte@scylladb.com>
2019-01-08 13:16:36 +02:00
Avi Kivity
775b7e41f4 Update seastar submodule
* seastar d59fcef...b924495 (2):
  > build: Fix protobuf generation rules
  > Merge "Restructure files" from Jesse

Includes fixup patch from Jesse:

"
Update Seastar `#include`s to reflect restructure

All Seastar header files are now prefixed with "seastar" and the
configure script reflects the new locations of files.

Signed-off-by: Jesse Haber-Kucharsky <jhaberku@scylladb.com>
Message-Id: <5d22d964a7735696fb6bb7606ed88f35dde31413.1542731639.git.jhaberku@scylladb.com>
"
2018-11-21 00:01:44 +02:00
Vladimir Krivopalov
0f298113c7 bytes: Add helper for turning bytes_view into sstring_view.
Signed-off-by: Vladimir Krivopalov <vladimir@scylladb.com>
2018-09-25 17:23:40 -07:00
Paweł Dziepak
f9940f620a types: move compare_unsigned() to bytes.hh
compare_unsigned() is a general utility function that compares two
bytes_view byte-by-byte. There is no need to include whole type.hh in
order to make it available.
2018-05-09 16:52:26 +01:00
Avi Kivity
2edf36f863 bytes: don't allocate NUL terminator
Since bytes is used to encapsulate blobs, not strings, there's no
need for a NUL terminator. It will never be passed to a function
that expects a C string.
Message-Id: <20180401151009.14108-1-avi@scylladb.com>
2018-04-02 19:23:06 +01:00
Avi Kivity
ebaeefa02b Merge seatar upstream (seastar namespace)
- introcduced "seastarx.hh" header, which does a "using namespace seastar";
 - 'net' namespace conflicts with seastar::net, renamed to 'netw'.
 - 'transport' namespace conflicts with seastar::transport, renamed to
   cql_transport.
 - "logger" global variables now conflict with logger global type, renamed
   to xlogger.
 - other minor changes
2017-05-21 12:26:15 +03:00
Paweł Dziepak
29430ba970 bytes: add bytes_mutable_view 2017-03-02 09:05:11 +00:00
Pekka Enberg
38a54df863 Fix pre-ScyllaDB copyright statements
People keep tripping over the old copyrights and copy-pasting them to
new files. Search and replace "Cloudius Systems" with "ScyllaDB".

Message-Id: <1460013664-25966-1-git-send-email-penberg@scylladb.com>
2016-04-08 08:12:47 +03:00
Tomasz Grabiec
eb1b21eb4b Introduce hashing helpers 2016-01-08 21:10:25 +01:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Avi Kivity
30b40bf7b1 db: make bytes even more distinct from sstring
bytes and sstring are distinct types, since their internal buffers are of
different length, but bytes_view is an alias of sstring_view, which makes
it possible of objects of different types to leak across the abstraction
boundary.

Fix this by making bytes a basic_sstring<int8_t, ...> instead of using char.
int8_t is a 'signed char', which is a distinct type from char, so now
bytes_view is a distinct type from sstring_view.

uint8_t would have been an even better choice, but that diverges from Origin
and would have required an audit.
2015-04-07 10:56:19 +03:00
Tomasz Grabiec
e50c5f8305 types: Implement operator<<(ostream&, const bytes_opt&) 2015-03-30 09:07:01 +02:00
Avi Kivity
1ac75b1609 db: add to_hex(bytes_view) variant
Useful for debugging.
2015-03-16 16:36:14 +02:00
Avi Kivity
6d18aa8f20 Decompose database.hh, types.hh into smaller headers
Avoid include hell for new code.
2015-03-04 16:18:48 +02:00