Commit Graph

79 Commits

Author SHA1 Message Date
Tomasz Grabiec
7b7b6dd301 lsa: Don't initialize segments 2015-08-06 18:56:48 +02:00
Tomasz Grabiec
d60c04542a lsa: Consistently use segment::at() to access data 2015-08-06 18:56:48 +02:00
Tomasz Grabiec
112385455e lsa: Drop unnecessary use of __int128_t in reclaim() 2015-08-06 18:56:48 +02:00
Tomasz Grabiec
6ae0747fe5 lsa: Use size_t for sizes 2015-08-06 18:40:06 +02:00
Tomasz Grabiec
f5fdaa61a2 lsa: Avoid overflow in reclaim() 2015-08-06 16:50:15 +02:00
Tomasz Grabiec
4389bc748e lsa: Fix target calculation in reclaim() 2015-08-06 16:19:48 +02:00
Tomasz Grabiec
e9322b9803 utils: Introduce managed_bytes<> class
It's a bytes container which can be used with LSA.
2015-08-06 14:05:16 +02:00
Tomasz Grabiec
5508d9ff17 utils: Introduce managed_ref<> 2015-08-06 14:05:16 +02:00
Tomasz Grabiec
df6f0c35df utils: lsa: Add reclaimer hook which compacts regions 2015-08-06 14:05:15 +02:00
Tomasz Grabiec
5a9e296803 utils: lsa: Introduce log-structured allocator 2015-08-06 14:05:15 +02:00
Tomasz Grabiec
581eceea41 utils: Introduce allocation_strategy 2015-08-06 12:52:43 +02:00
Pekka Enberg
a520f6499f utils/UUID: Add inequality operator overloading
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-03 13:41:16 +03:00
Raphael S. Carvalho
f71932f37f utils: add exponential_backoff_retry
This is a retry mechanism that exponentially increases sleep time
between retries.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-28 17:49:45 -03:00
Amnon Heiman
b2c5e2a7cc utils: adding the latency object
The latency object is used to simplify calculating latencies. It uses a
start and stop time_point so the latency can be queried multiple time.

The start need to be done explicitely and not in the constructor to
allow reuse of the object.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-26 10:57:32 +03:00
Amnon Heiman
2b584ec2ec Adding the histogram object
The histogram object is equivalent to the Histogram used in Origin. It
collect multiple values about the data:
Count, Min, Max, Sum, variance and the sum of square that are used for
std calculation.

It also contain a sample of the last n elements, that are stored in a
circular buffer.

The histogram is used by the API to report histogram statistics.

As the API does not support unsigned integer, the count is signed.

Typically the base type of the histogram is int64_t, so ihistogram was
defined as such.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-07-26 10:55:14 +03:00
Pekka Enberg
55858137e0 utils: Clean up runtime::get_uptime() API
Return a std::chrono::steady_clock::duration and switch the caller in
migration manager to also use proper C++ durations.

Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-22 14:56:52 +03:00
Pekka Enberg
e361f2a436 utils/runtime: Add uptime helpers
The functionality is similar to RuntimeMBean.getUptime() that's needed
in schema pulling logic.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-22 13:02:43 +03:00
Asias He
fa2aee57ac utils: Move util/serialization.hh to utils/serialization.hh
Now we will not have the ugly utils and util directories, only utils.
2015-07-21 16:12:54 +08:00
Vlad Zolotarov
6017d7f6bc sequenced_set: add a non-const version of get_vector()
This is meant to allow std::moving the returned object when needed.
Otherwise std::move(s.get_vector()) will be degraded to copying.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-07-17 10:24:57 +02:00
Pekka Enberg
c003f89484 utils/UUID_gen: Add bytes_view variant of get_name_UUID()
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-16 14:53:30 +03:00
Avi Kivity
99a15de9e5 logger: de-thread_local-ize logger
The logger class constructor registers itself with the logger registry,
in order to enable dynamically setting log levels.  However, since
thread_local variables may be (and are) initialized at the time of first
use, when the program starts up no loggers are registered.

Fix by making loggers global, not thread_local.  This requires that the
registry use locking to prevent registration happening on different threads
from corrupting the registry.

Note that technically global variables can also be initialized at the
point of first use, and there is no portable way for classes to self-register.
However this is the best we can do.
2015-07-14 17:18:11 +03:00
Calle Wilund
8a8694cbbf Add naive rate limiter object
This is mostly a placeholder, since the "limiting" is rather coarse
and stuttering
2015-07-08 10:50:46 +02:00
Paweł Dziepak
9c025946f0 uuid: handle invalid strings more gracefully
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-06 23:33:58 +02:00
Paweł Dziepak
9b89bad329 utils: allow creating uuid from sstring_view
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-07-06 23:26:47 +02:00
Asias He
c58829fa42 utils: Add serialize interface to UUID 2015-06-30 10:01:23 +08:00
Vlad Zolotarov
a4041a81a3 locator: added a sequenced_set class
This class implements an add-only vector that ensures that the elements are
unique.
As long as items are only added this class does essentially the same what Java's
LinkedHashSet does.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Moved sequenced_set to its own .hh file.
2015-06-29 18:44:19 +03:00
Gleb Natapov
969134280a initial mutation clustering code 2015-06-15 12:53:10 +03:00
Vlad Zolotarov
e850a723e4 class_registrator::create(): Enforce refference arguments
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-06-09 15:33:38 +03:00
Vlad Zolotarov
f1aa0df4c3 class_registrator: ensure the static member initialization order
There was a possibility for initialization disorder of static member _classes
and its usage in another static class.

Defining the _classes inside the static method that is called when it's accessed ensures
the proper initialization (aka "standard trick", quoting Avi ;)).

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-06-09 15:33:38 +03:00
Vlad Zolotarov
73278798a9 added missing methods (stubs) required for snitch implementation
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - storage_service: add a non-const version of get_token_metadata().
   - get_broadcast_address(): check if net::get_messaging_service().local_is_initialized()
     before calling net::get_local_messaging_service().listen_address().
   - get_broadcast_address(): return an inet_address by value.
   - system_keyspace: introduce db::system_keyspace::endpoint_dc_rack
   - fb_utilities: use listen_address as broadcast_address for now
2015-06-09 15:33:29 +03:00
Paweł Dziepak
bfe6446a89 class_registrator: make no_such_class message more informative
Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-06-08 15:24:14 +03:00
Asias He
8a578a1364 utils: Add UUID(const sstring& uuid_string) constructor
Construct a UUID from a UUID string.
2015-06-04 17:12:10 +08:00
Paweł Dziepak
43d915f881 class_registrator: check whether the class exists
Without the check added in this patch if the class doesn't exist
a std::bad_function_call is thrown which is not very informative.

Signed-off-by: Paweł Dziepak <pdziepak@cloudius-systems.com>
2015-06-02 14:11:34 +02:00
Asias He
e908c6a7ed UUID: Add default constructor
Needed by std::map operator[].
2015-06-01 11:24:38 +08:00
Glauber Costa
a209a5c2cb bloom filter: go back to size for bitset
max_size, as currently used, will return -1 and while it will fix the previous
bug, it will uncover another.

We can go back to using size, as long as we make sure that all sites correctly
pick a size when creating the bitset. Aside from that, for compatibility with
the java code, the total number of bits has to be a power of two.

The best way to achieve those goals, is to just set the size ourselves through
resize() in the filter constructor. num_blocks() * bits_per_block is guaranteed
to yield a power of two as we need, and in case one caller did not explicitly
set a size, it will be set from this moment on.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-28 17:12:39 +03:00
Glauber Costa
989bc91de3 bloom filter: use unsigned quantities for signed math
We are using signed quantities to be compatible with the code java uses.
However, the current code will eventually overflow.

To avoid that, let's cast the quantities to unsigned, and then back to signed

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-26 15:50:18 -04:00
Glauber Costa
17557ca3f2 bloom filter: fix incorrect size
There is a tricky bug in our current filter implementation: is_present will
return a different value depending on the order keys are inserted.

The problem here, is that _bitmap.size() will return the maximum *currently*
used bit in the set. Therefore, when we hash a given key, the maximum bit it
sets in the bitmap is used as "max" in the expression

    results[i] = (base % max);

If the next keys do not set any bit higher than this one, everything works as
expected, because the keys will always hash the same way.

However, if one of the following keys happens to set a bit higher than the
highest bit which was set at the time a certain key was set, it will hash using
two different values of "max" in the aforementioned expression; one at insertion,
and another one at the test.

We should be using max_size() to be sure that we will always have the same
hash results.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-26 15:50:05 -04:00
Glauber Costa
ba60176d4d bloom_filter: fix index calculation to work on int, not uint
Java uses long, so we should use int64_t. Using uint64_t causes the wrong
indexes to be calculated, and therefore, the filter to respond incorrectly
to a given key.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 09:01:44 +02:00
Glauber Costa
601bb48fe7 bloom_filter: no need for a unique_ptr
I actually wrote this code before I learned that we could just return a local
vector by copy without major hassles.

Never too late for a cleanup.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-21 09:00:37 +02:00
Glauber Costa
b86218b2ef bloom filter
This comes from Origin, but the changes I had to do are quite large.
These files also represents many files, but I found it to be inconvenient
to keep all the originals, simply because we would end up with way too many
files: one .cc and one .hh per filter + an enveloping .hh so users could include
without knowing which filter to use.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-19 11:22:41 -04:00
Glauber Costa
151feb8b6e bloom_filter: bloom calculations
Helpers to calculate the various parameters of a Bloom Filter

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-19 11:22:41 -04:00
Tomasz Grabiec
5bd68aa514 data_input: Support read<bytes_view>() 2015-05-06 16:40:48 +02:00
Calle Wilund
2a14037f79 db: add data_input constructors & methods
* Construct from temporary_buffer<>
* Construct from basic_string_view<*> (cast)
* Allow skipping bytes

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-05-06 16:45:06 +03:00
Glauber Costa
cccc3c8597 data_input: allow other sizes to be used for reading a blob.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-04-30 15:49:36 -04:00
Tomasz Grabiec
b28d26bc9f murmur_hash: Drop static qualifier from template definition
To avoid multiple definitions in different compilation units. Pointed
out by Avi.
2015-04-30 16:49:50 +02:00
Tomasz Grabiec
5fc149d454 utils: murmur_hash: Add input iterator based hash3_x64_128() version 2015-04-30 11:16:53 +02:00
Tomasz Grabiec
9197914697 utils: Fix indentation in murmur_hash.hh
We don't indent namespaces.
2015-04-30 11:16:53 +02:00
Tomasz Grabiec
a9972b4b28 Relax header dependencies 2015-04-24 18:01:01 +02:00
Asias He
ea08c7e000 utils: Fix bounded_stats_deque::add 2015-04-23 14:55:26 +08:00
Tomasz Grabiec
957544f69b utils: UUID_gen: Add support for name-based UUIDs (type 3) 2015-04-17 14:19:07 +02:00