The test tests/memcached/test_ascii_parser hung after the change to
consume(). The problem was that consume() notified the consumer of an
EOF by sending it an empty buffer, and then it expected to get back a
message that it shouldn't read more (by setting the unconsumed buffer),
if it didn't, it continued to send empty buffers in a never-ending loops.
So this patch changes consume() to send one empty buffer to the consumer
on the end-of-file, and then stop (regardless of what the consumer returns).
It would have probably made sense to *not* send an empty buffer to the
consumer after the data is done - not even once - but if we change this
behavior, it will break the existing tests.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Tested-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Our input_stream::consume() mechanism allows feeding data from an input
stream into a consumer, piece by piece, until the consumer doesn't want
any more. It currently assumed the input can block (when reading from disk),
but the consumption is assumed to be immediate. This patch adds support for
blocking in the consumption function: The consumer now returns a future
which it promises to fulfill after consuming the given buffer.
This patch goes further by somewhat simplifying (?) the interface of the
consumer. Instead of receiving a mysterious "done" lambda the consumer
is supposed to call when done (doesn't want any more input), the consumer
now returns a future<optional<temporary_buffer<char>>, which means:
1. The future is fulfilled when the consumer is done with this buffer
and either wants more - or wants to stop.
2. If the consumer wants to stop, it returns the *remaining* part of the
buffer it didn't want to process (this will be pushed back into the
input stream).
3. If the consumer is not done, and wants to consume more, it returns an
unset optional.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
This patchset adds thread support to seastar. Threads can use the normal
seastar API, and may also "block" by calling future<>::get() on an
unavailable future. Synchronous and asynchronous code may be intermixed.
Threads may not issue blocking operating system calls.
swapcontext() is expensive as it invokes system calls. Replace it with
setjmp()/longjmp(). We still use setcontext() initially, since that's
the most reasonable portable method of setting up a stack.
Context switch time (measured by thread_context_switch) is reduced to
120ns (from 450ns), with inefficiencies in the test itself and in future<>
dominating.
Add a thread class that can be used to launch a blockable thread of
execution. Within a thread, future<>::get() can be called on an
unavailable future, in which case it blocks until the future is made ready.
The implementation of sleep() looks like a game of Seastar golf - doing
something in the minimum number of lines possible :-) Unfortunately, it
looks very clever, but not quite right. sleep() usually works correctly,
but the sanitizer (in the debug build) catches a use after free.
The problem was that we delete an object which contains a timer which
contains the callback (and std::function) - from inside this callback.
The workaround in this patch is to use our future chaining to only delete
the sleeper object after its future became ready - and at that point, none
of the sleeper object or code is needed any more.
This patch also includes a regression test for this issue. The test looks
silly (just sleeps and checks nothing), but in the debugging build it
failed (with a sanitizer reporting use-after-free) before this patch.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Separating the initial value (and accumulator) from the reducer function
can result in simpler invocations.
Unfortunately, the name conflicts with another variant, so we have to name
the method map_reduce0.
When submit_to() calls to a different core, and when the function to
be executed is a temporary (the usual case), we copy it to the heap for
the duration of execution. However when the function happens to execute
locally, we don't copy it, which can lead to a use-after-free if the function
defers.
Fix by detecting the case of local execution of a temporary function, and
copying it to the heap in that case.
Obviously, I was sleeping or something when I wrote the reg/unreg code, since
using copy semantics for anchors is equivalent with double unregistrations.
Luckily, unregister was broken as well, so counters did stay active. Which
however broke things once actual non-persistent counters were added. Doh.
* Anchors must be non-copyable
* Above makes creating std::vector<registration> from initializer list
tricky, so added helper type "registrations" which inherits vector<reg>
but constructs from initializer_list<type_instance_id>, avoiding illegal
copying.
* Both register and unregister were broken (map semantics does not overwrite
on insert, only [] or iterator operation).
* Modified the various registration callsites to use registrations and move
semantics.
Collectd uses an exception to signal that the buffer space in the packet
is exhausted and a new packet needs to be started. This violates the
"exceptions are for exceptional conditions" guideline, but more practically,
makes it hard to use the gdb "catch throw" command to trap exceptions.
Fix by using a data member to store the overflow condition instead.
The scollectd is an infrastructure that allows different part of the
code to register internal counters and the infrastructure would send it
periodically to an external server.
This patch adds and API to the scollectd that allows to inquire a
register value and the names of the registered values.
The collectd_value structure is used to return a single value that can
be of type: double or signed and unsigned 64 bit long.
The definition of the API are found in scollectd_api.hh
The inquiries are for the local cpu, it is up to the caller to call a
relevent cpu.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds a static method to return a range object to smp.
with this patch it is possible to use:
for (auto i : smp::all_cpus())
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This change is intended to make close() flush the stream before
proceeding with the close itself. This improves the situation
where we have to guarantee that the stream is flushed before
closing it.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Allow enable_shared_from_this<T>::shared_from_this() to return
a shared_ptr<const T> when called on a const object.
Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
The foreign_ptr wrapper needs 'element_type' to be present in
shared_ptr to be able to access the data.
Fixes the following compilation failure when trying to use shared_ptr
with foreign_ptr:
In file included from tests/foreign_ptr_test.cc:24:0:
./core/distributed.hh: In instantiation of ‘class foreign_ptr<shared_ptr<basic_sstring<char, unsigned int, 15u> > >’:
tests/foreign_ptr_test.cc:28:54: required from here
./core/distributed.hh:272:56: error: no type named ‘element_type’ in ‘class shared_ptr<basic_sstring<char, unsigned int, 15u> >’
using element_type = typename PtrType::element_type;
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
small_pools, which are responsible for allocations up to 16kB, aren't able
to provide a buffer with alignment stricter than a page. This results
in aligned allocations being broken for buffers in the range 4kB - 16kB.
This patch make sure that if the alignment requirement is too big for
small_pool to handle allocate_large_aligned() is used instead.
Fixes#36.
Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
The proper way to test whether posix_memalign() failed is to check its
return value, not the content of the pointer.
This also silences "ignoring return value of 'posix_memalign()'" diagnostic
messages.
Signed-off-by: Paweł Dziepak <pdziepak@quarnos.org>
Reviewed-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
OSv does not supported timer_create/timer_settime and thread based signal handling.
This patch implements timer function using OSv native timer and timer handler thread witch pinned to the CPU same as reactor thread.
Signed-off-by: Takuya ASADA <syuu@cloudius-systems.com>
If the callback throws the program will segfault and GDB will be
useless in diagnosing the failure:
gdb$ run
...
thread_get_info_callback: cannot get thread info: generic error
gdb$
So let's fail in a better way.
Added to make clang happy, but causes compilation failure with more than
one bound value in value list. Using paranthesis instead.
Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
The current code assumes that the sstring will have the same char_type as the
output_stream. That was working well, until I was forced to change the type of
my basic_sstring to another one that is backed by signed chars.
Of course, the best solution for this would be to change the output_stream (as
well as the input_stream), to take a signed char as well.
And oh boy, have I tried. The data_sink assumes a char type, and when it tries
to allocate a new buffer from it, the buffer will have no other choice than to
be of a char type. Fix that one, and another one appears.
I eventually gave up when the code wouldn't compile because struct fragment has
a char type - and both using a template for such a simple struct, as well as
sprinkling casts all over the place where it is used, sounded like horrible
ideas to me.
It's true that quitters never win, and winners never quit. But for now, my
proposal would be to generalize the write code to accept basic_sstrings of
general types. At least the cast lives in a single place.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
All basic_sstring writes the same way. Using sstring as the signature would
require other users that are using other variants of basic_sstring to add their
own signatures.
This general version will cover those use cases as well.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
