remove() is a convenient glibc wrapper that will unlink() a file and rmdir() a
directory. It will allow us to operate on both without creating a separate
function for a directory.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
There is a hidden exception that could be thrown insize file::dma_get_bulk()
if file size is not aligned with fstream::_buffer_size. In this case
file::dma_read_bulk() will be given a _buffer_size as a length for the last data
chunk in the file too. file::dma_read_bulk() will get a short read (till EOF)
and then will try to read beyond it (by calling file::read_maybe_eof())
in order to differentiate between I/O error and EOF. file::read_maybe_eof()
will throw a file::eof_error exception to indicate the EOF, it will be caught
by file::dma_read_bulk() and since we have read some "good" bytes by now this
exception won't be forwarded further. However the damage by throwing the exception
has already been done and we want to avoid this in fstream flow (unless there are
real errors).
In order to prevent the above we will always request file::dma_read_bulk() to
read the amount of data it should be able to deliver (not beyond EOF).
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Make read_maybe_eof() return zero-sized buffer when pos is at or beyond
EOF. This will prevent an internal exception throwing inside "file" class
in cases when dma_read_bulk() is going to succeed and return read data, e.g. when
it was called with "offset" less than file size but "offset" + "range_size"
beyond EOF.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Move read_state helper class, file::dma_read_bulk() and file::read_maybe_eof()
definitions outside the class declaration in order to make reading the class
interface easier.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Requiring next_fn to be available at listen() time means you can't pass
subscriptions around, and it is often hard to create next_fn, since it
usually needs to capture 'this', while this also points to the subscription
itself, creating a chicken and egg problem.
Fix by separating the registration process into two steps: listen() creates
the subscription, and start() accepts the next callback and starts processing
events.
Move the get() logic in fstream.cc into the file::dma_read_bulk()
fixing some issues:
- Fix the funny "alignment" calculation.
- Make sure the length is aligned too.
- Added new functions:
- dma_read(pos, len): returns a temporary_buffer with read data and
doesn't assume/require any alignment from either "pos"
or "len". Unlike dma_read_bulk() this function will
trim the resulting buffer to the requested size.
- dma_read_exactly(pos, len): does exactly what dma_read(pos, len) does but it
will also throw and exception if it failed to read
the required number of bytes (e.g. EOF is reached).
- Changed the names of parameters of dma_read(pos, buf, len) in order to emphasize
that they have to be aligned.
- Added a description to dma_read(pos, buf, len) to make it even more clear.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
gcc makes poor inlining decisions sometimes, which cause all the value-
tracking optimizations to be lost. Forcing it to inline (particularly
around destructors) allows ready futures to be inlined with fewer always-
true tests and data movements.
From http://en.cppreference.com/w/cpp/language/constexpr:
A constexpr specifier used in an object declaration implies const.
However, We can not change from
static constexpr const char* TIME_FORMAT = "%a %b %d %I:%M:%S %Z %Y";
to
static constexpr char* TIME_FORMAT = "%a %b %d %I:%M:%S %Z %Y";
The compiler complains:
In file included from json/formatter.cc:22:0:
json/formatter.hh:132:42: error: deprecated conversion from string
constant to ‘char*’ [-Werror=write-strings]
static constexpr char* TIME_FORMAT = "%a %b %d %I:%M:%S %Z %Y";
Since, unlike const, constexpr does not modify a type. It just applies
to an object (or function), and incidentally implies const to the
top-level type.
The test tests/memcached/test_ascii_parser hung after the change to
consume(). The problem was that consume() notified the consumer of an
EOF by sending it an empty buffer, and then it expected to get back a
message that it shouldn't read more (by setting the unconsumed buffer),
if it didn't, it continued to send empty buffers in a never-ending loops.
So this patch changes consume() to send one empty buffer to the consumer
on the end-of-file, and then stop (regardless of what the consumer returns).
It would have probably made sense to *not* send an empty buffer to the
consumer after the data is done - not even once - but if we change this
behavior, it will break the existing tests.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Tested-by: Tomasz Grabiec <tgrabiec@cloudius-systems.com>
Our input_stream::consume() mechanism allows feeding data from an input
stream into a consumer, piece by piece, until the consumer doesn't want
any more. It currently assumed the input can block (when reading from disk),
but the consumption is assumed to be immediate. This patch adds support for
blocking in the consumption function: The consumer now returns a future
which it promises to fulfill after consuming the given buffer.
This patch goes further by somewhat simplifying (?) the interface of the
consumer. Instead of receiving a mysterious "done" lambda the consumer
is supposed to call when done (doesn't want any more input), the consumer
now returns a future<optional<temporary_buffer<char>>, which means:
1. The future is fulfilled when the consumer is done with this buffer
and either wants more - or wants to stop.
2. If the consumer wants to stop, it returns the *remaining* part of the
buffer it didn't want to process (this will be pushed back into the
input stream).
3. If the consumer is not done, and wants to consume more, it returns an
unset optional.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
This patchset adds thread support to seastar. Threads can use the normal
seastar API, and may also "block" by calling future<>::get() on an
unavailable future. Synchronous and asynchronous code may be intermixed.
Threads may not issue blocking operating system calls.
swapcontext() is expensive as it invokes system calls. Replace it with
setjmp()/longjmp(). We still use setcontext() initially, since that's
the most reasonable portable method of setting up a stack.
Context switch time (measured by thread_context_switch) is reduced to
120ns (from 450ns), with inefficiencies in the test itself and in future<>
dominating.
Add a thread class that can be used to launch a blockable thread of
execution. Within a thread, future<>::get() can be called on an
unavailable future, in which case it blocks until the future is made ready.
The implementation of sleep() looks like a game of Seastar golf - doing
something in the minimum number of lines possible :-) Unfortunately, it
looks very clever, but not quite right. sleep() usually works correctly,
but the sanitizer (in the debug build) catches a use after free.
The problem was that we delete an object which contains a timer which
contains the callback (and std::function) - from inside this callback.
The workaround in this patch is to use our future chaining to only delete
the sleeper object after its future became ready - and at that point, none
of the sleeper object or code is needed any more.
This patch also includes a regression test for this issue. The test looks
silly (just sleeps and checks nothing), but in the debugging build it
failed (with a sanitizer reporting use-after-free) before this patch.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Separating the initial value (and accumulator) from the reducer function
can result in simpler invocations.
Unfortunately, the name conflicts with another variant, so we have to name
the method map_reduce0.
When submit_to() calls to a different core, and when the function to
be executed is a temporary (the usual case), we copy it to the heap for
the duration of execution. However when the function happens to execute
locally, we don't copy it, which can lead to a use-after-free if the function
defers.
Fix by detecting the case of local execution of a temporary function, and
copying it to the heap in that case.
Obviously, I was sleeping or something when I wrote the reg/unreg code, since
using copy semantics for anchors is equivalent with double unregistrations.
Luckily, unregister was broken as well, so counters did stay active. Which
however broke things once actual non-persistent counters were added. Doh.
* Anchors must be non-copyable
* Above makes creating std::vector<registration> from initializer list
tricky, so added helper type "registrations" which inherits vector<reg>
but constructs from initializer_list<type_instance_id>, avoiding illegal
copying.
* Both register and unregister were broken (map semantics does not overwrite
on insert, only [] or iterator operation).
* Modified the various registration callsites to use registrations and move
semantics.
Collectd uses an exception to signal that the buffer space in the packet
is exhausted and a new packet needs to be started. This violates the
"exceptions are for exceptional conditions" guideline, but more practically,
makes it hard to use the gdb "catch throw" command to trap exceptions.
Fix by using a data member to store the overflow condition instead.
The scollectd is an infrastructure that allows different part of the
code to register internal counters and the infrastructure would send it
periodically to an external server.
This patch adds and API to the scollectd that allows to inquire a
register value and the names of the registered values.
The collectd_value structure is used to return a single value that can
be of type: double or signed and unsigned 64 bit long.
The definition of the API are found in scollectd_api.hh
The inquiries are for the local cpu, it is up to the caller to call a
relevent cpu.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This adds a static method to return a range object to smp.
with this patch it is possible to use:
for (auto i : smp::all_cpus())
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
