Often a single file is used in multiple fibers, and so it is wrapped in a
lw_shared_ptr. Remove the need for this by making files internally reference
counted.
Often, when we create a directory, it is useful to make sure that the path
leading to it exists.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
file_data_source_impl always calls file::size(), which can be slow. This
slows down applications that create many short-lived input streams on the
same file (for random-access processing of a subset of the data).
Fix by not calling size(), and letting the file code handle short reads
itself.
Our debugging builds always produce the annoying false warning like:
==1199==ERROR: AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete) on 0x62300003d500
This error could be silenced by setting ASAN_OPTIONS=alloc_dealloc_mismatch=0
(and we do this in test.py), but it's still annoying for manual runs, and
can mask real bugs of this type.
The problem appears to be that the address sanitizer doesn't understand
our new(with_align(..)) trick we used to allocate the reactor object,
and when we eventually use ordinary "delete" on this object, it thinks
we mixed malloc and delete.
This patch uses posix_memalign() and free() to allocate and free the
reactor structure, and ASAN stops complaining.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
SIGALRM is used by the boost unit test library, which interferes with our
use. This causes spurious unit test failures.
Switch to a real-time signal instead.
After the implementation of the code that uses the scollectd API was
modified, the get_collectd_value gets the collectd ID as a const
reference, to remove unnessary creation of shared_ptr.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
gate_closed_exception::what() is currently a private member, thus
calling the method when handling the exception isn't possible.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Execute a function making sure it is wrapped in a gate.
This can afford to be considerably simpler than do_with, since we are not
playing any games with object creation, and the gate itself can be assumed to
be relatively long lived.
Among other things, this will allow us to make changes to enter / leave without
breaking existing code, and experiment with stronger ways of doing leave - as
Avi suggested recently.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
.then()'s return type is a complex template, which needs to be mangled into
the function's name. Move the return type into a defaulted template type
parameter, so that the entire type expression is eliminated, being replaced
by the result type.
Saves about 1% compile time and 3% object size on futures_test.o.
We can use sstring as an argument to sprint. In some cases, we already have the
string in a variable and end up being forced to call c_str() from the caller
site. Accepting sstring as a parameter makes for a cleaner interface.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
This patch adds a "class future" method for handling an exception result
of the future. It is impossible to discard a future's exception while passing
through the value of the result (what will we pass in the case of
exception?), so we discard the result as well.
An example of how this can be used, to log an error (but otherwise do
nothing) if removing a file fails:
remove_file(filename).handle_exception(
[] (std::exception_ptr eptr) {
print("Exception when deleting file: %s", eptr);
});
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Closing a file can both expose latent errors that did not have the
opportunity to be reported earlier, and also may block. While both
are unlikely in the case of O_DIRECT files, better not to risk it.
the file_data_sink_impl::put() code assumes it is always called on buffers
with size multiple of dma alignment (4096), except the *last* one. After
writing one unaligned-size buffer, further writes cannot continue because
the offset into the file no longer has the right alignment! If a caller
does try to do that, there is a bug in the caller (it's not a run-time error,
it's a design bug), and better discover it quickly with an assert, as I do
in this patch.
I had such a caller in an example application, and it took me a whole day
of debugging just to figure out that this is where the caller actually had
a bug.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
Reviewed-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Our file-output code contains various layers making sometimes contradictory
assumptions, and it is a real art-form to make it all work together.
They usually do work together well, but there was one undetected bug for
large writes to a file output stream:
The problem is what happens when we try to write a large buffer (larger
than the output stream's buffer) in one output_stream::write() call.
By default, output_stream uses the efficient, zero-copy, implementation
which calls the underlying data sink's put function on the entire written
buffer, without copying it to the stream's buffer first.
Unfortunately, this solution does NOT work on *file* output streams.
Because of our use of AIO and O_DIRECT, we can only write from aligned
buffers, and at aligned (multiple of dma_alignment) sizes. Even a large
size cannot be fully written if not a multiple of dma_alignment, and
the need to align the buffers, and data already on the output_stream,
complicate things further.
Amazingly, we already had an option "_trim_to_size" in output_stream to
do the right thing, and we just need to enable it for file output stream.
In special cases (aligned position, aligned input buffer) it might be
possible to do something even more efficient - zero copy and just one
write request - but in the general case, _trim_to_size is exactly what
we needed.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
When creating a file object, we call fstat() to determine whether it's
a block device or a regular file. While unlikely, the fstat() call can
block. Use an ioctl() that we expect to fail on regular files instead.
Add a use_count() method for shared_ptr, like std::shared_ptr has.
We already had such a method for lw_shared_ptr, but not for shared_ptr.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
When closing the read-end side of the pipe, we need to notify the writer
only if it was blocked in write() on a full pipe - because if it wasn't
blocked, it will get the broken pipe on the next write() attempt.
We used queue::abort() for that, which is fine, because this function
won't do anything if nobody is waiting on a full buffer. But we still
call std::make_exception_ptr() unconditionally, which is slow (involves
throwing an exception and catching it) and an annoying false-alarm when
trying to debug with gdb's "catch throw" (which stops on any throw).
So this patch does
if (_buf.full()) {
_buf.abort(std::make_exception_ptr(broken_pipe_exception()));
}
So that in the typical case when the buffer was not full (hopefully it
is empty), we don't do anything.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
distributed<> has the connotation of being distributed across the network.
Sharded is more in line with seastar terms.
Keep the distributed<> name and header file as an alias, for
compatibility.
Similar to std::shared_mutex, allows shared and exclusive locking of a
resource (also known as a reader/writer lock). This implementation is
strictly FIFO.
