When using the set_routes with the registry builder, the shared_ptr of
the registry builder should be captured.
In the original implementation the api_registery_builder captured this
parameter, but as the shared_ptr was not captured it was deleted,
causing the http to crash when running in debug.
In this implementation the method that use the registry creates a lambda
function and inside it calls the set_api_doc method, this allows to
catch the shared_ptr so it lives until the function complete.
It also replaces the c style cast to static_cast and add a FIXME note to
the routes implementation, that handlers should be deleted to prevent
memory leak.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Function handler in general and json function in particular are the
easiest way to add logic to to handler.
While in some cases the method can return immediately, there are cases
when it is required to perform an async operation, to support those
cases, the function handler was modified to use future.
If it receives an old style function, it would wrap the results in a
make_ready_future. This you could still assign a function like:
new function_handler([](const_req req) {
return "hello";
});
It would no also support a function that return a future json so it is
no possible to assign logic like:
new function_handler([](std::unique_ptr<request> req) {
return make_ready_future<json::json_return_type>("json-future");
});
For the future case note that auto-boxing still works, although you now
need to use make_ready_future.
The json_path was also modified to accept the new kind of function, to
support the common case of route definition based on the code
generation.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This demonstrate how to use a swagger definition file to create an API.
The swagger file demo.json define one api called hello_world, it has
both query and path parameters and return an object as a result.
The handler implementation simply places the given parameters in the
return object.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
File handlers support reading local file and return it as the result
for the http request.
There are two kind of handler, a file handler will return a specific
file when called. A directory handler expect to find a path parameter
called 'path' and would concatinate this parameter to the direcoty path.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Most of the time, implementing an http handler consist of a small amount
of logic.
Function handlers are a simplified way of adding such a logic, they
accept a lambda expression of various types and eliminate the need to
create a type for the handlers.
The prefered way of creating a handler is by using the
json_request_function, it would use auto-boxing to return a json object.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
The http server handlers can sometimes need to perform async operation,
specifically, when reading files from the disk. To support async
behavior the handlers handle function will return a future, that will be
propagate back, when the future will be ready it will return the reply
to be sent.
When switching from direct function call to future, there is also a need
to switch from references to pointers.
Note that while the request will be discarded, the reply will be
propagate back via the future to the caller.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
This use the routes and the reqeuest found in the http directory and
move all files but main to the http directory
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
We would like to extend the httpd capabilities and use it for the API
implementation.
The first step is to make it a library with main that calls an instanse.
This break the implementation to a header file, implementation and main,
that simply calls the implementation.
Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
Basic explanation of the reclaimer algorithm:
- Each slab page has a descriptor containing information about it, such as
refcnt, vector of free objects, link into LRU, etc.
- The LRU list will only contain slab pages which items are unused, so as
to make the reclaiming process faster and easier. Maintaining the LRU of slab
pages has a performance penalty of ~1.3%. Shlomi suggested an approach where
LRU would no longer exist and timestamp would be used instead to keep track of
recency. Reclaimer would then iterate through all slab pages checking for an
unused slab page with the lowest timestamp.
- Reclaimer will get the least-recently-used slab page from the LRU list,
do all the management stuff required, and iterate through the page erasing any
of the items there contained. Once reclaimer was called, it's likely that slab
memory usage is calibrated, thus slab pages shouldn't be allocated anymore.
- Reclaimer is enabled by default but can be disabled by specifying the slab
size using the application parameter --max-slab-size.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
* Item structure was changed to work with slab, where its last field
is used to access key, value, and ascii prefix, respectively.
* Item structured was worked to have a smaller footprint. It was
reduced from 104 bytes to 72 bytes.
* Reclaimer was temporarily disabled.
* Global LRU was removed from the cache. Now LRU eviction is done
on a per-slab-class basis, whenever the slab allocator runs out of
slab pages.
* Fragmentation issue is naturally solved with the slab algorithm,
where slab classes have chunks of the same size.
Expiration time is a 32-bit value as specified by memcached protocol.
Thus, no need to store it as a 64-bit value. When the value is needed,
convert it to the target type. Change intended to save space from
item structure.
In addition, there is no need to insert an entry into _alive when
expiration time is zero, meaning item will never expire.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Flashcached integration was done relying on certain similarities between
flashcached and memcached. Old assumptions no longer hold.
As a result, the current code is terrible to integrate the slab allocator
into it, while keeping flashcached alive.
This patch reverts flashcached from memcached. It should be re-integrated
in the future, but definitely in a better way.
httpd uses recursion for its read loop:
future<> read() {
_read_buf.consume().then([] {
...
if more work:
return read();
});
}
However, after error handling was added, it looks like this:
future<> read() {
_read_buf.consume().then([] {
...
if more work:
return read();
}).rescue(...);
}
The problem is that rescue() is called for every iteration of the loop,
instead of for the loop in its entirety. This means that a rescue
continuation is allocated for every processed request, but they will only
be called after the entire loop terminates. This results in tons of
allocated memory.
Fix by moving error handling to the end of the loop (and incidentally using
do_until() instead of recursion).
I found wrk sometimes sends RST instead a FIN to close a connection. In
this case, we will reset the connection and go to CLOSED state. However
httpd will not delete this, so we will have leaked connections in CLOSED
state.
Fix by handling the exception and sending an empty response as we do in
EOF case. Here we do not pass the exception to upper layer again,
otherwise httpd will be very noise.
