Commit Graph

43077 Commits

Author SHA1 Message Date
Avi Kivity
6b84f666f7 smp: move smp::listen_one() into smp_message_queue class
All it does is access its internals.
2014-10-23 13:16:50 +03:00
Avi Kivity
1bdc2a1f6c smp: asynchronous functions
Allow functions run on another cpu to return a future.  In that case, the
result (or exception) is only returned when the future is resolved.

Note this has the potential for livelocks:

A: smp::submit_to(B, [] {
       ...
       return smp::submit_to(A, [] {
           ...
       });
    });

If this and its mirror image (B sending to A) happen concurrently, and if
both the A->B and B->A queue become full, the system will not be able to
make forward progress.  This can be fixed by making the inner submit_to()
use a different queue.
2014-10-23 13:04:52 +03:00
Asias He
e2c2580e81 tcp: Fix ACK with both data and FIN
local  send: <FIN>
remote send: <ACK>

In response to local <FIN> packet, remote can send <ACK> packet acking
both data and FIN.

Avoid consuming extra one byte in data handling.
2014-10-23 12:57:33 +03:00
Asias He
c3d225eb5f httpd: Support close initiated on server side
This makes ab work:

$ ab -n 1000 http://ip:10000/
2014-10-23 12:57:32 +03:00
Asias He
d12e495653 tcp: Support close initiated on server side 2014-10-23 12:57:32 +03:00
Tomasz Grabiec
cec8b6c5de core: fix SIGSEGV in packet::packet(fragment frag, packet&& x)
We move from x._impl in the initializer list, so it will hold nullptr
later.
2014-10-23 11:31:03 +03:00
Asias He
502ecb8485 Ignore cscope files 2014-10-23 10:46:55 +03:00
Asias He
0b3fc256b6 httpd: Drop dead code 2014-10-23 10:30:40 +03:00
Asias He
b7ab4621fe httpd: Allow tune the http server port 2014-10-23 10:30:40 +03:00
Asias He
9aaaded894 app template: Put options for application in app options group
E.g.:

App options:
  --help                                show help message
  --port arg (=10000)                   HTTP Server port
2014-10-23 10:30:40 +03:00
Avi Kivity
1f34dc4b2a syscall_work_queue: remove submit<void>
Not used with syscalls - they almost always return a result.
2014-10-23 10:19:55 +03:00
Avi Kivity
4b020f3874 smp: split inter_thread_work_queue
inter_thread_work_queue now serves two purposes: a minimal work queue
for running blocking syscalls in a thread context outside the main reactor,
and for passing messages between reactors on different threads.

In order to evolve the inter_thread_work_queue's second task, without harming
the first, split it into two classes, syscall_work_queue and
smp_message_queue. Currently they are exactly equivalent.
2014-10-23 10:13:56 +03:00
Tomasz Grabiec
a8828fdf8c core: use standard hash for sstring
Turns out that the current hash implementation yields very bad
hashes. This change improves memcache throughput by 70% on
muninn/huginn.
2014-10-22 18:51:55 +03:00
Gleb Natapov
544a990f80 timer_set: fix expired time cancellation
Expired timer cancellation is broken since timer_set assumes that a
timer that is being canceled is not expired yet. This patch fixes the
problem by moving expired timers management outside timer_set and
letting the code that uses it to managed cancellation of expired timers.
In memcached it can never happen, in core each queued timer gets expired
flag that tells if timer is queued in a timer set or in expired timer
list.
2014-10-22 18:27:00 +03:00
Tomasz Grabiec
a8c64386b2 memcache: enable hash caching and hash comparison
std::unordred_map does it by default for std::string. We switched to
boost::intrusive::unordered_set<>, which does not do it by default for
any key. I see ~4% improvement in throughput with this.
2014-10-22 17:41:57 +03:00
Asias He
aa06198f0a tcp: Remove tcb from tcbs when connection is closed
Currently tcb are inserted but never removed from tcbs.

This patch also removes an unnecessary <ACK> packet (packet #5 below) to
the client.

1) client: <FIN>
2) server: <ACK>
3) server: <FIN>
4) client: <ACK>
5) server: <ACK>
2014-10-22 16:39:49 +03:00
Avi Kivity
05effc0b2d array_map: fix initialization
Force value initialization of the array, rather than the default (default
initialization) which does nothing for primitive types such as pointers.
2014-10-22 16:37:36 +03:00
Calle Wilund
40db2c0ba1 Collectd 'daemon' module
Per-cpu value list registry with polling -> udp send

- Allows registration of metric values associated with
  collectd id path (plugin/[plugin-inst/]type[/type-instance]).
- Values are broadcast/sent at periodic intervals. (config)
- Config through seastar.conf / app-template.
- Value registration can be revoked safely, either manually or
  through anchor.

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2014-10-22 12:01:16 +03:00
Asias He
8b737c95e5 core: Add use_count for shared_ptr
This is useful when debugging shared_ptr.
2014-10-22 11:40:18 +03:00
Asias He
6561bde964 net: Add TCP option support
Maximum segment size and Window scale option are supported currently.
2014-10-22 10:28:06 +03:00
Asias He
5d19306435 httpd: Delete connection when client closes
When client closes the connection in one direction, make httpd close the
other direction too. This way, httpd will send back a <FIN> packet to
client after receiving client's <FIN> packet.
2014-10-22 10:27:27 +03:00
Avi Kivity
3ce9762fcd Merge branch 'memcache' of github.com:cloudius-systems/seastar-dev
"memcache: implement memory reclamation" from Tomasz.
2014-10-21 18:02:51 +03:00
Tomasz Grabiec
1d439ec2b6 memcache: try to handle hashtable resize failure
Big hashtables may fail to resize. Let's not crash when this happens
but evict some items and continue.
2014-10-21 16:58:35 +02:00
Tomasz Grabiec
137d465b72 memcache: hook up memory reclaimer
It implements a very simple reclaimer. When triggered tries to free
around 5 MB of data. This assumes that this amount is more than anyone
would want to allocate between low-memory event is detected and the
reclaimer task runs. This amount was chosen arbitrarily and I am not
sure if this amount is right.
2014-10-21 16:58:35 +02:00
Tomasz Grabiec
9478601abf core: introduce units.hh 2014-10-21 16:58:35 +02:00
Tomasz Grabiec
1a84e3560c core: implement "reclaimer" instance methods when running with default allocator 2014-10-21 16:43:09 +02:00
Tomasz Grabiec
bd31ea1f6a memcache: add stats for expired items 2014-10-21 16:43:09 +02:00
Tomasz Grabiec
0f1c499154 memcache: track items in an LRU list 2014-10-21 16:43:09 +02:00
Tomasz Grabiec
658fe77afa memcache: switch cache to use boost::intrusive::unordered_set
This will allow us to have LRU evictions and removal of expired items
which don't involve key lookups.
2014-10-21 16:43:09 +02:00
Tomasz Grabiec
4ac3c1d1e1 core: add to_sstring(sstring) overload
So that we don't need to special case templates.
2014-10-21 16:43:09 +02:00
Tomasz Grabiec
3eaac67a09 tests: memcache: make test_connection_statistics a bit more robust
It some times fails because the server records connection closure with
a delay.
2014-10-21 16:43:09 +02:00
Tomasz Grabiec
f921a9787b tests: make test_memcache.py return non-zero exit code on failure 2014-10-21 16:43:09 +02:00
Pekka Enberg
6723c0cbdd apps/memcache => apps/memcached
The protocol is called the "memcache protocol" but the server should
follow the same naming convention as httpd does.

It should not be a big deal but it annoys the hell out of simple people
like myself who have their brain hard-wired to type the final "d"...

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2014-10-21 17:30:14 +03:00
Avi Kivity
d1c8d6d64a Merge branch 'deleter'
"Deleter objects are relatively heavyweight since they need to remember
which destructor to call.  However, raw memory needs no destructor, and
we can exploit this fact."
2014-10-21 15:50:53 +03:00
Gleb Natapov
a1533cc671 memory: call MADV_DONTDUMP on the whole mmaped memory
We mmap 2*alloc bytes not alloc.
2014-10-21 12:20:22 +03:00
Avi Kivity
9cecc2ce79 Add .gitorderfile
With

  git config diff.orderfile .gitorderfile

patches become more readable, as header changes will come before source
changes.
2014-10-21 12:08:00 +03:00
Avi Kivity
b8b39499ff smp: cleanups
"smp" is always defined, no need to test for it.
2014-10-21 12:07:30 +03:00
Avi Kivity
5a12154f3a core: fix memory initialization order
Initizalize memory before initializing the reactor object, which might
itself require a lot of memory (e.g. the virtio ring).
2014-10-21 11:46:12 +03:00
Avi Kivity
91782ac6a2 virtio: optimize single-buffer packet deleter
Instead of allocating a vector to store the buffers to be destroyed, in the
case of a single buffer, use an ordinary free deleter.

This doesn't currently help much because the packet is share()d later on,
but if we may be able to eliminate the sharing one day.
2014-10-21 11:27:05 +03:00
Avi Kivity
61782fcc05 packet: add a vectored constructor with a deleter
Add packet(Iterator, Iterator, deleter).

(unfortunately we have both a template version with a template parameter
named Deleter, and a non-template version with a parameter called deleter.
Need to sort the naming out).
2014-10-21 11:24:27 +03:00
Avi Kivity
fe8ed68878 temporary_buffer: use raw object deleter to destroy allocated buffer
Removes an allocation.
2014-10-21 11:23:41 +03:00
Avi Kivity
f75d1822cc core: special-case deleter for raw memory
In many cases, a deleter is used to protect raw memory (e.g. a char array,
not something with a destructor).  In that case we can simply free() it,
so, the deleter need not remember which destructor needs to be called.

It does need to remember whether it's a raw object or not, so we take over
the least significant bit and use it as a marker, and store the pointer
to the object in the deleter, instead of using a proxy impl object to
control actual deletion.

If the deleter is subsequently share()d, we have to convert it back to
the standard form, since the reference count lives in the impl object.
2014-10-21 11:20:12 +03:00
Avi Kivity
5a6771ee33 Drop eclipse workaround for reference qualified member functions
Latest eclipse understands it.
2014-10-19 17:42:47 +03:00
Avi Kivity
7ac12f4839 Merge branch 'virtio'
Remove allocations from the virtio receive and transmit paths.
2014-10-19 10:58:30 +03:00
Glauber Costa
79b9053751 net: parse ethernet address from string
Given a string, return the corresponding ethernet address. This is useful
specially for xen, where we read the mac address from the xenstore.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2014-10-19 10:55:26 +03:00
Raphael S. Carvalho
932a6e8752 core: move file-specific content from reactor.hh to specialized header
No semantic change; just cleaning things up.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2014-10-18 16:42:08 +03:00
Avi Kivity
60b70974d4 Merge branch 'memcache' of github.com:cloudius-systems/seastar-dev
Memcache updates from Tomasz:

"memcache: towards ASCII protocol completeness

Contains also some bug fixes."
2014-10-18 16:33:20 +03:00
Tomasz Grabiec
7903acdeb8 tests: memcache: add incr/decr tests 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
f00bc22067 memcache: add support for 'incr' and 'decr' commands 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
3093e2c543 memcache: make data changes atomic
Transmission of data is not atomic so we cannot replace item's fields
in-place. This can lead to inconsistencies in "get" responses. We
should rather allocate a new item object replacing the one which is in
the cache.
2014-10-18 12:59:59 +02:00