Commit Graph

555 Commits

Author SHA1 Message Date
Tomasz Grabiec
1d439ec2b6 memcache: try to handle hashtable resize failure
Big hashtables may fail to resize. Let's not crash when this happens
but evict some items and continue.
2014-10-21 16:58:35 +02:00
Tomasz Grabiec
137d465b72 memcache: hook up memory reclaimer
It implements a very simple reclaimer. When triggered tries to free
around 5 MB of data. This assumes that this amount is more than anyone
would want to allocate between low-memory event is detected and the
reclaimer task runs. This amount was chosen arbitrarily and I am not
sure if this amount is right.
2014-10-21 16:58:35 +02:00
Tomasz Grabiec
9478601abf core: introduce units.hh 2014-10-21 16:58:35 +02:00
Tomasz Grabiec
1a84e3560c core: implement "reclaimer" instance methods when running with default allocator 2014-10-21 16:43:09 +02:00
Tomasz Grabiec
bd31ea1f6a memcache: add stats for expired items 2014-10-21 16:43:09 +02:00
Tomasz Grabiec
0f1c499154 memcache: track items in an LRU list 2014-10-21 16:43:09 +02:00
Tomasz Grabiec
658fe77afa memcache: switch cache to use boost::intrusive::unordered_set
This will allow us to have LRU evictions and removal of expired items
which don't involve key lookups.
2014-10-21 16:43:09 +02:00
Tomasz Grabiec
4ac3c1d1e1 core: add to_sstring(sstring) overload
So that we don't need to special case templates.
2014-10-21 16:43:09 +02:00
Tomasz Grabiec
3eaac67a09 tests: memcache: make test_connection_statistics a bit more robust
It some times fails because the server records connection closure with
a delay.
2014-10-21 16:43:09 +02:00
Tomasz Grabiec
f921a9787b tests: make test_memcache.py return non-zero exit code on failure 2014-10-21 16:43:09 +02:00
Pekka Enberg
6723c0cbdd apps/memcache => apps/memcached
The protocol is called the "memcache protocol" but the server should
follow the same naming convention as httpd does.

It should not be a big deal but it annoys the hell out of simple people
like myself who have their brain hard-wired to type the final "d"...

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2014-10-21 17:30:14 +03:00
Avi Kivity
d1c8d6d64a Merge branch 'deleter'
"Deleter objects are relatively heavyweight since they need to remember
which destructor to call.  However, raw memory needs no destructor, and
we can exploit this fact."
2014-10-21 15:50:53 +03:00
Gleb Natapov
a1533cc671 memory: call MADV_DONTDUMP on the whole mmaped memory
We mmap 2*alloc bytes not alloc.
2014-10-21 12:20:22 +03:00
Avi Kivity
9cecc2ce79 Add .gitorderfile
With

  git config diff.orderfile .gitorderfile

patches become more readable, as header changes will come before source
changes.
2014-10-21 12:08:00 +03:00
Avi Kivity
b8b39499ff smp: cleanups
"smp" is always defined, no need to test for it.
2014-10-21 12:07:30 +03:00
Avi Kivity
5a12154f3a core: fix memory initialization order
Initizalize memory before initializing the reactor object, which might
itself require a lot of memory (e.g. the virtio ring).
2014-10-21 11:46:12 +03:00
Avi Kivity
91782ac6a2 virtio: optimize single-buffer packet deleter
Instead of allocating a vector to store the buffers to be destroyed, in the
case of a single buffer, use an ordinary free deleter.

This doesn't currently help much because the packet is share()d later on,
but if we may be able to eliminate the sharing one day.
2014-10-21 11:27:05 +03:00
Avi Kivity
61782fcc05 packet: add a vectored constructor with a deleter
Add packet(Iterator, Iterator, deleter).

(unfortunately we have both a template version with a template parameter
named Deleter, and a non-template version with a parameter called deleter.
Need to sort the naming out).
2014-10-21 11:24:27 +03:00
Avi Kivity
fe8ed68878 temporary_buffer: use raw object deleter to destroy allocated buffer
Removes an allocation.
2014-10-21 11:23:41 +03:00
Avi Kivity
f75d1822cc core: special-case deleter for raw memory
In many cases, a deleter is used to protect raw memory (e.g. a char array,
not something with a destructor).  In that case we can simply free() it,
so, the deleter need not remember which destructor needs to be called.

It does need to remember whether it's a raw object or not, so we take over
the least significant bit and use it as a marker, and store the pointer
to the object in the deleter, instead of using a proxy impl object to
control actual deletion.

If the deleter is subsequently share()d, we have to convert it back to
the standard form, since the reference count lives in the impl object.
2014-10-21 11:20:12 +03:00
Avi Kivity
5a6771ee33 Drop eclipse workaround for reference qualified member functions
Latest eclipse understands it.
2014-10-19 17:42:47 +03:00
Avi Kivity
7ac12f4839 Merge branch 'virtio'
Remove allocations from the virtio receive and transmit paths.
2014-10-19 10:58:30 +03:00
Glauber Costa
79b9053751 net: parse ethernet address from string
Given a string, return the corresponding ethernet address. This is useful
specially for xen, where we read the mac address from the xenstore.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2014-10-19 10:55:26 +03:00
Raphael S. Carvalho
932a6e8752 core: move file-specific content from reactor.hh to specialized header
No semantic change; just cleaning things up.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2014-10-18 16:42:08 +03:00
Avi Kivity
60b70974d4 Merge branch 'memcache' of github.com:cloudius-systems/seastar-dev
Memcache updates from Tomasz:

"memcache: towards ASCII protocol completeness

Contains also some bug fixes."
2014-10-18 16:33:20 +03:00
Tomasz Grabiec
7903acdeb8 tests: memcache: add incr/decr tests 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
f00bc22067 memcache: add support for 'incr' and 'decr' commands 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
3093e2c543 memcache: make data changes atomic
Transmission of data is not atomic so we cannot replace item's fields
in-place. This can lead to inconsistencies in "get" responses. We
should rather allocate a new item object replacing the one which is in
the cache.
2014-10-18 12:59:59 +02:00
Tomasz Grabiec
e3fa358075 test: memcache: flush all keys after each test 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
6ed212e7c6 tests: memcache: add tests for 'stats' command 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
7ae9178439 memcache: introduce 'stats' command
Our version prints only a subset of statistics printed by stock memcache.
2014-10-18 12:59:59 +02:00
Tomasz Grabiec
4edfc4f0c2 core: implement to_sstring(const char*) 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
287ad01f76 memcache: make cache::size() return only live items 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
51f52ed2b5 tests: memcache: make tests more robust
By automatically detecting when memcache is up and running.
2014-10-18 12:59:59 +02:00
Tomasz Grabiec
e07e6ccbef tests: memcache: add tests for 'cas' and 'gets' 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
7c9e31a0b2 memcache: add 'cas' and 'gets' commands 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
f134fe1a94 tests: memcache: add tests for 'add' and 'replace' 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
5c649c54fb memcache: add support for 'add' and 'replace' commands 2014-10-18 12:59:59 +02:00
Tomasz Grabiec
d5a0e9654e tests: memcache: add tests for 'noreply' variant of 'set' and 'delete'
This also extracts some code to a common base class.
2014-10-18 12:59:59 +02:00
Tomasz Grabiec
f3ebe8abe9 memcache: support 'noreply' on 'set' and 'delete' 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
a1848ca6a7 tests: memcache: add test for 'version' command 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
84d5b7098f memcache: implement 'version' command 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
e30b097867 tests: memcache: fix assertion so that it works with stock memcached too 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
3347a78289 tests: memcache: add tests for 'flush_all' command 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
fdc4e7bbf5 tests: memcache: cleanup: rename methods 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
f2b1d16ce5 memcache: implement 'flush_all' command 2014-10-18 12:59:58 +02:00
Tomasz Grabiec
09faeff196 tests: distinguish between fast and slow tests
Most of the tests are fast and they cover most of the
functionality. The slow minority of tests takes significantly more
time to run. Developers should run tests frequently in order to get
feedback on correctness of their changes. The test runner
distinguishes now between fast and slow tests. When given '--fast'
switch it skips tests marked as slow.

$ time ./test.py
[8/8] PASSED tests/memcache/test.py --mode release
OK.

real	0m33.084s
user	0m0.501s
sys	0m0.271s

$ time ./test.py --fast
[8/8] PASSED tests/memcache/test.py --mode release --fast
OK.

real	0m1.012s
user	0m0.464s
sys	0m0.247s
2014-10-18 12:59:50 +02:00
Tomasz Grabiec
db5cdc3918 tests: memcache: add test exploiting string_builder issue 2014-10-16 12:35:09 +02:00
Tomasz Grabiec
5d6eaaea33 ragel: reset string builder on init
Otherwise it will contain stale data from previous unsuccessful pass.
2014-10-16 12:24:30 +02:00
Tomasz Grabiec
6b970957d9 core: never throw from keep_doing()
Exception thrown from async action should be returned in a failed
future rather than thrown to the caller of keep_doing().
2014-10-16 11:45:32 +02:00