The current shared_ptr implementation is efficient, but does not support
polymorphic types.
Rename it in order to make room for a polymorphic shared_ptr.
This is a little tricky, since we only know we want hugetlbfs after memory
has been initialized, so we start up in anonymous memory, and later
switch to hugetlbfs by copying it to hugetlb-backed memory and mremap()ing
it back into place.
With -fvisibility=hidden, all executable symbols are hidden from shared
objects, allowing more optimizations (especially with -flto). However, hiding
the allocator symbols mean that memory allocated in the executable cannot
be freed in a library, since they will use different allocators.
Fix by exposing these symbols with default visibility.
Fixes crash loading some dpdk libraries.
We store spans in freelist i if the span's size >= 2^i. However, when
picking a span to satisfy an allocation, we must use the next larger list
if the size is not a power of two, so that we can be sure that all spans on
that list can satisfy that request.
The current code doesn't do that, so it under-allocates, leading to memory
corruption.
Because memcpy() is declared by gcc as receiving non-null attributes, gcc
assumes that ptr != null, as it is passed into memcpy() (though with a size
of zero). As a result it ignores the null pointer check in ::free(), and
calls memory::free() directly, which does not expect a null pointer.
Fix by only calling memcpy() when the ptr is non-null.
cpu_pages::initialize() established the one-past-the-end page as a sentinel
to avoid boundary conditions checks. cpu_pages::do_resize() considers the
last page as the sentinel. This discrepancy causes the last page to be
considered free by do_resize, which promptly ends up as a use-after-free
page.
Fix by aligning do_resize() with initialize().
Allow memory users to declare methods of reclaiming memory (reclaimers),
and allow the main loop to declare a safe point for calling these reclaimers.
The memory mananger will then schedule calls to reclaimers when memory runs
low.
Add a compile-time option, DEFAULT_ALLOCATOR, to use the existing
memory allocator (malloc() and friends) instead of redefining it.
This option is a workaround needed to run Seastar on OSv.
Without this workaround, what seems to happen is that some code compiled
into the kernel (notably, libboost_program_options.a) uses the standard
malloc(), while inline code compiled into Seastar uses the seastar free()
to try and free that memory, resulting in a spectacular crash.
Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
With N3778, the compiler can provide us with the size of the object,
so we can avoid looking it up in the page array. Unfortunately only
implemented in clang at the moment.
Instead of rounding up to a power-of-two, have four equally spaced
regions between powers of two. For example:
1024
1280 (+256)
1536 (+256)
1792 (+256)
2048 (+256)
2560 (+512)
3072 (+512)
3584 (+512)
4096 (+512)
Allocate small objects within spans, minimizing waste.
Each object size class has its own pool, and its own freelist. On overflow
free objects are pushed into the spans; if a span is completely free, it is
returned to the main free list.
