Commit Graph

53948 Commits

Author SHA1 Message Date
Avi Kivity
60b4756a36 smp: allow tuning memory size 2014-10-13 13:03:48 +03:00
Avi Kivity
56598903f6 memory: add support for reconfiguring memory
- add memory to a cpu
- bind that memory to a NUMA node
2014-10-13 13:03:46 +03:00
Avi Kivity
e2ea9336cd sockaddr: declare operator==() in global namespace
Needed for argument-dependent lookup; pointed out by clang.
2014-10-13 11:57:53 +03:00
Avi Kivity
3397812551 build: disable debug information on clang do to missing support 2014-10-13 11:55:05 +03:00
Avi Kivity
51261a2ff1 build: disable mismatched tags warning 2014-10-13 11:18:06 +03:00
Avi Kivity
7f8d92ee3c posix-stack.cc: add missing copyright 2014-10-13 09:31:04 +03:00
Avi Kivity
4a66794331 Merge branch 'asias/net' of github.com:cloudius-systems/seastar-dev
Net queue from Asias.
2014-10-13 09:24:44 +03:00
Asias He
c2650971b9 virtio-net: Improve RX packet building
Use begin and end iterator to build a packet in one go.
2014-10-13 11:37:56 +08:00
Asias He
4d71913a5b virtio-net: Enable guest side TSO4 and UFO
With merge receive buffer support, we can handle large packet on the RX
path now.
2014-10-13 11:37:56 +08:00
Asias He
c5861476b9 net: Rename pseudo_header_checksum to tcp_pseudo_header_checksum
We already have udp_pseudo_header_checksum. Make the name more
consistent.
2014-10-13 11:37:56 +08:00
Asias He
2625dd5944 net: Introduce eth_protocol_num 2014-10-13 11:37:56 +08:00
Asias He
5cf3f200c5 net: Introduce ip_protocol_num
We use this in all the places where the ip protocol number is used.
2014-10-13 11:37:56 +08:00
Asias He
05c72b0808 net: UDP checksum offload and UPD fragmentation offload 2014-10-13 11:37:56 +08:00
Asias He
5c661b5034 tcp: Setup initial sequence numbers 2014-10-13 11:37:56 +08:00
Asias He
d5c8155e00 net: Split posix stack related code to net/posix-stack.cc
NOTE: This patch makes pollable_fd::get_file_desc public in order to
access it in posix_server_socket_impl::accept(). Maybe there is a better
solution. But, fow now, this avoids a lot of including game.
2014-10-13 11:33:57 +08:00
Asias He
0ffaa0cbdd net: Reorder net/stack.cc
Makes the file more readable. Now, The classes are defined in orders.
2014-10-13 11:33:47 +08:00
Avi Kivity
2d85ac902f build: disable sanitizers for static builds
Fixes static debug builds.
2014-10-12 19:56:49 +03:00
Avi Kivity
8d5924a734 posix: add function to pin the current thread 2014-10-12 16:37:30 +03:00
Avi Kivity
77f13169c0 core: add NUMA resource allocation functions
Add a wrapper around libhwloc to discover memory and processing resources,
and divide them among the configured cpu threads.
2014-10-12 16:35:48 +03:00
Avi Kivity
cca41658aa util: add conversions library
At this time, support parsing a memory size (e.g "2T" == 2 << 40).
2014-10-12 16:34:16 +03:00
Avi Kivity
d89e09c9ed memory: switch page_list to use page indices instead of pointers
Two motivations:
1. reduce the size of struct page
2. allow relocating the page array; needed for dynamically changing
   the amount of memory supported.
2014-10-12 14:47:11 +03:00
Tomasz Grabiec
1bdb73e5da core: fix buffer overflow in to_sstring() 2014-10-09 20:04:53 +03:00
Tomasz Grabiec
eac955cc73 core: simplify output_stream::write() and flush() results
The result is not used for anything and I am not sure what it could be
used for, as the result carries little (write) to none (flush)
information. So I went ahead and simplified it to be future<> so that
it is easier to return it in places which expect future<>.
2014-10-09 20:02:11 +03:00
Asias He
d90bab775d virtio-net: Merge receive buffer 2014-10-09 19:54:50 +03:00
Asias He
a111b2fa1e net: TCP segment offload and UDP fragmentation offload support
Only tx path is added for now. Will enable rx path when merge receive
buffer feature is supported in virtio-net.

UDP fragmentation offload is not hooked up, will do in a separate patch.
2014-10-09 19:52:55 +03:00
Asias He
f14c04a132 net: Add offload_info to packet class
This is useful for adding tcp segment offload in ethernet driver.
This way, we do not need to parse the TCP packet again.
2014-10-09 19:52:54 +03:00
Avi Kivity
37ef3bc899 memory: request huge page backing for heap
Worth about 12% on httpd.
2014-10-07 15:23:30 +03:00
Avi Kivity
c6069ac4d9 core: set memory reclaim hook
Allow the memory manager to call us back requesting a reclaim.  Push
the task to the front of the queue, so we don't run out of memory waiting
for it to fire.
2014-10-07 15:14:44 +03:00
Avi Kivity
6746bcfd68 memory: reclaim support
Allow memory users to declare methods of reclaiming memory (reclaimers),
and allow the main loop to declare a safe point for calling these reclaimers.

The memory mananger will then schedule calls to reclaimers when memory runs
low.
2014-10-07 15:14:44 +03:00
Avi Kivity
252ca596bb Merge branch 'smp'
From Gleb:

"This patchset adds smp support for networking. Posix and native stack
implementation are completely disjoint, but present the same API to an
application. Speaking about application API it is the same as before,
but application is expected to create server socket on all cpus, otherwise
some connection will stuck or reset."
2014-10-07 11:04:22 +03:00
Gleb Natapov
304f6a317e Run httpd on all available cpus 2014-10-07 11:04:07 +03:00
Gleb Natapov
2a662af092 Forward received packet according to classification 2014-10-07 11:04:05 +03:00
Gleb Natapov
cc9146403d Proxy net interface
Only one cpu can talk to virtio NIC directly (at least without
multiqueue), so other cpus receive packets from the one that drives
virtio and forward packets that need to be send back. This is done
by proxy net interface.
2014-10-07 11:04:03 +03:00
Gleb Natapov
c8ffffa557 Add packet::free_on_cpu() function
Some packets are processed by a cpu other than the one that allocates it
and its fragments. free_on_cpu() function should be called on a cpu that
does processing and it returns a packet that is deletable by the current
cpu. It is done by copying packet/packet::impl to locally allocated one
and adding new deleter that runs old deleter on original cpu.
2014-10-07 11:04:01 +03:00
Gleb Natapov
4e7d8a8506 Introduce packet classification mechanism
Classifier returns what cpu a packets should be processed on. It may
return special broadcast identifier. The patch includes classifier for
tcp, udp and arp. Arp classifier broadcasts arp reply to all cpus. Default
classifier does not forward packet.
2014-10-07 11:03:57 +03:00
Gleb Natapov
0b59abafa7 Add net::device::l2inject function
Will need it later to handle forwarded packets. Also save net::device
pointer in thread local variable to get to device instance easily. When
we ill have more then one device per cpu we will have to change to
something more sophisticated.
2014-10-07 11:03:52 +03:00
Gleb Natapov
19addf6e64 Implement smp handling for posix networking stack
The idea is that only one thread opens listen socket and runs accept().
Other threads emulate listen()/accept() by waiting for connected
socket from the main thread. Main thread distributes connected sockets
according to round robin pattern. Patch introduce new specialization
for server_socket_impl and network_stack: posix_ap_server_socket_impl
and posix_ap_network_stack respectively. _ap_ stand for auxiliary processor.
2014-10-07 11:03:49 +03:00
Gleb Natapov
526312253f Introduce smp::main_thread() function
Sometimes we need to know if we are running on main thread during engine
initialization, before engine._id is available. This function will be
used for that.
2014-10-07 11:03:43 +03:00
Gleb Natapov
850545df6d Catch exception in app template
Print it out and exit program gracefully.
2014-10-07 11:03:39 +03:00
Gleb Natapov
20dcbbd12d Provide engine::exit() call to exit smp program gracefully 2014-10-07 11:03:37 +03:00
Raphael S. Carvalho
5ca6ddc281 reactor: introduce reactor::stat
Basically, wrapping stat around _thread_pool as it might block
waiting for metadata to be read from the underlying device.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2014-10-06 21:37:30 +03:00
Avi Kivity
4a080eb008 core: switch to spsc_queue for interthread communication
It's much faster than a normal queue.
2014-10-06 21:37:22 +03:00
Avi Kivity
daca47af67 Merge branch 'tgrabiec/udp' of github.com:cloudius-systems/seastar-dev
Posix and native UDP from Tomek.
2014-10-06 20:00:28 +03:00
Tomasz Grabiec
e4d55258c1 tests: add UDP tests
To start the server:

  $ build/release/tests/udp_server

To start the client:

  $ build/release/tests/udp_client --server localhost:10000
2014-10-06 18:36:42 +02:00
Tomasz Grabiec
f907f6a999 reactor: add getter for network stack 2014-10-06 18:34:43 +02:00
Tomasz Grabiec
ba49d24b01 net: add support for UDPv4 in native and posix stack 2014-10-06 18:34:40 +02:00
Tomasz Grabiec
cf7a084457 core: move some of the networking abstractions to net/api.hh 2014-10-06 18:34:28 +02:00
Tomasz Grabiec
3775dae6fb net: convert ipv4_addr.host from array to uint32_t
It will be easier to convert it to a format on which the native stack
works.
2014-10-06 18:34:28 +02:00
Tomasz Grabiec
623129b45b net: add packet constructor which accepts static data 2014-10-06 18:34:28 +02:00
Tomasz Grabiec
269b05afc2 core: add queue::push_eventually()
It's an asynchronous version of blocking push().
2014-10-06 18:34:28 +02:00