This value is passed as an opaque parameter of the rte_pktmbuf_pool_init().
It should equal to a buffer size + RTE_PKTMBUF_HEADROOM.
The default value is 2K + RTE_PKTMBUF_HEADROOM.
PMD is using this value minus RTE_PKTMBUF_HEADROOM for configuring the Rx
data buffers' size when it configures the Rx HW ring.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Do not store the tcp header in unacked queue. When partial ack of a
segment happens, trim off the acked part of a segment. When retransmits,
recalculate the tcp header and retransmit only the unacked part.
We neglected to set offload_info::needs_csum on reset packets, resuling in
them being ignored by the recipient. Symptoms include connection attempts
to closed ports (seastar = passive end) hanging instead the active end
understanding the port is closed.
Allocate exactly the available fragment size in order to catch buffer
overflows.
We get similar behaviour in dpdk, since without huge pages, it must copy
the packet into a newly allocated buffer.
Two bugs:
1. get_header<type>(offset) was used with size given as the offset
2. opt_end failed to account for the mandatory tcp header, and thus was
20 bytes to large, resulting in overflow.
boost::join() provided by boost/algorithm/string.hpp conflicts with
boost::join() from boost/range/join.hpp. It looks like a boost issue
but let's not pollute the namespace unnecesssarily.
Regarding the change in configure.py, it looks like scollectd.cc is
part of the 'core' package, but it needs 'net/api.hh', so I added
'net/net.cc' to core.
If data buffers decoupling from the rte_mbuf is not available (hugetlbfs is not
available)copy the newly received data into the memory buffer we allocate and
build the "packet" object from this buffer. This will allow us returning the
rte_mbuf immediately which would solve the same issue the "decoupling" is solving
when hugetlbfs is available.
The implementation is simplistic (no preallocation, packet data cache alignment, etc.).
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
- Allocate the data buffers instead of using the default inline rte_mbuf
layout.
- Implement an rx_gc() and add an _rx_gc_poller to call it: we will refill
the rx mbuf's when at least 64 free buffers.
This threshold has been chosen as a sane enough number.
- Introduce the mbuf_data_size == 4K. Allocate 4K buffers for a detached flow.
We are still going to allocate 2K data buffers for an inline case since 4K
buffers would require 2 pages per mbuf due to "mbuf_overhead".
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
std::vector is promised to be continuous storage while std::deque is not.
In addition std::vector's semantics yields a simpler code than those of deque.
Therefore std::vector should deliver a better performance for a stack semantics
we need here.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Take into an account the alignment, header and trailer that mempool is adding
to the elements.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
- tcp.hh: Properly calculate the pseudo-header in the TSO case: it should be
calculated as if ip_len is zero.
- Enable TSO in the DPDK network backend.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
- Define MARKER type if not defined.
- Adjust the Tx zero-copy to the rte_mbuf layout in DPDK 1.7.x.
- README.md:
- Bump up the DPDK latest version to 1.8.0.
- Add a new DPDK configuration description.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Send packets without copying fragments data:
- Poll all the Tx descriptors and place them into a circular_buffer.
We will take them from there when we need to send new packets.
- PMD will return the completed buffers descriptors to the Tx mempool.
This way we are going to know that we may release the buffer.
- "move" the packet object into the last segment's descriptor's private data.
When this fragment is completed means the whole packet has been sent
and its memory may be released. So, we will do it by calling the packet's
destructor.
Exceptions:
- Copy if hugepages backend is not enabled.
- Copy when we failed to send in a zero-copy flow (e.g. when we failed
to translate a buffer virtual address).
- Copy if first frag requires fragmentation below 128 bytes level - this is
in order to avoid headers splitting.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v5:
- NULL -> nullptr across the board.
- Removed unused macros: MBUF_ZC_PRIVATE() and max_frags_zc.
- Improved the local variables localization according to Nadav's remarks.
- tx_buf class:
- Don't regress the whole packet to the copy-send if a single fragment failed to be sent
in a zero-copy manner (e.g. its data failed the virt2phys translation). Send only such a
fragment in a copy way and try to send the rest of the fragments in a zero-copy way.
- Make set_packet() receive packet&&.
- Fixed the comments in check_frag0(): we check first 128 bytes and not first 2KB.
starting from v2.
- Use assert() instead of rte_exit() in do_one_frag().
- Rename in set_one_data_buf() and in copy_one_data_buf(): l -> buf_len
- Improve the assert about the size of private data in the tx_buf class:
- Added two MARKER fields at the beginning and at the end of the private fields section
which are going to be allocated on the mbuf's private data section.
- Assert on the distance between these two markers.
- Replace the sanity_check() (checks that packet doesn't have a zero-length) in a
copy-flow by an assert() in a general function since this check
is relevant both for a copy and for a zero-copy flows.
- Make a sanity_check to be explicitly called frag0_check.
- Make from_packet() receive packet&&.
- In case frag0_check() fails - copy only the first fragment and
not the whole packet.
- tx_buf_factory class:
- Change the interface to work with tx_buf* instead of tx_buf&.
- Better utilize for-loop facilities in gc().
- Kill the extra if() in the init_factory().
- Use std::deque instead of circular_buffer for storing elements in tx_buf_factory.
- Optimize the tx_buf_factory::get():
- First take the completed buffers from the mempool and only if there
aren't any - take from the factory's cache.
- Make Tx mempools using cache: this significantly improves the performance despite the fact that it's
not the right mempool configuration for a single-producer+single-consumer mode.
- Remove empty() and size() methods.
- Add comments near the assert()s in the fast-path.
- Removed the not-needed "inline" qualifiers:
- There is no need to specify "inline" qualifier for in-class defined
methods INCLUDING static methods.
- Defining process_packets() and poll_rx_once() as inline degraded the
performance by about 1.5%.
- Added a _tx_gc_poller: it will call tx_buf_factory::gc().
- Don't check a pointer before calling free().
- alloc_mempool_xmem(): Use posix_memalign() instead of memalign().
New in v4:
- Improve the info messages.
- Simplified the mempool name creation code.
- configure.py: Opt-out the invalid-offsetof compilation warning.
New in v3:
- Add missing macros definitions dropped in v2 by mistake.
New in v2:
- Use Tx mbufs in a LIFO way for better cache utilization.
- Lower the frag0 non-split thresh to 128 bytes.
- Use new (iterators) semantics in circular_buffer.
- Use optional<packet> for storing the packing in the mbuf.
- Use rte_pktmbuf_alloc() instead of __rte_mbuf_raw_alloc().
- Introduce tx_buf class:
- Hide the private rte_mbuf area handling.
- Hide packet to rte_mbuf cluster translation handling.
- Introduce a "Tx buffers factory" class:
- Hide the rte_mbuf flow details:
mempool->circular_buffer->(PMD->)mempool
- Templatization:
- Make huge_pages_mem_backend a dpdk_qp class template parameter.
- Unite the from_packet_xxx() code into a single template function.
- Unite the translate_one_frag() and copy_one_frag() into a single
template function.
This function is needed when we want to estimate a number of memory we want to give to DPDK
when we can provide a mempool an external memory buffer.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
If seastar is configured to use hugetlbfs initialize mempools
with external memory buffer. This way we are going to better control the overall
memory consumption.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v2:
- Use char* instead of void* for pointer's arithmetics.
There is no reason for Rx and Tx pools to be of the same size:
Rx pool is 3 times the ring size to give the upper layers some time
to free the Rx buffers before the ring stalls with no buffers.
Tx has absolutely different constraints: since it provides a back pressure
to the upper layers if HW doesn't keep up there is no need to allow more buffers
in the air than the amount we may send in a single rte_eth_tx_burst() call.
Therefore we need 2 times HW ring size buffers since HW may release the whole
ring of buffers in a single rte_eth_tx_burst() call and thus we may be able to
place another whole ring of buffers in the same call.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
New in v4:
- Fixed the info message.
It is used to recover from a race where the sender is waiting for a
window update and the receiver is waiting for the sender to send more,
because somehow the window update carried in the ACK packet is not seen
by the sender.
This fix tcp_server rxrx test on DPDK. The problem is that when we
receive out of order packets, we will hold the packet in the ooo queue.
We do linearize on the incoming packet which will copy the packet and
thus free the old packet. However, we missed one case where we need to
linearize. As a result, the original packet will be held in the ooo
queue. In DPDK, we have fixed buffer in the rx pool. When all the dpdk
buffer are in ooo queue, we will not be able to receive further packets.
So rx hangs, even ping will not work.
This fix the following:
Server side:
$ tcp_server
Client side:
$ go run client.go -host 192.168.66.123 -conn 10 -test txtx
$ control-c
At this time, connection in tcp_server will be in CLOSED state (reset by
the remote), then tcp_server will call tcp::tcb::close() and wait for
wait_for_all_data_acked(), but no one will signal it. Thus we have tons
of leaked connection in CLOSED state.
We call output_one to make sure a packet with FIN is actually generated
and then sent out. If we only call output() and _packetq is not empty,
in tcp::tcb::get_packet(), packet with FIN will not be generated, thus
we will not send out a FIN.
This can happen when retransmit packets have been queued into _packetq,
then ACK comes which ACK all of the unacked data, then the application
call close() to close the connection.
We currently have RFC5681, a.k.a Reno TCP, as the congestion control
algorithms: slow start, congestion avoidance, fast retransmit, and fast
recovery. RFC6582 describes a specific algorithm for responding to
partial acknowledgments, referred to as NewReno, to improve Reno.
Follow RFC793 section "SEGMENT ARRIVES".
There are 4 major cases:
1) If the state is CLOSED
2) If the state is LISTEN
3) If the state is SYN-SENT
4) If the state is other state
Note:
- This change is significant (more than 10 pages in RFC793 describing
this segment arrival handling).
- More test is needed. Good news is, so far, tcp_server(ping/txtx/rxrx)
tests and httpd work fine.
Build a 128-entry redirection table to select which cpu services which
packet, when we have more cores than queues (and thus need to dispatch
internally).
Add a --hw-queue-weight to control the relative weight of the hardware queue.
With a weight of 0, the core that services the hardware queue will not
process any packets; with a weight of 1 (default) it will process an equal
share of packets, compared to proxy queues.
Unlike tcp::tcb::send() and tcp::connection::send() which send tcp
packets associated with tcb, tcp::send() only send packets associated
without tcb. We have a bunch of send() functions, rename it to make the
code more readable.
Tested with tcp_server + client.go using iptables dropping <SYN,ACK> or
<FIN,ACK> on client side.
I verified that the SYN or FIN packet is retransmitted and the
connection is closed after N (currently 5) retries.
