This is useful for features that are provided incrementally, so may not
be present on all hypervisors. If the value is not present, return a
user-provided default, which also has a system-provided default (0).
We current have one port per event channel. We need to have a list of
semaphores that will all be made ready when an interrupt kicks in. This is
useful in the case where both tx and rx are bound to the same event channel.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
If we do that, plus make it an instance method, we should be able to use
make_ready_port. This is consistent with the userspace implementation and
from that point any changes there will be propagated to both.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
The representation of an event channel as an integer poses a problem, in which
waiting on an integer port doesn't work well when the same event channel is
assigned for both tx and rx. The future will be ready for one of the sides, but
we won't process the other.
One alternative is to have conditions in the future processing, and in case the
event channels are bound to the same port, process both events. But a better
solution is to use a class to represent the bound ports, and instances of those
classes will have their own pending methods.
Infrastructure will be written in a following patch to make sure that all
listeners to the same port will be made ready when an interrupt kicks in
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
The backend may be completely silent about the existence of the split channels feature.
In that case, trying to read through the template directly would cause an exception,
since we can't convert the empty string.
The backend-id, OTOH, is guaranteed to exist and wasn't using the template signature.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
We copy our grant reference into a temporary, so free_ref() does not
clear the real entry, causing an assert() to trigger later on.
Fix by capturing the grant reference entry by reference.
With this, the xen network driver survives multiple trips around the ring.
Keep recycling free ring entries back into the receive ring so we can
receive more than 256 packets.
The code is a little lame at the moment since it writes the index and
notifies the host for every frame, but that can be adjusted later.
There is no reason to wait when pushing back a free id - there is nothing
that could possibly block there.
Switch from a queue<> to an std::queue<> and use a semaphore to guard
popping from the queue.
Running tcp stream test with --smp > 1, sometimes the server sends TSO
frame, sometimes it does not. If we set --smp = 1, the server always
sends TSO frame. This is because the proxy device does not parse all the
features in the opts. We should copy the _hw_features from the real
device but it is not easy. For now, we simply duplicate the parse code.
Fix UDP for memcache with native stack
memcached: apps/memcached/memcached.cc:807:
void memcache::assert_resolved(future<>): Assertion `f.available()'
failed.
Tomek writes:
UDP path relied on the fact that handle() could not block, because
the output stream does not block, and passed references to variables
which live on stack. Since you now can block in handle_get(), this
no longer holds. We should chnage that, ie allocate conversation
state like we do in TCP.
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
Fix tcp_server tx test. We still have more to do.
Native stack:
$ go run client-txtx.go
Bytes Received(MiB): 1000
Total Time(Secs): 1.567927562
Bandwidth(MiB/Sec): 637.7845662234746
Posix stack:
$ go run client-txtx.go
Bytes Received(MiB): 1000
Total Time(Secs): 1.014354958
Bandwidth(MiB/Sec): 985.8481906291427
Note: client-txtx uses 100 concurrent connections.
With TSO enabled, we can see a Ethernet frame larger than 64K on tap
device. This makes wireshark unable to handle. It complains:
The capture file appears to be damaged or corrupt.
(pcapng_read_packet_block: cap_len 65549 is larger than
WTAP_MAX_PACKET_SIZE 65535.)
handle buffer recycles. Right now it is very simple: allocate a new receive
buffer after a succesful receival, and mark the tx spot free when we get the tx
event notification.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Instead of returning a reference to a grant that is already present in an
array, defer the initialization. This is how the OSv driver handles it, and I
honestly am not sure if this is really needed: it seems to me we should be able
to just reuse the old grants. I need to check in the backend code if we can be
any smarter than this.
However, right now we need to do something to recycle the buffers, and just
re-doing the refs would lead to inconsistencies. So the best by now is to close
and reopen the grants, and then later on rework this in a way that works for
both the initial setup and the recycle.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Right now, we allocate the whole index, and notify the backend that we have
produced nr_ents indexes. If we do that, we cannot increment the producer index
when we receive a new package. This would make the index overflow, and
basically, it is the responsible for the biggest part of the slowdown we are
seeing.
Before this patch, we're seeing 2s RTT for pings. After the patch:
64 bytes from 192.168.100.79: icmp_seq=1 ttl=64 time=0.437 ms
64 bytes from 192.168.100.79: icmp_seq=2 ttl=64 time=0.431 ms
64 bytes from 192.168.100.79: icmp_seq=3 ttl=64 time=0.475 ms
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Aside from managing the grant references, we also need to manage the positional
indexes in the array. We need to keep track of which indexes are free, and
which are used. Because we need the actual position number to fill xen's data
structures, I figured we could use a queue and then fill it up with all the
integers in our range. The queue is already futurized, so that's easy.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Enhance gntref with some useful operations. Also provide a default object that
represents an invalid grant.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
Some packets, like arp replies, are broadcast to all cpus for handling,
but only packet structure is copied for each cpu, the actual packet data
is the same for all of them. Currently networking stack mangles a
packet data during its travel up the stack while doing ntoh()
translations which cannot obviously work for broadcaster packets. This
patches fixes the code to not modify packet data while doing ntoh(), but
do it in a stack allocated copy of a data instead.
Fixes the following link errors when Xen support is disabled:
build/release/net/native-stack.o: In function `net::add_native_net_options_description(boost::program_options::options_description&)':
/seastar/net/native-stack.cc:101: undefined reference to `get_xenfront_net_options_description()'
build/release/net/native-stack.o: In function `net::create_native_net_device(boost::program_options::variables_map)':
/seastar/net/native-stack.cc:93: undefined reference to `create_xenfront_net_device(boost::program_options::variables_map, bool)'
collect2: error: ld returned 1 exit status
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
Checksum offload cannot be disabled in Xen (or at least, I haven't figured
out how). Advertise it as enabled, so that tcp doesn't drop packets as
failing their checksum.
Still need to flesh out the transmit path.
With this, seastar sends SYN/ACK packets in response to connection requests.
The Xen code registers a function that calls semaphore::signal as
an interrupt handler, however that function is not smp safe and may crash,
and in events it generates are likely to be ignored, since they are just
appended to the reactor queue without any real wakeup to the reactor thread.
Switch to using an eventfd. That's still unsafe, but a little better, since
its signalling is smp safe, and will cause the reactor thread to wake up
in case it was asleep.
With this, we are able to receive multiple packets.
We prepared N buffers, but only told the host about one. This meant the host
stopped forwarding received packets almost immediately.
Fix by writing the Xen-visible ring index correctly.
We used gnttab_grant_foreign_access() instead of
gnttab_grant_foreign_access_ref(). While the two functions have similar
enough signatures, they do very different things.
With the change, we are able to receive packets from Xen, though we crash
immediately.
We used gnttab_grant_foreign_access() instead of
gnttab_grant_foreign_access_ref(). While the two functions have similar
enough signatures, they do very different things.
With the change, we are able to transmit packets through Xen.
This is the basic support for xenfront. It can be used in domU, provided there
is a network interface to be hijacked.
The code that follows, is just the mechanics of managing the grants, event
channels, etc.
However, it does not yet work: I can't see netback injecting any data into it.
I am still debugging the protocol, but I wanted to flush the current state.
Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
