Skip to content

Commit bb7015c

Browse files
ericsandurazvand
authored andcommitted
feat(guides): Port usoc22 High Performance session as a guide
Ports the guides from session 10 of usoc22 as it's own guide .mdx file for compatibility with the current /guides page. Signed-off-by: Eric Sandu <eric_andrei.sandu@stud.acs.upb.ro>
1 parent ef30b32 commit bb7015c

1 file changed

Lines changed: 259 additions & 0 deletions

File tree

Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
---
2+
title: High-Performance Networking
3+
description: |
4+
Developing highly specialized and performance-optimized unikernels with Unikraft by bypassing traditional OS layers and interacting directly with network drivers.
5+
---
6+
7+
## Overview
8+
9+
In this guide, we explore the development of highly specialized and performance-optimized unikernels.
10+
Traditionally, network-based applications in Unikraft are built on top of the socket API, which requires a multi-layered library stack:
11+
12+
```text
13+
.---------------------------.
14+
( Socket application )
15+
'---------------------------'
16+
|
17+
V
18+
+---------------------------+
19+
| libvfscore |
20+
+---------------------------+
21+
+---------------------------+
22+
| liblwip |
23+
+---------------------------+
24+
+---------------------------+
25+
| libuknetdev |
26+
+---------------------------+
27+
+---------------------------+
28+
| libkvmplat |
29+
+---------------------------+
30+
|
31+
V
32+
.---------------------------.
33+
( Virtual Network Interface )
34+
'---------------------------'
35+
```
36+
37+
The Virtual File System (VFS) layer (provided by `libvfscore`) and the TCP/IP network stack (provided by `liblwip`) are complex subsystems that can introduce additional overhead.
38+
For high-performance Network Functions (NFs), it is often more efficient to bypass these OS components and interact directly with the driver or hardware, similar to frameworks like Intel DPDK.
39+
40+
In this scenario, the library stack is simplified:
41+
42+
```text
43+
.---------------------------.
44+
( High performance NF )
45+
'---------------------------'
46+
|
47+
V
48+
+---------------------------+
49+
| libuknetdev |
50+
+---------------------------+
51+
+---------------------------+
52+
| libkvmplat |
53+
+---------------------------+
54+
|
55+
V
56+
.---------------------------.
57+
( Virtual Network Interface )
58+
'---------------------------'
59+
```
60+
61+
This guide demonstrates the process of developing a high-performance network packet generator using this direct-access model.
62+
63+
## Requirements
64+
65+
To develop and test high-performance networking unikernels, you need the Unikraft `kraft` tool and the following utilities:
66+
67+
- `qemu-kvm`
68+
- `qemu-system-x86_64`
69+
- `bridge-utils`
70+
- `ifupdown`
71+
- `tshark`
72+
- `tcpdump`
73+
74+
Installation on Debian/Ubuntu:
75+
76+
```console
77+
$ sudo apt-get -y install qemu-kvm qemu-system-x86 sgabios socat bridge-utils ifupdown tshark tcpdump
78+
```
79+
80+
## Getting Started
81+
82+
The development process typically begins with a template containing basic building blocks, such as utilities for crafting IPv4/UDP packets.
83+
84+
1. Create a copy of the packet generator template:
85+
86+
```console
87+
$ cp -a sol/pktgen path/to/your/copy
88+
$ cd path/to/your/copy
89+
```
90+
91+
1. Initialize and build the application with `kraft`:
92+
93+
```console
94+
$ kraft list update
95+
$ kraft list pull
96+
$ kraft configure
97+
$ kraft build
98+
```
99+
100+
1. Verify the resulting image by running it to see the Unikraft banner:
101+
102+
```console
103+
$ kraft run
104+
```
105+
106+
## Interacting with the Network Device
107+
108+
Direct interaction with network device drivers is achieved through Unikraft's internal `uknetdev` API.
109+
110+
### Library Dependency
111+
112+
First, ensure the application states a dependency on `libuknetdev` in its `Config.uk` file:
113+
114+
```kconfig
115+
depends on LIBUKNETDEV
116+
```
117+
118+
This provides access to the `<uk/netdev.h>` and `<uk/netbuf.h>` headers.
119+
120+
### Bringing Up the Interface
121+
122+
Bringing up a network interface involves transitioning the device through several configuration states before it can process traffic:
123+
124+
1. **Discovery**: Determine the number of available interfaces using `uk_netdev_count()`.
125+
126+
1. **Retrieval**: Retrieve the `struct uk_netdev *` handle for the desired device (e.g., device `0`).
127+
128+
1. **Configuration**: Specify how many receive (RX) and transmit (TX) queues the device should provide.
129+
Note that drivers typically require at least one queue in each direction even if only one is used.
130+
131+
```c
132+
struct uk_netdev_conf ifconf = {
133+
.nb_rx_queues = 1,
134+
.nb_tx_queues = 1
135+
};
136+
```
137+
138+
1. **Queue Setup**: Configure the RX and TX queues, specifying the size and the allocators for internal descriptors and buffers.
139+
140+
1. **Probing**: Use `uk_netdev_probe()` to transition the device from an unprobed to an unconfigured state.
141+
142+
1. **Starting**: Use `uk_netdev_start()` to enable the device for operational traffic.
143+
144+
### Testing Environment
145+
146+
To test the unikernel's networking capabilities, a network bridge must be created on the Linux host:
147+
148+
```console
149+
# Ensure permissions for STP and create bridge 'usocbr0'
150+
sudo sysctl -w net.bridge.bridge-nf-call-arptables=0
151+
sudo brctl addbr usocbr0
152+
sudo brctl setfd usocbr0 0
153+
sudo brctl stp usocbr0 off
154+
sudo ifconfig usocbr0 0.0.0.0 up
155+
156+
# Disable packet filtering on bridge interfaces
157+
echo 0 | sudo tee /proc/sys/net/bridge/bridge-nf-call-arptables
158+
echo 0 | sudo tee /proc/sys/net/bridge/bridge-nf-call-iptables
159+
echo 0 | sudo tee /proc/sys/net/bridge/bridge-nf-call-ip6tables
160+
```
161+
162+
Launch the unikernel attached to the bridge:
163+
164+
```console
165+
$ kraft run -b usocbr0
166+
```
167+
168+
## Packet Generation and Transmission
169+
170+
### Generating Frames
171+
172+
The provided `genpkt_udp4()` function (or the shorthand `genpkt_usoc21()`) generates Ethernet-IPv4-UDP frames.
173+
The generation requires specific parameters retrieved from the device via `uk_netdev_info_get()`, such as `ioalign` for buffer alignment and `nb_encap_tx` for headroom (reserved space for driver encapsulation).
174+
175+
For performance testing, minimum-sized packets (60 bytes) are recommended, as they place the most stress on software and hardware parsing components.
176+
177+
### Sending Packets
178+
179+
Transmission is performed using `uk_netdev_tx_one()`.
180+
It is critical to manage the packet buffers correctly:
181+
182+
- If a packet is successfully enqueued, the driver takes ownership and will free it automatically.
183+
- If transmission fails, the application must manually call `uk_netbuf_free()` to avoid memory leaks.
184+
185+
```c
186+
status = uk_netdev_tx_one(netif, 0, pkt);
187+
if (!uk_netdev_status_successful(status)) {
188+
uk_netbuf_free(pkt);
189+
}
190+
```
191+
192+
Traffic can be verified on the host using tools like `tshark` or `tcpdump` attached to the bridge:
193+
194+
```console
195+
$ tshark -i usocbr0
196+
```
197+
198+
## Performance Optimization
199+
200+
### Throttled Instrumentation
201+
202+
Measuring performance requires tracking the total number of packets and bytes sent.
203+
However, synchronous printing in Unikraft is expensive and can significantly slow down the application.
204+
It is recommended to throttle statistics printing using a bitmask or counter:
205+
206+
```c
207+
if ((total_nb_pkts & 0x3fffff) == 0x0) {
208+
print_netspeed(total_nb_pkts, total_nb_bytes);
209+
}
210+
```
211+
212+
### Optimization Techniques
213+
214+
Several techniques can be applied to increase the packet rate:
215+
216+
1. **Compiler Optimizations**: Enable *Link Time Optimizations* (LTO) and *Dead Code Elimination* (DCE) in the build options via `menuconfig`.
217+
218+
1. **Persistent Packets**: Instead of freeing and regenerating packets on transmission failure, retry sending until the TX queue has space.
219+
220+
1. **Packet Duplication**: Use `uk_netbuf_dup_single()` to `memcpy` from a primordial packet buffer, which can be cheaper than full header generation.
221+
222+
1. **Memory Pools**: Use `libukallocpool` for $O(1)$ allocation and deallocation.
223+
This replaces the general-purpose allocator with a pre-allocated list of objects.
224+
225+
```c
226+
/* Add 'depends on LIBUKALLOCPOOL' to Config.uk */
227+
pool = uk_allocpool_alloc(uk_alloc_get_default(), 1024, 2048, info.ioalign);
228+
struct uk_alloc *p = uk_allocpool2ukalloc(pool);
229+
```
230+
231+
1. **Zero-copy**: While virtio-net has current limitations, some drivers support increasing the `netbuf` reference counter to avoid freeing and reallocating the same packet for repeated transmission.
232+
233+
1. **Batching**: Sending multiple packets in a single batch reduces the overhead of notifying the device backend (functionality introduced in PR#243).
234+
235+
## Receiving Traffic
236+
237+
Implementing a receiver follows a similar pattern to the transmitter but requires a proper receive buffer allocation function (`alloc_rxpkts`).
238+
This callback is used by the driver to replenish the RX queue with empty buffers.
239+
240+
```c
241+
uint16_t alloc_rxpkts(void *argp, struct uk_netbuf *pkts[], uint16_t count) {
242+
uint16_t i;
243+
for (i=0; i<count; ++i) {
244+
pkts[i] = uk_netbuf_alloc_buf(p, 2048, info.ioalign, info.nb_encap_rx, 0, NULL);
245+
if (!pkts[i]) break;
246+
}
247+
return i;
248+
}
249+
```
250+
251+
The receive loop then polls the device:
252+
253+
```c
254+
status = uk_netdev_rx_one(netdev, 0, &pkt);
255+
if (uk_netdev_status_successful(status)) {
256+
// Process counters and free the received packet
257+
uk_netbuf_free(pkt);
258+
}
259+
```

0 commit comments

Comments
 (0)