Skip to content

fixup! net-tcp-bbr1: for testing, a copy of BBRv1 #10

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 28 commits into
base: v3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f601c9f
net-tcp_bbr: broaden app-limited rate sample detection
nealcardwell Jun 11, 2019
9cb2d74
net-tcp_bbr: v2: shrink delivered_mstamp, first_tx_mstamp to u32 to f…
nealcardwell Jun 25, 2018
7679309
net-tcp_bbr: v2: snapshot packets in flight at transmit time and pass…
nealcardwell Aug 5, 2017
4e589c6
net-tcp_bbr: v2: count packets lost over TCP rate sampling interval
nealcardwell Oct 13, 2017
ade2a0e
net-tcp_bbr: v2: export FLAG_ECE in rate_sample.is_ece
nealcardwell Nov 19, 2018
1f4015e
net-tcp_bbr: v2: introduce ca_ops->skb_marked_lost() CC module callba…
nealcardwell Aug 8, 2018
f82a3d3
net-tcp_bbr: v2: adjust skb tx.in_flight upon merge in tcp_shifted_skb()
nealcardwell May 2, 2019
bfa26db
net-tcp_bbr: v2: adjust skb tx.in_flight upon split in tcp_fragment()
nealcardwell May 2, 2019
0fa4869
net-tcp: add new ca opts flag TCP_CONG_WANTS_CE_EVENTS
yousukseung May 24, 2018
c20e56d
net-tcp: re-generalize TSO sizing in TCP CC module API
nealcardwell Sep 27, 2019
a5cc006
net-tcp: add fast_ack_mode=1: skip rwin check in tcp_fast_ack_mode__t…
nealcardwell Nov 16, 2019
4fef7ac
net-tcp_bbr: v2: record app-limited status of TLP-repaired flight
jianfenw Jun 19, 2020
dc4a1f8
net-tcp_bbr: v2: inform CC module of losses repaired by TLP probe
jianfenw Jun 16, 2020
9f5cbd8
net-tcp_bbr: v2: introduce is_acking_tlp_retrans_seq into rate_sample
nealcardwell Sep 21, 2020
40f1ce9
tcp: introduce per-route feature RTAX_FEATURE_ECN_LOW
Jul 14, 2023
aa27c22
net-tcp_bbr: v3: update TCP "bbr" congestion control module to BBRv3
nealcardwell Jun 11, 2019
5ad789e
net-tcp_bbr: v3: ensure ECN-enabled BBR flows set ECT on retransmits
Jun 11, 2021
a1d32ad
tcp: export TCPI_OPT_ECN_LOW in tcp_info tcpi_options field
nealcardwell Jul 24, 2023
a7743a2
net-test: add .config for kernel circa v5.10, with many TCP CC module…
soheilhy Jan 24, 2016
04ed1b4
net-test: adds a gce-install.sh script to build and install kernel on…
soheilhy Jan 24, 2016
7ce213b
net-test: scripts for testing BBR with upstream Linux kernels
nealcardwell Aug 29, 2018
d0d8043
net-tcp-bbr1: for testing, a copy of BBRv1
nealcardwell Jul 20, 2023
fea8e5a
net-test: udpate config.gce to recent kernel and enable BBR1 for testing
nealcardwell Jul 20, 2023
c27c87b
iproute2: a .patch file for iproute2: ss: output TCP BBRv3 diag infor…
nealcardwell Jul 23, 2023
cc97916
iproute2: a .patch file for iproute2: ip: introduce the ecn_low per-r…
nealcardwell Jul 23, 2023
a59d131
iproute2: a .patch file for iproute2: ss: display "ecn_low" if tcp_in…
nealcardwell Jul 24, 2023
7542cc7
net-tcp_bbr: v3: add a README.md for TCP BBR v3 release
nealcardwell Jul 23, 2019
69142bf
fixup! net-tcp-bbr1: for testing, a copy of BBRv1
eiffel-fl Aug 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,9 @@ x509.genkey
/allrandom.config
/allyes.config

# tmp build/install directory for /gce-install.sh:
/gce/

# Kconfig savedefconfig output
/defconfig

Expand Down
170 changes: 170 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# TCP BBR v3 Release

This document gives a quick overview of Google's TCP BBR v3 release for Linux,
and how to download, build, install, and test it.

The TCP BBR v3 release is intended to enable research collaboration and wider
testing. We encourage researchers to dive in and help evaluate/improve the BBR
algorithm and code. We welcome patches with good solutions to issues.

This document shows how to download, build, install, and test
a Linux kernel running TCP BBR v3.

## License

Like Linux TCP BBR v1, the v3 code is dual-licensed as both GPLv2 (like the
Linux kernel) and BSD. You may use it under either license.

## Viewing the TCP BBR v3 sources

You can view the current sources here:
[tcp_bbr.c](https://github.com/google/bbr/blob/v3/net/ipv4/tcp_bbr.c)

## Obtaining kernel sources with TCP BBR v3

There are two main options for downloading the code:

1. To create a new git repo starting from a Linux kernel with TCP BBR v3,
you can run:

```
git clone -o google-bbr -b v3 https://github.com/google/bbr.git
cd bbr/
```

2. To download the code into an existing git repo, you can use:

```
git remote add google-bbr https://github.com/google/bbr.git
git fetch google-bbr
git checkout google-bbr/v3
```

Note that if you already have a git repo that has imported the Linux source
tree, then the second option will be much faster and use much less space, since
it will only need to download the small deltas relative to the mainline Linux
source distribution.

## Building and installing the kernel

To build a Linux kernel with TCP BBR v3 support, copy that kernel to a target
(Debian or Ubuntu) test machine (bare metal or GCE), and reboot that machine,
you can use the following script, included in the TCP BBR v3 distribution:

```
./gce-install.sh -m ${HOST}
```

## Checking the kernel installation

Once the target test machine has finished rebooting, then ssh to the target
test machine and become root with sudo or equivalent. First check that the
machine booted the kernel you built above:

```
uname -a
```

You should see the branch name SHA1 hash, and build time stamp from the kernel
you built above.


Then check what congestion control modules are available with:
```
sysctl net.ipv4.tcp_available_congestion_control
```

You should see something like:
```
net.ipv4.tcp_available_congestion_control = reno bbr cubic dctcp
```

## Install test dependencies

Next, copy the test scripts to the target test machine with:

```
scp -r gtests/net/tcp/bbr/nsperf/ ${HOST}:/tmp/
```

Before running the tests for the first time, as a one-time step you'll need to
install the dependencies on the test machine, as root:

```
mv /tmp/nsperf /root/
cd /root/nsperf
./configure.sh
```

## Running TCP BBR v3 tests and generating graphs

To run the tests, ssh to the target test machine and become root with sudo or
equivalent. Then run the tests and generate graphs with:

```
cd /root/nsperf
./run_tests.sh
./graph_tests.sh
```

This will run for hours, and place the graphs in the ./graphs/ directory.

You can run and graph a subset of the tests by specifying the test by name as
an environment variable. For example:

```
cd /root/nsperf
tests=random_loss ./run_tests.sh
tests=random_loss ./graph_tests.sh
```

Enjoy!

## Release Notes and Details

### Introducing the ecn_low per-route feature

This new "ecn_low" per-route feature indicates that the given destination
network is a low-latency ECN environment, meaning both that (a) ECN CE marks
are applied by the network using a low-latency marking threshold and also that
(b) TCP endpoints provide precise per-data-segment ECN feedback in ACKs (where
the ACK ECE flag echoes the received CE status of all newly-acknowledged data
segments). This feature indication can be used by congestion control algorithms
to decide how to interpret ECN signals over the given destination network.

Basically, this "ecn_low" feature is for use when you know that any ECN marks
that the connections experience will be DCTCP/L4S-style ECN marks, rather than
RFC3168 ECN marks.

A patch for the iproute2 package to support the "ecn_low" feature is included
in the BBRv3 source branch
[here](https://github.com/google/bbr/blob/v3/0002-ip-introduce-the-ecn_low-per-route-feature.patch)

### iproute2 patches to support functionality in the BBRv3 source release

The BBRv3 source release includes several patches for the iproute2 package to
support functionality in the BBRv3 test kernel:

- patch "ss" tool to output congestion control state for BBRv3:
- [0001-ss-output-TCP-BBRv3-diag-information.patch](https://github.com/google/bbr/blob/v3/gtests/net/tcp/bbr/nsperf/0001-ss-output-TCP-BBRv3-diag-information.patch)
- patch "ip" tool to support ecn_low per-route feature:
- [0002-ip-introduce-the-ecn_low-per-route-feature.patch](https://github.com/google/bbr/blob/v3/gtests/net/tcp/bbr/nsperf/0002-ip-introduce-the-ecn_low-per-route-feature.patch)
- patch "ss" tool to show "ecn_low" state in TCP socket:
- [0003-ss-display-ecn_low-if-tcp_info-tcpi_options-TCPI_OPT.patch](https://github.com/google/bbr/blob/v3/gtests/net/tcp/bbr/nsperf/0003-ss-display-ecn_low-if-tcp_info-tcpi_options-TCPI_OPT.patch)

### Enabling ECN support

You can enable BBRv3 ECN support with commands like the following:

```
# 1: negotiate TCP ECN for active and passive connections:
sysctl net.ipv4.tcp_ecn=1
# 2: enable BBRv3 ECN logic using the "ecn_low" per-route feature, e.g. :
alias ip=/root/iproute2/iproute2/ip/ip
ip route change default via 192.168.0.100 dev ens4 features ecn_low
```

## FAQ

If you have questions about BBR, check the [BBR
FAQ](https://github.com/google/bbr/blob/master/Documentation/bbr-faq.md).
Loading