PTF tests for ONOS fabric.p4

This project contains PTF-based tests for fabric.p4, a P4 program distributed as part of ONOS, designed to work with Trellis, a set of SDN applications running on top of ONOS to provide the control plane for an IP fabric based on MPLS segment-routing.

PTF is a framework for data plane testing: https://github.com/p4lang/PTF

This project maintains the test case definition (written in Python), as well as the scripts to run them on different targets. Test cases are defined inside the directory tests/ptf/fabric.ptf Run scripts can be found in run/.

Currently, we provide scripts to test the following targets:

• stratum_bmv2 (run/bmv2/)
• tofino-model with stratum_bf (run/tm/)

To learn more about Stratum: https://github.com/stratum/stratum

Requirements

All scripts are based on a containerized version of the required tools and can be executed by installing the following dependencies:

• Docker (tested with v19.03, but it should work with older versions as well)
• make
• Bash-like Unix shell

Steps to run tests on stratum_bmv2

stratum_bmv2 is a version of BMv2's simple_switch (the reference P4 software switch) built with Stratum support to expose a P4Runtime interface that is used by the test cases to populate tables and other P4 objects.

1. Obtain the fabric.p4 pre-compiled artifacts for BMv2 (bmv2.json and p4info.txt). These files are distributed with ONOS:

   git clone https://github.com/opennetworkinglab/onos
   

2. Set the ONOS_ROOT environment variable to the location where you just cloned the ONOS repo:

   export ONOS_ROOT=<path-to-onos>
   

3. Run PTF tests:

   ./run/bmv2/run <profile> [test-case]
   

   To learn more about fabric.p4 "profiles" check these instructions.

   For example, to run all test cases for all profiles:

   ./run/bmv2/run all
   

   To run all test cases for the fabric-spgw profile:

   ./run/bmv2/run fabric-spgw
   

   To run a specific test case against a specific fabric profile (or all), for example test.FabricBridgingTest for the basic fabric profile:

   ./run/bmv2/run fabric TEST=test.FabricBridgingTest
   

4. If tests fail, check logs in:

   • run/bmv2/log for stratum_bmv2 logs
   • tests/ptf/ for PTF-related logs and PCAP traces (ptf.log and ptf.pcap)

Steps to run tests on tofino-model with stratum_bf

Requirements

Steps are similar to the previous case with a few differences:

• You will need to compile fabric.p4 for Tofino;
• You will need to create or obtain a containerized version of tofino-model;
• stratum_bf is used to provide a P4Runtime server to control tofino-model (a Docker image for it will be downloaded automatically.)

The run scripts assume that you have access to a containerized version of the Intel/Barefoot SDE that includes tofino-model. We do not provide such Docker image, but one can be easily generated by executing the SDE install instructions inside a Dockerfile.

The run script will use docker run to invoke the tofino-model command inside the container. For this reason, the script expects a Docker image that has the whole Barefoot SDE installed in it or just the tofino-model package. In both cases, the tofino-model executable should be on PATH.

IMPORTANT: make sure to reserve at least 8GB of RAM for your Docker host system (or VM if running Docker Desktop for Mac), otherwise tofino-model might fail to start or affect test results negatively.

Steps

1. Clone fabric-tofino repo and follow instructions to compile fabric.p4 for Tofino:

   https://github.com/opencord/fabric-tofino.git
   cd fabric-tofino
   <open README and follow instructions>
   

2. Set the FABRIC_TOFINO environment variable to the location where you cloned fabric-tofino:

   export FABRIC_TOFINO=<path-to-fabric-tofino>
   

3. Set the SDE_DOCKER_IMG environment variable to the location of a Docker image that can be downloaded via docker pull and that contains a Barefoot SDE installation, including tofino-model:

   export SDE_DOCKER_IMG=my-docker-repo/bf-sde:9.0.0
   

4. Run PTF tests using the run/tm/run script:

   ./run/tm/run <profile> [test-case]
   

   NOTE: Testing all profiles is not supported on this target. You must execute the run command for each profile.

   To run all test cases for the basic fabric profile:

   ./run/tm/run fabric
   

   To run a specific test case against a specific fabric profile, for example test.FabricBridgingTest for the fabric-spgw profile:

   ./run/tm/run fabric-spgw TEST=test.FabricBridgingTest
   

5. If tests fail, check logs in:

   • run/tm/log for tofino-model and stratum_bf logs
   • tests/ptf/ for PTF-related logs and PCAP traces (ptf.log and ptf.pcap)

Run tests on other targets

To run tests on targets other than BMv2 (e.g. a real Tofino-based switch), check the instructions available inside the tests/ptf directory.

Status

All test cases are executed daily against the ONOS 2.5 LTS branch (onos-2.5) using Travis CI. The current status is:

Migrating to Stratum Test Vectors

We are currently in the process of migrating the test runner framework from PTF to Stratum's testvectors-runner.

For this reason, some PTF-based test case definitions are currently instrumented to generate protobuf-based TestVectors (TVs) (under tests/ptf/testvectors). The long-term goal is to remove all PTF references from the codebase, but continue using Python as a convenient way to generate TVs. For now, we use a Python library (tvutils) to wrap PTF and P4Runtime calls in methods that generate TV's actions, stimuli and expectations instead of calling the corresponding PTF or P4Runtime gRPC methods.

Steps to generate TestVectors

TestVectors can be generated for bmv2 and tofino targets. The instructions are similar to running ptf tests on bmv2 and tofino model.

1. Obtain the fabric.p4 pre-compiled artifacts for BMv2 (bmv2.json and p4info.txt). These files are distributed with ONOS:

   git clone https://github.com/opennetworkinglab/onos
   

2. Set the ONOS_ROOT environment variable to the location where you just cloned the ONOS repo:

   export ONOS_ROOT=<path-to-onos>
   

3. Clone fabric-tofino repo and follow instructions to compile fabric.p4 for Tofino:

   https://github.com/opencord/fabric-tofino.git
   cd fabric-tofino
   <open README and follow instructions>
   

4. Set the FABRIC_TOFINO environment variable to the location where you cloned fabric-tofino:

   export FABRIC_TOFINO=<path-to-fabric-tofino>
   

5. Set the optional SDE_VERSION environment variable. Default value is 9.0.0

   export SDE_VERSION=9.0.0
   

6. Generate TestVectors using the run/tv/run script:

   ./run/tv/run <profile> [device] [portmap] [grpcaddr] [cpuport] [test-case]
   

   Default values for optional arguments are:

   1. device: tofino
   2. portmap: portmap.veth.json
   3. grpcaddr: 127.0.0.1:28000
   4. cpuport: 320 for tofino and 255 for bmv2

   Example command with all the optional arguments set:

   ./run/tv/run fabric DEVICE=tofino PORTMAP=port_map.hw.json GRPCADDR=10.128.13.111:28000 CPUPORT=320 TEST=test.FabricBridgingTest
   

   NOTE: Testing all profiles is not supported on this target. You must execute the run command for each profile.

   To generate TestVectors for the basic fabric profile:

   ./run/tv/run fabric
   

   To generate a specific test case for a specific fabric profile, for example test.FabricBridgingTest for the fabric-spgw profile:

   ./run/tv/run fabric-spgw TEST=test.FabricBridgingTest
   

Support

For help running the tests please write to the P4 Brigade mailing list:

https://groups.google.com/a/onosproject.org/forum/#!forum/brigade-p4