diff --git a/docs/loadbalancer/ONVM_LB_TopologyDoc.pdf b/docs/loadbalancer/ONVM_LB_TopologyDoc.pdf index f852ae7c4..bdba5f5ed 100644 Binary files a/docs/loadbalancer/ONVM_LB_TopologyDoc.pdf and b/docs/loadbalancer/ONVM_LB_TopologyDoc.pdf differ diff --git a/docs/loadbalancer/index.rst b/docs/loadbalancer/index.rst index d29c3e5c8..468e1d8b9 100644 --- a/docs/loadbalancer/index.rst +++ b/docs/loadbalancer/index.rst @@ -7,7 +7,7 @@ Getting Started Overview ----------------- -- In the following tutorial, we will explore a means of deploying and testing ONVM’s example Layer-3 round-robin load balancer. To do this, we will instantiate a Cloudlab experiment using the :code:`ONVM_LoadBalancer` profile; this topology (shown below) includes two backend servers and a single client, in addition to the ONVM load balancer. +- In the following tutorial, we will explore a means of deploying and testing ONVM’s load balancer. To do this, we will instantiate a Cloudlab experiment using the :code:`ONVM_LoadBalancer` profile; this topology (shown below) includes two backend servers and a single client, in addition to the ONVM load balancer. .. image:: ../images/lb-1.png @@ -15,18 +15,23 @@ Overview Cloudlab Node Setup ----------------- -- Open Cloudlab and start a new experiment. When prompted to choose a profile, select :code:`ONVM_LoadBalancer`. For this tutorial, set the number of backend servers to 2, and set the OS Image to :code:`ONVM UBUNTU20.04`. With regards to defining the physical node type, we will leave this blank and expect the default c220g2 nodes. In the following section (under "Finalize"), select the :code:`Cloudlab Wisconsin` cluster, and all remaining inputs are discretionary. +- Open Cloudlab and start a new experiment. When prompted to choose a profile, select :code:`ONVM_LoadBalancer`. For this tutorial, set the number of backend servers to 2, and set the OS Image to :code:`ONVM & Sledge UBUNTU20.04`. With regards to defining the physical node type, we will leave this blank and expect the default c220g2 nodes. In the following section (under "Finalize"), select the :code:`Cloudlab Wisconsin` cluster, and all remaining inputs are discretionary. - Begin by SSHing into each node within the experiment, and download the **Load Balancer Topology Template** `here `_. If you are using any Apple product to complete this tutorial, avoid using Preview as your PDF editor; autofill scripts will not apply. Google Chrome or Adobe Acrobat are viable alternatives. -- For every node, use :code:`ifconfig` to view all available network interfaces. Record the appropriate name, IPv4 (inet), and MAC address (ether) for each network interface in the Topology Template, as shown below. Note that the client side and server side nodes should be on a different IP subnets. The ONVM_LB node requires the use of two ports: one for connection to the client and one for connecting to the servers. It is recommended that you use the 10-Gigabit SFI/SFP+ network connections. Port IDs will be handled later. +- For every node, use :code:`ifconfig` to view all available network interfaces. Record the appropriate name, IPv4 (inet), and MAC address (ether) for each network interface in the Topology Template, as shown below. Note that the client side and server side nodes should be on a different IP subnets and the first 3 values of the client ip must match it's serverside onvm port. The first 3 values of the servers and their onvm port must also match. The ONVM_LB node requires the use of two ports: one for connection to the client and one for connecting to the servers. It is recommended that you use the 10-Gigabit SFI/SFP+ network connections. Port IDs will be handled later. .. image:: ../images/lb-2.png -- In the ONVM LB node, set up the environment using :code:`setup_cloudlab.sh` in the scripts directory. Once the ports have been successfully bound to the DPDK-compatible driver, start the manager with at least two available ports. Listed below are the abbreviated steps for binding available ports to the DPDK-bound driver. To start the manager, you may use :code:`./onvm/go.sh -k 3 -n 0xFF -s stdout`. - - #. Unbind the connected NICs: :code:`sudo ifconfig down` where represents the interface name (eg. :code:`enp6s0f0`) +Running ONVM Manager +----------------- + +- In the ONVM LB node + + #. Unbind both of the connected NICs: :code:`sudo ifconfig down` where represents the interface name (eg. :code:`enp6s0f0`) #. Navigate to the :code:`/local/onvm/openNetVM/scripts` directory and bind the NICs to DPDK using the command :code:`source ./setup_cloudlab.sh` #. Ensure that you see the two NICs in the section defining “Network devices using DPDK-compatible driver.” If you only see one NIC, it’s possible that you did not unbind the other NIC from the kernel driver using :code:`sudo ifconfig down`. Repeat step (i). - #. Navigate back to the openNetVM folder (:code:`cd ..`) and compile the Manager using :code:`cd onvm && make && cd ..` + #. Navigate back to the openNetVM folder (:code:`cd ..`), and compile the Manager using :code:`cd onvm && make && cd ..` + #. Navigate to examples folder :code:`cd examples && make && cd ..` + #. Run the manager :code:`./onvm/go.sh -k 3 -n 0xFF -s stdout` - At the top of the manager display (pictured below), you can observe two (or more) port IDs and their associated MAC addresses. Use these ID mappings to complete the Port ID sections of the **Topology Template**. @@ -34,10 +39,11 @@ Cloudlab Node Setup - Now that the **Topology Template** is complete, all commands within the PDF document should be populated. To complete our LB configuration, we must: - #. Specify the backend servers’ port information in **server.conf** + #. Specify the backend servers’ port information in **server.json** #. Define a static route in each of the backend servers to specify the correct gateway for which traffic will enter. -- To specify the server information for the load balancer, go to :code:`/examples/load_balancer/server.conf` and copy the information that is shown in the bottom left quadrant of your **Topology Template**. This includes the *"LIST_SIZE 2"* and the IP+MAC address of each port. +- To specify the server information for the load balancer, go to :code:`/examples/load_balancer/server.json` and copy the json code that is shown in the bottom left quadrant of your **Topology Template**. This includes the *"list_size 2"*, the policy, and the IP + MAC address + weight of each port. +- The Load Balancer currently supports random ("random"), weighted random ("weighted_random"), and round robin ("rrobin") packet distribution policies. The policy may be selected in the json using the strings provided in parenthesis. Weight fields are only required for weighted random policy. - To define the static routes, navigate to the two backend nodes (Server1 and Server2) and execute the respective commands shown on the bottom-center area of the **Topology Template**. This includes the :code:`sudo ip route add *` command for each server. Running The Load Balancer diff --git a/examples/Makefile b/examples/Makefile index a7b41bc26..e064ccd51 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -40,7 +40,9 @@ $(error "Please define RTE_SDK environment variable") endif # To add new examples, append the directory name to this variable -examples = bridge basic_monitor simple_forward speed_tester flow_table test_flow_dir aes_encrypt aes_decrypt flow_tracker load_balancer arp_response nf_router scaling_example load_generator payload_scan firewall simple_fwd_tb l2fwd test_messaging l3fwd fair_queue + +examples = bridge basic_monitor simple_forward speed_tester flow_table test_flow_dir aes_encrypt aes_decrypt flow_tracker load_balancer arp_response nf_router scaling_example load_generator payload_scan firewall simple_fwd_tb l2fwd test_messaging l3fwd fair_queue skeleton + ifeq ($(NDPI_HOME),) $(warning "Skipping ndpi_stats NF as NDPI_HOME is not set") diff --git a/examples/skeleton/Makefile b/examples/skeleton/Makefile new file mode 100644 index 000000000..6505a1485 --- /dev/null +++ b/examples/skeleton/Makefile @@ -0,0 +1,67 @@ +# openNetVM +# https://github.com/sdnfv/openNetVM +# +# BSD LICENSE +# +# Copyright(c) +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# The name of the author may not be used to endorse or promote +# products derived from this software without specific prior +# written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + + +# Default target, can be overriden by command line or environment +include $(RTE_SDK)/mk/rte.vars.mk +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +# binary name [EDIT & ADD NF TAG TO "examples" VARIABLE WITHIN MAKEFILE OF EXAMPLES DIRECTORY] +APP = skeleton + +# all source are stored in SRCS-y +SRCS-y := skeleton.c + +ONVM= $(SRCDIR)/../../onvm + +CFLAGS += $(WERROR_FLAGS) -O3 $(USER_FLAGS) + +CFLAGS += -I$(ONVM)/onvm_nflib +CFLAGS += -I$(ONVM)/lib +LDFLAGS += $(ONVM)/onvm_nflib/$(RTE_TARGET)/libonvm.a +LDFLAGS += $(ONVM)/lib/$(RTE_TARGET)/lib/libonvmhelper.a -lm + +# workaround for a gcc bug with noreturn attribute +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) +CFLAGS_main.o += -Wno-return-type +endif + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/examples/skeleton/README.md b/examples/skeleton/README.md new file mode 100644 index 000000000..fdd66f5d8 --- /dev/null +++ b/examples/skeleton/README.md @@ -0,0 +1,32 @@ +Skeleton +== +This is an example NF that acts as a basic skeleton NF. + +Compilation and Execution +-- +``` +cd examples +make +cd skeleton +./go.sh SERVICE_ID [-p TIME_PRINT_DELAY | -v PACKET_PRINT_DELAY] + +OR + +./go.sh -F CONFIG_FILE -- -- [-p PRINT_DELAY] + +OR + +sudo ./build/skeleton -l CORELIST -n 3 --proc-type=secondary -- -r SERVICE_ID -- [-p TIME_PRINT_DELAY | -v PACKET_PRINT_DELAY] +``` + +App Specific Arguments +-- + - `-p `: time between each print, e.g. `-p 1` prints after every second. + - `-v `: number of packets between each print, e.g. `-v 1` prints after every packet. + +Config File Support +-- +This NF supports the NF generating arguments from a config file. For +additional reading, see [Examples.md](../../docs/Examples.md) + +See `../example_config.json` for all possible options that can be set. diff --git a/examples/skeleton/go.sh b/examples/skeleton/go.sh new file mode 100755 index 000000000..03fc3bb36 --- /dev/null +++ b/examples/skeleton/go.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +#The go.sh script is a convinient way to run start_nf.sh without specifying NF_NAME + +NF_DIR=${PWD##*/} + +if [ ! -f ../start_nf.sh ]; then + echo "ERROR: The ./go.sh script can only be used from the NF folder" + echo "If running from other directory use examples/start_nf.sh" + exit 1 +fi + +# only check for running manager if not in Docker +if [[ -z $(pgrep -u root -f "/onvm/onvm_mgr/.*/onvm_mgr") ]] && ! grep -q "docker" /proc/1/cgroup +then + echo "NF cannot start without a running manager" + exit 1 +fi + +../start_nf.sh "$NF_DIR" "$@" diff --git a/examples/skeleton/skeleton.c b/examples/skeleton/skeleton.c new file mode 100644 index 000000000..36044b552 --- /dev/null +++ b/examples/skeleton/skeleton.c @@ -0,0 +1,329 @@ +/********************************************************************* + * openNetVM + * https://sdnfv.github.io + * + * BSD LICENSE + * + * Copyright(c) + * 2015-2021 George Washington University + * 2015-2021 University of California Riverside + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * skeleton.c - Template NF for development. [EDIT] + ********************************************************************/ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "onvm_nflib.h" +#include "onvm_pkt_helper.h" + +// Define NF Tag Appropriately [EDIT] +#define NF_TAG "skeleton" + +/* + * State structs [EDIT] + * Holds any variables which are universal to all files within the NF + * Listed below are common variables used within the struct + */ +struct skeleton_state { + + int displayType; + uint64_t start_time; + uint32_t delay; + uint32_t current_time; + uint32_t packets_processed; +}; + +/* + * Print a usage message + */ +static void +usage(const char *progname) { + printf("Usage:\n"); + printf("%s [EAL args] -- [NF_LIB args]\n", progname); + + // Additional Usage Messages [EDIT] + printf("%s -F [EAL args] -- [NF_LIB args] -- [NF args]\n\n", progname); + printf("Flags:\n"); + printf(" - `-p `: number of packets between each print, e.g. `-p 1` prints every packets.\n"); +} + +/* + * Parse the application arguments. + */ +static int +parse_app_args(int argc, char *argv[], const char *progname, struct skeleton_state *skeleton_data) { + int c; + + /* User can specify how they would like to display data. */ + + skeleton_data->displayType = -1; // displayType -1 = No Display + while ((c = getopt(argc, argv, "p:v:")) != -1) { + switch (c) { + case 'p': + skeleton_data->delay = strtoul(optarg, NULL, 10); + if (skeleton_data->delay < 1) { + RTE_LOG(INFO, APP, "Option -%c requires an argument.\n", optopt); + return -1; + } + skeleton_data->displayType = 0; // displayType 0 = Time-Based Delays + return optind; + case 'v': + skeleton_data->delay = strtoul(optarg, NULL, 10); + if (skeleton_data->delay < 1) { + RTE_LOG(INFO, APP, "Option -%c requires an argument.\n", optopt); + return -1; + } + skeleton_data->displayType = 1; // displayType 1 = Packet-Based Delays + return optind; + case '?': + usage(progname); + if (optopt == 'p') + RTE_LOG(INFO, APP, "Option -%c requires an argument.\n", optopt); + else if (optopt == 'v') + RTE_LOG(INFO, APP, "Option -%c requires an argument.\n", optopt); + else if (isprint(optopt)) + RTE_LOG(INFO, APP, "Unknown option `-%c'.\n", optopt); + else + RTE_LOG(INFO, APP, "Unknown option character `\\x%x'.\n", optopt); + return -1; + default: + usage(progname); + return -1; + } + } + return optind; +} + +/* + * Function for NFs with Displays + * [EDIT IF NF USES DISPLAY; ELSE DELETE] + */ +static void +do_stats_display(struct skeleton_state *skeleton_data) { + + const char clr[] = {27, '[', '2', 'J', '\0'}; + const char topLeft[] = {27, '[', '1', ';', '1', 'H', '\0'}; + + /* Clear screen and move to top left */ + printf("%s%s", clr, topLeft); + + printf("PACKETS\n"); + printf("-----\n"); + printf("Number of packet processed : %d\n", skeleton_data->packets_processed); + printf("Time : %d\n", skeleton_data->current_time); + + printf("\n\n"); +} + + +/* + * Handles each packet upon arrival [EDIT] + */ +static int +packet_handler(__attribute__((unused)) struct rte_mbuf *pkt, struct onvm_pkt_meta *meta, + struct onvm_nf_local_ctx *nf_local_ctx) { + + struct skeleton_state *skeleton_data; + skeleton_data = (struct skeleton_state *)nf_local_ctx->nf->data; + skeleton_data->packets_processed++; + + /* + * Timed display current_time (current_time stored in skeleton_state - keep/edit if NF uses display) + * After the reception of each packet, the current_time is incremented. Once we reach a defined + * number of packets, display and reset the current_time + */ + + if (skeleton_data->displayType == 1) { + if (skeleton_data->packets_processed % skeleton_data->delay == 0) { + do_stats_display(skeleton_data); + } + } + + /* + * meta->action + * Defines what to do with the packet. Actions defined in onvm_common.h + * meta->destination + * Defines the service ID of the NF which will receive the packet. + * + * Possible actions: + * ONVM_NF_ACTION_DROP - Drop packet + * NVM_NF_ACTION_NEXT - Go to whatever the next action is configured by the SDN controller in the flow table + * ONVM_NF_ACTION_TONF - Send the packet to the NF specified in the argument field (assume it is on the same host) + * ONVM_NF_ACTION_OUT - Send the packet out the NIC port set in the argument field + */ + meta->action = ONVM_NF_ACTION_DROP; + + /* Return 0 on success, -1 on failure */ + return 0; +} + +/* + * Performs action continuously; called every run loop regardless of packet reception [EDIT] + */ +static int +action(struct onvm_nf_local_ctx *nf_local_ctx){ + + struct skeleton_state *skeleton_data; + skeleton_data = (struct skeleton_state *)nf_local_ctx->nf->data; + + /* + * Get the current time, and, if the user has specified a timer-based print delay, + * check whether it is time to print + */ + __uint64_t time = (rte_get_tsc_cycles() - skeleton_data->start_time) / rte_get_timer_hz(); + if (skeleton_data->displayType == 0) { + __uint64_t delay = skeleton_data->delay; + __uint64_t current_time = skeleton_data->current_time; + if ((time%delay == 0) && (time != current_time)) { + skeleton_data->current_time = time; + do_stats_display(skeleton_data); + return 0; + } + } + + /* Update the current time */ + skeleton_data->current_time = time; + return 0; +} + +/* + * Initial setup function; called before NF receives any packets/messages [EDIT] + */ +static void +setup(struct onvm_nf_local_ctx *nf_local_ctx){ + + /* + * Initialize variables within state struct that must be defined + * before receiving packets/messages + */ + + struct skeleton_state *skeleton_data = nf_local_ctx->nf->data; + skeleton_data->start_time = rte_get_tsc_cycles(); + skeleton_data->current_time = 0; + skeleton_data->packets_processed = 0; + if (skeleton_data->displayType > -1) { + do_stats_display(skeleton_data); + } +} + +/* + * Handles each message upon arrival [EDIT] + */ +static void +handle_msg(__attribute__((unused))void *msg_data, __attribute__((unused))struct onvm_nf_local_ctx *nf_local_ctx) { + + /* Remove __attribute__((unused)) once implemented */ + +} + +/* + * Creates function table and local context. Runs NF. + */ +int +main(int argc, char *argv[]) { + + /* Handles command line arguments */ + int arg_offset; + + /* Local context holds NF and status */ + struct onvm_nf_local_ctx *nf_local_ctx; + + /* Declare function table: Holds pointers to the NF methods - setup, msg_handler, user_actions, and pkt_handler */ + struct onvm_nf_function_table *nf_function_table; + + /* Handles command line arguments */ + const char *progname = argv[0]; + + /* Initialize local context and start default signal handler */ + nf_local_ctx = onvm_nflib_init_nf_local_ctx(); + onvm_nflib_start_signal_handler(nf_local_ctx, NULL); + + /* Initialize function table and respective pointers to NF methods: */ + nf_function_table = onvm_nflib_init_nf_function_table(); + nf_function_table->pkt_handler = &packet_handler; + nf_function_table->setup = &setup; + nf_function_table->user_actions = &action; + nf_function_table->msg_handler = &handle_msg; + + /* If a termination signal is received or initiation is interrupted, exit */ + if ((arg_offset = onvm_nflib_init(argc, argv, NF_TAG, nf_local_ctx, nf_function_table)) < 0) { + onvm_nflib_stop(nf_local_ctx); + if (arg_offset == ONVM_SIGNAL_TERMINATION) { + printf("Exiting due to user termination\n"); + return 0; + } else { + rte_exit(EXIT_FAILURE, "Failed ONVM init\n"); + } + } + + /* Command line arguments */ + argc -= arg_offset; + argv += arg_offset; + + /* + * (1) Declare and initialize state struct to hold non-local data within the NF. Use rte_zmalloc + * void * rte_zmalloc (const char * type, size_t size, unsigned align) + * type: A string identifying the type of allocated objects. Can be NULL. + * size: Size (in bytes) to be allocated. + * align: If 0, the return is a pointer that is suitably aligned for any kind of variable + * (2) Assign the nf_local_ctx->nf->data pointer to the struct. + */ + struct skeleton_state *skeleton_data; + skeleton_data = (struct skeleton_state *) rte_zmalloc ("skeleton", sizeof(struct skeleton_state), 0); + nf_local_ctx->nf->data = (void *) skeleton_data; + + /* Invalid command-line argument handling */ + if (parse_app_args(argc, argv, progname, skeleton_data) < 0) { + onvm_nflib_stop(nf_local_ctx); + rte_exit(EXIT_FAILURE, "Invalid command-line arguments\n"); + } + + /* Begin running NF */ + onvm_nflib_run(nf_local_ctx); + + /* Once the NF has stopped running, free and stop */ + onvm_nflib_stop(nf_local_ctx); + printf("If we reach here, program is ending\n"); + return 0; +}