OSDOCS-11555:Configuring network observability network policy

OSDOCS-10036:Exporting Network Observability metrics to OpenTelemetry

OSDOCS-11731: Developer perspective for Network Observability

OSDOCS-10877: Virtualization in Network Observability

NetObserv FlowMetric API regeneration

Network Observability 1.7 regenerate flows format doc

OSDOCS-11821: NetObserv CLI updates

NetObserv 1.7 FlowCollector API regeneration

Network Observability 1.7 Release Notes

Author: skrthomas

_topic_maps/_topic_map.yml

@@ -2680,6 +2680,8 @@ Topics:
     File: network-observability-operator-monitoring
   - Name: Scheduling resources
     File: network-observability-scheduling-resources
+  - Name: Secondary networks
+    File: network-observability-secondary-networks
   - Name: Network Observability CLI
     Dir: netobserv_cli
     Topics:

modules/network-observability-SRIOV-configuration.adoc

@@ -1,6 +1,6 @@
 // Module included in the following assemblies:
 //
-// * network_observability/configuring-operator.adoc
+// * observability/network_observability/network-observability-secondary-networks.adoc
 
 :_mod-docs-content-type: PROCEDURE
 [id="network-observability-SR-IOV-config_{context}"]
@@ -16,7 +16,7 @@ In order to collect traffic from a cluster with a Single Root I/O Virtualization
 . Under the *Provided APIs* heading for the *NetObserv Operator*, select *Flow Collector*.
 . Select *cluster* and then select the *YAML* tab.
 . Configure the `FlowCollector` custom resource. A sample configuration is as follows:
-
++
 .Configure `FlowCollector` for SR-IOV monitoring
 [source,yaml]
 ----

modules/network-observability-cli-capturing-flows.adoc

@@ -25,6 +25,7 @@ $ oc netobserv flows --enable_filter=true --action=Accept --cidr=0.0.0.0/0 --pro
 ----
 live table filter: [SrcK8S_Zone:us-west-1b] press enter to match multiple regular expressions at once
 ----
+. Use the *PageUp* and *PageDown* keys to toggle between *None*, *Resource*, *Zone*, *Host*, *Owner* and *all of the above*.
 . To stop capturing, press kbd:[Ctrl+C]. The data that was captured is written to two separate files in an `./output` directory located in the same path used to install the CLI. 
 . View the captured data in the `./output/flow/<capture_date_time>.json` JSON file, which contains JSON arrays of the captured data.
 +

modules/network-observability-cli-capturing-packets.adoc

@@ -16,14 +16,15 @@ You can capture packets using the Network Observability CLI.
 +
 [source,terminal]
 ----
-$ oc netobserv packets tcp,80
+$ oc netobserv packets --action=Accept --cidr=0.0.0.0/0 --protocol=TCP --port=49051
 ----
 . Add filters to the `live table filter` prompt in the terminal to refine the incoming packets. An example filter is as follows:
 + 
 [source,terminal]
 ----
 live table filter: [SrcK8S_Zone:us-west-1b] press enter to match multiple regular expressions at once
 ----
+. Use the *PageUp* and *PageDown* keys to toggle between *None*, *Resource*, *Zone*, *Host*, *Owner* and *all of the above*.
 . To stop capturing, press kbd:[Ctrl+C].
 . View the captured data, which is written to a single file in an `./output/pcap` directory located in the same path that was used to install the CLI:
 .. The `./output/pcap/<capture_date_time>.pcap` file can be opened with Wireshark. 

modules/network-observability-create-network-policy.adoc

@@ -6,7 +6,54 @@
 :_mod-docs-content-type: PROCEDURE
 [id="network-observability-network-policy_{context}"]
 = Creating a network policy for Network Observability
-You might need to create a network policy to secure ingress traffic to the `netobserv` namespace. In the web console, you can create a network policy using the form view.
+If you want to further customize the network policies for the `netobserv` and `netobserv-privileged` namespaces, you must disable the managed installation of the policy from the `FlowCollector` CR, and create your own. You can use the network policy resources that are enabled from the `FlowCollector` CR as a starting point for the procedure that follows:
+
+.Example `netobserv` network policy
+[source,yaml]
+----
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+spec:
+  ingress:
+  - from:
+    - podSelector: {}
+    - namespaceSelector:
+        matchLabels:
+          kubernetes.io/metadata.name: netobserv-privileged
+  - from:
+    - namespaceSelector:
+        matchLabels:
+          kubernetes.io/metadata.name: openshift-console
+    ports:
+    - port: 9001
+      protocol: TCP
+  - from:
+    - namespaceSelector:
+        matchLabels:
+          kubernetes.io/metadata.name: openshift-monitoring
+  podSelector: {}
+  policyTypes:
+  - Ingress
+----
+
+.Example `netobserv-privileged` network policy
+[source,yaml]
+----
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: netobserv
+  namespace: netobserv-privileged
+spec:
+  ingress:
+  - from:
+    - namespaceSelector:
+        matchLabels:
+          kubernetes.io/metadata.name: openshift-monitoring
+  podSelector: {}
+  policyTypes:
+  - Ingress
+----
 
 .Procedure
 . Navigate to *Networking* -> *NetworkPolicies*.

modules/network-observability-deploy-network-policy.adoc

@@ -0,0 +1,40 @@
+// Module included in the following assemblies:
+
+// * networking/network_observability/network-observability-network-policy.adoc
+
+
+:_mod-docs-content-type: PROCEDURE
+[id="network-observability-deploy-network-policy_{context}"]
+= Configuring an ingress network policy by using the FlowCollector custom resource
+You can configure the `FlowCollector` custom resource (CR) to deploy an ingress network policy for Network Observability by setting the `spec.NetworkPolicy.enable` specification to `true`. By default, the specification is `false`.
+
+If you have installed Loki, Kafka or any exporter in a different namespace that also has a network policy, you must ensure that the Network Observability components can communicate with them. Consider the following about your setup:
+
+   * Connection to Loki (as defined in the `FlowCollector` CR `spec.loki` parameter)
+   * Connection to Kafka (as defined in the `FlowCollector` CR `spec.kafka` parameter)
+   * Connection to any exporter (as defined in FlowCollector CR `spec.exporters` parameter)
+   * If you are using Loki and including it in the policy target, connection to an external object storage (as defined in your `LokiStack` related secret)
+
+.Procedure
+. . In the web console, go to *Operators* -> *Installed Operators* page.
+. Under the *Provided APIs* heading for *Network Observability*, select *Flow Collector*.
+. Select *cluster* then select the *YAML* tab.
+. Configure the `FlowCollector` CR. A sample configuration is as follows:
++
+[id="network-observability-flowcollector-configuring-network-policy_{context}"]
+.Example `FlowCollector` CR for network policy
+[source, yaml]
+----
+apiVersion: flows.netobserv.io/v1beta2
+kind: FlowCollector
+metadata:
+  name: cluster
+spec:
+  namespace: netobserv
+  networkPolicy:
+    enable: true   <1>
+    additionalNamespaces: ["openshift-console", "openshift-monitoring"] <2>
+# ...
+----
+<1> By default, the `enable` value is `false`.
+<2> Default values are `["openshift-console", "openshift-monitoring"]`.

modules/network-observability-enriched-flows.adoc

@@ -6,10 +6,11 @@
 [id="network-observability-enriched-flows_{context}"]
 = Export enriched network flow data
 
-You can send network flows to Kafka, IPFIX, or both at the same time. Any processor or storage that supports Kafka or IPFIX input, such as Splunk, Elasticsearch, or Fluentd, can consume the enriched network flow data.
+You can send network flows to Kafka, IPFIX, the Red{nbsp}Hat build of OpenTelemetry, or all three at the same time. For Kafka or IPFIX, any processor or storage that supports those inputs, such as Splunk, Elasticsearch, or Fluentd, can consume the enriched network flow data. For OpenTelemetry, network flow data and metrics can be exported to a compatible OpenTelemetry endpoint, such as Red{nbsp}Hat build of OpenTelemetry, Jaeger, or Prometheus. 
 
 .Prerequisites
-* Your Kafka or IPFIX collector endpoint(s) are available from Network Observability `flowlogs-pipeline` pods.
+* Your Kafka, IPFIX, or OpenTelemetry collector endpoints are available from Network Observability `flowlogs-pipeline` pods.
+
 
 .Procedure
 
@@ -26,22 +27,41 @@ metadata:
   name: cluster
 spec:
   exporters:
-  - type: Kafka                         <3>
+  - type: Kafka                         <1>
       kafka:
         address: "kafka-cluster-kafka-bootstrap.netobserv"
-        topic: netobserv-flows-export   <1>
+        topic: netobserv-flows-export   <2>
         tls:
-          enable: false                 <2>
-  - type: IPFIX                         <3>
+          enable: false                 <3>
+  - type: IPFIX                         <1>
       ipfix:
         targetHost: "ipfix-collector.ipfix.svc.cluster.local"
         targetPort: 4739
         transport: tcp or udp           <4>
-
-
+ -  type: OpenTelemetry                 <1>
+      openTelemetry:
+        targetHost: my-otelcol-collector-headless.otlp.svc
+        targetPort: 4317
+        type: grpc                      <5>
+        logs:                           <6>
+          enable: true
+        metrics:                        <7>
+          enable: true
+          prefix: netobserv
+          pushTimeInterval: 20s         <8>
+          expiryTime: 2m
+   #    fieldsMapping:                  <9>
+   #      input: SrcAddr
+   #      output: source.address
 ----
-<1> The Network Observability Operator exports all flows to the configured Kafka topic.
-<2> You can encrypt all communications to and from Kafka with SSL/TLS or mTLS. When enabled, the Kafka CA certificate must be available as a ConfigMap or a Secret, both in the namespace where the `flowlogs-pipeline` processor component is deployed (default: netobserv). It must be referenced with `spec.exporters.tls.caCert`. When using mTLS, client secrets must be available in these namespaces as well (they can be generated for instance using the AMQ Streams User Operator) and referenced with `spec.exporters.tls.userCert`.
-<3> You can export flows to IPFIX instead of or in conjunction with exporting flows to Kafka.
+<1> You can export flows to IPFIX, OpenTelemetry, and Kafka individually or concurrently.
+<2> The Network Observability Operator exports all flows to the configured Kafka topic.
+<3> You can encrypt all communications to and from Kafka with SSL/TLS or mTLS. When enabled, the Kafka CA certificate must be available as a ConfigMap or a Secret, both in the namespace where the `flowlogs-pipeline` processor component is deployed (default: netobserv). It must be referenced with `spec.exporters.tls.caCert`. When using mTLS, client secrets must be available in these namespaces as well (they can be generated for instance using the AMQ Streams User Operator) and referenced with `spec.exporters.tls.userCert`. 
 <4> You have the option to specify transport. The default value is `tcp` but you can also specify `udp`.
-. After configuration, network flows data can be sent to an available output in a JSON format. For more information, see _Network flows format reference_.
+<5> The protocol of OpenTelemetry connection. The available options are `http` and `grpc`.
+<6> OpenTelemetry configuration for exporting logs, which are the same as the logs created for Loki. 
+<7> OpenTelemetry configuration for exporting metrics, which are the same as the metrics created for Prometheus. These configurations are specified in the `spec.processor.metrics.includeList` parameter of the `FlowCollector` custom resource, along with any custom metrics you defined using the `FlowMetrics` custom resource.
+<8> The time interval that metrics are sent to the OpenTelemetry collector.
+<9> *Optional*:Network Observability network flows formats get automatically renamed to an OpenTelemetry compliant format. The `fieldsMapping` specification gives you the ability to customize the OpenTelemetry format output. For example in the YAML sample, `SrcAddr` is the Network Observability input field, and it is being renamed `source.address` in OpenTelemetry output. You can see both Network Observability and OpenTelemetry formats in the "Network flows format reference".
+
+After configuration, network flows data can be sent to an available output in a JSON format. For more information, see "Network flows format reference".