KubeArmor Performance Benchmarking Data

Benchmarking data

You can setup the benchmarking environment by following this guide.

Config

• Node: 3 e2-custom-2-4096 (2vCPU. 4GB RAM), 1 e2-standard-4 = 4 Node Cluster
• Platform - GKE
• Workload -> Microservices-Demo
• Tool -> Locust Loadgenerator (request at front-end service)

Summary

• With default configuration (Visibility=Process, Network, and with policy enforcement), KubeArmor induces a CPU performance overhead of <3%.
• The memory overhead with BPF-LSM was roughly 250MB. With AppArmor it is <100MB.
• The primary performance overhead is induced when telemetry is turned on. By default, the File visibility is turned off by KubeArmor.
• High performance overhead when File visibility is turned on. This is expected since there would be several file open events per second and these events have to be shipped to userspace and then to the control plane, incurring a huge overhead. Hence, KubeArmor keeps the File visibility turned off by default. Note that it is still possible to audit specific sensitive folder/file paths as necessary.

Without Kubearmor

Average

Scenario	Users	Kubearmor CPU	Kubearmor Relay CPU (m)	Throughput (req/s)	Failed Requests	Micro-service CPU (Frontend)	Micro-service CPU (CartService)	Micro-service CPU (CurrencyService)	Kubearmor Memory	Kubearmor Relay Memory
without kubearmor	2000	-	-	385.61	0	1015.6m (23 replica)	356.6 (3 replica)	1168.54 (23 replica)	-	-

Readings

Scenario	Users	Kubearmor CPU	Kubearmor Relay CPU (m)	Throughput (req/s)	Failed Requests	Micro-service CPU (Frontend)	Micro-service CPU (CartService)	Micro-service CPU (CurrencyService)	Kubearmor Memory	Kubearmor Relay Memory
without kubearmor	2000	-	-	379.21	0	1393.5m (23 replicas)	332.7m (3 replica)	1325.7m (25 replica)	-	-
without kubearmor	2000	-	-	389.4	0	954.3m (23 replica)	388.35m (3 replica)	1129m (25 replica)	-	-
without kubearmor	2000	-	-	388.14	0	910m (23 replica)	401m (3 replica)	1011.4m (25 replica)	-	-
without kubearmor	2000	-	-	382.72	0	863.73m (23 replica)	351.8m (3 replica)	1119.7m (25 replica)	-	-
without kubearmor	2000	-	-	388.6	0	957.17m (23 replica)	301.6m (3 replica)	1256.9m (25 replica)	-	-

---

BPF LSM benchmarking data

Scenario	Users	Kubearmor CPU (Averaged Over 4 Replicas)	Kubearmor Memory (Averaged Over 4 Replicas)	Throughput (req/s)	Kubearmor Relay CPU (m)	Kubearmor Relay Memory	Percentage Drop	Micro-service CPU (Frontend)	Micro-service CPU (CartService)	Micro-service CPU (CurrencyService)
with Kubearmor (without policy)	2000	44.25m	236.2M	380.25	58.5m	32.9M	1.3	900.4m (23 replica)	367.85m (3 replica)	1032m (25 replica)
with Kubearmor (with process policy)	2000	47.28m	239.4M	381.08	64.6m	33.8M	1.1	904.2m (23 replica)	346.2m (3 replica)	1049.2m (25 replica)
with Kubearmor (with process, network policy)	2000	45.37m	238.8M	374.3	61.4m	33.4M	2.9	916m (23 replica)	330.65m (3 replica)	1076.5m (25 replica)
with Kubearmor (with process, network, file policy)	2000	45.52m	241.2M	368.58	66.6m	34.2M	4.4	894.5m (23 replica)	331.75m (3 replica)	1173.6m (25 replica)

AppArmor LSM Benchmarking Data

Pure AppArmor Benchmark, without System Monitor

Scenario	Users	Kubearmor CPU (Averaged Over 4 Replicas)	Kubearmor Memory (Averaged Over 4 Replicas)	Throughput (req/s)	Kubearmor Relay CPU (m)	Kubearmor Relay Memory	Percentage Drop	Micro-service CPU (Frontend)	Micro-service CPU (CartService)	Micro-service CPU (CurrencyService)
with Kubearmor (without policy)	2000	38.18m	89.43M	376.1	42.75m	51.32M	2.4	897.8m (23 replicas)	339.6m (3 replica)	1075.2m (25 replica)
with Kubearmor (with process policy)	2000	37.25m	98.45M	372.2	54.1m	49.5M	3.4	872.3 (23 replicas)	354.9m (3 replica)	1132.4m (25 replica)
with Kubearmor (with process, network policy)	2000	38.65m	98.67M	371.3	55.2m	48.5M	3.7	863.4m (23 replicas)	354.2m (3 replica)	1125.9m (25 replica)
with Kubearmor (with process, network, file policy)	2000	37.80m	98.35M	368.6	54.4m	50.12M	4.4	875m (23 replicas)	413.25m (3 replica)	1139.6m(25 replica)

Enabled System Monitor, Visbility set to None

We enable system monitor, so as we receive policy violation alerts

Scenario	Users	Kubearmor CPU (Averaged Over 4 Replicas)	Kubearmor Memory (Averaged Over 4 Replicas)	Throughput (req/s)	Kubearmor Relay CPU (m)	Kubearmor Relay Memory	Percentage Drop	Micro-service CPU (Frontend)	Micro-service CPU (CartService)	Micro-service CPU (CurrencyService)
with Kubearmor (without policy)	2000	49.62m	337.5M	376.7	54.1m	36.9M	2.3	969.9m (23 replica)	352.3m (3 replica)	1250.5m (25 replica)
with Kubearmor (with process policy)	2000	52.12m	325M	374.6	60.7m	35.8M	2.8	954.6m (23 replica)	346.3m (3 replica)	1246.8m (25 replica)
with Kubearmor (with process, network policy)	2000	53.40m	302.5M	373.2	64.6m	37.4M	3.2	989.4m (23 replica)	318.4m (3 replica)	1297.6m (25 replica)
with Kubearmor (with process, network, file policy)	2000	54.37m	302.5M	363.5	62.2m	37.2M	5.7	955.4m (23 replica)	362.7m (3 replica)	1209.1m (25 replica)

Benchmarking data with different modes of visibility

Scenario	Users	Kubearmor CPU (Averaged Over 4 Replicas)	Kubearmor Memory (Averaged Over 4 Replicas)	Throughput (req/s)	Kubearmor Relay CPU (m)	Kubearmor Relay Memory	Percentage Drop	Micro-service CPU (Frontend)	Micro-service CPU (CartService)	Micro-service CPU (CurrencyService)
with Kubearmor (visibility: none)	2000	49.22m	300M	373.3	61.04m	39.7M (1 replica)	3.2	1124.6m (21 replica)	345.17m (3 replica)	994.6m (23 replica)
with Kubearmor (visibility: process)	2000	46.77m	282.5M	375	60.4m	35.9M (1 replica)	2.7	1125.9m (21 replica)	335.1m (3 replica)	1246.9m (23 replica)
with Kubearmor (visibility: process, file)	2000	59.35m	275M	340.5	58.3m	34.83M (1 replica)	11.6	1232.5m (22 replica)	296.11m (2 replica)	1128.03m (22 replica))
with Kubearmor (visibility: process, network)	2000	64.27m	265M	354.8	94.75m	36.3M (1 replica)	7.9	1209.9m (20 replica)	335.02m (3 replica)	1136.1m (22 replica)
with Kubearmor (visibility: process, network, file)	2000	68.82m	277.5M	312.86	93.2m	35.2M (1 replica)	18	1237.1m (23 replica)	321.2 (3 replica)	1430.4 (26 replica)