add controller-driven kube-vip load balancer for K3S_BASE clusters#5683
Open
naiming-zededa wants to merge 2 commits intolf-edge:masterfrom
Open
add controller-driven kube-vip load balancer for K3S_BASE clusters#5683naiming-zededa wants to merge 2 commits intolf-edge:masterfrom
naiming-zededa wants to merge 2 commits intolf-edge:masterfrom
Conversation
ceda466 to
9c5b309
Compare
| // collectLBPoolStatus reads the kubevip ConfigMap from kube-system to get the configured | ||
| // load balancer pool, and gathers IPs currently allocated to LoadBalancer-type services. | ||
| // Returns nil if the kubevip ConfigMap does not exist (kubevip not yet deployed). | ||
| func collectLBPoolStatus(clientset *kubernetes.Clientset, services []types.KubeServiceInfo) *types.KubeLBPoolStatus { |
There was a problem hiding this comment.
Do we publish this LBPoolStatus in any eve pub/sub calls for collect info ?
Contributor
Author
There was a problem hiding this comment.
this LBPoolStatus is published by zedkube in KubeUserServices, so it is part of the collectinfo.
This PR implements controller-driven Kubernetes LoadBalancer services
for CLUSTER_TYPE_K3S_BASE in eve-k.
- pkg/pillar/types/clustertypes.go: Add LBInterfaceConfig (interface +
CIDR string) and LBInterfaces []LBInterfaceConfig to both
EdgeNodeClusterConfig and EdgeNodeClusterStatus.
- pkg/pillar/cmd/zedagent/parseconfig.go: Parse LoadBalancerService
from the controller proto and populate
EdgeNodeClusterConfig.LBInterfaces (K3S_BASE only; first
interface/CIDR entry applied).
- pkg/pillar/cmd/zedkube/clusterstatus.go: Relay LBInterfaces from
EdgeNodeClusterConfig into EdgeNodeClusterStatus on the bootstrap
node only; non-bootstrap nodes publish an empty list so they do not
trigger kube-vip setup.
- pkg/pillar/dpcmanager/dns.go: Filter kube-vip VIPs out of
DeviceNetworkStatus.AddrInfoList using the LBInterfaces CIDR range,
preventing VIPs from being used as source addresses for
controller-bound traffic.
- pkg/kube/cluster-init.sh: Add check_kubevip_lb loop that reads
EdgeNodeClusterStatus JSON each iteration and calls kubevip-apply.sh
or kubevip-delete.sh when the LB config changes. Persists
last-applied state to avoid redundant re-applies across restarts.
- pkg/kube/kubevip-apply.sh / kubevip-delete.sh: Scripts to
install/remove the kube-vip DaemonSet and kube-vip-cloud-provider
Deployment, configuring the IP pool via a kubevip ConfigMap.
- pkg/kube/kubevip-ds.yaml: kube-vip DaemonSet manifest (ARP mode,
control-plane nodes).
- pkg/kube/config.yaml: Disable k3s built-in ServiceLB (servicelb) and
Traefik for K3S_BASE — kube-vip replaces ServiceLB; users bring
their own ingress.
- pkg/pillar/docs/zedkube.md: Document the feature with an overview
diagram, data-flow, EVE-API proto, and DeviceNetworkStatus filtering
notes.
Signed-off-by: naiming-zededa <naiming@zededa.com>
… K3S_BASE - add the pillar vendor files updated eve-api Signed-off-by: naiming-zededa <naiming@zededa.com>
9c5b309 to
7952e18
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
PR dependencies
How to test and validate this PR
Create a native orchestration type cluster on EVE devices, and in the controller config the enabling of 'loadbalancing'
for the cluster. Specify the interface and IP prefix of the LB.
In the kubernetes side, user specify the helm/yaml definition of App and Service with type of 'loadbalancer'. Verify
the service has the IP address allocated on the interface for this service, and endpoint created for the App.
User the specify client to access that ip address and port. and verify when one device is down, the IP is reallocated
to another device of the cluster, and the app is still reachable.
there are many different ways to use the LB service, see examples in pkg/pillar/docs/zedkube.md for detail
Changelog notes
add controller-driven kube-vip load balancer for K3S_BASE clusters
PR Backports
Checklist
For backport PRs (remove it if it's not a backport):
And the last but not least:
check them.