node-public-ipv6 not having desired effect in IPv6 only cluster

[dvgt]

Expected Behavior

Setting the flannel.alpha.coreos.com/node-public-ipv6 annotation on a node should result in that IP address to be used as backend for the VXLAN tunnel.

Current Behavior

The IPv6 address that is getting used, is the first IPv6 address which is found on the interface that has the node-public-ipv6 address, which might not be the expected IPv6 address. Pod traffic going to other nodes are being sent using the first IPv6 address instead of the configured address from the annotation.

I think there is a small bug in match.go:LookupExtIface, called from main.go:282 or main.go:284. This passes the publicIP(v6) as argument to the function. When retrieving the interface for the given publicIP, the variable ifaceAddr is used initially

flannel/pkg/ipmatch/match.go

Line 73 in da774f2

if ifaceAddr = net.ParseIP(ifname); ifaceAddr != nil {

but when checking if a fallback is needed later, the ifaceV6Addr variable is used

flannel/pkg/ipmatch/match.go

Line 243 in da774f2

} else if ipStack == ipv6Stack && ifaceV6Addr == nil {

which is not set in the case of an IPv6-only IP stack. Then the code falls back to using the first IPv6 address of the detected interface.

This does not happen in case of dual-stack because the ifaceV6Addr variable is set there

flannel/pkg/ipmatch/match.go

Line 94 in da774f2

if ifaceV6Addr = net.ParseIP(opts.PublicIPv6); ifaceV6Addr != nil {

Possible Solution

A possible solution could be to add ifaceV6Addr = ifaceAddr and further use ifaceV6Addr above line

flannel/pkg/ipmatch/match.go

Line 82 in da774f2

iface, err = ip.GetInterfaceByIP6(ifaceAddr)

and potentially set ifaceAddr = nil to avoid processing it as an IPv4 address.

Steps to Reproduce (for bugs)

1. Setup an IPv6-only cluster (similar to Inconsistent behavior between public-ipv6 annotations and public-ipv6 cli option #1813, but without the IPv4 addresses)
2. Add a second IPv6 address on the interface used for pod-to-pod communication on a certain node.
3. Set the flannel.alpha.coreos.com/node-public-ipv6 annotation to the original IPv6 address of the node.
4. Restart canal pods.

Context

Your Environment

• Flannel version: rancher/hardened-flannel:v0.25.4-build20240610
• Backend used: vxlan
• Etcd version: rancher/hardened-etcd:v3.5.13-k3s1-build20240531
• Kubernetes version (if used): v1.30.2+rke2r1
• Operating System and version: Oracle Linux Server 8.10

[rbrtbnfgl]

Did you disable IPv4? On the net-conf configuration you should specify EnableIPv6: true and EnableIPv4: false

[dvgt]

This is the config map that's in use:

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: rke2-canal
    meta.helm.sh/release-namespace: kube-system
  creationTimestamp: "2024-09-18T12:15:44Z"
  labels:
    app.kubernetes.io/managed-by: Helm
  name: rke2-canal-config
  namespace: kube-system
data:
  canal_iface: ""
  canal_iface_regex: ""
  cni_network_config: |-
    {
      "name": "k8s-pod-network",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "calico",
          "log_level": "info",
          "datastore_type": "kubernetes",
          "nodename": "__KUBERNETES_NODE_NAME__",
          "mtu": __CNI_MTU__,
          "ipam": {
              "type": "host-local",
              "ranges": [
                  [
                      {
                          "subnet": "usePodCidrIPv6"
                      }
                  ]
              ]
          },
          "policy": {
              "type": "k8s"
          },
          "kubernetes": {
              "kubeconfig": "__KUBECONFIG_FILEPATH__"
          }
        },
        {
          "type": "portmap",
          "snat": true,
          "capabilities": {"portMappings": true}
        },
        {
          "type": "bandwidth",
          "capabilities": {"bandwidth": true}
        }
      ]
    }
  masquerade: "true"
  net-conf.json: |
    {
      "EnableIPv4": false,
      "IPv6Network": "A:B:C:D::/108",
      "EnableIPv6": true,
      "Backend": {
        "Type": "vxlan"
      }
    }
  typha_service_name: none
  veth_mtu: "1450"

Note: The IPv6 address prefix was changed to A:B:C:D.

[rbrtbnfgl]

I checked from the code and probably there something that shouldn't be done when the IPv6 is configured. You are right, I'll do some tests to check if it's fixed.

[dvgt]

Thanks for looking into it.
From my point of view, something is rather missing when IPv6 is configured, i.e. the ifaceV6Addr is not set in case of ipv6Stack in the if-statement:

flannel/pkg/ipmatch/match.go

Line 73 in da774f2

if ifaceAddr = net.ParseIP(ifname); ifaceAddr != nil {