[on hold] RFC3: simultaneous tunnels #540

bgm-malbeclabs · 2025-06-09T22:17:03Z

Summary of Changes

ben-malbeclabs · 2025-06-10T22:44:38Z

rfcs/rfc3-simultaneous-tunnels.md

+      AccountType account_type
+      Pubkey owner
+      Pubkey device_pk
+      string name


Something worth considering would be to have in the Device class the device_type or an equivalent field that allows us validate that the interface name is sensible.

Sorry, the device_type is there as a switch, but something more like device_model could capture this.

Is there a list of approved device_models that we support that we could validate against? or is this more for something that we could check in the future

We have two models: 7130LBR and 7280CR3. Both have very different conventions around interface naming e.g. swA/B/C on the former and EthA/B on the latter.

Can be future, but we need to figure out how we want to fail if these sorts of fields prove to have garbage data.

ben-malbeclabs · 2025-06-10T22:55:01Z

rfcs/rfc3-simultaneous-tunnels.md

+    * can a user set their own termination "points" etc
+
+#### Controller
+* *optional*: configuration for tunnel termination loopbacks generated in device template


My preference would be to do this in the controller templating.

Agreed; can we launch multicast without the controller templating? (my thought is yes but could be convinced otherwise of course). Feels like the templating could be a separate RFC to sunset ansible so that we don't block multicast too long but again open to suggestions otherwise.

I think we can take parts of Ansible config generation process and move it to the controller over time. I think here we just need to add loopback interfaces, and I would hope it isn't too much work. We need a way to allocate these from the dz_prefix pool and pass them from the activator to the controller I guess.

Feel free to correct me if it is more work than I appreciate, we can continue to push it manually for now.

bgm-malbeclabs · 2025-06-10T23:11:44Z

rfcs/rfc3-simultaneous-tunnels.md

+  3. Initiate latency probes per tunnel termination
+  4. Store results as <Device: Interface: LatencyResult> and serve via /latency endpoint for CLI
+
+#### Activator


@elitegreg are there any things that are missing from the activator / smart contract side?

Not that I can think of, but I'm not sure I'm grasping the Activator change listed.

elitegreg · 2025-06-12T17:01:54Z

rfcs/rfc3-simultaneous-tunnels.md

+
+## Motivation
+
+Multiple tunnel support is required now that DoubleZero supports IBRL and multicast. In fact, multicast can not be publicly released wthout multiple tunnel support due to restrictions in the Linux kernel.


Not a huge deal, but the limitations aren't from Linux but rather GRE.

elitegreg · 2025-06-12T17:06:19Z

rfcs/rfc3-simultaneous-tunnels.md

+    }
+
+    Interface --> Device : device_pk
+````


@juan-malbeclabs you probably understand this better than me. Wouldn't this design require loading all interfaces in order to reduce to the set for device_pk? Would it be better to have a list of interface pubkeys in the device? Then they can be looked up by the account pubkey?

Although I conceptually agree with the diagram, I would store all interfaces in a second account linked to the Device. We would create a DeviceInterfaceList account. This way, we would need to read both the list of Devices and the list of Interfaces (one account per Device with multiple Interfaces).

The goal is to be able to modify this account whenever an interface is created or deleted, without having to load the Device information.

elitegreg · 2025-06-12T17:08:25Z

rfcs/rfc3-simultaneous-tunnels.md

+### Service Changes
+
+#### CLI
+1. The CLI currently selects the tunnel termination endpoint for a user connection based on min(latency) across all DZDs. In the event there is an existing tunnel terminated on the DZD, we need to select the next best endpoint on the same DZD.


Is it not fair to just assume that the latency of one interface is as good as another on the same DZD and just iterate the interfaces as more tunnels are added?

elitegreg · 2025-06-12T17:12:36Z

rfcs/rfc3-simultaneous-tunnels.md

+  3. Initiate latency probes per tunnel termination
+  4. Store results as <Device: Interface: LatencyResult> and serve via /latency endpoint for CLI
+
+#### Activator


Not that I can think of, but I'm not sure I'm grasping the Activator change listed.

Copilot

Pull Request Overview

This PR introduces RFC3, a design proposal for enabling simultaneous tunnel support in DoubleZero by defining a new Interface data structure, updating service components, and outlining related on-chain and network changes.

Add initial RFC document outlining motivation, design, and components affected.
Define new Interface structure alongside Device, and detail CLI, daemon, activator, and controller updates.
Document security considerations, backwards compatibility impact, and open questions.

Comments suppressed due to low confidence (1)

rfcs/rfc3-simultaneous-tunnels.md:54

The Mermaid code block is opened with three backticks but closed with four; use three backticks consistently.

</details>

rfcs/rfc3-simultaneous-tunnels.md

juan-malbeclabs · 2025-06-18T18:18:19Z

rfcs/rfc3-simultaneous-tunnels.md

+
+#### CLI
+1. The CLI currently selects the tunnel termination endpoint for a user connection based on min(latency) across all DZDs. In the event there is an existing tunnel terminated on the DZD, we need to select the next best endpoint on the same DZD.
+2. Users need to be able perform CRUD operations on the on-chain interfaces i.e. `doublezero interface create`.


Are the interfaces associated with the devices created by the contributor? In this context, who is the User?
Who has the authority to execute the doublezero interface create command?

nikw9944 · 2025-06-18T18:01:57Z

rfcs/rfc3-simultaneous-tunnels.md

+
+#### CLI
+1. The CLI currently selects the tunnel termination endpoint for a user connection based on min(latency) across all DZDs. In the event there is an existing tunnel terminated on the DZD, we need to select the next best endpoint on the same DZD.
+2. Users need to be able perform CRUD operations on the on-chain interfaces i.e. `doublezero interface create`.


I would think contributors need to do this, not users.

nikw9944 · 2025-06-18T18:04:00Z

rfcs/rfc3-simultaneous-tunnels.md

+3. Users need to be able to display interfaces listed on-chain via `doublezero interfaces list` or some derivative command.
+
+#### Daemon
+Latency probing changes are needed for this as the current implementation looks at the public_ip field of device record to probe each DZD:


I'm not understanding the benefit (or necessity) of testing latency to every tunnel termination interface on every DZD. Why would we expect different IPs on the same router to have different latency characteristics?

nikw9944 · 2025-06-18T18:06:53Z

rfcs/rfc3-simultaneous-tunnels.md

+
+2. **Require users to obtain a second public address.** While this would satisfy the requirement of a unique (source, destination) tunnel IP endpoint per tunnel, it pushes this issue back on the users of DoubleZero and possibly prevents user uptake at the expense of more engineering work.
+
+3. **Adapt the devices table in the current smart contract to fit a second tunnel (i.e. multicast) endpoint.** While this seems like significantly less work on its face, we end up needing to touch the same portions of the stack as a more ideal solution as they all need to be taught to understand this field.


This also has the downside of limiting us to 2 tunnels, so it's not a general solution

nikw9944 · 2025-06-18T18:54:03Z

rfcs/rfc3-simultaneous-tunnels.md

+
+```mermaid
+classDiagram
+    class Interface {


Today we can infer the interface that a Link is associated with by looking at the link's tunnel_net. But now that we have Interface, should we make this explicit by adding an optional relationship between Interface and Link?

!#475 - simultaneous tunnels RFC

4271a11

bgm-malbeclabs marked this pull request as draft June 9, 2025 22:17

bgm-malbeclabs changed the title ~~!#475 - simultaneous tunnels RFC~~ RFC3: simultaneous tunnels Jun 9, 2025

bgm-malbeclabs and others added 4 commits June 9, 2025 19:38

!#475 - simultaneous tunnels RFC

410a31b

updated

a8740e2

updates

56f3e86

add color

87757e0

ben-malbeclabs reviewed Jun 10, 2025

View reviewed changes

!#475 - simultaneous tunnels RFC

768b224

bgm-malbeclabs commented Jun 10, 2025

View reviewed changes

!#475 - simultaneous tunnels RFC

b714e73

elitegreg reviewed Jun 12, 2025

View reviewed changes

bgm-malbeclabs requested a review from Copilot June 12, 2025 18:29

Copilot AI reviewed Jun 12, 2025

View reviewed changes

rfcs/rfc3-simultaneous-tunnels.md Outdated Show resolved Hide resolved

rfcs/rfc3-simultaneous-tunnels.md Outdated Show resolved Hide resolved

rfcs/rfc3-simultaneous-tunnels.md Outdated Show resolved Hide resolved

bgm-malbeclabs added 2 commits June 12, 2025 11:54

!#475 - simultaneous tunnels RFC

0ad808f

!#475 - simultaneous tunnels RFC

01aa22f

juan-malbeclabs reviewed Jun 18, 2025

View reviewed changes

nikw9944 requested changes Jun 18, 2025

View reviewed changes

bgm-malbeclabs changed the title ~~RFC3: simultaneous tunnels~~ [on hold] RFC3: simultaneous tunnels Jul 9, 2025


		## Motivation

		Multiple tunnel support is required now that DoubleZero supports IBRL and multicast. In fact, multicast can not be publicly released wthout multiple tunnel support due to restrictions in the Linux kernel.


		2. Require users to obtain a second public address. While this would satisfy the requirement of a unique (source, destination) tunnel IP endpoint per tunnel, it pushes this issue back on the users of DoubleZero and possibly prevents user uptake at the expense of more engineering work.

		3. Adapt the devices table in the current smart contract to fit a second tunnel (i.e. multicast) endpoint. While this seems like significantly less work on its face, we end up needing to touch the same portions of the stack as a more ideal solution as they all need to be taught to understand this field.

[on hold] RFC3: simultaneous tunnels #540

Are you sure you want to change the base?

[on hold] RFC3: simultaneous tunnels #540

Uh oh!

Conversation

bgm-malbeclabs commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bgm-malbeclabs Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ben-malbeclabs Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

bgm-malbeclabs commented Jun 9, 2025 •

edited

Loading

bgm-malbeclabs Jun 10, 2025 •

edited

Loading

ben-malbeclabs Jun 12, 2025 •

edited

Loading