Add CLN support #713

macgyver13 · 2025-04-23T14:41:31Z

Replaces #685 (Continues work started by Dan, but only focuses on CLN support at this time, he faced stability issues with eclair, will revisit that integration in the future)

This PR adds CLN pod support to warnet and SimLN plugin. Noteable changes:

commander/templates/rbac.yaml now requires exec rule
SimLN plugin copies certs/key from source cln pod to simln pod
ln_framework/ln.py::run_command has to determine if being executed inside or outside cluster to make connection to cln node correctly via kubernetes.stream
added sleep(0.25) before most thread.start() to greatly increase reliability of ln_init.py
deploy.py now searches node-defaults.yaml to determine if ln_init is required
lnd nodes need to be indexed in the same block before cln nodes, otherwise the expected channel id will not be correct

There is a new test ln_mixed_test.py that is partially implemented until the following decision is determined - how should LNNode access LND nodes from testing instance outside of cluster?:

port forward? - seems like a lot of overhead / clunky
use rpc instead of REST? - duplicates warnet/ln.py
use kubectl to issue curl commands? - messy / clunky
use simln plugin which will act inside cluster avoiding all the issues? - a drawback for simln is the activity is sendkey based instead of invoice / pay

issues: - restoring wallet produces same spend address on two nodes - lnd node connecting to cln produces infinite reconnect issue - system resource issue - two cln nodes can not find each other - gossip error

Tune sleep / pauses down Set simln container restartPolicy: Never

Remove all unnecessary thread sleeps

* main: allow scenarios to run locally without a cluster (for --help) control: add "please install helm" add metricsExport:true Update apidocs on replace master build with 28.1 add example to build image docs TypeError: build() missing 1 required positional argument: 'registry' handle missing key in base config lint: action all ruff fixes ci: bump actions, and ruff version. Use action remove unused docker_registry param build: update image build for cmake check for None, replace with [] Update install.md with a link to the scaling tips remove added print remove unused import reverse graph_test change did not improve stability make local_test_runner executable Resolve ruff formatting issues - Add local_test_runner script - add delay to graph_test expect - increases reliability # Conflicts: # resources/scenarios/commander.py # resources/scenarios/ln_init.py

migrate references from -ln to -lnd remove eclair references for now support ln: type: True in node-defaults.yaml update cln test data reorder channel index for stability LNNode shares commander logger

Update documentation to address CLN support Make ln_framework/ln.py work in commander or local device Add CLN support to warnet/ln.py Add short sleep between thread starts to improve stability

bdp-DrahtBot · 2025-04-23T14:41:39Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#686 (cli: add 'port_forward' subcommand for ln by f3r10)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

pinheadmz

Starting to review this, will add more comments and address your questions once ive got my head around it

test/data/LN_10.json

src/warnet/graph.py

reverse LN_10.json sequence change

…od metadata

pinheadmz

really really REALLY great work, thank you. I reviewed all the changes and left comments throughout.

Next I'm going to try answer your open questions in the PR description, and then try to figure out whats up with the test failures.

resources/charts/bitcoincore/charts/cln/values.yaml

resources/charts/bitcoincore/charts/lnd/templates/_helpers.tpl

resources/charts/commander/templates/rbac.yaml

resources/scenarios/ln_framework/ln.py

resources/plugins/simln/charts/simln/values.yaml

resources/scenarios/ln_init.py

src/warnet/deploy.py

pinheadmz · 2025-04-24T18:38:20Z

src/warnet/ln.py

+    if "cln" in pod.metadata.labels["app.kubernetes.io/name"]:
+        pubkey_key = "id"


this kind of thing is ok for now and probably will be ok for eclair as well -- but its a tad codesmelly. feels like implementation-specific methods should stay in their classes, even as jsut static functions

pinheadmz · 2025-04-24T18:39:01Z

test/data/ln_mixed/network.yaml

+    ln:
+      cln: true


just wondered what would happen in the edge case

ln: cln: true lnd: true

I bet helm would add both charts 😬

tested this, lnd started when both were defined

test/ln_mixed_test.py

pinheadmz · 2025-04-24T18:54:45Z

Now that i've read through the entire patch i think i might have opinions about these questions, lets see...

commander/templates/rbac.yaml now requires exec rule

I think you only needed this to use lightning-cli inside cln containers, and im hoping we dont need to do that, but this isn't a bad change necessarily

SimLN plugin copies certs/key from source cln pod to simln pod

yes I think this is the only way to do it for now

ln_framework/ln.py::run_command has to determine if being executed inside or outside cluster to make connection to cln node correctly via kubernetes.stream

everything inside scenarios/ should only ever be executed inside the cluster. The only exception is if a user needs help: warnet run ... -- --help which requires the scenario file to be run locally just to gather the argument parser comments.

added sleep(0.25) before most thread.start() to greatly increase reliability of ln_init.py

Not sure why you needed this? What happens otherwise?

deploy.py now searches node-defaults.yaml to determine if ln_init is required

yes thank you yes!

lnd nodes need to be indexed in the same block before cln nodes, otherwise the expected channel id will not be correct

Hm I didn't notice this. Did you see how we use decreasing transaction fees in the channel opens to ensure they are in the expected order in a block? Hopefully when you open a channel from cln you can also set the fee rate, and the process will continue to be implementation-agnostic

There is a new test ln_mixed_test.py that is partially implemented until the following decision is determined - how should LNNode access LND nodes from testing instance outside of cluster?:

port forward? - seems like a lot of overhead / clunky
use rpc instead of REST? - duplicates warnet/ln.py
use kubectl to issue curl commands? - messy / clunky
use simln plugin which will act inside cluster avoiding all the issues? - a drawback for simln is the activity is sendkey based instead of invoice / pay

I don't think this should ever happen - was it happening before? Scenarios are executed entirely inside the cluster.

reduce ln_init changes start migration of CLN to REST

other clean up for comments remove rbac exec change

danvergara and others added 16 commits March 23, 2025 21:07

add eclair chart

e0def8f

add eclair as a dependency

1f44794

add cln chart

6c54c7b

add improvements to the cln node

80c3211

fixes for eclair

02609bc

checkpoint cln integrated into ln_framework

814793c

issues: - restoring wallet produces same spend address on two nodes - lnd node connecting to cln produces infinite reconnect issue - system resource issue - two cln nodes can not find each other - gossip error

move certs from cln nodes to simln

4df0c33

adjust cln to start grpc

4c409fd

enable grpc plugin to respond to remote nodes

7831413

Add styling to standard out for improved readability

9fdd5f8

Tune sleep / pauses down Set simln container restartPolicy: Never

Fix wait for gossip loop

a6fc662

Remove all unnecessary thread sleeps

fix linter findings

5767109

don't setup a ln connection to itself

ff9d32e

migrate references from -ln to -lnd remove eclair references for now support ln: type: True in node-defaults.yaml update cln test data reorder channel index for stability LNNode shares commander logger

Add ln_mixed node test

cda355b

Update documentation to address CLN support Make ln_framework/ln.py work in commander or local device Add CLN support to warnet/ln.py Add short sleep between thread starts to improve stability

switch from print to logger

f1a8b37

bdp-DrahtBot mentioned this pull request Apr 23, 2025

cli: add 'port_forward' subcommand for ln #686

Open

correct formatting ruff finding

6355e85

pinheadmz reviewed Apr 23, 2025

View reviewed changes

test/data/LN_10.json Outdated Show resolved Hide resolved

src/warnet/graph.py Outdated Show resolved Hide resolved

macgyver13 added 2 commits April 23, 2025 16:19

revert -lnd changes based on comments

f1d7b39

reverse LN_10.json sequence change

Remove all dependences on -cln or -lnd name to determine type - use p…

10531d5

…od metadata

pinheadmz reviewed Apr 24, 2025

View reviewed changes

macgyver13 added 4 commits April 24, 2025 20:34

address comments

3d846d3

reduce ln_init changes start migration of CLN to REST

remove RPC requirements for CLN - fetch rune via http transfer from pod

f5aba7c

other clean up for comments remove rbac exec change

Add debug logs to deploy step

4fb0225

try wait_for_pod_ready

7890267

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CLN support #713

Add CLN support #713

macgyver13 commented Apr 23, 2025 •

edited

Loading

bdp-DrahtBot commented Apr 23, 2025

pinheadmz left a comment

pinheadmz left a comment

pinheadmz Apr 24, 2025

pinheadmz Apr 24, 2025

macgyver13 Apr 24, 2025

macgyver13 Apr 25, 2025

pinheadmz commented Apr 24, 2025

		if "cln" in pod.metadata.labels["app.kubernetes.io/name"]:
		pubkey_key = "id"

Add CLN support #713

Are you sure you want to change the base?

Add CLN support #713

Conversation

macgyver13 commented Apr 23, 2025 • edited Loading

bdp-DrahtBot commented Apr 23, 2025

Conflicts

pinheadmz left a comment

Choose a reason for hiding this comment

pinheadmz left a comment

Choose a reason for hiding this comment

pinheadmz Apr 24, 2025

Choose a reason for hiding this comment

pinheadmz Apr 24, 2025

Choose a reason for hiding this comment

macgyver13 Apr 24, 2025

Choose a reason for hiding this comment

macgyver13 Apr 25, 2025

Choose a reason for hiding this comment

pinheadmz commented Apr 24, 2025

macgyver13 commented Apr 23, 2025 •

edited

Loading