[CI test] Integrate secp256k1lab as subtree

[theStack]

This PR moves the secp256k1lab (formerly called secp256k1proto) folder to an external package (living at https://github.com/theStack/secp256k1lab currently, created with the uv package manager), integrating it as git-subtree here [1]:

$ git subtree add --prefix=python/secp256k1lab --squash https://github.com/theStack/secp256k1lab secp256k1lab-draft04

Supersedes #81, using the suggested syspath extension approach [1] [2] [3] to avoid the ugly import hack that was used before.

[1] #81 (comment)
[2] #81 (comment)
[3] https://github.com/BlockstreamResearch/bip-frost-dkg/tree/syspath

[jonasnick]

This is looking good. Added some comments.

[theStack]

Rebased on master and addressed the review comments above (#83 (comment), #83 (comment), #83 (comment)). I btw thought it might be nice to squash the first three commits into one, but somehow always the rename of the vendored package (src/secp256k1lab => python/secp256k1lab/src/secp256k1lab, apparently happening in the merge commit) gets lost 🤔

[real-or-random]

I think it will be fair to list @sipa as an author:

Suggested change

	authors = [
	{ name = "Tim Ruffing", email = "[email protected]" },
	{ name = "Jonas Nick", email = "[email protected]" },
	{ name = "Sebastian Falbesoner", email = "[email protected]" }
	]
	authors = [
	{ name = "Pieter Wuille", email = "[email protected]" },
	{ name = "Tim Ruffing", email = "[email protected]" },
	{ name = "Jonas Nick", email = "[email protected]" },
	{ name = "Sebastian Falbesoner", email = "[email protected]" }
	]
	maintainers = [
	{ name = "Tim Ruffing", email = "[email protected]" },
	{ name = "Jonas Nick", email = "[email protected]" },
	{ name = "Sebastian Falbesoner", email = "[email protected]" }
	]

based on:
https://github.com/bitcoin/bitcoin/commits/e486597f9a57903600656fb5106858941885852f/test/functional/test_framework/crypto/secp256k1.py

[real-or-random]

You've added authors but not maintainers. Was this intentional? (I also don't like the duplication too much, but well, it's correct...)

[theStack]

Oh, that was unintentional, missed the maintainers part of your suggested diff. Done.

[real-or-random]

I btw thought it might be nice to squash the first three commits into one, but somehow always the rename of the vendored package (src/secp256k1lab => python/secp256k1lab/src/secp256k1lab, apparently happening in the merge commit) gets lost 🤔

I think I've seen this before. Just a shot in the dark but perhaps this helps: https://stackoverflow.com/questions/30136558/how-to-squash-commits-which-have-merge-commit-in-between

[theStack]

Added the secp256k1lab suggestions in branch https://github.com/theStack/secp256k1lab/tree/secp256k1lab-draft02 and updated the subtree integration in this PR accordingly. Still a bunch of TODOs (#83 (comment), plus determining a reasonable required Python version), I hope to tackle them this week. I guess with the ruff and mypy integration I can pretty much reuse the CI config here with minor adaptions, or is there anything special to be considered if it's a (uv) package?

[real-or-random]

I guess with the ruff and mypy integration I can pretty much reuse the CI config here with minor adaptions, or is there anything special to be considered if it's a (uv) package?

This is not strictly necessary, but since we use uv anyway: we won't need pip install foo, and we can just use uvx foo (see uv help tool).

plus determining a reasonable required Python version)

Something like uv run --python 3.9 should do it. But you'll need to execute the BIP code: uv run --python 3.9 python/tests.sh (or something like this, I haven't tested.) The thing is that we can't really run the library, and there are no tests, so the best test we have is running the BIP code...

[theStack]

I guess with the ruff and mypy integration I can pretty much reuse the CI config here with minor adaptions, or is there anything special to be considered if it's a (uv) package?

This is not strictly necessary, but since we use uv anyway: we won't need pip install foo, and we can just use uvx foo (see uv help tool).

Ah nice, that's very handy. Added two GHA jobs that directly run ruff/mypy using uvx (inspired by the warnet repo where they do something similar: https://github.com/bitcoin-dev-project/warnet/blob/296e0ba28b071365872afa28036af298d5c750a8/.github/workflows/test.yml#L15). I kept it very simple for now, but of course happy to pick up suggestions w.r.t. version pinning, caching etc.

plus determining a reasonable required Python version)

Something like uv run --python 3.9 should do it. But you'll need to execute the BIP code: uv run --python 3.9 python/tests.sh (or something like this, I haven't tested.) The thing is that we can't really run the library, and there are no tests, so the best test we have is running the BIP code...

That worked and the tests succeeded, so I changed to required minimum version to 3.9. ✔️ They also did with Python 3.8 by the way, but I guess there is not much point in requiring a minimum version that's not supported anymore (according to https://endoflife.date/python).

Added the secp256k1lab suggestions in branch https://github.com/theStack/secp256k1lab/tree/secp256k1lab-draft03 and updated the subtree integration in this PR accordingly. #77 is still unadressed, will look into that within the next days.

[theStack]

Another push, based on https://github.com/theStack/secp256k1lab/tree/secp256k1lab-draft04, where #77 has been adressed (see also discussion in #79). The last two commits here can be squashed if we want to avoid situations where intermediate commits don't pass the tests. I think we are quite close, once regarding code/CI there's no outstanding comments anymore, what's left to do in the secp256k1lab repo is

• add at least a rudimentary README.md (including that cute lab dog meme? :P or another logo, if someone has ideas)
• maybe go over commit history once more and refine if needed (squash, add co-authors for squashed-in fixes if we really care?)
• add at least a very basic test first (I used this one locally: https://github.com/theStack/secp256k1lab/blob/add_smoke_test/smoke_test.py); could still be added later though
• move over to (yet-to-be-created) repo in the secp256k1lab organization, once 1.0.0 is ready
• announce somewhere?

[real-or-random]

Great! I didn't have a proper look but two quick nits:

• The COPYING file should be in the repo root.
• Can you change my email address in the toml file to [email protected]? I recently switched to that one for Bitcoin development. (Sorry, had I missed myself this when I suggested the changes...)

[real-or-random]

• add at least a very basic test first (I used this one locally: theStack/secp256k1lab@add_smoke_test/smoke_test.py); could still be added later though

I think this is good. Another good candidate for a test are the BIP340 test vectors (see https://github.com/bitcoin/bips/blob/master/bip-0340/reference.py#L147) but let's add this later then.

[theStack]

Pushed again based on https://github.com/theStack/secp256k1lab/tree/secp256k1lab-draft05, with the COPYING file moved to the repo root and a rudimentary README.md (could use some vamping up for sure...).

Can you change my email address in the toml file to [email protected]? I recently switched to that one for Bitcoin development. (Sorry, had I missed myself this when I suggested the changes...)

Seems that this was done already in an earlier iteration (probably after your suggestion #83 (comment)). The "crypto@" address is still used in the author metadata of your commits though, happy to change those if you want to.

[jonasnick]

Changelog says 1.0 while pyproject.toml says 0.1. Otherwise, looks good to me.