Resolving anyio Dependency Conflict in p2pclient During Migration

[seetadev]

🧩 Context

While working on the anyio migration for py-libp2p, I’ve encountered a persistent version conflict involving the p2pclient dependency.

The p2pclient library installs an internal version of anyio as soon as it’s committed or pushed.
This creates a direct conflict with the anyio version specified in our project’s pyproject.toml.

Despite multiple attempts to reconcile this mismatch —
✅ Bumping the anyio version,
✅ Overriding with dependency constraints, and
✅ Using a separate constraints.txt —
the conflict remains unresolved.

I wanted to open this discussion to document the issue, explore solutions, and gather insights from contributors.

---

⚙️ Problem Summary

• Library involved: p2pclient
• Issue: Conflicting internal anyio dependency
• Observed behavior: The internal anyio version from p2pclient overrides or conflicts with the project’s declared version.
• Impact: Prevents consistent dependency resolution during local builds and CI runs for the anyio migration branch.

---

Analysis

The likely cause is that p2pclient pins a specific anyio version (possibly anyio==2.x) that conflicts with the newer version used by the ongoing py-libp2p migration (anyio>=3.x).

---

Potential Solutions

Below are the possible paths we could take to resolve this dependency issue:

Option 1: Fork and Upgrade p2pclient

• Fork the p2pclient repo.

• Update its anyio version to match the project’s (e.g., anyio>=3,<5).

• Test compatibility locally.

• Use the forked repo in pyproject.toml (via Git URL):

  p2pclient = { git = "https://github.com/<your-handle>/p2pclient", branch = "anyio3-migration" }

• ✅ Pros: Full control, immediate unblock.

• ⚠️ Cons: Maintenance overhead; must track upstream updates.

---

Option 2: Dependency Override

• Use dependency resolution features to override p2pclient’s internal version (works if version constraints aren’t strict).

• Example (Poetry):

  [tool.poetry.dependencies]
  anyio = ">=3.7.1,<4.0"
  
  [tool.poetry.dev-dependencies]
  pip = ">=24.0"

• ✅ Pros: Minimal code changes.

• ⚠️ Cons: Can fail if p2pclient pins exact version numbers.

---

Option 3: Upstream PR / Issue

• Open an issue or PR on p2pclient requesting that its dependency on anyio be loosened (e.g., anyio>=3,<5).
• Include test results showing backward compatibility.
• ✅ Pros: Sustainable, community-friendly fix.
• ⚠️ Cons: Depends on upstream responsiveness.

---

🧠 Recommended Approach

We should first:

1. Confirm p2pclient’s latest pinned version of anyio from its setup.py / pyproject.toml.
2. Open a PR or issue upstream proposing a version bump, referencing this discussion.
3. As a fallback, prepare a temporary fork for internal development continuity.

---

🧰 Next Steps

1. Investigate which version p2pclient currently depends on.

2. Decide whether to:

   • Submit a PR upstream, or
   • Use a fork while the PR is under review.

3. Update documentation in py-libp2p noting this compatibility caveat.

4. Share reproducible steps for anyone facing similar dependency lock issues.

---

🗣️ Open Questions

• Has anyone previously run into dependency locking between p2pclient and anyio during testing?
• Would Sumanjeet, Luca or Emmanuel be open to a lightweight PR updating the dependency constraint in p2pclient?
• Are there known runtime incompatibilities when upgrading to anyio>=3 for p2pclient?

---

🔗 Related References

1. Current PR: Replace async service with aniyo service #973
2. Earlier attempt by Khwahish and Vraj: replace async_service with anyio_service #597
3. Collaboration at go-libp2p daemon bindings for python: Bump Anyio and ensure all test cases are passing seetadev/py-libp2p-daemon-bindings#3

---

Appreciation

Thank you to everyone contributing to the migration work!
Special thanks to Parth, Sumanjeet and Emmanuel working on the anyio transition — your effort strengthens the foundation for async I/O in py-libp2p.

Feedback and insights are welcome — let’s collaborate on the cleanest path forward.

[seetadev]

CCing @sumanjeet0012 , @parth-soni07 and @acul71.

[sumanjeet0012]

@seetadev Thank you for bringing this to our attention.

I've noted that the latest version of p2pclient was released on February 9, 2022. Given the lack of recent maintenance activity, I believe the most pragmatic approach would be to fork the repository and update the anyio dependency to a compatible version.

Our proposed plan of action is as follows:

1. Fork and Update: Create a fork of p2pclient with the updated anyio dependency
2. Upstream Contribution: Submit a Pull Request to the official p2pclient repository with our changes
3. Contingency Plan: If the PR is not merged in a reasonable timeframe, we can proceed with our maintained fork

To ensure reliability and mitigate the risk of a single point of failure (such as account suspension), we should maintain multiple clones of the fork across different github accounts.

[seetadev]

@seetadev Thank you for bringing this to our attention.

I've noted that the latest version of p2pclient was released on February 9, 2022. Given the lack of recent maintenance activity, I believe the most pragmatic approach would be to fork the repository and update the anyio dependency to a compatible version.

Our proposed plan of action is as follows:

1. Fork and Update: Create a fork of p2pclient with the updated anyio dependency
2. Upstream Contribution: Submit a Pull Request to the official p2pclient repository with our changes
3. Contingency Plan: If the PR is not merged in a reasonable timeframe, we can proceed with our maintained fork

To ensure reliability and mitigate the risk of a single point of failure (such as account suspension), we should maintain multiple clones of the fork across different github accounts.

@sumanjeet0012 :The new anyio3-migration branch is live and ready for review. The fork is based on the updated p2pclient implementation from py-libp2p-daemon-bindings which included prior improvements from Namay and Emmanuel.

Updates we need verify and implement if not fully completed:

• Upgraded anyio dependency to >=3,<5, ensuring full compatibility with the ongoing py-libp2p migration.
• Verified dependency resolution consistency across pyproject.toml and constraints.txt.
• Local and CI builds tested cleanly under the new async stack.

Next steps:

1. @parth-soni07 and you can open a PR upstream to the original p2pclient repository for visibility and potential merge.
2. Meanwhile, our forked version can serve as the active dependency reference for the py-libp2p migration branch.

This should unblock further async migration work and stabilize dependency management. Great teamwork and alignment from everyone involved — thank you for the collaboration, Sumanjeet! 🚀

We will get this repository available within py-libp2p and also get it integrated with py-libp2p's CI/CD system.

[parth-soni07]

Thank you @seetadev & @sumanjeet0012 , I will certainly start working on this asap and update you with my progress as I move.