Add UFACTORY 850 as one of the robots supported, introduce various HRI utilities for teleop and next episode hinting while reseting #493
Conversation
- Added extensions to manipulator to support 850 - Create Motors' UFactory abstraction - Proposed a new example demonstrating local setup Signed-off-by: Víctor Mayoral Vilches <[email protected]>
Signed-off-by: Víctor Mayoral Vilches <[email protected]>
Signed-off-by: vmayoral <[email protected]>
Signed-off-by: vmayoral <[email protected]>
Signed-off-by: vmayoral <[email protected]>
This is useful while recording many episodes, since you just loose track of them and the verbose outputs from the teleop don't allow for clarity on this regard Signed-off-by: vmayoral <[email protected]>
Signed-off-by: vmayoral <[email protected]>
Signed-off-by: vmayoral <[email protected]>
Signed-off-by: vmayoral <[email protected]>
Also, it's a derivative of the previous ones Signed-off-by: vmayoral <[email protected]>
|
Wow really cool! We are in the middle of a refactor. We will come back to this PR soon. In the meantime, I am curious if other people working on UFACTORY 850 are able to use this arm with LeRobot :) |
|
Let me know @Cadene if you'd like other flavours of xArm robots tested. I'm rather certain it will, as the UFACTORY API is rather well structured and known reproducible. I've got an |
|
@vmayoral Sounds good ;) Any chance you could run |
| @@ -497,15 +515,21 @@ def teleop_step( | |||
| # Cap goal position when too far away from present position. | |||
| # Slower fps expected due to reading from the follower. | |||
| if self.config.max_relative_target is not None: | |||
| present_pos = self.follower_arms[name].read("Present_Position") | |||
| if self.robot_type == "u850": | |||
There was a problem hiding this comment.
This approach of adding if/elif throughout the codebase doesn't seem scalable.
What do folks think about creating an abstract base class that all robots can implement their wrapper API's within, so we don't have to edit so much code in a brittle way to add new robot support?
I can take this on- I'm currently working on adding support for the elephant robotics MyArmM&C series.
The following abstract class would describe the current robot supported by manipulator.py, but personally I don't like the API and I think it's wayyy too specific to one robot:
from abc import ABC, abstractmethod
class BaseRobotWrapper(ABC):
@abstractmethod
def read(self, cmd: Literal["Present_Position]) -> npt.NDArray[np.float64]:
pass
@abstractmethod
def write(self, cmd: Literal["Operating_Mode", "Torque_Enable", "Goal_Position", "Lock", "Mode", "P_Coefficient", "I_Coefficient", "D_Coefficient", "Acceleration", "Maximum_Acceleration"], val: int, motors: str | list[str] | None=None) -> None:
pass
Something more generic could be made, leaving the API high level:
class BaseRobotWrapper(ABC):
@abstractmethod
def read_angles(self) -> npt.NDArray[np.float64]:
"""Read arbitrary number of joints to the robot. Mirrors write_angles"""
@abstractmethod
def write_angles(self, angles: npt.NDArray[np.float64]) -> None:
"""Write arbitrary number of joints to the robot. Mirrors read_angles"""
@abstractmethod
def set_follower_presets() -> none:
"""This would replace:
- set_koch_robot_preset
- set_aloha_robot_preset
- set_so100_robot_preset
- ...
"""
@abstractmethod
def set_leader_presets() -> None:
pass
@abstractmethod
def reset() -> None:
"""Used when 'reset_environment' is called. Optional to implement."""There was a problem hiding this comment.
@apockill, I've started reviewing my code following @Cadene's requests above, but I also like your suggestion and would be happy to work together on this. Frankly, an abstraction layer such as the one you propose would facilitate future ports, but I'm unsure if this is something desired at all, so I just took the most straightforward approach for my weekend of hacking.
Nevertheless, I think your suggestion is needed and a code rebase is probably the best way to do it. Happy to participate on that as well. For now, here, I'll focus on my initial intent and contribution.
There was a problem hiding this comment.
Okay, sounds good! Maybe let's keep things as is, once xarm / myarm support is merged in I can jump in and write wrappers for all the currently supported robots.
The only hard thing will just be to have everyone test their bots out for regressions.
This PR comes as a result of the Oct 2024 Hackathon wherein I participated remotely. Results were successful within the weekend of hacking, so I thought I'd give back and contribute. Results first:
lerobotteleop operational in UFACTORY 850topwristThe contributions of this PR are summarized below:
xArmWrapperof the UFACTORY Python SDK, usable with other arms from the same vendormanipulator.pywith minimal changes (attempted respecting the already existing structure) to support the new arm testedSome notes and considerations: