Skip to content

Commit 474223b

Browse files
committed
Add O6 somehand hand-pose support
1 parent a7d51f5 commit 474223b

15 files changed

Lines changed: 315 additions & 69 deletions

File tree

AGENTS.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -155,8 +155,8 @@ target_dof_pos = clip(action, -10, 10) × action_scale + default_dof_pos
155155
- `gripper` mode reuses `Pico4InputProvider.get_controller_snapshot()` for Pico grip/trigger open-close control and supports LinkerHand L6 and O6
156156
- `vr_hand_pose` mode reuses `Pico4InputProvider.get_hand_snapshot()` and somehand 0.2.0 public `somehand.api` for continuous Pico hand-pose retargeting; do not start a second `PicoBridge` for hand control
157157
- Teleopit owns Pico 26-joint hand-state to 21-landmark conversion; do not import `somehand.pico_input`
158-
- LinkerHand O6 supports only `hands.mode=gripper`; its default `close_pose` is `[86, 73, 118, 111, 110, 111]`
159-
- L6 `gripper` mode uses the configured `hands.linkerhand_l6.speed` (default `[50]*6`); O6 `gripper` mode uses `hands.linkerhand_o6.speed` (default `[255]*6`); `vr_hand_pose` always sets LinkerHand L6 speed to `[255]*6`
158+
- LinkerHand O6 supports `hands.mode=gripper|vr_hand_pose`; its default `close_pose` is `[86, 73, 118, 111, 110, 111]`
159+
- L6 `gripper` mode uses the configured `hands.linkerhand_l6.speed` (default `[50]*6`); O6 `gripper` mode uses `hands.linkerhand_o6.speed` (default `[255]*6`); `vr_hand_pose` always sets LinkerHand L6/O6 speed to `[255]*6`
160160
- `vr_hand_pose` defaults to a low-latency somehand path: `hands.somehand.rate_hz=60`, `max_iterations=12`, `temporal_filter_alpha=1.0`, and `output_alpha=1.0`; this prioritizes response speed over smoothing
161161
- LinkerHand control is active in all sim2real modes when `hands.enabled=true`; shutdown and hand-runtime failure must send the configured open pose
162162
- In `vr_hand_pose` mode, missing/inactive hand pose holds the last commanded pose for that side instead of opening the hand

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ Full docs at **[BotRunner64.github.io/Teleopit](https://BotRunner64.github.io/Te
134134
### v0.4.0 (2026-06-25)
135135

136136
- Improved Pico realtime control with pico-bridge 0.2.1, `ARMS` mode, armed sim2real mocap entry, and retargeter-preserving pause/arms resets.
137-
- Added optional LinkerHand L6/O6 sim2real control, including Pico gripper input and low-latency L6 `vr_hand_pose`.
137+
- Added optional LinkerHand L6/O6 sim2real control, including Pico gripper input and low-latency L6/O6 `vr_hand_pose`.
138138
- Added manual Pico sim2real HDF5 recording and an interactive Pico motion recorder for training NPZ clips.
139139
- Refined the training data path with minimal HDF5 shards, explicit precompute, rewind sampling, and updated tracking rewards.
140140

docs/docs/configuration/config-reference.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -128,9 +128,10 @@ installs of `third_party/linkerhand-python-sdk` and `third_party/somehand`.
128128
When enabled, hand control remains active in all sim2real modes.
129129
`gripper` supports `linkerhand_l6` and `linkerhand_o6` by interpolating Pico
130130
trigger input between the configured open and close poses. `vr_hand_pose` is
131-
L6-only: missing hand pose holds the last command for that side, L6 speed is
132-
set to the maximum, and Teleopit converts Pico hand state to 21 landmarks before
133-
calling somehand 0.2.0 through `somehand.api` only.
131+
supported by `linkerhand_l6` and `linkerhand_o6`: missing hand pose holds the
132+
last command for that side, the selected hand speed is set to the maximum, and
133+
Teleopit converts Pico hand state to 21 landmarks before calling somehand 0.2.0
134+
through `somehand.api` only.
134135

135136
| Field | Description | Default |
136137
|-------|-------------|---------|
@@ -144,9 +145,10 @@ calling somehand 0.2.0 through `somehand.api` only.
144145
| `hands.linkerhand_l6.speed` | L6 speed used by `gripper`; `vr_hand_pose` overrides this to maximum speed | see config |
145146
| `hands.linkerhand_l6.open_pose` / `close_pose` | Six-value L6 open/closed poses | see config |
146147
| `hands.linkerhand_o6.left_can` / `right_can` | CAN channels for each O6 hand | `can0` / `can1` |
147-
| `hands.linkerhand_o6.speed` | O6 speed used by `gripper` | see config |
148+
| `hands.linkerhand_o6.speed` | O6 speed used by `gripper`; `vr_hand_pose` overrides this to maximum speed | see config |
148149
| `hands.linkerhand_o6.open_pose` / `close_pose` | Six-value O6 open/closed poses | see config |
149-
| `hands.somehand.config_path` | Official somehand 0.2.0 bi-hand L6 config used by `vr_hand_pose` | see config |
150+
| `hands.somehand.l6_config_path` | Official somehand 0.2.0 bi-hand L6 config used by L6 `vr_hand_pose` | see config |
151+
| `hands.somehand.o6_config_path` | Official somehand 0.2.0 bi-hand O6 config used by O6 `vr_hand_pose` | see config |
150152
| `hands.somehand.rate_hz` | Low-latency `vr_hand_pose` command rate in Hz | `60.0` |
151153
| `hands.somehand.max_iterations` | somehand solver iteration cap for `vr_hand_pose` | `12` |
152154
| `hands.somehand.temporal_filter_alpha` | somehand input landmark smoothing alpha; `1.0` disables smoothing delay | `1.0` |

docs/docs/getting-started/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ Install those packages directly after initializing the submodules:
6868
git submodule update --init --recursive
6969
pip install -e third_party/linkerhand-python-sdk
7070
pip install -e third_party/somehand
71-
scripts/setup/download_somehand_l6_assets.sh
71+
scripts/setup/download_somehand_assets.sh
7272
```
7373

7474
These packages are only required when `hands.enabled=true`.

docs/docs/tutorials/pico-sim2real.md

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -183,11 +183,11 @@ Pico sim2real can drive LinkerHand hands from Pico input:
183183
trigger closes that hand. This mode supports `hands.driver=linkerhand_l6` and
184184
`hands.driver=linkerhand_o6`; speed and open/close poses come from the matching
185185
driver config.
186-
- `vr_hand_pose`: L6-only mode that retargets Pico hand pose through somehand and
187-
commands the continuous L6 hand target. If a hand pose disappears, that side
188-
keeps its last commanded pose. This mode uses Teleopit's Pico landmark adapter
189-
and the public `somehand.api` from somehand 0.2.0. It always sets L6 speed to
190-
the maximum.
186+
- `vr_hand_pose`: retargets Pico hand pose through somehand and commands the
187+
continuous L6 or O6 hand target. If a hand pose disappears, that side keeps its
188+
last commanded pose. This mode uses Teleopit's Pico landmark adapter and the
189+
public `somehand.api` from somehand 0.2.0. It always sets the selected hand
190+
speed to the maximum.
191191

192192
When `hands.enabled=true`, hand control remains active in all sim2real modes.
193193
Shutdown and hand-runtime failure send the configured open pose.
@@ -199,7 +199,7 @@ the main Pico profile:
199199
git submodule update --init --recursive
200200
pip install -e third_party/linkerhand-python-sdk
201201
pip install -e third_party/somehand
202-
scripts/setup/download_somehand_l6_assets.sh
202+
scripts/setup/download_somehand_assets.sh
203203
```
204204

205205
Bring up the CAN interfaces before testing or running hand control:
@@ -229,7 +229,8 @@ python scripts/dev/test_linkerhand_l6.py \
229229
--right-can can1
230230
```
231231

232-
To test O6 with live Pico gripper input, add `--mode gripper`.
232+
To test O6 with live Pico gripper input, add `--mode gripper`. To test O6 with
233+
live Pico hand-pose retargeting, add `--mode vr_hand_pose`.
233234

234235
Then enable L6 gripper control in Pico sim2real:
235236

@@ -251,7 +252,7 @@ hands.linkerhand_o6.left_can=can0
251252
hands.linkerhand_o6.right_can=can1
252253
```
253254

254-
For continuous VR hand-pose control, use:
255+
For continuous L6 VR hand-pose control, use:
255256

256257
```bash
257258
hands.enabled=true
@@ -261,6 +262,16 @@ hands.linkerhand_l6.left_can=can0
261262
hands.linkerhand_l6.right_can=can1
262263
```
263264

265+
For continuous O6 VR hand-pose control, switch the driver and CAN keys:
266+
267+
```bash
268+
hands.enabled=true
269+
hands.driver=linkerhand_o6
270+
hands.mode=vr_hand_pose
271+
hands.linkerhand_o6.left_can=can0
272+
hands.linkerhand_o6.right_can=can1
273+
```
274+
264275
## Optional RealSense Preview
265276

266277
Stream the G1 RealSense color camera back to the Pico headset:

docs/i18n/zh-Hans/docusaurus-plugin-content-docs/current/configuration/config-reference.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,9 @@ MuJoCo 窗口显示重定向参考;`sim2sim`、`mocap`、`camera` 和 `all`
145145
`hands.enabled=true` 要求 `input.provider=pico4`,并以本地 editable 方式安装
146146
`third_party/linkerhand-python-sdk``third_party/somehand`。启用后,手控会在所有 sim2real 模式中保持生效。
147147
`gripper` 支持 `linkerhand_l6``linkerhand_o6`,会用 Pico trigger 在配置的张开和闭合姿态之间插值。
148-
`vr_hand_pose` 只支持 L6:手部 pose 消失时,对应侧会保持上一条命令;L6 速度会设为最大值;
149-
Teleopit 会先将 Pico 手部状态转成 21 个 landmarks,再只通过 somehand 0.2.0 公开的 `somehand.api` 调用。
148+
`vr_hand_pose` 支持 `linkerhand_l6``linkerhand_o6`:手部 pose 消失时,对应侧会保持上一条命令;
149+
所选手的速度会设为最大值;Teleopit 会先将 Pico 手部状态转成 21 个 landmarks,
150+
再只通过 somehand 0.2.0 公开的 `somehand.api` 调用。
150151

151152
| 字段 | 说明 | 默认值 |
152153
|---|---|---|
@@ -162,9 +163,10 @@ Teleopit 会先将 Pico 手部状态转成 21 个 landmarks,再只通过 someh
162163
| `hands.linkerhand_l6.trigger_deadzone` | trigger 两端死区 | `0.05` |
163164
| `hands.linkerhand_l6.open_pose` / `close_pose` | L6 的 6 维张开/闭合姿态 | 见配置 |
164165
| `hands.linkerhand_o6.left_can` / `right_can` | 左右 O6 手 CAN 通道 | `can0` / `can1` |
165-
| `hands.linkerhand_o6.speed` | `gripper` 使用的 O6 速度 | 见配置 |
166+
| `hands.linkerhand_o6.speed` | `gripper` 使用的 O6 速度`vr_hand_pose` 会覆盖为最大速度 | 见配置 |
166167
| `hands.linkerhand_o6.open_pose` / `close_pose` | O6 的 6 维张开/闭合姿态 | 见配置 |
167-
| `hands.somehand.config_path` | `vr_hand_pose` 使用的 somehand 双手 L6 配置 | 见配置 |
168+
| `hands.somehand.l6_config_path` | L6 `vr_hand_pose` 使用的 somehand 双手 L6 配置 | 见配置 |
169+
| `hands.somehand.o6_config_path` | O6 `vr_hand_pose` 使用的 somehand 双手 O6 配置 | 见配置 |
168170
| `hands.somehand.rate_hz` | 低延时 `vr_hand_pose` 命令频率(Hz) | `60.0` |
169171
| `hands.somehand.max_iterations` | `vr_hand_pose` 的 somehand solver 迭代上限 | `12` |
170172
| `hands.somehand.temporal_filter_alpha` | somehand 输入 landmarks 平滑 alpha;`1.0` 表示关闭平滑延时 | `1.0` |

docs/i18n/zh-Hans/docusaurus-plugin-content-docs/current/getting-started/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ submodule 后,直接安装这些包:
6868
git submodule update --init --recursive
6969
pip install -e third_party/linkerhand-python-sdk
7070
pip install -e third_party/somehand
71-
scripts/setup/download_somehand_l6_assets.sh
71+
scripts/setup/download_somehand_assets.sh
7272
```
7373

7474
只有在 `hands.enabled=true` 时才需要安装这些包。

docs/i18n/zh-Hans/docusaurus-plugin-content-docs/current/tutorials/pico-sim2real.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -172,9 +172,9 @@ Pico sim2real 可以用 Pico 输入控制 LinkerHand:
172172
- `gripper`:按住同侧 grip 作为 deadman,同侧 trigger 控制对应手闭合。
173173
该模式支持 `hands.driver=linkerhand_l6``hands.driver=linkerhand_o6`
174174
速度和张开/闭合姿态来自对应 driver 配置。
175-
- `vr_hand_pose`只支持 L6,通过 somehand 重定向 Pico 手部 pose,并下发连续 L6 手部目标。
175+
- `vr_hand_pose`:通过 somehand 重定向 Pico 手部 pose,并下发连续 L6 或 O6 手部目标。
176176
如果某侧手部 pose 消失,该侧会保持上一条手势命令。这个模式使用 Teleopit 的
177-
Pico landmark 适配器和 somehand 0.2.0 公开的 `somehand.api`并始终将 L6
177+
Pico landmark 适配器和 somehand 0.2.0 公开的 `somehand.api`并始终将所选手的
178178
速度设为最大值。默认配置使用 60 Hz 的低延时 somehand 路径并减少平滑,所以响应会更快,
179179
但可能比标准 somehand 设置更抖。
180180

@@ -186,7 +186,7 @@ Pico sim2real 可以用 Pico 输入控制 LinkerHand:
186186
git submodule update --init --recursive
187187
pip install -e third_party/linkerhand-python-sdk
188188
pip install -e third_party/somehand
189-
scripts/setup/download_somehand_l6_assets.sh
189+
scripts/setup/download_somehand_assets.sh
190190
```
191191

192192
测试或运行手控前,先开启 CAN 接口:
@@ -215,7 +215,8 @@ python scripts/dev/test_linkerhand_l6.py \
215215
--right-can can1
216216
```
217217

218-
如果要用实时 Pico gripper 输入测试 O6,再加 `--mode gripper`
218+
如果要用实时 Pico gripper 输入测试 O6,再加 `--mode gripper`。如果要用实时 Pico
219+
手部 pose 重定向测试 O6,再加 `--mode vr_hand_pose`
219220

220221
然后在 Pico sim2real 中启用 L6 gripper 控制:
221222

@@ -237,7 +238,7 @@ hands.linkerhand_o6.left_can=can0
237238
hands.linkerhand_o6.right_can=can1
238239
```
239240

240-
连续 VR 手部 pose 控制使用:
241+
连续 L6 VR 手部 pose 控制使用:
241242

242243
```bash
243244
hands.enabled=true
@@ -247,6 +248,16 @@ hands.linkerhand_l6.left_can=can0
247248
hands.linkerhand_l6.right_can=can1
248249
```
249250

251+
连续 O6 VR 手部 pose 控制切换 driver 和 CAN 配置键:
252+
253+
```bash
254+
hands.enabled=true
255+
hands.driver=linkerhand_o6
256+
hands.mode=vr_hand_pose
257+
hands.linkerhand_o6.left_can=can0
258+
hands.linkerhand_o6.right_can=can1
259+
```
260+
250261
## 可选 RealSense 预览
251262

252263
将 G1 RealSense 彩色相机推送回 Pico 头显:

scripts/dev/test_linkerhand_l6.py

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,8 @@
3434
O6_OPEN_POSE = [250, 250, 250, 250, 250, 250]
3535
O6_CLOSE_POSE = [86, 73, 118, 111, 110, 111]
3636
O6_DEFAULT_SPEED = [255, 255, 255, 255, 255, 255]
37-
DEFAULT_SOMEHAND_CONFIG_PATH = "third_party/somehand/configs/retargeting/bihand/linkerhand_l6_bihand.yaml"
37+
DEFAULT_L6_SOMEHAND_CONFIG_PATH = "third_party/somehand/configs/retargeting/bihand/linkerhand_l6_bihand.yaml"
38+
DEFAULT_O6_SOMEHAND_CONFIG_PATH = "third_party/somehand/configs/retargeting/bihand/linkerhand_o6_bihand.yaml"
3839
OPEN_CLOSE_HOLD_S = 1.0
3940
GRIPPER_RATE_HZ = 30.0
4041
VR_HAND_POSE_RATE_HZ = 60.0
@@ -56,7 +57,7 @@ def parse_args() -> argparse.Namespace:
5657
"--driver",
5758
choices=["linkerhand_l6", "linkerhand_o6"],
5859
default="linkerhand_l6",
59-
help="Hand driver to test. O6 currently supports open_close and gripper only.",
60+
help="Hand driver to test.",
6061
)
6162
parser.add_argument(
6263
"--mode",
@@ -77,8 +78,6 @@ def parse_args() -> argparse.Namespace:
7778
help='RS485 serial port such as /dev/ttyUSB0; "None" uses CAN',
7879
)
7980
args = parser.parse_args()
80-
if args.driver == "linkerhand_o6" and args.mode == "vr_hand_pose":
81-
raise SystemExit("hands.driver=linkerhand_o6 supports only --mode open_close or gripper")
8281
args.speed = list(O6_DEFAULT_SPEED if args.driver == "linkerhand_o6" else DEFAULT_SPEED)
8382
args.open_pose = list(O6_OPEN_POSE if args.driver == "linkerhand_o6" else OPEN_POSE)
8483
args.close_pose = list(O6_CLOSE_POSE if args.driver == "linkerhand_o6" else CLOSE_POSE)
@@ -113,7 +112,8 @@ def make_config(args: argparse.Namespace, *, mode: str) -> dict[str, object]:
113112
"frame_timeout_s": FRAME_TIMEOUT_S,
114113
driver_section: driver_cfg,
115114
"somehand": {
116-
"config_path": DEFAULT_SOMEHAND_CONFIG_PATH,
115+
"l6_config_path": DEFAULT_L6_SOMEHAND_CONFIG_PATH,
116+
"o6_config_path": DEFAULT_O6_SOMEHAND_CONFIG_PATH,
117117
"rate_hz": VR_HAND_POSE_RATE_HZ,
118118
"max_iterations": 12,
119119
"temporal_filter_alpha": 1.0,
@@ -260,9 +260,6 @@ def run_gripper(args: argparse.Namespace) -> None:
260260

261261

262262
def run_vr_hand_pose(args: argparse.Namespace) -> None:
263-
if args.hand_type != "both":
264-
raise SystemExit("hands.mode=vr_hand_pose currently requires --hand-type both")
265-
266263
config = make_config(args, mode="vr_hand_pose")
267264
provider = make_pico_provider()
268265
device, mapper = build_driver_runtime(config, driver=args.driver)
File renamed without changes.

0 commit comments

Comments
 (0)