From Hand to Hand: A Shared Wearable Interface and Functional Benchmark for Dexterous Manipulation
Policy Learning Across Hands
We collect 70 demonstrations per task–hand pair and train Diffusion Policy with wrist RGB and relative EE pose. Policies are evaluated on Zipper, Lid, and Coin for both WUJI Hand v1 and LEAP Hand on a UFactory xArm 6.
WUJI Hand v1
LEAP Hand
WUJI Hand v1
LEAP Hand
(Wearable, Unified Dexterous Interface) supports collection in everyday environments after brief familiarization. On LEAP Hand, an expert operator achieves 3.62–7.71× more successful completions per five-minute window than vision-based teleoperation on Zipper, Coin, and Lid.
Method Overview
This video includes audio.
Collecting demonstrations for dexterous manipulation is difficult to scale across robot hands with different kinematics and contact geometry. Existing interfaces often require embodiment-specific sensing hardware, motion retargeting, or visual correction. We present , a Wearable, Unified Dexterous Interface that separates a shared sensorized exoskeleton from interchangeable passive hands and linkages. This design reuses the sensing hardware across embodiments and supports robot-free demonstration collection without visual correction within the supported workspace. We evaluate demonstration collection and policy learning with WUJI Hand v1 and LEAP Hand. To assess functional manipulation, we introduce , an eight-task suite inspired by occupational therapy that spans precision grasping, tool use, pulling, rotation, compression, and fastening. A 12-participant study evaluates usability and task performance, and demonstrations collected with WUDI support autonomous policies on three tasks across both hands. In an expert-operator comparison on LEAP Hand, WUDI yields 3.62–7.71× more successful task completions per collection window than a vision-based teleoperation baseline.
Mechanical Interface
Three components implement the separation of sensing and embodiment: a shared exoskeleton (16 resistive rotary sensors, ~$120 sensing assembly), interchangeable passive hands derived from target kinematics, and embodiment-specific four-bar linkages that transmit MCP/PIP flexion.
WUJI Hand v1 (left) · LEAP Hand (right). A shared sensorized exoskeleton connects to interchangeable passive hands through embodiment-specific linkages.
Shared across every hand
Sensorized exoskeleton
Joint sensing
Worn by the operator and left unchanged when the passive hand is replaced. Each finger accommodates MCP abduction/adduction and flexion plus PIP flexion; the thumb adds CMC, MCP and IP rotation. Sixteen resistive rotary sensors measure joint angles or linkage rotation relative to the supporting segment.
Localization
The iPhone on top streams its 6-DoF ARKit pose over USB via Record3D, and a keyed mount fixes the phone-to-palm transform, so the wrist is localized without a robot arm or external tracking. Pose samples are matched to the joint trajectory by timestamp and supply the policy’s relative end-effector poses.
- 15 operator-side DoF
- 16 rotary encoders
- 6-DoF wrist pose from ARKit
- ~$120 system, excl. phone & camera
Embodiment-specific
Four-bar linkages
Transmission
Couple the passive hand to the shared exoskeleton, accommodating MCP flexion, MCP abduction/adduction and PIP flexion. Following DEXOP, standoffs fix the base separation; the first coupler connects the two PIP axes and serves as the second linkage’s moving ground, and the second coupler connects the middle-link mounts.
Joints
Revolute joints on the linkage mounts connect the couplers: those of the first linkage system (green) and the serial joints of the second (red). A new hand needs only its own coupler dimensions and mount positions; thumb couplers are designed per embodiment.
- 2 serial four-bars per finger
- New hand = printed parts + fasteners
LEAP
Interchangeable
Passive hand + wrist camera
A replica of the target robot hand, derived from its kinematic description and link meshes. It reproduces the robot’s supported articulation and contact geometry but carries no encoders. A RealSense D405 below the palm records RGB-D from the viewpoint the robot will have at deployment.
No visual gap
Because the passive hand looks like the robot hand, collection frames match deployment frames, so policies train on them directly, with no segmentation, inpainting, or rendered hand replacement.
- RealSense D405 RGB-D
- CAD derived from the robot model
Explore & Extend WUDI
Inspect how the shared WUDI exoskeleton adapts to six robot hands through embodiment-specific couplers and mounts.
LEAP Hand
Reconstruction & Workspace
Mechanism-based reconstruction on LEAP recovers passive-hand joint angles with MAE 1.34°–4.20° and RMSE ≤ 5.50°. Motion capture validates the reconstruction; workspace bars compare WUDI with Vision teleop.
Motion-capture validation
Fingertip workspace coverage
Normalized against Vision teleop
LEAP Hand
WUJI Hand v1
Benchmark
(Occupational Therapy-inspired Hand Evaluation and tRaining) is an eight-task suite for functional dexterity during human-operated demonstration collection and autonomous execution. Tasks span precision grasp (Coin), grip–torque (Lid), tool use (Write), constrained pulling (Zipper), rotation (Spin), compression (Squeeze), and fastening (Buckle / Unbuckle).
Buy the OT busy board
User Study
A 12-participant study with the WUJI Hand v1 configuration measured usability and task performance. Donning took 16.8±6.3 s. Success reached 100% on Zipper, Spin, and Lid; no successes were observed on Squeeze or Unbuckle under the optional-continuation protocol. Perceived success averaged 3.83/5.