Vision-based hand control
This open-source project combines HaMeR’s three-dimensional hand reconstruction with optimization-based inverse kinematics to retarget human hand motion to an articulated robotic hand in simulation.
The system takes monocular RGB images as input, without data gloves or motion-capture equipment. It connects visual perception, three-dimensional hand representations, and robot control, extending my interest in low-cost dexterous manipulation.
Pipeline
- Reconstruct three-dimensional hand information from monocular RGB input.
- Retarget motion using optimization-based inverse kinematics.
- Use the retargeted motion to control an articulated robotic hand in simulation.
The project focuses on motion retargeting between human and robotic hands, examining how visual input can be converted into executable robot motion.