Teleoperation with Apex
Gento Teleoperation Apex is the VR teleoperation system for Marvin Pro, Skye, and Luna. The operator sees a live first-person view through a VR headset, and their motion is mapped to the robot with low latency. Apex also records robot data and video for demonstration datasets, and provides ROS 2 interfaces for your own applications. Manus data gloves with a Wuji Hand are optional.
Where to start
| Your robot | Start here |
|---|---|
| Marvin Pro, current version (Tianzhun controller) | Wiring, then Marvin Pro: first teleoperation |
| Marvin Pro, earlier version (Orin controller) | Earlier Marvin Pro (Orin) |
| Skye or Luna | Skye and Luna: first teleoperation |
To install, check, or update the software, see Install or update Apex. After every power-up of the robot's controller, start the cameras before you teleoperate.
Software layers
The Apex host software connects to the robot controller, manages teleoperation modules and control modes, displays camera streams, records and plays back data, and provides runtime logs. A delivered software set includes the robot controller layer, the Teleop service, and the frontend. Their version numbers refer to different components.
| Software layer | Current documentation baseline | Purpose |
|---|---|---|
| Marvin Pro controller | gento-apex 1.1.7.1 | Current Pro source-review target with Gento SDK 4.7.1; the source still marks it as unreleased |
| Skye/Luna controller | v1.0.6.95g-beta-10-g14cfc0d | Current Gento source-review snapshot; use the customer package listed in the delivery manifest |
| Teleop service and frontend | Matched to the delivery release | Processes teleoperation data and provides the host interface; do not mix it with another controller release |
| Headset client | Matched to the delivery release | Provides video and teleoperation input |
Versions continue to evolve. These numbers identify the current documentation baseline; always use matching software from one delivery release.
Apex Teleop interface by robot
| Product | Interface | Notes |
|---|---|---|
| Marvin Pro (current Tianzhun version) | Current interface | Current controller source-review target is gento-apex 1.1.7.1, which is not yet marked as a formal release |
| Marvin Pro (historical Orin version) | Classic interface | For previously delivered systems |
| Skye / Luna | Classic interface | Current source-review snapshot is v1.0.6.95g-beta-10-g14cfc0d |
Marvin Pro supports Pico and Meta Quest headsets. Skye and Luna full-body teleoperation uses Pico with waist and leg trackers.
VR headset
Use Ethernet port 1 when connecting the headset to a Tianzhun controller and Ethernet port 2 with a Lingjing Thor controller. Both controllers use 6.6.7.100 as the default connection address. Follow the delivered configuration if the site network parameters have been changed.
| Headset | Guides |
|---|---|
| Meta Quest | App installation · User manual |
| Pico | App installation · User manual |
The headset app's privacy policy describes what data it handles.
More guides
- Teleoperation modes and input sources: full-body, dual-arm, and mobile-base operation, and IK sources
- Data-Processing-Tool: turn recordings into LeRobot datasets
- VLA guide: train and deploy vision-language-action policies on recorded data
- Troubleshooting and FAQ
- Advanced: configuration changes, topic whitelist, and communication diagnostics for Marvin Pro and Skye/Luna
- ROS 2 interfaces: Marvin Pro and Skye and Luna
- End effectors: ApexTool grippers, DM gripper CAN protocol, and Wuji Hand 2