Skip to main content

Teleoperation with Apex

Gento Teleoperation Apex is the VR teleoperation system for Marvin Pro, Skye, and Luna. The operator sees a live first-person view through a VR headset, and their motion is mapped to the robot with low latency. Apex also records robot data and video for demonstration datasets, and provides ROS 2 interfaces for your own applications. Manus data gloves with a Wuji Hand are optional.

Where to start​

Your robotStart here
Marvin Pro, current version (Tianzhun controller)Wiring, then Marvin Pro: first teleoperation
Marvin Pro, earlier version (Orin controller)Earlier Marvin Pro (Orin)
Skye or LunaSkye and Luna: first teleoperation

To install, check, or update the software, see Install or update Apex. After every power-up of the robot's controller, start the cameras before you teleoperate.

Software layers​

The Apex host software connects to the robot controller, manages teleoperation modules and control modes, displays camera streams, records and plays back data, and provides runtime logs. A delivered software set includes the robot controller layer, the Teleop service, and the frontend. Their version numbers refer to different components.

Software layerCurrent documentation baselinePurpose
Marvin Pro controllergento-apex 1.1.7.1Current Pro source-review target with Gento SDK 4.7.1; the source still marks it as unreleased
Skye/Luna controllerv1.0.6.95g-beta-10-g14cfc0dCurrent Gento source-review snapshot; use the customer package listed in the delivery manifest
Teleop service and frontendMatched to the delivery releaseProcesses teleoperation data and provides the host interface; do not mix it with another controller release
Headset clientMatched to the delivery releaseProvides video and teleoperation input

Versions continue to evolve. These numbers identify the current documentation baseline; always use matching software from one delivery release.

Apex Teleop interface by robot​

ProductInterfaceNotes
Marvin Pro (current Tianzhun version)Current interfaceCurrent controller source-review target is gento-apex 1.1.7.1, which is not yet marked as a formal release
Marvin Pro (historical Orin version)Classic interfaceFor previously delivered systems
Skye / LunaClassic interfaceCurrent source-review snapshot is v1.0.6.95g-beta-10-g14cfc0d

Marvin Pro supports Pico and Meta Quest headsets. Skye and Luna full-body teleoperation uses Pico with waist and leg trackers.

VR headset​

Wired Headset Connection

Use Ethernet port 1 when connecting the headset to a Tianzhun controller and Ethernet port 2 with a Lingjing Thor controller. Both controllers use 6.6.7.100 as the default connection address. Follow the delivered configuration if the site network parameters have been changed.

HeadsetGuides
Meta QuestApp installation · User manual
PicoApp installation · User manual

The headset app's privacy policy describes what data it handles.