Skip to main content

Start and view the cameras

On robots with onboard AI compute (NVIDIA Orin or Thor), including Marvin Pro, Luna, and Skye, the head and wrist cameras run on the Orin or Thor. Start them after every power-up, before you teleoperate or record data.

Start the cameras​

  1. Log in to the Orin or Thor. The factory login is nvidia / nvidia (Thor: user / 1). See Connect to the Orin or Thor.

    ssh nvidia@6.6.7.100
  2. Start the cameras. The script's folder depends on the robot.

    Luna V3.0, which has the S56 stereo camera:

    cd ~/s56+shw5g/cam_geac
    ./rb_camera.sh init

    Other robots:

    cd /home/nvidia/cam_geac
    ./rb_camera.sh init
  3. Wait about 20 seconds for the cameras to come up.

View a camera image​

To check that a camera works, open its live preview with argus_camera. Choose the camera with -d:

argus_camera -d 0
-dCamera
0, 1Head cameras
4, 5Wrist cameras
Connect a monitor to see the image

argus_camera shows the preview on the Orin's or Thor's own display. If you run it over SSH, it may stop with EGL_KHR_stream not supported. This means there's no display to show the image: connect an external monitor to the Orin or Thor and run the command in a terminal on that screen.

To use the RealSense RGB-D cameras in ROS 2, see RealSense cameras (ROS 2). If you can't log in, see If you can't log in.

If argus_camera fails with Failed to create CaptureSession​

If argus_camera stops with errors like these, followed by Segmentation fault:

Error generated. .../Dispatcher.cpp, createSession:1490 Failed to create CaptureSession

The Apex camera service is most likely still using the cameras. Stop it and restart the Argus daemon, then run argus_camera again:

sudo systemctl stop apex-camera
sudo systemctl restart nvargus-daemon
argus_camera -d 0

When you've finished checking the image, start the camera service again so that Apex gets its camera feeds back:

sudo systemctl start apex-camera

If the error persists, see Camera, Argus, H264, and WebRTC in the teleoperation FAQ.

Apex camera service (gmsl_quadcam)​

This section describes the slot configuration, ROS 2 interfaces, WebRTC video, and MCAP recording workflow of the current gmsl_quadcam four-camera component. Camera models and device numbers vary by delivery, so verify the configuration on the target system.

Camera slots​

The component provides four fixed logical slots:

SlotMosaic positionTypical use
head_leftTop leftLeft head camera
head_rightTop rightRight head camera
wrist_leftBottom leftLeft wrist camera
wrist_rightBottom rightRight wrist camera

CUDA processing combines the four inputs into a 2x2 mosaic. Each slot independently defines its model, physical device, rotation, fit mode, calibration, and depth settings. Set an unused slot to source: "none".

A common configuration uses SH5 head cameras and D405 wrist cameras. SH5 uses CSI/Argus, while SH3 and D405 use V4L2. Device numbers depend on enumeration. Check them before editing the configuration:

v4l2-ctl --list-devices
v4l2-ctl --list-formats-ext

Start from a terminal​

On a delivered system, start Camera from Apex. To launch the installed component from a terminal:

source /etc/apex/apex_ros_env.sh
ros2 launch gmsl_quadcam quad_csi_quickview.launch.py

To use another configuration:

ros2 launch gmsl_quadcam quad_csi_quickview.launch.py \
params_file:=/absolute/path/to/quad_quickview.yaml

This launch already starts the WebRTC signaling server. Do not start another process on the same port.

Configuration file​

The main configuration is normally installed at:

/opt/kernelmind/apex/install/gmsl_quadcam/share/gmsl_quadcam/config/quad_quickview.yaml

Model defaults are stored under camera_profiles/ in the same directory. The main file contains profile references, physical connections, and mounting orientation:

quad_csi_quickview:
ros__parameters:
camera_profiles:
sh5: "camera_profiles/sh5.yaml"
sh3: "camera_profiles/sh3.yaml"
d405: "camera_profiles/d405.yaml"

camera:
head_left:
profile: "sh5"
source: "csi://0"
flip_method: false
head_right:
profile: "sh5"
source: "csi://1"
flip_method: true
wrist_left:
profile: "d405"
source: "/dev/video5"
depth_source: "/dev/video4"
flip_method: false
wrist_right:
profile: "d405"
source: "/dev/video7"
depth_source: "/dev/video6"
flip_method: true

Back up the deployed configuration before editing it. Active color and depth sources must be unique, all four slot keys must exist, and at least one camera must remain enabled.

Common parameters​

ParameterDescription
profileCamera model registered under camera_profiles
sourcecsi://N, /dev/videoN, /dev/gmslcamN, or none
depth_sourceIndependent V4L2 device used when the profile enables depth
fpsPer-camera capture negotiation rate; it does not define the mosaic rate
flip_methodtrue rotates the image by 180 degrees
fit_modecontain preserves aspect ratio with black bars; stretch fills the cell
calibration.enabledEnables CUDA undistortion for the slot
depth.enabledEnables the paired D405 depth device
depth.publish_rawPublishes the raw 16UC1 depth image

Disable a slot​

quad_csi_quickview:
ros__parameters:
camera:
head_right:
source: "none"

The disabled cell remains black and displays CAM N DISABLED. Do not set all four slots to none.

ROS topics​

The default mosaic.fps is 30 Hz. Most single-camera, undistorted, H.264, JPEG, and compressed-depth topics use subscriber-driven publication. Their actual rate cannot exceed the valid input and mosaic configuration; measure it on the target with ros2 topic hz.

Per-camera color images​

TopicTypeFormatPublication condition
/camera/left_eye/image_nv12sensor_msgs/msg/ImageNV12Exported when a subscriber exists
/camera/right_eye/image_nv12sensor_msgs/msg/ImageNV12Exported when a subscriber exists
/camera/left_wrist/image_nv12sensor_msgs/msg/ImageNV12Exported when a subscriber exists
/camera/right_wrist/image_nv12sensor_msgs/msg/ImageNV12Exported when a subscriber exists
/camera/left_eye/image_nv12_undistortedsensor_msgs/msg/ImageNV12Slot calibration enabled and a subscriber exists
/camera/right_eye/image_nv12_undistortedsensor_msgs/msg/ImageNV12Slot calibration enabled and a subscriber exists
/camera/left_wrist/image_nv12_undistortedsensor_msgs/msg/ImageNV12Slot calibration enabled and a subscriber exists
/camera/right_wrist/image_nv12_undistortedsensor_msgs/msg/ImageNV12Slot calibration enabled and a subscriber exists

The mosaic path does not depend on these ROS images. NVMM-to-CPU export occurs only when a per-camera color topic has a subscriber.

Mosaic and depth images​

TopicTypeCompressedImage.formatPublication condition / description
/quad_tile/compressedsensor_msgs/msg/CompressedImageh264Raw four-camera mosaic; starts its encoder when subscribed
/quad_tile/compressed_undistortedsensor_msgs/msg/CompressedImageh264Mosaic processed according to each slot's calibration; ROS messages publish when subscribed
/quad_tile/jpeg/compressedsensor_msgs/msg/CompressedImagejpegProcessed JPEG mosaic; Reliable, KeepLast(1), default 640x360 at quality 55
/camera/<name>/depth/image_rawsensor_msgs/msg/Image16UC1depth.enabled=true and publish_raw=true
/camera/<name>/depth/image_raw/compressedsensor_msgs/msg/CompressedImageh264Depth enabled; grayscale mapping and encoding run when subscribed

/recorder/quad_tile/compressed_undistorted is written directly to MCAP and is not exposed as a normal ROS publisher.

WebRTC video​

The default signaling URL is ws://127.0.0.1:8554, with rooms 10, 11, and 12. All rooms share the same processed mosaic H.264 stream; they are not separate layouts or encoders.

The default WebRTC output is 1920x1488, and the mosaic reads the latest frames at 30 Hz. An active slot that receives no frame for more than 500 ms displays STALE.

Record to MCAP​

Recording service:

/recorder/set_recording
marvin_msgs/srv/VideoCapture

Start recording:

ros2 service call /recorder/set_recording marvin_msgs/srv/VideoCapture \
"{start_stop: true, save_dir: ''}"

Stop recording:

ros2 service call /recorder/set_recording marvin_msgs/srv/VideoCapture \
"{start_stop: false, save_dir: ''}"

An empty save_dir uses recorder.directory. The default is /media/marvin/BAG_STORAGE/recorded_bags. Recordings contain the processed mosaic H.264 stream in MCAP storage.

RealSense D435​

The D435 RGB-D camera uses its own configuration and isn't part of the four-camera mosaic. See With Apex installed on the RealSense page.

Quick checks​

source /etc/apex/apex_ros_env.sh

ros2 node list
ros2 topic list | grep -Ei 'camera|image|compressed|quad'
ros2 topic info /quad_tile/compressed_undistorted -v
ros2 topic hz /quad_tile/compressed_undistorted
SymptomInterpretation and action
No data on /quad_tile/compressedCheck for subscribers first; its encoder remains stopped without subscribers by design
No compressed depth dataVerify that depth is enabled and a subscriber exists
CAM N DISABLEDThe slot source is none
CAM N STALEThe slot has received no new frame for more than 500 ms
WebRTC reports unavailableCheck Camera, ensure only one signaling server is running, and inspect room, SDP, and ICE logs
Repeated capture or Argus errorsCheck the GMSL link, sensor mode, cable, device number, and driver

For full communication diagnostics, see Skye/Luna modular communication diagnostics or Marvin Pro modular communication diagnostics.