Skip to main content

RealSense cameras (ROS 2)

Marvin Pro, Luna, and Skye have Intel RealSense RGB-D cameras. This guide shows how to start a RealSense camera in ROS 2 and read its images, calibration, and frames, as a starting point for your own vision code.

RobotRealSense cameras
Marvin ProD435 (head)
LunaD455C (head, standard); D435 (chest, optional); D405GA (wrist, optional)
SkyeD435 (head, standard); D435 (chest, optional); D405GA (wrist, optional)

These steps work for both the D435 and the D455. The same ROS 2 driver also supports the D405, but its available streams can differ; use ros2 topic list to see what your camera publishes.

Before you start​

Run these commands on the robot's Orin or Thor. Log in with ssh nvidia@6.6.7.100; see Connect to the Orin or Thor. The commands use ROS 2 Humble. In every new terminal, load ROS 2 first:

source /opt/ros/humble/setup.bash

Install and start the camera node​

  1. Install the RealSense ROS 2 driver:

    sudo apt install ros-humble-realsense2-camera
  2. Start the camera node:

    ros2 launch realsense2_camera rs_launch.py

    Leave it running, and open a second terminal for the commands below.

Check topics and data​

  1. List all ROS 2 topics, then only the camera topics:

    ros2 topic list
    ros2 topic list | grep camera
  2. Print one RGB image message and one depth image message:

    ros2 topic echo /camera/camera/color/image_raw --once
    ros2 topic echo /camera/camera/depth/image_rect_raw --once

    Image messages include every pixel value. Add --no-arr to show only the header and image size.

  3. Check how often the RGB and depth images are published:

    ros2 topic hz /camera/camera/color/image_raw
    ros2 topic hz /camera/camera/depth/image_rect_raw
  4. Show each topic's publishers, subscribers, and message type:

    ros2 topic info /camera/camera/color/image_raw
    ros2 topic info /camera/camera/depth/image_rect_raw
    ros2 topic type /camera/camera/color/image_raw
    ros2 topic type /camera/camera/depth/image_rect_raw

Read the calibration​

To readCommand
RGB camera intrinsicsros2 topic echo /camera/camera/color/camera_info --once
Depth camera intrinsicsros2 topic echo /camera/camera/depth/camera_info --once
Depth-to-color extrinsicsros2 topic echo /camera/camera/extrinsics/depth_to_color --once

camera_info gives the focal length, principal point, and distortion parameters. The extrinsics give the position of the depth camera relative to the RGB camera.

Check node parameters​

List the node's parameters, then read the depth and color stream profiles (resolution and frame rate):

ros2 param list /camera/camera
ros2 param get /camera/camera depth_module.depth_profile
ros2 param get /camera/camera rgb_camera.color_profile

Check the TF frames​

ros2 run tf2_tools view_frames
ros2 topic echo /tf_static --once

view_frames listens to the TF tree for a few seconds and saves it as a PDF in the current folder. /tf_static shows the fixed transforms between the camera's frames. Use TF to convert between camera and robot coordinates.

Check the device and node​

To checkCommand
Connected RealSense devices (USB)rs-enumerate-devices
Live view in the RealSense Viewerrealsense-viewer
Running nodesros2 node list
The camera node's topics, services, and parametersros2 node info /camera/camera

rs-enumerate-devices and realsense-viewer come with Intel's RealSense SDK tools. realsense-viewer opens a window, so run it on a monitor connected to the Orin or Thor, or in a NoMachine session.

Common topics​

DataTopicUse
RGB image/camera/camera/color/image_rawInput for object detection and vision algorithms
Depth image/camera/camera/depth/image_rect_rawDistance measurement and 3D localization
RGB camera parameters/camera/camera/color/camera_infoFocal length, principal point, distortion
Depth camera parameters/camera/camera/depth/camera_infoFocal length, principal point, distortion
Depth-to-color extrinsics/camera/camera/extrinsics/depth_to_colorRelative position of the depth and RGB cameras
TF/tf, /tf_staticTransforms between camera and robot frames

The topic names start with /camera/camera because the launch file's default namespace and camera name are both camera. If you launch with a different camera_namespace or camera_name, the topic names change to match.

With Apex installed​

On a robot with Apex, the D435 is started by the Apex camera package with its own configuration file, config/rgbd_cameras.yaml, instead of rs_launch.py. It isn't part of the four-camera mosaic used for teleoperation.

source /etc/apex/apex_ros_env.sh
ros2 launch gmsl_quadcam rgbd_cameras.launch.py

Its topics are under /camera/d435/ rather than /camera/camera/:

/camera/d435/color/image_raw
/camera/d435/depth/image_rect_raw
/camera/d435/color/camera_info
/camera/d435/depth/camera_info

For the head and wrist camera mosaic, see Apex camera service.

Troubleshooting​

ProblemWhat to check
ros2: command not foundRun source /opt/ros/humble/setup.bash in that terminal.
No camera topics after launchRun rs-enumerate-devices to check that the camera is detected. If another program is already using the camera, stop it first.
realsense-viewer doesn't open over SSHIt needs a display. Connect a monitor or use NoMachine.

For the robot's other cameras, see Start and view the cameras.