RealSense cameras (ROS 2)
Marvin Pro, Luna, and Skye have Intel RealSense RGB-D cameras. This guide shows how to start a RealSense camera in ROS 2 and read its images, calibration, and frames, as a starting point for your own vision code.
| Robot | RealSense cameras |
|---|---|
| Marvin Pro | D435 (head) |
| Luna | D455C (head, standard); D435 (chest, optional); D405GA (wrist, optional) |
| Skye | D435 (head, standard); D435 (chest, optional); D405GA (wrist, optional) |
These steps work for both the D435 and the D455. The same ROS 2 driver also supports the D405, but its available streams can differ; use ros2 topic list to see what your camera publishes.
Before you start
Run these commands on the robot's Orin or Thor. Log in with ssh nvidia@6.6.7.100; see Connect to the Orin or Thor. The commands use ROS 2 Humble. In every new terminal, load ROS 2 first:
source /opt/ros/humble/setup.bash
Install and start the camera node
-
Install the RealSense ROS 2 driver:
sudo apt install ros-humble-realsense2-camera -
Start the camera node:
ros2 launch realsense2_camera rs_launch.pyLeave it running, and open a second terminal for the commands below.
Check topics and data
-
List all ROS 2 topics, then only the camera topics:
ros2 topic listros2 topic list | grep camera -
Print one RGB image message and one depth image message:
ros2 topic echo /camera/camera/color/image_raw --onceros2 topic echo /camera/camera/depth/image_rect_raw --onceImage messages include every pixel value. Add
--no-arrto show only the header and image size. -
Check how often the RGB and depth images are published:
ros2 topic hz /camera/camera/color/image_rawros2 topic hz /camera/camera/depth/image_rect_raw -
Show each topic's publishers, subscribers, and message type:
ros2 topic info /camera/camera/color/image_rawros2 topic info /camera/camera/depth/image_rect_rawros2 topic type /camera/camera/color/image_rawros2 topic type /camera/camera/depth/image_rect_raw
Read the calibration
| To read | Command |
|---|---|
| RGB camera intrinsics | ros2 topic echo /camera/camera/color/camera_info --once |
| Depth camera intrinsics | ros2 topic echo /camera/camera/depth/camera_info --once |
| Depth-to-color extrinsics | ros2 topic echo /camera/camera/extrinsics/depth_to_color --once |
camera_info gives the focal length, principal point, and distortion parameters. The extrinsics give the position of the depth camera relative to the RGB camera.
Check node parameters
List the node's parameters, then read the depth and color stream profiles (resolution and frame rate):
ros2 param list /camera/camera
ros2 param get /camera/camera depth_module.depth_profile
ros2 param get /camera/camera rgb_camera.color_profile
Check the TF frames
ros2 run tf2_tools view_frames
ros2 topic echo /tf_static --once
view_frames listens to the TF tree for a few seconds and saves it as a PDF in the current folder. /tf_static shows the fixed transforms between the camera's frames. Use TF to convert between camera and robot coordinates.
Check the device and node
| To check | Command |
|---|---|
| Connected RealSense devices (USB) | rs-enumerate-devices |
| Live view in the RealSense Viewer | realsense-viewer |
| Running nodes | ros2 node list |
| The camera node's topics, services, and parameters | ros2 node info /camera/camera |
rs-enumerate-devices and realsense-viewer come with Intel's RealSense SDK tools. realsense-viewer opens a window, so run it on a monitor connected to the Orin or Thor, or in a NoMachine session.
Common topics
| Data | Topic | Use |
|---|---|---|
| RGB image | /camera/camera/color/image_raw | Input for object detection and vision algorithms |
| Depth image | /camera/camera/depth/image_rect_raw | Distance measurement and 3D localization |
| RGB camera parameters | /camera/camera/color/camera_info | Focal length, principal point, distortion |
| Depth camera parameters | /camera/camera/depth/camera_info | Focal length, principal point, distortion |
| Depth-to-color extrinsics | /camera/camera/extrinsics/depth_to_color | Relative position of the depth and RGB cameras |
| TF | /tf, /tf_static | Transforms between camera and robot frames |
The topic names start with /camera/camera because the launch file's default namespace and camera name are both camera. If you launch with a different camera_namespace or camera_name, the topic names change to match.
With Apex installed
On a robot with Apex, the D435 is started by the Apex camera package with its own configuration file, config/rgbd_cameras.yaml, instead of rs_launch.py. It isn't part of the four-camera mosaic used for teleoperation.
source /etc/apex/apex_ros_env.sh
ros2 launch gmsl_quadcam rgbd_cameras.launch.py
Its topics are under /camera/d435/ rather than /camera/camera/:
/camera/d435/color/image_raw
/camera/d435/depth/image_rect_raw
/camera/d435/color/camera_info
/camera/d435/depth/camera_info
For the head and wrist camera mosaic, see Apex camera service.
Troubleshooting
| Problem | What to check |
|---|---|
ros2: command not found | Run source /opt/ros/humble/setup.bash in that terminal. |
| No camera topics after launch | Run rs-enumerate-devices to check that the camera is detected. If another program is already using the camera, stop it first. |
realsense-viewer doesn't open over SSH | It needs a display. Connect a monitor or use NoMachine. |
For the robot's other cameras, see Start and view the cameras.