Restrict ROS2 DDS communication to specific network interfaces
One of the main architectural differences between ROS1 and ROS2 is that ROS1 requires a discovery mechanism running on a single network node while ROS2 provides decentralized discovery. As a consequence the average user is rarely required to consider specific implementation details of ROS2 networking. However, every now and then very specific circumstances arise in which expert knowledge of ROS2 networking principles and configuration is necessary to achieve your desired outcomes.
Note: The content of this article applies to ROS2 Humble Hawksbill using eProsima’s Fast RTPS DDS. Same or similar principles apply to other ROS2 versions and DDS, though the specific implementation details may differ.
In its default configuration Fast RTPS is publishing its UDP multicast discovery packages using all available network interfaces. However, when running ROS2 nodes within a Docker container a situation can arise where ROS2 nodes running on a different computer (but on the same network) are not visible within the Docker container.
Let’s consider the following real-world example, where ROS2 nodes are executed within a Docker container on the Pika Spark micro robot control system: The Pika Spark has two available network interfaces: a) Gigabit-Ethernet (192.168.1.x/24 – static IP configuration) which is used for exchanging data using a custom protocol with a Livox Mid-360 laser scanner and b) WiFi (192.168.39.x/24 – dynamic IP configuration by WiFi Access Point) for communicating with a ROS2 based control software running on another host.
The Docker image is started using --network host which is transparently mapping the network interfaces of the host (in this case a Pika Spark micro robot control system) into the Docker container.
docker run -it --network host pika_spark_docker_image bash
In the default configuration ROS2 nodes running inside the Docker container will send their UDP multicast discovery packets using both Gigabit-Ethernet and WiFi. However, since the Livox Mid-360 laser scanner is not a ROS-enabled device it is not necessary to send ROS2 network traffic over Gigabit-Ethernet.
In order to limit ROS2 network traffic to a specific interface – WiFi in my example – a Fast RTPS default profile file must be created. The Fast RTPS default profile file is a text file encoding Fast RTPS specific configuration options using XML. Using interface whitelisting the user can specify one or multiple network interface IP addresses to be used for ROS2 communication. In my example I want to restrict ROS2 network traffic to Pika Spark’s WiFi interface – thereby only one network interface (identified by its IP address) needs to be whitelisted: 192.168.39.27. This leads to a Fast RTPS default profile file as show here:
<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
<transport_descriptors>
<transport_descriptor>
<transport_id>CustomUDPTransport</transport_id>
<type>UDPv4</type>
<interfaceWhiteList>
<address>192.168.39.27</address>
</interfaceWhiteList>
</transport_descriptor>
<transport_descriptor>
<transport_id>CustomTCPTransport</transport_id>
<type>TCPv4</type>
<interfaceWhiteList>
<address>192.168.39.27</address>
</interfaceWhiteList>
</transport_descriptor>
</transport_descriptors>
<participant profile_name="CustomTransportParticipant" is_default_profile="true">
<rtps>
<userTransports>
<transport_id>CustomUDPTransport</transport_id>
<transport_id>CustomTCPTransport</transport_id>
</userTransports>
<useBuiltinTransports>false</useBuiltinTransports>
</rtps>
</participant>
</profiles>
Note: Unfortunately available documentation on interface whitelisting is both sparse – and partially wrong. Here are the two relevant points – not mentioned in the official documentation – which took me the better part of a weekend to put together:
1) is_default_profile="true" needs to be set to true – otherwise your configuration will not be used. Doing so also necessitates that you specify both CustomUDPTransport and CustomTCPTransport within CustomTransportParticipant, because you can not create more than one default participant profile.
2) <useBuiltinTransports>false</useBuiltinTransports> needs to be set to false. This is necessary because Fast RTPS attempts to use shared memory for default communication which creates a problem in a setup where ROS2 nodes are running in different Docker containers on the same host. Alternately shared memory can be shared between host and Docker container by mounting the shared memory using -v /dev/shm:/dev/shm.
Once the creation of the Fast RTPS default profile file is complete its location need to be made known to the Fast RTPS DDS implementation. This is achieved by setting the FASTRTPS_DEFAULT_PROFILES_FILE environment variable, i.e.
export FASTRTPS_DEFAULT_PROFILES_FILE=~/fastrtps-whitelist-interfaces.xml
The selection of Fast RTPS DDS as ROS2 DDS middleware is done by setting the RMW_IMPLEMENTATION environment variable to rmw_fastrtps_cpp, i.e.
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
A final and very important point: before you continue working with ROS2 you need to make sure that the ROS2 daemon is stopped by invoking
ros2 daemon stop
The ROS2 daemon is a background service which is started automatically on the first time you are invoking any ROS2 command, i.e. ros2 node list or ros2 launch or … . Since the ROS2 DDS configuration is only parsed during daemon start none of the previous configuration changes will be in effect until the daemon is stopped and started again.
Sometimes – as is the case for WiFi in my example – an IP address is assigned dynamically using DHCP. Since interface whitelisting requires the specification of an IP address (instead simply of network devices) I’ve created a shell script which can be invoked to create a Fast RTPS default configuration file. The shell script accepts one or multiple network device names as parameter and retrieves and writes their IP address in the Fast RTPS default configuration file.
./fastrtps-whitelist-interfaces.sh wlan0 eth0 ...
Alexander Entinger is a highly experienced embedded engineer with a focus on robotic systems. By providing hard-won expertise designing embedded systems for real-world embedded applications Alexander Entinger is helping companies developing robotic systems or components for robotic systems to achieve their desired business outcomes.