How to accellerate ROS 2 discovery using Fast DDS discovery server

By default ROS 2 is using the simple discovery protocol for node, topic and service discovery via UDP multicast communication. While this approach does not require a central server (as was the case in ROS “1”), it can be problematic with systems containing a large number of nodes or when routing ROS 2 communication over WiFi networks, which not always take kindly to multicast traffic.

As a solution for a dynamic centralized discovery mechanism eProsima’s Fast DDS discovery server can be used. Doing so considerably reduces the number of packets sent during the discovery phase, as benchmarked in the large scale open-rmf/rmf_demos “Clinic World” scenario:

Note: The content of this article applies to ROS 2 Jazzy Jalisco. Same or similar principles apply to other versions of ROS 2, though specific implementation and configuration details may differ.

First step is installing the Fast DDS discovery server via sudo apt-get install fastdds-tools. Once installed the Fast DDS discovery server can be brought up via fastdds discovery --server-id 0. Since you can use multiple discovery servers simultaneously, parameter --server-id is used to set the unique number/ID of a specific discovery server instance.

By default the server can be reached via 0.0.0.0:11811. Should you required a specific IP address and port configuration you can do so via parameters --ip-address and --port.

1

fastdds discovery --server-id 0 --ip-address 127.0.0.1 --port 11811

It can be convenient to automatically start the Fast DDS discovery server during system startup. The bash script setup_fastdds_discovery_server.sh reproduced below obtains the IP address of the eno1 interface and binds the Fast DDS discovery server to it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

#!/bin/bash
source /opt/ros/jazzy/setup.bash

set -euo pipefail
IFS=$'\n\t'

IP=$(ip -4 addr show eno1 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')
echo "ROS_DISCOVERY_SERVER: using IP address: $IP"

export ROS_DISCOVERY_SERVER=$IP:11811
fastdds discovery --server-id 0 --ip-address $IP

Further create a file fastdds-discovery-server.service in /etc/systemd/system and point it to the location of above bash script (i.e. /opt/fastdds/setup_fastdds_di...).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

[Unit]
Description=ROS FastDDS Discovery Server Service
After=network.target

[Service]
Type=simple
User=MY_USER
ExecStart=/opt/fastdds/setup_fastdds_discovery_server.sh
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Test your configuration with sudo systemctl start fastdds-discovery-server.service, sudo systemctl status fastdds-discovery-server.service and sudo systemctl status fastdds-discovery-server.service. Finally, enable startup at boot time via

1

sudo systemctl enable fastdds-discovery-server.service

As a last step all nodes need to be made aware of the discovery server. This is achieved by setting the environment variable ROS_DISCOVERY_SERVER. Note: It is highly recommended to first stop all running ROS 2 nodes and the ROS 2 daemon (ros2 daemon stop).

1
2

ros2 daemon stop
export ROS_DISCOVERY_SERVER=127.0.0.1:11811

Attention: An important caveat to mention is that many ROS 2 introspection tools (such as ros2 topic [list|echo|...]) DO NOT WORK out of the box when you are using the Fast DDS discovery server. In order to re-enable those tools each ROS 2 participatent needs to be configured as a super client. This can be done by providing a custom FastRTPS profile, see content of fastdds_super_client_configuration.xml below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

<?xml version="1.0" encoding="UTF-8" ?>
<dds>
    <profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
        <participant profile_name="super_client_profile" is_default_profile="true">
            <rtps>
                <builtin>
                    <discovery_config>
                        <discoveryProtocol>SUPER_CLIENT</discoveryProtocol>
                        <discoveryServersList>
                            <RemoteServer prefix="44.53.00.5f.45.50.52.4f.53.49.4d.41">
                                <metatrafficUnicastLocatorList>
                                    <locator>
                                        <udpv4>
                                            <address>127.0.0.1</address>
                                            <port>11811</port>
                                        </udpv4>
                                    </locator>
                                </metatrafficUnicastLocatorList>
                            </RemoteServer>
                        </discoveryServersList>
                    </discovery_config>
                </builtin>
            </rtps>
        </participant>
    </profiles>
</dds>

The <RemoteServer prefix="..." value needs to be adjusted depending on the discovery server id and is shown during discovery server startup:

1
2
3
4
5

...
Server ID:          0
Server GUID prefix: 44.53.00.5f.45.50.52.4f.53.49.4d.41
Server Addresses:   UDPv4:[127.0.0.1]:11811
...

Furthermore, the ROS 2 system needs to be aware of the special FastRTPS configuration. This can be achieved by setting the environment variable FASTRTPS_DEFAULT_PROFILES_FILE to point to the location of fastdds_super_client_configuration.xml. Overall I recommend using a script such as setup_fastdds.sh shown below to configure your shell environment when using a Fast DDS discovery server.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

#!/bin/bash

source /opt/ros/jazzy/setup.bash
ros2 daemon stop

export ROS_DISCOVERY_SERVER="127.0.0.1:11811"
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
export FASTRTPS_DEFAULT_PROFILES_FILE=$(pwd)/fastdds_super_client_configuration.xml

echo "ROS_DISCOVERY_SERVER           = " $ROS_DISCOVERY_SERVER
echo "RMW_IMPLEMENTATION             = " $RMW_IMPLEMENTATION
echo "FASTRTPS_DEFAULT_PROFILES_FILE = " $FASTRTPS_DEFAULT_PROFILES_FILE

ros2 daemon start

Keep in mind to source setup_fastdds.sh or . setup_fastdds.sh which executes the script in your current shell environment, preserving exported environment variables after termination of the script.

More detailed information can be found at the eProsima`s Fast DDS and ROS 2/Tutorials/Advanced documentation.