Guide: Debugging ROS & ROS2 Systems with ROSA [Work in Progress]

Introduction

Embodied agents like ROSA are revolutionizing the way we interact with complex robotic systems. By leveraging the power of Large Language Models (LLMs) and custom tools, ROSA can execute complex workflows, chain together sequences of tool calls, and automate routine tasks for debugging and troubleshooting robotic systems built on ROS and ROS2. Unlike traditional methods, ROSA can perform multiple operations in parallel, significantly reducing the time spent waiting for responses and aggregating results. This guide will demonstrate how to use ROSA to inspect, debug, and troubleshoot live ROS systems using natural language, and how to maximize the efficiency of your development and operations.

Important

This guide is a work-in-progress. Please check back regularly for updates.

Basics

ROSA simplifies interaction with your ROS system by allowing you to use natural language queries to perform common tasks. Instead of manually typing commands into the terminal, you can simply ask ROSA, and it will use its suite of tools to provide you with the information you need.

Listing Primitives

One of the most fundamental tools in the ROS developers toolbox is the ability to get lists of running nodes, topics, services, and so on. This is usually done in the CLI using the list subcommand (e.g. rosnode list and ros2 topic list). ROSA is well-equipped to make this easier than ever by applying filters and formatting the output in Markdown. For example, to get a list of all active nodes in your ROS system, you can simply ask ROSA:

User: "Give me a list of nodes."

ROSA will process your request and utilize its built-in tools to retrieve the information:

ROSA:

Invoking rosnode_list with {}

Here are the active nodes:

- /turtlebot3_core
- /turtlebot3_teleop
- /rviz
- /rosout
- /my_custom_node

Is there anything else you'd like to know?

Tip

ROSA can easily provide information from multiple sources, all from a single query.

Try compound queries like: "Give me a list of nodes, topics, services, parameters, and log files."

Using Filters and Namespaces

Have you ever tried to get a list of ROS topics using the CLI, only to find the list is far too long? If so, you most likely use a tool like grep to limit the output based on a pattern. ROSA can perform these same filtering options using regular expressions. Simply specify the filters you want to apply using natural language, and ROSA will automatically apply the filters for you. For example:

User: "Show me all of the camera topics."

ROSA will understand that you only want to see topics with the word "camera" in it.

ROSA:

Invoking ros2_topic_list with {"pattern": ".*camera.*"}

Sure, here are the camera topics:

- /camera_front/rgb
- /camera_front/info
- /camera_back/rgb
- /camera_back/info

Would you like to know more about these topics?

Tip

You can also specify a specific namespace.

Try constrained queries like "Show me all of the services in the /localization namespace."

Getting Detailed Information

A good starting point for debugging is to get a list of the relevant primitives (nodes, topics, etc.). But once we know which primitives we're interested in, we typically want to dig deeper to figure out why things are not working as expected. That's where the various info subcommands come in (e.g. rosnode info or ros2 node info), and this is where ROSA really shines! Unlike the standard CLI tools that only take single parameters at a time (e.g. ros2 node info /my_node), ROSA is able to take entire lists of parameters and return information all in one go. For instance:

User: "Give me a list of subscribers for IMU sensor topics."

ROSA will use a single tool call, passing in the list of topics, and receiving a dictionary that contains the requested information for each.

ROSA:

Invoking ros2_topic_list with `{"pattern": ".imu."}

Invoking ros2_topic_info with {"topics": ['/imu/center', '/imu/head']}

Here is a list of subscribers for the camera RGB topics:

- **/imu/center**
  - **Subscribers:**
    - /vio
- **/imu/head**
  - **Subscribers:**
    - /vio
    - /human_interface

Note

There are two things we'd like to point out in this example:

1. Notice how ROSA first retrieved the list of imu topics before trying to get the list of subscribers. This is expected behavior and highlights the adaptability and context awareness of ROSA.
2. The ros2_topic_info tool normally returns additional information like publishers and message type. However, since we asked only for the subscribers, ROSA omitted the other details.

You can ask for detailed information about nodes, topics, services, parameters, and more.

Tip

You can request more detailed information like the message type of a topic, and also request detailed information of the message type as well.

Try context-dependent queries like "Give me info on the /camera_front/rgb topic, and include details about its message type."

Chaining Tools

One of ROSA's greatest strengths is its ability to chain multiple tool calls in sequence to fulfill complex requests. This allows you to gather comprehensive information without manually executing multiple commands.

Suppose you want to use information from a log file to diagnose a problem with dataflow between two nodes. The interaction might look this like:

User: "I noticed the /slam node is not working. Can you check the log files and investigate any errors?"

This is a rather complex query, and there's no telling if the log files will be contain any information about the error. Still, ROSA is well equipped with the right tools for the job. The following chain of tool invocations may ensue:

ROSA:

Invoking roslog_list with {}

Invoking roslog_read with {"log_filename": "slam_node.log", "num_lines": 50, "level_filter": "ERROR"}

Invoking rostopic_list with {"pattern": ".*slam.*"}

Invoking rostopic_info with {"topics": ["/point_cloud"]}

Invoking rostopic_echo with {"topic": "/point_cloud", "delay": 1.0, "timeout": 10.0}

Invoking roslog_read with {"log_filename": "lidar_node.log", "num_lines": 50}

It appears that the `/slam` node is expecting data from the `/lidar` node over the `/point_cloud` topic. 
However, no data was published to this topic in the last 10 seconds.

The log file for the `/lidar` node shows an error message at 08:03:01 that indicates a potential power failure.

Suggestion: ensure that the LiDAR sensor is properly connected to the device, and that it is receiving sufficient power.

Please let me know if the problem persists.

Important

Notice that in this interaction, ROSA happened to choose a sequence of tool calls that led to a reasonable suggestion for how to resolve the issue. In practice, it doesn't always work perfectly. There are many reasons why a chain of tool calls might not result in a successful outcome.

More importantly, it is possible for ROSA to get confused and attempt to carry out actions that bring it no closer to achieving a satisfactory solution. In such cases, ROSA will eventually reach an iteration limit (currently set to 30 tool calls). You can also prevent runaway tool calls by pressing ctrl+C if you think the agent is getting off track.

We highly recommend working collaboratively with ROSA to resolve issues. Rather than saying something analogous to "this is my problem, please fix it", you will assuredly get better results by trying to break the problem into smaller, more manageable pieces. For instance, "this is my problem, can you help me figure out it?", followed by "let's check the log files first", and so on.

Parallel Tool Calls

ROSA can execute multiple tools in parallel, significantly reducing the time required to gather information from various sources. This is particularly useful when you need a broad overview of your ROS system. For instance, if you want a snapshot of the current state of your ROS system, you can ask:

User: "Please give me diagnostics information about the battery, power consumption, and CPU usage."

Note that the data we're interested is inter-dependent. That is, we want to know at a specific moment in time the readings from each of the corresponding subsystems. We say this data should be "colocated in time," since taking measurements at different points in time may lead to errors in downstream tasks. In this case, ROSA will be able to execute tool calls in parallel to ensure the readings are close together in time.

ROSA:

Invoking rostopic_echo with {"topic": "/battery_status"} and rostopic_echo with {"topic": "/power_status"} and rostopic_echo with {"topic": "/cpu_status"}

Here is the diagnostic information:

* Battery Level: 78%
* Power Consumption: 21W
* CPU Utilization: 6.7% 

What else can I help you with?

Tip

In some cases, ROSA is unaware that certain tools should be called in sequence or in parallel. You can influence this behavior for specific tools in the following ways:

1. Specify that a specific tool could/should be used in parallel in the tool docstring
2. Tell ROSA to do things in parallel or in sequence in the query (e.g. "Call tools X, Y, Z in parallel")
3. Customize ROSA with RobotSystemPrompts that specify the circumstances under which ROSA should use parallel or sequential tool calls.

Common Scenarios

In this section, we'll explore some common debugging and troubleshooting scenarios and demonstrate how ROSA can assist in resolving them efficiently.

Note

The following scenarios may rely on the existence of tools that are not included with the standard ROSA distribution. For instance, the ability for ROSA to read specific configuration files is a work-in-progress and not released with the open-source version (yet). However, we are including these examples here to demonstrate not only the current capabilities of ROSA, but also possibilities for how it might evolve in the future.

If you have any suggestions for how to improve ROSA, or other common scenarios you think should be listed here, please open an issue or send an email to [email protected]

Scenario 1: Nodes Not Communicating

Problem

You're noticing that certain nodes are not communicating as expected. Messages aren't being received, or topics seem inactive.

Traditional Approach

Typically, you would manually check for mismatched topic names, incompatible message types, or network issues by running a series of commands like rostopic list, rostopic info, and rosnode info.

Using ROSA

With ROSA, you can diagnose the issue more efficiently.

User: "Why isn't the /my_custom_node receiving messages on the /scan topic?"

ROSA will:

1. Use rosnode_info to get information about /my_custom_node.
2. Use rostopic_info to check the /scan topic.
3. Cross-reference publishers and subscribers.

ROSA:

I found that /my_custom_node is subscribed to the /scan topic, which is of type sensor_msgs/LaserScan. The /scan topic is being published by /turtlebot3_core with the same message type. However, there is a mismatch in the expected message queue size for /my_custom_node, which is set to zero. You may need to adjust the queue size in your subscriber to ensure messages are received correctly.

Benefit

ROSA automates the diagnostic process by chaining together multiple tool calls and analyzing the results, saving you time and effort.

Scenario 2: Node Crashes on Startup

Problem

A node crashes immediately after launch, and you're unsure why.

Traditional Approach

You might check logs, run the node with verbose output, or use debugging tools like gdb.

Using ROSA

Ask ROSA for assistance.

User: "Why does /my_faulty_node crash right after I launch it?"

ROSA will:

1. Attempt to retrieve logs related to /my_faulty_node.
2. Analyze error messages.
3. Suggest possible causes.

ROSA:

The node /my_faulty_node is crashing due to a missing dependency: the message type custom_msgs/ImportantData is not found. Please ensure that the custom_msgs package is properly built and sourced. You can try running `rospack find custom_msgs` to check if it's available.

Benefit

ROSA quickly identifies the root cause by accessing and interpreting log data, providing actionable advice.

Scenario 3: Parameters Not Set Correctly

Problem

A node isn't behaving as expected due to incorrect or missing parameters.

Traditional Approach

Manually inspect parameter settings using rosparam list and rosparam get, and check launch files for configuration errors.

Using ROSA

Consult ROSA to verify parameter settings.

User: "Are the parameters for /my_control_node set correctly?"

ROSA will:

1. Retrieve the parameters for /my_control_node.
2. Compare them against expected values.
3. Highlight discrepancies.

ROSA:

The parameters for /my_control_node are:

- **gain**: 1.0
- **setpoint**: 10.0
- **mode**: 'auto'

However, the 'mode' parameter is expected to be 'manual' based on your configuration files. This mismatch might be causing unexpected behavior.

Benefit

ROSA streamlines parameter verification, helping you quickly identify and correct misconfigurations.

Scenario 4: Transform (TF) Issues

Problem

Your robot's coordinate frames are not aligning correctly, causing issues with navigation or sensor data interpretation.

Traditional Approach

Use tools like rosrun tf view_frames and rosrun tf tf_echo to inspect the TF tree and individual transforms.

Using ROSA

Ask ROSA to analyze the TF frames.

User: "Is there any problem with the TF frames related to /base_link?"

ROSA will:

1. Retrieve the TF tree.
2. Analyze the transforms involving /base_link.
3. Identify any discrepancies or missing frames.

ROSA:

The transform from /odom to /base_link is not being published. This may be due to the robot_state_publisher node not running or not configured properly. Ensure that robot_state_publisher is active and that it has the correct URDF model.

Benefit

ROSA provides targeted analysis of TF issues, saving you from manually sifting through the TF tree.

Scenario 5: Service Calls Not Working

Problem

Service calls fail or do not receive a response.

Traditional Approach

You would use rosservice list, rosservice call, and rosservice info to debug service issues.

Using ROSA

Ask ROSA to investigate the issue.

User: "Why can't I call the service /reset_odometry?"

ROSA will:

1. Check if the service /reset_odometry is available.
2. Verify the node offering the service is running.
3. Analyze any errors.

ROSA:

The service /reset_odometry is not available. The node that provides this service, /turtlebot3_core, is not currently running. Please start the /turtlebot3_core node to make the service available.

Benefit

ROSA consolidates multiple diagnostic steps into a single interaction, quickly pointing you toward the solution.

Tips for Effective Interaction with ROSA

To get the most out of ROSA, consider the following tips:

• Be Specific in Your Queries: Clearly state what information you're seeking or the problem you're facing. This helps ROSA select the appropriate tools and provide precise answers.

  Example: Instead of saying "Something is wrong with my robot," try "Why isn't /my_navigation_node receiving laser scan data from /scan topic?"

• Leverage Natural Language: Don't hesitate to use natural language to describe complex tasks. ROSA is designed to understand and process such queries.

• Ask for Additional Details: If the initial response doesn't fully address your needs, ask follow-up questions or request more information.

• Verify Actions Before Execution: When performing actions that could affect your system's state, confirm with ROSA before proceeding.

Safety Considerations

While ROSA is a powerful tool, it's essential to use it responsibly:

• Validate Critical Actions: Before executing commands that modify your system or robot, ensure that the actions are safe and won't cause unintended consequences.

• Implement Constraints: Customize ROSA's behavior by adding safety checks and constraints, especially when controlling physical robots.

• Protect Sensitive Information: Use ROSA's blacklist feature to prevent access to sensitive topics, nodes, or services.

Conclusion

ROSA represents a significant advancement in how developers can interact with ROS systems. By harnessing AI and custom tools, it streamlines the debugging and troubleshooting process, saving time and reducing errors. Whether you're diagnosing communication issues between nodes or verifying system configurations, ROSA provides an efficient and user-friendly approach to managing complex robotic systems.

By integrating ROSA into your workflow, you can enhance productivity, reduce development time, and focus more on innovation rather than routine tasks. As AI continues to evolve, tools like ROSA will become indispensable assets in the field of robotics.