As an engineer analyzing robot logs, you’re likely asking questions such as:
Have we encountered this type of event before?
How often does this issue occur?
Can we identify similar patterns in past logs?
How can we set up alerts so we know if it happens again?
These questions are critical for diagnosing system behaviors, anticipating failures, and enhancing future performance.
However, answering them presents two key challenges:
(1) Data volume and format: Robots generate immense amounts of data—sometimes tens or hundreds of gigabytes per mission—stored in specialized formats like ROS bags or MCAP files. Parsing these logs to extract relevant segments can be labor-intensive, often requiring custom scripts or manual inspection. The complexity compounds when dealing with fleets of robots, accumulating data across hundreds of missions or runs.
(2) Data diversity and pattern complexity: Robot logs are inherently multimodal, containing sensor data such as images, point clouds, time-series signals, and textual logs. While simple issues can be flagged by looking for discrete values or conditions (e.g., vehicle_gps_lost = True), more complex events—such as repeated instability in drone flights or subtle gripper failures—demand more advanced methods.
At Roboto, our goal is to address challenge (1) by simplifying the process of retrieving task-specific data at scale, allowing roboticists to manage and analyze large datasets more efficiently. At the same time, we’re continuously expanding our tooling to help make challenge (2) easier as well—offering features that support engineers in identifying, classifying, and analyzing complex patterns in multimodal data.
In this post, we’ll walk through a practical use case where we use Roboto’s Python SDK to retrieve IMU data from drone racing logs and then apply similarity search to identify and classify repeated patterns in flight events.
Scenario: As a systems engineer, you're tasked with investigating hard landings in your drone's latest flight campaign. To do this, you need to generate landing videos from the drone’s forward-facing camera and extract the corresponding IMU signals. These will then be reviewed by the controls team to assess performance.
While this sounds straightforward, the challenge is that your ROS bags contain raw IMU and image data—there are no annotations or flags indicating hard landing events. Typically, this would require:
(1) Downloading dozens of raw flight logs, each several gigabytes in size.
(2) Manually reviewing flights in a visualizer to identify hard landings and record the timestamps.
(3) Writing custom scripts to extract images and IMU signals, then compiling them into videos.
(4) Uploading these videos to a shared storage system for team access.
This process can be painstaking, often taking hours or even days when dealing with a large number of flights.
Our Approach
(1) Single Annotation: Begin by manually reviewing one of the flights in Roboto’s web-based visualizer to identify and annotate a landing event. This involves adding an event to the relevant IMU topic, which will serve as a reference for future searches.
(2) Data Retrieval: Utilize Roboto’s SDK in a Python script (Jupyter notebook) to efficiently retrieve IMU signals from all relevant flights.
(3) Similarity Search: Apply a similarity search algorithm to the retrieved IMU signals to identify similar hard landings.
(4) Artifact Generation: Generate videos of the identified landing sequences, and share the notebook for further review.
By implementing this approach, you can streamline the entire process and save considerable time, enabling you to focus on the core analysis.
For this demo, we ingested 25 drone flights from UZH-FPV Drone Racing. The flights cover various drone racing scenes, both indoor and outdoor, using forward and downward-facing cameras.
To reproduce this demo: create your own account, then install the Python SDK and configure a personal access token. You can see the ingested flights in the public collection on Roboto and run the full notebook from our repository by following the README under:
https://github.com/roboto-ai/roboto-python-sdk/tree/main/examples
(1) Open a dataset in the visualizer. Create panels for the image topics, and plot the [x, y, z] values from /snappy_imu/linear_acceleration. Refer to our guide on Visualizing ROS Data.
(2) Identify the drone's landing by looking for spikes in the linear acceleration signals, indicating contact with the ground.
(3) Create an event around the landing, setting the Event Scope to "All Topics in this Panel." See our guide on Create Events on Data for more details.
The first cell in the notebook sets up the Roboto SDK by configuring the client, connecting to the account with the public Drone Racing logs, and initializing RobotoSearch to query the data.
import roboto
query_client = roboto.query.QueryClient(
owner_org_id="og_najtcyyee2qa" # Drone Racing (public)
)
roboto_search = roboto.RobotoSearch(query_client)
In the next cell, we’re going to do the following:
(1) Retrieve the Event: The event is loaded using roboto.Event. This event holds the specific data that we annotated earlier. It will be used as an input for the similarity search.
(2) Prepare Topics to Search: Use roboto_search.find_topics to search for topics named /snappy_imu with RoboQL syntax. The search space could be further constrained by including additional metadata (e.g. drone software version) and tags (e.g. outdoor)
(3) Extract Signal Data from Event: The linear_acceleration and angular_velocity data is extracted from the /snappy_imu topic event into a pandas dataframe; this will serve as the query signal.
(4) Find Similar Signals: With roboto.analytics.find_similar_signals, the topics prepared in (2) are efficiently searched for matches with the query signal.
import roboto.analytics
event = roboto.Event.from_id("ev_6funfjngoznn17x3")
topics_to_search = roboto_search.find_topics(
"topic.name = '/snappy_imu'"
)
query_signal = event.get_data_as_df(
message_paths_include=["linear_acceleration", "angular_velocity"]
)
matches = roboto.analytics.find_similar_signals(
query_signal,
topics_to_search,
max_matches_per_topic=1,
normalize=True
)
Extracting event-related data and finding patterns across matching topics is simple, with no need to manually process entire logs. Roboto handles the heavy lifting—from data retrieval to comparison—so you can stay focused on solving the problem.
Now let’s run some utility code for visualization and look at some of the results:
from match_visualization_utils import print_match_results
print_match_results(
matches[:5],
image_topic="/snappy_cam/stereo_l",
)
The first column displays the distance scores of the closest matches. The second column shows the plots of the matched IMU subsequences, while the third column presents the corresponding image sequences for each match. The first row, with a distance score of zero, represents the query signal. As hoped, the top matches correspond to hard landing sequences in other flights.
Find Hard Landings
How does find_similar_signals work?
The find_similar_signals search is powered by Mueen's Algorithm for Similarity Search (MASS), which uses Euclidean distance profiles to identify matching subsequences in time-series data. We apply this method across multiple signals, then combine the results to find the closest match. This approach has also proven effective for detecting other types of events. Let's explore a few more examples.
Find Calibration Sequences for Downward Facing Camera
In the example below, we're looking for IMU patterns which correspond to the calibration sequences before takeoff.
Find Left Turn Sequences
In the example below, we tried to find similar sequences for an aggressive left turn. The results qualitatively make sense.
Note that we only used angular_velocity to find turns, as it directly captures rotational motion. Dropping linear_acceleration makes sense because turns are primarily characterized by changes in angular velocity, while linear acceleration data may introduce noise unrelated to the rotational dynamics of the turn. It's generally a good idea to experiment with including different signals in the similarity search to better capture events.
Limitations
The approach using Euclidean distance profiles may struggle in a few cases. First, noise or signal distortions can cause the algorithm to misinterpret small variations as dissimilar, leading to missed matches. Second, temporal misalignment can prevent the algorithm from recognizing similar events that are slightly shifted in time, a problem better handled by methods like Dynamic Time Warping (DTW). Additionally, events with different semantic meanings—such as a takeoff versus manually lifting the drone - can have similar IMU signatures, potentially leading to false positives.
Conclusion
In this post, we showed how Roboto’s SDK empowers you to quickly extract the most relevant slices of your robotics data - like IMU signals with associated images - so you can dive straight into more advanced tasks, such as identifying interesting events using similarity search.
By simplifying the retrieval process for ROS data, we’ve demonstrated how you can streamline workflows that usually take hours into just minutes. We're excited to be adding more powerful techniques for pattern matching in robotics data, so stay tuned for new features and examples in the coming months. In the meantime, give it a try and share what you discover!
This demo and the accompanying open source notebook use data from UZH-FPV Drone Racing, published by the Robotics and Perception Group (RPG) at the University of Zurich.
