Following the Giants: Robots Track Whales in Real Time

by

Author: Denis Avetisyan

A new system combines autonomous drones, acoustic sensors, and machine learning to enable close-range observation of sperm whales in their natural environment.

The system localized a whale’s trajectory-interpolated from visual and depth data gathered via on-body tags-and autonomously maneuvered a hydrophone array to resolve directional ambiguity during a ten-minute simulation that incorporated acoustic angle-of-arrival (AOA) errors with a standard deviation of $5^{\circ}$, ultimately achieving rendezvous within 118 meters.
The system localized a whale’s trajectory-interpolated from visual and depth data gathered via on-body tags-and autonomously maneuvered a hydrophone array to resolve directional ambiguity during a ten-minute simulation that incorporated acoustic angle-of-arrival (AOA) errors with a standard deviation of $5^{\circ}$, ultimately achieving rendezvous within 118 meters.

This work details a real-time whale rendezvous system integrating acoustic and VHF sensing, autonomous UAV control, and reinforcement learning for improved sperm whale tracking and data collection.

Tracking marine life in dynamic ocean environments presents significant challenges for long-term ecological study. This is addressed in ‘Real-time Remote Tracking and Autonomous Planning for Whale Rendezvous using Robots’, which details an integrated system for autonomously approaching and tracking sperm whales at sea. By combining acoustic and VHF sensing with model-based reinforcement learning, the authors demonstrate real-time whale rendezvous capable of enabling close-range data collection. Could this approach revolutionize our ability to study the behavior of marine mammals in their natural habitat and inform conservation efforts?

The Rhythms of Existence: Tracking Cetacean Movements

The pursuit of precise sperm whale tracking underpins a wealth of critical marine research and conservation initiatives. These intelligent cetaceans exhibit complex social structures, deep-sea foraging behaviors, and vast migratory patterns – all demanding detailed observation for comprehensive understanding. Reliable tracking data informs studies of whale communication, predator-prey interactions, and the impact of environmental stressors like noise pollution and climate change. Furthermore, accurate population monitoring, facilitated by tracking technology, is essential for implementing effective conservation strategies, protecting critical habitats, and mitigating threats to this vulnerable species, ultimately ensuring the long-term health of ocean ecosystems.

Efforts to pinpoint sperm whale locations using sound face significant hurdles due to the very nature of underwater acoustics and the ocean itself. Sperm whales produce clicks for echolocation, but determining a precise direction from these sounds is often ambiguous; a single hydrophone struggles to differentiate between a whale directly ahead versus one at an angle. Furthermore, the marine environment introduces complexities like sound reflection and refraction off the seafloor, thermoclines, and even other whales, creating “ghost” signals or distorting the true source. These factors combine to make accurate acoustic tracking challenging, requiring sophisticated signal processing and multiple hydrophone arrays to mitigate errors and achieve reliable localization – a process that remains computationally intensive and prone to inaccuracies in dynamic ocean conditions.

The inability to consistently track sperm whales presents a significant obstacle to both population assessment and behavioral ecology. Without reliable positional data, determining population size, distribution, and migratory patterns remains imprecise, complicating conservation strategies. More critically, understanding the nuances of sperm whale social structure, foraging behavior, and communication-all vital to their survival-is severely hampered. Current tracking limitations obscure crucial details about dive depths, prey selection, and the impact of environmental stressors, ultimately hindering efforts to protect this iconic species and its role within the marine ecosystem. Consequently, a clearer picture of sperm whale life history is essential for informed management and ensuring their long-term viability.

This system integrates hydrophone arrays, VHF sensing payloads on a catamaran and UAV, and a whale dive model with a model-based reinforcement learning approach to estimate whale locations and control UAVs via a local network and distributed key-value store.
This system integrates hydrophone arrays, VHF sensing payloads on a catamaran and UAV, and a whale dive model with a model-based reinforcement learning approach to estimate whale locations and control UAVs via a local network and distributed key-value store.

A Convergence of Sensing: Mapping the Submerged World

A hydrophone array is deployed to capture underwater vocalizations produced by sperm whales. This array consists of multiple acoustic sensors and is towed behind a catamaran platform, enabling broad survey coverage while minimizing vessel noise interference. The physical arrangement of hydrophones within the array is critical for beamforming and subsequent direction-finding algorithms. Specifically, the array geometry allows for the estimation of the direction-of-arrival (DOA) of sperm whale clicks and codas, forming the foundation for whale localization efforts. Data acquisition parameters, including sampling rate and array depth, are optimized to effectively capture the frequency range and signal characteristics of sperm whale vocalizations, typically between 100 Hz and 25 kHz.

Acoustic Angle-of-Arrival (AOA) estimation is performed on the hydrophone array data using PAMGuard, a software suite designed for passive acoustic monitoring. This process determines the direction from which underwater sounds originate, providing initial cues regarding the sperm whale’s location. PAMGuard utilizes algorithms to analyze the time difference of arrival of acoustic signals at each hydrophone in the array, calculating the angle relative to the array’s heading. While initial AOA estimates are subject to inaccuracies due to multipath propagation and noise, they serve as critical input for subsequent localization refinements and provide a broad search area for whale detection.

VHF sensing is integrated to detect whale surfacing events, supplementing acoustic localization data. This system utilizes a custom-developed sensing payload capable of detecting surface events at a range of up to 2 km. Detection of surfacing provides crucial temporal context, confirming the presence of a whale at a specific location and time, and directly enhances localization accuracy by validating or correcting acoustic bearings derived from the hydrophone array. The VHF system functions independently of acoustic data, providing redundant detection and improving the robustness of the overall tracking system, particularly in challenging acoustic environments.

Signal Phase-Based Angle-of-Arrival (AOA) techniques enhance the precision of acoustic bearing estimates by analyzing the phase differences of signals received across multiple hydrophones in the array. This method exploits the wave nature of sound; differences in the arrival time, and therefore phase, of a signal at each hydrophone are directly related to the source’s direction. By precisely measuring these phase differences and applying trigonometric calculations, the AOA can be determined with higher resolution than methods relying solely on time-difference-of-arrival. This refinement is critical for minimizing localization error, particularly in multi-path environments where signals may reflect off the seafloor or surface, creating spurious bearings. The technique effectively reduces the angular uncertainty, leading to more accurate whale localization estimates.

A catamaran-based system, utilizing both acoustic and VHF signal detection alongside a UAV for autonomous navigation, successfully locates and rendezvous with surfaced sperm whales in a fielded deployment in Dominica.
A catamaran-based system, utilizing both acoustic and VHF signal detection alongside a UAV for autonomous navigation, successfully locates and rendezvous with surfaced sperm whales in a fielded deployment in Dominica.

Navigating Uncertainty: Probabilistic Estimation and Aerial Convergence

A Particle Filter is utilized to estimate whale location by integrating data from both acoustic and VHF sources. This filter operates by maintaining a set of weighted particles, each representing a possible whale position in a defined state space. Acoustic data, such as Time Difference of Arrival (TDOA) measurements, and VHF data, including Angle of Arrival (AOA), are used to update the weights of these particles based on their likelihood of representing the true whale location. The resulting distribution of weighted particles, known as the belief location, provides a probabilistic representation of the whale’s position, effectively quantifying the uncertainty inherent in the localization process. This probabilistic estimate, rather than a single point estimate, is then used for subsequent control actions.

The Particle Filter addresses directional ambiguity in whale localization by integrating data from multiple sources – acoustic and VHF signals – and representing the whale’s possible locations as a probability distribution comprised of weighted particles. Ambiguity arises because a single Angle of Arrival (AOA) measurement doesn’t uniquely define a location; a particle filter resolves this by maintaining a set of hypotheses (particles), each representing a possible whale location. Each particle’s weight is updated based on its consistency with incoming measurements; particles aligned with multiple, corroborating data sources – such as both acoustic and VHF detections – receive higher weights, while those inconsistent with multiple sources are downweighted or eliminated. This probabilistic approach effectively reduces uncertainty and provides a more reliable estimate of the whale’s position despite inherent ambiguities in individual measurements.

The belief location, initially estimated via a Particle Filter incorporating acoustic and VHF data, is further refined using a Gaussian Mixture Model (GMM). This model addresses the scenario where Angle of Arrival (AOA) measurements originate from multiple whale groups, potentially leading to ambiguous localization. The GMM clusters AOA measurements based on their statistical characteristics, effectively modeling the probability distribution of whale positions contributed by each group. By representing the data as a weighted sum of Gaussian distributions – each representing a potential whale group – the GMM provides a more accurate and nuanced probabilistic estimate of the whale’s location compared to a single, uniform distribution.

The probabilistic estimate, derived from the fusion of acoustic and VHF data via a Particle Filter and refined by a Gaussian Mixture Model, directly informs autonomous navigation of the Unmanned Aerial Vehicle (UAV). This estimate is utilized as the target location for the UAV’s path planning algorithms, enabling it to navigate towards the estimated whale position without requiring manual control. Field trials demonstrated successful rendezvous within a 200-meter radius of surfaced whales, validating the efficacy of this probabilistic estimation approach for autonomous UAV control in whale tracking applications.

Autonomous aerial rendezvous with whale groups in Dominica successfully identified vocalizing (triangles) and non-vocalizing (stars) groups through acoustic angle of arrival estimation and onboard visual confirmation.
Autonomous aerial rendezvous with whale groups in Dominica successfully identified vocalizing (triangles) and non-vocalizing (stars) groups through acoustic angle of arrival estimation and onboard visual confirmation.

Echoes of Progress: Validation and Future Trajectories

Rigorous testing demonstrated the system’s high performance in simulated environments. Utilizing the DSWP Dataset and Software-In-The-Loop (SITL) simulations, the unmanned aerial vehicle (UAV) consistently achieved a 98.3% success rate in rendezvous maneuvers with sperm whales. These simulations, mirroring real-world flight conditions, also revealed an average flight time of 15 minutes was sufficient to complete the rendezvous procedure. This high degree of accuracy and efficiency, validated through comprehensive simulations, establishes a strong foundation for deploying the system in practical field studies and highlights its potential for reliable whale tracking and data collection.

The system’s capacity to successfully rendezvous with sperm whales is notably improved through the incorporation of a predictive Whale Dive Model. This model analyzes patterns in whale diving behavior – specifically, dive duration and depth – to forecast surfacing events with increased accuracy. By anticipating when and where a whale is likely to reappear, the unmanned aerial vehicle (UAV) can proactively adjust its flight path, reducing search time and maximizing the probability of a successful encounter. This predictive capability is crucial, as sperm whales spend a significant portion of their time submerged, making encounters during brief surfacing periods challenging; the model effectively narrows the window of opportunity, boosting rendezvous success rates and providing researchers with more consistent data collection opportunities.

The developed system represents a substantial advancement in sperm whale tracking methodologies, moving beyond traditional, often labor-intensive, observational techniques. By automating the rendezvous process with high accuracy, researchers gain access to a significantly increased volume of data regarding whale behavior, social structures, and migratory patterns. This enhanced data collection capability unlocks new avenues for ecological research, potentially revealing critical insights into sperm whale communication, foraging strategies, and the impacts of environmental changes on this vulnerable species. Furthermore, the efficiency of the automated tracking system allows for long-term monitoring initiatives, providing a more comprehensive understanding of whale populations and informing effective conservation efforts.

Continued development centers on augmenting the unmanned aerial vehicle’s perceptual capabilities through an expanded sensor suite, incorporating modalities beyond the current system to achieve a more robust understanding of the marine environment. Simultaneously, researchers are devising adaptive control algorithms designed to interpret and respond to the complexities of ocean conditions – accounting for variables like sea state, currents, and atmospheric disturbances – and the nuanced, often unpredictable, behaviors exhibited by sperm whales. This iterative process of sensor integration and algorithmic refinement promises to yield a system capable of maintaining reliable tracking and rendezvous success even in challenging conditions, ultimately facilitating more detailed and ecologically valuable data collection.

Real-time detection of whale surfacing events is achievable using VHF signal data correlated with tag depth, even during dives exceeding 800 meters.
Real-time detection of whale surfacing events is achievable using VHF signal data correlated with tag depth, even during dives exceeding 800 meters.

The pursuit of real-time tracking, as detailed in this study, reveals a fundamental truth about complex systems. The integration of acoustic and VHF sensing, alongside autonomous UAV control, isn’t merely about locating sperm whales; it’s about adapting to a constantly shifting environment. This resonates with the understanding that all architectures live a life, and the system presented here demonstrates a capacity for graceful aging through continuous learning and adaptation. Ada Lovelace observed, “The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform.” The research embodies this sentiment-the robots don’t independently find the whales, but execute the precisely ordered algorithms enabling rendezvous, mirroring the machine’s reliance on human instruction and highlighting the iterative nature of improvement within a defined system.

What Lies Ahead?

The pursuit of autonomous proximity to complex marine life-as demonstrated by this work-is less about achieving a technological pinnacle and more about acknowledging the inevitable entropy of any tracking system. Versioning algorithms for localization-acoustic and VHF combined-are, at their core, a form of memory, striving to retain signal fidelity against the relentless current of noise. The system’s performance, though promising, merely delays the eventual divergence between estimated and actual whale position-a divergence inherent in all observation.

Future iterations will undoubtedly focus on sensor fusion beyond the current modalities. However, the true challenge isn’t simply adding layers of data, but grappling with the fundamental limitations of inference. Each refinement represents a localized victory against decay, but the arrow of time always points toward refactoring. A robust solution will necessitate not just improved algorithms, but a deeper understanding of the whales’ own agency-their capacity to evade tracking, forcing a continuous recalibration of the autonomous planning.

The ultimate metric isn’t range or duration of contact, but the system’s capacity to gracefully degrade. To build a truly resilient system is to accept that complete knowledge is an illusion, and to engineer for uncertainty-a system that doesn’t merely track a whale, but anticipates its departure from the model.

Original article: https://arxiv.org/pdf/2512.05808.pdf

Contact the author: https://www.linkedin.com/in/avetisyan/

See also:

2025-12-08 20:58