Visual tracking with spiking neural networks and its application to aerosol event tracking

Aerosol plumes, such as intense smoke or dust, affect population health by causing pollu-tion peaks. These plumes are responsible for millions of deaths per year and are mainly due to human activities. Therefore, we need an efficient and robust automatic plume track-ing system that enables faster air quality prediction and supports environmental disaster response (e.g., volcanic eruptions or pollution peaks). To achieve this, satellites should be able to move their instruments directly to areas of interest to obtain the maximum amount of data from specific events. Furthermore, due to the limited energy resources on-

board satellites, such tracking models should consume as little energy as possible. Unlike standard artificial neural networks, Spiking Neural Networks (SNNs) consume very little energy on suitable hardware, making them good candidates for this task. However, no atmospheric plume datasets are available for Visual Object Tracking (VOT). In addition, deep learning-based VOT methods have never been applied to atmospheric plumes, let alone low-energy SNN trackers. This absence of trackers for such tasks raises the question of the capacity of a tracker to extract relevant optical signatures from aerosol plumes. To that end, we first introduce a new VOT dataset designed for atmospheric aerosol plume tracking. This dataset is based on the Copernicus Atmosphere Monitoring Service Euro-

pean Air Composition reanalysis 4 (CAMS EAC4), a multi-year aerosol dataset produced by the European Centre for Medium‑Range Weather Forecasts (ECMWF). This dataset focuses on sea salt and dust aerosol plumes. Then, inspired by the work of Bertinetto et al., we adapt the Fully-Convolutional Siamese Network (SiamFC) tracker to SNNs. We introduce a spiking SiamFC architecture, trained end-to-end with backpropagation and the surrogate gradient method. In this model, we introduce new spiking mechanisms: synchronised threshold adaptation, and a head-asymmetry mechanism that accumulates spikes in the instance branch. Our results show that our spiking SiamFC yields higher or similar performances to comparable SNN trackers, on generic object benchmarks. Fi-nally, we evaluate our spiking SiamFC model on our aerosol plume tracking dataset. Results show that our spiking tracker can learn plume features, paving the way for future developments in VOT applied to aerosol plumes.