Perception-to-Pursuit: Track-Centric Temporal Reasoning for Open-World Drone Detection and Autonomous Chasing

AI summary

Overview

• Research area: Robotics / autonomous counter-drone systems, combining multi-object tracking, trajectory prediction, and pursuit control.
• Technical level: Intermediate (requires familiarity with transformers, tracking metrics, and basic kinematics, but the core argument is accessible).
• Scope: The paper proposes a track-centric temporal reasoning framework (Perception-to-Pursuit, P2P) that predicts drone trajectories which are not only accurate but physically reachable by an interceptor, and introduces a metric to measure that reachability.

What This Paper Is About

Existing drone tracking systems are optimized to predict where a drone will be, not to check whether an autonomous chaser could actually get there in time. The authors report that state-of-the-art tracking methods produce trajectories that are physically impossible to intercept 99.9% of the time. P2P addresses this by learning from sequences of compact motion features — velocity, acceleration, scale, and smoothness — so that predictions of future position are constrained by what an interceptor with realistic speed and acceleration limits can feasibly reach.

Key Contributions

1. Pursuit-aware temporal reasoning: A 12-frame causal transformer that reasons over sequences of 8-dimensional motion tokens to jointly predict drone identity, behavior, maneuver intent, and future trajectory. The approach is motion-centric rather than appearance-based, so it does not rely on prior object models.
2. The Intercept Success Rate (ISR) metric: A measure of the fraction of predicted trajectories that satisfy bang-bang optimal control reachability constraints for an interceptor with v_max = 15 m/s and a_max = 5 m/s² (critical distance d_c = 22.5 m).
3. Empirical validation on Anti-UAV-RGBT: 226 real drone sequences, with 28.12 pixel ADE and 0.597 ISR, reported as a 77% improvement in trajectory prediction and a 597× improvement in pursuit feasibility over tracking-only baselines, plus 100% drone classification accuracy.
4. Interpretable motion token design: An 8-D representation (position, velocity, acceleration, scale, smoothness) replacing raw pixel-level bounding box input, which the authors report enables real-time processing at 323 FPS.

Main Findings

• Trajectory accuracy: P2P achieves 28.12 pixel Average Displacement Error (ADE) and 41.14 pixel Final Displacement Error (FDE) on the Anti-UAV-RGBT test set of 8,092 examples, versus 122.83 ADE and 53.24 FDE for naive velocity extrapolation and 122.45 ADE / 52.53 FDE for tracking-only.
• Pursuit feasibility: P2P reaches 0.597 ISR, meaning roughly 60% of predicted trajectories permit a kinematically feasible intercept plan. Tracking-only and naive velocity baselines both score 0.001 ISR, meaning 99.9% of their predictions yield impossible pursuit plans. The frame-based baseline scores 1.000 ISR but 0.000 classification accuracy, since it assumes a stationary target at the current position.
• Open-world discrimination: The motion-based model reaches 100% drone classification accuracy, while the appearance-agnostic baselines score 0.000.
• Ablations: Removing multi-task learning gives 35.4 ADE / 0.512 ISR / 0.987 accuracy; removing temporal attention gives 42.1 ADE / 0.423 ISR / 0.956 accuracy; removing acceleration features gives 31.8 ADE / 0.541 ISR / 0.998 accuracy; a window of W = 6 gives 33.2 ADE / 0.529 ISR / 0.991 accuracy; a window of W = 18 gives 29.1 ADE / 0.584 ISR / 1.000 accuracy.
• Component importance: Removing self-attention degrades ADE by 50% and ISR by 29%; acceleration features contribute a 13% ADE improvement; W = 12 is reported as the balance point between context and efficiency, with W = 18 offering only marginal gains.
• Qualitative cases: In an evasive-maneuver example, P2P reaches ADE = 18.3 and ISR = 0.89, while the tracking-only baseline produces ADE = 156.2 and ISR = 0.003. In a rapid direction-change example, P2P reaches ADE = 31.2 and ISR = 0.78, versus ADE = 198.4 and ISR = 0.02 for the baseline.
• Failure modes: P2P struggles with sudden unpredictable maneuvers such as emergency stops, and with occlusions that break temporal continuity. The authors state these cases account for the 40% where ISR = 0.
• Runtime: On an NVIDIA T4 GPU, motion token extraction takes 0.8 ms, transformer inference 2.3 ms, for a 3.1 ms end-to-end total of 323 FPS.

Methodology in Plain English

The system takes a stream of tracked bounding boxes and converts each one into a small 8-number token describing where the drone is, how fast it is moving, how its speed is changing, how large it appears, and how smooth its recent path has been. Instead of feeding thousands of pixels to a network, the model works only on these compact motion tokens.

A transformer with causal masking — meaning each position can only look at earlier positions — reads a 12-frame window of these tokens and predicts 20 frames ahead. The same internal representation feeds four outputs: whether the object is a drone, which behavior category it exhibits (hover, loiter, approach, evade, or pass-by), a maneuver-aggressiveness intent score, and the future trajectory. All four tasks are trained together with a weighted loss combining binary cross-entropy for drone classification, categorical cross-entropy for behavior, mean squared error for intent, and Smooth L1 loss for trajectory.

The distinct step is the evaluation. For each predicted future position, the authors compute the minimum time an interceptor would need to travel that distance, using bang-bang optimal control that first accelerates at the maximum rate and then cruises at maximum speed. If that minimum time is within the prediction horizon, the prediction counts as feasible. Averaging this over all predictions gives the Intercept Success Rate. Training used the Anti-UAV-RGBT dataset (226 sequences, reported as 160 train and 67 validation, with 40,458 training examples generated by a sliding window with a step of 5 frames and an 80/20 train/validation split), an AdamW optimizer at a learning rate of 10⁻³ and weight decay of 10⁻⁴, batch size 128, 50 epochs, loss weights of 1.0, 1.0, 0.5, and 0.5, and gradient clipping at 1.0. The transformer uses d = 128, 4 layers, and 4 attention heads.

Why This Matters

Impact on research: The paper argues that prediction accuracy and prediction actionability are separate properties, and that a tracker can score well on error metrics while producing trajectories no physical agent could follow. The ISR formulation gives the field a way to score predictions against controller limits, and the 100% motion-only classification result suggests temporal motion patterns may generalize to novel objects where appearance-based models do not.

Real-world applications:

• Counter-drone defense at airports, stadiums, and critical infrastructure, where an interceptor must actually reach a hostile drone.
• Airspace security for public events and government facilities requiring autonomous monitoring and response.
• Autonomous inspection or escort tasks where a following vehicle must maintain a physically achievable path relative to a moving target.
• Safety monitoring systems that need to judge whether a detected intruder can be intercepted before it reaches a protected zone.

Industry relevance: The reported 323 FPS on an NVIDIA T4 GPU places the method within real-time budgets for deployed perception stacks, and the compact 8-D token representation lowers the compute cost relative to pixel-based pipelines. The authors note that full deployment would still require integration with perception systems, control loops, and hardware testing.

Future Directions

1. Cross-dataset validation: The authors state that the evaluation uses only Anti-UAV-RGBT, and suggest CST Anti-UAV and Drone-vs-Bird as datasets that would strengthen generalization claims.
2. Adaptive interceptor constraints: ISR currently assumes fixed v_max = 15 m/s and a_max = 5 m/s². The authors propose adaptive constraints based on interceptor type.
3. Multi-drone pursuit: The current framework handles a single target; extending it to multi-drone pursuit with coordination constraints is listed as important future work.
4. Real-world deployment: Moving from the 323 FPS benchmark to integrated perception, control loops, and hardware testing remains open, as does improving robustness to sudden maneuvers and occlusions that currently account for the 40% of cases with ISR = 0.

Target Audience

Robotics and autonomous systems researchers working on tracking, trajectory prediction, or pursuit and interception; engineers building counter-drone or airspace-security systems; and machine learning practitioners interested in multi-task temporal transformers with physically grounded evaluation metrics. Readers focused on pure detection accuracy or appearance-based recognition will find the motion-centric framing the paper's main point of departure.

Authors’ abstract

Autonomous drone pursuit requires not only detecting drones but also predicting their trajectories in a manner that enables kinematically feasible interception. Existing tracking methods optimize for prediction accuracy but ignore pursuit feasibility, resulting in trajectories that are physically impossible to intercept 99.9% of the time. We propose Perception-to-Pursuit (P2P), a track-centric temporal reasoning framework that bridges detection and actionable pursuit planning. Our method represents drone motion as compact 8-dimensional tokens capturing velocity, acceleration, scale, and smoothness, enabling a 12-frame causal transformer to reason about future behavior. We introduce the Intercept Success Rate (ISR) metric to measure pursuit feasibility under realistic interceptor constraints. Evaluated on the Anti-UAV-RGBT dataset with 226 real drone sequences, P2P achieves 28.12 pixel average displacement error and 0.597 ISR, representing a 77% improvement in trajectory prediction and 597x improvement in pursuit feasibility over tracking-only baselines, while maintaining perfect drone classification accuracy (100%). Our work demonstrates that temporal reasoning over motion patterns enables both accurate prediction and actionable pursuit planning.