Real-Time Localization Framework for Autonomous Basketball Robots

AI summary

Overview

Research area: Robotics — vision-based self-localization for autonomous mobile robots, applied to a competitive basketball-playing robot.

Technical level: Intermediate. The paper assumes familiarity with neural network basics, camera image processing, and robotics simulation tooling (Gazebo, ROS 2), but the core idea — map floor markings to a position with a small neural network — is explainable to a general technical reader.

Scope: The paper proposes and evaluates a hybrid localization pipeline that uses only a monocular camera view of the basketball court floor to predict a robot's x and y position, and compares the resulting lightweight feedforward network against MobileNetV2 and EfficientNet-B0 on an NVIDIA Jetson Orin Nano.

What This Paper Is About

For the Robocon 2025 "Robot Basketball" competition, two robots had to play basketball on a 15 m × 8 m court with baskets at 2.4 m height, moving quickly and shooting from many positions. The robots therefore needed fast, reliable knowledge of where they were on the court at all times. The paper's goal is a localization system that determines the robot's position using only visual information from the white field lines on the court floor, without relying on encoders, IMUs, or depth sensors.

Key Contributions

1. A two-step localization pipeline that preprocesses a monocular camera image (RGB to HSV conversion, masking white regions with two HSV ranges, and a radial-scan downsampling routine) before feeding it to a neural network.
2. A lightweight feedforward neural network — a flattening layer plus four fully connected layers (1024, 256, 64, and a 2-dimensional output) — that predicts the robot's x and y coordinates and is deliberately kept simple to reduce inference time.
3. A simulation-based dataset of 6,283 images with synchronized position labels, collected in a Gazebo digital twin of the Robocon 2025 arena using ROS 2 and a TimeSynchronizer.
4. A comparison against two edge-oriented baseline models, MobileNetV2 and EfficientNet-B0, on physical hardware (NVIDIA Jetson Orin Nano) using 100 samples, plus an explainability analysis with Integrated Gradients.

Main Findings

• Simulation dataset: 6,283 images were captured in Gazebo and split into training and test sets at a 0.9 to 0.1 ratio, with care taken to cover every part of the arena.
• Training setup: The model was trained for 15 epochs with the Adam optimizer, an initial learning rate of 10⁻⁴, MSE cost, and batch size 8.
• Loss versus iterations: For the x loss, a Pearson correlation test gave r = −0.32, df = 7, t ≈ −0.89, and p = 0.40 — above the 0.05 significance level, so no statistically significant relationship. For the y loss, r = −0.67 and t ≈ −2.39 with p = 0.01, below 0.05, indicating a statistically significant negative correlation where y loss tends to decrease as iterations increase.
• Reported per-iteration losses (Table II): x losses ranged from 0.1375 m (15 iterations) to 0.1639 m (25 iterations), and y losses from 0.1519 m (200 iterations) to 0.1950 m (15 iterations), across 9 listed data pairs. The final listed row gives 0.1463 (x) and 0.1506 (y) without an iteration number in the extracted text.
• Model comparison (Table III, 100 samples on physical hardware): The feedforward network achieved the lowest MSE (0.040), the fastest inference (3.283 ms), the highest RAM usage (1.3 GB), and a temperature of 43.62 °C. MobileNetV2 recorded MSE 3.380, 4.213 ms, 387 MB, 43.69 °C. EfficientNet-B0 recorded MSE 3.778, 4.373 ms, 323 MB, 43.97 °C.
• Accuracy stated in the conclusion: The authors report an average prediction error of approximately 0.06 meters.
• Explainability: Integrated Gradients attributions often concentrated on regions containing field lines and other spatial cues, suggesting the model relies heavily on geometric features.
• Failed prior approaches: An odometry-based method (wheel encoders, IMU, TF-Luna LiDAR) suffered from the sensor's limited effective range of approximately 3 to 4 meters, suboptimal accuracy in that range, and no clearly distinguishable surface on the basket. A YOLOv8n basket detector with manually selected distance zones was coarse and pilot-dependent. A YOLOv8 plus MiDaS monocular depth pipeline required manual ground-truth measurements at three positions to fit a linear regression for converting relative depth to metric distance.

Methodology in Plain English

The team built a digital twin of their robot and a replica of the Robocon 2025 arena in the Gazebo simulator, controlled through ROS 2. They teleoperated the robot around the arena with a joystick (using the joy package in ROS 2 Humble), capturing camera images alongside the robot's true x and y coordinates, synchronized using a TimeSynchronizer.

For localization, each incoming 640 × 480 × 3 image is first converted from RGB to HSV, and white areas — the court's field lines — are masked out using two predefined HSV ranges. The masked image then undergoes a radial scan: rays are cast from a point at the bottom center of the image outward at angles from 0 to 180 degrees in steps of 2, and a pixel is marked when the scan transitions out of a white (255) value. This turns the image into a compact representation of where the floor lines are.

The first 200 rows of pixels are discarded as uninformative, leaving a 640 × 280 × 1 image, which is flattened into a 179,200-element vector. This vector passes through four fully connected layers (1024, 256, 64, and 2 units), with ReLU activations after the first three, and the final 2-element output is the predicted x and y position. The authors chose this simple design because the input images are simple and inference speed matters.

For comparison, the same task was attempted with MobileNetV2 and EfficientNet-B0, two efficient convolutional architectures suited to edge devices. All latency and resource measurements were taken on an NVIDIA Jetson Orin Nano (1024-core Ampere GPU with 32 Tensor Cores, 6-core Arm Cortex-A78AE CPU, 8 GB RAM) with batch size 1; the models were trained on an Apple MPS backend and exported to ONNX for inference on the Jetson.

Why This Matters

The paper shows that a very small, purpose-built neural network can localize a robot on a marked indoor surface from a single ordinary monocular camera, beating much larger pretrained architectures on both error and latency in this specific setting. Because it avoids omnidirectional or fisheye optics, it sidesteps the distortion-correction pipelines that dominate the related work the authors review, making the approach portable to standard camera modules.

Real-world applications:

• Warehouse and factory autonomous mobile robots that navigate using painted floor lanes and intersection markers.
• Service and hospitality robots operating in indoor spaces with taped or painted guidance markings.
• Educational and competition robotics, including robot sports leagues, where small embedded compute budgets are the norm.
• Any indoor robot that needs a cheap fallback localization layer when odometry drifts or GPS is unavailable.

Industry relevance: The model comparison directly targets the edge-deployment trade-off that matters commercially — the authors' feedforward model is fastest and most accurate here (3.283 ms, MSE 0.040) but costs 1.3 GB of RAM, while MobileNetV2 (387 MB) and EfficientNet-B0 (323 MB) are far lighter with higher MSE. That is a concrete deployment decision for anyone shipping vision-based localization on embedded hardware like the Jetson Orin Nano.

Future Directions

• Fuse the vision system with IMU, wheel odometry, or LiDAR using filtering techniques such as Kalman or particle filters to produce more robust and reliable localization.
• Explore other neural network architectures to improve prediction accuracy.
• Optimize the model for deployment on embedded platforms with limited computing resources through pruning, quantization, or hardware acceleration.
• Open question raised by the paper's own numbers: Table II reports per-iteration losses in the range of roughly 0.14 to 0.19 meters, while the conclusion reports an average prediction error of approximately 0.06 meters; the paper does not reconcile these two figures, and it also does not report validation on a physical court rather than the Gazebo digital twin.

Target Audience

Readers who will benefit most are robotics students and competition teams (Robocon and similar leagues), embedded vision engineers choosing between lightweight model architectures, and researchers working on vision-based indoor self-localization who want a concrete data point on how far a simple fully connected network can go when the input representation is engineered carefully. The paper is also useful to practitioners evaluating deep learning models under real latency, memory, and thermal constraints on a Jetson-class device.

Authors’ abstract

Localization is a fundamental capability for autonomous robots, enabling them to operate effectively in dynamic environments. In Robocon 2025, accurate and reliable localization is crucial for improving shooting precision, avoiding collisions with other robots, and navigating the competition field efficiently. In this paper, we propose a hybrid localization algorithm that integrates classical techniques with learning based methods that rely solely on visual data from the court's floor to achieve self-localization on the basketball field.