Gesture Recognition on Horizon X3 Pi
Efficient gesture recognition and mobile-robot deployment (undergraduate thesis)
Undergraduate Thesis Edge AI YOLOv5s ROS2
Real-time gesture recognition. YOLOv5s on Horizon X3 Pi after optimization — 30 FPS detection with 0.36% mAP loss versus the GPU baseline.
Gesture-controlled mobile robot. Recognized gestures mapped to ROS2 motion commands driving the platform in real time.
Human tracking. Detection-driven person tracking module integrated with the mobile-robot motion-control node.
TL;DR
An undergraduate-thesis project on deploying efficient gesture recognition on the Horizon X3 Pi edge board and integrating it into a mobile-robot system. The work covers model selection, edge-side optimization, and ROS2 integration with simulation and real-robot validation.
Contributions
- Edge-deployable detector. Trained YOLOv5s on the HaGRID static-gesture dataset, reaching 86.60% mAP, and benchmarked against the YOLO and DETR families.
- Edge optimization. Optimized and deployed YOLOv5s on the Horizon X3 Pi, improving inference from 20 FPS to 30 FPS with only 0.36% mAP degradation.
- Robot integration. Built ROS2 motion-control nodes and integrated gesture recognition with human tracking on the lab’s mobile robot.
Method & Platform
The pipeline combines a YOLOv5s detector (HaGRID-trained) with Horizon-toolchain-based quantization and graph optimization for the X3 Pi BPU. Recognized gestures are mapped to discrete motion primitives published as ROS2 Twist messages; a parallel human-tracking module shares the detection backbone to drive the platform’s follow behavior.