The Challenge
Educational and commercial micro-UAVs (like the DJI Ryze Tello) are lightweight and affordable, but they lack onboard computing power, LiDAR, and stereo cameras required for autonomous tracking.
Moreover, standard visual trackers quickly fail when the target turns around, crosses behind obstacles, or steps into crowded environments where multiple people intersect.
The Solution: An off-board architecture that decouples physical sensing and execution from compute-heavy artificial intelligence, combining human pose estimation with deep feature re-identification and anatomical depth estimation.
System Architecture
Perception & Pose Estimation (YOLOv8-Pose)
Extracts 17 skeletal keypoints per individual in real time at 30+ FPS. Rather than simple bounding boxes, full pose topology enables body orientation detection and gesture command decoding.
Visual Memory & Re-Identification (BoT-SORT + OSNet)
BoT-SORT tracks spatial-temporal velocity while OSNet extracts a 512-dimensional visual embedding of the target's clothing and appearance. When the pilot is occluded or walks behind obstacles, the system re-acquires the exact same target upon reappearance.
Hybrid Monocular Distance Estimation
Converts 2D pixel coordinates into real-world distance ($Z$) using human biometric invariant proportions. Transitions dynamically between facial width (close range) and biacromial shoulder span (mid range), with virtual nose projection when the pilot faces away.
Decoupled 3-Axis PID Flight Controller
Three independent PID control loops calculate real-time flight velocity commands: Yaw PID keeps target centered horizontally, Altitude PID tracks eye/shoulder height, and Pitch PID maintains target distance (1.8m safe distance).
Touchless Skeletal Flight Gestures
Operating the UAV without physical keyboards or transmitters by calculating geometric vectors between shoulders, elbows, and wrists:
- ASCEND: Right arm raised vertical (>150°) +0.4 m/s
- DESCEND: Right arm angled down (<30°) -0.4 m/s
- SWITCH TARGET: Both arms horizontal T-Pose Cycle ID
- EMERGENCY LAND: Arms crossed over chest (X) Auto-Land
Reverse / Back-Facing Tracking
Standard facial trackers fail when the user turns their back. This project introduces a Virtual Nose Formulation: when facial confidence drops below 0.3, the system computes the geometric midpoint between left and right shoulders, projecting an invariant tracking anchor.
Field Validation & Experimental Results
Indoor Arena
Controlled lighting and obstacle occlusion testing. Verified that BoT-SORT + OSNet maintained identity with 94.2% accuracy after 3-second full column occlusions.
Outdoor Field
Evaluated with wind gusts up to 12 km/h and direct sunlight shadows. Decoupled PID controllers compensated for outdoor drift while keeping distance within ±15 cm tolerance.
Latency Benchmark
End-to-end latency measured at ~78 ms (Wi-Fi streaming 35ms + YOLO/OSNet inference 28ms + PID calculation 2ms + UDP command 13ms), well within dynamic flight limits.