Taxiing Operations via Gesture Understanding (TOPGUN)

Unmanned Aerial Systems (UAS) play an increasingly important role in many military scenarios. For the Navy to incorporate UASsuch as those currently being developed under the Unmanned Combat Air System Carrier Demonstration (UCAS-D) programinto its critical missions without impacting the current sortie rate, an efficient protocol is needed for handling UAS on aircraft carrier decks. We propose to design and develop a system for Taxiing OPerations via Gesture UNderstanding (TOPGUN). The fundamental premise of our approach is to use the same communication modality (i.e., gestures) for controlling UAS as for piloted aircraft. Our solution uses on-board cameras to track flight deck directors and to recognize their gestures without requiring modifications to operator equipment or to the carrier deck. The underlying software will build on state-of-the-art video-based human tracking, pose estimation, and gesture recognition technologies previously developed by our team to produce an efficient and reliable system for gesture-based control of UAS during carrier deck operations.