Image-based Tracking and Resource Allocation Algorithms for Unmanned Aerial Vehicles

We propose to develop advanced algorithms and software to collaboratively track people and vehicles in an urban environment using video imagery (visible, IR) from a network of unmanned aerial vehicles (UAVs). The tracking and data fusion architecture will have one fusion node in a surveillance region, with a number of UAVs associated with the fusion node. We propose the multiple hypotheses tracking (MHT) algorithm for each UAV local tracker, as well as the fusion node. This will provide cost effective algorithm and software development and maintenance. The local UAV tracker will process target centroid pixel locations and feature measurements extracted from its video imagery. The video processor at each UAV will perform frame registration, change detection, target segmentation, target location measurement, and target feature measurement. The pixel location measurement in the image plane is a nonlinear function of the 3D target location in the global coordinate frame. This nonlinear mapping depends on the camera location, orientation, focal length, and intrinsic camera parameters. Thus the local UAV tracker will use a nonlinear filtering algorithm such as the extended Kalman filter (EKF) or unscented Kalman filter (UKF). The UAV tracker will also process track data as feedback received from the fusion node. The fusion node will process track data received from a number of local UAV MHT trackers. We shall explore a number of track-to-track association and fusion algorithms in Phase I, and recommend the best candidate track-to-track association and fusion algorithm for Phase II. Furthermore, we will address dynamic, collaborative UAV trajectory planning and camera pointing based on the Fisher information, which is a natural choice in the context of multi-platform tracking and fusion. Proof of concept for the selected architecture and algorithms will be demonstrated using simulated and real video data.