Panoptic Guardian

Complex, chaotic environments are a common feature of the modern battlefield. Threats can exist at various elevations, ranges, and headings and can use civilian crowds to mask their movements and intentions. Maintaining situational awareness in these conditions can be overwhelming. Panoptic Guardian will provide additional robotic eyes to reduce risk to a vehicle and its crew by identifying threats requiring specific attention. Panoptic Guardian employs a modular processing architecture and hierarchical inference scheme to achieve real-time detection, classification, and tracking of military targets. It will operate in complex, cluttered environments with moving people and vehicles, both day and night. In contrast to existing systems, Panoptic Guardian will identify objects that not only look like but also move and behave like threats, decoupling target motion from vehicle ego-motion to improve precision and recall. It will take video from multiple vehicle-mounted thermal cameras, including pan–tilt–zoom (PTZ) cameras, and generate georeferenced threat alerts by combining INS/GPS data with computer-vision-based depth estimation. The user can also manually identify targets for further interrogation and tracking with the PTZ camera. Automatic re-calibration routines using natural image features will ensure robustness against incidental deflections of camera-to-vehicle mounting due to the harsh environment.