Multi-modal Performance Driven Sensing

To provide complementary data and enable the synergistic gain from the fusion of multiple sensing modalities, there have been recent efforts to incorporate multiple sensors on the same platform. This approach may require vastly different detector material and apertures (optical lenses and antennas) due to the physics involved. However, within a limited spectral range (e.g., visible through infrared electro-optical wavelengths) one can envision an integrated multi-modality sensor that does not violate the laws of physics and takes advantage of the ongoing advances in microelectromechanical systems (MEMS) and nanotechnology. The miniature nature of these devices allow concepts that can be integrated at or near the detector focal plane, thus leading to compact size and much simplified alignment techniques. On-device co-alignment addresses one of the limitations of multiple single-mode sensors, which is the inability to collect perfectly aligned imagery across modalities thereby limiting the possible fusion gain. In addition to gains possible from integrated multi-modality sensors, there is an opportunity for improved ISR performance from sensors designed to optimally adapt based on feedback from integrated real-time exploitation algorithms in a performance driven sensing paradigm. BENEFIT: The basic research conducted under this proposed effort will identify robust candidate technology that can then be further developed through additional research and engineering to provide a transformational integrated sensing capability to serve primarily Department of Defense ISR and SSA technology needs, but also could be adapted for civilian and homeland security applications.  The integrated capability for tracking objects of interest (persons, vehicles, etc.) would have clear benefits to homeland security, particularly for public events and transportation centers. The integrated multi-modality devices could lead to broader applications of traditional urban mapping and site monitoring technology and expand those commercial markets. In Phase I, the specific problem of multi-modal sensor design and feature-aided data exploitation is addressed. The Phase~II effort will culminate in the fabrication of sensor designs for model validation, but will also provide prototypes that can be used for marketing of these sensor technologies. The developmental sections of the exploitation software can be readily translated into a real-time system for direct use by the military and civilian reconnaissance communities. The solutions to these problems are of direct interest to the U.S. DoD as well as the Department of Homeland Security (DHS).  )