Subspace Tracking and Manifold Learning Based Heterogeneous Data Fusion for Unexpected Event Discovery

In many DoD intelligence, surveillance, and reconnaissance (ISR) scenarios, the data streams not only have a high dimensionality, but also belong to different sensing modalities. Current ISR surveillance typically employs stove-piped exploitation systems. Based on a single sensing modality, a stove-piped system senses, captures, and processes only one kind of target signal or signature. IFT proposes a subspace tracking and manifold learning based heterogeneous data fusion approach, for unexpected event discovery, from upstream sensor data. In the Phase I research, we have developed a prototype with four main components: (i) CCS and PETRELS based joint subspace tracking for linear modalities; (ii) joint sparsity support recovery for mixed or nonlinear modalities; (iii) joint manifold learning framework for nonlinear or mixed modalities with training data; and (iv) a multi-viewer, multi-layer visualization and prototype with GUI. We have tested our prototype on AFRL datasets. Supportive and promising results have been obtained. In Phase II, we refine and expand the Phase I technologies to unify real-time upstreaming data discovery, pattern of life, and anomaly detection, as well as diffusion based distributed paradigm. We evaluate the performance of the developed executable prototypical system by using realistic metrics, real-world data, and operational constraints.