Adaptive multi-sensor wide area situational awareness system

ABSTRACT: Confronted by a vast quantity of data, presented piecemeal, sporadically and at varying levels of detail, the human analyst is often overwhelmed when trying to effectively monitor even medium-sized areas of interest. Offline, there is a wealth of data, resolution, and time to pick through and find activities of interest. Given a large amount of high resolution data, we can simulate situations where we only have low resolution data simply by down-sampling. Our approach to exploiting these data is to impute high level features from low level data by learning the association between low and high in the offline setting. Expert annotation of scenes, direct user input, and a priori knowledge of class structure may be available: we will bootstrap from this information by employing a recently developed form of semi-supervised learning that will also tap into the vast quantity of unlabeled data. At the core of our learning algorithm will be a robust multi-modal, multi-expert classifier. Fed into this classifier will be a novel, advanced activity representation derived from the data through interaction with expert knowledge. Our network management system will ultimately exploit these insights produced by this system at each stage to optimize network performance. BENEFIT: The core of what we develop will be software libraries for creating the advanced activity representation, imputing features and training the classification system; network management software with a user interface; and a well-documented API. This system will be capable off-the-shelf of linking in with existing sensor networks, but also provide the capability to the user to retrain on new data or add in new inputs/annotations/descriptions. We will sell these libraries and interface as a standalone product or as a plugin to already existing data management systems. In these cases, we would typically perform some custom engineering work to integrate the software into the client's specific platform and tailor the system for custom vehicle capabilities or requirements. Each component library of our system has value in itself: the low-to-high resolution feature imputation can be used in any application where the supply of high resolution data is limited; the multi-modal, multi-expert classifier does not require imputed features; and the advanced activity representation is derived separately from interaction of data and annotation/user-input and has application on its own in surveillance applications. Assuming successful completion of a 9-month Phase I and a 24 month Phase II, we would expect that initial sales of the final, full version of the system would commence approximately 39 months after Phase I award, including an additional 6 months for a final quality assurance revision and customer interaction. It is hard to predict the potential license income, but it is likely that a price-point of under $5K per seat could be reached (with appropriate price breaks as number of runtime licenses increases). The DoD market size is tied to the ultimate end market size for ground stations for activity monitoring within the Joint Services. However, given that sales to military and civilian defense organizations are often very slow in developing because of funding, political, and procedural issue, it is likely that a Phase III effort will ramp up over time, with reasonable expectation being in the low tens of units per year initially. We will pursue commercial markets such as security surveillance systems and targeted marketing. In the former, we expect a straightforward applicability of the product, though perhaps in a scaled-down form. In the latter, we expect the training interface to be essential to adoption because it allows for customization of the product to disparate input sources and classes. Our focus on human activity, in particular our use of annotations/descriptions to develop an advanced activity representation allows a more rapid adaptation of our product to targeted marketing in social networking.