Multi-modal Synthetic Sensor Data Generator with Real-World Environmental Effects and Sensor Physics

ABSTRACT: There is a need to develop a sufficient-fidelity simulation tool to create multi-sensor datasets for human activity detection and recognition. To address this need, Intelligent Automation Inc. (IAI) and Infoscitex Corporation (IST) propose a novel, integrated, sufficient-fidelity simulation tool called HumanView (HumV). The proposed software will leverage existing tools, integrate commercial, proprietary and COTS multi-physics simulation tools and material properties databases. The key elements of the proposed architecture are: The HumV Models module will be a data store that will contain 3D human models, 3D geometric models of environment, relevant sensor models (RF, LIDAR and Hyperspectral) and platform models (UAV, aircraft) on which the sensors reside. The HumV editor module will allow researchers to discover the available models, manage the configuration associated with a model using a drag-n-drop interface, and create specific activities for the model, and associate mathematical models where appropriate. The HumV simulator module will allow the user to simulate multiple scenarios that will generate the synthetic sensor data for analytics. The goal of the Phase I program is to demonstrate the feasibility of HumV.Specific focus areas will include developing the overall architecture, and developing key HumV components.; BENEFIT: Development of HumV will allow the military to generate realistic and physically sound sensor imagery data from virtual 3D scenes, expanding capabilities of ISR applications. The technology would also be useful in other arenas such as analytical technology development for homeland security applications. HumV would be beneficial to developers of first responder support equipment for operations such as search and rescue.Commercial application of HumV include the fast growing field of robotics. Just as LIDAR simulators are used by roboticists today to test their algorithms, HumV may be used by developers of self-driving cars (which use a variety of sensors) and any other Artificial Intelligence or robotic systems that interact with humans, enabling the development of algorithms for navigating in human environments.