STTR Phase I:3D Media Synchronization for Educational Tele-gaming

This Small Business Technology Transfer Phase I project aims to enable novel teleimmersive interfaces for digital gaming via the development of multi-site 3D multimedia integration and synchronization technologies. These interfaces will allow geographically distributed individuals to interact seamlessly with each other and learn in shared 3D virtual gaming environments. The focus of this project is to develop, evaluate, and transfer teleimmersive technologies that synchronize multiple correlated and spatial audio and 3D video streams in distributed multi-site games. These technologies are required to create interfaces that will bring together learners in a novel manner and potentially transform cultural education based on digital gaming. Current multimedia technologies only fully address the synchronization of pairs of audio and 2D video streams. The objectives of the proposed work are to (i) implement site-to-site 3D multi-audio/video streams synchronization solutions through rugged prototyping and experimental validation, (ii) identify and incorporate serious gaming requirements, usability and evaluation factors into the teleimmersive interface, and, (iii) design, develop and evaluate the extension of the synchronization capabilities for multi-site, 3D multi-video/audio streams. The project builds on research conducted at the University of Illinois. The broader impact/commercial potential of this project lies in that the project will enable the creation of a 4D teleimmersive interface that represents a paradigm shift in the manner participants can be engaged in serious gaming through real-time teleimmersion across distances. These new environments will promote cultural awareness and understanding by providing (i) real-time teleimmersive dialogue between gamers/learners separated by geography, (ii) simulated travel to other cultures, and (iii) the means for structured interactions with individuals and groups from other cultures. The proposed interface will allow participants (and any physical objects they wish to bring with them) to be immersed in a shared virtual space and interact freely with each other, easily bringing into the games the complexity of human behavior. The commercial potential lies in greatly reducing game development time and costs by replacing the programming of 3D spaces and avatar behavior with 3D live capture and recordings of individuals in virtual spaces while retaining maximum flexibility and authenticity. The lower game development costs will enable the deployment of many small individual learning-games that may otherwise simply be too costly to develop using current methods. The teleimmersive interfaces and their underlying synchronization technologies will also enable new social interactions and collaboration capabilities in the educational games that are of particular interest to girls and women.