A Semantic Based Surveillance Video Indexing and Retrieval System

Large volume of data collected in real-time video surveillance poses a challenging task for retrieval and analysis. The ability to conduct content-based retrieval of surveillance image/video is critical for security surveillance systems to perform forensic analysis, to discover trends and patterns of threat activities, and to evaluate the system performance. The project objective is the development of an innovative Semantic-Based Video Indexing and Retrieval (SBVIR) system to help security professionals rapidly, effectively and intuitively search for intelligent information from large corpus of surveillance videos. Based on advanced classification techniques, SBVIR provides users with the opportunity to query video content using semantic concepts selected from a large dictionary of objects and activities, display the history linked to a given target/activity, and detect anomalies. The user can interact with the system to increase accuracy and relevance of retrieved data. In Phase I, the proposed concepts of semantic video indexing and retrieval have been proven and demonstrated through a working toolset in C++. In Phase II the algorithm suite will be refined and enhanced for the extended ontology of objects and events. Phase II will develop a complete prototype system and demonstrate forensic ability for surveillance systems in both maritime and land-based environments.