Attention-Based Vision for Autonomous Vehicles

This small business innovation Phase I project aims to develop a robust architecture for an attention based visual system for autonomous vehicle applications. Such a system is a critical front-end component for highly cognitive systems that can recognize objects and extract scene semantics from high resolution, multimodal image streams. The proposed solution is based on a bottom-up and top-down visual attention mechanism coupled with a hierarchical object recognition framework. This Phase I project aims to develop a complete specification, at an algorithmic level, of the proposed visual attention framework to determine the feasibility and project the performance that could be realized in a Phase II prototype. In addition to the size, weight, and power constraints imposed by a general vehicle platform, autonomous vehicles impose that visual information be captured and processed with short latency to afford real-time decision making. As such our solution will employ an appropriate combination of software and custom hardware accelerators to realize the latency and throughput requirements of an autonomous vehicle.