Computing a Semantic View of a Scene for Surveillance from Stereo and Discreet LIDAR

We propose an innovative semantic approach to surveillance that combines the relative strengths and weaknesses of the passive 2D (single camera), passive 3D (stereo), and active 3D (LIDAR) sensing technologies to optimize performance. Our technology uses the high accuracy of LIDAR selectively to complement a passive stereo system when the cost in computation, energy consumption, and risk of exposure can be outweighed by a significant improvement in the performance of a system operating in purely passive mode. In this combined mode of operation, the active sensor would be used only infrequently and only in a very small area, permitting a relatively fast scan of the selected area with lower risk of exposure and lower computational burden. We illustrate a scenario where a (passive) stereo vision system and an (active) LIDAR system cooperate to render a semantic view of the activities that occur in a busy street intersection with individual pedestrians and vehicles represented by icons, and labeled with markers that describe their trajectories and activities. To maximize stealth, the LIDAR is only used rarely and on small parts of the scene when a probabilistic analysis shows the benefits of the requested active sensing outweighing the expense and risk of exposure.