ViRaCom: Integrating Vision and Radar for Multi-Modality Joint Sensing and Communication in Cooperative Driving Automation

In the evolving domain of Cooperative Driving Automation (CDA), the integration of vehicular and infrastructural systems is pivotal for transforming transportation, aiming to bolster safety, mobility, environmental sustainability, situational awareness, and traffic flow efficiency. Despite the advancement of CDA algorithms and the availability of communication technologies like Cellular Vehicle-to-Everything (C-V2X), the challenge of melding these technologies for real-time CDA persists, hindered by the limitations of standalone sensing and communication systems. Addressing this challenge, we introduce ViRaCom, a novel plug-and-play system that amalgamates RadCom—a joint radar and communication technology—with cost-effective cameras to facilitate robust and efficient cooperative multi-modality sensing and communication under various conditions, including adverse weather and low ambient lighting. While RadCom contributes to communication and radar-based object detection, its efficacy is constrained by low spatial resolution and environmental clutter. Conversely, vision-based sensing offers high resolution and rich semantic information, complementing radar's geometric data. This integration not only capitalizes on the mature radar and camera technologies prevalent in the automotive industry but also aligns with the objectives of the SBIR call, proposing a compact, integrated module for multi-modal perception.