SENSAI: digital twin for next-gen automotive sensors
Software modeling can prove the value of sensors before the hardware is introduced. Imec’s SENSAI program expands these models to next-gen infrared, lidar and radar sensors.
Do we want future vehicles to make intelligent decisions? Then we need to make their perception close to perfect. To achieve this, we have to fix the blind spots of current sensor systems. They need to map out the environment and all of its occupants in all weather and lighting conditions and achieve long-range detection, especially in front of the vehicle.
This requires a combination of sensor types and the use of many of these sensors around the vehicle: ultrasound, (infrared) cameras, radar, and lidar.
Such a robust sensor system is only economically and practically feasible if the individual sensors are compact, low-power and low-cost – on a level that current technologies do not yet attain.
Virtual validation
The good news is that advances in semiconductor technology are enabling a new generation of highly integrated sensors, such as quantum dot SWIR cameras, solid-state lidars and 140-GHz radar.
Car manufacturers and their suppliers are eager to capitalize on the promises of these technologies. However, the traditional process of evaluating these next-gen sensors is cumbersome and slow. They need to:
- Develop working prototypes for field tests.
- Integrate these prototypes into existing sensor systems.
- Capture and analyze huge amounts of data.
Virtual simulation using digital twin technology would allow to do this in a much more cost-efficient way. However, current models are not yet equipped to deal with sensors that are in the early stages of development.
Imec’s SENSAI program aims to expand digital twin technology to deal with next-gen sensor technologies through physics-based sensor modeling and by leveraging the power of AI.
Imec’s 140 GHz radar demo platform
Research program geared towards the entire automotive ecosystem
In its SENSAI program, imec will build virtual models of its groundbreaking sensors, based on ground truth experimental data using imec’s sensor prototypes. Using artificial intelligence, the amount of data can then be expanded in a virtual environment, and combined with models for commercially available sensors.
Ultimately, the goal is to predict the performance improvements of sensor systems that incorporate new sensors. And to demonstrate these improvements by re-training reference sensor fusion models and quantifying performance improvements for specific driving scenarios and corner cases.
Simulations of RGB and infrared camera images, both without and (on the bottom row) with heavy fog.
SENSAI is geared to a wide range of actors in the ecosystem, from OEMs to Tier-1 and Tier-2s, who want to explore the benefits of next-gen sensors and derive sensor specifications in the earliest stages of technology development. Participants can contribute to shaping the research roadmap and get insights into all results.
Want to join imec’s SENSAI program or talk about your automotive sensing challenge?
Click the button below to get in touch.
