Self-Supervised Scene Flow Estimation for Automotive Applications Using Off-the-Shelf mm-Wave Radar

Topic Description

In the automotive industry, precise environment perception is crucial for advanced driver-assistance systems (ADAS) to improve the safety. Scene flow estimation involves determining the 3D motion vectors of all points in a scene, providing a detailed understanding of object movements and spatial relationships. It plays a vital role in understanding dynamic environments.  Traditional approaches rely heavily on LiDAR or stereo cameras, which can be expensive and computationally demanding. Recent millimeter-wave (mmWave) radar technology advancements offer a cost-effective and robust alternative. mmWave radar offers several advantages for automotive sensing, including robustness to weather conditions, low cost, and compact size. This thesis explores self-supervised scene flow estimation using commercially available mmWave radar technology, complemented by camera data for depth completion, to enhance 3D perception in automotive applications. The successful completion of this project could significantly enhance the capabilities of automotive sensing systems.

Type of work:

1. Literature Review and Analysis: Review existing literature on scene flow estimation and self-supervised learning techniques.
2. Algorithm Development: Create self-supervised learning algorithms for scene flow estimation, leveraging the complementary strengths of radar and camera data
3. Field Testing and Validation: Conduct field tests to validate the performance of the proposed system in real-world automotive scenarios, we will make use of available data.
4. Performance Evaluation: Assess the system's accuracy, robustness, and computational efficiency compared to traditional methods.

Required qualifications:

• Following an MSc in a field related to one or more: Electrical engineering, Computer Science, or Applied Computer Science.
• Experience with image processing, signal processing, and computer vision. Some knowledge of radar concepts is a plus.
• Experience with machine learning and statistics.
• Strong programming skills (Python).
• Interest in developing state-of-the-art Machine Learning methods and conducting experiments.
• Ability to write scientific reports and communicate research results at conferences in English.

Type:

• Master Thesis internship @ imec (6 months)
• Preceded by (optional) summer internship (1-3 months) @ imec

(the summer internship alone is not possible)

Responsible scientist(s): 

Prof. Hichem Sahli (imec / VUB)  hichem.sahli@imec.be 
Dr. Seyed Hamed Javadi (imec) hamed.javadi@imec.be

Supervisor:  Piet Wambacq (VUB) 

References:

1. Li, Z., et al. (2023). Deep Learning for Scene Flow Estimation on Point Clouds: A Survey and Prospective Trends. Computer Graphics Forum, 42: e14795. https://doi.org/10.1111/cgf.14795.
2. Ling, F. et al. (2024). milliFlow: Scene Flow Estimation on mmWave Radar Point Cloud for Human Motion Sensing. arXiv, https://arxiv.org/abs/2306.17010. 
3. Ding, F. et al. (2023). Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision,  CVPR’23 pp. 9340-9349, doi: 10.1109/CVPR52729.2023.00901. 
4. Hao, Y. et al. (2024). Bootstrapping Autonomous Driving Radars with Self-Supervised Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 15012-15023).

Type of Project: Combination of internship and thesis 

Master's degree: Master of Engineering Science 

Master program: Electromechanical engineering; Computer Science 

Duration:  9 months 

For more information or application, please contact Hichem Sahli (hichem.sahli@imec.be)  and Hamed Javadi (hamed.javadi@imec.be)

Imec allowance will be provided for students studying at a non-Belgian university.