3D Body Reconstruction and pose estimation for automotive radar-based Vulnerable Road User Detection

Introduction

With the rise of autonomous vehicles, there is a growing need for advanced sensing technologies capable of accurately detecting and reconstructing vulnerable road users (VRUs), such as pedestrians, cyclists, and motorcyclists. Ensuring the safety of VRUs in complex urban environments is critical, and traditional sensors such as cameras and LiDAR struggle under adverse conditions like poor lighting or bad weather. MmWave radars, on the other hand, offer robustness in such scenarios, making it a promising tool for VRU detection. However, mmWave radar data typically have low spatial resolution, making the tasks of 3D skeletal pose estimation and full body reconstruction challenging. This PhD research aims to address these challenges by developing advanced algorithms for both skeletal pose estimation and 3D body reconstruction using mmWave radar in automotive settings.

Several recent studies have made progress in VRU detection and reconstruction using mmWave radar, though challenges remain regarding accuracy, real-time processing, and data sparsity. [1] explores the feasibility of human skeletal pose estimation from mmWave radar point clouds. While effective for basic motion detection, the system struggles with real-time performance and precise reconstruction, indicating the need for further refinement in pose estimation models. [2] presents a self-supervised approach that improves 3D pose estimation across different viewpoints. This method could be adapted by leveraging shape consistency to overcome the limitations of mmWave radar’s sparse and noisy data, enhancing skeletal pose estimation for VRUs. [3] introduced pseudo-labeling to enhance pose estimation from sparse radar data, providing a potential solution for handling mmWave radar’s low-resolution point clouds. [4] investigates point augmentation strategies that enrich sparse radar data. Augmenting point clouds could lead to better body reconstruction, particularly in cases where direct skeletal information is unavailable or ambiguous. [5] combines mmWave radar with visual and inertial sensors to refine point cloud quality. This multimodal approach holds promise for improving both skeletal pose estimation and full 3D body reconstruction, by compensating for the radar’s inherent limitations. The authors of [6] and [7] explored deep learning models for 3D object reconstruction using mmWave radar, demonstrating that deep learning can enhance body reconstruction's fidelity in single and multi-object scenarios.

Research Objectives

This PhD research seeks to bridge the gap between skeletal pose estimation and 3D body reconstruction for VRU detection using mmWave radar. The main objectives are:

1. Developing advanced skeletal pose estimation models: Leveraging deep learning and self-supervised techniques to enhance skeletal pose detection accuracy and real-time capability using mmWave radar.
2. Designing robust 3D body reconstruction methods: Creating algorithms capable of reconstructing a full 3D representation of VRUs, even from sparse mmWave radar data, using techniques such as point augmentation.
3. Validating in real-world automotive scenarios: Ensuring the algorithms are robust and reliable in complex, real-time automotive applications, addressing challenges related to diverse road environments and dynamic interactions.

Required qualifications:

• An MSc in a field related to one or more: Electrical engineering, Computer Science, or Applied Computer Science.
• Experience with image processing, signal processing, and computer vision. Some knowledge of radar concepts is a plus.
• Experience with machine learning and statistics.
• Strong programming skills (Python).
• Interest in developing state-of-the-art Machine Learning methods and conducting experiments.
• Ability to write scientific reports and communicate research results at conferences in English.

References