Vision-to-inertial measurement unit knowledge transfer for wearable action recognition: Application to fall detection
Loading...
Author
Soltani, R.
Vuagniaux, R.
Saeedi, S.
El Achkar, C. M.
Türetken, E.
Dia, M.
Lemay, M.
Maamari, N.
DOI
Abstract
Human activity recognition using wearable sensors is often limited by the availability and diversity of inertial data, especially for rare or safety-critical events. Fall detection systems are a prominent example, as real fall recordings from older adults are scarce and difficult to collect. To address this challenge, we propose a cross-modal approach that transfers knowledge from vision to wearables by reconstructing 3D human poses from video and generating realistic synthetic inertial measurement unit (IMU) signals. The models are pretrained and fine-tuned on publicly available datasets to estimate accurate world-frame full-body 3D poses from videos and use them to simulate IMU signals. The proposed IMU estimator achieves low median errors (0.02 g and 31.9 d/s), demonstrating a close match between simulated and real inertial signals. For fall detection, when used for impact detection, the simulated IMUs achieve 100% sensitivity and specificity. Converting detected impacts into confirmed falls is more conservative, with a fall-confirmation sensitivity of 40% while specificity remains 100%. Overall, these results show that synthetic IMU signals extracted from real-world video provide high-fidelity motion and impact information that can complement wearable datasets. Beyond fall detection, this vision-to-IMU knowledge transfer approach supports more robust and scalable IMU-based action recognition across a wide range of human activities.
Publication Reference
CSEM Scientific and Technical Report 2025, p. 101–102
Year
2025