Validation of wearable step-counting devices in daily life
No Thumbnail Available
Author
Soltani, Ramin Abolfazl
Moufawad El Achkar, Christopher
Pilkar, Rakesh
Zorman, Sylvain
Patterson, Matthew R
Lemay, Mathieu
DOI
10.1515/bmt-2025-1001
Abstract
Methods To evaluate the step-counting algorithms (CSEM, CPIWv1, and CPIWv2) in daily life situations, we have employed two independent datasets as follows: • Clemson dataset (n=30) consisting of laboratory-based walking trials of varying regularities • OxWalk dataset (n=39) consisting of one hour of free-living activity of healthy participants For both datasets, the ground truths were employed to compute performance parameters such as mean absolute per-cent error (MAPE), root-mean-squared-error (RMSE), mean bias with 95% confidence intervals (CI), and Pearson’s correlation coefficient (r). In addition, the first and the second algorithms of this study are designed based on machine learning techniques and the third one is baased on motion-frequency detection analysis for step counting estimation.
Results For the Clemson dataset, CSEM’s algorithm estimated the step counts with the highest accuracy (MAPE: 10.93±16.73%; RMSE of 55.84 steps) with all walk types (regular, semiregular, irregular) combined. CPIWv1 estimated steps with MAPE of 112.52±154.2% and RMSE of 444.81 steps. CPIWv2 estimated steps with MAPE: 114.94±155.28% and RMSE of 451.86 steps. When performance metrics were categorized per walk type, all algorithms showed the highest accuracies (MAPE < 3%) for the regular trials and lowest accuracies (MAPE > 25%) for the irregular trials. While CPIW algorithms’ performances were comparable to CSEM’s algorithm for the regular and semiregular walk, both CPIW algorithms performed poorly for irregular walk (MAPE > 300%, RMSE > 700 steps), significantly overestimating the step counts. On the OxWalk dataset, The CSEM algorithm performed better than the UWF algorithms with the lowest MAPE (24.24±26.92 %) and RMSE (306.76 steps) metrics for the OxWalk dataset. The CSEM algorithm also showed the smallest LoA (95% CI: -318.95to 674.23, CI range: 993.18). In addition, Pearson’s correlation statistics showed a strong linear relationship between the estimated and ground truth step counts for all three algorithms (CSEM: r(37)=0.99, p<<0.05; UWFv1: r(37)=0.97, p<<0.05; UWFv2: r(37)=0.99, p<<0.05).
Conclusion This study provided unbiased evidence on algorithms’ performances to support the deployment of step count algorithms in the free-living environment which is essential for impactful application of wearable digital health technologies in clinical research. Here, we systematically evaluated three wrist-based step-counting algorithms in both controlled and freeliving environments. While all algorithms performed well during regular walking, their accuracy significantly decreased for irregular movements, particularly in real-life conditions. Among them, the motion frequency-based algorithm (CSEM) demonstrated the best overall performance on both in-lab and free-living situations. While the results are promising, they also emphasize the need for further refinement of step-counting algorithms to enhance their reliability in everyday activities.
Publication Reference
BMT 2025, Muttenz (Switzerland)
Year
2025-09-11