Hybrid Deep Learning Models for Free-Living Imbalanced Human Activity Recognition: Comparative Study
DOI:
https://doi.org/10.29207/resti.v10i2.6936Keywords:
attention mechanism, deep learning, HARTH, human activity recognition, imbalanced dataset, wearable sensorsAbstract
This study presents a comparative evaluation of three hybrid deep learning models for human activity recognition (HAR) in free-living and highly imbalanced conditions: 1DCNN-ResBLSTM-Attention (Model A), Attention-Mechanism-Based Deep Learning Feature Combination (Model B), and Time-Reversal-1DCNN-ResLSTM-Attention (Model C). Each architecture integrates convolutional layers for feature extraction, recurrent networks for temporal modeling, and attention mechanisms to enhance relevant representations. The HARTH v2.0 dataset, comprising 31 subjects and 15 activity classes under strong class imbalance, is used for evaluation. Results show that soft labeling consistently improves performance by better capturing transitional uncertainty in windowed sensor data. Model A achieves the highest accuracy (96.21%) and macro-averaged F1-score (88.17%), followed by Model C with comparable performance at lower computational cost, while Model B underperforms on minority classes due to limitations of spectrogram-based representations. Across all models, persistent confusion is observed among activities with similar motion patterns, such as walking, standing, and shuffling, indicating intrinsic ambiguity in sensor signals. This study provides a controlled and standardized comparison of hybrid architectures under realistic conditions, revealing both performance trade-offs and shared limitations. The findings highlight the importance of modeling uncertainty and temporal context for improving robustness, particularly transitional and underrepresented activities.
Downloads
References
A. O. Ige and M. H. M. Noor, “A Survey on Unsupervised Learning for Wearable Sensor-Based Activity Recognition,” Applied Soft Computing, vol. 127, p. 109363, 2022, doi: 10.1016/j.asoc.2022.109363.
M. A. A. Al-qaness, A. M. Helmi, A. Dahou, and M. A. Elaziz, “The Applications of Metaheuristics for Human Activity Recognition and Fall Detection Using Wearable Sensors: A Comprehensive Analysis,” Biosensors, vol. 12, no. 10, p. 821, 2022, doi: 10.3390/bios12100821.
M. S. Irwanto, F. A. Bachtiar, and N. Yudistira, “Klasifikasi Aktivitas Manusia Menggunakan Algoritme Computed Input Weight Extreme Learning Machine dengan Reduksi Dimensi Principal Component Analysis,” Jurnal Teknologi Informasi dan Ilmu Komputer (JTIIK), vol. 9, no. 6, pp. 1195–1202, 2022, doi: 10.25126/jtiik.2022965504.
M. Karim, S. Khalid, A. Aleryani, J. Khan, I. Ullah, and Z. Ali, “Human Action Recognition Systems: A Review of the Trends and State-of-the-Art,” IEEE Access, vol. 12, pp. 36372–36390, 2024, doi: 10.1109/ACCESS.2024.3373199.
M. H. Abdullah and M. A. Ahmed, “Human Activity Recognition using Accelerometer & Gyroscope Smartphone Sensor by Extract Statistical Features,” Journal of Robotics and Control (JRC), vol. 5, no. 5, pp. 1390–1398, 2024, doi: 10.18196/jrc.v5i5.22443.
S. Zhang, Y. Li, S. Zhang, F. Shahabi, S. Xia, Y. Deng, and N. Alshurafa, “Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances,” Sensors, vol. 22, no. 4, p. 1476, 2022, doi: 10.3390/s22041476.
M. El Ghazi and N. Aknin, “A Systematic Review of Non-Intrusive Human Activity Recognition in Smart Homes Using Deep Learning,” IAES International Journal of Artificial Intelligence (IJ-AI), vol. 13, no. 3, pp. 3188–3202, 2024, doi: 10.11591/ijai.v13.i3.pp3188-3202.
M. Zaher, A. S. Ghoneim, L. Abdelhamid, and A. Atia, “Unlocking the Potential of RNN and CNN Models for Accurate Rehabilitation Exercise Classification on Multi-Datasets,” Multimedia Tools and Applications, vol. 84, no. 3, pp. 1261–1301, 2024, doi: 10.1007/s11042-024-19092-0.
I. D. Luptáková, M. Kubovčík, and J. Pospíchal, “Wearable Sensor-Based Human Activity Recognition with Transformer Model,” Sensors, vol. 22, no. 5, p. 1911, 2022, doi: 10.3390/s22051911.
K. K. Sahoo, R. Ghosh, S. Mallik, A. Roy, P. K. Singh, and Z. Zhao, “Wrapper-Based Deep Feature Optimization for Activity Recognition in the Wearable Sensor Networks of Healthcare Systems,” Scientific Reports, vol. 13, no. 1, p. 965, 2023, doi: 10.1038/s41598-022-27192-w.
A. Logacjov, S. Herland, A. Ustad, and K. Bach, “SelfPAB: Large-Scale Pre-Training on Accelerometer Data for Human Activity Recognition,” Applied Intelligence, vol. 54, no. 6, pp. 4545–4563, 2024. doi: 10.1007/s10489-024-05322-3.
R. R. Chowdhury, R, Kapila, A. Panse, X. Zhang, D. Teng, R. Kulkarni, D. Hong, R. K. Gupta, and J. Shang, “ZeroHAR: Sensor Context Augments Zero-Shot Wearable Action Recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 15, pp. 16046–16054, 2025, doi: 10.1609/aaai.v39i15.33762.
F. J. Ordóñez and D. Roggen, “Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition,” Sensors, vol. 16, no. 1, p. 115, 2016, doi: 10.3390/s16010115.
M. H. M. Noor, S. Y. Tan, and M. N. Ab Wahab, “Deep Temporal Conv-LSTM for Activity Recognition,” Neural Processing Letters, vol. 54, no. 5, pp. 4027–4049, 2022, doi: 10.1007/s11063-022-10799-5.
S. P. Singh, A. Lay-Ekuakille, D. Gangwar, M. K. Sharma, and S. Gupta, “Deep ConvLSTM with Self-Attention for Human Activity Decoding Using Wearables,” 2020, doi: 10.48550/ARXIV.2005.00698.
Y. Li and L. Wang, “Human Activity Recognition Based on Residual Network and BiLSTM,” Sensors, vol. 22, no. 2, p. 635, 2022, doi: 10.3390/s22020635.
J. Zhang, Y. Liu, and H. Yuan, “Attention-Based Residual BiLSTM Networks for Human Activity Recognition,” IEEE Access, vol. 11, pp. 94173–94187, 2023, doi: 10.1109/ACCESS.2023.3310269.
M. Akter, S. Ansary, M. A.-M. Khan, and D. Kim, “Human Activity Recognition Using Attention-Mechanism-Based Deep Learning Feature Combination,” Sensors, vol. 23, no. 12, p. 5715, 2023, doi: 10.3390/s23125715.
S. AlMuhaideb, L. AlAbdulkarim, D. M. AlShahrani, H. AlDhubaib, and D. E. AlSadoun, “Achieving More with Less: A Lightweight Deep Learning Solution for Advanced Human Activity Recognition (HAR),” Sensors, vol. 24, no. 16, p. 5436, 2024, doi: 10.3390/s24165436.
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity Recognition Using Cell Phone Accelerometers,” ACM SIGKDD Explorations Newsletter, vol. 12, no. 2, pp. 74–82, 2011, doi: 10.1145/1964897.1964918.
N. Sikder and A.-A. Nahid, “KU-HAR: An Open Dataset for Heterogeneous Human Activity Recognition,” Pattern Recognition Letters, vol. 146, pp. 46–54, 2021, doi: 10.1016/j.patrec.2021.02.024.
D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, “A Public Domain Dataset for Human Activity Recognition Using Smartphones,” in Proceedings of the 21st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 2013, pp. 437–442.
A. Logacjov, K. Bach, A. Kongsvold, H. B. Bårdstu, and P. J. Mork, “HARTH: A Human Activity Recognition Dataset for Machine Learning,” Sensors, vol. 21, no. 23, p. 7853, 2021, doi: 10.3390/s21237853.
T. Gong, J. Jeong, T. Kim, Y. Kim, J. Shin, and S.-J. Lee, “NOTE: Robust Continual Test-Time Adaptation Against Temporal Correlation,” in Proceedings of the Conference on Neural Information Processing Systems (NeurIPS), 2023.
N. Bento, J. Rebelo, M. Barandas, A. V. Carreiro, A. Campagner, F. Cabitza, and H. Gamboa, “Comparing Handcrafted Features and Deep Neural Representations for Domain Generalization in Human Activity Recognition,” Sensors, vol. 22, no. 19, p. 7324, 2022, doi: 10.3390/s22197324.
A. Ustad, A. Logacjov, S. Ø. Trollebø, P. Thingstad, B. Vereijken, K. Bach, and N. S., Maroni, "Validation of an Activity Type Recognition Model Classifying Daily Physical Behavior in Older Adults: The HAR70+ Model," Sensors, vol. 23, no. 5, p. 2368, 2023, doi: 10.3390/s23052368.
A. Logacjov and K. Bach, “Self-Supervised Learning with Randomized Cross-Sensor Masked Reconstruction for Human Activity Recognition,” Engineering Applications of Artificial Intelligence, vol. 128, p. 107478, 2024, doi: 10.1016/j.engappai.2023.107478.
D. Jin, Y. Hu, B. Chen, G. He, J. Chen, and Z. Shen, “TIAN: A Time Series Imaging Association Network for Human Abnormal Behavior Detection,” Information Fusion, vol. 118, p. 102906, 2025, doi: 10.1016/j.inffus.2024.102906.
F. R. Albogamy, “Federated Learning for IoMT-Enhanced Human Activity Recognition with Hybrid LSTM-GRU Networks,” Sensors, vol. 25, no. 3, p. 907, 2025, doi: 10.3390/s25030907.
A. Mulyani, S. Khoerunisa, and D. Kurniadi, “Comparison of KNN and SVM Algorithms Performance using SMOTE to Classify Diabetes,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi, vol. 14, no. 1, pp. 25–34, 2025, doi: 10.22146/jnteti.v14i1.15198.
R. Haddadj, A. L. Nordstoga, T. I. L. Nilsen, E. S. Skarpsno, A. Kongsvold, M. Flaaten, J. Schipperijn, K. Bach, and P. J. Mork, “Volume and Intensity of Walking and Risk of Chronic Low Back Pain,” JAMA Network Open, vol. 8, no. 6, p. e2515592, 2025, doi: 10.1001/jamanetworkopen.2025.15592.
K. Ellis, J. Kerr, S. Godbole, and G. Lanckriet, “Multi-Sensor Physical Activity Recognition in Free-Living,” in Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, 2014, pp. 431–440, doi: 10.1145/2638728.2641673.
S. Hurtado, J. García-Nieto, A. Popov, and I. Navas Delgado, “Human Activity Recognition from Sensorised Patient’s Data in Healthcare: A Streaming Deep Learning-Based Approach,” International Journal of Interactive Multimedia and Artificial Intelligence, vol. 8, no. 1, pp. 23–37, 2023, doi: 10.9781/ijimai.2022.05.004.
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;





