Fatigue Detection Through Car Driver’s Face Using Boosting Local Binary Patterns

Grandhys  Setyo Utomo; Ema Rachmawati; Febryanti  Sthevanie

doi:10.29207/resti.v7i5.4798

Grandhys Setyo Utomo Telkom University
Ema Rachmawati Telkom University
Febryanti Sthevanie Telkom University

DOI: https://doi.org/10.29207/resti.v7i5.4798

Keywords: Fatigue Detection, Adaboost, Boosting Local Binary Patterns, Support Vector Machine

Abstract

The general population is concerned with traffic accidents. Driver fatigue is one of the leading causes of car accidents. Several factors, including nighttime driving, sleep deprivation, alcohol consumption, driving on monotonous roads, and drowsy and fatigue-inducing drugs, can contribute to fatigue. This study proposes a facial appearance-based driver fatigue detection system. This is based on the assumption that facial features can be used to identify driver fatigue. We categorize driver conditions into three groups: normal, talking, and yawning. In this study, we used Adaboost to propose Boosting Local Binary Patterns (LBP) to improve the image features of fatigue drivers in the Support Vector Machine (SVM) model. The experimental results indicate that the system's optimal performance achieves an accuracy value of 93.68%, a recall value of 94%, and a precision value of 94%.

Downloads

Download data is not yet available.

Author Biographies

Grandhys Setyo Utomo, Telkom University

Febryanti Sthevanie, Telkom University

References

Kementerian Komunikasi dan Informatika Republik Indonesia, “Rata-rata Tiga Orang Meninggal Setiap Jam Akibat Kecelakaan Jalan,” Aug. 22, 2017. https://www.kominfo.go.id/index.php/content/detail/10368/rata-rata-tiga-orang-meninggal-setiap-jam-akibat-kecelakaan-jalan/0/artikel_gpr (accessed Dec. 20, 2022).

L. H. Tjakranata, B. L. Priatna, and Bianpoen, “Pengaruh Kelelahan Pengemudi terhadap Frekuensi Kecelakaan Lalu Lintas: studi kasus Pengemudi Bus di Jalan Tol Jakarta-Cikampek,” Universitas Indonesia, Jakarta, 1997.

P. Badgujar and P. Selmokar, “Driver gaze tracking and eyes off the road detection,” Mater Today Proc, Oct. 2022, doi: 10.1016/J.MATPR.2022.10.046.

W. Li, J. Huang, G. Xie, F. Karray, and R. Li, “A survey on vision-based driver distraction analysis,” Journal of Systems Architecture, vol. 121, p. 102319, Dec. 2021, doi: 10.1016/J.SYSARC.2021.102319.

G. Sikander and S. Anwar, “Driver Fatigue Detection Systems: A Review,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 6, pp. 2339–2352, Jun. 2019, doi: 10.1109/TITS.2018.2868499.

M. K. Kamti and R. Iqbal, “Evolution of Driver Fatigue Detection Techniques—A Review From 2007 to 2021,” Transp Res Rec, vol. 2676, no. 12, pp. 486–507, Jul. 2022, doi: 10.1177/03611981221096118.

Kementerian Perhubungan Republik Indonesia, “Kelelahan Jadi Faktor Utama Penyebab Kecelakaan,” Aug. 28, 2012. https://dephub.go.id/post/read/kelelahan-jadi-faktor-utama-penyebab-kecelakaan-14470 (accessed Dec. 20, 2022).

M. Marsaid, M. Hidayat, and A. Ahsan, “Faktor yang Berhubungan Dengan Kejadian Kecelakaan Lalu Lintas Pada Pengendara Sepeda Motor di Wilayah Polres Kabupaten Malang,” Journal of Nursing Science Update (JNSU), vol. 1, no. 2, pp. 98–112, Nov. 2013.

M. Ramzan, H. U. Khan, S. M. Awan, A. Ismail, M. Ilyas, and A. Mahmood, “A Survey on State-of-the-Art Drowsiness Detection Techniques,” IEEE Access, vol. 7, pp. 61904–61919, 2019, doi: 10.1109/ACCESS.2019.2914373.

Q. Ji and X. Yang, “Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance,” Real-Time Imaging, vol. 8, no. 5, pp. 357–377, Oct. 2002, doi: 10.1006/RTIM.2002.0279.

J. Zhang, Y. Wu, Y. Chen, J. Wang, J. Huang, and Q. Zhang, “Ubi-Fatigue: Toward Ubiquitous Fatigue Detection via Contactless Sensing,” IEEE Internet Things J, vol. 9, no. 15, pp. 14103–14115, Aug. 2022, doi: 10.1109/JIOT.2022.3146942.

A. Kircher, M. Uddman, and J. Sandin, “Vehicle control and drowsiness,” 2002, Accessed: Dec. 20, 2022. [Online]. Available: http://urn.kb.se/resolve?urn=urn:nbn:se:vti:diva-4880

J. Healey and R. Picard, “Smart Car: Detecting driver stress,” Proceedings - International Conference on Pattern Recognition, vol. 15, no. 4, pp. 218–221, 2000, doi: 10.1109/ICPR.2000.902898.

D. Huang, C. Shan, M. Ardabilian, Y. Wang, and L. Chen, “Local binary patterns and its application to facial image analysis: A survey,” IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 41, no. 6, pp. 765–781, Nov. 2011, doi: 10.1109/TSMCC.2011.2118750.

R. Senaratne, B. Jap, S. Lal, A. Hsu, S. Halgamuge, and P. Fischer, “Comparing two video-based techniques for driver fatigue detection: classification versus optical flow approach,” Machine Vision and Applications 2011 22:4, vol. 22, no. 4, pp. 597–618, May 2011, doi: 10.1007/S00138-011-0321-4.

W. Liu, H. Sun, and W. Shen, “Driver fatigue detection through pupil detection and yawing analysis,” ICBBT 2010 - 2010 International Conference on Bioinformatics and Biomedical Technology, pp. 404–407, 2010, doi: 10.1109/ICBBT.2010.5478931.

D. F. Dinges and R. Grace, “PERCLOS: A Valid Psychophysiological Measure Of Alertness As Assessed By Psychomotor Vigilance,” Washington, DC : Federal Highway Administration, Office of Motor Carriers, Office of Motor Carrier Research and Standards, Oct. 1998.

A. al Redhaei, Y. Albadawi, S. Mohamed, and A. Alnoman, “Realtime Driver Drowsiness Detection Using Machine Learning,” 2022 Advances in Science and Engineering Technology International Conferences, ASET 2022, 2022, doi: 10.1109/ASET53988.2022.9734801.

T. Azim, M. A. Jaffar, M. Ramzan, and A. M. Mirza, “Automatic fatigue detection of drivers through yawning analysis,” Communications in Computer and Information Science, vol. 61, pp. 125–132, 2009, doi: 10.1007/978-3-642-10546-3_16/COVER.

S. Abtahi, M. Omidyeganeh, S. Shirmohammadi, and B. Hariri, “YawDD: A yawning detection dataset,” Proceedings of the 5th ACM Multimedia Systems Conference, MMSys 2014, pp. 24–28, 2014, doi: 10.1145/2557642.2563678.

S. Abtahi, B. Hariri, and S. Shirmohammadi, “Driver drowsiness monitoring based on yawning detection,” Conference Record - IEEE Instrumentation and Measurement Technology Conference, pp. 1606–1610, 2011, doi: 10.1109/IMTC.2011.5944101.

L. Pauly and D. Sankar, “Detection of drowsiness based on HOG features and SVM classifiers,” Proceedings of 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks, ICRCICN 2015, pp. 181–186, Mar. 2016, doi: 10.1109/ICRCICN.2015.7434232.

X. Li, L. Hong, J. chun Wang, and X. Liu, “Fatigue driving detection model based on multi-feature fusion and semi-supervised active learning,” IET Intelligent Transport Systems, vol. 13, no. 9, pp. 1401–1409, Sep. 2019, doi: 10.1049/IET-ITS.2018.5590/CITE/REFWORKS.

B. C. Yin, X. Fan, and Y. F. Sun, “Multiscale Dynamic Features Based Driver Fatigue Detection,” Intern J Pattern Recognit Artif Intell, vol. 23, no. 3, pp. 575–589, Nov. 2009, doi: 10.1142/S021800140900720X.

Y. Luo, C. M. Wu, and Y. Zhang, “Facial expression recognition based on fusion feature of PCA and LBP with SVM,” Optik - International Journal for Light and Electron Optics, vol. 124, no. 17, pp. 2767–2770, Sep. 2013, doi: 10.1016/J.IJLEO.2012.08.040.

C. Chen, H. Seo, C. H. Jun, and Y. Zhao, “Pavement crack detection and classification based on fusion feature of LBP and PCA with SVM,” International Journal of Pavement Engineering, vol. 23, no. 9, pp. 3274–3283, 2021, doi: 10.1080/10298436.2021.1888092.

H. Yang, L. Liu, W. Min, X. Yang, and X. Xiong, “Driver Yawning Detection Based on Subtle Facial Action Recognition,” IEEE Trans Multimedia, vol. 23, pp. 572–583, 2021, doi: 10.1109/TMM.2020.2985536.

F. Wang, Z. Li, F. He, R. Wang, W. Yu, and F. Nie, “Feature Learning Viewpoint of Adaboost and a New Algorithm,” IEEE Access, vol. 7, pp. 149890–149899, 2019, doi: 10.1109/ACCESS.2019.2947359.

C. Xiao, N. Chen, C. Hu, K. Wang, J. Gong, and Z. Chen, “Short and mid-term sea surface temperature prediction using time-series satellite data and LSTM-AdaBoost combination approach,” Remote Sens Environ, vol. 233, p. 111358, Nov. 2019, doi: 10.1016/J.RSE.2019.111358.

Z. Mehmood and S. Asghar, “Customizing SVM as a base learner with AdaBoost ensemble to learn from multi-class problems: A hybrid approach AdaBoost-MSVM,” Knowl Based Syst, vol. 217, p. 106845, Apr. 2021, doi: 10.1016/J.KNOSYS.2021.106845.

W. Deng and R. Wu, “Real-Time Driver-Drowsiness Detection System Using Facial Features,” IEEE Access, vol. 7, pp. 118727–118738, 2019, doi: 10.1109/ACCESS.2019.2936663.

K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks,” IEEE Signal Process Lett, vol. 23, no. 10, pp. 1499–1503, Oct. 2016, doi: 10.1109/LSP.2016.2603342.

Y. Freund, “Boosting a Weak Learning Algorithm by Majority,” Inf Comput, vol. 121, no. 2, pp. 256–285, Sep. 1995, doi: 10.1006/INCO.1995.1136.

Y. Zhang and C. Hua, “Driver fatigue recognition based on facial expression analysis using local binary patterns,” Optik (Stuttg), vol. 126, no. 23, pp. 4501–4505, Dec. 2015, doi: 10.1016/J.IJLEO.2015.08.185.

N. Sun, W. Zheng, C. Sun, C. Zou, and L. Zhao, “Gender classification based on boosting local binary pattern,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3972 LNCS, pp. 194–201, 2006, doi: 10.1007/11760023_29/COVER.

P. Yang, S. Shan, W. Gao, S. Z. Li, and D. Zhang, “Face recognition using Ada-Boosted Gabor features,” Proceedings - Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 356–361, 2004, doi: 10.1109/AFGR.2004.1301556.

B. Akrout and W. Mahdi, “Yawning detection by the analysis of variational descriptor for monitoring driver drowsiness,” IPAS 2016 - 2nd International Image Processing, Applications and Systems Conference, Mar. 2017, doi: 10.1109/IPAS.2016.7880127.