1. Home
  2. Vol 7, No 4 (2023)
  3. Thanathamathee

An Optimized Machine Learning and Deep Learning Framework for Facial and Masked Facial Recognition

Putthiporn Thanathamathee, Siriporn Sawangarreerak, Prateep Kongkla, Dinna Nina Mohd Nizam

Abstract


In this study, we aimed to find an optimized approach to improving facial and masked facial recognition using machine learning and deep learning techniques. Prior studies only used a single machine learning model for classification and did not report optimal parameter values. In contrast, we utilized a grid search with hyperparameter tuning and nested cross-validation to achieve better results during the verification phase. We performed experiments on a large dataset of facial images with and without masks. Our findings showed that the SVM model with hyperparameter tuning had the highest accuracy compared to other models, achieving a recognition accuracy of 0.99912. The precision values for recognition without masks and with masks were 0.99925 and 0.98417, respectively. We tested our approach in real-life scenarios and found that it accurately identified masked individuals through facial recognition. Furthermore, our study stands out from others as it incorporates hyperparameter tuning and nested cross-validation during the verification phase to enhance the model's performance, generalization, and robustness while optimizing data utilization. Our optimized approach has potential implications for improving security systems in various domains, including public safety and healthcare.

 

Doi: 10.28991/ESJ-2023-07-04-010

Full Text: PDF


Keywords


Masked Face Recognition; Deep Learning; Hyperparameter Tuning; Grid Search; Nested Cross-Validation.

References


Khel, M. H. K., Kadir, K., Albattah, W., Khan, S., Noor, M. N. M. M., Nasir, H., ... & Khan, A. (2021). Real-time monitoring of COVID-19 SOP in public gathering using deep learning technique. Emerging Science Journal, 5(Special issue), 182-196. doi:10.28991/esj-2021-SPER-14.

Damer, N., Boutros, F., Süßmilch, M., Kirchbuchner, F., & Kuijper, A. (2021). Extended evaluation of the effect of real and simulated masks on face recognition performance. IET Biometrics, 10(5), 548–561. doi:10.1049/bme2.12044.

Alzu’bi, A., Albalas, F., Al-Hadhrami, T., Younis, L. B., & Bashayreh, A. (2021). Masked face recognition using deep learning: A review. Electronics (Switzerland), 10(21), 2666. doi:10.3390/electronics10212666.

Ullah, N., Javed, A., Ali Ghazanfar, M., Alsufyani, A., & Bourouis, S. (2022). A novel DeepMaskNet model for face mask detection and masked facial recognition. Journal of King Saud University - Computer and Information Sciences, 34(10), 9905–9914. doi:10.1016/j.jksuci.2021.12.017.

Ejaz, Md. S., Islam, Md. R., Sifatullah, M., & Sarker, A. (2019). Implementation of Principal Component Analysis on Masked and Non-masked Face Recognition. 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT). doi:10.1109/icasert.2019.8934543.

Maharani, D. A., Machbub, C., Rusmin, P. H., & Yulianti, L. (2020). Improving the Capability of Real-Time Face Masked Recognition using Cosine Distance. 2020 6th International Conference on Interactive Digital Media (ICIDM), Bandung, Indonesia. doi:10.1109/icidm51048.2020.9339677.

Montero, D., Nieto, M., Leskovsky, P., & Aginako, N. (2022). Boosting Masked Face Recognition with Multi-Task ArcFace. 2022 16th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Dijon, France. doi:10.1109/sitis57111.2022.00042.

Hariri, W. (2022). Efficient masked face recognition method during the COVID-19 pandemic. Signal, Image and Video Processing, 16(3), 605–612. doi:10.1007/s11760-021-02050-w.

Boutros, F., Damer, N., Kirchbuchner, F., & Kuijper, A. (2022). Self-restrained triplet loss for accurate masked face recognition. Pattern Recognition, 124, 108473. doi:10.1016/j.patcog.2021.108473.

Golwalkar, R., & Mehendale, N. (2022). Masked-face recognition using deep metric learning and FaceMaskNet-21. Applied Intelligence, 52(11), 13268–13279. doi:10.1007/s10489-021-03150-3.

Hong, J. H., Kim, H., Kim, M., Nam, G. P., Cho, J., Ko, H.-S., & Kim, I.-J. (2021). A 3d Model-Based Approach For Fitting Masks To Faces In The Wild. 2021 IEEE International Conference on Image Processing (ICIP), Anchorage, AK, USA. doi:10.1109/icip42928.2021.9506069.

Ejaz, Md. S., & Islam, Md. R. (2019). Masked Face Recognition Using Convolutional Neural Network. 2019 International Conference on Sustainable Technologies for Industry 4.0 (STI). doi:10.1109/sti47673.2019.9068044.

Deng, H., Feng, Z., Qian, G., Lv, X., Li, H., & Li, G. (2021). MFCosface: a masked-face recognition algorithm based on large margin cosine loss. Applied Sciences (Switzerland), 11(16), 7310. doi:10.3390/app11167310.

Li, Y., Guo, K., Lu, Y., & Liu, L. (2021). Cropping and attention based approach for masked face recognition. Applied Intelligence, 51(5), 3012–3025. doi:10.1007/s10489-020-02100-9.

Moungsouy, W., Tawanbunjerd, T., Liamsomboon, N., & Kusakunniran, W. (2022). Face recognition under mask-wearing based on residual inception networks. Applied Computing and Informatics, 1-14. doi:10.1108/ACI-09-2021-0256.

Talahua, J. S., Buele, J., Calvopina, P., & Varela-Aldas, J. (2021). Facial recognition system for people with and without face mask in times of the covid-19 pandemic. Sustainability (Switzerland), 13(12), 6900. doi:10.3390/su13126900.

Song, Z., Nguyen, K., Nguyen, T., Cho, C., & Gao, J. (2022). Spartan Face Mask Detection and Facial Recognition System. Healthcare (Switzerland), 10(1), 87. doi:10.3390/healthcare10010087.

Kim, M., Jain, A. K., & Liu, X. (2022). AdaFace: Quality Adaptive Margin for Face Recognition. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr52688.2022.01819.

Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). ArcFace: Additive Angular Margin Loss for Deep Face Recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2019.00482.

Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., & Liu, W. (2018). CosFace: Large Margin Cosine Loss for Deep Face Recognition. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. doi:10.1109/cvpr.2018.00552.

Lu, H., & Zhuang, Z. (2022). ULN: An efficient face recognition method for person wearing a mask. Multimedia Tools and Applications, 81(29), 42393–42411. doi:10.1007/s11042-022-13495-7.

Ge, Y., Liu, H., Du, J., Li, Z., & Wei, Y. (2023). Masked face recognition with convolutional visual self-attention network. Neurocomputing, 518, 496–506. doi:10.1016/j.neucom.2022.10.025.

Wang, Y., Li, Y., & Zou, H. (2023). Masked Face Recognition System Based on Attention Mechanism. Information (Switzerland), 14(2), 87. doi:10.3390/info14020087.

Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23(10), 1499–1503. doi:10.1109/LSP.2016.2603342.

Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding for face recognition and clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi:10.1109/cvpr.2015.7298682.

Elgeldawi, E., Sayed, A., Galal, A. R., & Zaki, A. M. (2021). Hyperparameter tuning for machine learning algorithms used for arabic sentiment analysis. Informatics, 8(4), 79. doi:10.3390/informatics8040079.

Jiang, X., & Xu, C. (2022). Deep Learning and Machine Learning with Grid Search to Predict Later Occurrence of Breast Cancer Metastasis Using Clinical Data. Journal of Clinical Medicine, 11(19), 5772. doi:10.3390/jcm11195772.

Saad, M. H., Hashima, S., Sayed, W., El-Shazly, E. H., Madian, A. H., & Fouda, M. M. (2023). Early Diagnosis of COVID-19 Images Using Optimal CNN Hyperparameters. Diagnostics, 13(1), 76. doi:10.3390/diagnostics13010076.

Zöller, M. A., & Huber, M. F. (2021). Benchmark and Survey of Automated Machine Learning Frameworks. Journal of Artificial Intelligence Research, 70, 409–472. doi:10.1613/JAIR.1.11854.

Tahkola, M., & Guangrong, Z. (2022). ATSC-NEX: Automated Time Series Classification With Sequential Model-Based Optimization and Nested Cross-Validation. IEEE Access, 10, 39299–39312. doi:10.1109/ACCESS.2022.3166525.

Wainer, J., & Cawley, G. (2021). Nested cross-validation when selecting classifiers is overzealous for most practical applications. Expert Systems with Applications, 182, 115222. doi:10.1016/j.eswa.2021.115222.

Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on Image Data Augmentation for Deep Learning. Journal of Big Data, 6(1). doi:10.1186/s40537-019-0197-0.

Peng, S., Huang, H., Chen, W., Zhang, L., & Fang, W. (2020). More trainable inception-ResNet for face recognition. Neurocomputing, 411, 9–19. doi:10.1016/j.neucom.2020.05.022.

Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014). Learning Face Representation from Scratch. doi:10.48550/arXiv.1411.7923.

Huang, G. B., Mattar, M., Berg, T., & Learned-Miller, E. (2008). Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Workshop on faces in 'Real-Life' Images: detection, alignment, and recognition, 17-20 October, 2008, Marseille, France.

Kaur, A., & Kaur, I. (2018). An empirical evaluation of classification algorithms for fault prediction in open source projects. Journal of King Saud University - Computer and Information Sciences, 30(1), 2–17. doi:10.1016/j.jksuci.2016.04.002.

Chen, S., He, H., & Garcia, E. A. (2010). RAMOBoost: Ranked minority oversampling in boosting. IEEE Transactions on Neural Networks, 21(10), 1624–1642. doi:10.1109/TNN.2010.2066988.

Khamparia, A., & Singh, K. M. (2019). A systematic review on deep learning architectures and applications. Expert Systems, 36(3), e12400. doi:10.1111/exsy.12400.

Krstajic, D., Buturovic, L. J., Leahy, D. E., & Thomas, S. (2014). Cross-validation pitfalls when selecting and assessing regression and classification models. Journal of Cheminformatics, 6(1), 1–15. doi:10.1186/1758-2946-6-10.

Fayed, H. A., & Atiya, A. F. (2019). Speed up grid-search for parameter selection of support vector machines. Applied Soft Computing Journal, 80, 202–210. doi:10.1016/j.asoc.2019.03.037.


Full Text: PDF

DOI: 10.28991/ESJ-2023-07-04-010

Refbacks

  • There are currently no refbacks.


Copyright (c) 2023 Putthiporn Thanathamathee, Siriporn Sawangarreerak, Prateep Kongkla, Dinna Nina Mohd Nizam