Detecting Genuine Versus Fake Emotions: A Dual-Task Deep Learning Approach Using Facial Expression Analysis
Downloads
Facial expression recognition (FER) is a relevant field of study with applications in human-computer interaction, healthcare, and security. Although recent approaches demonstrate excellent outcomes on the recognition of basic emotions, the authenticity of expressions (genuine versus fake) remains unexplored. In this work, we propose a dual-task deep learning framework based on EfficientNet-B0, enhanced with a lightweight squeeze-and-excitation (SE) attention mechanism, to collaboratively work on multiclass emotion recognition (seven categories: angry, disgust, fear, happy, neutral, sad and surprise) and authenticity classification (genuine vs fake). The architecture leverages a shared backbone for representing feature, followed by task-dedicated branches trained using categorical cross-entropy and focal loss, respectively. To overcome the lack of publicly available benchmarks incorporating authenticity labels, we designed a curated dataset annotated with both emotional categories and authenticity information. Experimental evaluation demonstrates that the proposed dual-task model with the SE attention mechanism achieves 98.5% accuracy for emotion recognition and 92.2% accuracy for authenticity prediction, emphasizing both the effectiveness of the framework and the inherent challenges of authenticity detection. Moreover, we present a deployable real-time system demonstrating the feasibility of integrating authenticity-aware FER into practical applications such as e-learning analytics, security surveillance, and affective computing.
Downloads
[1] Ullah, S., Ou, J., Xie, Y., & Tian, W. (2024). Facial expression recognition (FER) survey: a vision, architectural elements, and future directions. PeerJ Computer Science, 10, e2024. doi:10.7717/PEERJ-CS.2024.
[2] Agung, E. S., Rifai, A. P., & Wijayanto, T. (2024). Image-based facial emotion recognition using convolutional neural network on emognition dataset. Scientific Reports, 14(1), 14429. doi:10.1038/s41598-024-65276-x.
[3] Kaur, S., & Kulkarni, N. (2024). FERFM: An Enhanced Facial Emotion Recognition System Using Fine-tuned MobileNetV2 Architecture. IETE Journal of Research, 70(4), 3723–3737. doi:10.1080/03772063.2023.2202158.
[4] Scarpazza, C., Gramegna, C., Costa, C., Pezzetta, R., Saetti, M. C., Preti, A. N., Difonzo, T., Zago, S., & Bolognini, N. (2025). The Emotion Authenticity Recognition (EAR) test: normative data of an innovative test using dynamic emotional stimuli to evaluate the ability to recognize the authenticity of emotions expressed by faces. Neurological Sciences, 46(1), 133–145. doi:10.1007/s10072-024-07689-0.
[5] Hasan, M. J., Mohammed, N., Rahman, S., & Koehn, P. (2025). HadaSmileNet: Hadamard fusion of handcrafted and deep-learning features for enhancing facial emotion recognition of genuine smiles. arXiv Preprint, arXiv:2509.18550. doi:10.48550/arXiv.2509.18550.
[6] Sujigarasharma, K., Rathi, R., Visvanathan, P., & Kanchana, R. (2022). Emotion-based human-computer interaction. Multidisciplinary Applications of Deep Learning-Based Artificial Emotional Intelligence, 136–150. doi:10.4018/978-1-6684-5673-6.ch009.
[7] Jia, S., Wang, S., Hu, C., Webster, P. J., & Li, X. (2021). Detection of Genuine and Posed Facial Expressions of Emotion: Databases and Methods. Frontiers in Psychology, 11. doi:10.3389/fpsyg.2020.580287.
[8] Bhagat, D., Vakil, A., Gupta, R. K., & Kumar, A. (2024). Facial Emotion Recognition (FER) using Convolutional Neural Network (CNN). Procedia Computer Science, 235, 2079–2089. doi:10.1016/j.procs.2024.04.197.
[9] Ballesteros, J. A., Ramírez V, G. M., Moreira, F., Solano, A., & Pelaez, C. A. (2024). Facial emotion recognition through artificial intelligence. Frontiers in Computer Science, 6. doi:10.3389/fcomp.2024.1359471.
[10] Ruchita Mathur, & Vaibhav Gupta. (2024). Emotion detection from facial images: A hybrid approach to feature extraction and classification. World Journal of Advanced Research and Reviews, 24(2), 2227–2234. doi:10.30574/wjarr.2024.24.2.3620.
[11] Anand, M., & Babu, S. (2024). Multi-class Facial Emotion Expression Identification Using DL-Based Feature Extraction with Classification Models. International Journal of Computational Intelligence Systems, 17(1), 25. doi:10.1007/s44196-024-00406-x.
[12] P. Manimohan, C.S. Keerthi, Sunkara Kavya Sudha, Devireddy Mourya Chandra Reddy, C. Mahendra, & Raginutala Nagarjuna. (2024). Human Emotion Detection Using CNN and Transfer Learning. International Research Journal on Advanced Engineering and Management, 2(05), 1365–1371. doi:10.47392/irjaem.2024.0188.
[13] Khuntia, A., & Kale, S. (2024). Real time emotion analysis using deep learning for education, entertainment, and beyond. arXiv Preprint, arXiv:2407.04560. doi:10.48550/arXiv.2407.04560.
[14] Islam, U., Mahum, R., AlSalman, A., Sharaf, M., Hassan, H., & Huang, B. (2023). Facial Emotions Detection using an Efficient Neural Architecture Search Network. Researchsquare (Preprint), 1-21. doi:10.21203/rs.3.rs-2526836/v1.
[15] Haider, I., Yang, H. J., Lee, G. S., & Kim, S. H. (2023). Robust Human Face Emotion Classification Using Triplet-Loss-Based Deep CNN Features and SVM. Sensors, 23(10), 4770. doi:10.3390/s23104770.
[16] Singh, A. (2024). Realtime Facial Emotion Detection. International Journal for Research in Applied Science and Engineering Technology, 12(3), 3226–3229. doi:10.22214/ijraset.2024.59630.
[17] Hakim, G. J. P., Simangunsong, G. A., Rangga Wasita Ningrat, Jonathan Cristiano Rabika, Muhammad Rafi’ Rusafni, Endang Purnama Giri, & Gema Parasti Mindara. (2024). Real-Time Facial Emotion Detection Application with Image Processing Based on Convolutional Neural Network (CNN). International Journal of Electrical Engineering, Mathematics and Computer Science, 1(4), 27–36. doi:10.62951/ijeemcs.v1i4.123.
[18] Ashraf, A., Gunawan, T. S., Arifin, F., Kartiwi, M., Sophian, A., & Habaebi, M. H. (2023). Enhanced Emotion Recognition in Videos: A Convolutional Neural Network Strategy for Human Facial Expression Detection and Classification. Indonesian Journal of Electrical Engineering and Informatics, 11(1), 286–299. doi:10.52549/ijeei.v11i1.4449.
[19] Pruthviraja, D., Kumar, U. M., Parameswaran, S., Chowdary, V. G., & Bharadwaj, V. (2024). Deep convolutional neural network architecture for facial emotion recognition. PeerJ Computer Science, 10, 1–20. doi:10.7717/peerj-cs.2339.
[20] Cardaioli, M., Miolla, A., Conti, M., Sartori, G., Monaro, M., Scarpazza, C., & Navarin, N. (2022). Face the Truth: Interpretable Emotion Genuineness Detection. Proceedings of the International Joint Conference on Neural Networks, 1–8. doi:10.1109/IJCNN55064.2022.9892298.
[21] Miolla, A., Cardaioli, M., & Scarpazza, C. (2023). Padova Emotional Dataset of Facial Expressions (PEDFE): A unique dataset of genuine and posed emotional facial expressions. Behavior Research Methods, 55(5), 2559–2574. doi:10.3758/s13428-022-01914-4.
[22] Annadurai, S., Arock, M., & Vadivel, A. (2023). Real and fake emotion detection using enhanced boosted support vector machine algorithm. Multimedia Tools and Applications, 82(1), 1333–1353. doi:10.1007/s11042-022-13210-6.
[23] Sunil, M. P., Hariprasad, S. A., Shrishti, S., & Sriharshini, S. (2023). Discrimination Between Fake and Real Emotion Using Modified CNN Model. In Lecture Notes in Networks and Systems: Volume 615, LNNS, 407–416. doi:10.1007/978-981-19-9304-6_38.
[24] Arslan, E. E., Akşahin, M. F., Yilmaz, M., & Ilgın, H. E. (2024). Towards Emotionally Intelligent Virtual Environments: Classifying Emotions through a Biosignal-Based Approach. Applied Sciences (Switzerland), 14(19), 8769. doi:10.3390/app14198769.
[25] Jia, J., Zhang, H., & Liang, J. (2025). Bridging discrete and continuous: A multimodal strategy for complex emotion detection. Proceedings of the 2025 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 1–6. doi:10.1109/MLSP62443.2025.11204253
[26] Govea, J., Navarro, A. M., Sánchez-Viteri, S., & Villegas-Ch, W. (2024). Implementation of deep reinforcement learning models for emotion detection and personalization of learning in hybrid educational environments. Frontiers in Artificial Intelligence, 7. doi:10.3389/frai.2024.1458230.
[27] Evangeline, D., & Parkavi, A. (2024). Facial Emotion Recognition of Online Learners Using a Hybrid Deep Learning Model. International Journal of Intelligent Engineering and Systems, 17(6), 735–751. doi:10.22266/ijies2024.1231.56.
[28] Rathod, V., Chohan, A., Nema, S., Nawghare, A., Devikar, P., & Agrawal, R. (2022). Improved remote mental health illness assessment and detection using facial emotion detection and speech emotion detection. International Journal of Health Sciences, 9577–9590. doi:10.53730/ijhs.v6ns2.7508.
[29] Chethan, R., & Patel, G. L. V. (2024). Deep learning-based emotion detection system. International Journal of Scientific Research in Engineering and Management, 8, 1–13. doi:10.55041/IJSREM36445.
[30] Barnwal, M. A. L., & Barik, M. R. (2025). Human Mood Detection using Image Processing and Machine Learning and Deep Learning. International Journal of Soft Computing and Engineering, 14(6), 28–31. doi:10.35940/ijsce.i9700.14060125.
[31] Zhang, C. (2024). Image-based Facial Emotion Detection SYSTEM. Computer Life, 12(2), 20–26. doi:10.54097/b02twk08.
[32] Singh, M. A. (2024). Real-Time Emotion Recognition System Using Facial Expressions. International Journal of Scientific Research in Engineering and Management, 8(4), 1–5. doi:10.55041/ijsrem31021.
[33] Ton-That, A. H., & Cao, N. T. (2022). Facial Expression Recognition Using a Novel Modeling of Combined Gray Local Binary Pattern. Advances in Human-Computer Interaction, 1–12. doi:10.1155/2022/6798208.
[34] Diya, S. T., Ferdos, M. J., & Rahman, M. M. (2025). Genuine and Fake Facial Emotion Dataset (GFFD-2025). Mendeley Data, 1. doi:10.17632/wmfd4p3z32.1.
- This work (including HTML and PDF Files) is licensed under a Creative Commons Attribution 4.0 International License.
