Quarterly Journal of Information and Communication Technology ​

Emotion Recognition Based on Improved VGG-16 Network

Document Type : Original Research Article

Author

Assistant Professor, Department of Computer Engineering, FSh.C., Islamic Azad University, Fouman, Iran

10.22034/apj.2025.730009
Abstract
Background and Objectives: In recent years, the use of deep learning methods to analyze electroencephalogram and physiological signals for emotion recognition has attracted the attention of many researchers and scholars. However, the lack of expertise in extracting features from electroencephalogram signals and classifying emotions based on these signals is considered one of the main challenges of this discussion.
Methods: In this study, signal-to-image conversion will be used to prepare input data for a pre-trained network. The innovation of the research is in how to prepare the input data to the pre-trained network. The innovation of the proposed method is also the conversion of signal to image as input features to the learning model in the recorded data for emotion recognition. Also, a hybrid architecture based on VGG-16 network and fuzzy layer is used to optimally solve the problem and increase the emotion recognition rate.
Findings: Given the problems and limitations of data collection and the sensitivity of electroencephalogram data to environmental noise and similar problems, recording electroencephalogram signals in an isolated environment is required. Considering the above, it is better to use a dataset whose accuracy has been previously confirmed by researchers and scholars. In this article, the public DEAP dataset was used for research and hypothesis testing.
Conclusion: By examining the results of implementing the proposed method, the emotion recognition rate shows an accuracy of 90.89 percent.

Keywords


[1] Pranav EKamal S, Chandran C S, Supriya M H, facial emotion recognition using deep convolutional neural network, 6th International Conference on Advanced Computing & Communication System (ICACCS) , Kochi, India, 2020.
[2] Jaiswal A, Facial Emotion Detection Using Deep Learning, International Conference for Emerging Technology (INCET), Belgaum, India, 2020.
[3] Akinpelu S, Lightweight Deep Learning Framework for Speech Emotion Recognition, IEEE Access, 2023.
[4] Tarunika P A K, "Applying Machine Learning Techniques for Speech Emotion Recognition, in 2018 9th International Conference on Contemporary Computing and Communication Networks (ICCCNT), Bengaluru, India, 2018.
[5] Torres M, Evaluation of Interpretability for Deep Learning Algorithms in EEG Emotion Recognition: A Case Study in Autism, in Artificial Intelligence in Medicine, 2023.
[6] Bazgir O, Emotion recognition with machine learning using EEG signals, Journal of Electrical and Computer Engineering, 2023.
[7] Raveendrababu Vempati L D S, A systematic review on automated human emotion recognition using electroencephalogram signals and artificial intelligence, Resulted in Engineering, 2023; 18: 101027.
[8] Suhaimi N J T, EEG-based emotion recognition: A state of the art review of current trends and opportunities, Computational Intelligence and Neuroscience, 2022: 8875426.
[9] Yuwei Chen J H, Deep Learning-Based Emotion Detection", Journal of Computer and Communications, 2022; 10: 57-71.
[10] Phan T, EEG-based emotion recognition by convolutional neural network with multi-scale kernels, Sensors, 2021; 21(15).
[11] Kwon H, Electroencephalography Based Fusion Two-Dimensional (2D)-Convolution Neural Networks (CNN) Model for Emotion Recognition System, Sensors, 2018; 18(5).
[12] Zhong Q, Electroencephalogram access for emotion recognition based on a deep hybrid network, Frontiers in Human Neuroscience, 2020.
[13] Cho H H J. Spatio-temporal representation of an electroencephalogram for emotion recognition using a three-dimensional convolutional neural network, Sensors, 2020; 20(12).
[14] Zhang A, et al. EEG data augmentation for emotion recognition with a multiple generator conditional Wasserstein GAN, Complex & Intelligent Systems, 2021: 1-13.
[15] Luo Y, et al. Data augmentation for enhancing EEG-based emotion recognition with deep generative models, Journal of Neural Engineering, 2020; 17(5): 056021.
[16] Gonzalez H A, Yoo J, Elfadel I M. EEG-based emotion detection using unsupervised transfer learning, in 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC), 2019, IEEE.
[17] Yin Z, et al. Cross-subject EEG feature selection for emotion recognition using transfer recursive feature elimination, Frontiers in neurorobotics, 2017; 11: 19.
[18] Cai J, Chen W, Yin Z. Multiple transferable recursive feature elimination technique for emotion recognition based on EEG signals, Symmetry, 2019; 11(5): 683.
[19] Cimtay Y, Ekmekcioglu E. Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition,S ensors, 2020; 20(7): 20-34.
[20] Yin Z, et al. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model, Computer methods and programs in biomedicine, 2017; 140: 93-110.
[21] Alhagry S, Fahmy A A, El-Khoribi R A. Emotion recognition based on EEG using LSTM recurrent neural network, International Journal of Advanced Computer Science and Applications, 2017; 8(10).
[22] Salama E S. EEG-based emotion recognition using 3D convolutional neural networks, International Journal of Advanced Computer Science and Applications, 2018; 9(8).
[23] Zheng  W L, Zhu J-Y, Lu B-L, Identifying stable patterns over time for emotion recognition from EEG, IEEE transactions on affective computing, 2017; 10(3): 417-429.
[24] Li, X. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network, IEEE international conference on bioinformatics and biomedicine (BIBM), 2016.
[25] Tripathi S. Using deep and convolutional neural networks for accurate emotion classification on DEAP data, in Proceedings of the AAAI Conference on Artificial Intelligence. 2017.
[26] Aslan Z, Akin M. Automatic Detection of Schizophrenia by Applying Deep Learning over Spectrogram Images of EEG Signals, Traitement du Signal, 2020; 37(2).
[27] Raghu S. EEG based multi-class seizure type classification using convolutional neural network and transfer learning, Neural Networks, 2020; 124: 202-212.
[28] Bizopoulos P, Lambrou G I, Koutsouris D. Signal2image modules in deep neural networks for eeg classification, in 2019 41st Annual international conference of the IEEE engineering in medicine and biology society (EMBC). 2019.
[29] Prasomphan S. Improvement of speech emotion recognition with neural network classifier by using speech spectrogram, in 2015 International Conference on Systems, Signals and Image Processing (IWSSIP). 2015.
[30] Acharya R. Multi-Class Emotion Classification Using EEG Signals, in Advances in Computing, Singapore, 2021; 1367: 474-485.
[31] Choi E. K. D. Arousal and valence classification model based on long short-term memory and deap data for mental healthcare management, Healthc, 2018; 4(24): 309-316.
[32] Alhagry S. Emotion recognition based on EEG using LSTM recurrent neural network, Emotion, 2017; 8(10): 355-358.