Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques

Pavan Nageswar Reddy Bodavarapu; P V V S Srinivas

doi:10.17485/IJST/v14i12.14

Article

Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques

VIEWS 2569
PDF 1016

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v14i12.14

Year: 2021, Volume: 14, Issue: 12, Pages: 971-983

Original Article

Facial expression recognition for low resolution images using convolutional neural networks and denoising techniques

Pavan Nageswar Reddy Bodavarapu^1*, P V V S Srinivas²

¹ Student, CSE Department, Koneru Lakshmaiah Education Foundation, 534406, India. Tel.: +91-9182811847
² Assistant Professor, CSE Department, Koneru Lakshmaiah Education Foundation, 502355, India

*Corresponding author
Tel: +91-9182811847
Email: [email protected]

Received Date:05 January 2021, Accepted Date:21 March 2021, Published Date:13 April 2021

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Background/Objectives: There is only limited research work is going on in the field of facial expression recognition on low resolution images. Mostly, all the images in the real world will be in low resolution and might also contain noise, so this study is to design a novel convolutional neural network model (FERConvNet), which can perform better on low resolution images. Methods: We proposed a model and then compared with state-of-art models on FER2013 dataset. There is no publicly available dataset, which contains low resolution images for facial expression recognition (Anger, Sad, Disgust, Happy, Surprise, Neutral, Fear), so we created a Low Resolution Facial Expression (LRFE) dataset, which contains more than 6000 images of seven types of facial expressions. The existing FER2013 dataset and LRFE dataset were used. These datasets were divided in the ratio 80:20 for training and testing and validation purpose. A HDM is proposed, which is a combination of Gaussian Filter, Bilateral Filter and Non local means denoising Filter. This hybrid denoising method helps us to increase the performance of the convolutional neural network. The proposed model was then compared with VGG16 and VGG19 models. Findings: The experimental results show that the proposed FERConvNet_HDM approach is effective than VGG16 and VGG19 in facial expression recognition on both FER2013 and LRFE dataset. The proposed FERConvNet_HDM approach achieved 85% accuracy on Fer2013 dataset, outperforming the VGG16 and VGG19 models, whose accuracies are 60% and 53% on Fer2013 dataset respectively. The same FERConvNet_HDM approach when applied on LRFE dataset achieved 95% accuracy. After analyzing the results, our FERConvNet_HDM approach performs better than VGG16 and VGG19 on both Fer2013 and LRFE dataset. Novelty/Applications: HDM with convolutional neural networks, helps in increasing the performance of convolutional neural networks in Facial expression recognition.

Keywords: Facial expression recognition; facial emotion; convolutional neural network; deep learning; computer vision

References

Kopparapu S, Kumar M, Satish. Identifying Optimal Gaussian Filter for Gaussian Noise Removal. In: 2011 Third National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics . .
Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision. IEEE. .
Deledalle CA, Duval V, Salmon J. Non-local Methods with Shape-Adaptive Patches (NLM-SAP) Journal of Mathematical Imaging and Vision. 2012;43(2):103–120. Available from: https://dx.doi.org/10.1007/s10851-011-0294-y
Deng J, Pang G, Zhang Z, Pang Z, Yang H, Yang G. cGAN Based Facial Expression Recognition for Human-Robot Interaction. IEEE Access. 2019;7:9848–9859. Available from: https://dx.doi.org/10.1109/access.2019.2891668
Zhang H, Jolfaei A, Alazab M. A Face Emotion Recognition Method Using Convolutional Neural Network and Image Edge Computing. IEEE Access. 2019;7:159081–159089. Available from: https://dx.doi.org/10.1109/access.2019.2949741
Wang Y, Li Y, Song Y, Rong X. The Application of a Hybrid Transfer Algorithm Based on a Convolutional Neural Network Model and an Improved Convolution Restricted Boltzmann Machine Model in Facial Expression Recognition. IEEE Access. 2019;7:184599–184610. Available from: https://dx.doi.org/10.1109/access.2019.2961161
Jian BL, Chen CL, Huang MW, Yau HT. Emotion-Specific Facial Activation Maps Based on Infrared Thermal Image Sequences. IEEE Access. 2019;7:48046–48052. Available from: https://dx.doi.org/10.1109/access.2019.2908819
Pham TTD, Kim S, Lu Y, Jung SW, Won CS. Facial Action Units-Based Image Retrieval for Facial Expression Recognition. IEEE Access. 2019;7:5200–5207. Available from: https://dx.doi.org/10.1109/access.2018.2889852
Lo L, Xie HX, Shuai HH, Cheng WH. MER-GCN: Micro-Expression Recognition Based on Relation Modeling with Graph Convolutional Networks. 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). 2020. doi: 10.1109/mipr49039.2020.00023
Melinte DO, Vladareanu L. Facial Expressions Recognition for Human–Robot Interaction Using Deep Convolutional Neural Networks with Rectified Adam Optimizer. Sensors. 2020;20(8):2393. Available from: https://dx.doi.org/10.3390/s20082393
Ch S, Kiruthika K, Priya B, Jayalakshmi R, J. Detection ofHuman Facial Expression using CNN Model. International Journal of Engineering and Advanced Technology. 2020;9(5):2249–8958. doi: 10.35940/ijeat.E9615.069520
Sreenivas V, Namdeo V, Kumar EV. Group based emotion recognition from video sequence with hybrid optimization based recurrent fuzzy neural network. Journal of Big Data. 2020;7(1):1–21. Available from: https://dx.doi.org/10.1186/s40537-020-00326-5
Lu P, Song B, Xu L. Human face recognition based on convolutional neural network and augmented dataset. Systems Science & Control Engineering. 2020. Available from: https://dx.doi.org/10.1080/21642583.2020.1836526
Deng L, Wang Q, Yuan D. Dynamic Facial Expression Recognition Based on Deep Learning. .
Kim JH, Kim BG, Roy PP, Jeong DM. Efficient Facial Expression Recognition Algorithm Based on Hierarchical Deep Neural Network Structure. IEEE Access. 2019;7:41273–41285. Available from: https://dx.doi.org/10.1109/access.2019.2907327
Yue Z, Yanyan F, Shangyou Z, Bing P. Facial Expression Recognition Based on Convolutional Neural Network. In: 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS). .
Filntisis PP, Efthymiou N, Koutras P, Potamianos G, Maragos P. Gerasimos Potamianos, and Petros Maragos. “Fusing Body Posture With Facial Expressions for Joint Recognition of Affect in Child-Robot Interaction. IEEE Robotics and Automation Letters. 2019;4(4):4011–4018. doi: 10.1109/LRA.2019.2930434

Copyright

© 2021 Bodavarapu & Srinivas. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)