1. Home
  2. Vol 8, No 2 (2024)
  3. Goh

Comparison of Activation Functions in Convolutional Neural Network for Poisson Noisy Image Classification

Khang Wen Goh, Sugiyarto Surono, M. Y. Firza Afiatin, K. Robiatul Mahmudah, Nursyiva Irsalinda, Mesith Chaimanee, Choo Wou Onn

Abstract


Deep learning, specifically the Convolutional Neural Network (CNN), has been a significant technology tool for image processing and human health. CNNs, which mimic the working principles of the human brain, can learn robust representations of images. However, CNNs are susceptible to noise interference, which can impact classification performance. Choosing the right activation function can improve CNNs performance and accuracy. This research aims to test the accuracy of CNN with ResNet50, VGG16, and GoogleNet architectures combined with several activation functions such as ReLU, Leaky ReLU, Sigmoid, and Tanh in the classification of images that experience Poisson noise. Poisson noise is applied to each test data to evaluate CNN accuracy. The data used in this study consists of three scenarios of different numbers of classes, namely 3 classes, 5 classes, and 10 classes. The results showed that combining ResNet50 with the ReLU activation function produced the best performance in class recognition in each scenario of the number of classes experiencing Poisson noise interference. The model achieved 97% accuracy for 3-class data, 95% for 5-class data, and 90% for 10-class data. These results show that using ResNet50 with the ReLU activation function can provide excellent resistance to Poisson noise in image processing. It was found that as the number of classes increases, the accuracy of image recognition tends to decrease. This shows that the more complex the image classification task is with a larger number of classes, the more difficult it is for CNNs to distinguish between different classes.

 

Doi: 10.28991/ESJ-2024-08-02-014

Full Text: PDF


Keywords


Activation Function; Classification; Convolutional Neural Network; Poisson Noise.

References


Naranjo-Torres, J., Mora, M., Hernández-García, R., Barrientos, R. J., Fredes, C., & Valenzuela, A. (2020). A review of convolutional neural network applied to fruit image processing. Applied Sciences (Switzerland), 10(10), 3443. doi:10.3390/app10103443.

Chauhan, R., Ghanshala, K. K., & Joshi, R. C. (2018). Convolutional Neural Network (CNN) for Image Detection and Recognition. 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC), Jalandhar, India. doi:10.1109/icsccc.2018.8703316.

Soydaner, D. (2020). A Comparison of Optimization Algorithms for Deep Learning. International Journal of Pattern Recognition and Artificial Intelligence, 34(13), 2052013. doi:10.1142/S0218001420520138.

Voulodimos, A., Doulamis, N., Doulamis, A., & Protopapadakis, E. (2018). Deep Learning for Computer Vision: A Brief Review. Computational Intelligence and Neuroscience, 2018. doi:10.1155/2018/7068349.

Mohammed, N. A., Abed, M. H., & Albu-Salih, A. T. (2022). Convolutional neural network for color images classification. Bulletin of Electrical Engineering and Informatics, 11(3), 1343–1349. doi:10.11591/eei.v11i3.3730.

Badža, M. M., & Barjaktarović, M. C. (2020). Classification of brain tumors from MRI images using a convolutional neural network. Applied Sciences (Switzerland), 10(6), 1999. doi:10.3390/app10061999.

Zhao, J., & Li, L. (2021). Research on image classification algorithm based on convolutional neural Network-TensorFlow. Journal of Physics: Conference Series, 2010(1), 12096. doi:10.1088/1742-6596/2010/1/012096.

Ashraf, S., Kadery, I., Chowdhury, A. A., Mahbub, T. Z., & Rahman, R. M. (2019). Fruit Image Classification Using Convolutional Neural Networks. International Journal of Software Innovation, 7(4), 51–70. doi:10.4018/IJSI.2019100103.

Narin, A., Kaya, C., & Pamuk, Z. (2021). Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks. Pattern Analysis and Applications, 24(3), 1207–1220. doi:10.1007/s10044-021-00984-y.

Ashhar, S. M., Mokri, S. S., Rahni, A. A. A., Huddin, A. B., Zulkarnain, N., Azmi, N. A., & Mahaletchumy, T. (2021). Comparison of deep learning convolutional neural network (CNN) architectures for CT lung cancer classification. International Journal of Advanced Technology and Engineering Exploration, 8(74), 126–134. doi:10.19101/IJATEE.2020.S1762126.

ERDEM, E., & AYDİN, T. (2021). Detection of Pneumonia with a Novel CNN-based Approach. Sakarya University Journal of Computer and Information Sciences, 4(1), 26–34. doi:10.35377/saucis.04.01.787030.

Xu, Z., Guo, X., Zhu, A., He, X., Zhao, X., Han, Y., & Subedi, R. (2020). Using deep convolutional neural networks for image-based diagnosis of nutrient deficiencies in rice. Computational Intelligence and Neuroscience, 7307252. doi:10.1155/2020/7307252.

Li, Q., Shen, L., Guo, S., & Lai, Z. (2020). Wavelet Integrated CNNs for Noise-Robust Image Classification. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, United States. doi:10.1109/cvpr42600.2020.00727.

Pei, Y., Huang, Y., Zou, Q., Zhang, X., & Wang, S. (2021). Effects of Image Degradation and Degradation Removal to CNN-Based Image Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4), 1239–1253. doi:10.1109/TPAMI.2019.2950923.

Busch, N. A., & Micheli-Tzanakou. (1989). Neural network design: methodology. International Joint Conference on Neural Networks, Washington, United States. doi:10.1109/ijcnn.1989.118439.

Wang, Y., Li, Y., Song, Y., & Rong, X. (2020). The influence of the activation function in a convolution neural network model of facial expression recognition. Applied Sciences (Switzerland), 10(5), 1897. doi:10.3390/app10051897.

Fausett, L. V. (2006). Fundamentals of neural networks: architectures, algorithms, and applications. Pearson Education India, Noida, India.

Momeny, M., Latif, A. M., Agha Sarram, M., Sheikhpour, R., & Zhang, Y. D. (2021). A noise robust convolutional neural network for image classification. Results in Engineering, 10, 100225. doi:10.1016/j.rineng.2021.100225.

Phaladisailoed, T., & Numnonda, T. (2018). Machine Learning Models Comparison for Bitcoin Price Prediction. 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE), Bali, Indonesia. doi:10.1109/iciteed.2018.8534911.

Hendarto, J. (2017). Denoising Grayscale Images Using the Fractal Method. PRODUKTIF: Jurnal Ilmiah Pendidikan Teknologi Informasi, 1, 1-12. (In Indonesian).

Owotogbe, J. S., Ibiyemi, T. S., & Adu, B. A. (2019). A comprehensive review on various types of noise in image processing. International Journal of Scientific & Engineering Research, 10(11), 388-393.

Thakur, R. S., Chatterjee, S., Yadav, R. N., & Gupta, L. (2021). Image De-Noising with Machine Learning: A Review. IEEE Access, 9, 93338–93363. doi:10.1109/ACCESS.2021.3092425.

Desai, M., & Shah, M. (2021). An anatomization on breast cancer detection and diagnosis employing multi-layer perceptron neural network (MLP) and Convolutional neural network (CNN). Clinical EHealth, 4, 1–11. doi:10.1016/j.ceh.2020.11.002.

Patil, A., & Rane, M. (2021). Convolutional Neural Networks: An Overview and Its Applications in Pattern Recognition. Smart Innovation, Systems and Technologies, 195, 21–30. doi:10.1007/978-981-15-7078-0_3.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). A Book on Deep Learning. MIT Press Book, Cambridge, United States.

Zhou, Z., Yuan, H., & Cai, X. (2023). Rock Thin Section Image Identification Based on Convolutional Neural Networks of Adaptive and Second-Order Pooling Methods. Mathematics, 11(5), 1245. doi:10.3390/math11051245.

Monnet, M., Gebran, H., Matic-Flierl, A., Kiwit, F., Schachtner, B., Bentellis, A., & Lorenz, J. M. (2023). Pooling Techniques in Hybrid Quantum-Classical Convolutional Neural Networks. 2023 IEEE International Conference on Quantum Computing and Engineering (QCE), Bellevue, United States. doi:10.1109/qce57702.2023.00074.

Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8(1), 53. doi:10.1186/s40537-021-00444-8.

Rafidison, M. A., Ramafiarisona, H. M., Randriamitantsoa, P. A., Rafanantenana, S. H. J., Toky, F. M. R., Rakotondrazaka, L. P., & Rakotomihamina, A. H. (2023). Image Classification Based on Light Convolutional Neural Network Using Pulse Couple Neural Network. Computational Intelligence and Neuroscience, 2023, 1–17. doi:10.1155/2023/7371907.

Abdulkadirov, R., Lyakhov, P., & Nagornov, N. (2023). Survey of Optimization Algorithms in Modern Neural Networks. Mathematics, 11(11), 2466. doi:10.3390/math11112466.

Kinsley, H. & Kukieła, D. (2020). Neural Networks from Scratch in Python. Available online: https://nnfs.io/ (accessed on March 2024).

Mao, A., Mohri, M., & Zhong, Y. (2023). Cross-entropy loss functions: Theoretical analysis and applications. International Conference on Machine Learning, 23-29 July, 2023, Honolulu, United States.

Singarimbun, R. N., Nababan, E. B., & Sitompul, O. S. (2019). Adaptive Moment Estimation to Minimize Square Error in Backpropagation Algorithm. 2019 International Conference of Computer Science and Information Technology (ICoSNIKOM), Medan, Indonesia. doi:10.1109/icosnikom48755.2019.9111563.

Kaggle. (2021). miniImageNet: re-processed miniImageNet dataset for few-shot learning research. Kaggle, San Francisco, United States. Available online: https://www.kaggle.com/datasets/arjunashok33/miniimagenet (accessed on March 2024).

Deepak, S., & Ameer, P. M. (2019). Brain tumor classification using deep CNN features via transfer learning. Computers in Biology and Medicine, 111, 103345. doi:10.1016/j.compbiomed.2019.103345.


Full Text: PDF

DOI: 10.28991/ESJ-2024-08-02-014

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Khang Wen Goh, Sugiyarto Surono, Muhammad Yahya Firza Afiatin, Kunti Robiatul Mahmudah, Mesith - Chaimanee, Choo Wou Onn