Performance Analysis of Different Optimizers, Batch Sizes, and Epochs on Convolutional Neural Network for Image Classification

Abstract

The important thing in the field of deep learning is to find out the appropriate hyper-parameter for image classification. In this study, the main objective is to investigate the performance of various hyper-parameters in a convolutional neural network model based on the image classification problem. The dataset was obtained from the Kaggle dataset. The experiment was conducted through different hyper-parameters. For this proposal, Stochastic Gradient Descent without momentum (SGD), Adaptive Moment Estimation (Adam), Adagrad, Adamax optimizer, and the number of batch sizes (16, 32, 64, 120), and the number of epochs (50, 100, 150) were considered as hyper-parameters to determine the losses and accuracy of a model. In addition, Binary Cross-entropy Loss Function (BCLF) was used for evaluating the performance of a model. In this study, the VGG16 convolutional neural network was used for image classification. Empirical results demonstrated that a model had minimum losses obtain by Adagrad optimizer in the case of 16 batch sizes and 50 epochs. In addition, the SGD with a 32 batch sizes and 150 epochs and the Adam with a 64 batch sizes and 50 epochs had the best performance based on the loss value during the training process. Interestingly, the accuracy was higher while performing the Adagrad and Adamax optimizer with a 120 batch sizes and 150 epochs. In this study, the Adagrad optimizer with a 120 batch sizes and 150 epochs performed slightly better among those optimizers. In addition, an increasing number of epochs can improve the performance of accuracy. It can help to create a broader scope for further experiments on several datasets to perceive the suitable hyper-parameters for the convolutional neural network. Dataset: https://www.kaggle.com/c/dogs-vs-cats/data