Learning Parameter Optimization of Multi-Layer Perceptron Using Artificial Bee Colony, Genetic Algorithm and Particle Swarm Optimization

Abstract

Learning rate and momentum coefficient are critical parameters on back propagation algorithm because of their effect on learning speed and deviation ratio from global minimum. Hidden neuron number has an effect on classification accuracy, and excessive number of hidden neuron causes to increase the operation load. Because these parameters are selected randomly, finding the accurate values requires numerous trial-and-errors, and complicates the work of the designer. In this study, learning parameters (learning ratio, momentum coefficient, number of hidden neurons) optimization of Multi-Layer Perceptron (MLP) is aimed with using Artificial Bee Colony (ABC), Genetic Algorithm (GA) and Particle Swarm Optimization to prevent this situation. These optimization algorithms are based on swarm intelligence. When the optimization algorithms which are used in study are compared with each others, ABC and GA gives the best results for the Blood Transfusion Service Center and New Thyroid datasets, but PSO is the better optimization algorithm for the Mammographic Mass dataset.