Scalable Curriculum Learning for Artificial Neural Networks

Abstract

Learning process of people usually starts with easy samples and goes towards hard ones. Using this method for machine learning is called curriculum learning. Samples are given in an order related to their difficulty level, rather than in random order. The aim of this approach is to create models that have better generalization performance. In existing studies, difficulty levels of the samples were determined by prior knowledge and given to the system. However, this is not a scalable approach for every application. Because of that, such studies were usually carried out in very limited application areas. In this study, a new approach is proposed that automatically generates difficulty levels of the samples from data sets. In this way, it is possible to overcome mentioned constraint in the implementations. Thus, curriculum and anti-curriculum learning methods could be applied on many different application areas. In the experiments where artificial neural networks are used as learners, more successful results were obtained with curriculum and anti- curriculum learning compared with the models where samples were given in random order. After various methods have been tried for determining the difficulty ratings of the samples, this study showed that ensemble learning-based approach is more successful.