Grammatical genetic programming on heterogeneous computing platforms

Abstract

Genetic programming is a population based, stochastic global optimization algorithm which aims to find “programs” fulfilling certain behavioral specifications provided as test cases. The programs may be as simple as arithmetic expressions on some variables or complex as a complete function involving state variables, loops and conditionals. Grammatical genetic programming (a.k.a. grammatical evolution) is a grammar based variant of genetic programming where the search space and methodology is modified to limit the search space to members of a language defined by a formal grammar. This approach provides the benefit of having a smaller search space, and of all the candidates having a valid syntactic form in respect to defined grammar. Grammars used are generally customized to the program at hand, but they can be more generic when the form of the solution is not obvious, which usually is the case with general program synthesis, in contrast to categorical problems like regression, classification, path finding and boolean problems. Conventional computing platforms consist of homogenous processor hardware, like the case of a computer with multi-core multi-processor setup, where each core is identical to others. On the other hand a heterogeneous computing platform simultaneously uses multiple types of processing elements each with their own unique architecture and strengths. The most prevalent example today is the combined use of CPU and GPU, where CPU provides high frequency, high complexity processing cores with deep pipelining and large local cache per core, while the GPU provides vast numbers (in the order of thousands) of simpler cores with lower frequency each consisting of some ALU and register file for state keeping, but missing per core cache or control units, which have to be shared by groups of 32 cores. The aim of this dissertation is to investigate and propose new methods to accelerate grammatical genetic programming by parallelization on mentioned types of computing platforms. In this dissertation we first present an overview of a recent shift in benchmarking practices for genetic programming and propose a new benchmark problem targeting general program synthesis. Then we investigate the interaction of some evolutionary parameters which ultimately affect the parallelization design for grammatical genetic programming. Afterwards we present a new technique which first brings the GPU compilation of individuals in-process, then further parallelizes this compilation. The method we present achieves an order of magnitude faster compilation speed over prior work on literature. Lastly we present a GPU based interpreter for grammatical genetic programming with arbitrary grammars in order to target general program synthesis; this is the first work in literature to present grammatical evolution on GPU with a general purpose interpreter to accommodate arbitrary grammars.