A Stochastic Computing Method For Generating Activation Functions in Multilayer Feedforward Neural Networks

Abstract

Stochastic computing using basic arithmetic logic elements based on stochastic bit sequences provides very beneficial solutions in terms of speed and hardware cost, relative to deterministic calculation. Studies for the realization of tangent hyperbolic and exponential functions used in the development of activation functions in Artificial Neural Networks by stochastic methods exist in the literature. The techniques presented using state transitions on finite state machines were constructed on the basis of two different forms of finite state machines, one-dimensional (Linear) and two-dimensional. In this analysis, in terms of both error rate and circuit cost, the advantageous two-dimensional finite state machines based stochastic computing approach for tangent hyperbolic and exponential functions is presented. The presented approach is implemented on Field Programmable Gate Array and the results are given for hardware simulation. The dataset used for the classification process in a decentralized smart grid control has been applied to the multilayer feedforward neural network and deterministic computing, for the stability classification which is carried out separately with the linear finite state machines based stochastic computing and the proposed 2D finite state machines based stochastic computing methods.