A New Approach to Emulate CNN on FPGAs for Real Time Video Processing

Abstract

A new processor architecture implementing the Discrete Time Cellular Neural Networks (DT-CNN) on FPGA is proposed. This architecture intends to process video images real time with 3x3 CNN templates and without the use of an external memory. The absence of the external memory decreases the cost and complexity of the system. The architecture is based on a single pipelined cell which is employed to emulate a CNN with larger number of neurons. The video source must transmit the data to the FPGA in a progressive manner like the VGA, DVI, progressive cameras or digital interfaced CMOS cameras. Interlaced video signals like PAL, SECAM and NTSC are not suitable for this architecture. The video image pixels are processed and outputted simultaneously with the incoming pixel values. The output signal of the system is synchronized to the input signal with some video line latency, where latency is due to the pipelined structure of the system. Proposed architecture is also realized on a Xilinx Virtex-II FPGA in Celoxica RC203 board. The main features of the architecture are: 1) No external memory is required. 2) Computation of each pixel is done in three FPGA clock periods. 3) The number of Enter iterations required for the computation of the output values is determined by the number of implemented processor units (PUs).