Turkish fingerspelling recognition system using axis of least inertia based fast alignment

Abstract

Fingerspelling is used in sign language to spell out names of people and places for which there is no sign or for which the sign is not known. In this work we describe a Turkish fingerspelling recognition system that recognizes all 29 letters of the Turkish alphabet. A single representative frame is extracted from the sign video, since that frame is enough for recognition purposes of the letters mentioned. Processing a single frame, instead of the whole video, increases speed considerably. The skin regions in the representative frame are extracted by color segmentation in YCrCb space before clearing noise regions by morphological opening. A novel fast alignment method that uses the angle of orientation between the axis of least inertia and y axis is applied to hand regions. This method compensates small orientation differences but increases big ones. This is desirable when differentiating the fingerspelling signs, some of which are close in shape but different in orientation. Also the use of minimum bounding square is advised, which helps in resizing without breaking the alignment. Binary values of this minimum bounding square are directly used as feature values, and that allowed experimenting with different classification schemes. Features like mean radial distance and circularity are also used for increasing success rate. Classifiers like kNN, SVM, Naive Bayes, and RBF Network are experimented with, and INN and SVM are found to be the best two of them. The video database was created by 3 different signers, a set of 290 training videos, and a separate set of 174 testing videos are used in experiments. The best classifiers INN and SVM achieved a success rate of 99.43% and 98.83% respectively.