Dynamic and static feature fusion for increased accuracy in signature verification

Abstract

The success rate in offline signature verification studies has reached high and limiting levels recently. However, any increase in this performance is and will be highly valuable in terms of fraud detection. This study assesses the impact of the sound arising from the friction of pen and paper on handwritten signature verification. A dataset was built containing static data from the signature image and dynamic data from the signature sound by taking samples from 75 participants according to different combinations of pen, paper types, and mobile phone models for recording the sounds of the signatures with their internal microphones. It was aimed to increase verification success by fusing dynamic and static features. From the static data, the features are extracted by the LBP and SIFT algorithms. For dynamic data, spectral flux onset envelopes and spectral centroids of audio signals are plotted and converted to image files. Thus, the dynamic data of the signature sound signal became static data and as in the static image of the signature, feature extraction was performed with the LBP and SIFT algorithms. Classification is performed with the OC-SVM algorithm. Moreover, instead of LBP and SIFT features, another verification method with the deep features obtained with a CNN-based model was also proposed and comparatively analyzed. Test results indicate that the aforementioned fusion of these two traits leads to increased signature verification success rates (statistical significance test results are provided), without incurring large costs, considering the sensor availability and acquisition times.