A Hybrid CGAN-FNN Framework for Robust Hand Gesture Recognition in Real-Time Systems

Abstract

This study proposes a novel approach that combines Conditional Generative Adversarial Network (CGAN) and feedforward Neural Network (FNN) architectures to generate high-variability, balanced sEMG datasets targeting enhancing NN classification performance while ensuring its applicability to real-time operating systems. The low-quality nature of sEMG datasets, influenced by factors such as muscle fatigue, noise, and physiological differences among patients, negatively impact the performance of both probabilistic models and NN models, making data generation a crucial task. To evaluate the impact of CGAN-generated data on classification performance, Naive Bayes (NB) and Gaussian Mixture Model (GMM) are employed as comparative classification methods, providing insights into the performance changes of the proposed FNN model. The developed CGAN model is designed to generate 1,000 new fake data samples for each of the five fundamental hand gestures (extension, flexion, fist, rest, and spread) by conditioning the generator and discriminator models with the class labels from the utilized real dataset. A normal distribution filter was applied to the generated dataset to enhance its real-data likeness. As the result of hyperoptimization and sliding window application, the proposed FNN model demonstrated real-time compatible high classification results, including but not limited to 88% classification accuracy with the usage of only 6 layers in its single hidden layer, within an inference time of 6 milliseconds. This classification result is 14.3 % better than probabilistic methods tested under the same conditions.