GESTURE RECOGNITION FOR CLASSICAL DANCE FORMS USING COMPUTER VISION

Abstract [English]

The symbolic communication in classical dance forms is based on codified hand gestures (mudras), postures and the use of rhythmic sequences of movement. Nevertheless, systematic computational identification of fine-grained dance gestures is not widely studied because subtle articulation variations, costume coverups and lack of annotated data exist. The paper suggests a spatial-temporal deep learning model of gesture recognition in classical dance with computer vision methods. This approach combines skeleton extraction using pose estimation, calculating of joint-angle features to achieve rotational invariance, convolutional neural network (CNN)-based spatial embedding and Long Short-Term Memory (LSTM) based temporal modeling to model dynamic gestures development. The hybrid representation is a mixture between the skeletal precision and the contextual visualization, consequently, granting the opportunity to distinguish visually similar mudras. The experimental evaluation of a curated dataset of classical dance gestures evidence reveals that the provided model will be more successful than the default CNN-only and skeleton-based models and will have more successful results in terms of accuracy, precision, recall, and F1-score. The training and validation analysis is the argument of constant convergence and high level of generalization between performers. The findings confirm that angular skeletal modeling and temporal deep learning are effective in the recognition of fine-grained gestures. In addition to the performance of the classification, the framework can also benefit digital cultural heritage preservation, smart systems of dance tutoring, and AI-based performance analytics. The research study provides a strong base to implement computer vision and the application of deep learning to the systematic study of performing arts.