NEURAL NETWORK–BASED MODELS FOR GESTURE RECOGNITION AND CHOREOGRAPHIC PATTERN SYNTHESIS

Shraddha Sharma; Bhushankumar Nemade; Sheetal Mahadik; Bijith Marakarkandy; Pravin Jangid; Sandeep Kelkar; P. V. Chandrika

doi:10.29121/shodhkosh.v7.i1s.2026.7197

Authors

Shraddha Sharma Shree L R Tiwari College of Engineering, Mumbai University, India
Bhushankumar Nemade Shree L R Tiwari College of Engineering, Mumbai University, India
Sheetal Mahadik Shree L R Tiwari College of Engineering, Mumbai University, India
Bijith Marakarkandy Department of E-Business, Prin. L.N. Welingkar Institute of Management Development and Research (WeSchool), Mumbai, India
Pravin Jangid Shree L R Tiwari College of Engineering, Mumbai University, India
Sandeep Kelkar Department of E-Business, Prin. L.N. Welingkar Institute of Management Development and Research (WeSchool), Mumbai, India
P. V. Chandrika Department of E-Business, Prin. L.N. Welingkar Institute of Management Development and Research (WeSchool), Mumbai, India

DOI:

https://doi.org/10.29121/shodhkosh.v7.i1s.2026.7197

Keywords:

Gesture Recognition, Computational Choreography, Human–AI Co-Creation, Motion Synthesis, Perceptual Evaluation, Real-Time Interaction

Abstract [English]

The understanding of gestures and the synthesis of choreography can be viewed as two distinct sides of the human-AI interaction problem, which cannot be viewed as complementary and must be addressed through joint modeling of perception, synthesis, and real-time interaction. An interactive multimodal neural architecture consisting of spatial-temporal gesture encoding, latent motion representation learning, and style-conditioned choreography synthesis is proposed to facilitate end-to-end transfer of human movement from sense to expressive synthesized movement. The semantic consistency constraints in joint optimization will be used to ensure consistency between the perceived gesture intent and the synthesized choreography, while an edge cloud deployment approach will be utilized to facilitate interactive latency and energy-efficient execution. The experimental evaluation on benchmark datasets and live co-creative applications demonstrate high recognition accuracy, smooth and diverse motion synthesis, and successful semantic agreement and consistency in co-creating real-time settings. The formal user study also reveals high levels of perceptual realism, sense of expression, usability, and creative satisfaction, which verifies the framework as an excellent collaborative partner and not a passive generative tool. Managerial analysis Networks have lower production costs, scalable deployment opportunities, and therapeutic engagement of benefits in the areas of creative media, rehabilitation, and social robotics. The findings place gesture-based creative AI as a promising foundation of embodied intelligent interaction, and future research directions include the integration of emotion in creative choreography synthesis, adaptive reinforcement learning co-creation, and extreme low-latency edge synthesis

References

Awan, M. J., Rahim, M. S. M., Salim, N., Mohammed, M. A., Garcia-Zapirain, B., and Abdulkareem, K. H. (2021). Efficient Detection of Knee Anterior Cruciate Ligament from Magnetic Resonance Imaging Using Deep Learning Approach. Diagnostics, 11(1), 105. https://doi.org/10.3390/diagnostics11010105 DOI: https://doi.org/10.3390/diagnostics11010105

Chen, M., et aL. (2022). Data Augmentation and Intelligent Fault Diagnosis of Planetary Gearbox using ILoFGAN under extremely limited samples. IEEE Transactions on Reliability, 72(3), 1029–1037. https://doi.org/10.1109/TR.2022.3215243 DOI: https://doi.org/10.1109/TR.2022.3215243

Côté-Allard, U., et al. (2019). Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 27(4), 760–771. https://doi.org/10.1109/TNSRE.2019.2896269 DOI: https://doi.org/10.1109/TNSRE.2019.2896269

Demolder, C., et al. (2021). Recent Advances in Wearable Biosensing Gloves and Sensory Feedback Biosystems for Enhancing Rehabilitation, Prostheses, Healthcare, and Virtual Reality. Biosensors and Bioelectronics, 190, 113443. https://doi.org/10.1016/j.bios.2021.113443 DOI: https://doi.org/10.1016/j.bios.2021.113443

Gourikeremath, G., and Hiremath, R. (2025). Institutional Repositories in Karnataka Universities: Status Assessment, AI-Assisted Framework Development and Future Research Directions. ShodhAI: Journal of Artificial Intelligence, 2(1), 63–75. https://doi.org/10.29121/shodhai.v2.i1.2025.48 DOI: https://doi.org/10.29121/shodhai.v2.i2.2025.48

Jiang, D., et al. (2019). Gesture Recognition Based on Binocular Vision. Cluster Computing, 22(5), 13261–13271. https://doi.org/10.1007/s10586-018-1844-5 DOI: https://doi.org/10.1007/s10586-018-1844-5

Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., and Fu, Y. (2021). Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble. arXiv Preprint arXiv:2110.06161.

Liu, M., et al. (2021). Training with Agency-Inspired Feedback From an Instrumented Glove to Improve Functional Grasp Performance. Sensors, 21(4), 1173. https://doi.org/10.3390/s21041173 DOI: https://doi.org/10.3390/s21041173

Lu, C., Amino, S., and Jing, L. (2023). Data Glove with Bending Sensor and Inertial Sensor Based on Weighted DTW Fusion for Sign Language Recognition. Electronics, 12(3), 613. https://doi.org/10.3390/electronics12030613 DOI: https://doi.org/10.3390/electronics12030613

Lu, N., Wu, Y., and Feng, L. (2019). Deep Learning for Fall Detection: 3D-CNN Combined with LSTM on Video Kinematic Data. IEEE Journal of Biomedical and Health Informatics, 23(1), 314–323. https://doi.org/10.1109/JBHI.2018.2808281 DOI: https://doi.org/10.1109/JBHI.2018.2808281

Mastoi, Q., Memon, M. S., Lakhan, A., Mohammed, M. A., Qabulio, M., Al-Turjman, F., and Abdulkareem, K. H. (2021). Machine Learning-Data Mining Integrated Approach for Premature Ventricular Contraction Prediction. Neural Computing and Applications. https://doi.org/10.1007/s00521-021-05820-2 DOI: https://doi.org/10.1007/s00521-021-05820-2

Mohammed, M. A., et al. (2020). Voice Pathology Detection and Classification Using Convolutional Neural Network Model. Applied Sciences, 10(11), 3723. https://doi.org/10.3390/app10113723 DOI: https://doi.org/10.3390/app10113723

Nangare, V., Sachin, B., Khatri, A., and Mundhe, B. (2025). Deepfake Video Detection Using Neural Networks. International Journal of Recent Advances in Engineering and Technology, 14(1s), 123–128.

Wang, Z. J., Li, Y. J., Dong, L., Li, Y. F., and Du, W. H. (2023). RUL Prediction of Bearing Using Fusion Network Through Feature Cross Weighting. Measurement Science and Technology, 34(10), 105908. https://doi.org/10.1088/1361-6501/acdf0d DOI: https://doi.org/10.1088/1361-6501/acdf0d

Wu, J., Tian, Q., and Yue, J. (2022). Static Gesture Recognition Based on Residual Dual Attention and Cross-Level Feature Fusion Module. Computer Systems and Applications, 31(1), 111–119.

Yan, S., et al. (2023). FGDAE: A New Machinery Anomaly Detection Method Towards Complex Operating Conditions. Reliability Engineering and System Safety, 236, 109319. https://doi.org/10.1016/j.ress.2023.109319 DOI: https://doi.org/10.1016/j.ress.2023.109319

Yedder, H. B., Cardoen, B., and Hamarneh, G. (2021). Deep Learning for Biomedical Image Reconstruction: A Survey. Artificial Intelligence Review, 54(1), 215–251. https://doi.org/10.1007/s10462-020-09861-2 DOI: https://doi.org/10.1007/s10462-020-09861-2

Yuanyuan, S., Yunan, L., Xiaolong, F., Kaibin, M., and Qiguang, M. (2021). Review of Dynamic Gesture Recognition. Virtual Reality and Intelligent Hardware, 3(3), 183–206. https://doi.org/10.1016/j.vrih.2021.05.001 DOI: https://doi.org/10.1016/j.vrih.2021.05.001

Zhao, H., Zhou, Y., Zhang, L., Peng, Y., Hu, X., Peng, H., and Cai, X. (2020). Mixed YOLOv3-Lite : A Lightweight Real-Time Object Detection Method. Sensors, 20(7), 1861. https://doi.org/10.3390/s20071861 DOI: https://doi.org/10.3390/s20071861