EXPLORING EMOTIONAL EXPRESSION IN DIGITAL ART THROUGH DEEP LEARNING TECHNIQUES
DOI:
https://doi.org/10.29121/shodhkosh.v6.i3s.2025.6952Keywords:
Digital Art, Emotional Expression, Deep Learning, Affective Computing, Emotion Embeddings, Vision Transformers, Valence–Arousal ModelAbstract [English]
The expression of emotion is a characteristic but difficult feature of digital art that is most frequently expressed in abstract visual features instead of direct semantics. This paper explores how deep learning methods can be used to learn and analyze emotional expression in digital artwork. The proposed hybrid model that integrates Convolutional Neural Networks and Vision Transformers will be able to capture local visual features, including color and texture as well as global compositional structure. A selected collection of various digital artworks is modeled and cited by a hybrid emotion system based on discrete categories and dimensional valence-arousal models. The experimental findings prove that the proposed hybrid method is more successful than CNN and transformer baselines on both emotion classification and regression problems with a higher F1-score, reduced error in prediction, and increased correlation with human emotional ratings. Embedding-level and qualitative analyses also indicate that the learned representations are able to maintain emotional continuity as well as ambiguity in artistic expression. The results affirm that emotion in digital art is multidimensional and optimal with regard to integrated local-global feature learning. The presented work contributes to the development of affective computing in the world of creativity and offers a premise to the study of emotional art, curating it, and creative collaboration between humans and AI.
References
De Lope, J., and Grana, M. (2022). A Hybrid Time-Distributed Deep Neural Architecture for Speech Emotion Recognition. International Journal of Neural Systems, 32(5), 2250024. https://doi.org/10.1142/S0129065722500241 DOI: https://doi.org/10.1142/S0129065722500241
Deonna, J., and Teroni, F. (2025). The Creativity of Emotions. Philosophical Explorations, 28(2), 165–179. https://doi.org/10.1080/13869795.2025.2471824 DOI: https://doi.org/10.1080/13869795.2025.2471824
Dupré, D., Krumhuber, E. G., Küster, D., and McKeown, G. J. (2020). A Performance Comparison of Eight Commercially Available Automatic Classifiers for Facial Affect Recognition. PLOS ONE, 15(4), e0231968. https://doi.org/10.1371/journal.pone.0231968 DOI: https://doi.org/10.1371/journal.pone.0231968
Fisher, H., Reiss, P. T., Atias, D., Malka, M., Shahar, B., Shamay-Tsoory, S., and Zilcha-Mano, S. (2023). Facing Emotions: Between- and Within-Sessions Changes in Facial Expression During Psychological Treatment for Depression. Clinical Psychological Science. Advance online publication. https://doi.org/10.1177/21677026231195793 DOI: https://doi.org/10.1177/21677026231195793
Fu, H., et al. (2023). Cross-Corpus Speech Emotion Recognition Based on Multi-Task Learning and Subdomain Adaptation. Entropy, 25(1), 124. https://doi.org/10.3390/e25010124 DOI: https://doi.org/10.3390/e25010124
Galanos, T., Liapis, A., and Yannakakis, G. N. (2021). Affectgan: Affect-Based Generative Art Driven by Semantics. In Proceedings of the 9th International Conference on Affective Computing and Intelligent Interaction Workshops (ACIIW) (1–7). IEEE. https://doi.org/10.1109/ACIIW52867.2021.9666317 DOI: https://doi.org/10.1109/ACIIW52867.2021.9666317
Ghanem, B., Rosso, P., and Rangel, F. (2020). An Emotional Analysis of False Information in Social Media and News Articles. ACM Transactions on Internet Technology, 20(1), 1–18. https://doi.org/10.1145/3381750 DOI: https://doi.org/10.1145/3381750
Krumhuber, E. G., Küster, D., Namba, S., and Skora, L. (2021). Human and Machine Validation of 14 Databases of Dynamic Facial Expressions. Behavior Research Methods, 53(2), 686–701. https://doi.org/10.3758/s13428-020-01443-y DOI: https://doi.org/10.3758/s13428-020-01443-y
Krumhuber, E. G., Skora, L. I., Hill, H. C. H., and Lander, K. (2023). The Role of Facial Movements in Emotion Recognition. Nature Reviews Psychology, 2(5), 283–296. https://doi.org/10.1038/s44159-023-00172-1 DOI: https://doi.org/10.1038/s44159-023-00172-1
Küster, D., Steinert, L., Baker, M., Bhardwaj, N., and Krumhuber, E. G. (2022). Teardrops on My Face: Automatic Weeping Detection from Nonverbal Behavior. IEEE Transactions on Affective Computing. Advance Online Publication. https://doi.org/10.1109/TAFFC.2022.3228749 DOI: https://doi.org/10.1109/TAFFC.2022.3228749
Lin, C., Bulls, L. S., Tepfer, L. J., Vyas, A. D., and Thornton, M. A. (2023). Advancing Naturalistic Affective Science with Deep Learning. Affective Science, 4(4), 550–562. https://doi.org/10.1007/s42761-023-00215-z DOI: https://doi.org/10.1007/s42761-023-00215-z
Lu, C., et al. (2022). Progressively Discriminative Transfer Network for Cross-Corpus Speech Emotion Recognition. Entropy, 24(8), 1046. https://doi.org/10.3390/e24081046 DOI: https://doi.org/10.3390/e24081046
Mendes, C., Pereira, R., Ribeiro, J., Rodrigues, N., and Pereira, A. (2023). Chatto: An Emotionally Intelligent Avatar for Elderly Care in Ambient Assisted Living. In Proceedings of the International Symposium on Ambient Intelligence (Lecture Notes in Networks and Systems, Vol. 770, pp. 93–102). Springer. https://doi.org/10.1007/978-3-031-43461-7_10 DOI: https://doi.org/10.1007/978-3-031-43461-7_10
Radlak, K., and Smolka, B. (2012). A Novel Approach to the Eye Movement Analysis Using a High Speed Camera. In Proceedings of the 2nd International Conference on Advances in Computational Tools for Engineering Applications (ACTEA) (145–150). IEEE. https://doi.org/10.1109/ICTEA.2012.6462854 DOI: https://doi.org/10.1109/ICTEA.2012.6462854
Ren, Z., et al. (2023). VEATIC: Video-Based Emotion and Affect Tracking in Context Dataset. Arxiv Preprint arXiv:2309.06745. https://doi.org/10.1109/WACV57701.2024.00441 DOI: https://doi.org/10.1109/WACV57701.2024.00441
Siddiqui, M. F. H., Dhakal, P., Yang, X., and Javaid, A. Y. (2022). A Survey on Databases for Multimodal Emotion Recognition and an Introduction to the VIRI Database. Multimodal Technologies and Interaction, 6(6), 47. https://doi.org/10.3390/mti6060047 DOI: https://doi.org/10.3390/mti6060047
Swain, M., Routray, A., and Kabisatpathy, P. (2018). Databases, Features and Classifiers for Speech Emotion Recognition: A Review. International Journal of Speech Technology, 21(1), 93–120. https://doi.org/10.1007/s10772-018-9491-z DOI: https://doi.org/10.1007/s10772-018-9491-z
Walker, S. A., Double, K. S., Kunst, H., Zhang, M., and MacCann, C. (2022). Emotional Intelligence and Attachment in Adulthood: A Meta-Analysis. Personality and Individual Differences, 184, 111174. https://doi.org/10.1016/j.paid.2021.111174 DOI: https://doi.org/10.1016/j.paid.2021.111174
Whitehill, J., Serpell, Z., Lin, Y.-C., Foster, A., and Movellan, J. R. (2014). The Faces of Engagement: Automatic Recognition of Student Engagement from Facial Expressions. IEEE Transactions on Affective Computing, 5(1), 86–98. https://doi.org/10.1109/TAFFC.2014.2316163 DOI: https://doi.org/10.1109/TAFFC.2014.2316163
Wu, C.-H., Chuang, Z.-J., and Lin, Y.-C. (2006). Emotion Recognition from Text Using Semantic Labels and Separable Mixture Models. ACM Transactions on Asian Language Information Processing, 5(2), 165–183. https://doi.org/10.1145/1165255.1165259 DOI: https://doi.org/10.1145/1165255.1165259
Yang, Y., et al. (2023). Facial Expression Recognition with Contrastive Learning and Uncertainty-Guided Relabeling. International Journal of Neural Systems, 33(5), 2350032. https://doi.org/10.1142/S0129065723500326 DOI: https://doi.org/10.1142/S0129065723500326
Zhou, E., and Lee, D. (2024). Generative Artificial Intelligence, Human Creativity, and Art. PNAS Nexus, 3(5), pgae052. https://doi.org/10.1093/pnasnexus/pgae052 DOI: https://doi.org/10.1093/pnasnexus/pgae052
Zong, Y., et al. (2022). Adapting Multiple Distributions for Bridging Emotions from Different Speech Corpora. Entropy, 24(9), 1250. https://doi.org/10.3390/e24091250 DOI: https://doi.org/10.3390/e24091250
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Dr. Prashant Wakhare, Dr. Riyazahemed A Jamadar, Dr. Sanjay Bhilegaonkar, Pallavi Mulmule
This work is licensed under a Creative Commons Attribution 4.0 International License.
With the licence CC-BY, authors retain the copyright, allowing anyone to download, reuse, re-print, modify, distribute, and/or copy their contribution. The work must be properly attributed to its author.
It is not necessary to ask for further permission from the author or journal board.
This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.
