REINFORCEMENT LEARNING IN MUSICAL IMPROVISATION

Abstract [English]

As a synthesis of real-time choices, awareness of style and expressiveness, musical improvisation is a complicated creative process. Implementing a conventional generative music system that uses supervised learning will tend to favor the approach that is only adaptive and exploratory of the approach used in improvisational performance. The paper explores how reinforcement learning (RL) can be applied to musical improvisation and presents the task as a sequence decision-making problem where an intelligent agent is trained to produce musically coherent and musical sequences by being exposed to an evaluative environment. The structure of music is represented as a Markov decision process with the states representing melodic intervals, harmonic context and rhythmic patterns, and actions representing the choice of notes, continuity of phrases and strategies of ornamentation. One of the main works offered by this work is the design of multi-objective reward functions which balance tonal and temporal coherence, stylistic and creative novelty. The offered framework of RL-based improvisation is a combination of feature extraction, a learning agent, evaluative feedback modules, and a symbolic output generator. The training is performed with datasets based on professional improvisation recording, with auxiliary rule-based musical constraints, aesthetical feedback by the listener in addition to model based evaluators. Q-learning, Deep Q-Networks, Proximal Policy Optimization and hybrid RL-deep learning models are used to make comparative experiments.