Real-Time Symbolic Music Accompaniment Generation for Edge Devices
Loading...
Author
Vuagniaux, Rémy
Fragnière, Gaspard
Türetken, Engin
DOI
Abstract
Music generation is a complex and challenging problem that
has made significant progress through recent machine learning solutions.
The challenge lies not only in rendering natural-sounding audio but also
in capturing the underlying musical structure. This paper presents a
real-time system that generates a musical accompaniment for an input
lead melody in the MIDI format. In this paper, we introduce REMI-
Block, a novel tokenization approach, and show that it is suited for ac-
companiment generation. Our method uses the GPT-2 architecture [11],
optimized for efficient on-device performance, to generate accompani-
ments that are rhythmically and harmonically coherent with a musi-
cian’s performance. The model used in this papers reaches a perplex-
ity of 7.69 on the test set, proving the model’s ability to understand
musical language. Additionally, using two metrics - groove and scale
consistency - proposed in [3], we show that the generated accompani-
ments closely match the ground truth, differing by at most 3.4%. This
further highlights our model’s ability to capture the intricacies of mu-
sical language. The model weights and inference code is available at
https://github.com/UncleBen420/JazzyGPT2
Publication Reference
Distributed Computing and Artificial Intelligence, 22nd International Conference (DCAI 2025). In: Lecture Notes in Networks and Systems, vol. X. Springer, (year 2025).
Year
2025-06