AI-enhanced music improvisation system

16/01/2026

Here is my new version of AI system for music improvisation. It now involves a local LLM (Llama-3.1:8B) and the drums is routed to a RAVE pretrained neural synth.

12/10/2025

Here a some improvisation with the API of OpenAI. The model is gpt-4.1-mini. The guitarist is Olivier Jambois.

1 AI-Rythmic improviser

Figure 1: The AI is asked to generate a two-bar drum pattern subdivided into sixteenth notes, corresponding to a 32-step grid for six instruments. The tempo, swing, and dynamics for the left and right channels are also requested. A Pure Data patch handles the playback of the data sent by the AI. The drum sounds are based on individual audio samples of each instrument to achieve greater realism.

Improvisation with a free drums playing:

Impro 1
Impro 2
Impro 3
Impro 4

Improvisation with a creative comping requirement

2 AI textural improviser

Figure 2: in the second setup, we designed a multi-effect unit within a Pure Data patch. It includes a standard delay whose controllable parameters are the delay time (50–2000 ms), feedback (0–1), and effect level (0–1). The patch also features a pitch shifter capable of transposing the guitar signal within a range of ±12 semitones.
A second, more sophisticated delay is also implemented. It allows the recording of four-second guitar loops, which can then be transposed or played in reverse, similar to modern delay pedals such as the Digitech PDS8000 or the Red Panda Tensor. The AI controls all these parameters and adjusts them in each sequence according to the guitarist’s playing.