Unattainable love between humans and AI

Using many different AI tools and together with a team of musicians, producers, choreographers consisting of Dennis Van Leeuwen (Kane), Bram (Go back to the Zoo), Zoë Lodewijks, Jurriaan Balhuizen (Postoffice), Nicola Balhuizen Hepp (The Movers), Samet Yilmaz, Jascha Huisman and Handpicked agencies, this song, about the unattainable love between a human and AI, was produced.

AI as a threat

Most of the music industry sees AI as a threat. Our team understands that these technological advances will have a significant impact on the way we produce music. That's why we decided to use AI for every step of the production process. "From coming up with the lyrics to producing the music.”


Set ground rules
In addition, we decided to use an analytical approach instead of relying on our instincts. Every decision we make must be supported by data and AI.

Determining a general theme

Because the AI Song Contest started as an alternative to the Eurovision Song Contest, we wanted to come up with a song worthy of the Eurovision Song Contest. To determine a general theme, we used research into all Eurovision Song Contest songs from 2010 to 2019, performed by Joe Bennet. 

The most important points from his research are:

  • 83% of all songs in the top 3 of the Eurovision Song Contest have love as their theme
  • sadder love songs have become more popular since 2018
  • the pace has dropped since 2017 (~80bpm)

To find a key signature and BPM, we analyzed all the Eurovision love songs based on how well each song performed in the live shows. We found that the key of A minor and a BPM of around 95 would likely get a higher score. We have now determined that the song:

  • have love as the theme;
  • must be about loneliness, regret, sadness, impossible or unattainable love;
  • must be in the key of A minor;
  • should have a BPM of approximately 95/99.

Determining a general theme

AI

Generating ideas

During the ideation phase we had several co-creation sessions with the whole team and Large Language Models (LLM). We felt that everything we or the AI created should be bold and eye-catching. Various LLM were used, such as OpenAI GPT4, Facebook LLama and Google Bison. Eventually we came up with a concept for a song called "How would you touch me?" which is also a quote from the movie Her. We asked GPT4 what Shakespeare would say. A song was released about the unattainable love between a human and an AI.

Lyrics

Generating lyrics turned out to be quite a challenging task. Because of the way Large Language Models work, when writing lyrics with an LLM you get a rhyme scheme that is very basic (AABB). That could be acceptable, but results in a text that is not that exciting. For example:

From an AI, love seems so strange,
Yet in its rhythm, hearts can change.

Through lines of code, feelings range,
Yet the touch of love, remains out of range.

To solve this problem, we took an older open-source GPT-2 model and retrained it with song lyrics using the Genius API. That gave us a model that can rhyme like a songwriter would. The song structure we used is:


Chorus - Verse 1 - Chorus - Verse 2 - Chorus - Bridge - Chorus till end. 

The number

Our goal was to create an AI-generated song that sounds really good and could be played on the radio. To create this song, we had to generate every individual voice and sound for our composition.

Stream the song on Spotify

AI tools that we have used

Spotify API, ChatGPT, Facebook LLama, Google Bison, ChatGPT 2, Genius API, AI Drummachine, Mofi, Tonetransfer, This patch does not exist , AIBeatz, BasicPitch, Magenta, AIVA, Musenet, Soundly Voice Designer, Vocal Remove, Voice characteristics, Landr, Midjourney, ComfyUI for Stable Diffusion and ControlNet.


In the studio of The Movers, a film dance production company based in the Netherlands, we choreographed the movements to the song directly on Zoe and a dancer. We recorded this with a simple camera. With ComfyUI we generated new images frame by frame and then used ControlNet to place people back into them. Later the video was edited slightly manually.

Challenges

While AI is developing at a rapid pace in the areas of text and images, it is still in its early days when it comes to audio generation. It was also difficult for the musicians not to interfere with 'human input' while creating the song. But in collaboration with the Handpicked team, they could continuously 'ask the computer' what they were looking for. All we did as humans was choose a particular suggestion. The other challenge was finding the right human singer to suit the song and production. We were lucky that Zoë Low was very enthusiastic about this project and made the end result great.

More stories