Blog: Synthetic Media

Vocaloid is a synthesizing software used to produce singing using virtual singers. The marketing around the Vocaloid software relies on several AI idol personas that act as mascots for different voice options. Some of these virtual singers have gained massive fandoms further popularizing the software. Hatsune Miku, a japanese manga-style teen with turquoise hair is by far the most famous. While not actually utilizing AI it has always seemed quite clear that both the company and the massive fandom around this product are invested in realizing a future with AI idols and performers. In 2020, with its most recent release, Vocaloid became one step closer to this. VOCALOID:AI utilizes deep learning to synthesize singing and include unique mannerisms and nuances from a predetermined vocalist. This can be done in any language, with any melodies and lyrics. For its debut this project was used to reproduce the singing of legendary Japanese performer Hibari Misora in commemoration of the 30th anniversary of her death. Using this tech, a likeness of Misora performed a new song live. I’ve included a video of the performance below.

“Actual recordings of the artist's songs and speech made while she was still alive were used as machine learning data to reproduce her singing. [These samples were provided by Misora’s former record label Nippon Columbia.] The singing source data used for machine learning included background musical accompaniment, but Yamaha's accompaniment suppression technology allowed for the generation of high-quality machine learning data to further improve the quality of the singing voice.” --Vocaloid.com

The big ethical question here would be around the rights of a deceased artist. It’s likely the record label does own Misora’s samples and has the legal right to do with them what they wish, but when the technology being used to recreate her didn’t exist at the time she signed the contract it does raise concerns over consent. While nobody who is dead really cares about this, family members, fans, and those with financial investments may raise concerns. With the release of Aaliyah’s music on streaming platforms this year we saw a lot of head butting between her uncle/former manager and her estate. I can’t imagine what sort of conflict there would’ve been if they were trying to reproduce Aaliyah and create new music.