AI Company Develops Digital Replicas of Voice Actors

Cissy Jones, a highly credited voice actor known for her work in popular video games such as Starfield and Baldur’s Gate III, has portrayed numerous fictional mishaps throughout her career. According to Jones, dying from a gunshot is relatively straightforward, while death by fire can be a prolonged and tiresome experience. In one particular instance, she portrayed an astronaut in space who woke up with a sense of unease, and after just ten minutes into the scene, Jones admits to nearly losing consciousness.

“Imagine your salary depends on whether you can sustain it,” Jones said

If Jones and his business partners are successful, he may not need to someday soon. That kind of work, he said, is simply outsourced to his “digital twin,” the AI powerhouse.

We are now on WhatsApp. Click to join.

Jones is co-founder and vice president of strategic partnerships at Morpheme, a startup that aims to harness artificial intelligence to edit vocal performances in everything from animated series to video games. It’s the kind of project that’s recently excited tech investors — and creative professionals of all kinds, from screenwriters to actors. Workers in the video game industry worry that AI could eliminate entire job categories, such as quality assurance reviewers.

Morpheme’s artificial intelligence software records actors’ voices and then creates a model of their voices that can be used to change, expand and enliven future productions. More recently, Morpheme has pitched the technology to entertainment companies, including several top gaming companies, which it declined to identify, citing non-disclosure agreements.

Among other things, Morpheme develops a library of stress sounds, such as heavy breathing or shouting, which customers can reuse throughout the development process. That way, actors like Jones don’t have to push themselves so hard, and the producers can get all the bloody screams they need. The software is also able to reproduce the performer’s voice in other languages, making it cheaper and faster for companies to translate products from different cultures.

Morpheme is entering an increasingly crowded market. In 2022, investors invested $378.6 million in speech-related AI startups across 47 deals, according to data from PitchBook. Some use AI to clone voices, change voices, and create voice from text.

Recently, unions have raised urgent concerns about how entertainment giants will incorporate such AI technology. It remains contentious in the ongoing Hollywood actors’ strike, and it’s flaring up on another front as well.

In September, video game voice actors from the SAG-AFTRA union approved a strike amid ongoing negotiations with top game companies such as Electronic Arts Inc., Take-Two Interactive Software Inc. and Activision Blizzard Inc. About 2,600 performers work under Interactive Media. agreement. Artificial intelligence has been a sticking point. Actors fear that entertainment companies are using artificial intelligence to reproduce their voices without permission or payment, reducing the value of their work.

“The unregulated use of artificial intelligence poses an existential threat to anyone who makes a living from their voice, image or performance,” union director Ray Rodriguez said.

Jones, for one, believes he and his colleagues would be better off embracing the technology, albeit under the right circumstances. “It’s not going backwards,” he said. “It will get better, cheaper and faster.”

Privately-funded Morpheme, which is looking for additional investors, supports the idea that amid growing anxiety and potential workplace disruptions, entertainment companies are embracing some rules of engagement. Currently, Morpheme’s more than 20 employees are in regular discussions with voice actors, Hollywood agents, entertainment companies and SAG-AFTRA, hoping to find a middle ground between competing interests.

A SAG-AFTRA spokesperson said in a statement that the union has had “ongoing discussions” with Morphem, noting that “the use of any digital reproduction” must include a set of provisions, including “safe storage” and “proper compensation.” A spokesperson for the game companies negotiating the interactive media contract did not comment on the matter.

A key part of Morpheme’s sales pitch is that it has created a fair salary model for actors. Under a broad and flexible agreement, Morpheme lets performers agree to how their AI-engineered voice is used and compensates them for its continued use with a license fee. After about a year, they have the option to renew or renegotiate. To keep score, Morpheme is also developing technology to track and monitor how companies use AI-generated or edited audio.

Recently, Morpheme joined the National Association of Voice Actors (NAVA) in a new campaign that includes “consent, compensation and control” as a mantra for the use of artificial intelligence.

“If we humans mess this up, we’re going to destroy ourselves,” said Morpheme founder and CEO Brett Shapiro, who has worked in tech and entertainment for decades, including a stint at game company Cosmic Forces.

Unlike their relatives in film or television, video game voice actors do not receive residual payments after their recording sessions. Now, some gaming operators are seeing the emerging artificial intelligence technology as an opportunity to collect additional fees along the way beyond the basic minimum. According to Morphem’s deal, actors who are unavailable or unable to work on a new project can put their “digital twin” to work and receive extra cash in return.

“For me as a voice actor, the licensing opportunity is to get paid to create that voice,” said Tim Friedlander, founder and president of NAVA. “The more my character becomes popular and people interact with it, the more I get compensated.”

At the same time, unauthorized use of artificial intelligence technology is already on the rise.

“We see people ripping off publicly available content to create whatever they want,” Jones said.

Jones has noticed fans on TikTok mixing up her previous scenes with words she “never said before, very clearly a robot.” Some creators have complied with his takedown request, others have not.

“There’s no consideration of the person behind the original art,” he said.

One more thing! ReturnByte is now on WhatsApp channels! Follow us by clicking the link to never miss any updates from the world of technology. Click here to join now!