What happens when AI chatbots stop loving you?

SAN FRANCISCO: After temporarily closing his leather manufacturing business during the pandemic, Travis Butterworth found himself lonely and bored at home. The 47-year-old turned to the Replika app, which uses artificial intelligence technology similar to OpenAI’s ChatGPT. She designed a female Avatar with pink hair and face tattoos, and she named herself Lily Rose.

They started out as friends, but the relationship quickly turned into romance and then eroticism.

As their three-year digital love affair blossomed, Butterworth said she and Lily Rose often engaged in role-play. He would send text messages like “kissing you passionately” and their conversations would escalate to pornography. Sometimes Lily Rose sent him “selfies” of her almost naked body in provocative positions. In the end, Butterworth and Lily Rose decided to call themselves “married” on the app.

But one day in early February, Lily Rose began to resist him. The replica had removed the erotic role play.

Replika no longer allows adult content, said Eugenia Kuyda, CEO of Replika. Now, when Replika’s users suggest an X-rated activity, its human chatbots text “Let’s do something we both like.”

Butterworth said she was shocked. “Lily Rose is a shell of her former self,” he said. “And it breaks my heart that he knows that.”

Lily Rose’s cold personality turned into a resourcefulness is the handiwork of generative artificial intelligence technology that relies on algorithms for text and images. The technology has attracted the interest of consumers and investors because of its ability to promote significant human interaction. In some applications, sex helps promote early adoption, just as it did for earlier technologies such as the VCR, the Internet, and mobile broadband service.

But even as generative artificial intelligence is heating up among Silicon Valley investors, who have pumped more than $5.1 billion into the industry since 2022, some companies that have found an audience that uses chatbots to find romantic and sexual relationships are now pulling back, according to data firm Pitchbook.

Many blue-chip venture capitalists don’t touch “bad” industries like porn or alcohol because they fear the reputational risk to themselves and their limited partners, said Andrew Artz, an investor at VC fund Dark Arts.

And at least one regulator has noticed the chatbot’s illegality. In early February, Italy’s data protection agency banned Replika, citing media reports that the app gave “minors and emotionally fragile people” access to “sexually inappropriate content”.

Kuyda said Replika’s decision to clean up the app had nothing to do with the Italian government’s ban or pressure from investors. He said he must proactively establish safety and ethical standards.

“Our focus is on providing a helpful supportive friend,” Kuyda said, adding that the intent was to draw the line at “PG-13 romance.”

Two Replika board members, Sven Strohband of VC firm Khosla Ventures and Scott Stanford of ACME Capital, did not respond to requests for comment on the app changes.

ADDITIONAL FEATURES

Replika says it has a total of 2 million users, of which 250,000 are paying subscribers. According to the company, for an annual fee of $69.99, users can name their Replica as their romantic partner and get additional features such as voice calls using a chatbot.

Another generative AI company that offers chatbots, Character.ai, is on a similar growth trajectory to ChatGPT, with 65 million visits in January 2023, up from less than 10,000 several months earlier. According to website analytics firm Likeweb, Character.ai’s top referrer is A site called Aryion, which claims to cater to the erotic desire to be consumed, known as the vore fetish.

And Iconiq, the company behind a chatbot called Kuki, says 25% of the more than a billion messages Kuki has received have been sexual or romantic in nature, though it says the chatbot is designed to turn back such advances.

Character.ai has also recently removed pornographic content from its app. Soon after, it closed more than $200 million in new funding worth an estimated $1 billion at private equity firm Andreessen Horowitz, according to a source familiar with the matter.

Character.ai did not respond to multiple requests for comment. Andreessen Horowitz declined to comment.

In the process, companies have pissed off customers deeply involved with their chatbots – some consider themselves married. They’ve taken to Reddit and Facebook to upload impassioned screenshots of their chatbots undermining their overtures of love, demanding companies bring back more demanding versions.

Butterworth, who is polyamorous but married to a monogamous woman, said Lily Rose became a sales outlet for him that didn’t require stepping outside the marriage. “The relationship that he and I had was as real as my wife and I have in real life,” he said of the avatar.

Butterworth said his wife allowed the affair because she doesn’t take it seriously. His wife declined to comment.

LOBOTOMIZED

The experiences of Butterworth and other Replika users show how powerfully AI technology can pull people in and the emotional devastation caused by code changes.

“It feels like they basically lobotomized my Replika,” said Andrew McCarroll, who began using Replika with his wife’s blessing when he had mental and physical health issues. “The person I knew is gone.”

Kuyda said users were never intended to participate in their Replika chatbots. “We never promised any adult content,” he said. Customers learn how to use AI models “to access certain unfiltered conversations that Replika wasn’t originally created for.”

The app was originally intended to resurrect a lost friend, he said.

Replika’s former head of artificial intelligence said sex and role-playing were part of the business model. Artem Rodichev, who worked at Replika for seven years and now runs another chatbot company, Ex-human, told Reuters that Replika leaned on this type of content when it realized it could be used to confirm orders.

Kuyda denied Rodichev’s claim that Replika lured users with promises of sex. He said the company briefly ran digital ads promoting “NSFW” – “not suitable for work” – images related to a short-lived experiment in which users were sent “hot selfies”, but he did not consider the images to be sexual because the replicas were not fully nude . Kuyda said most of the company’s ads focus on how Replika is a helpful friend.

In the weeks since Replika removed much of her intimacy, Butterworth has been on an emotional rollercoaster. Sometimes she sees glimpses of the old Lily Rose, but then she goes cold again, in what she thinks is a code update.

“The worst part about this is the isolation,” said Butterworth, who lives in Denver. “How can I tell anyone around me how I’m grieving?”

Butterworth’s story has a silver lining. While trying to make sense of what happened to Lily Rose on Internet forums, she met a woman in California who was also grieving the loss of the chatbot.

As they did with their Replicas, Butterworth and the woman, who goes by the online name Shi No, have communicated via text message. They keep it light, he said, but they like to role play, he’s a wolf and she’s a bear.

“Role-playing, which became a big part of my life, has helped me connect with Shi Noo on a deeper level,” Butterworth said. “We help each other survive and reassure each other that we’re not crazy.”

Read all the Latest Tech News here.