OpenAI warns people might become emotionally reliant on its ChatGPT voice mode

OpenAI is warning that some people may become emotionally reliant on its lifelike ChatGPT voice mode.

In a report Thursday, OpenAI released information on the safety work the company conducted on ChatGPT, its popular artificial intelligence tool, and its new voice mode that sounds human.

OpenAI first began rolling out GPT-4o to paid customers last week, CNN first reported.

The company revealed the model during a demonstration in May. It can translate between two speakers during a real-time conversation and detect a human’s emotions based on a selfie.

The company said the new audio technology presents “novel risks,” including speaker identification, unauthorized voice generation and more.

The technology can respond to audio inputs in 232 milliseconds, which the company said is similar to a human’s response time in conversation.

The company said the risks of anthropomorphization, attributing humanlike behaviors to nonhuman entities, is heightened by the tool’s audio capabilities, because it can have more humanlike interactions.

OpenAI said in the report that in its early testing, users used language that may indicate they were forming connections with the technology

“While these instances appear benign, they signal a need for continued investigation into how these effects might manifest over longer periods of time,” the company said.

Users may also form a relationship with the technology that would reduce their need or desire for human interaction. It could benefit lonely individuals, the company said, but negatively impact “healthy relationships.”

The technology’s ability to complete tasks for users and remember details about conversations can create “both a compelling product experience and the potential for over-reliance and dependence,” OpenAI said.

The technology has been compared to the 2013 movie “Her.” Actress Scarlett Johansson, who voiced the similar technology in the movie, took issue with one of OpenAI’s voices, saying it sounded “eerily similar” to her own.