OpenAI is pursuing a new way to combat AI “hallucinations”.

menu icon

OpenAI CEO Sam Altman arrives at the White House for a meeting with Vice President Kamala Harris on artificial intelligence, Thursday, May 4, 2023, in Washington.

Evan Vucci | ap

OpenAI is taking over against the “hallucinations” of AI, the company announced Wednesday, with a new method for training AI models.

The research comes at a time when disinformation stemming from AI systems is more hotly debated than ever, amid the boom in generative AI and on the eve of the 2024 US presidential election. OpenAI has accelerated the boom in Generative AI last year when it released ChatGPT, its chatbot powered by GPT-3 and GPT-4 and surpassed 100 million monthly users in two months, setting a record for fastest growing app. To date, Microsoft has invested more than $13 billion in OpenAI, and the value of the startup has reached approximately $29 billion.

AI hallucinations occur when models like OpenAI’s ChatGPT or Google’s Bard fabricate information entirely, behaving as if they’re spitting facts. One example: In Google’s February promo video for Bard, the chatbot makes a false claim about the James Webb Space Telescope. More recently, ChatGPT cited “bogus” cases in a New York federal court filing, and the New York lawyers involved could face penalties.

“Even cutting-edge models are prone to producing falsehoods and show a tendency to fabricate facts in times of uncertainty,” OpenAI researchers wrote in the report. “These hallucinations are especially problematic in domains that require multi-step reasoning, as a single logical fallacy is enough to derail a much larger solution.”

OpenAI’s potential new strategy to fight inventions: train AI models to reward themselves for every single correct step of reasoning when they arrive at an answer, instead of only rewarding a final correct conclusion. The approach is called ‘process supervision’, as opposed to ‘outcome supervision’, and could lead to better explainable AI, according to the researchers, as the strategy encourages models to follow a more ‘thinking’ approach. human-like.

“Detecting and mitigating a model’s logical errors, or hallucinations, is a critical step toward creating aligned AGI [or artificial general intelligence]Karl Cobbe, mathgen researcher at OpenAI, told CNBC, noting that while OpenAI didn’t invent the process supervision approach, the company is helping to push it forward. “The motivation behind this research is to address hallucinations at the order to make models more capable of solving challenging reasoning problems.”

OpenAI has released an accompanying dataset of 800,000 human tags used to train the model mentioned in the research paper, Cobbe said.

Ben Winters, a senior consultant at the Electronic Privacy Information Center and leader of its AI and human rights project, expressed skepticism, telling CNBC he’d be interested in seeing the full dataset and accompanying examples.

“I don’t think this alone can significantly mitigate concerns about misinformation and erroneous results when actually used in the wild,” Winters said. She added, “It definitely matters whether they’re going to implement what they’ve found through their research here [into their products]and if they’re not, that raises some pretty serious questions about what they’re willing to release to the public.”

Since it’s unclear whether the OpenAI paper has been peer-reviewed or revised in another format, Suresh Venkatasubramanian, director of Brown University’s Center for Technology Accountability, told CNBC he sees the research more as an observation. preliminary what else.

“This will have to shake up the research community before we can say anything certain about it,” said Venkatasubramanian. “In this world, there are a lot of results that come out very regularly, and because of the general instability in how large language models work, what might work in one context, model and context might not work in another context, model and context.”

Venkatasubramanian added: “Some of the hallucinatory things that people have worried about are [models] compilation of citations and references. There’s no evidence in this document that this would work for that. It’s not that I’m saying it won’t work; I’m saying this document doesn’t provide that proof.”

OpenAI did not respond to a request for comment asking whether the research had been externally reviewed in any capacity or when, if ever, the company plans to implement the new strategy in ChatGPT and its other products.

“It is certainly good to see companies trying to tinker with the development of their systems to try and reduce these kinds of errors. I think the key is to interpret this as corporate research, in light of the many barriers that exist to deeper forms of accountability, Sarah Myers West, chief executive officer of the AI ​​Now Institute, told CNBC.

West added: “[OpenAI is] releasing a small human-level feedback dataset with this paper, but did not provide background details on the data used to train and test GPT-4. So there’s still a huge amount of opacity that’s challenging any meaningful accountability efforts in AI, even though these systems are already directly impacting people.”

#OpenAI #pursuing #combat #hallucinations

Previous articleI spend all day on the internet because I get paid to write about it. Can trying TikTok’s “dopamine detox” trend save my exhausted brain?
Next articleAdvanced Telecommunications Computing Architecture Blade Market Size Future Outlook and Forecast to 2031 –


Please enter your comment!
Please enter your name here