OpenAI is Pursuing a New Way to Fight ‘AI Hallucination’

To combat the “AI Hallucination” problem, OpenAI revealed a new strategy in their research paper wherein ChatGPT will be trained to reward itself upon generating each correct step of reasoning.

ChatGPT, which is the most popular AI chatbot in the world right now has been going through continuous improvements and updates to get better and more trustworthy amidst the “AI hallucinations “concerns. 

OpenAI, the parent of ChatGPT has time and again warned users not to trust the answers generated by the AI chatbot blindly since the chatbot has the ability to fabricate information. To tackle this issue, the company has finally unveiled its latest approach to make ChatGPT a more reliable tool.

“AI hallucinations” is the term that is used when the information provided by the AI chatbot is either misinterpreted or fabricated in such a way that looks like truth or facts. This can be quite dangerous since the information can lead to destructive outcomes.

According to the new research report by OpenAI, the company is going to implement a new strategy wherein the AI chatbot will be thoroughly trained to reward itself every time it processes a correct step of reasoning before giving the final answer. This process is labeled as “process supervision”.

“Even state-of-the-art models are prone to producing falsehoods —they exhibit a tendency to invent facts in moments of uncertainty. These AI hallucinations are particularly problematic in domains that require multi-step reasoning since a single logical error is enough to derail a much larger solution.”, the researchers mentioned in the report.

The company was already implementing the “outcome supervision” approach wherein the AI chatbot is trained to reward itself upon generating a successful correct final outcome. While both approaches are similar, the process supervision approach will enable the AI chatbot to generate more conscious and reliable results. 

Since the supervision will take place at each step by following a human-like-thinking approach, this could prevent the AI system to generate any possible false outcomes caused due to even a single step and lead to a better explainable AI.

A researcher from OpenAI, Karl Cobbe in an interview with CNBC said, “Detecting and mitigating a model’s logical mistakes, or hallucinations, is a critical step towards building aligned AGI [or artificial general intelligence],” He further added, “The motivation behind this research is to address hallucinations in order to make models more capable at solving challenging reasoning problems.”

The company backing its research also revealed a dataset of almost 800,000 human labels that were used to train the AI model. Although it is still not estimated how well the approach will work. While the research by OpenAI is quite commendable, there still prevails concerns about transparency and security while using AI systems.