What is an AI Hallucination?

Have you ever had a strange dream that didn’t make any sense? Maybe the pizza in the takeout box started talking to you or a cat with rainbow-striped fur crossed your path. Or maybe something was just slightly off, and you woke up wondering whether it was a dream or it really happened.

When humans create those thoughts, we don’t call them hallucinations. But when a system built on artificial intelligence (AI), such as ChatGPT, responds to a serious question with a response that ranges from inaccurate to silly, we do. In the world of AI, a “hallucination” is an instance in which an AI tool makes up something that isn’t true or doesn’t exist. This article explains how to recognize an AI hallucination, how and why they happen and what to do about them.

 

Introduction to AI and Machine Learning

The terms “AI” and “machine learning” (ML) are a bit vague and maybe a bit scary, so let’s clear up what they are:

  • AI is software—very sophisticated computer code.
  • ML is a subset of AI, and is even more sophisticated computer code.

Okay, so then what is AI really?

To keep it simple, think of AI as a super-smart robot brain. During development, humans give this brain huge amounts of information called “data.”

So. how huge is the data set? Well, let’s start with all the information on the Internet. Everything. Every Facebook and Insta post, every word of every encyclopedia, every book, every song, and then keep going.

You get the picture. It’s a lot of information. And when the brain has all the data it needs, developers say it has been “trained”. Once ‘trained’ itis ready for people to use it to do things that would normally require human intelligence, like organizing some information, analyzing other information, and writing a detailed report about all that information it organized and analyzed. But, unlike a human brain, the AI system didn’t “think” when it did all that work.

Yep, that’s right. It didn’t think. Instead, it predicted what the user—the person who asked it to perform the task—wanted based on the way the user worded their question and all the data the AI system has been trained on, and then put then information together to meet that prediction. And it did it all in a couple of minutes.

ML takes AI a step further because it enables the AI brain to learn from examples. You might be wondering ‘how can something learn if it can’t think?’ Well, to train an ML system to learn how to do something, you provide it with a lot of information about the thing you want it to learn about. For instance, if you want it to learn what a cat looks like, you enter information about cats, such as drawings, photographs, and other types of images of cats, into the system and tell it that all of those images are cats. This information is called a ‘training dataset’. Then, you could enter a cat photo that had never been entered into that system before—a test dataset—and ask the system to describe what it is. The system would be able to correctly identify the object in the photo as a cat because it has been trained to determine (or ‘learned’), what cats look like.

All of this training and learning is possible because of the way AI and ML systems are structured. The structure of their software is described as a “neural network,” but it’s easier to think of the structure as the system’s brain cells. A neural network is made up of lots of tiny parts that work together to understand and learn from the data presented to them. Just like human brains have neurons that help us think, learn, connect information and create memories, neural networks have “nodes” that enable AI/ML systems to make connections between items in their dataset and be trained.

 

Defining AI Hallucination

So, what exactly is an AI hallucination? Imagine you asked an AI system, for instance ChatGPT, to write a report about a famous person from history, but instead of giving you all factual information, it includes statements about spouses and children the person never had, homes in places they never lived, and accomplishments they never achieved. That’s a hallucination.

 

Here are a few funny examples of AI hallucinations:

While these examples are funny because they are so ridiculous and obviously wrong, some hallucinations can be difficult to find and could lead to serious consequences if they are believed by the reader. For instance, recently, lawyers arguing a case   This is usually perfectly acceptable, but in this instance, the problem was those cases did not exist. The AI tool the lawyers used to help with their research “hallucinated” the cases, including names, dates, details, and legal citations. Everything was fake. The lawyers, not realizing the AI tool could do such a thing, did not fact-check the information and ended up being fined thousands of dollars and sanctioned by the court.

 

Causes of AI Hallucinations

It’s important to understand that AI hallucinations are not “machine learning errors.” The system is doing exactly what it has been trained to do, which is to predict and return a response to a question. As explained above, these systems do not think. They do not have a conscience or moral code. They do not understand what a fact is, what truth is, or what falsehoods are. They simply make a prediction about what the user wants in a response based on the question asked, and then the system provides that information. So, when a system returns a hallucination in its response, it is not “broken.” It did what was asked of it: return an answer. The answer just might have included no facts, or it might have misstated the facts. So, it’s best to just be aware that p

Another misunderstanding about these AI systems is based on an assumption that these tools work the same as a search engine, like Google Search or Bing, because the user interface resembles a search engine’s user interface. But there is a very important difference between a search engine and an AI system. This difference is that a search engine returns answers with links to published information that can be viewed and verified, while an AI system returns newly generated information that the user must verify independently because, as noted above, it doesn’t guarantee facts.

 

Now that we’ve covered how AI hallucinations happen, let’s address why they happen. There are several causes, including:

  1. Overfitting: An AI system is limited to using the information it was trained on. If that dataset is small or limited in terms of its topics, the system can use only that information to create predictions. It would be like studying only one book called All About Cats, for a test and then having to answer questions about dogs during the exam.
  2. Biased Data: Similar to Overfitting, if the information used to train the AI system is imbalanced or unfair, the system has to create its predictions based on that data. For example, if is trained on a dataset that only includes information about weather systems in Antarctica, but asked a question about weather in the tropics, it will use the information it has to create a response, which will be factually wrong and probably quite funny.
  3. Complex Neural Networks: As mentioned previously, neural networks are very complicated, and sometimes they can arrange information in odd and unexpected ways. Imagine a huge machine with lots of gears that must spin at specific times for the machine to work properly. Then one gear gets out of sync and the whole machine starts making weird noises. That’s the same sort of glitch that can cause neural network issues.
  4. Expectations vs. Reality: Because AI systems are so good at responding to requests using the same kind of language, it’s easy to think of them as a super-smart friend who always tells the truth. But AI is designed to be good at generating text and images. It is not designed to be accurate. So, a big step toward preventing unpleasant surprises is not to expect AI to always be correct.

 

 

Impact of AI Hallucinations

While AI hallucinations can be funny or even ridiculous, they can also be very serious, harmful, and even dangerous. For instance, if a system is trained on information that contains negative social stereotypes about certain groups of people or professions, and that system is used to make decisions about who can get a loan for a car or a business, that would cause financial harm to some people. When systems that make decisions that can cause physical injury, such as self-driving cars or medical diagnostic systems, are trained on biased data, it can lead to extremely serious outcomes.

But the dangers aren’t limited just to drastic outcomes. For example, let’s say someone heard that something unusual went on at a gathering in their town over the weekend and asked an AI system “What happened in [town] on Saturday?” The system might not have access to recent information, so it will “guess” or even combine unrelated facts to provide an answer. The person receiving that made-up information doesn’t realize it’s fake and doesn’t bother to check the facts. But they still share it on a social platform. Suddenly that fake information is spreading like wildfire, and causes actual issues in that town.

 

Detecting and Mitigating AI Hallucinations

We’ve been discussing things that can go wrong with AI systems but, in general, AI reliability is very, very good and getting better every day. The systems aren’t perfect, though, and until AI systems can be trained to separate fact from fiction and truth from lies, individual users need to understand clearly what the systems can and cannot do. The companies that bring this technology into their firms must also understand how the systems work and how they must be maintained. Here are a few suggestions:

  1. Cleaning Data: Check the data used to train the system for accuracy and fairness to ensure it doesn’t use flawed logic or lead to bad decisions.
  2. Training AI Better: During training, test the AI system to see if it is making up things and then fix its learning process. It’s like making sure a student understands a subject well before a big exam.
  3. Regular Checks: Routine updates and checks can help catch and correct any mistakes before they become big problems—just like regular dental visits can catch cavities when they are small.
  4. Transparency: When companies developing these systems are open about how AI works and how it learns, we can understand its decisions better and spot any errors more easily.

For more detailed ways to make AI more reliable, you can check out AutogenAI’s guide on improving AI content reliability.

 

How AutogenAI is Pioneering in AI Content Reliability

AutogenAI has taken a novel, but very practical approach to reducing hallucinations and making sure its AI systems are reliable. We’re developing new methods of improving how the AI system learns and checks its own answers for accuracy, and have built in features to enable human fact-checking and verification, making our solution a better, easier solution for writers.

AI hallucinations are like the weird dreams our brains have—they don’t always make sense and can be a bit confusing. By understanding how they happen and finding ways to fix them, we can make our interactions with AI better and more trustworthy. Remember, it’s important to use AI wisely, write clear questions, and not expect the answers to always be perfect.

If you want to learn more about how AutogenAI can help with making AI tools more reliable for tasks like writing bids and proposals, contact AutogenAI today.