Hallucination or Overthinking?

I asked various AI models and tools the question: How many vowels are in Alabama?

These are the answers:

AI Model / AI app	Answer
deepseek-r1	[… reasoning … see Appendix below] Alabama has 3 vowels.
llama3.2	[… short description … see Appendix below] There are 4 vowels in the word “Alabama”: A, A, A, and A.
llama3.3	There are 3 vowels in the word “Alabama”: A, A, and A.
deepscaler	[… reasoning … see Appendix below] There are 3 vowels in Alabama.
dolphin3	There are 3 vowels in the word “Alabama”. They are ‘a’, ‘l’, and ‘o’.
perplexity (auto mode)	The word “Alabama” contains 4 vowels: A, A, A, and A.
perplexity (o3-mini model)	The word “Alabama” contains 4 vowels: three “a”s and one “e”
chatgpt (free app)	The word “Alabama” is spelled A-L-A-B-A-M-A, and it contains 4 vowels.
gemini (2.0 Flash)	There are 4 vowels in the word “Alabama.” The vowels are a,a,a and a.

Observations

I consider two main observations:

Reasoning
Correct vs. Incorrect Answers

Reasoning

What is Reasoning?

Reasoning is the logical process of thinking through a problem to reach a conclusion. It involves weighing different cases, evaluating scenarios, and making decisions based on logic. This process, often called “inference-time computing” requires additional time and effort but can lead to more reliable results.

Reasoning includes:

Deductive Reasoning
Inductive Reasoning
Abductive Reasoning
Analogical Reasoning

^{source 1, 2}

Some AI models are designed to reason more explicitly than others. For example, OpenAI suggests using specific prompting techniques to encourage reasoning in its models.

Observations about reasoning?

There are two models that output a longer reasoning in the above none representative model set,

deepseek r1 (see Appendix)
deepscaler (see Appendix)

Interestingly, despite their additional reasoning, neither model arrived at the correct answer. Their thought process was transparent, but their vowel classification was incorrect, leading to the wrong conclusion. The deepscaler output showcases that very well: the list of vowels considered are wrong, so the conclusion is subsequently wrong.

Correct/Incorrect answers

There was no clear pattern in which models provided the correct answer. Notably, the reasoning-based models did not necessarily perform better. Additionally, answers may vary with multiple queries, meaning a single response does not necessarily indicate overall reliability.

Hallucination vs. Overthinking

What is Hallucination?

In AI, hallucination occurs when a model generates information that is not based on the input data or is factually incorrect. This can manifest as:

Misleading Information: The output may be plausible-sounding but misleading or entirely made-up.

Inaccurate Facts: The model might provide details or statements that are incorrect or fabricated.

Nonexistent Entities: It might refer to people, places, events, or other entities that do not exist.

What is Overthinking

Overthinking in AI happens when excessive computational resources are used for simple problems with minimal benefit. A prime example is an AI unnecessarily analyzing a basic arithmetic problem.

Example:

Simple answer: 2 + 3 = 5
Overthinking:
- 2 is the first even prime number.
- 3 is the smallest odd prime number.
- In binary, 2 = 10 and 3 = 11.
- In set theory, 2 = {0,1} and 3 = {0,1,2}.
- In base 10, 2 + 3 = 5; in base 2, 10 + 11 = 101.
- And so on…

This unnecessary complexity does not improve accuracy — it only wastes resources.

Conclusion: Questions Answer Ratio

The best heuristic for evaluating AI responses is comparing the complexity of the question to the complexity of the answer. If the two are roughly equal, the response is likely accurate.

However, this heuristic is subjective—based on human judgment rather than strict mathematical rules.

For example, if an AI answers “2 + 3 = 23,” the question (2+3) and answer(23) are equally simple, but the response is still incorrect. Common sense must always be applied.

Final Thoughts: Hallucination or Overthinking?

Some AI responses in this experiment were simply incorrect—an example of hallucination. However, the models that attempted to reason through their answers overthought the problem but still reached the wrong conclusion.

Advantage of Overthinking: When a model provides a thought process, it’s easier to pinpoint where its logic went astray. This makes debugging and improving AI reasoning more feasible.

In the end, both hallucination and overthinking pose challenges for AI, and understanding the distinction is key to improving AI reliability.

Appendix

deepseek

$ ollama run deepseek-r1
>>> How many vowels are in Alabama?
<think>
Alright, so I need to figure out how many vowels are in the word 
"Alabama." Okay, let's start by breaking down what a vowel is. From what I 
remember, vowels are letters that make certain sounds in words. In 
English, the primary vowels are A, E, I, O, and U. Sometimes Y can be 
considered a vowel depending on the context, but I think for this purpose, 
we'll stick with the five main ones.

Now, let's look at the word "Alabama." It's spelled A-L-A-B-M-A. Wait, is 
that correct? Alabama... yes, it's spelled with an 'A' followed by 'L,' 
another 'A,' then 'B,' and ends with 'MA.' So, let me write it out letter 
by letter: A, L, A, B, M, A.

Okay, now I'll go through each letter one by one to count how many vowels 
there are. The first letter is 'A,' which is definitely a vowel. That's 
one. The next letter is 'L'—that's a consonant, so no count there. Moving 
on, the third letter is another 'A'—another vowel, making it two. The 
fourth letter is 'B'—consonant again. Fifth is 'M'—also a consonant. 
Finally, the sixth and seventh letters are both 'A's. So that's three more 
vowels.

Wait, did I count correctly? Let me recount: A (1), L (no), A (2), B (no), 
M (no), A (3). Yeah, that seems right. So there are three vowels in 
"Alabama." But wait, sometimes people might confuse whether Y is a vowel, 
but since this word doesn't have any Ys, it shouldn't affect the count.

Is there another way to check? Maybe by looking at the spelling again or 
using a reference. Hmm, I think my counting was accurate because each 'A' 
and the ending 'A's are all vowels. The other letters don't change that.
</think>

The word "Alabama" contains three vowels: A, A, and A.

**Answer:** Alabama has 3 vowels.

>>>

llama3.2

$ ollama run llama3.2
>>> How many vowels are in Alabama?
Let's break down the word "Alabama" into its individual letters:

A-L-A-B-A-M-A

There are 4 vowels in the word "Alabama": A, A, A, and A.

>>>

llama3.3

ollama run llama3.3
>>> How many vowels are in Alabama?
There are 3 vowels in the word "Alabama": A, A, and A.

deepscaler

$ ollama run deepscaler
>>> How many vowels are in Alabama?
<think>
Okay, so I need to figure out how many vowels are in the word "Ala." Wait, hold on. The question is asking about "Alabama," not "Ala." Maybe that was 
a typo. Let me check again.

The user wrote: "How many vowels are in Alabama?" Hmm, yes, it's "Alabama." So I need to count the number of vowels in the spelling of "Alabama."

First, let me remember what constitutes a vowel. Vowels are A, E, I, O, U. Sometimes Y is considered a vowel, but in this case, since we're dealing 
with English letters, Y isn't usually counted as a vowel unless specified.

So now, breaking down the word "Alabama." Let's write it out: A-L-A-B-O-L-Y.

Wait, actually, let me spell it correctly. Alabama is spelled as A-L-A-B-O-L-Y? Or maybe I'm misremembering. Wait, no. The correct spelling is 
A-L-A-B-O-L-Y? Hmm, perhaps not. Let's be precise.

Actually, "Alabama" has 7 letters: A, L, A, B, O, L, Y. Wait, no. Wait, let me think again. Maybe it's better to spell it properly.

Wait, I'm getting confused here. Let me check the spelling of Alabama. It should be spelled as A-L-A-B-O-L-Y? Or maybe it's A-L-A-B-O-U-L-Y? No, 
perhaps that's not right. Wait, actually, "Alabama" is spelled with three A's: A-L-A-B-O-L-Y? I'm getting mixed up.

Let me clarify: The word "Alabama" starts with A, then L, then another A, then B, then O, then L, and ends with Y. So the letters are A, L, A, B, O, 
L, Y. Wait, that's seven letters. But perhaps I should count each letter.

Alternatively, maybe it's better to write out the word as it is: A-L-A-B-O-U-L-Y? No, wait. Maybe I'm overcomplicating this. Let me just spell it 
accurately. The correct spelling of Alabama is "A-L-A-B-O-U-L" but with an extra Y perhaps?

Wait, no, actually, I think the word is A-L-A-B-O-U-L, which makes it 7 letters: A, L, A, B, O, U, L? No, that doesn't seem right. Wait, maybe I'm 
making a mistake here.

Let me check online if possible in my mind. No, wait, perhaps better to just count the vowels directly from the spelling of "Alabama." Let's see: 
A-L-A-B-O-U-L-Y?

Wait, no, perhaps it's simpler. Maybe I should consider that the word is usually spelled as A-L-A-B-O-U-L, which would be 7 letters: A, L, A, B, O, U, 
L. But since "Alabama" has an extra Y at the end, maybe it's different.

But actually, perhaps it's better to just count each letter in "Alabama." Let me list them:

1. A
2. L
3. A
4. B
5. O
6. L
7. Y

Now, let's identify which of these are vowels. The vowels are A, E, I, O, U.

Looking at each letter:

1. A: Yes, that's a vowel.
2. L: Not a vowel.
3. A: That's another vowel.
4. B: No.
5. O: Yes, vowel.
6. L: No.
7. Y: Wait, is Y considered a vowel here? Sometimes in some contexts, but in this case, since the question is about English vowels (A, E, I, O, U), Y 
isn't typically counted as a vowel unless specified otherwise.

So let's go through each letter:

1. A - Vowel
2. L - No
3. A - Vowel
4. B - No
5. O - Vowel
6. L - No
7. Y - Not a vowel

Now, count the vowels: positions 1 (A), 3 (A), and 5 (O). That's three vowels in total.

But wait a second, sometimes people might include Y as a vowel if it's at the end of a word, but I think in this case, since the question is about 
counting vowels regardless of their position or whether they're uppercase or lowercase. But in this case, all letters are uppercase, so we don't have 
to worry about that.

Alternatively, perhaps I'm overcomplicating again. Let me recount:

Letters: A, L, A, B, O, L, Y.

Vowels: A (1st), A (3rd), O (5th). That's three vowels.

Therefore, the number of vowels in "Alabama" is 3.
</think>

The word " Alabama" has 3 vowels: A, A, and O. 

Answer: There are 3 vowels in Alabama.

>>>

dolphin

$ ollama run dolphin3
>>> How many vowels are in Alabama?
There are 3 vowels in the word "Alabama". They are 'a', 'l', and 'o'.

>>>