When it comes to estimating how good we are at something, research consistently shows that we tend to rate ourselves as slightly better than average. This tendency is stronger in people who perform low on cognitive tests. It’s known as the Dunning-Kruger Effect (DKE) –– the worse people are at something the more they tend to overestimate their abilities and the “smarter” they are, the less they realise their true abilities.

However, a study led by Aalto University reveals that when it comes to AI, specifically, Large Language Models (LLMs), the DKE doesn’t hold, with researchers finding that all users show a significant inability to assess their performance accurately when using ChatGPT. In fact, across the board, people overestimated their performance. On top of this, the researchers identified a reversal of the Dunning-Kruger Effect –– an identifiable tendency for those users who considered themselves more AI literate to assume their abilities were greater than they really were.

‘We found that when it comes to AI, the DKE vanishes. In fact, what’s really surprising is that higher AI literacy brings more overconfidence,’ says Professor Robin Welsch. ‘We would expect people who are AI literate to not only be a bit better at interacting with AI systems, but also at judging their performance with those systems – but this was not the case.’

The finding adds to a rapidly growing volume of research indicating that blindly trusting AI output comes with risks like ‘dumbing down’ people’s ability to source reliable information and even workforce de-skilling. While people did perform better when using ChatGPT, it’s concerning that they all overestimated that performance.

‘AI literacy is truly important nowadays, and therefore this is a very striking effect. AI literacy might be very technical, and it’s not really helping people actually interact fruitfully with AI systems’, says Welsch.

‘Current AI tools are not enough. They are not fostering metacognition [awareness of one’s own thought processes] and we are not learning about our mistakes,’ adds doctoral researcher Daniela da Silva Fernandes. ‘We need to create platforms that encourage our reflection process.’

The article was published on October 27th in the journal Computers in Human Behavior.

The researchers designed two experiments where some 500 participants used AI to complete logical reasoning tasks from the US’s famous Law School Admission Test (LSAT). Half of the group used AI and half didn’t. After each task, subjects were asked to monitor how well they performed –– and if they did that accurately, they were promised extra compensation.

‘These tasks take a lot of cognitive effort. Now that people use AI daily, it’s typical that you would give something like this to AI to solve, because it’s so challenging’, Welsch says.

The data revealed that most users rarely prompted ChatGPT more than once per question. Often, they simply copied the question, put it in the AI system, and were happy with the AI’s solution without checking or second-guessing.

‘We looked at whether they truly reflected with the AI system and found that people just thought the AI would solve things for them. Usually there was just one single interaction to get the results, which means that users blindly trusted the system. It’s what we call cognitive offloading, when all the processing is done by AI’, Welsch explains.

This shallow level of engagement may have limited the cues needed to calibrate confidence and allow for accurate self-monitoring. Therefore, it’s plausible that encouraging or experimentally requiring multiple prompts could provide better feedback loops, enhancing users’ metacognition, he says.

So what’s the practical solution for everyday AI users?

‘AI could ask the users if they can explain their reasoning further. This would force the user to engage more with AI, to face their illusion of knowledge, and to promote critical thinking,’ Fernandes says.

IMAGE CREDIT: ThisIsEngineering