When AI Gets It Wrong: What Auditors Must Know—and Why Skepticism Still Matters

Artificial intelligence (AI) is rapidly becoming part of the auditor’s toolkit. It can process thousands of transactions in seconds, detect outliers that might go unnoticed by humans, and even suggest conclusions based on large volumes of data. On paper, it sounds like the perfect assistant. But in practice, things are not always that simple.

As auditors, we know that trust is earned, not assumed. And just because a machine produces information quickly doesn’t mean it’s always right. It’s in this fast-paced, data-driven environment that professional skepticism—that essential mindset of critical thinking—becomes more important than ever.

When AI “hallucinates”: What does that mean?

You may have heard the term hallucination in conversations about AI. In this context, it doesn’t mean seeing visions. It refers to instances when an AI system provides a confident answer that’s not accurate. The machine isn’t lying on purpose—it doesn’t even know it’s wrong. It’s just filling in the blanks with something that sounds plausible.

Imagine asking an AI tool to explain a transaction it doesn’t recognize. Instead of saying “I don’t know,” it might invent a reason or fabricate a description. That’s a hallucination. In an audit, this kind of response can be dangerous if it’s taken at face value.

So, can AI be trusted?

Not on its own. AI doesn’t have intent—it doesn’t plan to deceive. But that’s exactly why we must be careful. AI models follow patterns based on the data they were trained on and the instructions we give them. If either of those is unclear, incomplete, or biased, the results can be misleading.

This is where the role of the auditor is irreplaceable. It’s our job to ask questions, to double-check, and to investigate further when something doesn’t add up—even if the data came from a highly advanced system.

The “black box” problem

Many AI tools, especially the most powerful ones, are what we refer to as black boxes. You feed in data, and you get an output—but you don’t know how the system arrived at that result. That’s a problem in audit work, where transparency and traceability are essential.

We need to be able to explain how we reached our conclusions, especially when dealing with regulators, boards, or stakeholders. If we can’t trace the logic behind the AI’s result, we risk basing our opinion on something we don’t fully understand.

A new skillset for auditors

This doesn’t mean we should reject AI. On the contrary, we should learn how to use it wisely. But it does mean we need to expand our skills—not just in accounting standards like IFRS (International Financial Reporting Standards) or audit procedures under the ISAs (International Standards on Auditing), but also in areas like:

• How AI systems process information
• How to detect when an AI tool is “guessing”
• How to ask the right questions to get reliable outputs
• How to explain our use of AI to clients and regulators

I often tell my team: “AI doesn’t replace your judgment—it tests it.” And I refer to professional skepticism as colmillo, that instinct we develop over time that tells us when something doesn’t feel quite right.

Why do some developers resist controls?

Here’s something many people outside our profession don’t realize: some AI developers actively avoid adding controls to their systems. Why? Because they fear those controls might slow things down or reduce performance. But in audit, speed should never come at the cost of quality. We have to hold the line.

Final thoughts

AI is a powerful tool, but it is not a shortcut. It can enhance our work—but only if we stay engaged, critical, and informed. The more we rely on automation, the more essential it becomes to use our human judgment.

As auditors, our greatest strength is not how fast we work—it’s how well we think. In a world of artificial intelligence, human skepticism remains our best defense.

---

Jorge Gutiérrez Guillén
Authorized Public Accountant – Managing Partner
JGutierrez Auditores Consultores S.A.
📩 [email protected] | 📞 +506 8811-5090
🌐 www.consultoresjg.com

#AuditingInTheAIera #ProfessionalSkepticism #HumanJudgmentMatters #AIandAudit #EthicalAuditing