Test Adversarially

Problem

Fluency masks two kinds of failure

AI models are optimized to be helpful and fluent. They produce confident, well-structured output whether or not the underlying claims are accurate.

Users who evaluate output by how it reads - coherent, professional, detailed - miss factual errors, logical gaps, and fabricated specifics that hide inside polished prose. Trusting an output and knowing it is valid are not the same thing. Fluency is a style property, not an accuracy signal. A confidently wrong answer reads exactly like a confidently right one.

There is a second failure mode that is harder to catch: the model can be wrong not because it hallucinated, but because it never had what it needed. Missing context, proprietary data the model couldn't access, or nuance that only exists in your organization's lived experience - these produce confident output that is structurally incomplete. The prose looks right. The reasoning holds together. But a critical variable wasn't in the room.

Adversarial testing surfaces both. Prompting for critique catches fabrications and logical gaps. Prompting for what's missing surfaces the blind spots the model couldn't know to flag on its own.

Value

Errors caught before they leave your hands

Errors caught before they leave your hands instead of after

Decisions grounded in verified claims instead of fluent-sounding assertions

A professional track record of accurate AI-assisted work that builds trust over time

Context

When this pattern applies

Any situation where AI-generated output will be used to inform decisions, shared externally, or relied on by others. Applies to research summaries, code, recommendations, analysis, and any output where errors carry real consequence.

Solution

Prompt the model to attack its own output

After generating output you intend to use, prompt the model to critique it directly. Two prompts that work:

Adversarial Review "Now act as a skeptical expert reviewer. Identify every claim in what you just wrote that could be wrong, outdated, or unsupported. List specific weaknesses, not general caveats."

Critic's Lens "What would a critic say about this analysis? What's missing? What assumptions am I making that might not hold?"

Context Gap Check "What context, data, or organizational nuance would you have needed to give a more accurate or complete answer? What did you have to assume because I didn't provide it?"

Then ask the harder question: after testing, what gives you independent grounds to know this output is valid - not just trust it? That answer comes from domain expertise, source verification, or data you can check directly. Testing surfaces what the model got wrong. Validation is what you bring to the table that the model never had.

Verify flagged items against primary sources. Treat the critique as a list of leads, not a verdict: research from Google DeepMind found that models struggle to correct their own reasoning without outside feedback, and sometimes make it worse (Huang et al., ICLR 2024). Asking for critique gets you candidates to check. Checking them is still your job.

Spot-check what wasn't flagged. Pick two or three claims the review passed over and verify them too. If those fail, the review missed things, and the whole output needs a closer look.

The two-minute version "List the three claims in what you just wrote most likely to be wrong, and what I'd check to confirm each one."

Source: Huang et al., "Large Language Models Cannot Self-Correct Reasoning Yet", Google DeepMind, ICLR 2024.

Notes

Run it in a separate session

Run the adversarial review in a fresh session by default. A new session hasn't "committed" to the first output and will challenge it more freely. Use the same session only when the review needs context the fresh one doesn't have.

What skipping this costs. In October 2025, Deloitte Australia agreed to partially refund a AU$440,000 government report after a researcher found fabricated references and a fabricated court quote. The revised report disclosed that generative AI had been used. (Associated Press, October 7, 2025)

Adversarial testing is active: you prompt for critique in the moment. Building verification into the workflow itself is the structural version, and a candidate for a future pattern.

Pattern contributor Justin J MacBale Contributed two insights that shaped this pattern: that AI output can fail not from hallucination but from missing context, data, or nuance the model never had access to; and that trusting an output and knowing it is valid are not the same thing - a distinction that defines where adversarial testing ends and human validation begins.