Models Are Not Good at Math

Problem

Confident numbers that are quietly wrong

Language models generate text. They are quite good at recalling and restating numbers - a cost figure from a document, a headcount from a table, a metric from a report. What they are not good at is using multiple numbers together to calculate something new. When you ask an AI to compute a budget variance, a resource utilization rate, or a weighted score across a portfolio, it produces a result that looks like a calculation but is not. It is a prediction of what a plausible answer might look like, generated the same way it generates a sentence.

The danger is not that the model refuses or hedges. The danger is that it answers with full confidence. It shows its work. The steps look reasonable. And the final number is wrong in a way that is nearly impossible to spot without independently verifying the math - because the model was never actually calculating. It was generating text that resembles a calculation.

In portfolio management, these errors compound. Consider a program manager who asks AI to calculate resource utilization across eight active projects. The model returns a confident 73% figure with a clean breakdown by project. It looks right. It gets pasted into the steering deck. Three weeks later, a resource manager flags that two projects are badly understaffed - the real number was 91%. The scope approvals, new project intake decisions, and staffing freezes made at that steering meeting were all built on a number the model generated but never actually calculated.

Value

Numbers you can stake a decision on

Calculations are performed by a deterministic engine, not a probabilistic one - the same inputs produce the same outputs every time

The code is auditable - anyone can read the logic, challenge an assumption, and rerun it with different inputs

The calculation becomes a reusable asset - run it next quarter with updated data, not a one-time answer you can't reproduce

Context

When this pattern applies

Any time you are asking AI to produce a number that will inform a decision, this pattern applies. That includes budget calculations, resource allocation models, ROI estimates, schedule variance analysis, weighted scoring of options, and any aggregation across a data set.

It applies regardless of whether you think the math is simple. Simple arithmetic is where AI errors are hardest to catch - the steps look right, the magnitude feels right, and the error is small enough to survive casual review but large enough to matter when the stakes are high.

It does not apply when you are asking AI to explain a concept, interpret a result someone else calculated, or reason about a quantitative question without producing a specific number. There, the model's probabilistic reasoning is appropriate. The line is: if you are going to use the number, use code to produce it.

Solution

Ask AI to write the calculation, not perform it

The shift is straightforward. Instead of pasting data and asking for an answer, ask AI to write a formula or script that produces the answer. Then run it yourself - in Excel, in Google Sheets, in Python, in whatever environment you have available. The model handles the logic. A deterministic engine handles the arithmetic.

Don't do this

"Here are my project costs by month. What is the total spend variance against budget?"

Do this instead

"Here are my project costs by month. Write an Excel formula that calculates total spend variance against budget. Include a comment explaining the logic so I can verify it before I use it."

The second prompt produces something you can run, inspect, and hand to a colleague. The first produces a number you have to trust blindly or verify manually - which defeats the purpose of asking AI in the first place.

A spreadsheet isn't error-free either - a well-known audit found errors in 49 of 54 operational spreadsheets (Panko, University of Hawaii, reported by Computerworld, 2004). That's the point: a formula can be audited and fixed. A number in a chat window can't.

For most enterprise practitioners, the spreadsheet formula is the right default. If you need something more repeatable across data sets, a Python script adds that. Either way: ask AI to write the calculation, then run it yourself.

The prompt that works: "Write an Excel formula that calculates [specific calculation] from this data. Explain the logic so I can verify each step before I use it."

One more step: After the script runs, ask AI to review the output for reasonableness - not to recalculate, but to flag if the result looks out of range given the inputs. This uses the model's pattern recognition strength rather than its arithmetic weakness.

Notes

Why the model sounds certain when it is wrong

Language models are trained to produce fluent, confident text. Hedging on arithmetic would make outputs feel less useful, so the training pushes toward certainty. This is not a bug that will be patched - it is a property of how these systems work. The confidence in the answer is not evidence of accuracy in the calculation.

Some models have gotten better at arithmetic on simple, well-formed problems. None are reliable enough to stake portfolio decisions on without verification. But the more important point is this: even if a model could calculate perfectly, a formula in a spreadsheet or a script in a file is still better than a number in a chat window. It is auditable. It is rerunnable. A colleague can challenge the logic without asking you to reconstruct it. That is a practice argument, not a capability argument - and it holds regardless of how capable models become.

The deeper point: AI is a reasoning and synthesis tool. Spreadsheets and code are calculation tools. Using each for what it is actually good at is not a limitation on how you use AI - it is what using AI well looks like.