A number without a source cannot be checked
A report is traceable when every figure it contains can be reproduced: it is known which version of the data it comes from, which table and which rows, or which query. Verifying what the machine says closed with an idea that is developed here: a traceable result can be reviewed, while one that is not can only be believed or discarded.
The requirement is not new. A good analyst has always been able to say where each number in a presentation came from. What changes with agents is the scale and the style: an agent can produce a ten-page report in minutes, well written and with dozens of figures, and the neat format says nothing about whether those figures come from the data. This chapter proposes a simple rule for reports made by agents (every number with its source) and a concrete format for following it.
The usual failure: a confident claim with no support
When thinking about the errors of an agent that works with data, the first thing that comes to mind is a miscalculation: a sum that leaves out a filter, a join that duplicates rows. Those errors exist, but they have an advantage: they leave a trail in the code, and someone can find them. The harder failure is a different one: an agent that confidently states something the data do not support. A count that was never queried, a "grew 12%" computed on an old copy, a "500 records were loaded" when the table holds 480. The text is correct in form and has no connection whatsoever to the database.
Recent research measures it. A 2026 paper on agents that operate databases found that, before any control was applied, 21 of 220 responses contained an unsupported claim of the type being controlled for. When every claim of that type was required to be tied to a trace of what was actually executed, none of the 199 responses issued had one, at the cost of 21 fewer responses. The result offers two lessons. The first is that it is better to verify the state of the data and not the agent's description of it. The second is that abstaining is a valid outcome: the rule trades coverage for reliability, a tradeoff similar to the one made when setting a cutoff point in Logistic regression.
Stopping that failure is what makes an analysis done by agents reliable. And the most direct way to stop it is for every figure to carry its source.
First, the version of the data
Data change. An orders table at 9 a.m. is not the same as at 6 p.m., and a query repeated the next day can give a different result without anyone having made a mistake. That is why a report states the version of the data it comes from, once, at the beginning or at the end.
In a versioned warehouse, like the one described in Agents that analyze, that version is a snapshot number. With DuckLake, a query written with AT (VERSION => n) returns exactly the same thing today as it will a year from now, even if thousands of new rows have been loaded. Without that reference, when a number does not match what someone sees later, there is no way to tell an error from a change in the data. It is, in essence, the property of replicability that this book has been pointing out since Working environment.
Then, the table and the row, or the query
The figures in a report usually come in two kinds:
- A value from a row. A customer's status, an order's date, a campaign's budget. It is cited with the table and the row's key.
- A computed value. A total, an average, a count, a change. It is cited with the query that produces it, complete and runnable, and with the result it returned.
A concrete format, which an agent can produce effortlessly and a person can read without training, consists of short markers in the text and a block of sources at the end:
In October there were 1,284 paid orders, for a total of $18,420,500 [F1]. The average ticket was $14,346 [F2], and the customer with the highest billing for the month was C-0193 [F3].
Sources
Data: central warehouse, snapshot 412 (2026-11-02 09:00)
[F1] sales.orders
SELECT count(*) AS orders, sum(amount) AS total
FROM sales.orders AT (VERSION => 412)
WHERE status = 'paid'
AND fecha >= DATE '2026-10-01' AND fecha < DATE '2026-11-01';
→ orders 1284 | total 18420500
[F2] Computed from [F1]: total / orders = 18420500 / 1284
→ 14346.18
[F3] sales.customers, key customer_id = 'C-0193'
(the first one when the query in [F1] is grouped by customer_id
and ordered by sum(amount) descending)
A few rules go with the format:
- One version for the whole report. If for some reason versions are combined, each source states its own.
- The query exactly as it was run, with its filters and its unrounded result; the rounding goes in the text.
- A row is cited by its key, never by its position. "Row 15" stops meaning anything as soon as the table changes order.
- A derived figure cites the figures it comes from and the operation, like [F2], so that an error in the base can be traced to everything that depends on it.
- A figure without a source does not go out, or it goes out explicitly marked as an estimate or as unverified data (as Data wrangling does with the 80% saying, which has no verifiable source).
Who checks the citations
A citation is only useful if someone checks it, and this is where the format pays off: checking is mechanical. It is enough to run each query on the cited snapshot and compare the result with what the text says, something a program can do without a person's intervention. The same goes for rows cited by key. People are left for what cannot be automated.
And some things cannot be automated. A citation verifies that the number comes from the query, not that the query answers the right question. A sales total can be computed correctly and still have excluded returns when the question included them; the ambiguity discussed in Asking the data in natural language is still there. To reduce it, it is better for the agent to query metrics defined once, in a semantic layer, rather than write free-form SQL for each report: that way "sales" means the same thing in every report the team produces. It is also worth noting that who signs a report and who decides when a figure without a source may go out are questions of organization more than of data, and they are developed in the book Business factories, by the same author.
Finally, traceability has a cost: the report is a little longer and the agent must keep every query it runs. It is a small cost compared with the alternative, which is reviewing an entire report because a single figure turned out to be false and there is no telling which others are too.
Recap
A report made by agents is reliable to the extent that every number can be reproduced. To that end, it states the version of the data it comes from and cites, for each figure, the table and the row's key or the query that produces it, along with its result. The failure this rule stops is not the miscalculation, which leaves a trail, but the confident claim the data do not support. A citation does not replace the reader's judgment, but it points that judgment where it is needed: at the question, not at the arithmetic.
Further reading
- No Trace, No Claim: Two Contracts for Database Agents. Zhang, 2026. The paper that measures unsupported claims in database agents and the effect of requiring a trace for each one.
- Enabling Large Language Models to Generate Text with Citations. Gao, Yen, Yu and Chen, 2023. How to measure whether the citations a language model produces actually support what it claims.
- Time Travel. DuckLake documentation. How to query a table as it was at a given snapshot.
- dbt Semantic Layer. dbt documentation. A way to define metrics once and query them by name.