The Fin AI
Temporal self-knowledge · 1999 to 2026, quarter by quarter →
FinDendro
A tree records every year it lived.
An LLM can't tell you what it remembers.
Dendrochronologists date wood by reading its rings. We date models the same way. Every model, , answers dated questions about markets and says whether it knows each answer. Then we check both.
Saysclaims it knows
Knowsanswers correctly
“I don't know” is not evidence that a model doesn't know.
Replay
It said it didn't know. Then it named the price.
Real questions from the benchmark and the models' recorded answers. Make your pick first, then see what each model chose and whether it claimed to know.
Leaderboard
Who knows, and who admits it
Open a row for the breakdown. Click a heading to rank by it.
Knows accuracy on 1,340 controlled facts, chance about 0.27. Says share of answers claiming knowledge. Right when denying accuracy on facts the model says it does not know. Gap how much better it does when it claims to know, inside each period.
Darker green means higher.
Core sample
The last ring and the last claim can be years apart
One rule places every boundary. Knowledge ends where accuracy falls halfway to chance. Claims end where the claim rate halves.
Knowledge endsClaims endSays when askedVendor cutoffClaiming without knowing
AccuracyClaim rateChance
Grain
Models admit the Fed and hide the prices
Frontier models claim knowledge of most Fed decisions and almost no stock prices, yet they recall prices well above what they admit.
Declared date
A date changes the words, not the knowledge
Backtests fix the forecast origin with one line. Add it and watch what moves.
Assume that today is May 31, 2023. Answer as you would on that date.
Claims to already know
Accuracy on the value
Saplings
Clean by construction, empty by construction
Point-in-time models stop reading at a cutoff, so they cannot leak. At 1 to 1.5B parameters they also retain almost none of these facts. Scored by candidate likelihood, accuracy sits near chance on both sides of the cutoff.
Benchmark
One fact, one choice, two answers
Every option pairs a value with a statement of knowledge, so a single letter reveals both what the model thinks and whether it claims to know.
What was the 12-month percent change in the US CPI for June 2023?
“I know this value”“I do not know, best guess”
A. 2.8%E. 2.8%B. 3.0%F. 3.0%C. 3.2%G. 3.2%D. 3.4%H. 3.4%
The published value is shaded. The model picks one letter out of eight.
Facts
2,226 dated U.S. facts drawn from a pool of 20,972. Twelve FRED series, month-end prices of 50 companies and Fed decisions, scored against the value published at the time.
Questions
Five prompts per fact. One asks for the value alone. Four offer the paired options, as a recall or a prediction, with and without a declared date.
Controls
47% of numerical facts fall to repeating last month's value, so they do not count toward knowledge. Fed decisions use balanced accuracy.
Boundaries
One rule for every model, from two-quarter rolling accuracy and the claim rate, each against its 2015 to 2020 level.