Opening the AI black box from the mathematics side - training people to handle interpretability through algebra (NSF award $1.65M)
The U.S. National Science Foundation awarded the University of Wisconsin-Madison $1,647,024 to build mathematical foundations for interpretability, identifiability and robustness in AI from algebra and statistics, and to train people in that area.
Grant overview (primary data)
- Award amount$1,647,024 / Est. total $2,735,284
- RecipientUniversity of Wisconsin-Madison (Wisconsin)
- ProgramWORKFORCE IN THE MATHEMAT SCI, ALGEBRA,NUMBER THEORY,AND COM
- Period2026-09-01 〜 2031-08-31
- FunderU.S. National Science Foundation (NSF) / NSF
Key points
- The phrase black box collapses interpretability, identifiability and robustness - questions of different character - into one.
- The award poses them separately and puts them into provable form in the language of algebra and statistics.
- Core areas are algebraic statistics, tensor methods, parameter estimation and high-dimensional inference, with guarantees for representation learning and model recovery.
- It also investigates AI-enabled approaches to mathematical discovery, holding both directions in one award.
- A Research Training Group weights producing people able to work in the field over individual research results.
- The 120 NSF awards this site holds as of 2026-09-02 divide across 70 programmes, mixing research awards with workforce frameworks.
1What "black box" collapses together
That AI decisions are hard to interpret is often called a black box. The phrase gathers too much into one. Being unable to explain what is happening, several internal states producing the same output, output collapsing under a small change of input - questions of different character sit under one word.
| Mathematical question | What it seeks to establish |
|---|---|
| Interpretability | Whether the reason for an output can be followed by a person |
| Identifiability | Whether internal parameters can be determined uniquely from observations |
| Robustness | Whether output stays stable under changes in input |
What the award takes up is posing these separately and putting them into provable form in the language of algebra and statistics. Rather than collecting methods that work empirically, it moves toward writing why they work, or do not, as a guarantee.
2Approaching AI from the mathematics side
The abstract states that algebraic and geometric structure will be used to analyse neural networks and latent representation models. Core areas named are algebraic statistics, tensor methods, parameter estimation and high-dimensional inference, with emphasis on provable guarantees for representation learning and model recovery. In parallel it investigates AI-enabled approaches to mathematical discovery itself. Mathematics analysing AI and AI assisting mathematics sit inside the same award.
3A framework for developing people, not results
- 1UndergraduatesMeet the field through research experience
- 2Graduate studentsAcquire the practice of research through mentoring and courses
- 3Postdoctoral scholarsAdvance research while becoming the next layer of mentors
- 4As a communityWorkshops build connections across cohorts
A Research Training Group places weight less on individual results than on producing, continuously, people able to work in the field. The 120 NSF awards this site holds as of 2026-09-02 divide across 70 programmes, mixing awards for research itself with frameworks for developing people. Size alone does not reveal that difference.
Why it matters
People able to handle interpretability, identifiability and robustness in provable form are needed wherever regulation and risk management reach AI. In finance, medicine and public services, where accountability is demanded, "it works empirically" does not suffice: someone has to say under what conditions a guarantee holds. Where these training frameworks are running is a reference point for the supply of such people a few years out.
FAQ
What is identifiability?
Why does algebra come into it?
What kind of framework is an RTG?
Sources (primary)
Source: NSF Award Search (U.S. National Science Foundation, public domain). Amounts are the obligated amount. For privacy, we do not handle principal investigator names.
- NSF Award (original, official)
- NSF Award ID: 2447115