Mathematical foundations of alignment in generative AI — NSF AI grant $1M (UPenn)
The NSF awarded about $1M to research on the mathematical foundations of "alignment" in generative AI — tackling risks like bias, unsafe, and misleading outputs in large language and diffusion models, and adapting public models to fairness, safety, reliability, and truthfulness.
Grant overview (primary data)
- Award amount$1,000,000
- RecipientUniversity of Pennsylvania (Pennsylvania)
- ProgramMSPA-INTERDISCIPLINARY, IIS Special Projects, Special Projects - CCF, EPCL: Energy, Power, Control,
- Period2025-10-01 〜 2028-09-30
- FunderU.S. National Science Foundation (NSF) / NSF
Key points
- Tackles bias and unsafe/misleading outputs of generative AI (LLMs, diffusion models) via "alignment"
- Studies retraining generic public models to meet fairness, safety, reliability, robustness, truthfulness
- Framed as necessary research as AI integrates into society and the economy
- About $1M, University of Pennsylvania, 2025–2028
- Shows the U.S. funding AI safety as foundational research (alongside regulation)
- The funding sources named across the 120 NSF AI-related awards this site holds as of 2026-09-01 run to 182 mentions of Research and Related Activities and 28 of STEM Education.
The NSF awarded about $1,000,000 to "Mathematical Foundations of Alignment in Generative AI (MFAI)" at the University of Pennsylvania (NSF Award 2502489; Oct 2025 – Sep 2028).
1The mathematics behind LLMs and diffusion models
Per the abstract, generative large language models (LLMs) and generative diffusion models (GDMs) can produce content with striking resemblance to human-made content, yet that content can introduce serious risks in specific applications: these models can reproduce biases in their training data and generate unsafe, misleading, false, or objectionable content. The project tackles these challenges within the general framework of alignment.
2Adapting general models to specific goals
Large pretrained models for image and language generation are available in the public domain but are generic; most users want to retrain them to fit their specific goals and principles. Success would make it easier to incorporate fairness, safety, reliability, robustness, and truthfulness — a necessary development for tools deeply integrated into society and the economy. The technical approach builds on properties of alignment problems in generative models.
3Which purse the money comes from
Counting the funding sources named across the 120 NSF AI-related awards this site holds as of 2026-09-01, NSF Research and Related Activities dominates at 182 mentions, then STEM Education at 28, with the H-1B fund and the trust fund at four each.
A single award can draw on more than one source, so the mentions outnumber the awards. The H-1B fund is an education purse financed by work-visa application fees, and part of it reaches the training of people for AI. Which purse an award is drawn from says what it was put there to do.
Why it matters
Shows the U.S. funding generative-AI safety/reliability (alignment) as foundational research. Bias, misinformation, and safety are universal concerns, and the regulation-plus-research approach is a useful reference.
FAQ
What is AI "alignment"?
Why does it matter?
Sources (primary)
Source: NSF Award Search (U.S. National Science Foundation, public domain). Amounts are the obligated amount. For privacy, we do not handle principal investigator names.
- NSF Award (original, official)
- NSF Award ID: 2502489