NSF AI grant $1M: mathematical foundations of alignment in generative AI (UPenn)
The NSF awarded about $1M to research on the mathematical foundations of "alignment" in generative AI — tackling risks like bias, unsafe, and misleading outputs in large language and diffusion models, and adapting public models to fairness, safety, reliability, and truthfulness.
Grant overview (primary data)
- Award amount$1,000,000
- RecipientUniversity of Pennsylvania（PA）
- ProgramMSPA-INTERDISCIPLINARY, IIS Special Projects, Special Projects - CCF, EPCL: Energy, Power, Control,
- Period2025-10-01 〜 2028-09-30
- FunderU.S. National Science Foundation (NSF) / NSF
Key points
- Tackles bias and unsafe/misleading outputs of generative AI (LLMs, diffusion models) via "alignment"
- Studies retraining generic public models to meet fairness, safety, reliability, robustness, truthfulness
- Framed as necessary research as AI integrates into society and the economy
- About $1M, University of Pennsylvania, 2025–2028
- Shows the U.S. funding AI safety as foundational research (alongside regulation)
The NSF awarded about $1,000,000 to "Mathematical Foundations of Alignment in Generative AI (MFAI)" at the University of Pennsylvania (NSF Award 2502489; Oct 2025 – Sep 2028).
Per the abstract, generative large language models (LLMs) and generative diffusion models (GDMs) can produce content with striking resemblance to human-made content, yet that content can introduce serious risks in specific applications: these models can reproduce biases in their training data and generate unsafe, misleading, false, or objectionable content. The project tackles these challenges within the general framework of alignment.
Large pretrained models for image and language generation are available in the public domain but are generic; most users want to retrain them to fit their specific goals and principles. Success would make it easier to incorporate fairness, safety, reliability, robustness, and truthfulness — a necessary development for tools deeply integrated into society and the economy. The technical approach builds on properties of alignment problems in generative models.
Why it matters
Shows the U.S. funding generative-AI safety/reliability (alignment) as foundational research. Bias, misinformation, and safety are universal concerns, and the regulation-plus-research approach is a useful reference.
FAQ
What is AI "alignment"?
Why does it matter?
Sources (primary)
Source: NSF Award Search (U.S. National Science Foundation, public domain). Amounts are the obligated amount. For privacy, we do not handle principal investigator names.
- NSF Award (original, official)
- NSF Award ID: 2502489