How does the brain decide what to say next? — decoding conversational speech with LLMs, NSF grant ~$2.06M (USC)
A ~$2.06 million ($2,057,137) NSF grant to the University of Southern California. Subjects hold weekly conversations for a full year while their brain activity is recorded with fMRI; large language models are then mapped onto that activity to reveal how the brain drives a conversation forward. The resulting dataset will be shared widely as a community benchmark.
Grant overview (primary data)
- Award amount$2,057,137
- RecipientUniversity of Southern California (California)
- ProgramCross-Directorate Activities
- Period2026-07-01 〜 2031-06-30
- FunderU.S. National Science Foundation (NSF) / NSF
Key points
- Recipient: University of Southern California; ~$2.06M ($2,057,137); July 2026–June 2031 (5 years)
- A CRCNS (computational neuroscience) proposal targeting the understudied side of language: production, not comprehension
- Longitudinal fMRI dataset: the same pairs converse weekly for a year, capturing accumulated conversational memory
- LLM internals are mapped onto brain activity to identify the neural signatures that drive the next utterance
- The dataset will be shared widely — a benchmark asset for naturalistic-dialogue neuroscience
- The same pairs talk weekly for a year; activity on the speaking side is mapped onto LLM representations and left as a public dataset.
1About $2.06 million to USC to read brain activity during conversation with LLMs
NSF has obligated $2,057,137 to the University of Southern California for "Generative models of neural activity driving conversational speech in longitudinal dialogues," a proposal under CRCNS, the collaborative program in computational neuroscience.
It records with fMRI the brain activity of participants who hold weekly conversations with the same partner, and aligns it with the internal representations of large language models to find how the brain decides what to say next. The resulting data will be shared widely as a benchmark for natural dialogue. The period is five years, from July 2026 to June 2031.
We use language to organize thought, externalize ideas, and explain reasoning — yet how the brain decides what to say next has remained a blank spot in neuroscience. Comprehension has been studied extensively; production has not, largely because natural conversation is hard to recreate in a laboratory.
This project's novelty is a longitudinal design that sidesteps the problem: the same pairs of participants converse weekly for a year, so the accumulating shared memory that real conversation rests on is captured along with the dialogue itself.
2Using AI as an instrument, not a subject
The second key move is using AI as an instrument rather than a subject. LLMs are the first generation of models that can turn natural-language input into meaningful responses, and their internal representations serve as concrete hypotheses for the computation of "composing a reply." Aligning those representations with fMRI signals gives researchers a ruler for locating where and how that computation unfolds in the brain.
AI is used to read the brain; what is learned feeds back into the design of future AI — a bootstrapping loop emblematic of where the field now stands.
3A public dataset as a deliverable
Just as important, the deliverable is not only papers but a public dataset. Naturalistic conversational fMRI data is extremely expensive to collect; shared, it lets researchers worldwide test hypotheses on common ground. It is a textbook case of an NSF award doubling as investment in field-wide research infrastructure.
4A year of talking to the same person
What stands out in this study is less the measurement than how it is arranged. A conversation is not a single exchange; it rests on an accumulation of shared memory. To measure that accumulation, the same pairs keep talking, weekly, for a year.
- 1The same pairs talk every weekThe build-up of shared memory, which a one-off session cannot produce, is written into the design
- 2Brain activity is recorded while speakingThe side that produces language, not the side that listens and understands, is what gets captured
- 3Activity is mapped onto LLM representationsAI is used as an instrument rather than a subject, to locate where the computation that assembles a response takes place
- 4The result is left as a public datasetfMRI during natural conversation is costly to acquire; as a shared asset it lets other researchers test hypotheses on the same ground
That last step counts as a deliverable is not incidental. Alongside particular findings, the award doubles as investment in research infrastructure the whole field will use.
Why it matters
Understanding speech production underpins more human-like conversational AI, assistive technology for speech disorders, and brain–machine interfaces. The two-way cycle — measuring the brain with AI, refining AI with brain science — plus the open-dataset infrastructure angle make this a useful reference point for the AI × neuroscience trend.
FAQ
Why use LLMs to study the brain?
What is new here?
Who can use the results?
Sources (primary)
Source: NSF Award Search (U.S. National Science Foundation, public domain). Amounts are the obligated amount. For privacy, we do not handle principal investigator names.
- NSF Award (original, official)
- NSF Award ID: 2605721