You do not need a frontier model to verify citations — calibrating the "judge LLM" for source attribution
Reinforcement learning increasingly uses a judge LLM to score rubric criteria, acting as the reward model. Before trusting that signal, we must know how capable and biased the judge is. On citation quality in deep-research systems, the authors score 8 off-the-shelf judges over 1,248 human-reviewed decisions and find cheaper judges stay competitive. But at equal F1, judges differ sharply in pass-rate drift and false-positive/negative rates — directional bias that scalar F1 hides and an RL loop amplifies.
Paper overview (our summary)
- Field (arXiv category)cs.CL
- AuthorsEthan Leung, Elias Lumer, Corey Feld, et al. (6)
- Submitted2026-07-09
- arXiv ID2607.08700v1
Key points
- A judge LLM acts as the reward model in RL; its capability and bias must be calibrated before it is trusted
- Setting: citation quality in deep research (each citation judged on source relevance and factual support)
- 8 judges from 3 families scored over 1,248 human-reviewed decisions (378 hard cases) — cheaper models stay competitive
- GPT-5-mini strongest on source-relevance F1 (0.908); factual support statistically indistinguishable across judges
- At equal F1, pass-rate drift and false-positive/negative rates differ widely — directional bias hidden by scalar F1, amplified by RL
This work measures how capable a judge LLM — an LLM that grades other LLMs outputs — must be, and how biased it is.
1Judge LLMs behind the reward model
Reinforcement learning increasingly relies on a judge LLM to score each rubric criterion, and that judge acts as the reward model during training. Before such a signal can be trusted, we need to know how capable the judge must be and how biased it is.
2Citation quality as the test case
The authors study this calibration question through citation quality in deep-research systems, where a search-grounded LLM must support each claim it writes with a cited source. Citation quality is a structured rubric task in which each attribution-citation pair is judged along two dimensions — source relevance and factual support — both requiring an LLM.
3Eight off-the-shelf judges across 1,248 decisions
On an adversarial long-form benchmark, they score 8 off-the-shelf judges from 3 model families against gold labels over 1,248 rubric decisions, all human-reviewed, 378 of them hard cases adjudicated from judge disagreements.
Cheaper judges stay competitive on both dimensions: GPT-5-mini attains the strongest source-relevance pass-class F1 (0.908, kappa=0.636), while on factual support no single model dominates (overlapping confidence intervals).
Why it matters
Directly relevant to LLM-as-a-judge, RLHF/RLAIF, and deep-research evaluation. The findings — a calibrated cheaper judge suffices, and do not select on F1 alone — bear directly on the cost and quality of building reward models and automated evaluators.
FAQ
What is a judge LLM?
Why is equal F1 still risky?
Sources (primary)
Source: arXiv (descriptive metadata is CC0 public domain). Summaries are our own; see arXiv for the original text and PDF.
- arXiv abstract page (original, official)
- PDF (arXiv)
- arXiv ID: 2607.08700