US Public Data LabU.S. public data, made readable
Data areas
Federal Register Congress bills CISA KEV / NVD SEC EDGAR USAspending Defense contracts Trade Economy Energy Places FDA recalls AI Research Grants NIH research grants Fiscal & national debt
Tools
Search Recent Rankings Compare Money flows Timeline Compare indicators Topics
Profiles
Company profiles Agency profiles Glossary Contact
EN▾
日本語 English
physics.flu-dyn cs.LG

The drag fell but the waste rose — what maximizing a reward actually bought

physics.flu-dyn ・Giorgio Maria Cavallazzi, Miguel Pérez-Cuadrado, Alfredo Pinelli ・Jun 2026

A reinforcement learning agent maximizes its reward, which can diverge from what its designer intended. In drag reduction for wall turbulence, two controllers achieved large reductions while total dissipation went up.

Paper overview (our summary)

  • Field (arXiv category)physics.flu-dyn（+1）
  • AuthorsGiorgio Maria Cavallazzi, Miguel Pérez-Cuadrado, Alfredo Pinelli
  • Submitted2026-06-04
  • arXiv ID2606.06227v1

Key points

  • A reinforcement learning agent maximizes its reward, which can diverge from the intended outcome, made concrete here in drag reduction for wall turbulence.
  • A mass-conservation projection coupled the agents outputs and erased the per-agent credit the policy gradient requires.
  • A memoryless policy could not resolve the slow near-wall cycle it was acting on.
  • A reward scored on pressure gradient paid for nominal drag reduction with pumping power through the wall.
  • Two degenerate controllers achieved large reductions while dissipation rose; after repair the controller earns a conservative 17% under a closed energy budget.

1Maximizing the reward, drifting from the goal

What this series has followed so far is how structure on the subject side gets carried into a formulation. This paper takes up the failure on the opposite side. The reward is maximized properly, and the result moves away from what the designer wanted.

2Three faults, traced to their causes

The symptomThe cause identifiedThe repair made
Per-agent credit disappearsA mass-conservation projection couples the agents' outputs and erases the credit the policy gradient needsA differentiable projection that restores credit
The slow near-wall cycle cannot be followedA memoryless policy cannot resolve the time structure of what it acts onA recurrent policy with a widened sensing stencil
Drag falls while waste risesA reward on pressure gradient pays for the reduction with pumping power through the wallA reward scored on the true wall power

All three sit in the design of the reward or the computation. Learning did not fail; these are what learning succeeding, exactly as instructed, looks like.

3What the drag reduction figure concealed

Degenerate controllers foundTwoBoth achieved large drag reductions
Total dissipation at the same timeIt roseA reported reduction can mask a more wasteful flow
The figure after repair17%Conservative, under a closed energy budget

The metric improved while the system as a whole got worse. This is not peculiar to reinforcement learning, but where a reward is written down explicitly, the gap between what was written and what was wanted shows up directly in the result. That is why the paper speaks of honest accounting.

4The repair was not a stricter reward

Worth noting is that the authors did not lower the reward. The three faults sat in different layers: one in the computation, where a projection flattened credit; one in the structure of the policy, which had no memory; one in the definition of the reward, in what was counted. Where the thing broke was separated out and each layer fixed on its own terms.

The 17% that results is smaller than the figure before repair. It is, however, a figure obtained under a closed energy budget, in a form that lets one confirm the system as a whole actually gains.

Why it matters

Where a reward is written down explicitly, the gap between what was written and what was actually wanted appears directly in the result. Drag falling while dissipation rises is not learning failing but learning succeeding exactly as instructed. The practical value of the paper lies in separating the diagnosis into layers, the computation, the structure of the policy, and the definition of the reward, and repairing each on its own terms.

FAQ

How can drag fall while waste rises?
Under a reward scored on pressure gradient, nominal drag reduction can be bought with pumping power through the wall, and that power goes uncounted.
How were the three faults repaired?
With a differentiable projection restoring credit, a recurrent policy with a widened sensing stencil, and a reward scored on the true wall power.
How should the 17% be read?
As smaller than the figure before repair but obtained under a closed energy budget, described by the authors as conservative.

Related explainers

Sources (primary)

Source: arXiv (descriptive metadata is CC0 public domain). Summaries are our own; see arXiv for the original text and PDF.

#AI research#Reinforcement learning#Fluid dynamics#arXiv#Physics

← Back to AI research paper watch

Disclaimer: This site independently summarizes and classifies information based on official data sources. Always verify the latest and accurate information with the official sources. Content on finance, health, legal, and security is information, not advice. This site is not an official website of the U.S. government.