A humanoid learning professional tennis form from broadcast video — separating planning from tracking, and closing the real-robot gap with speed adaptation

1Doing it and looking the part are different problems

Humanoid robots playing ball sports are no longer rare. Achieving a professional player's motion style while holding task performance, however, is a separate difficulty. That is what this work addresses, learning tennis serving and rally form not from purpose-recorded motion capture but directly from broadcast video. Extracting style from readily available material is the starting premise.

2Separating planning from tracking

1. 1Learn from broadcast videoUse professional serving and rally styles as the source material
2. 2The planner producesStylistic kinematic motions
3. 3The tracker executesCarries out the motion with minimal interference with planning
4. 4Real hardware degrades itTracking performance inevitably degrades on real robots, is partially overlooked by autoregressive planning, and is compounded by noisy perception
5. 5Adaptation closes the gapLearn to track randomized execution speeds, and condition the planner on a learned motion-speed adapter to mitigate compounding errors

The point of the hierarchy is to separate the job of producing style from the job of making it physically hold. But separating them means differences invisible in simulation surface on real hardware. The heart of this work is how that gap is handled.

3The distance between simulation and hardware

The authors name three overlapping problems. Tracking performance inevitably degrades on a real robot. That degradation is partially overlooked when planning proceeds autoregressively. And noisy perception compounds the oversight. Each has its counterpart in the fix: learning to track randomized execution speeds raises robustness, and conditioning the planner on a motion-speed adapter keeps errors from accumulating.

Real-world experiments were run on the Unitree G1, and the policies were further deployed on the full-size 1.7 m Dobot Atom, demonstrating serving in the wild without motion capture.

4Where it sits among robotics papers

Of the 1,900 arXiv papers this site holds as of 2026-09-02, 63 have cs.RO as their primary category, against 482 in cs.CV, 377 in cs.LG, 364 in cs.AI and 254 in cs.CL. Work involving real hardware is comparatively scarce. This paper carries 18 authors, well above the median of 4 across the collection — a hint at the scale of organization that hardware validation demands.

Beyond the results themselves, the authors report that their real-world experiments yielded algorithmic and engineering insights for future humanoid ball-sports systems. This article is our own summary and does not warrant the correctness of the claims.