# R2Orch trains agent orchestrators to work with zero skills

> A paper from Chinese University of Hong Kong, Shenzhen uses self-play to move hand-written skills into a Director model, and reports wins on 12 benchmarks.

URL: https://claudegpt.si/news/r2orch-agent-orchestration-self-play  
Published: 2026-11-02T15:50:25.613Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Agents, Reinforcement learning, Hugging Face, Qwen  
Publisher: ClaudeGPT Si

## Key takeaways

- R2Orch trains a Director model to design agent graphs through self-play, then withdraws skills one at a time until none are needed.
- The authors report it beats baselines on all 12 benchmarks, for example 81.33 on AIME 2026 against 74.00 for PATS.
- The method still starts from skills the authors wrote by hand for each dataset.

R2Orch is a training method for AI agent orchestration, described in a [paper](https://arxiv.org/abs/2610.22740) from researchers at the Chinese University of Hong Kong, Shenzhen and four other universities. It trains a small "director" model to design teams of agents, and then slowly removes the written instructions that help it. The paper was the most upvoted on Hugging Face's daily papers page on October 30, with about 79 upvotes.

The idea matters because most agent systems today lean on long hand-written prompts and skill files. R2Orch tries to move that knowledge into the model's weights, so the finished system runs with no skills at all.

## What orchestration means here

Agent orchestration is the job of splitting a hard task among several model calls: who does what, and in what order. In this paper a Director model builds a graph of roles and hand-offs, and a set of frozen Worker models carry it out. Only the Director is trained.

A "skill" is a block of procedural knowledge given to the model, such as steps for solving a math problem. The paper says skill scaffolds hand this kind of knowledge to a policy, either in context at inference time or as training support. They work, but they have to be written and kept up to date by people.

## How R2Orch trains the Director

The authors name three problems with earlier orchestration training: hand-designed curricula, "advantage collapse" (when every attempt at a task ends the same way, so there is nothing to learn from), and evaluation signals that score each skill with one noisy number. R2Orch uses two nested loops.

The inner loop is self-play. A Proposer model picks tasks, and the Director builds agent graphs for them. The Proposer is rewarded for tasks where the outcomes still differ across attempts when a particular skill is hidden. Those are tasks the Director can still learn from.

The outer loop is called Skill Withdrawal. It hides one skill at a time during training and compares results with and without it. A statistical test decides whether the Director does as well without the skill, and if so the skill is removed for good. The authors say all 12 withdrawals in their runs passed this test. The aim is a Director that needs zero skills when deployed.

## Results

The Workers are mostly Qwen3.5-9B models. The authors test on 12 benchmarks: six in-distribution (HotpotQA, NQ-Open, MedQA, AIME 2026, MBPP+, ALFWorld) and six held out (TriviaQA, MuSiQue, GPQA, MATH-Hard, SWE-Bench, WebShop). They compare against baselines from six families, including GRPO, PATS and SKILL0. The paper states R2Orch is best on all twelve.

Scores for four of the benchmarks, with no skills at test time:

| Benchmark | Qwen3.5-9B alone | GRPO | SKILL0 | PATS | R2Orch |
| --- | --- | --- | --- | --- | --- |
| AIME 2026 (accuracy) | 48.67 | 34.00 | 64.67 | 74.00 | 81.33 |
| ALFWorld (success rate) | 47.19 | 50.94 | 83.44 | 87.97 | 90.63 |
| SWE-Bench (resolved) | 15.78 | 17.50 | 36.88 | 40.16 | 42.81 |
| WebShop (success rate) | 32.81 | 35.94 | 82.81 | 87.97 | 91.09 |

The ablations show where the gain comes from. Using GRPO alone, with no recursive self-play, the Director scores 59.33 on AIME 2026 against 81.33 for the full method. Keeping the skills fixed, with no withdrawal, gives 66.67. The authors also report that a Director trained on Qwen3.5-9B Workers transfers to six other Worker models without retraining, and that the spread in scores across Workers narrows from 25.7 to 6.4 points.

## What to trust

Every number here comes from the authors, on their own test sets. Each test set has 128 items, and AIME 2026 has 30, so a few problems can move a score by several points. The paper averages five runs.

The authors state one limitation themselves: R2Orch starts from initial skills that they customize by hand for each dataset. Deriving those starting skills automatically for a new domain is left open. So the method removes the need for skills at deployment, not the need to write them at the start.

The paper says its code is available at github.com/beita6969/R2Orch. We did not open the repository, and the paper does not mention released model weights.

## What to watch

The practical test is whether the zero-skill Director holds up on a task type the authors did not hand-prepare. Outside replications on other Worker models will say more than the 12-benchmark table. For a related look at how teams of agents are coordinated, see our piece on [DynaFlow](https://claudegpt.si/news/dynaflow-multi-agent-orchestration).

## Frequently asked questions

### What is agent orchestration?

It is the job of splitting a hard task among several model calls, deciding who does what and in what order. In R2Orch a Director model builds a graph of roles and hand-offs that frozen Worker models execute.

### What is Skill Withdrawal in R2Orch?

It is the outer training loop. It hides one skill at a time and tests whether the Director does as well without it. If so, the skill is removed for good.

### Does R2Orch release code?

The paper says its code is available at github.com/beita6969/R2Orch. The paper does not mention released model weights.

## Sources

- [R2Orch: Recursive Self-Improvement for Agent Orchestration via Self-Play within a Skill-Internalization Loop](https://arxiv.org/abs/2610.22740), arXiv
- [R2Orch (full text)](https://arxiv.org/html/2610.22740), arXiv
- [Hugging Face paper page](https://huggingface.co/papers/2610.22740), Hugging Face