# Trajectory2Gym trains small tool-use agents on grounded data

> A paper says successful runs make poor training data unless every action was visible to the learner, and reports Qwen3 gains of 17 to 30 points on agent benchmarks.

URL: https://claudegpt.si/news/trajectory2gym-tool-use-agents-training  
Published: 2026-10-27T14:56:58.932Z  
Author: ClaudeGPT Si Newsroom  
Section: Research  
Topics: Hugging Face, Qwen, AI agents, Fine-tuning, Research  
Publisher: ClaudeGPT Si

## Key takeaways

- Trajectory2Gym keeps a training step only if the agent could see what it relied on, and it adds data for under-covered tool-use paths.
- Across five Qwen3 sizes from 1.7B to 30B-A3B, the authors report a 16.9 to 29.8 point rise in the three-benchmark mean.
- Ablations are single runs and public-checkpoint comparisons are not matched; code, 34,772 interactions and five checkpoints are released.

Trajectory2Gym is a method for building training data for tool-use agents, described in a new paper on arXiv ([2610.33568](https://arxiv.org/html/2610.33568)). Its claim is simple: a successful run is not automatically a good lesson. The authors fine-tuned five Qwen3 models on data built this way and report gains on three agent benchmarks at every size.

The paper had 325 upvotes on Hugging Face's daily papers page on October 27, 2026, and the authors have released code, data and checkpoints.

## The problem: successful runs can teach the wrong thing

A tool-use agent is a language model that calls APIs to get a job done, such as changing a flight booking or checking an order. The usual way to train one is to let a strong model attempt many tasks, keep the runs that succeed, and fine-tune a smaller model to imitate them. This is called supervised fine-tuning, or SFT.

The authors say success is a weak filter, and point to two gaps.

The first is coverage. In their pool of 34,772 candidate trajectories, 40.9% contain at most two decisions the model would be trained on. Only 10.0% include a correct decision made after a real error. Agents need the second kind, because real tool calls fail.

The second is grounding. An API call can succeed even though the agent had no way to know the argument it used. The paper's example is an identifier the agent never saw in the conversation. Imitating that action teaches the student to guess.

## What the Trajectory2Gym team did

Trajectory2Gym has three stages.

1. **Synthesis.** The team defines a shared vocabulary of capabilities: actions, arguments, outcomes, recovery after errors, branching, handoff, termination and horizon. A generator builds tasks for a target path through that vocabulary, a teacher model runs them in the real environment, and a trajectory is kept only if the needed dependencies show up in the executed log. Paths with too little support get more tasks.
2. **Grounded supervision.** A step becomes a training target only if the call executed validly and everything it relied on was visible in the history the student sees. Rejected calls and their error messages stay in the context but carry no training loss. Teacher-only hints never reach the student.
3. **Structural-support learning.** Training data is chosen so each capability path is covered to a target depth, which grows with model size (1, 2, 3, 4 and 8 from the 1.7B to the 30B-A3B model). The loss gives each trajectory equal weight, however many training examples it produces. The authors call this parent-normalized SFT.

## Results

The models are Qwen3 at 1.7B, 4B, 8B, 14B and 30B-A3B, each trained for one epoch on 11.6k to 19.9k trajectories. The benchmarks are Tau1, Tau2 (customer-service tasks with simulated users and tools) and BFCL, a function-calling test. The paper reports that the equal-weight mean across the three rises by 16.9 to 29.8 points at every size.

| Model | Tau mean before | Tau mean after | BFCL overall before | BFCL overall after |
| --- | --- | --- | --- | --- |
| Qwen3-1.7B | 15.93 | 35.97 | 55.88 | 66.48 |
| Qwen3-4B | 21.70 | 56.70 | 62.92 | 74.85 |
| Qwen3-8B | 26.44 | 62.65 | 67.30 | 78.62 |
| Qwen3-14B | 28.52 | 69.44 | 72.01 | 79.68 |
| Qwen3-30B-A3B | 36.36 | 75.47 | 77.58 | 80.93 |

Some of the specifics stand out. The 4B model scores 69.17 on Tau2, against 28.33 for the untouched 30B-A3B model. On Tau2 Telecom, the 30B-A3B model goes from 12.50 to 95.00. In the multi-turn part of BFCL, the 1.7B model goes from 12 to 48.

For reference, the paper lists GPT-5.2 at a Tau mean of 76.99 and Claude-4.5-Opus at 79.38, numbers it takes from reported results. The 30B-A3B model reaches 75.47. The authors also re-ran six public agent-training checkpoints, and their 8B model's Tau2 score of 74.17 is at least 28 points above every one of them.

In matched 8B ablations, removing parent normalization hurt most: the three-benchmark mean fell from 67.97 to 59.94. Dropping decision grounding cost 2.75 points, and dropping targeted expansion cost 3.55.

## What the paper admits

- Each ablation is a single run. The authors note that one Tau2 Airline task moves the three-benchmark mean by 0.56 points, so they call gains of a few points directional.
- More support is not always better. In the depth test at 8B, the shallowest setting scored highest (72.28, against 69.44 at depth 4).
- The comparisons with public checkpoints are not matched in data, model family or context budget, as the authors say themselves.
- Live Simple, a BFCL category, drops by 6 to 13 points at four of the five sizes.
- Tau1 Airline is a seen-task setting, and the method so far covers only supervised training on one epoch. On-policy training and web, software and terminal agents are listed as future work.

## Code, data and weights

The [GitHub repository](https://github.com/yihaohu0118/Trajectory2Gym) is under Apache-2.0. It holds the interaction representation, target construction, selection code, the parent-normalized training, and adapters for the three benchmarks. The 34,772 candidate interactions (about 125 MB) are on Hugging Face as mis2/Trajectory2gym, and five checkpoints are published. The README says the code alone does not reproduce the reported numbers, because that also needs the benchmark environments and the training data, and that the listed scores are inherited from earlier experiments rather than rerun for the release.

## Why it matters

Most of the paper's lesson is about data, not architecture. Small open models gain the most when training examples are checked for what the student could actually see and for whether they cover failure and recovery. For anyone fine-tuning an agent, that is a cheap thing to audit in an existing dataset. The headline numbers come from the authors' own runs, so treat the size of the gains as a claim until others reproduce them.

## Frequently asked questions

### What is Trajectory2Gym?

It is a method for building supervised training data for tool-use agents. It synthesizes tasks around under-covered tool-call patterns and trains only on actions the agent could justify from its visible history.

### What is decision grounding in tool-use agents?

A training step counts as grounded when its tool call executed validly and everything it depended on appeared in the history the student model sees. Teacher-only hints are excluded.

### Is Trajectory2Gym open source?

The code is released under Apache-2.0, with 34,772 candidate interactions and five checkpoints on Hugging Face. Benchmark packages and some data keep their own licenses, and the code alone does not reproduce the paper's numbers.

## Sources

- [Trajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents](https://arxiv.org/html/2610.33568), arXiv
- [Trajectory2Gym on Hugging Face Papers](https://huggingface.co/papers/2610.33568), Hugging Face
- [Trajectory2Gym repository](https://github.com/yihaohu0118/Trajectory2Gym), GitHub