WorkFactory builds 112,686 work tasks to train small agents

WorkFactory is a method from IQuest Research for building training tasks for AI agents that do professional work, described in a paper posted to arXiv on October 27, 2026. The team made 112,686 tasks, trained three small Qwen models on them, and says the largest result reaches the range of frontier models on a work benchmark.

The interesting part is not the models. It is how the tasks are made. Most of the effort goes into making sure each task can actually be solved from the files it ships with.

The problem: tasks that cannot be done

To train an agent on office work, you need many tasks, each with a request, input files and a good example of the finished work. An LLM can write those tasks cheaply. The trouble is that generated tasks are often broken. A spreadsheet is missing a number the report needs, two files disagree, or the request leaves out what the deliverable should contain. A teacher model that attempts such a task produces a bad demonstration, and the student learns from it.

How WorkFactory builds a task

The paper describes four stages.

1. Intent synthesis. Task ideas come from three places: an occupational hierarchy based on the Standard Occupational Classification, documented skills from skills.sh, and real professional deliverables, where a model works backward to the request that would have produced them. A reviewer model filters out incoherent or implausible ideas.
2. Graph-guided construction. A planner starts from the intended output and works backward to build an "information support graph". It lists each fact the output needs, which file carries it, and how the facts depend on each other. Files are then generated to match the plan.
3. Execution-based repair. A probe solver tries the task using only what a solver would see. If it hits a blocker, a refiner edits the request and files, keeping the same file count, types and dependencies. A fresh probe checks the fix. This runs for up to three rounds.
4. Teacher trajectories. GLM-5.3 attempts the surviving tasks in clean sandboxes, using agent harnesses including Stirrup and Claude Code. Simple filters drop repeated reasoning, repeated tool calls and thin deliverables. The recorded runs are used for supervised fine-tuning.

In a 5,000-task analysis set, 69.58% of tasks needed no repair, 20.24% needed one round and 7.70% needed two. 124 tasks (2.48%) hit the three-round limit, and 24 of those stayed blocked and were dropped.

The data and the models

The corpus covers 612 occupations in 23 major groups. A task has 4.15 reference files on average, averaging 1.56 MiB each, and the files span 317 types. An average teacher run took 57 assistant turns and 63 tool calls.

The team fine-tuned Qwen3.5-4B, Qwen3.6-27B and Qwen3.6-35B-A3B (a mixture-of-experts model, which runs only part of its network for each token). The results are called WorkPilot-4B, WorkPilot-27B and WorkPilot-35B-A3B.

Results

The paper reports four benchmarks. On GDPval-AA, scores are Elo ratings, a ranking number built from head-to-head comparisons. The paper says unmarked GDPval-AA numbers are local two-anchor estimates, not official scores. The official v2.1 figure is marked only for some baselines, including the Qwen3.6-35B-A3B base.

For comparison, the paper lists GLM-5.3 at 1646 Elo, DeepSeek-V4.1-Flash at 1600, Qwen3.8-Max at 1596, GPT-5.6 Sol at 1588 and Kimi-K3 at 1524. WorkPilot-27B is 12 Elo above DeepSeek-V4.1-Flash and 34 below GLM-5.3. On JobBench, DeepSeek-V4.1-Flash (67.69) and GLM-5.3 (66.35) still beat it (63.05).

The gains come early. By 30,000 tasks, the 35B-A3B and 27B models had captured 88% and 94% of their total Elo gain. Going from 75,000 to 110,000 tasks added 13 points on the 35B-A3B model. That suggests task quality matters more than raw count past a point.

Which part matters: ablations

On 5,000-task sets, the authors compare three setups on GDPval-AA Elo. Plain artifact-level planning gives 1245. Adding the information support graph gives 1293 (+48). Adding execution-based repair gives 1326 (+33).

The defect counts are more telling. Cross-file inconsistency falls from 32.82% of tasks with planning alone to 4.84% after repair. Missing evidence falls from 9.62% to 2.02%. The graph was not a pure win: it raised leakage (where the answer is exposed in the inputs) from 1.92% to 3.36% and self-inconsistency from 3.28% to 4.46%. The authors say explicit dependencies expose derived facts and make constraints denser. Repair brought both back down.

Teacher choice also matters. On a 30,000-task subset, a student trained on GLM-5.3 runs reached 1483 Elo, against 1248 for one trained on DeepSeek-V4-Flash-0731 runs. The base model scored 879.

Cost

The paper estimates solving cost per GDPval-AA task at about $0.012 for WorkPilot-4B, $0.031 for the 35B-A3B model and $0.396 for the 27B model. It says all three sit on the non-dominated cost and Elo frontier among the models shown. These figures cover inference only.

Limits

• The authors name two limits. The teacher is text-only, so the models were trained and tested without visual inputs. And the work is supervised fine-tuning only, with reinforcement learning left for later.
• Several headline GDPval-AA numbers are local estimates calibrated to the benchmark, not official leaderboard entries. Compare them with care.
• All results come from the authors' own evaluation. We found no independent replication.
• Web-fetch tool failures stayed around 40 to 48% for the 35B-A3B model after training, so some agent weaknesses were not fixed.

Code, data and weights

The paper points to the IQuestLab/WorkFactory repository for data and models, under an Apache-2.0 license. It says only a subset of the task data and the model weights are released, so the full 112,686-task corpus is not confirmed as public. When we opened the repository, it held a README, figures and a license, with no source code, dataset files or weights visible, and the dataset and model links were in-page anchors. Treat the release as announced, not yet verified.

Why it matters

Small open models are catching up on office-style agent tasks, and this paper argues the data is the lever. A 4B model went from 661 to 1375 Elo on one benchmark, and the cost estimates are a few cents per task. The checks that decide whether a synthetic task is solvable, rather than the volume of tasks, look like the transferable idea. Qwen models are also in the news this week: see our report on Qwen 3.8 weights under Apache 2.0.

Watch for the repository's data and weights, and for results from anyone outside IQuest Research who trains on them.