PDRVLA teaches robot models physics without running video

PDRVLA is a robot control model from researchers at Peking University and JD.com, described in an arXiv paper posted on October 27, 2026. It is a vision-language-action model, meaning one network that reads camera images and an instruction and outputs robot arm movements. The authors say it learns physics from video without generating any video while the robot runs, and it posts the highest scores they report on four simulated benchmarks.

Robots that follow instructions often fail on physical details, such as how heavy an object is. The paper's answer is a small add-on module that teaches the model how things move, and then drops the expensive part before deployment.

The problem: physics knowledge is slow to use

Vision-language-action models (VLAs) usually start from a vision-language model trained on images and text. The authors say such models encode physical properties of objects weakly and adapt poorly when something changes, such as an object's mass.

Video world models, which predict how a scene will look next, do capture dynamics. But generating future video inside the control loop adds latency, and a robot arm cannot wait for it.

What the team built

PDRVLA uses Qwen3.5-4B as its vision-language backbone. On top of it sits a Physical Dynamics Module: 50 learnable queries pass through 4 Transformer blocks and produce 50 "physical tokens" that feed the action predictor. The module has about 57.3 million parameters, 1.4% of the backbone.

During training, the module learns by predicting future RGB, depth and optical-flow features (optical flow is the motion of pixels between frames). A frozen video world model, LingBot-Video-Dense-1.3B, gives extra supervision. At deployment the video model is removed, so the robot runs only the backbone, the module and the action expert.

The action expert is a diffusion-transformer of about 150 million parameters. It predicts chunks of 8 actions using 4 denoising steps, over a 58-dimensional action vector with separate left and right arm slots.

Training data

The paper says all training data is public. It includes 1.26 million real-robot trajectories (14,893 hours), 1.22 million UMI trajectories (10,259 hours), 1.87 million simulated trajectories (8,868 hours), and large image-text sets: 8.05 million general and 23.63 million spatial and embodied examples. The team also converted 0.90 million human egocentric trajectories from EgoDex into robot data for 12 arm models, and added subtask and embodied chain-of-thought annotations. Chain-of-thought here means the model writes out intermediate reasoning about the task.

Results on four simulated benchmarks

Success rates, in percent, as reported by the authors:

The margins over the next best system are small: 0.2 points on LIBERO, 0.9 on LIBERO-Plus, 0.4 on RoboTwin 2.0 and 0.7 on SimplerEnv. The authors took baseline numbers from the cited papers under each paper's own protocol and did not re-run them.

The picture is mixed in places. On LIBERO-Plus, which perturbs the setup, PDRVLA leads in 4 of 7 categories. Its weakest is robot initial states at 75.2, where Qwen-RobotManip scores higher. On SimplerEnv's Carrot to Plate task it scores 64.6 against 85.4 for InternVLA-A1.5.

How much the physics module helps

The ablations are the most informative part. On 828 held-out clips, adding world-model supervision cut average prediction error from 0.6439 to 0.6354, a 1.32% drop. Gains were 4.06% for RGB and 2.12% for depth. The 0.62% gain for flow has a confidence interval that includes zero.

The extras are less clear. Generating chain-of-thought at inference gave no consistent gain: on LIBERO Long, success rose from 92.8 to 96.0 when it was turned off. Refreshing subtasks every 1 or 8 action chunks lifted RoboCasa365 results from 26.25 to about 32, but refreshing every 32 chunks dropped them to 20.63.

What we don't know

• No real-robot results. Every evaluation is in simulation.
• No code, weights or project page appear in the paper text we read.
• The comparison to rivals rests on other teams' published numbers, and the gaps are small.

The authors list richer prediction targets, such as contact states and material properties, as future work. The thing to watch is whether a real-robot evaluation or a code release follows.

Paper: PDRVLA on arXiv, also listed on Hugging Face Daily Papers.