CoRL 2026 Spotlight 10th Conference on Robot Learning

Chunk-wise
Model-based Policy Optimization

Weixi Song1,2,3Donglin Wang3,†Yong-Lu Li2,4,†

1 Zhejiang University2 Shanghai Innovation Institute3 Westlake University4 Shanghai Jiao Tong University

† Corresponding authors

Paper arXiv Code
01 / PRESENTATION

The idea in two minutes.

02:14 / RESEARCH OVERVIEW

From the recurrent-rollout bottleneck to collaborative chunk-wise modeling and real-robot deployment.

02 / METHOD

One chunk. One forward pass.

Model gradients for learning.
A direct policy for deployment.

CMPO computation graph: the policy predicts an action chunk; the world model predicts the latent chunk; reward and terminal value yield the actor objective.
Collaborative chunk-wise policy update.
Figure 1 from the paper.
1

Predict an action chunk

Encode the current state, then generate H future actions with a single actor query.

at:t+H−1 = π(zt)
2

Evaluate the future together

The chunk world model maps the current latent state and the entire action sequence directly to future latent states.

zt+1:t+H = f(zt, at:t+H−1)
3

Optimize through the model

Predicted rewards and a terminal critic form an (H−1)-step value expansion. Gradients pass through a single chunk prediction instead of a recurrent rollout.

AT DEPLOYMENT

Predict a chunk, execute its first action, and repeat from the next observation. No test-time planning is used in the main experiments.

03 / EXPERIMENTS

Efficient learning.
Responsive control.

19 whole-body control tasks.
61-dimensional action space.

CMPO achieves the strongest average performance among the evaluated baselines and outperforms them on the majority of selected HumanoidBench tasks. All main experiments use a chunk horizon of H = 3.

Learning curves for the average across 19 tasks, comparing CMPO, MAAC, TD-MPC2, DreamerV3, SAC, and PPO.
HumanoidBench learning curves. Average over 19 tasks. Mean of 3 runs; shaded regions show 95% confidence intervals.
COMPUTATIONAL EFFICIENCY

More time learning.
Less time planning.

Single-call predictions reduce sequential training work. At inference, the actor directly outputs actions without iterative MPC optimization.

Table 1. Single NVIDIA RTX 4090; identical settings; no Torch Compile. Training time measured over 200k steps. Lower is better.

Training and inference wall-time
Task / measurementTD-MPC2CMPO
Walk Training / 200k steps3.57 h1.74 h
Run Training / 200k steps3.54 h1.72 h
Walk Inference / step40.8 ms1.4 ms
Run Inference / step39.5 ms1.3 ms
04 / SIM-TO-REAL

From simulation
to standing on two legs.

Unitree Go2 quadruped.
Bipedal locomotion.

Unitree Go2 transitions into an upright posture in the real world and in simulation; a plot compares CMPO and SAC linear velocity tracking reward.
Real-world and simulated standing locomotion, alongside the simulation training curve. Figure 3 from the paper.

Trained in Isaac Gym and transferred to the physical Go2, the policy controls all 12 actuated joints through residual joint-position commands. CMPO discovers standing earlier and achieves higher tracking reward than SAC in the reported simulation experiment.

7/10

successful hardware trials for CMPO
versus 4/10 for SAC · Maintaining a bipedal posture for 20 seconds

05 / ANALYSIS

Why the chunk model matters.

01

A shorter gradient path

Compared with recurrent MAAC updates, CMPO exhibits more stable gradient norms and smaller excursions as the tested horizon increases.

02

Both components matter

Keeping the chunk actor but reverting to a single-step world model substantially degrades performance in the reported ablation.

03

Prediction still has limits

Chunking removes recurrent temporal depth, but direct long-horizon prediction remains difficult. Performance drops at H = 5 in the horizon study.

Gradient norm comparisons between CMPO and MAAC over training and across horizons.
Figure 4. The left panel uses separate vertical scales for the two methods; the right panel compares gradient norms across horizons on a shared scale.
06 / CITATION

Cite this work.

Chunk-wise Model-based Policy Optimization
Conference on Robot Learning, 2026

BIBTEX
@inproceedings{song2026cmpo,
  title     = {Chunk-wise Model-based Policy Optimization},
  author    = {Song, Weixi and Wang, Donglin and Li, Yong-Lu},
  booktitle = {Conference on Robot Learning},
  year      = {2026}
}