Move or scroll to enter

OpenWAM

OpenWAM: An Open, Modular Exploration Towards Systematic World–Action Model Pretraining

Yuran Wang^1*‡, Siqiao Huang^2*‡, Mingleyang Li^3*, Chenhao Zhang^3*, Jiaqi Liang^3*, Weiyang Jin⁴, Yue Chen³, Xuemin Chi⁵, Donghao Zhou⁶, Qize Yu³, Yu-Kai Wang³, Yuhan Rui³, Shenzhe Yao², Zhen Yuan⁴, Zhenhao Shen³, Kefei Zhu³, Zijie Zhu⁴, Ning Gao⁷, Xiaowei Chi³, Guanqi He², Shanghang Zhang³, Hao Dong³, Lin Shao^1†, Hang Zhao^2†

* Equal contribution ‡ Project lead † Equal advising

CodearXivsoonModel & Data

Overview of the OpenWAM stack: OpenWAM-Infra on the left, OpenWAM-Study in the middle, OpenWAM-α on the right. — Figure 1. A modular infrastructure, a controlled study over six design questions, and OpenWAM-α, the model those answers produce.

Abstract

World–Action Models inherit world knowledge from video-generative priors, and channel it into executable control signals through embodied experience. Existing systems, however, are monolithic: the generative backbone, visual representation, architecture, information flow, inference procedure, and training data are tightly coupled, obscuring which design choices matter and why. We introduce OpenWAM, an open research stack that turns world–action pretraining into a controlled experimental program. OpenWAM-Infra factorizes the WAM design space into composable modules with unified training, inference, deployment, and evaluation. On this substrate, OpenWAM-Study examines three questions through controlled experiments: what to inherit, how world and action learning interact, and how their synergy scales; and distills three principles: upstream knowledge transfers through a sufficiently capable generative backbone and a compact, information-rich latent space; world–action synergy requires dedicated action capacity, explicit world-to-action information flow, and synchronized joint denoising; and embodied pretraining principally improves out-of-domain generalization, with one-stage co-training over egocentric and robot data integrating world coverage and action grounding. Composing these principles, we build OpenWAM-α, an open WAM pretrained on roughly 6,400 hours of egocentric human and robot data and evaluated across simulation and real-world benchmarks. Across the eight simulation benchmarks and the real-robot experiments, which together span embodiments from single-arm and bimanual manipulation to dexterous hands, OpenWAM-α delivers consistently excellent performance, sustaining its top-tier standing from simulation to the physical world. We release the full stack: infrastructure, evaluation protocols, pretrained models, and data recipes, to facilitate future research.

Video

A three-minute tour of the infrastructure, the study and the model.

One stack, three layers

Modularity makes world–action modeling a controlled experimental program rather than a set of coupled implementation choices.

OpenWAM-Infra

A modular infrastructure for world–action modeling

Factorizes the WAM design space into composable modules (dataloaders, encoders, backbones, architectures, attention masks) behind one trainer, one policy server, and one evaluation protocol spanning simulation and real robots.

4 visual encoders
5 video backbones
6 architecture variants
4 attention masks

OpenWAM-Study

Design principles for world–action synergy

Six controlled questions: architecture, backbone, visual representation, video↔action interaction, pretraining recipe, denoising strategy. Each is tested under the conditions the previous one established.

Q1 Architecture
Q2 Backbone
Q3 Representation
Q4 Interaction
Q5 Data recipe
Q6 Denoising

OpenWAM-α

From principles to a pretrained model

The recipe instantiated at scale on egocentric human and robot data through a unified action space, then evaluated across eight simulation benchmarks and three real-robot platforms.

518M frames (≈6,400 h)
80-D unified action space
8 simulation benchmarks
3 real-robot platforms

What the controlled study found

Three questions, asked in sequence. Each answer fixes the setup for the next.

Finding 1What world knowledge should a WAM inherit?
A WAM inherits upstream world knowledge most effectively through a sufficiently capable generative backbone and a compact, information-rich visual representation space. Reconstructive encoders are not the only option; representation encoders with dimension compression are also performant.
Finding 2How can we create synergy between world and action learning?
World–action synergy requires explicit world-to-action information flow during training, and synchronized joint denoising at inference.
Finding 3How can this synergy be scaled across domains?
Embodied pretraining primarily expands OOD generalization. Robot trajectories preserve action grounding, egocentric video broadens transfer, and one-stage co-training integrates both effectively. At pretrained scale, mutual world–action visibility is consistently preferred.

The accumulated recipe

Stage	Finding	Carried-forward default
Inherit	Capable video backbones and compact representation latents transfer the strongest upstream priors.	Wan2.2-TI2V-5B; compact latent
Interact	Dedicated action capacity and world-to-action visibility are necessary; synchronized denoising performs best.	Dual joint self-attention; synchronized denoising
Consolidate	Embodied pretraining primarily improves OOD generalization and consistently favors mutual visibility.	One-stage ego + robot co-training; mutual visibility

OpenWAM-α

The recipe instantiated at scale: a dual-system model with mutual world–action visibility, co-trained in one stage on 518.5M frames (6,369 hours) of egocentric human and robot data through an 80-D unified action space.

Visual encoderQ3
Wan2.2-VAEfrozen
Frozen. Encodes causally: the first frame alone, then groups of four. The first latent frame therefore stays a clean anchor of the present.
World streamQ2
Wan2.2-TI2V-5B
Pretrained DiT, width 3072, 3D RoPE over the (frame, height, width) grid. Denoises the future frames of the video window in latent space.
Action streamQ1
ActionDiT
Dedicated backbone, width 1024, 1D RoPE over the chunk index. One token per noised action step.
CouplingQ1
Joint self-attention
All 30 paired layers act as bridge layers, projecting both residual widths into a shared 24-head × 128-dim attention space.
VisibilityQ4
Mutual mask
Both streams read each other freely; clean first-frame rows attend to neither noised future frames nor actions.
Language
umT5frozen
Frozen. Maps the instruction into a 4096-dim context that both streams consume through their own cross-attention.

From simulation to the physical world

OpenWAM-α holds its standing across eight simulation benchmarks and three real-robot platforms, including a dexterous hand whose embodiment never appears in the mixture above.

OpenWAM-αPrevious WAM SOTAPrevious VLA SOTA

Hover or tap an axis

LIBEROSimulation

Single-arm · Average success rate

OpenWAM-α 99.3vs best WAM -0.1vs best VLA +0.1

RoboDojo bimanual, real robot: 37.6 / 24.4score / SR
Simulation coverage: 8benchmarks
Pretraining data: 6,400hours
Open checkpoints: 46models

Real-robot rollouts

The same policy across three real-robot experiments, from a single-arm gripper to bimanual platforms to a dexterous hand. Every task and camera view, scrollable left and right.

Single-arm gripper

Six tasks, each from the fixed scene camera.

6 clips

Hang M

Chili into drawer

Hang cup

Jenga

Jenga into drawer

Stack rings

Hang M

Chili into drawer

Hang cup

Jenga

Jenga into drawer

Stack rings

Bimanual — RoboDojo

Successful runs across the three bimanual platforms, four each. OpenWAM-α averages 37.6 score / 24.4 SR over all eighteen tasks, against 22.9 / 12.8 for the next best.

12 clips

cover blocksARX X5

insert tubesARX X5

pack and pour fruitARX X5

store in safeARX X5

fill pen holderPiper

put objects into basketPiper

stack bowlsPiper

stand up bottlesPiper

cap penPiper X

disassemble legoPiper X

pack objects into backpackPiper X

sweep blocksPiper X