Training Object Permanence in World Models

Human preference leaderboard of the 14 evaluated models on the 300-question WROP exam. Twenty raters produced 361 blind pairwise judgments between the fourteen models (50–52 per model; ties count 0.5 for each side); strengths are Bradley–Terry maximum-likelihood estimates, regularised by one virtual draw per model, rescaled to an Elo scale with mean 1500. 95% confidence intervals come from 1,000 rater-clustered bootstrap resamples; overlapping intervals should not be read as significant rank differences. Score rate is the raw win rate. Wan 3.0 Prime and MiniMax H3 have identical records and tie for first.

Rank	Model	Class	Elo	95% CI	Score rate	Games
1	Wan 3.0 Prime	Reference-to-video	1723.6	[1629.2, 1864.9]	77.9%	52
2	MiniMax H3	Reference-to-video	1723.6	[1644.5, 1837.6]	77.9%	52
3	PWM-WROP (ours)	True continuation	1679.5	[1603.5, 1781.5]	73.1%	52
4	Seedance 2.5	Reference-to-video	1649.6	[1554.2, 1751.5]	69.6%	51
5	Runway Aleph 2	Edit / transfer	1518.3	[1425.1, 1621.6]	52.9%	51
6	Wan-VACE 14B	Edit / transfer	1506.7	[1429.9, 1590.8]	51.0%	52
7	Gemini Omni Flash 1.1	Edit / transfer	1492.5	[1404.1, 1572.9]	49.0%	52
8	Kling O3 Pro	Edit / transfer	1471.3	[1404.0, 1534.0]	46.2%	52
9	Grok Imagine (video extend)	True continuation	1457.0	[1362.7, 1555.1]	44.2%	52
10	LTX-2.3 Extend	True continuation	1453.4	[1363.6, 1545.8]	43.1%	51
11	Cosmos3 Super	Edit / transfer	1409.2	[1318.2, 1488.7]	37.3%	51
12	LTX-2.3 Dev	Edit / transfer	1398.9	[1297.7, 1482.3]	36.5%	52
13	HY-OmniWeaving	Edit / transfer	1268.5	[1159.4, 1344.2]	21.0%	50
14	MAGI-1 24B	True continuation	1248.0	[1137.2, 1315.6]	19.2%	52

Answers of every model to every question: the benchmark dataset. Model weights: PWM-WROP. Method and per-family results: the paper.