Mundo 將模擬器納入世界模型，應檢查甚麼？

A scene you can edit, simulate and revisit is more useful than a video you can only watch. That’s the interesting idea in Mundo, the world-model project presented on Hugging Face: keep an explicit representation of the scene, run its dynamics, then use a video model to render what happens. [1]

The separation gives you something valuable to inspect before admiring the pixels: the state that is supposed to explain them.

I built a small MuJoCo example to make one consequence concrete. Two cubes with different masses follow the same falling trajectory. Apply the same horizontal force, and their paths separate. The first video alone wouldn’t tell you which mass was right. 🔎

Authored synthetic scene, simulated with MuJoCo 3.17.0. This illustrates a physical inference problem; it is not output from Mundo’s reconstruction or video model.

Follow the state through the picture

Mundo’s project describes a 3D proxy with objects and layout, a scene program containing physical properties, a simulator that updates the world, and a video renderer conditioned on the resulting proxy frames and context images. The intermediate representation connects geometry, actions and appearance. [2]

That creates useful debugging questions. If an object ends up in the wrong place, inspect its simulated trajectory. If the trajectory is sensible but the generated object changes shape, inspect the rendering. If the initial geometry is wrong, fix that before interpreting the interaction.

I’d separate the checks this way:

These questions can fail independently. A convincing render is helpful for inspection, but agreement with a proxy and agreement with reality are separate evaluations. A beautiful cube can still have a fictional mass. 🎬

The mass disappears from free fall

In the deliberately simple scene I used, there is uniform gravity, no contact, no air drag and no joint damping. Newton’s equation in the vertical direction is:

mass × vertical acceleration = mass × gravity
vertical acceleration        = gravity

Mass cancels. Give a 1 kg cube and a 10 kg cube the same initial pose and velocity, and the ideal falling paths match.

Now add a known sideways force F. Horizontal displacement from rest becomes:

x(t) = F × t² / (2 × mass)

Mass no longer cancels. This is an example of identifiability: whether the observations distinguish the parameter you’re trying to infer. Adding more pictures of the same uninformative motion doesn’t necessarily solve it. Changing the intervention can.

Run the counterexample 🧪

Save the following as mass_demo.py. With uv installed, I ran it using:

uv run --python 3.12 --with mujoco==3.17.0 python mass_demo.py

The official Python bindings expose MuJoCo’s model, simulation data and stepping functions. This example needs no graphical viewer. [3]

import mujoco

def run(mass, force):
    model = mujoco.MjModel.from_xml_string(f'''<mujoco>
      <option timestep="0.001" gravity="0 0 -9.81" integrator="RK4"/>
      <worldbody><body name="cube" pos="0 0 2">
        <freejoint/><geom type="box" size="0.05 0.05 0.05" mass="{mass}"/>
      </body></worldbody>
    </mujoco>''')
    data = mujoco.MjData(model)
    body = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_BODY, "cube")
    for _ in range(400):
        data.xfrc_applied[body, 0] = force
        mujoco.mj_step(model, data)
    t = data.time
    x, z = data.qpos[0], data.qpos[2]
    assert abs(x - force * t*t / (2 * mass)) < 1e-10
    assert abs(z - (2 - 9.81 * t*t / 2)) < 1e-10
    return x, z

for force in (0, 1):
    for mass in (1, 10):
        x, z = run(mass, force)
        print(f"mass={mass:2d} kg force={force} N: x={x:.4f} m z={z:.4f} m")

Output:

mass= 1 kg force=0 N: x=0.0000 m z=1.2152 m
mass=10 kg force=0 N: x=0.0000 m z=1.2152 m
mass= 1 kg force=1 N: x=0.0800 m z=1.2152 m
mass=10 kg force=1 N: x=0.0080 m z=1.2152 m

✅ I also checked the full trajectories against the analytic positions, rather than checking only the last frame. The numerical tolerance here is a local integration check under the stated assumptions; it says nothing about reconstruction accuracy in a real room.

State replay at 10× slower playback. The outlines are a 2D schematic drawn from saved simulation positions, not generated video. The static chart above shows the same trajectories.

Choose an action that reveals the hidden parameter

The practical payoff is an evaluation design.

Suppose two reconstructed scenes look equally plausible, but they assign different masses to an object. Replaying a gravity-only fall won’t distinguish those masses in this ideal setup. A calibrated applied force can. If the force is unknown too, the motion may reveal only a force-to-mass ratio, leaving another ambiguity.

That suggests saving the physical parameters alongside the scene, recording which came from measurement and which were inferred, and testing actions that distinguish plausible alternatives. For a robot interaction, I'd want the action and state traces as well as the rendered observation.

⚠️ A physics engine advances the model you give it. It cannot establish that your inferred parameters describe the real object. MuJoCo’s own computation documentation treats contact models as approximations with distinct strengths and weaknesses. Even a correctly implemented solver leaves a modeling question to answer. [4]

Mundo’s explicit state makes those questions easier to pose. That is its useful architectural contribution here: scene edits and interventions have a representation you can inspect and compare, instead of disappearing into a video prompt.

The authors also report limitations with small-object interactions, visual consistency over extended changes, and generation speed. Their repository lists code and weights for the end of November. The launch examples therefore provide a system to examine, while the executable example above uses standalone MuJoCo. [1, 5]

I’d judge a world-model demo by keeping its state trace and its video side by side. The interesting moment is when they disagree: that tells you where to start looking. 💡

References

1. Mundo: from seeing a place to simulating what happens there.
2. Mundo project and technical overview.
3. MuJoCo Python bindings.
4. MuJoCo computation and contact modeling.
5. Mundo repository and release TODO.

Asset credits: original chart and animated schematic from the saved synthetic MuJoCo trajectories.

AI disclosure

AI was used to draft the article.