mujoco-playground
Installation
SKILL.md
MuJoCo Playground Sim-to-Real Composition (playground_sim2real.py)
Compose live MuJoCo Studio visualization, perception, IK, collision-aware planning, and visual servoing on mujoco_playground's out-of-the-box so101 and yam 50 Hz DeviceLayer stacks with identical code in mode="sim" and mode="real":
-
Workspace Setup:
- Clone and sync:
git clone https://github.com/google-deepmind/mujoco_playground.git && cd mujoco_playground && make sync-cpu(ormake sync/make sync-cu12on Linux CUDA; adduv sync --extra so101or--extra yam --default-index https://pypi.org/simpleon robot hosts, and pass--no-synctouv run). - Install composed skill packages into
.venv:uv pip install --python .venv/bin/python --default-index https://pypi.org/simple ruckig opencv-python-headless scipy mink daqp. SetMUJOCO_GL=osmesaonly on headless Linux (omit on macOS).
- Clone and sync:
-
Unified
sim/realDeviceLayerExecution (PlaygroundSim2Real):- Wrap
SO101Follower/SO101FollowerDevice(so101_aloha_table, arm inrad, gripper in[0, 100]) orYamFollower/YamFollowerDevice(yam_aloha_table, arm inrad, gripper in[0, 1]) viawith PlaygroundSim2Real(robot="yam", mode="sim", setup_scene=...) as env:, passingsetup_scene(spec)(mujoco-spec-editing/mujoco-python) to customize objects, containers, conveyors, or camera/base poses for any scene. - Never bypass the follower
DeviceLayeror read privileged objectmj_data.xposinside controllers;PlaygroundSim2Realsteps throughfollower/goal_position/*, readsfollower/joint_position/*+ camera streams, and cosine-interpolates safely toFOLLOWER_HOMEonclose().
- Wrap