Environments

RoboDojo: install and run one episode

Read shared setup for host slots, provider profiles and perception services. RoboDojo uses its Isaac Sim environment, X5 arms and the upstream CuRobo v2 fork. The official task success function supplies the verdict.

At a glance

Two fixed-base ARX X5 arms (6-DoF each) in Isaac Sim. 54 admitted tasks, each with a reviewed completion contract: the 42 official tasks and the *_random variants of the 12 generalization tasks (below). Each task's action limit and completion contract is on RoboDojo tasks. Needs an NVIDIA GPU with a working Vulkan stack for Isaac Sim, and a qualified lane (below).

Observation head (fixed), left_wrist and right_wrist (on the arms) RGB-D with per-observation calibration; both arms' link6 poses, normalized gripper encoder readings, controller feedback, the last action's receipt and the remaining budget, including native actions left
Action Sparse per-arm motion proposals through the Supervisor (move_pose, move_to, rotations, set_gripper, recover_pose, wait_ticks), plus grasp_plan / grasp_execute and align_plan; poses name the link6 flange. A move may use up to 250 native actions of the task's allowance, one trajectory, and other commands 20 or 30
Success RoboDojo's official task result. It ends the episode the moment it holds; the default surface has no check_task, and finish_episode can only give up
Task prompt The task's public instruction, plus its reviewed completion contract where one exists

Robot facts: ARX X5 · frames and units · policy I/O.

Install the isolated runtime

The maintained installer reference contains the exact sources, asset revisions, package manifest and graphics patch. Its tested recipe uses Python 3.11.15, Isaac Sim 5.1.0.0 and Torch 2.7.0/CUDA 12.8. Use a permanent root:

export ORA_DOJO_INSTALL=/srv/ora/robodojo
./scripts/setup-robodojo sources --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo assets --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo sdk --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo resolve --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo packages --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo bind-assets --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo doctor --root "$ORA_DOJO_INSTALL"
./scripts/setup-robodojo manifest --root "$ORA_DOJO_INSTALL"
export ROBODOJO_ROOT="$ORA_DOJO_INSTALL/source/RoboDojo"
export ROBODOJO_ASSET_ROOT="$ORA_DOJO_INSTALL/assets/Assets"
export ROBODOJO_PYTHON="$ORA_DOJO_INSTALL/env/bin/python"
uv run --locked ora doctor --env robodojo

Pinned revision

Since 2026-11-14 ORA pins RoboDojo f57e163 (XPolicyLab 5d0be14) and one Hugging Face asset tree, 1127bda, for every task; it replaced RoboDojo ee67a146 with assets a14409d7, whose swap_T layouts were no longer served and came from a separate successor checkout (assets-35efbc7d, ROBODOJO_SWAP_T_ASSET_ROOT, now retired). What the bump changes:

  • swap_blocks can now succeed. Upstream 487e425 pushes a spring button's restored drive target to PhysX on reset; at ee67a146 the target never reached PhysX, the button settled partly sagged, none of its presses registered and the task could not reach the official success. press_by_number's buttons go through the same reset, so its reset state changes too.
  • organize_table and stack_blocks_by_language deliver new instructions: organize_table no longer asks to open the drawer and put the remaining items inside (its contract's fourth clause changed with it), and stack_blocks_by_language gains the missing space in "in the order of".
  • Results for these four tasks (swap_blocks, press_by_number, organize_table, stack_blocks_by_language) from before and after the bump are not pooled. Every other task's source, layouts and contract are byte-identical; its records were re-issued only for the new pins (task-contract registry-v9).
  • The *_random variants (stack_bowls_random, push_T_random and the others, each its own upstream task module with its own cluttered layouts) run with their base task's native-action limit and physics device. Since 2026-11-14 each has a reviewed completion contract (task-contract registry-v10, the default), so the task roster, the composer and ora run offer all 54 tasks. A variant's record reviews its own layout 0 against its base task's record: eleven keep their base task's instruction, and the policy reads the base task's contract text unchanged. Nine of them also keep its success check byte for byte; stack_blocks_random and make_toast_random change only private tolerances (the stacking offset, the slice height in the toaster). sweep_blocks_random asks to sweep "the objects" and scores every toy brick, earbud and paper ball (four to six per layout) instead of three blocks, so it has its own text. The layout-portability table v5 extends each variant's contract to its layouts 0-9 (hang_mugs_random: all but layout 5, whose clutter holds a cup that could pass for a fourth mug), so --layout k runs those layouts. No variant has a reviewed layout pool, so a variant runs one-round episodes. registry-v9 and layout-portability v4 stay in the tree for specs composed before v10; nothing in the tree composes them.

Card and task qualification records made at the earlier pins do not carry over; each lane is re-qualified on an installation of the new pins.

sources, assets and sdk may run independently. Later stages depend on those inputs. Installation does not launch a simulator or qualify physics. The installer preserves source/package/asset receipts and supports reuse of matching components. Isaac SDK and assets are large separate downloads; consult their licenses and check local disk capacity first. No portable VRAM or installation-time minimum has been established by ORA. Keep the three ROBODOJO_* paths in your private .env (see .env.example).

Vulkan loader and compatibility layer

Isaac Sim renders through Vulkan, so the host needs a Vulkan loader (libvulkan.so.1, for example Ubuntu's libvulkan1). If you cannot install it system-wide, put a directory holding it on LD_LIBRARY_PATH before starting ORA; the isolated worker inherits that variable.

NVIDIA 595 drivers report an unbounded maxMemoryAllocationSize, which crashes Isaac Sim 5.1 at RTX startup. On these drivers qualification and preflight require the process-local Khronos Profiles layer that caps the limit. Build it once into the runtime root (needs Git, CMake, a C++17 compiler and network access for the layer's pinned dependencies; a few minutes):

./scripts/setup-robodojo graphics-compat --root "$ORA_DOJO_INSTALL"

The stage clones KhronosGroup/Vulkan-Profiles at the pinned revision, applies the tracked core-identity patch, builds only the layer, and writes graphics-compat/ with its hash receipt installation.json. It changes no driver, SDK or shell profile; the worker activates the layer for itself and checks the repaired properties before every start. The installer reference explains the repair. graphics-compat/ belongs to that runtime root: the worker finds it next to env/ (ROBODOJO_PYTHON is <root>/env/bin/python), and the Kit cache and data directories must be real directories inside the same root, not symlinks into another installation.

uv run --locked ora doctor --env robodojo reports vulkan_loader and, on drivers that need it, vulkan_compatibility_layer.

Code-mode sandbox

RoboDojo's default policy is code mode (robodojo_toolkit_v7, since 2026-11-05; see code mode by default): the model's programs run in a bubblewrap sandbox. The host needs bubblewrap (bwrap on PATH, at ~/.local/bin/bwrap or named by ORA_CODE_MODE_BWRAP; ora setup --env robodojo unpacks one under its prefix and names it there when the machine has none) and unprivileged user namespaces, which ORA cannot install (bwrap --unshare-all --ro-bind / / true must succeed; see the one-GPU guide for what to install or allow), and the sandbox interpreter, built from its lock:

uv venv --python 3.11 "$PREFIX/envs/code-mode"
uv pip install --python "$PREFIX/envs/code-mode/bin/python" --require-hashes --no-deps \
  --only-binary :all: --compile-bytecode -r ora/agent/code_mode/sandbox-requirements.txt
export ORA_CODE_MODE_PYTHON="$PREFIX/envs/code-mode/bin/python"

./scripts/ora setup --env robodojo builds the same at <prefix>/envs/code-mode; without ORA_CODE_MODE_PYTHON the sandbox uses that path under ORA_GPU_PREFIX (else ~/.local/share/ora/gpu). ora doctor --env robodojo reports code_mode_bwrap, code_mode_user_namespaces and code_mode_interpreter, and launch preflight's code_mode_sandbox refuses a code-mode plan, with what to install or set, when any is missing. A RunSpec with "code_mode": null runs tool calls and needs none of them.

Qualify the lane

Inspect supported task identities:

./scripts/ora spec robodojo tasks

On the GPU/slot registered in your host profile, obtain a card qualification and a task record. These are simulator-only development checks, not model episodes:

./scripts/ora qualify robodojo card stack_bowls \
  --gpu 1 --resource-profile robodojo-gpu1 --yes
./scripts/ora qualify robodojo task stack_bowls \
  --gpu 1 --resource-profile robodojo-gpu1 --card PATH_TO_CARD_RECORD --yes

Each command runs two full-length simulator episodes. On one data-center GPU lane, stack_bowls took 38 minutes for the card and about 60 minutes for the task record; budget both before an episode on a new lane. Reuse a still-valid card record whose actual source, controller, device and asset dependencies match. A task record admits the default layout 0 only. For one episode on another official layout (--layout k, a layout the reviewed layout-portability table lists for the task; see Seeds and case lists) or a multi-round session, use the lane operator's smoke and offline revalidate commands with the exact layout or schedule; see their --help.

Records are written to robodojo-compat/qualification/ under the outputs root (ORA_OUTPUTS_ROOT when set, otherwise <workspace>/outputs), outside Git. The task step and the composer bind them as workspace-relative outputs/robodojo-compat/... pairs; with a separate outputs root, ORA creates the workspace link outputs/robodojo-compat onto it. A record elsewhere is refused.

Compose and launch

Obtain a perception readiness report, then:

./scripts/ora spec robodojo compose stack_bowls \
  --experiment dojo-example --gpu 1 --resource-profile robodojo-gpu1 \
  --task-qualification PATH_TO_TASK_RECORD \
  --service-report PATH_TO_READINESS_REPORT --cohort dojo-example-g001

Use the printed path in the review/launch commands. The composer inherits the selected default tools, provider, task instruction and a physics ceiling of 1000 times the task's official native-action limit, past anything a run reaches, so max_robot_decisions, max_model_calls and wall_time_s end it; a success past the official limit is recorded apart from the official one. Execution capture grows with the native actions a run takes (a fasten_screws run of 1704 wrote 1.1 GB), so budget disk from those allowances. The result records the official boolean verdict separately from infrastructure faults. Check actual elapsed time and tokens in the run rather than treating the budget as expected duration.

A missing Isaac import usually means the wrong interpreter or an incomplete SDK stage. Rendering failures require checking Vulkan/EGL and driver components. Asset verification failures should be repaired through the installer; do not replace source pins or qualification hashes to force admission. Physics devices and layout schedules are task-bound, so evidence for one device/layout cannot silently qualify another.

Free play

ora play --env robodojo plays one of ORA's own Isaac Sim tabletops (scene set isaac-tabletop-v1: breakfast, desk, toys, kitchen_tidy, blocks) with no task, on the same installation, embodiment, cameras and default tools (without check_task and finish_episode). The official tasks are not loaded in free play. Before the first session, audit the scenes on your RoboDojo slot; the audit needs no task record or card qualification:

./scripts/ora play audit --env robodojo --resource-profile robodojo-gpu1
./scripts/ora play --env robodojo --provider PROFILE --service-report PATH_TO_READINESS_REPORT --confirm-launch

A scene runs on its own worker backend (robodojo_play) with the scene's physics device (cpu for breakfast, whose toaster and egg carton are articulated; cuda:0 for the others). Its objects carry gentler PhysX limits than RoboDojo's defaults (depenetration 0.5 m/s, speed 3 m/s, spin 720 deg/s): an object the arm sweeps or the gripper squeezes tips over or slides away instead of flying off the table. After each action the scene keeps running, both arms holding their targets, until its objects are at rest, for at most 2 s of simulated time: physics otherwise advances only inside an action, so an object the arm knocked over would hang in mid-fall until the next one. The settle belongs to the action's recorded motion and is logged per action in native/post-action-settle.jsonl; it is not part of the receipt and does not count toward the session's physics budget. A free-play run records its execution capture at 30 fps of simulated time (one frame every ninth 4 ms physics step) with the operator camera at 1280 × 720 beside the three 640 × 480 robot cameras, so the web page and an exported video show the motion as it happened; benchmark episodes keep 10 fps and have no settle. Measured on toys, an action takes about 1.1 s more wall time than at 10 fps and 640 × 480 (2.7 s instead of 1.6 s on average) and writes three times the frames' bytes (24 MB instead of 8 MB), plus the settle's own time while something still moves (0.5 s of simulated time after a sweep cost 3.8 s). None of this changes the RoboDojo worker that benchmark episodes and their card and task records bind. The free-play guide lists the scenes, the audit's checks and how /reset moves between seeds. A scene's audit stays valid until the play code or that scene's own sources (its YAML, its entry in the set's index.yml, the layout generator) change; editing one scene re-audits only that scene.

Source: docs/environments/robodojo.md