ORA v0.1: Robot-use agents now write code, move smoothly, and reach 40% on RoboDojo
TL;DR
ORA is an open harness: a language model operates a robot with cameras and tools, and no robot training[1]. v0.1 with the open-weight DeepSeek-V4.1-Flash[2] solves 39.9% of RoboDojo[3] zero-shot, up from 5.2% in October's preview[4, 5].
- Write code.The model writes Python against the same robot tools; one turn carries 3.6 robot actions instead of 1.1.
- Move smoothly.Every move is planned around what the depth cameras see; moves stopped by a collision fall from 14.1% to 7.0% on RoboDojo.
- Work across open models.In code mode on 39 RoboDojo tasks, six open models solve from 48.7% (DeepSeek) down to 5.1% (Nemotron).
- Cold-start post-training.On RoboLab-120, Qwen3.8-27B solves 5.4% alone, 52.5% inside ORA, and 61.9% with PEAR's training on top[6].
Overview
How ORA works, what v0.1 solves, and six solved episodes you can inspect.
ORA is a robot-use agent[1]: it brings the loop of computer-use agents[7, 8, 9] to robots, as a line of recent systems does[10, 11, 12, 13, 14, 15, 16, 17, 18]. The model asks for an action; the harness, with its manual, tools and context, carries it out on the robot and returns what happened; the model decides again. Language models drove robots zero-shot before these agents[19, 20]; ORA's loop is built on Pi[21].
The agent loop, around a robot
Tool calls. Each turn the model sends one tool call; the harness runs it and one result comes back before the model decides again.
Code mode. Each turn the model writes one program, which calls the same tools from a persistent workspace, up to 12 robot motions admitted one at a time, before one result comes back.
RoboDojo, zero-shot
Share of cases solved
We run the baselines and ORA v0.1 on DeepSeek-V4.1-Flash. Successes after the benchmark's step limit count.
How successes are counted
Across four benchmarks
Same model, run by us, % solved
RoboDojo
Uni-Robot-AgentORA, OctoberORA v0.1RoboLab
Uni-Robot-AgentORA, Octobernot run in OctoberORA v0.1RoboTwin
RPentORA, OctoberORA v0.1LIBERO-Pro
RPent*ORA, OctoberORA v0.1*RPent there runs a policy trained on LIBERO and an explored memory. October's LIBERO-Pro ran on other seeds; RoboLab was not run in October.
The numbers in full
What one turn carries
One call per turn against one program per turn
tool callsprograms
Robot actionsper acting turn
Reasoningper action, characters
DeepSeek-V4.1-Flash, 59 RoboDojo cases in each setting.
Code mode
Moves stopped by a collision
Share of moves, %; lower is better
before the motion stackv0.1
Other parts of the harness changed between the two sets of runs too.
Smooth motion
Open models in v0.1
RoboDojo, 39 tasks, share of cases solved
tool callsprograms, a hue per maker
Four layouts per task for DeepSeek, two for the others.
Analysis across models
A harness and training, on one model
Qwen3.8-27B on RoboLab-120, % of episodes solved, scored as PEAR scores
PEAR's paperPEAR's paper, trainedour runour run, PEAR-trained
The model nobody trained, inside the harness, is ahead of the trained model without one; the trained model inside the harness is ahead of both.
Harness and model
Solved tasks
A tower from planks24 decisions · 85 motions · 73 s of motion
TaskBuild a tower using the wooden blocks and wooden boards.
1× · 73 s of motion- 1 of 24 · cycle_supR.py
- 2 of 24 · place_supR.py
- 3 of 24 · cycle_boardL.py
- 4 of 24 · place_boardL.py
- 5 of 24 · place_boardL2.py
- 6 of 24 · retract_verifyL.py
- 7 of 24 · cycle_upperL.py
- 8 of 24 · cycle_upperR.py
- 9 of 24 · place_upperR.py
- 10 of 24 · cycle_boardM.py
- 11 of 24 · place_boardM.py
- 12 of 24 · fix_boardM.py
- 13 of 24 · place_boardM2.py
- 14 of 24 · relay_tan.py
- 15 of 24 · stage_tan.py
- 16 of 24 · place_boardS.py
- 17 of 24 · place_boardS2.py
- 18 of 24 · place_roof.py
- 19 of 24 · place_roof2.py
- 20 of 24 · place_roof3.py
- 21 of 24 · verify_tower.py
- 22 of 24 · roof_yaw_do.py
- 23 of 24 · roof_yaw_do2.py
- 24 of 24 · roof_place3.py
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- return_home
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- rotate_wrist
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- rotate_wrist
- move_pose
- move_pose
- set_gripper
- return_home
- wait_ticks
- move_pose
- move_pose
- set_gripper
- move_pose
- rotate_wrist
- move_pose
- move_pose
- set_gripper
- return_home
Eggs into the holder32 decisions · 102 motions · 80 s of motion
TaskPlace the four eggs from the basket into the egg holder, then close the lid.
1× · 80 s of motion- 1 of 32 · step_up.py
- 2 of 32 · grasp1.py
- 3 of 32 · lift_clear.py
- 4 of 32 · basket_grip.py
- 5 of 32 · basket_grip2.py
- 6 of 32 · basket_grip3.py
- 7 of 32 · grasp2.py
- 8 of 32 · place1.py
- 9 of 32 · place2.py
- 10 of 32 · place3.py
- 11 of 32 · retract.py
- 12 of 32 · pick_place.py
- 13 of 32 · place4.py
- 14 of 32 · place5.py
- 15 of 32 · check_state.py
- 16 of 32 · survey.py
- 17 of 32 · pickplace2.py
- 18 of 32 · pickplace3.py
- 19 of 32 · place6.py
- 20 of 32 · home_measure.py
- 21 of 32 · last_egg.py
- 22 of 32 · place_last.py
- 23 of 32 · final_check.py
- 24 of 32 · seat_eggs.py
- 25 of 32 · seat2.py
- 26 of 32 · lid_probe.py
- 27 of 32 · free_arm.py
- 28 of 32 · escape.py
- 29 of 32 · escape2.py
- 30 of 32 · escape3.py
- 31 of 32 · lid_push2.py
- 32 of 32 · final.py
- move_pose
- grasp_execute
- grasp_execute
- grasp_execute
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- return_home
- set_gripper
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- return_home
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- return_home
- move_to
- move_pose
- recover_pose
- return_home
- rotate_wrist
- move_to
- move_to
- move_to
- move_to
- move_to
- move_to
- move_to
- move_to
- move_to
- move_to
- move_to
- return_home
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- return_home
Folding a cloth22 decisions · 97 motions · 74 s of motion
TaskFold the clothes neatly.
1× · 74 s of motion- 1 of 22 · step1.py
- 2 of 22 · step2.py
- 3 of 22 · step3.py
- 4 of 22 · step4.py
- 5 of 22 · step5.py
- 6 of 22 · fold_body.py
- 7 of 22 · unstick.py
- 8 of 22 · unstick2.py
- 9 of 22 · fold_R.py
- 10 of 22 · fold_R3.py
- 11 of 22 · fold_R4.py
- 12 of 22 · fold_R5.py
- 13 of 22 · fold_R6.py
- 14 of 22 · liftcheck.py
- 15 of 22 · fold_R7.py
- 16 of 22 · fold2.py
- 17 of 22 · fold3.py
- 18 of 22 · fold4.py
- 19 of 22 · pullback.py
- 20 of 22 · fold5.py
- 21 of 22 · fold6.py
- 22 of 22 · finish.py
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- set_gripper
- recover_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- set_gripper
- recover_pose
- move_pose
- set_gripper
- move_pose
- set_gripper
- recover_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- recover_pose
- recover_pose
- set_gripper
- return_home
Nuts onto bolts8 decisions · 39 motions · 28 s of motion
TaskInsert and tighten each screw into the nut of the same color.
1× · 28 s of motion- 1 of 8 · place.py
- 2 of 8 · place2.py
- 3 of 8 · pair.py
- 4 of 8 · pair.py
- 5 of 8 · pair.py
- 6 of 8 · pair2.py
- 7 of 8 · pair2.py
- 8 of 8 · final_check.py
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- return_home
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- set_gripper
- move_pose
- return_home
A xylophone, key by key6 decisions · 33 motions · 27 s of motion
TaskPick up the mallet and strike all xylophone keys from left to right.
1× · 27 s of motion- 1 of 6 · step_grasp.py
- 2 of 6 · step_rotate.py
- 3 of 6 · step_park.py
- 4 of 6 · step_strike.py
- 5 of 6 · step_strike2.py
- 6 of 6 · step_key7.py
- move_pose
- move_pose
- set_gripper
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
- move_pose
A game of tic-tac-toe15 decisions · 99 motions · 81 s of motion
TaskPlay tic-tac-toe as the first player and fill the board with the opponent.
1× · 81 s of motion- 1 of 15 · do_grasp.py
- 2 of 15 · do_place.py
- 3 of 15 · do_home.py
- 4 of 15 · do_wait.py
- 5 of 15 · step.py
- 6 of 15 · step_place.py
- 7 of 15 · do_wait2.py
- 8 of 15 · step.py
- 9 of 15 · place22.py
- 10 of 15 · do_wait2.py
- 11 of 15 · step.py
- 12 of 15 · verify_all.py
- 13 of 15 · do_wait2.py
- 14 of 15 · final_move.py
- 15 of 15 · place_final.py
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- move_pose
- move_pose
- set_gripper
- wait_ticks
- return_home
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- move_pose
- move_pose
- set_gripper
- wait_ticks
- return_home
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- move_pose
- move_pose
- set_gripper
- wait_ticks
- return_home
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- move_pose
- move_pose
- set_gripper
- wait_ticks
- return_home
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- wait_ticks
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- grasp_execute
- move_pose
- move_pose
1. Code mode
The same harness, as a Python library in a workspace that persists.
Code mode adds no robot capability: the cameras, the motion stack and the supervision that admits one robot command at a time are the same. What changes is the interface. Instead of one tool call per turn, the model writes Python in a persistent workspace and can run up to 12 robot motions in one decision. Programs run in a sandbox[27] with no network and reach the robot only through the same supervised tools. Programs as an agent's action have a history: language-model programs for robot control[28, 29], executable code in place of one tool call per turn[30, 31], and code mode for tool use[32, 33].
The gain over tool calls
On build_tower, the tool-call run gives up after 94 model calls with the planks still on the table; the code-mode run stacks the tower. One of its programs finds a block, measures it, grasps, lifts, places, and checks the height before it hands back. Across 59 paired RoboDojo cases a turn carries 3.62 robot actions instead of 1.10, and solved cases go from 6 to 20.
Tools written on the fly
Given a workspace, models fill it with tools of their own[34, 35]. Qwen[36] and DeepSeek build perception and geometry helpers, a median of 122 and 37 functions per episode, and import them in later turns; Qwen3.8-27B[37] writes 46, Inkling[38] 12, Muse[39] 7 and Nemotron[40] 4. More code is not more success: DeepSeek's failed episodes hold more functions than its solved ones.
Language models now make their own tools, on the fly
34,327 functions the models wrote in 727 RoboDojo episodes, code mode
Each dot is one function, a sample of 2,496, placed by how alike their source is.
Each square is one function of a model's median episode, on the same cases for every model.
37functions
62%of its helpers are used again on a later turn
122functions
67%of its helpers are used again on a later turn
12functions
53%of its helpers are used again on a later turn
7functions
32%of its helpers are used again on a later turn
46functions
65%of its helpers are used again on a later turn
4functions
75%of its helpers are used again on a later turn
2. Smooth motion
Every move is planned around what the depth cameras see and timed inside joint limits.
Between asking for a move and seeing the next picture, the model sees nothing. In October a move was a straight run at the target, as fast as the controller would go. Arms clipped boxes, hit each other and knocked finished towers over.
DigitsRoboDojo
The hand lets go of a digit and whips away; the digit goes with it.
A bowl by the cabinetLIBERO-Pro
Carrying a bowl out, the arm drives into the cabinet top and drops it.
A finished towerRoboDojo
Both arms pull away from the tower at full speed; planks and blocks burst across the table.
Pack the boxRoboDojo
The arm swings into the open box and sends it 0.9 m into the far corner.
Xylophone, two armsRoboDojo
Holding a mallet, the right arm swings into the left; the mallet falls to the table.
Build a towerRoboDojo
A 0.12 s sweep along a plank shoots it up on end.
Push the TRoboDojo
Coming down beside the T, the gripper lifts it onto its edge.
XylophoneRoboDojo
A quarter turn of the wrist whips the arm up; the mallet is gone.
Moves stopped by a collision% of moves, lower is better
beforev0.1
Other parts of the harness changed between the two sets of runs too.
In v0.1 every move goes through one tool, move_pose. The harness builds a 1 cm voxel map from the robot's depth cameras, plans around it with cuRobo[43], and times the path inside joint limits[44, 45]. A request that would collide, press into a surface or wind the wrist is refused before anything moves, with a named reason the model can act on. A guard stops an arm that falls behind its command.
Timed, not thrown.The same way out, paced inside each joint's limits: 0.80 s instead of 0.36 s. The digit moves 2 mm where before it flew 60 cm.
Beforethe run, as recorded
ORA v0.1same request, same state
and 2 more arguments
The hand moved 0.455 m in 0.36 s.
The digit: 597 mm, turned 145°.
It ran for 0.80 s.
The digit: 2 mm, turned 15°.
The hand's speed m/s, measured by the controller every 4 ms
beforeORA v0.1
time from the start of the move
Every path is timed inside each joint's speed and acceleration limits. The pace never changes the path.
where the model sees itIn the call, speed; not set here, so fast, the default. In the receipt, motion.duration_s and motion.speed.planned_peak_linear_m_s.
Up and over.The planned path lifts the hand 15 cm over the open box, which moves 2 cm. Before, the arm swung into it and it flew 0.9 m.
Beforethe run, as recorded
ORA v0.1same request, same state
and 4 more arguments
The hand moved 0.17 m in 0.48 s.
The box: 900 mm, turned 103°; the toothbrush: 881 mm, turned 76°.
It ran for 0.88 s; the hand rose 15 cm on the way.
The box: 21 mm, turned 12°; the toothbrush: 9 mm, turned 102°.
The hand's height above where it started cm, measured by the controller every 4 ms
beforeORA v0.1
time from the start of the move
A world of 1 cm voxels from the robot's own depth cameras, rebuilt for every move.
A path through that world from the measured joints: here, up and over the box.
where the model sees itIn the call, interaction: "free_space" asks for a planned path. In the receipt, motion.route: "global" says one ran.
It stops itself.Both arms leave on a planned, timed path; when their joints fall behind, the move stops after 0.24 s and the tower stands.
Beforethe run, as recorded
ORA v0.1same request, same state
and 8 more arguments
Recorded as move_pose_fast, which is move_pose at its default speed today; replayed as move_pose.
The hands moved 0.27 m (left) and 0.31 m (right) in 0.48 s.
The tower's pieces: up to 857 mm.
It stopped after 0.24 s of the 0.64 s planned, the hands 0.22 m and 0.26 m short of their goals.
The tower's pieces: up to 16 mm; it stands.
While the arms move, a guard reads how far each joint falls behind its command and stops a move that stays behind.
where the model sees itIn the receipt, failure_code: "sustained_joint_tracking_error", and each arm's pose_error: how far it still is from its goal.
Refused before it moves.At the goal the mallet in the hand would be 3.6 cm inside the other arm, so nothing moves, and the model is told why.
Beforethe run, as recorded
ORA v0.1same request, same state
and 2 more arguments
The hand moved 0.175 m in 0.12 s.
The mallet: 343 mm, turned 82°, dropped 13 cm onto the table.
Nothing moved.
The other arm is in the world where it stands, and what the hand holds moves with the hand.
Every goal is tested before anything moves; one that puts the mallet inside the other arm is refused.
where the model sees itA refusal in place of a receipt: code: "goal_unreachable", and the reason in words.
Refused: the wrist would unwind.For a 90° turn of the hand the planned path would turn it 252°, the long way round; nothing moves.
Beforethe run, as recorded
ORA v0.1same request, same state
The hand moved 0.304 m in 0.64 s, cut short.
The mallet: 701 mm, turned 92°.
Nothing moved.
How far the hand would turn degrees, the planner's answer to this request
The timed path is checked before anything moves: a hand that would turn more than 45° beyond what was asked is refused.
where the model sees itA refusal in place of a receipt: code: "paced_path_wrist_unwind", with requested_turn_deg and path_turn_deg.
move_posereworkedPut the hand at a poseabsolutedeltaworld_axeshold_current
One call is one move. The harness compiles the target, plans the path from the measured joints, runs it as one motion and answers with a receipt. In the preview a free move on RoboDojo was one inverse-kinematics solution and an interpolation between joints, with no check for collisions. Each arm gets a position (absolute, a delta, or held) and an orientation: a quaternion, a turn about an axis, held, or world_axes, which states the hand's pose as two directions, the way it approaches and the way the jaws close.
Comes back The compiled pose, the receipt (reached or not, how far from the target), the route taken and the speed the plan reached.
move_pose({"targets": {"left": {
"approach_world": [0, 0, -1],
"interaction": "free_space",
"jaw_line_world": [0, 1, 0],
"orientation_mode": "world_axes",
"position_m": [-0.29, -0.14, 0.97],
"position_mode": "absolute"}}})
→ left arm 0.1 mm / 0.0° from target, reached, TCP moved 218 mmRecorded: arrange largest number, layout 0, tool calls, decision 1
interactionSay what kind of move it is; the harness picks the pathfree_spaceprecisioncontact_approach
free_space: a collision-free path planned around what the cameras see now and the other arm, a held object moving with the hand. precision: the straight line of the hand when the whole change is within 50 mm and 15 degrees, a planned path otherwise. contact_approach: straight when it is short; when it is longer, a planned path to a point just short of contact, a stop, then the straight line for the last 40 mm. The model states a physical intent and never names a controller.
Comes back motion.route says which path ran: a planned one, a straight line, or the two in stages.
move_pose({
"slow_approaching": true,
"speed": "normal",
"targets": {"left": {
"interaction": "precision",
"orientation_mode": "hold_current",
"position_m": [-0.12671, -0.06119, 0.9316],
"position_mode": "absolute"}}})
→ left arm 0.1 mm / 0.0° from target, reached, TCP moved 43 mmRecorded: arrange largest number, layout 0, tool calls, decision 7
speednewSet the pace, never the pathfastnormalcarefulslow_approachingspeed_m_s
fast, the default: each joint at up to 4 rad/s. normal: about 0.3 m/s at the hand, for moves while holding something. careful: about 0.1 m/s, for the last centimetres of an approach or a placing. slow_approaching slows only the last 2 cm of a move. speed_m_s gives a top speed directly. Every plan is timed inside the joints' speed and acceleration limits.
Comes back The caps that applied and the top speed the planned path reaches under them.
move_pose({
"expect_contact": true,
"speed": "careful",
"targets": {"right": {
"approach_world": [0, 1, 0],
"interaction": "contact_approach",
"jaw_line_world": [-1, 0, 0],
"orientation_mode": "world_axes",
"position_m": [0.066, -0.122, 0.8516],
"position_mode": "absolute"}}})
→ right arm 1.9 mm / 0.4° from target, reached: arm not settled, TCP moved 49 mmRecorded: insert tubes, layout 0, tool calls, decision 20
dry_runnewLook at a move before making ittrue
Nothing moves and nothing is spent. The planned path is drawn on the camera images that see it, with its start and end, and the answer gives, per arm, how far the hand travels and turns, how high the path rises above its ends, and the least clearance between the jaws and what the depth images show, with where that is. The same call on the same observation then runs exactly the path shown. Every moving call takes it.
Comes back A picture and the numbers of the plan. It predicts no contact and nothing after the first contact.
pl = tools.move_pose(targets=tgt(p, it), dry_run=True)
print("dry", name, "feasible", pl.feasible, "code", pl.failure_code, "|", pl.summary[:160])
if not pl.feasible:
handoff("approach path not feasible", stage=name, code=pl.failure_code, summary=pl.summary)
→ dry grasp feasible True code None | left: 19 native actions, hand travels 0.289 m and turns 2 deg, rises 43 mm above its higher end at 42% of the path, least clearance 2 mm at 74% of the path, entRecorded: pour by language, layout 3, code mode, decision 1, step grasp.py lines 27 to 30
reach_checknewAsk whether the arm can hold a pose, without movingposesinteraction
A read-only question to the planner, in programs: for up to 16 poses of one arm at once, whether each can be reached, by which route, and how far its joints stay from their limits. A program uses it to choose among grasps or to check a whole pick and place before the first motion.
Comes back Per pose: reachable, a reason when it is not, the route, and the margin to the nearest joint limit in degrees.
res = query("reach_check", arm=arm, poses=poses)
→ reach unc-red grasp: True None margin=17.638704
→ reach unc-red lift: False ik_failed margin=NoneRecorded: cover blocks, layout 5, code mode, decision 8, cycle.py lines 28 to 28
expect_contactnewMean to touchtrue
For a move that is meant to push, press or move an object. It follows the straight line the whole way, at any length and speed, and the guard that stops an arm as soon as it meets resistance stands down; the hard limit on how far the arm may fall behind its command stays.
Comes back The receipt as for any move; a stop on such a move says that it was stopped despite expecting contact.
move_pose({
"expect_contact": true,
"slow_approaching": true,
"speed": "normal",
"targets": {"left": {
"interaction": "contact_approach",
"orientation_mode": "hold_current",
"position_m": [0.0496, -0.1785, 0.874],
"position_mode": "absolute"}}})
→ left arm 1.4 mm / 0.2° from target, reached, TCP moved 4.8 mmRecorded: play Xylophone, layout 1, tool calls, decision 69
the receiptreworkedWhat comes back: reached, stopped, or refused with a reasonreachednot_reachedrefused
A move that ran ends reached, or not_reached with what stopped it and how far along its path it was. A move the planner will not make is refused before anything moves, with a named reason, such as goal_unreachable, goal_in_collision, goal_below_support, path_blocked, inter_arm_collision or paced_path_wrist_unwind, and where it can, with something that would work. In a program a refusal is an exception, ToolRefused, that the program can catch and branch on.
Comes back The reason by name, and the measured state; nothing is executed on a refusal.
move_pose({
"slow_approaching": true,
"speed": "normal",
"targets": {"right": {
"interaction": "precision",
"orientation_mode": "hold_current",
"position_m": [0.0418, -0.0612, 0.928],
"position_mode": "absolute"}}})
→ right arm 14 mm / 0.9° from target, not reached: controller tracking error, TCP moved 58 mmRecorded: arrange largest number, layout 0, tool calls, decision 49
robot.ToolRefused: move_pose: no joint configuration reaches this pose even with nothing in the way; with the fingertips at the same point, an approach tilted 30 degrees plans: (0.15, 0.48, -0.87) in world coordinates instead of (0.00, 0.00, -1.00); the other arm does not reach this pose with nothing in the way; nothing was executed → refused: goal_unreachable
Recorded: cover blocks, layout 5, code mode, decision 9
recover_pose · return_homenewUndo a move, or start overarms
recover_pose backs an arm out the way it came, one of its last five moves per call, along the path it measured on the way in. return_home takes an arm straight back to the joints it started the episode in, by a planned path, keeping the grippers as they are.
Comes back A receipt, as for any move.
recover_pose({"arms": ["left"]})
→ left arm 0.1 mm / 0.0° from target, reached, TCP moved 68 mmRecorded: arrange largest number, layout 0, tool calls, decision 12
return_home({"arms": ["left"]})
→ left arm 0.0 mm / 0.0° from target, reached, TCP moved 450 mmRecorded: arrange largest number, layout 0, tool calls, decision 26
On matched cases, moves stopped by a collision fall from 14.1% to 7.0% on RoboDojo and from 18.4% to 6.8% on RoboTwin[24]. Light contact is not solved: about one move in ten still shifts an object by 5 mm or more.
3. Analysis across models
Six open models, the same prompt, tools and budgets, 39 RoboDojo tasks.
In code mode DeepSeek-V4.1-Flash solves 44.9% of these cases (48.7% over four layouts) and Qwen3.8-Flash-Next 35.9%; few of their programs fail (10% and 13%). Inkling-2-Small[38] solves 21.8% with programs of nine lines. Muse Spark Open (16.7%), Qwen3.8-27B (15.4%) and Nemotron-3.5-Super-VL (5.1%) lose about a third of their programs to a crash; for Muse and Nemotron it is mostly a refused call they did not catch. More reasoning is not what separates them: Qwen3.8-27B reasons the most, a median of 3,390 tokens per call against DeepSeek's 2,083. The open models were served with vLLM[46] and SGLang[47].
Across open models
Code mode, share of RoboDojo cases solved, 39 tasks
Four layouts per task for DeepSeek, two for the others.
- DeepSeek-V4.1-Flash: with programs 35 of 78 cases solved, a median of 2,083 reasoning tokens per model call, programs of 35 lines (median), 10% of programs fail.
- Qwen3.8-Flash-Next: with programs 28 of 78 cases solved, a median of 1,072 reasoning tokens per model call, programs of 40 lines (median), 13% of programs fail.
- Inkling-2-Small: with programs 17 of 78 cases solved, a median of 1,000 reasoning tokens per model call, programs of 9 lines (median), 21% of programs fail.
- Muse Spark Open: with programs 13 of 78 cases solved, a median of 341 reasoning tokens per model call, programs of 24 lines (median), 32% of programs fail.
- Qwen3.8-27B: with programs 12 of 78 cases solved, a median of 3,390 reasoning tokens per model call, programs of 41 lines (median), 31% of programs fail.
- Nemotron-3.5-Super-VL: with programs 4 of 78 cases solved, a median of 180 reasoning tokens per model call, programs of 17 lines (median), 39% of programs fail.
The traces are open: 3,557 episodes from seven models on five benchmarks, with everything each model was given and everything it wrote, are on Hugging Face.
Across the six, the RoboDojo result lines up with public scores for agentic coding far more than with those for visual reasoning.
Robot use against public benchmarks
Across the six models, r runs from 0.38 to 1.00 on evaluations where the model acts and from −0.21 to 0.45 on those where it answers questions; it is 0.84 on the Intelligence Index, a composite of nine evaluations.
The model acts agentic evaluations
The model answers question sets
4. Harness and model
What a harness adds, and what training adds on top of it.
Training a reasoning model to use robots[6, 49, 50] or a vision-language-action model[51] changes the model; a harness changes what the model is given. PEAR trains Qwen3.8-27B[37] with reinforcement learning in simulation to drive a robot with no tools. We ran the base model and PEAR's released checkpoint[52] inside ORA and scored them the way PEAR does.
On RoboLab-120[26] the base model goes from 5.4% alone to 52.5% inside ORA, ahead of the trained model alone (37.3%). The trained model inside ORA reaches 61.9%. RoboDojo's long-horizon tasks stay at 0% either way.
One model, with and without training, with and without a harness
No training: 5.4 becomes 52.5 in ORA. PEAR's training adds on top: 61.9.
All 120 tasks
Qwen3.8-27B on RoboLab-120, share of episodes solved, %
Qwen3.8-27Bno harness, PEAR's paper
PEAR-trainedno harness, PEAR's paper
Qwen3.8-27Bin ORA, our run
PEAR-trainedin ORA, our run
By task difficulty
trained, no harnessas released, in ORAtrained, in ORA
Simple128 episodes in ORA
Moderate78 episodes in ORA
Complex34 episodes in ORA
120 tasks. No harness: PEAR's paper, 10 episodes per task. In ORA: 2 episodes per task, 240 and 239 episodes.
No training: 0.08 becomes 10.0 in ORA. PEAR's training adds on top: 15.8.
All 54 settings
Qwen3.8-27B on RoboDojo, mean of five capability dimensions, %
Qwen3.8-27Bno harness, PEAR's paper
PEAR-trainedno harness, PEAR's paper
Qwen3.8-27Bin ORA, our run
PEAR-trainedin ORA, our run
By capability
as released, in ORAtrained, in ORA
Generalization
Precision
Long-horizon
Memory
Open-ended
54 settings. No harness: PEAR's paper, 10 episodes per setting. In ORA: 2 episodes per setting, the same 106 episodes for both models.
ORA is open source. To run it, go to the repository.
References
- 1Isola (2026). Robot-Use Agents.
- 2DeepSeek (2026). DeepSeek-V4.1-Flash-1017.
- 3Chen et al. (2026). RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies.
- 4Yang et al. (2026). ORA: Open Robot-Use Agents Zero-Shot Solve Robot Tasks for $0.35.
- 5Yang et al. (2026). ORA: Open Robot-Use Agents.
- 6Tang et al. (2026). Training Reasoning Models in Simulation to Natively Use Real Robots.
- 7OpenAI (2025). Computer-Using Agent.
- 8Anthropic (2024). Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku.
- 9Xie et al. (2024). OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments.
- 10Chu et al. (2026). OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies.
- 11Yuan et al. (2026). Democratizing Robot-Use Agents: A Unified Harness with Open Models.
- 12Zhang et al. (2026). Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents.
- 13Si et al. (2026). Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens.
- 14Huang et al. (2026). Scaling Online Exploration for Robot Agents.
- 15Lu et al. (2026). ASPIRE: Agentic /Skills Discovery for Robotics.
- 16Yang et al. (2026). Astranomy: An Empirical Study of GPT-6 Astra as a Native Robot Policy.
- 17OpenRoboKit (2026). RoboSkills: A minimal set of agent workflows and tools for robotics.
- 18StellarEdge AI Technical Team (2026). Auriga Technical Report: A Task-Agnostic Agent Harness for Zero-Shot Embodied Manipulation.
- 19Ahn et al. (2022). Do As I Can, Not As I Say: Grounding Language in Robotic Affordances.
- 20Huang et al. (2023). VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models.
- 21Earendil Works (2025). Pi.
- 22RoboDojo Team (2026). RoboDojo Sim Leaderboard, Agent Track.
- 23Zhang et al. (2026). An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond.
- 24Chen et al. (2025). RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation.
- 25Zhou et al. (2025). LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorization.
- 26Yang et al. (2026). RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies.
- 27Containers (2016). Bubblewrap: low-level unprivileged sandboxing tool.
- 28Liang et al. (2022). Code as Policies: Language Model Programs for Embodied Control.
- 29Fu et al. (2026). CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation.
- 30Yao et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models.
- 31Wang et al. (2024). Executable Code Actions Elicit Better LLM Agents.
- 32Varda and Pai (2025). Code Mode: the better way to use MCP.
- 33Anthropic (2025). Code execution with MCP: Building more efficient agents.
- 34Wang et al. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models.
- 35Cai et al. (2023). Large Language Models as Tool Makers.
- 36Qwen Team (2026). Qwen3.8-Flash-Next.
- 37Qwen Team (2026). Qwen3.8-27B.
- 38Thinking Machines Lab (2026). Inkling-2-Small.
- 39Meta (2026). Muse-Spark-Open-816B-A42B.
- 40NVIDIA (2026). NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16.
- 41Reimers and Gurevych (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.
- 42van der Maaten and Hinton (2008). Visualizing Data using t-SNE.
- 43Sundaralingam et al. (2023). cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation.
- 44Pham and Pham (2018). A New Approach to Time-Optimal Path Parameterization Based on Reachability Analysis.
- 45Berscheid and Kröger (2021). Jerk-limited Real-time Trajectory Generation with Arbitrary Target States.
- 46Kwon et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention.
- 47Zheng et al. (2023). SGLang: Efficient Execution of Structured Language Model Programs.
- 48Artificial Analysis (2026). Artificial Analysis Intelligence Index.
- 49Liu et al. (2026). AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics.
- 50Fan et al. (2026). CaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents.
- 51Black et al. (2025). π0.5: a Vision-Language-Action Model with Open-World Generalization.
- 52MIT HAN Lab (2026). Qwen3.8-27B-PEAR.
- 53OpenRUA Team (2026). Behind the RoboDojo Boom: Revisiting Embodied Evaluation of Robot-Use Agents.
- 54Zhao et al. (2023). Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware.
- 55Haddadin et al. (2022). The Franka Emika Robot: A Reference Platform for Robotics Research and Education.
- 56Khazatsky et al. (2024). DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset.
- 57OpenAI (2026). GPT-6 Astra: A new generation of intelligence.
- 58Mittal et al. (2025). Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning.
- 59Xiang et al. (2020). SAPIEN: A SimulAted Part-based Interactive ENvironment.
- 60Zhu et al. (2020). robosuite: A Modular Simulation Framework and Benchmark for Robot Learning.
- 61Liu et al. (2023). LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.
- 62Carion et al. (2026). SAM 3: Segment Anything with Concepts.
- 63Sundermeyer et al. (2021). Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes.
Citation
To cite this post:
@misc{ora2026v01,
title = {{ORA} v0.1: Robot-Use Agents Now Write Code, Move Smoothly, and Reach 40\% on {RoboDojo}},
author = {Yuanbo Yang and Junda Chen and Quanquan Peng and Yulun Wu and Shaoxiong Duan and Hao Zhang},
year = {2026},
month = nov,
howpublished = {FastRUA Blog},
url = {https://fastrua.ai/blog/2026-11-code-mode/}
}
Appendix
How successes are counted
RoboLab[26] and RoboDojo[3] limit the simulator steps of an episode. ORA records the benchmark's verdict at the limit and lets the episode go on; this post counts a success reached later too, except in section 4, which follows PEAR[6] and counts only successes within the limit. RoboTwin[24] and LIBERO-Pro[25] end an episode at their limit.
| Comparison | All successes | Within the step limit | |
|---|---|---|---|
| RoboDojo, 198 cases | ORA v0.1 | 79 (39.9%) | 21 (10.6%) |
| Uni-Robot-Agent | 55 (27.8%) | 32 (16.2%) | |
| RoboLab, 300 cases | ORA v0.1 | 235 (78.3%) | 200 (66.7%) |
| Uni-Robot-Agent | 187 (62.3%) | 179 (59.7%) | |
| RoboLab, 500 cases | ORA v0.1 | 79.2% | 67.0% |
| RoboTwin, 235 cases | RPent | 167 (71.1%) | 151 (64.3%) |
| Paired experiment, 59 cases | tool calls | 6 | 2 |
| programs | 20 | 5 | |
| Open models: DeepSeek, 156 cases | tool calls | 58 | 36 |
| programs | 76 | 30 | |
| Qwen, 78 cases | tool calls | 29 | 17 |
| programs | 28 | 7 | |
| Inkling-2-Small, 78 cases | tool calls | 12 | 6 |
| programs | 17 | 6 | |
| Muse, 78 cases | tool calls | 9 | 6 |
| programs | 13 | 8 | |
| Qwen3.8-27B, 78 cases | tool calls | 10 | 6 |
| programs | 12 | 5 | |
| Nemotron, 78 cases | tool calls | 3 | 3 |
| programs | 4 | 2 |
Within the limit Uni-Robot-Agent[11] solves more RoboDojo cases than ORA, and for DeepSeek and both Qwen models tool calls solve more than programs. RoboDojo's leaderboard[22] counts within the limit only, so 10.6% is the number nearest to it. We count later successes because the limits were set for policies that act at a fixed control rate[53], and a late success passes the benchmark's own check. Uni-Robot-Agent ran with its own limit of 3 hours per episode, ORA with 6.
The numbers in full
ORA v0.1 with DeepSeek-V4.1-Flash, zero-shot, every success counted:
| Benchmark | Robot | Cases | Programs | Tool calls |
|---|---|---|---|---|
| RoboTwin 2.0, clean to randomized | two-arm ALOHA[54] | 235 | 79.6% (187) | 80.4% |
| LIBERO-Pro | Franka Panda[55] | 800 | 87.5% (700) | 85.4% |
| RoboLab, 100 of its 120 tasks | DROID[56] Franka | 500 | 79.2% (396) | |
| RoboDojo | two-arm ARX X5 | 198 | 39.9% (79) |
Baselines, run by us on the same cases: Uni-Robot-Agent[11] solves 55 of 198 on RoboDojo and 187 of 300 on RoboLab; RPent[12] solves 167 of 235 on RoboTwin, 738 of 800 on LIBERO-Pro with its LIBERO VLA[51] and explored memory, and 11 of 200 on RoboDojo with its VLA. On RoboDojo's leaderboard[22, 23], read on 16 November 2026, GPT-6-Astra[57] scores 22.48% and DeepSeek V4.1 Flash 1.92%.
The motion stack
Under every planned move, on every simulator[58, 59, 60]:
| Stage | What it does |
|---|---|
| Collision world | Rebuilt for each request from the robot's own depth cameras only (RoboDojo: the head and both wrists; RoboTwin: four cameras; LIBERO[61]: two), as 1 cm voxels. The robot's own body is removed; the start pose and a box around the hand at the goal are cleared; a held object moves with the hand; space no camera sees counts as free. A goal that puts a fingertip more than 1 cm below the surface under it is refused before planning. |
| Path | cuRobo[43] plans from the measured joints, with the other arm as an obstacle where it stands. The path is checked a second time against the scene without the clearing at the goal. |
| Wrist | Along the path the hand may turn at most 45 degrees more than between its two ends; beyond that the move is refused (paced_path_wrist_unwind). With empty jaws the same grasp turned a half-turn about the approach is tried as well. |
| Timing | TOPP-RA[44], then Ruckig[45]. fast: each joint at up to 4 rad/s and 40 rad/s². normal: 0.3 m/s and 60 degrees/s at the hand. careful: 0.1 m/s and 20 degrees/s. slow_approaching: the last 2 cm at careful. |
| Contact | A move into contact is a straight line when it is within 50 mm and 15 degrees; a longer one is planned to a point short of contact and is straight for its last 40 mm. With expect_contact the whole move is straight. The guard that stops a blocked arm reads how far the joints fall behind their commands; there is no force sensing. |
| Refusal | Nothing moves and nothing is spent. The model is told one of goal_unreachable, goal_in_collision, goal_below_support, start_in_collision, path_blocked, paced_path_wrist_unwind, paced_path_unreachable, self_collision, table_collision, inter_arm_collision, contact_path_blocked, paced_motion_budget or invalid_motion_arguments. |
Cost, and every model call by call
A model call is not cheaper in ORA than in Uni-Robot-Agent[11]: a median of 190k input tokens against 133k on RoboDojo. An episode is, because code mode makes about eight times fewer calls (a median of 58 against 462).
- Model calls per episode, median: Uni-Robot-Agent 462, ORA 58.
- Input tokens per call, median: 133k and 190k.
- Input tokens per solved case: 211.9M and 31.5M.
- Context resets per episode: 5.0 and 0.0.
| DeepSeekV4.1-Flash | Qwen3.8-Flash-Next | Inkling2-Small | MuseSpark Open | Qwen3.8-27B | Nemotron3.5-Super-VL | |
|---|---|---|---|---|---|---|
| Episodes | ||||||
| Casesone episode each | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 |
| Solvedshare of cases | 45% | 36% | 22% | 17% | 15% | 5% |
| Thinking, per model call | ||||||
| Reasoning tokensmedian; quartiles and 90th percentile | 2.1k800 to 4.6k, p90 8.4k | 1.1k139 to 3.2k, p90 6.3k | 1.0k272 to 2.5k, p90 4.9k | 341163 to 642, p90 1.1k | 3.4k542 to 8.4k, p90 16.4k | 18040 to 767, p90 2.8k |
| Calls with no reasoningshare of model calls | 2.8% | 0% | 0% | 0% | 0% | 0% |
| What a decision carries | ||||||
| Model calls per decisiontotal calls over total decisions | 2.8 | 3.9 | 3.2 | 2.1 | 3.1 | 3.1 |
| Robot actions per acting turnmean | 3.9 | 4.2 | 2.1 | 3.9 | 3.0 | 1.6 |
| Acting turns with several actionsshare | 73% | 68% | 41% | 66% | 61% | 28% |
| The programs | ||||||
| Lines per programmedian; quartiles | 3523 to 54 | 4020 to 72 | 96 to 15 | 2414 to 39 | 4123 to 66 | 1712 to 26 |
| Programs that end in an errorshare of programs | 10% | 12% | 21% | 32% | 31% | 39% |
| Robot tool calls | ||||||
| Moves refusedshare of motion calls, before anything moves | 5% | 5% | 16% | 9% | 6% | 26% |
| Calls not accepted, by reasonshare of all robot tool calls | 3% | 5% | 10% | 37% | 6% | 19% |
| Context and length | ||||||
| Input per model callthousand tokens; median over episodes | 187k | 95k | 98k | 91k | 106k | 85k |
| Model calls per episodemedian | 58 | 146 | 136 | 88 | 104 | 139 |
| Decisions per episodemedian | 18 | 31 | 40 | 37 | 33 | 40 |
| How episodes end | ||||||
| Endingshare of cases | 45% | 36% | 22% | 17% | 15% | 5% |
| DeepSeekV4.1-Flash | Qwen3.8-Flash-Next | Inkling2-Small | MuseSpark Open | Qwen3.8-27B | Nemotron3.5-Super-VL | |
|---|---|---|---|---|---|---|
| Episodes | ||||||
| Casesone episode each | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 | 78layouts 3 and 4 |
| Solvedshare of cases | 40% | 37% | 15% | 12% | 13% | 4% |
| Thinking, per model call | ||||||
| Reasoning tokensmedian; quartiles and 90th percentile | 1.3k347 to 3.7k, p90 7.8k | 1.3k329 to 3.2k, p90 6.1k | 877230 to 2.4k, p90 4.9k | 12649 to 287, p90 587 | 2.0k480 to 5.5k, p90 12.8k | 15351 to 467, p90 1.6k |
| Calls with no reasoningshare of model calls | 1.1% | 0% | 0% | 0% | 0% | 0% |
| What a decision carries | ||||||
| Model calls per decisiontotal calls over total decisions | 1.9 | 2.3 | 2.5 | 2.0 | 2.1 | 2.6 |
| Robot actions per acting turnmean | 1.2 | 1.1 | 1.2 | 1.3 | 1.1 | 1.2 |
| Acting turns with several actionsshare | 6% | 4% | 5% | 8% | 4% | 6% |
| Robot tool calls | ||||||
| Moves refusedshare of motion calls, before anything moves | 12% | 12% | 28% | 26% | 16% | 41% |
| Calls not accepted, by reasonshare of all robot tool calls | 8% | 12% | 20% | 20% | 13% | 38% |
| Context and length | ||||||
| Input per model callthousand tokens; median over episodes | 232k | 109k | 111k | 100k | 113k | 102k |
| Model calls per episodemedian | 66 | 140 | 158 | 111 | 130 | 160 |
| Decisions per episodemedian | 34 | 64 | 65 | 59 | 73 | 60 |
| How episodes end | ||||||
| Endingshare of cases | 40% | 37% | 15% | 12% | 13% | 4% |
Not accepted:move refusedbad argumentsper-program limitother
Ending:solvedgave upout of budgetended by a failed check