Robots and tasks

RoboTwin tasks

50 RoboTwin tasks. For each: the public instruction and the public completion contract ORA gives the model, which restates in plain words what RoboTwin's own check_success requires. Success is decided only by that official check at the end of the round; the contract never replaces it. Generated from ora/environment/benchmarks/robotwin-instructions-v1.json and ora/policy/environments/robotwin/task-contracts/registry-v4.json.

Task Instruction
adjust_bottle Pick up the bottle on the table and hold it head up.
beat_block_hammer Grab the hammer on the table and beat the block with it.
blocks_ranking_rgb Move the three blocks to the center of the table and arrange them red, green and blue from left to right.
blocks_ranking_size Move the three blocks to the center of the table and arrange them from largest to smallest, from left to right.
click_alarmclock Click the button on the top of the alarm clock.
click_bell Click the top center of the bell.
dump_bin_bigbin Grab the small bin and pour the balls into the big bin.
grab_roller Use both arms to grab the roller on the table.
handover_block Grasp the red block with the left arm, hand it over to the right arm and place it on the blue pad.
handover_mic Grasp the microphone with one arm and hand it over to the other arm.
hanging_mug Pick up the mug with the left arm, rotate it and set it down in the middle of the table, then hang it on the rack with the right arm.
lift_pot Use both arms to lift the pot.
move_can_pot Pick up the can and move it beside the pot.
move_pillbottle_pad Use one arm to pick up the pill bottle and place it onto the pad.
move_playingcard_away Pick up the playing card and move it away from the near edge of the table.
move_stapler_pad Move the stapler to a colored mat.
open_laptop Use one arm to open the laptop.
open_microwave Use one arm to open the microwave.
pick_diverse_bottles Pick up one bottle with one arm and the other bottle with the other arm.
pick_dual_bottles Pick up one bottle with one arm and the other bottle with the other arm.
place_a2b_left —
place_a2b_right —
place_bread_basket Put every bread on the table into the basket.
place_bread_skillet Put the bread on the table into the skillet.
place_burger_fries Use both arms to pick up the hamburger and the fries and put them onto the tray.
place_can_basket Put the can into the basket with one arm, then lift the basket with the other arm.
place_cans_plasticbox Use both arms to pick up the cans and place them into the plastic box.
place_container_plate Place the container onto the plate.
place_dual_shoes Use both arms to put the two shoes into the shoe box with the shoe tips pointing left.
place_empty_cup Place the empty cup on the coaster.
place_fan Grab the fan and place it on a colored mat, facing the robot.
place_mouse_pad Grab the mouse and place it on a colored mat.
place_object_basket Put the target object into the basket, then grab the basket with the other arm and move it slightly away.
place_object_scale Use one arm to grab the object and put it on the scale.
place_object_stand Place the object on the stand.
place_phone_stand Pick up the phone and put it on the phone stand.
place_shoe Use one arm to grab the shoe from the table and place it on the mat.
press_stapler Use one arm to press the stapler.
put_bottles_dustbin Grab the bottles and put them into the dustbin to the left of the table.
put_object_cabinet Open the cabinet drawer with one arm and use the other arm to put the object on the table into the drawer.
rotate_qrcode Pick up the QR code board and rotate it so the code faces you.
scan_object Pick up the scanner with one arm and the object with the other, then scan the object.
shake_bottle Shake the bottle with the appropriate arm.
shake_bottle_horizontally Shake the bottle horizontally with the appropriate arm.
stack_blocks_three Move the three blocks to the center of the table and stack the blue block on the green block and the green block on the red block.
stack_blocks_two Move the two blocks to the center of the table and stack the green block on the red block.
stack_bowls_three Stack the three bowls on top of each other.
stack_bowls_two Stack the two bowls on top of each other.
stamp_seal Grab the stamp and stamp it onto the target colored mat.
turn_switch Use the arm to click the switch.

adjust_bottle

Instruction. Pick up the bottle on the table and hold it head up.

Completion contract.

  • Pick up the bottle and hold it upright with its cap at the top.
  • Keep the bottle raised well clear of the table, and carry it outward toward the side on which its base end lay at the start.

beat_block_hammer

Instruction. Grab the hammer on the table and beat the block with it.

Completion contract.

  • Strike the top centre of the block with the hammer head, making physical contact with the block.

blocks_ranking_rgb

Instruction. Move the three blocks to the center of the table and arrange them red, green and blue from left to right.

Completion contract.

  • Arrange the blocks in a compact row from the robot's left to right: red, green, then blue.
  • Keep the blocks at the same depth on the table, near one another.
  • Leave both grippers open when the placement is judged.

blocks_ranking_size

Instruction. Move the three blocks to the center of the table and arrange them from largest to smallest, from left to right.

Completion contract.

  • Arrange the blocks in a compact row from the robot's left to right: largest, medium, then smallest.
  • Keep the blocks at the same depth on the table, near one another.
  • Leave both grippers open when the placement is judged.

click_alarmclock

Instruction. Click the button on the top of the alarm clock.

Completion contract.

  • Press the button on top of the alarm clock with a gripper, making physical contact.
  • Keep the gripper on the side where the alarm clock started set to close when that contact occurs.

click_bell

Instruction. Click the top center of the bell.

Completion contract.

  • Press the top centre of the bell with a gripper, making physical contact.
  • Keep the gripper on the side where the bell started set to close when that contact occurs.

dump_bin_bigbin

Instruction. Grab the small bin and pour the balls into the big bin.

Completion contract.

  • Pour all the balls from the small bin into the large bin beside the table.
  • Keep the small bin raised well above the tabletop while all the balls are down in the large bin.

grab_roller

Instruction. Use both arms to grab the roller on the table.

Completion contract.

  • Use both arms to hold the roller.
  • Lift the roller clear of the table and keep both grippers set to close while it is raised.

handover_block

Instruction. Grasp the red block with the left arm, hand it over to the right arm and place it on the blue pad.

Completion contract.

  • Pass the red block from the left arm to the right arm, then place it on the blue pad.
  • Set the block down on the end that was underneath it at the start, with that end centred on the pad.
  • Open the right gripper after placing the block.

handover_mic

Instruction. Grasp the microphone with one arm and hand it over to the other arm.

Completion contract.

  • Pick up the microphone with the arm on the side where it starts, then pass it to the other arm.
  • Hold the microphone well above the table on the receiving arm's side of the workspace.
  • Keep the receiving gripper set to close on the microphone and open the giving gripper; maintain gripper contact with the microphone.

hanging_mug

Instruction. Pick up the mug with the left arm, rotate it and set it down in the middle of the table, then hang it on the rack with the right arm.

Completion contract.

  • Hang the mug by its handle on the rack's horizontal hanging arm, with the handle seated along the arm rather than at its tip.
  • Leave the mug clear of the table with its handle raised on the rack, and open the right gripper.

lift_pot

Instruction. Use both arms to lift the pot.

Completion contract.

  • Lift the pot clear of the table and keep it upright.
  • Keep the left gripper at the left handle and the right gripper at the right handle while the pot is raised.

move_can_pot

Instruction. Pick up the can and move it beside the pot.

Completion contract.

  • Set the can upright on the table beside the pot, on the same side of the pot where the can started.
  • Keep the can close beside the pot and level with it from front to back.
  • Leave both grippers open when the placement is judged.

move_pillbottle_pad

Instruction. Use one arm to pick up the pill bottle and place it onto the pad.

Completion contract.

  • Stand the pill bottle upright on its base on the pad, centred on it and resting at pad level.
  • Leave both grippers open when the placement is judged.

move_playingcard_away

Instruction. Pick up the playing card and move it away from the near edge of the table.

Completion contract.

  • Move the playing cards sideways toward an outer left or right edge of the table, well away from the middle.
  • Leave both grippers open when the placement is judged.

move_stapler_pad

Instruction. Move the stapler to a colored mat.

Completion contract.

  • Set the stapler down centred on the coloured mat, resting on its base at mat level.
  • Align the stapler's length with the long edge of the mat.
  • Leave both grippers open when the placement is judged.

open_laptop

Instruction. Use one arm to open the laptop.

Completion contract.

  • Open the laptop lid clearly beyond its starting position.
  • Use the left arm when the keyboard faces toward the robot's left, and the right arm when it faces toward the robot's right.
  • Keep that gripper close to the lid's free edge while the laptop is open.

open_microwave

Instruction. Use one arm to open the microwave.

Completion contract.

  • Swing the microwave door wide open, beyond a small crack.

pick_diverse_bottles

Instruction. Pick up one bottle with one arm and the other bottle with the other arm.

Completion contract.

  • Pick up both bottles, one with each arm, and hold them raised well clear of the table at the same time.
  • While holding them up, bring the left-starting bottle to just left of the robot's centre line and the right-starting bottle to just right of it, close together side by side on the robot's half of the table, nearer the table's middle than its near edge.

pick_dual_bottles

Instruction. Pick up one bottle with one arm and the other bottle with the other arm.

Completion contract.

  • Pick up both bottles, one with each arm, and hold them raised well clear of the table at the same time.
  • While holding them up, bring the left-starting bottle to just left of the robot's centre line and the right-starting bottle to just right of it, close together side by side on the robot's half of the table, nearer the table's middle than its near edge.

place_a2b_left

Instruction. null

Completion contract.

  • Place the object named first to the robot's left of the object named second.
  • Arrange them beside one another at a similar depth, with their centres separated but still close together.
  • Leave both grippers open when the placement is judged.

place_a2b_right

Instruction. null

Completion contract.

  • Place the object named first to the robot's right of the object named second.
  • Arrange them beside one another at a similar depth, with their centres separated but still close together.
  • Leave both grippers open when the placement is judged.

place_bread_basket

Instruction. Put every bread on the table into the basket.

Completion contract.

  • Put every piece of bread present on the table into the basket, gathered toward the middle of its interior.
  • Leave both grippers open when the placement is judged.

place_bread_skillet

Instruction. Put the bread on the table into the skillet.

Completion contract.

  • Place the bread in the middle of the skillet's cooking surface, supported by the skillet.
  • Keep both the bread and the skillet's cooking surface raised clear of the tabletop when the placement is judged.

place_burger_fries

Instruction. Use both arms to pick up the hamburger and the fries and put them onto the tray.

Completion contract.

  • Put the hamburger on the left side of the tray and the fries on the right side, as seen from the robot, with each item near the middle of its side.
  • Leave both items resting on the tray together.
  • Leave both grippers open when the placement is judged.

place_can_basket

Instruction. Put the can into the basket with one arm, then lift the basket with the other arm.

Completion contract.

  • Put the can inside the basket, then lift the basket with the other arm while the can stays inside.
  • Keep the basket's opening facing upward and the can in contact with the basket, clear of the table.

place_cans_plasticbox

Instruction. Use both arms to pick up the cans and place them into the plastic box.

Completion contract.

  • Place both cans inside the plastic box, one in the half of the box nearer the robot and the other in the half farther from the robot, each near the middle of its half rather than side by side or on the rim.
  • Leave both grippers open when the placement is judged.

place_container_plate

Instruction. Place the container onto the plate.

Completion contract.

  • Set the container down on the plate, centred on it and resting at plate level.
  • Leave both grippers open when the placement is judged.

place_dual_shoes

Instruction. Use both arms to put the two shoes into the shoe box with the shoe tips pointing left.

Completion contract.

  • Place both shoes down in the shoe box with their soles down and toes pointing to the robot's left.
  • Place the shoe that started on the left toward the nearer part of the box, and the shoe that started on the right toward the farther part.
  • Leave both grippers open when the placement is judged.

place_empty_cup

Instruction. Place the empty cup on the coaster.

Completion contract.

  • Set the empty cup down on the coaster with its base centred on the coaster.
  • Leave both grippers open when the placement is judged.

place_fan

Instruction. Grab the fan and place it on a colored mat, facing the robot.

Completion contract.

  • Set the fan upright, centred on the coloured mat at the mat's starting location.
  • Turn the fan's front grille to face straight across the table toward the edge nearest the robot, square to that edge, rather than angled toward the robot itself.
  • Leave both grippers open when the placement is judged.

place_mouse_pad

Instruction. Grab the mouse and place it on a colored mat.

Completion contract.

  • Place the mouse centrally on the coloured mat, resting normally on its base.
  • Align the mouse's long axis with the mat's long axis, running front to back.
  • Leave both grippers open when the placement is judged.

place_object_basket

Instruction. Put the target object into the basket, then grab the basket with the other arm and move it slightly away.

Completion contract.

  • Put the target object inside the basket, then lift the basket with the other arm while the target object stays inside.
  • Keep the basket's opening facing upward and the target object in contact with the basket, clear of the table.

place_object_scale

Instruction. Use one arm to grab the object and put it on the scale.

Completion contract.

  • Set the object on the scale's weighing surface, centred over that surface.
  • Release it and open the gripper on the side where the object started.

place_object_stand

Instruction. Place the object on the stand.

Completion contract.

  • Set the object on the display stand, centred over its top surface.
  • Leave both grippers open when the placement is judged.

place_phone_stand

Instruction. Pick up the phone and put it on the phone stand.

Completion contract.

  • Seat the phone upright in the stand's cradle with its bottom edge down, the short edge that was nearer the robot when the phone lay on the table at the start, leaning back against the stand's support.
  • Leave both grippers open when the placement is judged.

place_shoe

Instruction. Use one arm to grab the shoe from the table and place it on the mat.

Completion contract.

  • Place the shoe centred on the mat, sole down, with its toe pointing to the robot's left along the mat's long edge.
  • Leave both grippers open when the placement is judged.

press_stapler

Instruction. Use one arm to press the stapler.

Completion contract.

  • Press the stapler's top pressing area with a gripper, making physical contact.

put_bottles_dustbin

Instruction. Grab the bottles and put them into the dustbin to the left of the table.

Completion contract.

  • Put all three bottles into the dustbin to the robot's left, leaving them down inside it rather than on its rim or outside it.
  • All three bottles must be inside together.

put_object_cabinet

Instruction. Open the cabinet drawer with one arm and use the other arm to put the object on the table into the drawer.

Completion contract.

  • Open the cabinet drawer with one arm and place the object inside the drawer with the other arm.
  • Set the object down on the drawer floor, centred side to side, in the middle of the drawer or toward its front by the handle rather than toward the back, slightly above its original tabletop height.
  • Release the object and open the gripper on the side where the object started.

rotate_qrcode

Instruction. Pick up the QR code board and rotate it so the code faces you.

Completion contract.

  • Set the QR-code sign back on the table on its base, with the printed display standing and facing the robot directly.
  • Align the sign squarely with the table. Judge its final facing direction from the current view; the required turn depends on its starting orientation.
  • Leave both grippers open when the placement is judged.

scan_object

Instruction. Pick up the scanner with one arm and the object with the other, then scan the object.

Completion contract.

  • Hold the scanner in one gripper and the object in the other, keeping both grippers set to close.
  • Aim the scanner's scanning window squarely at the centre of the object's base, the face it stood on at the start, and bring the window close to that face, with the object in front of the window.

shake_bottle

Instruction. Shake the bottle with the appropriate arm.

Completion contract.

  • Pick up the bottle and shake it back and forth while holding it clear of the table.
  • Keep the bottle raised above the tabletop while doing so.

shake_bottle_horizontally

Instruction. Shake the bottle horizontally with the appropriate arm.

Completion contract.

  • Pick up the bottle and shake it back and forth horizontally while holding it clear of the table.
  • Keep the bottle raised above the tabletop while doing so.

stack_blocks_three

Instruction. Move the three blocks to the center of the table and stack the blue block on the green block and the green block on the red block.

Completion contract.

  • Stack the green block centrally on top of the red block.
  • Stack the blue block centrally on top of the green block, making one tower.
  • Leave both grippers open when the placement is judged.

stack_blocks_two

Instruction. Move the two blocks to the center of the table and stack the green block on the red block.

Completion contract.

  • Stack the green block centrally on top of the red block.
  • Leave both grippers open when the placement is judged.

stack_bowls_three

Instruction. Stack the three bowls on top of each other.

Completion contract.

  • Nest the three bowls into one low stack on the table, centred over one another.
  • Leave both grippers open when the placement is judged.

stack_bowls_two

Instruction. Stack the two bowls on top of each other.

Completion contract.

  • Nest the two bowls into one low stack on the table, centred over one another.
  • Leave both grippers open when the placement is judged.

stamp_seal

Instruction. Grab the stamp and stamp it onto the target colored mat.

Completion contract.

  • Set the seal down on the target coloured mat with its stamp face centred on the mat, and leave it standing there.
  • Leave both grippers open when the placement is judged.

turn_switch

Instruction. Use the arm to click the switch.

Completion contract.

  • Press the switch toggle over to its fully switched position, close to the end of its travel.