These functions compose or inspect configuration. Resolving a plan does not launch an episode. Source, host, controller, provider, and qualification gates still apply at launch.
from ora.orchestrator.spec import (
load_run_spec,
resolve_run_spec,
validate_run_plan,
)
load_run_spec
load_run_spec(path: Path) -> dict[str, Any]
resolve_run_spec
resolve_run_spec(
spec: Mapping[str, Any],
*,
root: Path = PROJECT_ROOT,
) -> dict[str, Any]
Validate one RunSpec and bind it into a content-addressed RunPlan.
Sections resolve in this fixed order, each raising the first error it
finds, so a RunSpec with several invalid fields always reports the same
one. The policy and runtime sections default and normalize shallow copies
of spec["policy"] and spec["runtime"].
validate_run_plan
validate_run_plan(plan: Mapping[str, Any]) -> dict[str, Any]
Verify one persisted RunPlan's complete content-addressed identity.
Benchmark builders
from ora.orchestrator.spec.benchmark import (
build_robotwin_spec,
build_libero_spec,
build_libero_pro_spec,
)
build_robotwin_spec
build_robotwin_spec(
task: str,
*,
seed: int,
experiment: str,
resource_profile: str,
seed_manifest: Path | None = None,
expert_unsolved: Sequence[Path] | None = None,
expert_unsolved_reason: str | None = None,
task_config: str = 'demo_randomized',
episode_names: Mapping[str, Any] | None = None,
host: str | None = None,
config: str | None = None,
provider_profile: str | None = None,
provider_catalog: Path | None = None,
budgets: Mapping[str, int] | None = None,
service_report: Path | None = None,
cohort_id: str | None = None,
milestone: str | None = None,
evidence_tier: str = 'behavioral',
rounds: int | None = None,
baseline: str | None = None,
root: Path = PROJECT_ROOT,
) -> dict[str, Any]
One RoboTwin cell (task, seed, scene configuration) on the default baseline.
seed_manifest must admit the seed (admitted_seed_manifest); the
spec names its content-addressed import path and compose imports it.
Instead, expert_unsolved names official scan manifests that show the
expert failed this seed, with expert_unsolved_reason: the cell then
runs under a labelled qualification.exception
(backends.robotwin.qualification_exception), for an external case list scored on its
own denominator. Exactly one of the two is given.
build_libero_spec
build_libero_spec(
suite: str,
task_id: int,
*,
seed: int,
experiment: str,
resource_profile: str,
host: str | None = None,
config: str | None = None,
provider_profile: str | None = None,
provider_catalog: Path | None = None,
budgets: Mapping[str, int] | None = None,
service_report: Path | None = None,
cohort_id: str | None = None,
milestone: str | None = None,
evidence_tier: str = 'behavioral',
rounds: int | None = None,
baseline: str | None = None,
root: Path = PROJECT_ROOT,
) -> dict[str, Any]
One original LIBERO cell (base suite, task, initial state seed) on the LIBERO-Pro backend.
The four source-paired base suites (spatial, object, goal, 10) run on the
same pinned LIBERO-Pro code and data as the core suites; LIBERO-90 is not
an official evaluation suite and has no pinned language here. The policy
receives the official instruction (base_task_language); when the BDDL
file words the same goal differently, the plan records both texts verbatim
so launch preflight admits exactly that divergence.
build_libero_pro_spec
build_libero_pro_spec(
suite: str,
task_id: int,
*,
seed: int,
experiment: str,
resource_profile: str,
host: str | None = None,
config: str | None = None,
provider_profile: str | None = None,
provider_catalog: Path | None = None,
budgets: Mapping[str, int] | None = None,
service_report: Path | None = None,
cohort_id: str | None = None,
milestone: str | None = None,
evidence_tier: str = 'behavioral',
rounds: int | None = None,
baseline: str | None = None,
root: Path = PROJECT_ROOT,
) -> dict[str, Any]
One LIBERO-Pro core cell (suite, task, initial state seed) on the default baseline.
The official core evaluation runs initial states 0-49 of each task
(libero_pro.official_core_evaluation); the Runtime selects state
seed % 50, so a seed outside that range would repeat one.