ora toolkit provider-replay serves a sealed run's recorded model replies as a
read-only tape and compares every request a candidate run sent with the source
run's. It never launches ORA, a simulator or a scheduler.
The old-tool treatment, which derived a candidate RunSpec for the tape
(provider-replay spec, the allocated router-task and
scripts/dev/router-provider-replay), was retired on 2026-11-01 with its replay
baseline native_full_explicit_motion_direct_replay_v1. The toolkit treatment
(robodojo_toolkit_legacy_replay_v1 on robodojo_placement_toolkit_v3) was
retired on 2026-10-31. Sealed replays keep their recorded baselines and
definitions.
Serve a tape
./scripts/ora toolkit provider-replay serve \
"$SOURCE_RUN" --output "$TAPE_OUTPUT" --port 18080
It returns the original OpenAI response chunks, including tool-call IDs,
reasoning and usage, in their recorded order. It records incoming requests
outside the source run. A consumed reply is never reused after a disconnect.
Exhausted tapes and stream-mode mismatches are errors. Loopback is the default;
use --listen only when the operator deliberately needs another host to
connect. --port 0 lets the OS choose a free port; --ready-file publishes the
bound endpoint atomically.
A fixed response tape does not rewrite old step or target references, so an unchanged tape can fail to find a previously recorded reference; record that divergence instead of mapping numbers.
Compare two runs
./scripts/ora toolkit provider-replay compare \
"$SOURCE_RUN" "$CANDIDATE_RUN" --output "$COMPARISON_OUTPUT"
This reads sealed or compressed request artifacts without changing either run.
Each request gets a JSON file with both model-call IDs, observation steps, raw
byte hashes and equality, every differing JSON field, image
count/order/label/reference checks and each paired image's PSNR and fraction of
pixels differing by more than 16. 48–52 dB is labelled as the M7.4b
rendering-noise reference; lower values are separate. That label is a measured
comparison range, not a causal proof or an exception to equality. Exact pixels
use pixels_equal=true, psnr_db=null. Unmatched requests and images are
explicit. Output directories must be new and outside both runs. Exit 1 means
any request bytes or count differ.
comparison.json.policy_configuration.source/candidate records each saved
RunPlan's full baseline reference (including its hash), require_live_marker,
the saved surface and implementation_commit. These are
comparison metadata, not a normalization or part of request equality. Missing
plans are reported as null. The report also carries each run's toolkit
protocol compatibility; a version difference is a warning, not a request
difference.