Does the real Opentrons simulator catch the protocol faults a small stdlib liquid-handling twin catches?
research/osrobot-labreal
Does the real Opentrons simulator catch the faults the stdlib liquid-handling twin catches? On the same 522 seeded faults it raises on 270 and the twin flags 366; everything the simulator catches the twin catches too, while tip reuse, a 5 µL dispense and a half-empty source pass the simulator. Mostly policy differences; the fault set was written for the twin (stylised).
What it found
The 522 seeded faults of osrobot-lab (three protocols, 11 fault kinds) were translated into Opentrons Python API scripts and run through the real simulator (opentrons==8.0.0, OT-2, p300). It raises on 270 of them against the twin's 366, and every one it catches the twin also catches. It catches scripting errors (unknown labware or well, missing or doubled tip pick-up, 350 µL or 5000 µL aspirate) and accepts tip reuse, a 5 µL dispense, a tip left on and a half-empty source, even with starting liquids declared. Those are mostly policy differences, not simulator misses. Stylised: the fault set was written for the twin.
A figure from its own output
Paper, code and results
More in Open-source purpose-built robots
- Does searching an arm’s joint count, link lengths and mount, with its controller tuned in the loop, beat a fixed three-link body at the same cost?Does searching a planar arm's joint count, links and mount, with its controller tuned, beat a fixed three-link body at equal cost? On a shelf task with a tool-down constraint, 0.95 held-out success against 0.26; on a long position-only reach, no gain (stylised, kinematic).
- How heavy must a purpose-built UAV for targeted observation be, and when does a hover-capable airframe stop being worth its weight?How heavy must a purpose-built UAV for targeted observation be, and when is a hover-capable airframe worth its weight? At 3 km out and 20 min on station a fixed wing closes at 1.27 kg against 2.14 kg for a quadrotor and 2.08 kg for a VTOL wing, and the quadrotor stops closing at 45 min (stylised, assumed constants, no flight stack run).
- Does a steady wind change which purpose-built observation UAV is lightest, and what is wind worth to a multirotor?Does a steady wind change which purpose-built observation UAV is lightest? At 3 km and 20 min on station the fixed wing stays lightest from 0 to 18 m/s (1.27 to 1.50 kg) while the quadrotor goes 2.14 to 7.0 kg; the multirotor's still-air win at 1 min on station ends near 10 m/s. Stylised, steady known wind only.
- How much heavier must an observation UAV be if the wind is uncertain, and is designing to a wind quantile cheaper than a bigger battery reserve?How much heavier must an observation UAV be if the wind is uncertain? Designing to the 95th-percentile of wind ~ N(8, 3) m/s costs the fixed wing +3.7% and the multirotor +11.7% mass, and a bigger battery reserve buys about the same reliability for about the same mass; stylised, one steady wind per mission.
- If an observation UAV only launches when the forecast wind is below a limit, how much does forecast error bring failures back, and what does it cost in scrubbed missions?If an observation UAV launches only when the forecast wind is under a limit, how much does forecast error bring failures back? A perfect forecast turns every failure into a scrub (fixed wing, wind ~ N(5, 4) m/s: 4.6% scrubbed at the 95th-percentile limit); a 2 m/s error gives 6.8% failures at the median limit but 1.1% at the 95th. Stylised, one steady wind per mission.
- If a scrubbed observation-UAV launch can be retried later the same day, how much does persistent wind limit what retrying buys, and does retrying bring failures back?If a scrubbed observation-UAV launch can be retried later the same day, how much does persistent wind limit the gain, and do retries bring failures back? Fixed wing, median-wind limit, perfect forecast: eight slots complete 99.7% of days with independent wind but 71.5% with persistence 0.95; with a 2 m/s forecast error eight slots raise failures from 6.8% to 13.6% of days (23.7% at persistence 0.95). Stylised.
All 44 Open-source purpose-built robots projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.