Early beta · launching soon
Find out why your robot policy fails before it ships.
Trialbase runs your policy across thousands of randomized simulated worlds and groups the failures by cause.
- runs
- 300
- passed
- 247
- failed
- 53
- cluster A41 of 300
Gripper slips when surface friction is low
friction < 0.4 · mass > 300 g
- cluster B8 of 300
Target hidden behind a distractor object
distractor within 4 cm
- cluster C4 of 300
Grasp times out in low light
light < 120 lux
The problem
A robot policy usually gets judged on a few hand-built scenes and someone watching rollouts. It looks fine on twenty trials, so it goes to the real arm, and then it drops the cup every time the table has just been wiped.
That failure was findable in simulation. Nobody had built a scene with a slick table, and a single success rate would have hidden it anyway.
What Trialbase does
Builds the task for you
Describe the task in a sentence, or hand us your own meshes and scene files, and Trialbase sets up the environment, the success check and the reset.
Varies everything that varies in a real cell
Lighting, object pose, surface friction, clutter, camera noise and distractor objects get sampled across thousands of variations, not the three you had time to build.
Groups the failures
Failed episodes are clustered by the conditions they share and what the robot did, so you read five causes instead of three hundred videos.
Replays every episode
Each run keeps its seed and its parameters, so you can watch any failure again and reproduce it exactly.
Blocks regressions in CI
A new checkpoint cannot merge if it fails scenarios the previous one passed.
Tells you what sim cannot
Every report says which physical effects we model and which we don't, so you know how far to trust a pass.
How it works
policy
Bring a policy
A checkpoint, a vision-language-action model, or your model plus the harness that runs it.
task
Describe the task
“Pick the mug and place it on the shelf.” Add your own assets if you have them.
sweep
Run the sweep
We generate the variations and run your policy across all of them in parallel.
clusters
Read the clusters
Failures come back grouped by cause, each with replays and the conditions they share.
gate
Gate the next checkpoint
Save the sweep as a baseline and every new checkpoint gets checked against it.
What you get back
For each sweep
- A success rate with its uncertainty
- Failure clusters ranked by size
- The conditions each cluster shares
- A replay for every episode
- The seed and parameters to reproduce any run
For each new checkpoint
The scenarios it newly fails, the scenarios it newly passes, and a pass or block result your CI can read.
| scenario | 120k | 140k |
|---|---|---|
| low light, clutter 6 | pass | pass |
| friction 0.35, mug 320 g | fail | pass |
| distractor beside target | pass | fail |
| camera noise high | pass | pass |
gate: blocked, 1 scenario newly fails
Sim to real: what we model, what we don't
modelled
- Rigid-body contact and friction
- Object pose and mass
- Lighting and shadows
- Camera noise and small calibration error
- Clutter and distractor objects
not modelled well yet
- Cloth, cable and other deformable objects
- Liquids
- Wear on grippers and surfaces
- The exact timing of your real control stack
A pass in Trialbase means the policy survived the conditions we can simulate. Treat it as a filter before real-robot time. We say so in every report.
Who it is for
- Robot learning teams training manipulation policies who need to know a checkpoint is worth real-robot time before they spend it.
- Teams fine-tuning large vision-language-action models who change a lot at once and need to see what broke.
- The people who own releases, when a regression showed up on hardware and nobody could reproduce it.
Where the beta is today
In the beta today
- Tabletop manipulation tasks: pick, place, stack and open
- Randomization sweeps across all six factors
- Failure clustering
- A replay for every episode
Next
- Regression gates in CI
- Importing your own robot and object assets
Later
- Mobile robot tasks
- Humanoid tasks
- Running Trialbase on your own hardware
What you get
Free runs while we are in beta, a direct line to the founder, and a real say in what gets built next.
What we ask
A short call to start, honest feedback after your first sweeps, and a note when something we report looks wrong.
Access is by conversation. Write to the founder and tell us what your robot does.
Questions
How much does it cost?
Pricing is not set yet. Early beta teams run free in exchange for feedback, and we will tell you well before that changes.
What happens to my policy and assets?
We use them only to run your evaluations. They are not shared or used to train anything, and we delete them when you ask.
What kinds of policies can I bring?
In the beta, a checkpoint we can load, or a harness that takes observations and returns actions. If yours works differently, write to us and we will tell you honestly whether it fits.
Can I use my own robot and objects?
Asset import is next on the list. Today we build scenes from a task description and a set of common tabletop objects. If you need your own arm now, send us the model and we will see what we can do.
Does a pass in simulation mean it works on the real robot?
No. It means the policy handled every condition we could simulate. We publish what we do not model so you can judge the gap yourself.
When will Trialbase be generally available?
We have no date yet. We will open it wider once CI gates and asset import are solid with the beta teams.
Why we are building this
Robot policies are getting more general faster than the ways we test them are improving. A model that can fold, pick and open things is still judged by a person watching a few dozen rollouts, and real-robot time is the slowest and most expensive part of the loop.
We are building Trialbase so that time goes to checkpoints that have already survived thousands of hard variations, and so that when a policy fails, the first thing you see is why.
founders
- Founder nameFounder
We are in early beta. If you want in, write to the founder.
Tell us what your robot does, what you are training, and where it breaks today.