Lab automation
One camera. Every tube in its rack.
A 5-axis arm with a single camera on its wrist maps the tubes lying on a bench, picks each one square to the table, stands it upright and drops it into the rack for its cap colour. No demonstrations, no hand-placed coordinates.
It picks four in five.
An unedited run of the pipeline: map the bench, pick each tube square to the table, stand it up and drop it into its rack. The arm gets 80% success on the pick-up and 50% on the drop into the rack hole.
The pick has a tolerance of about 5 mm and the camera clears it easily. The drop is the hard half: a 13 mm tube in a roughly 15 mm hole leaves about 1 mm of clearance, below what an open-loop move can hit. Closing that gap takes a contact search at the rim, not better pixels.
How it works
The arm never plans a whole path from one picture and hopes. It runs a short loop, over and over: see the error, measure what a joint does to it, solve the move on its own kinematic model, move a little, and look again. That loop is visual servoing, and the solver inside it is ordinary inverse kinematics, run on the robot's own description.
1 · See. The wrist camera finds the cap and compares it with a fixed cell between the jaws. The difference, a pixel vector, is the only thing the camera has to judge, and it judges it well: pixels near the jaws are exact even when the camera's idea of distance is not.
2 · Measure. Before correcting, the arm turns its base a few degrees and watches how far the cap moves. That ratio, pixels per degree, turns the error into a joint step. It is measured at this pose, on this arm, every time, so nothing depends on a sign convention: on our arm the base turns the opposite way to intuition, and a hard-coded sign drove it steadily away from the tube while it believed it was converging.
3 · Solve. Every joint angle comes from inverse kinematics on the robot's own URDF, with the wrist pitch held. Started from one pose, the solver gave up at 70° and gripped tubes at a slant; started from several, it finds a hand square to the table out to 30 cm. The last centimetres are a straight vertical drop at that fixed angle, so the fingers land on the tube instead of sweeping it away.
Then the loop runs again from the new pose, until the cap sits in the cell. The same three steps carry the held tube over a free hole in the rack; only the target changes. The rest of the run, mapping the bench once, twisting the wrist to the tube's direction, judging the grip by where the jaws stop, and checking the rack from above, is built around this loop.
Why an end-to-end VLA didn't work
Before this pipeline we trained SmolVLA end to end on teleoperated demonstrations of the same task, recorded on an X250 arm: two cameras and six joints in, joint actions out. It didn't sort tubes reliably, and the reasons were in the data and the task, not the model size.
Too few examples of the rule. Blue and green go to the black rack, red to the silver one, and the policy had to infer that from pixels with about 43 episodes per colour. Green really had 22: 21 of its 43 were exact duplicates. A rule that fits in a three-row lookup table was being learned from a few dozen examples.
The instruction said nothing. Every frame of every episode carried the same sentence, "Pick up the blood tube and place it in the rack", one instruction for about 69,000 frames. It never said which phase, which tube or which rack, so the language half of the model had nothing to do.
Noisy, fixed-length demonstrations. Episodes were cut at a fixed 20 seconds, not when the task ended, so on average 18% of each is the arm idling after the release. One in eleven taught the wrong thing: recording stopped while still holding the tube, the grasp was missing, or nothing was picked. And the training stack loaded one recording session at a time, so early runs saw 18 to 20 episodes, not 117.
So the pipeline splits the work. Geometry, the arm's own kinematics and sensing do what they can do exactly, and learning goes where it pays off: the teleop episodes that were too few for a whole policy are plenty to train the detector that sees the tubes. The VLA stays as a baseline to beat.
It collects its own training data
Every attempt is recorded as an episode: the camera frames before and after, the joint states along the way, and an outcome the robot assigns itself from the grip and a look from above: placed, missed the hole, missed the grasp, dropped in carry, not found. No one watches the run to label it. Development alone left 672 labelled episodes on disk, sorted by what went wrong.
That changes where data comes from. Teleoperation gave us 117 demonstrations, uneven in speed and style, and one in eleven taught the wrong thing. The pipeline can run unattended and produce hundreds of consistent, labelled episodes a day, with its failures already sorted into the cases a model most needs to see.
Failures are kept, not thrown away. A miss is the most useful episode there is: it shows exactly where the robot's picture of the world is wrong. The top camera measures how far a dropped tube landed from its hole, and that error becomes a correction for the next drop; a missed grasp shows which approach went wrong. Successes teach what to do, failures teach what not to do and how to recover.
Those episodes train the parts that learning does best: the tube detector from the frames, and a short insertion policy from the drops, the misses included. Each model that replaces a hand-built step makes the next round of runs, and so the next round of data, better. The precise controller is the data engine for the learned one, a system that improves itself instead of waiting on a human with a teleoperation rig.
What's next: training it to near perfect
Close the last millimetre. The drop misses on 1 mm of clearance, below what a camera-driven move can hit. The next fixes are local: feel the rim through the gripper current and search outward until the tube falls in, servo on the hole with the wrist camera instead of trusting the overhead calibration, and let cheap hardware help, with chamfered holes and fingers that centre the tube.
Then let it teach itself the task. Perfecting one task happens fastest on the real arm, because what is left is the last millimetre, and that is where a simulator is least right. The robot corrects its own systematic errors from each measured miss, then tunes its own settings overnight, keeping whatever raises the success rate, successes and failures both counting. On top of the controller, a small policy learns only the final correction from those same episodes.
Then train, in three steps. Collect data with the current method: the robot runs unattended and labels every attempt itself, hundreds of consistent episodes a day instead of 117 teleoperated ones. Train in simulation on that data, which is where breadth comes from: new objects, lighting and layouts. Then train GR00T on it, with a learned detector replacing the colour thresholds along the way.
A fair rematch for the VLA. The same data fixes what held SmolVLA back: an instruction that names the tube and the rack, episodes that end when the task does, no duplicates or failed runs, and the free holes given as input.
Other projects
RAX grew out of lab automation work: a benchtop station for clinical specimen receiving at Kinesis Labs, and robot-run lab assays at BioRobotix.