Control Stack
A seven-dimensional end-effector delta action space drives translation, rotation, and gripper commands.
CORE Labs
Cognitive robotics - SO-101 robotic arm
Vial Sort
A language-conditioned vial sorting project for the Waveshare SO-101 arm. The robot observes a tabletop scene with two 6x1 vial racks and three camera views, receives an instruction such as placing a red vial into a target rack position, and executes the corresponding pick-and-place behavior through a pi0 vision-language-action pipeline.
Team
Research question
Which training setup produces a higher success rate for VLA-based vial sorting on the SO-101 robot arm: a static tabletop dataset or a domain-randomized dataset?
SO-101pi0 + LoRALeRobotJetson AGX Thor
Experiment log
Milestone 04Autonomous inference
ACT, pi0, and pi0.5 policy runs
The autonomous comparison shows the sequence of policies built so far: ACT as the pick-and-place baseline, pi0 as a stronger policy that did not reliably follow the language prompt, and pi0.5 as the current model with improved prompt-following.
Outcome·Early tests on the previous dataset guide the retraining plan for the new dataset.
04/04
Milestones
4 videosAbout
RobotSO-101 6-DOF arm
ComputeNVIDIA Jetson Thor
Workspace2 x 6-slot racks + bin
Cameras3 x RGB, 640x480 @ 30fps
FrameworkLeRobot
PoliciesACT, pi0, pi0.5
VLA Training
The vial dataset is converted into a VLA-compatible format and pi0 is fine-tuned with LoRA.
Robot Deployment
Three RGB camera views and robot-side Jetson inference connect language instructions to pick-and-place actions.
Expected outcome
A complete VLA pipeline for language-conditioned vial sorting.
The project should produce documented components for end-effector control, inverse kinematics, dataset conversion, LoRA fine-tuning, deployment, and evaluation. The evaluation compares success rate, robustness, latency, and failure modes across controlled lab scenarios.
Evaluation metrics
Task success rateScenario difficultyGrip failuresDropped vialsWrong placementLatency