Cognitive Robotics

DyNAMO
Our research advances Cognitive Robotics by bridging perception, reasoning, and action. At the core is Dynamo, a comprehensive framework for dynamic manipulation and operational intelligence.
Dynamo is a cognitive robotic system built to adapt to unstructured environments and learn through interaction. By integrating foundation models with rigorous control theory, it achieves versatile, robust behavior in real-world settings.

Vial Sort
Language-conditioned vial sorting on a low-cost SO-101 arm, driven by a pi0.5 vision-language-action policy fine-tuned from 150 teleoperated demonstrations with its vision and language backbones frozen.
A hierarchical pick-and-place pipeline built with LeRobot: a planner grounds which vial goes where and emits a source-to-destination command, then the flow policy executes the spatial move from three RGB views, inferring on the Jetson beside the arm. Evaluation across autonomous rollouts localizes the limit at grasping rather than at language or targeting.

Leader Following
Leader following on a Ridgeback mobile base: camera-based detection singles out a chosen person by appearance, while fused point-cloud perception tracks the leader and visualizes the intended follow path in RViz.
The Ridgeback picks one designated person out of a multi-person scene, then fuses camera detection with point-cloud perception to follow them through clutter. Two iterations compare 2D and full 3D sensing for environment understanding.
Project Unknown
A new initiative building upon Video Joint Embedding Predictive Architectures.
Autonomous Driving
Our autonomous driving research combines cutting-edge algorithms with practical implementation, resulting in robust solutions for real-world scenarios.
Traffic Network Builder
Interactive tools for constructing and simulating complex traffic networks to rigorously test autonomous systems.
Benchmarking Framework
A comprehensive benchmarking suite for evaluating autonomous driving algorithms across various metrics and scenarios.
Real-Time Corridor Planning
Advanced corridor planning using cubic spirals for smooth, kinematically feasible trajectory generation in real-time.
Real-World 1:10 Scale Autonomous Driving
Testing real-world 1:10 scale autonomous driving capabilities, showcasing reliable navigation and control.