Cognitive Robotics

DyNAMO
Cognitive roboticsDynamic manipulationOperational intelligence

DyNAMO

Our research advances Cognitive Robotics by bridging perception, reasoning, and action. At the core is Dynamo, a comprehensive framework for dynamic manipulation and operational intelligence.

Dynamo is a cognitive robotic system built to adapt to unstructured environments and learn through interaction. By integrating foundation models with rigorous control theory, it achieves versatile, robust behavior in real-world settings.

View Technical Overview →
Team
David Szilagyi
Pratham Rathod
Shidan Chen
Vial Sort
VLA policyLeRobotJetson inference

Vial Sort

Language-conditioned vial sorting on a Waveshare SO-101 arm using a pi0 vision-language-action model fine-tuned with LoRA. Two experiments compare static versus domain-randomized training data.

A VLA-based pick-and-place pipeline built with LeRobot: the robot receives a natural language instruction, observes the workspace from three RGB camera views, and executes the requested vial arrangement on Jetson inference. The project studies whether randomized training data improves transfer when lighting, rack placement, and vial colors change.

View Technical Overview →
Team
David Szilagyi
Sari Abdan
Leader Following
RidgebackPerson trackingSensor fusion3D perception

Leader Following

Leader following on a Ridgeback mobile base: camera-based detection singles out a chosen person by appearance, while fused point-cloud perception tracks the leader and visualizes the intended follow path in RViz.

The Ridgeback picks one designated person out of a multi-person scene, then fuses camera detection with point-cloud perception to follow them through clutter. Two iterations compare 2D and full 3D sensing for environment understanding.

View Technical Overview →
Team
David Szilagyi
Pratham Rathod
Coming Soon

Project Unknown

A new initiative building upon Video Joint Embedding Predictive Architectures.

Autonomous Driving

Our autonomous driving research combines cutting-edge algorithms with practical implementation, resulting in robust solutions for real-world scenarios.

Simulation

Traffic Network Builder

Interactive tools for constructing and simulating complex traffic networks to rigorously test autonomous systems.

Evaluation

Benchmarking Framework

A comprehensive benchmarking suite for evaluating autonomous driving algorithms across various metrics and scenarios.

Motion Planning

Real-Time Corridor Planning

Advanced corridor planning using cubic spirals for smooth, kinematically feasible trajectory generation in real-time.

Field Test

Real-World 1:10 Scale Autonomous Driving

Testing real-world 1:10 scale autonomous driving capabilities, showcasing reliable navigation and control.