PROJECT_MODULES
Vision-Language-Action System for Humanoid Manipulation
Built an end-to-end VLA pipeline converting unseen YouTube cooking videos into robot commands. Trained a 19M-parameter Transformer on 1.6k+ clips, achieving 77-85% action accuracy, 90%+ precision on key primitives, 60-70% zero-shot transfer to Western cuisine, and 85-90% task success in AI2-THOR.
O-VLA: Trajectory Optimization Safety Layer
Morphology-agnostic trajectory optimizer transforming unsafe VLA outputs into collision-free, constraint-satisfying paths. Achieved 100% constraint satisfaction, 60-100% collision reduction across 5 robot morphologies (6-23 DOF) in 0.03-0.44s, and 34-45% energy reduction through RNEA-based coupled dynamics optimization.
CrossViT: Cross-Embodiment Visual Perception
Trained cross-attention ViT conditioned on 32-dim kinematic features for embodiment-aware representations. Achieved 99.3-100% embodiment classification vs 20% baseline. Demonstrated zero-shot generalization to unseen humanoid robots.
Humanoid Motion Planning
Whole-body locomotion and manipulation stack for Unitree G1 using ZMP preview control, A* footstep planning, MPC balance optimization, RL locomotion, and Jacobian IK. Reduced energy use by 49% vs PD control, remained stable under 70-80N perturbations, and achieved 75%+ success on coordinated walk-and-reach tasks.
VLM-Based Autonomous Navigation
Intent-based autonomous navigation for GEM e4 using GPT-4o VLM. Built full pipeline: BEV perception, collision-free planning, Stanley control. Achieved 90% end-to-end task success and 88% accuracy on KITTI real-world dataset.
Open-Vocabulary 6D Pose Estimation
Built a zero-shot 6D pose pipeline combining VLM semantic inventory, Gemini query enrichment, SAM-3 segmentation, and Objaverse mesh retrieval. Achieved 76.5-100% ADD-S AUC on YCB-Video and developed an RGB-D variant using LangSAM, InstantMesh, and ensemble mesh selection.
Pedestrian-Aware AV Safety System
Built a real-time pedestrian safety stack for GEM e4 using LiDAR clustering and RGB-D detection. Implemented trajectory prediction and Time-to-Collision estimation for proactive speed adaptation, achieving 90% success with a state-machine safety controller and Stanley control.
Dynamic Object Handover System
Built a vision-based dynamic handover system with real-time MediaPipe hand tracking, Jacobian IK, velocity limiting, and command smoothing. Integrated gesture recognition for adaptive hand following and autonomous object release.
Terrain-Aware Navigation
Developed a perception pipeline converting depth clouds to elevation maps for quadruped footstep placement.
RL Locomotion + Safety
Trained terrain-adaptive locomotion with PPO and integrated a Control Barrier Function as a real-time safety filter. Achieved zero-fall locomotion, rejected 99% of unsafe actions, and retained 90% of the original PPO policy's traversal speed.
Quadruped Locomotion (RL)
Trained ANYmal quadruped policies using Proximal Policy Optimization. Achieved robust traversal on irregular terrain with 0-fall safety constraints.
VIO + Footstep Planning
Fused Visual-Inertial Odometry with footstep planning. Enabled autonomous navigation in GPS-denied environments with <10cm drift.
EMG-Controlled Prosthetic Arm
Built a 5-DOF prosthetic arm controlled via EMG signals. Deployed real-time gesture classification on ESP32 using XGBoost, achieving 96% accuracy.