Haresh Karnan

I am a PhD candidate at The University of Texas at Austin, working on Reinforcement Learning, Computer Vision and Artificial Intelligence for Robotics. I'm a member of the Learning Agents Research Group (LARG), advised by Peter Stone, and co-advised by Joydeep Biswas in the Computer Science department. I'm also a part of the UT Austin RoboCup@Home team.

I've previously interned at Amazon Scout where I worked on Vision based Robot Localization for their package delivery robot - Scout.

In my past life, I worked with Dr. Robert Skelton at Texas A&M University, College Station, on localization, sensing and control of Tensegrity robots.

Email  /  CV  /  Biography  /  Google Scholar  /  LinkedIn

profile photo
Publications
3DSP Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation Haresh Karnan, Anirudh Nair, Xuesu Xiao, Garrett Warnell, Soeren Pirk, Alexander Toshev, Justin Hart, Joydeep Biswas, Peter Stone
IEEE Robotics and Automation Letters (IEEE RA-L) , 2022
Paper / Short Video / Long Video / Poster / Dataset

We introduce a large-scale, first-person-view dataset of socially compliant robot navigation demonstrations. SCAND consists of 138 trajectories, 25 miles of socially compliant navigation demonstrations collected on 2 robots by 4 human demonstrators within the UT Austin campus.

3DSP High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics
Haresh Karnan, Kavan Singh Sikand, Pranav Atreya, Sadegh Rabiee, Xuesu Xiao, Garrett Warnell, Joydeep Biswas, Peter Stone
International Conference on Intelligent Robots and Systems (IROS) , 2022
Paper / Video

We introduce VI-IKD, a visual-inertial IKD model that learns to anticipate the kinodynamic interactions of the vehicle with the terrain ahead, enabling accurate high-speed navigation.

3DSP Optim-FKD: High-Speed Accurate Robot Control using Learned Forward Kinodynamics and Non-linear Least Squares Optimization Pranav Atreya, Haresh Karnan, Kavan Singh Sikand, Sadegh Rabiee, Xuesu Xiao, Garrett Warnell, Joydeep Biswas, Peter Stone
International Conference on Intelligent Robots and Systems (IROS) , 2022
Paper / Video

In this work, we introduce Optim-FKD, an approach for high-speed navigation that uses a learned forward kinodynamics model (FKD) coupled with non-linear least squares optimization for multi-horizon control.

3DSP VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation
Haresh Karnan, Garrett Warnell, Xuesu Xiao, Peter Stone
International Conference on Robotics and Automation (ICRA), 2022
Paper / Video / Poster

Learning to imitate from an expert's video-only demonstration can be hard when there is significant viewpoint mismatch between the expert and the imitator agents. In this work, we propose VOILA, an IfO algorithm that imitates an expert driver's navigation policy from a single video-only demonstration, overcoming viewpoint mismatch.

3DSP Adversarial Imitation Learning from Video using a State Observer
Haresh Karnan, Garrett Warnell, Faraz Torabi, Peter Stone
International Conference on Robotics and Automation (ICRA), 2022
Paper / Video / Poster

SOTA approaches in Imitation from Video only demonstrations exhibit poor sample efficiency when learning with access to proprioceptive features of the imitator. To tackle this, we introduce VGAIfO-SO, an IfO algorithm that uses self-supervision to improve sample efficiency.

3DSP Grounded Action Transformation for Sim-to-Real Reinforcement Learning
Josiah Hanna, Siddharth Desai, Haresh Karnan, Garrett Warnell, Peter Stone
Springer. Machine Learning, 2021
Paper

Sim-to-real is the problem of learning a control policy in an inaccurate simulated world such that the learned policy when transferred to the real-world, performs well. In this article, we explore the proposed black-box Sim-to-Real algorithm GAT, and its extension SGAT.

3DSP An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch
Siddharth Desai, Ishan Durugkar, Haresh Karnan, Josiah Hanna, Garrett Warnell, Peter Stone
Neural Information Processing Systems (NeurIPS), 2020
NeurIPS site / Paper / Poster / arXiv

In this work, we propose the GARAT algorithm, which treats the Sim-to-Real problem as an Imitation from Observation (IfO) problem and uses advances in the IfO literature to transfer a control policy from a source domain to a target domain, using Adversarial Imitation Learning.

3DSP Stochastic Grounded Action Transformation for Robot Learning in Simulation
Haresh Karnan, Siddharth Desai, Josiah Hanna, Garrett Warnell, Peter Stone
International Conference on Intelligent Robots and Systems (IROS), 2020
Long Video / Short Video / Paper / Poster / arXiv

Real world dynamics are often stochastic and robot simulators have an inaccurate approximation of real world dynamics. In this work, we propose a Sim-to-Real algorithm called SGAT and transfer a Humanoid walk from Simulation to Real world.

3DSP Reinforced Grounded Action Transformation for Sim-to-Real Transfer
Haresh Karnan, Siddharth Desai, Josiah Hanna, Garrett Warnell, Peter Stone
International Conference on Intelligent Robots and Systems (IROS), 2020
Long Video / Short Video / Paper / Poster / arXiv

In this work, we propose a Sim-to-Real algorithm called RGAT to ground an inaccurate simulator with data from the real world, using Reinforcement Learning.

3DSP Visual Feedback Control of Tensegrity Robotic Systems
Haresh Karnan, Raman Goyal, Manoranjan Majji, Robert Skelton, Puneet Singla
International Conference on Intelligent Robots and Systems, 2017
Video / Paper

Tensegrity mechanisms are known for their minimal-mass and flexible properties. In this work, we propose using vision based sensing for shape control of such soft robotic manipulators.

Workshops, Symposiums, Extended Abstracts
3DSP VOILA: Visual Observation-only Imitation Learning for Autonomous navigation
Haresh Karnan, Garrett Warnell, Xuesu Xiao, Peter Stone
AAAI ML4NAV Spring Symposium, 2021
Talk / Paper
3DSP Solving Service Robot Tasks: UT Austin Villa@Home 2019 Team Report
Rishi Shah, Yuqian Jiang, Haresh Karnan, Gilberto Briscoe-Martinez, Dominick Mulder, Ryan Gupta, Rachel Schlossman, Marika Murphy, Justin W. Hart, Luis Sentis, Peter Stone
AAAI AI-HRI Symposium, 2019
Video / Paper / arXiv
3DSP Extended Abstract: An Imitation from Observation Approach to Sim-to-Real Transfer
Siddharth Desai, Ishan Durugkar, Haresh Karnan, Josiah Hanna, Garrett Warnell, Peter Stone
Robotics Science and Systems, Sim-to-Real Workshop (RSS), 2020
Video / Paper / Poster

In this work, we propose the GARAT algorithm, which treats the Sim-to-Real problem as an Imitation from Observation (IfO) problem and uses advances in the IfO literature to transfer a control policy from a source domain to a target domain, using Adversarial Imitation Learning.

3DSP Visual Servoing of Unmanned Surface Vehicle from Small Tethered Unmanned Aerial Vehicle
Haresh Karnan, Aritra Biswas, Pranav Vaidik Dhulipala, Jan Dufek, Robin Murphy
arXiv preprint, 2017
Paper / arXiv

Template Credits