Haresh Karnan
I am a PhD candidate at The University of Texas at Austin, specializing in Reinforcement Learning, Computer Vision and Artificial Intelligence for Robotics. I'm advised by Peter Stone, and affiliated with the Learning Agents Research Group (LARG) at UT Austin Computer Science department. I'm also a member of the UT Austin RoboCup@Home team.
I was previously an Applied Science Intern at Amazon Scout where I worked on Deep Learning and Computer Vision for their package delivery robot.
In my past life, I worked with Dr. Robert Skelton at Texas A&M University, College Station, on state estimation and control of Tensegrity robots.
Email  / 
CV  / 
Biography  / 
Google Scholar  / 
LinkedIn
|
|
|
Learning to Extrapolate Human Preferences for Preference-Aligned Path Planning
Haresh Karnan, Elvin Yang, Garrett Warnell, Joydeep Biswas, Peter Stone
Accepted for Publication @ ICRA 2024!
Paper / Video / Poster / PATERN @ Lake Austin
Robots navigating off-road environments typically require additional human feedback to learn operator preferences for visually novel terrains. In this work, we introduce a framework to extrapolate operator preferences from known terrains by leveraging multi-modality.
|
|
Targeted Learning: A Hybrid Approach to Social Robot Navigation
Amir Hossain Raj, Zichao Hu, Haresh Karnan, Rohan Chandra, Amirezza Payandeh, Luisa Mao, Peter Stone, Joydeep Biswas, Xuesu Xiao
Accepted for Publication @ ICRA 2024!
Paper / Video
In this work, we challenge the prevailing shift towards solely utilizing learning-based approaches for robot navigation in social contexts, advocating instead for a hybrid approach that toggles between classical and learning-based motion planners.
|
|
Self-Supervised Terrain Representation Learning from Unconstrained Robot Experience
Haresh Karnan, Elvin Yang, Daniel Farkash, Garrett Warnell, Joydeep Biswas, Peter Stone
Conference on Robot Learning (CoRL), 2023
Project Page / Paper / Video / STERLING @ Lake Austin / Poster / Code
Visual terrain awareness is crucial for autonomous off-road navigation. We propose a novel way to learn relevant terrain representations from unconstrained, multi-modal robot experience collected with any policy. We show how such learned representations enable aligning off-road navigation behaviors with human operator preferences.
|
|
Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation
Haresh Karnan, Anirudh Nair, Xuesu Xiao, Garrett Warnell, Soeren Pirk, Alexander Toshev, Justin Hart, Joydeep Biswas, Peter Stone
IEEE Robotics and Automation Letters (IEEE RA-L) , 2022
Paper / Short Video / Long Video / Poster / Dataset
We introduce a large-scale, first-person-view dataset of socially compliant robot navigation demonstrations. SCAND consists of 138 trajectories, 25 miles of socially compliant navigation demonstrations collected on 2 robots by 4 human demonstrators within the UT Austin campus.
|
|
High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics
Haresh Karnan, Kavan Singh Sikand, Pranav Atreya, Sadegh Rabiee, Xuesu Xiao, Garrett Warnell, Joydeep Biswas, Peter Stone
International Conference on Intelligent Robots and Systems (IROS) , 2022
Paper / Video
We introduce VI-IKD, a visual-inertial IKD model that learns to anticipate the kinodynamic interactions of the vehicle with the terrain ahead, enabling accurate high-speed navigation.
|
|
Optim-FKD: High-Speed Accurate Robot Control using Learned Forward Kinodynamics and Non-linear Least Squares Optimization
Pranav Atreya, Haresh Karnan, Kavan Singh Sikand, Sadegh Rabiee, Xuesu Xiao, Garrett Warnell, Joydeep Biswas, Peter Stone
International Conference on Intelligent Robots and Systems (IROS) , 2022
Paper / Video
In this work, we introduce Optim-FKD, an approach for high-speed navigation that uses a learned forward kinodynamics model (FKD) coupled with non-linear least squares optimization for multi-horizon control.
|
|
Autonomous Ground Navigation in Highly Constrained Spaces: Lessons Learned From the Benchmark Autonomous Robot Navigation Challenge at ICRA 2022
Xuesu Xiao, Zifan Xu, Zizhao Wang, Yunlong Song, Garrett Warnell, Peter Stone, Tingnan Zhang, Shravan Ravi, Gary Wang, Haresh Karnan, Joydeep Biswas, Nicholas Mohammad, Lauren Bramblett, Rahul Peddi, Nicola Bezzo, Zhanteng Xie, Philip Dames
Robotics and Automation Magazine (RA-M) , 2022
Paper
Won the 1st place in BARN Challenge @ ICRA 2022
|
|
VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation
Haresh Karnan, Garrett Warnell, Xuesu Xiao, Peter Stone
International Conference on Robotics and Automation (ICRA), 2022
Paper / Video / Poster
Imitating an expert's video-only demonstration can be challenging in the presence of egocentric viewpoint mismatch. In VOILA, we introduce a unique viewpoint-invariant reward function to effectively imitate demonstrations on a physically different agent.
|
|
Adversarial Imitation Learning from Video using a State Observer
Haresh Karnan, Garrett Warnell, Faraz Torabi, Peter Stone
International Conference on Robotics and Automation (ICRA), 2022
Paper / Video / Poster
SOTA approaches in Imitation from Video-only demonstrations exhibit poor sample efficiency when learning with access to proprioceptive features of the imitator. To tackle this, we introduce VGAIfO-SO, an IfO algorithm that improves sample efficiency through self-supervision.
|
|
Grounded Action Transformation for Sim-to-Real Reinforcement Learning
Josiah Hanna, Siddharth Desai, Haresh Karnan, Garrett Warnell, Peter Stone
Springer. Machine Learning, 2021
Paper
Sim-to-real is the problem of learning a control policy in an inaccurate simulated world such that the learned policy when transferred to the real-world, performs well. In this article, we explore the proposed black-box Sim-to-Real algorithm GAT, and its extension SGAT.
|
|
An Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch
Siddharth Desai, Ishan Durugkar, Haresh Karnan, Josiah Hanna, Garrett Warnell, Peter Stone
Neural Information Processing Systems (NeurIPS), 2020
NeurIPS site / Paper / Poster / arXiv
In this work, we propose the GARAT algorithm, which treats the Sim-to-Real problem as an Imitation from Observation (IfO) problem and uses advances in the IfO literature to transfer a control policy from a source domain to a target domain, using Adversarial Imitation Learning.
|
|
Stochastic Grounded Action Transformation for Robot Learning in Simulation
Haresh Karnan, Siddharth Desai, Josiah Hanna, Garrett Warnell, Peter Stone
International Conference on Intelligent Robots and Systems (IROS), 2020
Long Video / Short Video / Paper / Poster / arXiv
Real world dynamics are often stochastic and robot simulators have an inaccurate approximation of real world dynamics. In this work, we propose a Sim-to-Real algorithm called SGAT and transfer a Humanoid walk from Simulation to Real world.
|
|
Reinforced Grounded Action Transformation for Sim-to-Real Transfer
Haresh Karnan, Siddharth Desai, Josiah Hanna, Garrett Warnell, Peter Stone
International Conference on Intelligent Robots and Systems (IROS), 2020
Long Video / Short Video / Paper / Poster / arXiv
In this work, we propose a Sim-to-Real algorithm called RGAT to ground an inaccurate simulator with data from the real world, using Reinforcement Learning.
|
|
Visual Feedback Control of Tensegrity Robotic Systems
Haresh Karnan, Raman Goyal, Manoranjan Majji, Robert Skelton, Puneet Singla
International Conference on Intelligent Robots and Systems, 2017
Video / Paper
Tensegrity mechanisms are known for their minimal-mass and flexible properties. In this work, we propose using vision based sensing for shape control of such soft robotic manipulators.
|
Workshops, Symposiums, Extended Abstracts
|
|
Aligning Robot Navigation Behaviors with Human Intentions and Preferences
Haresh Karnan
IPPC: Integrated Planning, Perception and Control Workshop, IROS, 2023
Paper / Poster
The research explored in this paper helps address challenges of value alignment in machine learning for robot navigation. It presents new algorithms and a dataset designed to teach mobile robots navigation behaviors that align with human intentions and preferences. These innovations are applicable in diverse settings, including indoor, outdoor, and human-occupied environments.
|
|
Self-Supervised Terrain Representation Learning from Unconstrained Robot Experience
Haresh Karnan, Elvin Yang, Daniel Farkash, Garrett Warnell, Joydeep Biswas, Peter Stone
Pretraining for Robotics Workshop, ICRA, 2023
Project Page / Paper / Video / STERLING @ Lake Austin / Poster
Visual terrain awareness is crucial for autonomous off-road navigation. We propose a novel way to learn relevant terrain representations from unconstrained, multi-modal robot experience collected with any policy. We show how such learned representations enable aligning off-road navigation behaviors with human operator preferences.
|
|
Learning to Extrapolate Human Preferences for Preference-Aligned Path Planning
Elvin Yang, Haresh Karnan, Garrett Warnell, Joydeep Biswas, Peter Stone
Pretraining for Robotics Workshop, ICRA, 2023
Paper / Poster
Robots navigating off-road environments typically require additional human feedback to learn operator preferences for visually novel terrains. In this work, we introduce a framework to extrapolate operator preferences from known terrains by leveraging multi-modality.
|
|
Principles and Guidelines for Evaluating Social Robot Navigation Algorithms
Anthony Francis, Claudia Perez-D'Arpino, Chengshu Li, Fei Xia,..., Haresh Karnan, Peter Stone
ArXiv, 2023
Paper
We introduce a standardized criteria and a metrics framework for evaluating social robot navigation in human occupied spaces. Key principles include safety, comfort, and social competency. This approach fosters fair algorithm comparison across diverse robots, simulators, and datasets, accelerating progress akin to computer vision and traditional robot navigation.
|
|
Benchmarking Social Robot Navigation Across Academia and Industry
Anthony Francis, Claudia Perez-D'Arpino, Chengshu Li, Fei Xia, Alexandre Alahi, Aniket Bera, Abhijat Biswas, Joydeep Biswas, Hao-Tien Lewis Chiang, Michael Everett, Sehoon Ha, Justin Hart, Haresh Karnan, Tsang-Wei Edward Lee, Luis J. Manso, Reuth Mirksy, Soren Pirk, Phani Teja Singamaneni, Peter Stone, Ada V. Taylor, Peter Trautman, Nathan Tsoi, Marynel Vazquez, Xuesu Xiao, Peng Xu, Naoki Yokoyama, Roberto Martın-Martın, Alexander Toshev
HRI in Academia and Industry, AAAI Spring Symposum, 2023 (Best Paper Award Nominee)
Paper
|
|
VOILA: Visual Observation-only Imitation Learning for Autonomous navigation
Haresh Karnan, Garrett Warnell, Xuesu Xiao, Peter Stone
AAAI ML4NAV Spring Symposium, 2021
Talk / Paper
|
|
Solving Service Robot Tasks: UT Austin Villa@Home 2019 Team Report
Rishi Shah, Yuqian Jiang, Haresh Karnan, Gilberto Briscoe-Martinez, Dominick Mulder, Ryan Gupta, Rachel Schlossman, Marika Murphy, Justin W. Hart, Luis Sentis, Peter Stone
AAAI AI-HRI Symposium, 2019
Video / Paper / arXiv
|
|
Extended Abstract: An Imitation from Observation Approach to Sim-to-Real Transfer
Siddharth Desai, Ishan Durugkar, Haresh Karnan, Josiah Hanna, Garrett Warnell, Peter Stone
Robotics Science and Systems, Sim-to-Real Workshop (RSS), 2020
Video / Paper / Poster
In this work, we propose the GARAT algorithm, which treats the Sim-to-Real problem as an Imitation from Observation (IfO) problem and uses advances in the IfO literature to transfer a control policy from a source domain to a target domain, using Adversarial Imitation Learning.
|
|
Visual Servoing of Unmanned Surface Vehicle from Small Tethered Unmanned Aerial Vehicle
Haresh Karnan, Aritra Biswas, Pranav Vaidik Dhulipala, Jan Dufek, Robin Murphy
arXiv preprint, 2017
Paper / arXiv
|
|