Khimya Khetarpal

Khimya Khetarpal

I am a fourth-year computer science Ph.D. candidate with Doina Precup in the Reasoning and Learning Lab at McGill University and Mila, Montreal.

Prior to doctorate studies, I worked in the Perceptual Computing space at Intel, with application to self-driving cars. I did my Masters at University of Florida, where I was advised by Eakta Jain. Before joining graduate school, I worked as a Research Associate at Indian Institute of Technology, Kanpur, where I was advised by Laxmidhar Behera. Broadly, my research interests span reinforcement learning, computer vision, and robotics.

Email  /  CV  /  GitHub  /  Google Scholar  /  Twitter  /  YouTube

profile photo
  • DeepMind Fall 2019
    Ph.D. Research Intern
  • McGill University 2017 - Now
    Ph.D. in Computer Science
  • Intel 2016 - 2017
    Perceptual Computing Engineer
  • Univerity of Florida 2014 - 2016
    Masters in Computer Engineering
  • IIT Kanpur 2013 - 2014
    Research Associate
  • Robert Bosch 2011-2012
    Embedded Software Development
  • VIT University 2007 - 2011
    B.Tech in Electronics & Communication Engineering



Highlights and News



Research

My research focuses on learning generalized temporal abstractions across both action and perception grounded in theoretical foundations of reinforcement learning. Understanding how humans perceive their world and reason with abstract knowledge has inspired much of my research on visual attention, state and temporal abstraction moving towards lifelong learning agents. Representative papers are highlighted.














sym The Paradox of Choice: Using Attention in Hierarchical Reinforcement Learning
Khimya Khetarpal*, Andrei Nica*, Doina Precup
Under Review
sym Temporally Abstract Partial Models
Khimya Khetarpal, Gheorghe Comanici, Doina Precup
Under Review
sym Towards Continual Reinforcement Learning: A Review and Perspectives
Khimya Khetarpal*, Matthew Riemer*, Irina Rish, Doina Precup
Preprint
sym Learning Robust State Abstractions for Hidden-Parameter Block MDPs
Amy Zhang, Shagun Sodhani, Khimya Khetarpal, Joelle Pineau
International Conference on Learning Representations (ICLR), 2021
sym Self-Supervised Attention-Aware Reinforcement Learning
Haiping Wu, Khimya Khetarpal, Doina Precup
Association for the Advancement of Artificial Intelligence (AAAI), 2021
sym Variance Penalized On-Policy and Off-Policy Actor-Critic
Arushi Jain, Gandharv Patil, Ayush Jain, Khimya Khetarpal, Doina Precup
Association for the Advancement of Artificial Intelligence (AAAI), 2021
sym What can I do here? A Theory of Affordances in Reinforcement Learning (Featured in MIT Technology Review)
Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici, David Abel, Doina Precup
International Conference on Machine Learning (ICML), 2020
 
sym Options of Interest: Temporal Abstraction with Interest Functions
Khimya Khetarpal, Martin Klissarov, Maxime Chevalier-Boisvert, Pierre-Luc Bacon, Doina Precup
Association for the Advancement of Artificial Intelligence (AAAI), 2020
sym Value Preserving State-Action Abstractions
David Abel, Nathan Umbanhowar, Khimya Khetarpal, Dilip Arumugam, Doina Precup, and Michael L. Littman
International Conference on Artificial Intelligence and Statistics (AISTATS), 2020
sym Learning Generalized Temporal Abstractions across Both Action and Perception (Scholarship Award)
Khimya Khetarpal
Association for the Advancement of Artificial Intelligence (AAAI), 2019
Doctorial Consortium Track
sym Learning Options with Interest Functions (3 Minute Thesis Finalist)
Khimya Khetarpal, Doina Precup
Association for the Advancement of Artificial Intelligence (AAAI), 2019
Student Abstract Track
sym Variational State Encoding as Intrinsic Motivation in Reinforcement Learning
Martin Klissarov*, Riashat Islam*, Khimya Khetarpal, Doina Precup
The Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM), 2019
sym Attend before you act: Leveraging human visual attention for continual learning (Best Paper Award- 3rd Place)
Khimya Khetarpal, Doina Precup
Lifelong Learning: A Reinforcement Learning Approach Workshop (ICML), 2018
sym Safe option-critic: Learning safety in the option-critic architecture
Arushi Jain*, Khimya Khetarpal*, Doina Precup
Adaptive Learning Agents Workshop, (ICML), 2018.
Invited for submission to special issue of The Knowledge Engineering Review (Cambridge University Press journal)
sym Re-evaluate: Reproducibility in evaluating reinforcement learning algorithms
Khimya Khetarpal*, Zafarali Ahmed*, Andre Cianflone, Riashat Islam, Joelle Pineau
Reproducibility in Machine Learning Workshop, (ICML), 2018.
sym Environments for Lifelong Reinforcement Learning
Khimya Khetarpal*, Shagun Sodhani*, Sarath Chandar, Doina Precup
Continual Learning Workshop, Workshop, (NeurIPS), 2018.
sym Creating Segments and Effects on Comics by Clustering Gaze Data
Ishwarya Thirunarayanan, Khimya Khetarpal, Sanjeev Koppal, Olivier Le Meur, John Shea and Eakta Jain
ACM Transactions on Multimedia Computing, Communications, and Applications, (TOMM), 2017.
sym A Preliminary Benchmark Of Four Saliency Algorithms On Comic Art
Khimya Khetarpal, Eakta Jain
International Workshop on Multimedia Artworks Analysis (MMArt),
(IEEE ICME), 2016.
sym Mobile robot navigation using evolving neural controller in unstructured environments
AwhanPatnaik, Khimya Khetarpal, Laxmidhar Behera
International Conference on – Advances in Control and Optimization of Dynamical Systems,
(IFAC), 2014.



Talks
sym Towards Continual Reinforcement Learning: A Review and Perspectives
RIKEN Center for Advanced Intelligence Project- Approximate Bayesian Inference Team (Japan), Invited Talk, 2021
sym What can I do here? A Theory of Affordances in Reinforcement Learning (Featured in MIT Technology Review)
International Conference on Machine Learning (ICML), Virtual Vienna, 2020 Northeast Reinforcement Learning and Decision Making Symposium (NERDS), 2020
sym Options of Interest: Temporal Abstraction with Interest Functions
Association for the Advancement of Artificial Intelligence (AAAI), New York, 2020.
sym Learning Generalized Temporal Abstractions across Both Action and Perception
Association for the Advancement of Artificial Intelligence (AAAI), Hawaii, 2019
Doctorial Consortium Track, (Mentor: Michael Littman)
sym Learning Options with Interest Functions
Association for the Advancement of Artificial Intelligence (AAAI), Hawaii, 2019
Student Abstract Track, (3 Minute Thesis Finalist)
sym Attend before you act: Leveraging human visual attention for continual learning (Best Paper Award- 3rd Place)
Lifelong Learning: A Reinforcement Learning Approach Workshop (ICML), Stockholm, 2018
Mentoring
Gabriela Moisescu-Pareja (McGill, CS Undergraduate)


Haiping Wu (McGill, CS Masters)

Work led by Haiping on "Self-Supervised Attention-Aware Reinforcement Learning" accepted to AAAI, 2021. Also to appear at NeurIPS Workshop on Object Representations for Learning and Reasoning. See preliminary draft here.

Teaching
sym Teaching Assistant, COMP-767 Reinforcement Learning, Winter 2020

Teaching Assistant, COMP-208 Computers in Engineering, Winter 2018

Guest Lecture, Hierarchical RL, Management Studies, 2019 [slides]
sym Reinforcement Learning, IVADO Deep Learning Summer School, 2019 [slides]
sym Lecturer, Reinforcement Learning, 2020 [slides]

Lecturer, Deep Reinforcement Learning, 2019 [slides]

Teaching Assistant, 2018 [resources]
Organizational Roles
sym Continual Reinforcement Learning, Un-Workshop WiML, ICML 2020

Lifelong Learning: A Reinforcement Learning Approach (LLARLA), RLDM 2019

Multi-Task and Lifelong Reinforcement Learning Workshop, ICML 2019

Area Chair, Women in Machine Learning (WiML), NeurIPS 2018
Reviewing
sym Reviewer, AISTATS ('21), ICLR ('20), NeurIPS ('20)

Reviewer, NeurIPS, Deep RL Workshop ('20), Reproducibility Challenge ('19)

Program Committee, Continual Learning Workshop, NeurIPS 2018

Reviewer, AI for Social Good Workshop, NeurIPS 2018

© Khimya Khetarpal
Base Design & CSS courtsey Jon Barron.