Research

Papers/ Technical Reports

• Robust Policy Gradient with Successor Features for Transfer in Reinforcement Learning
  Ehsan Mehralian
  MS Thesis, 2020.
  abstract

  Humans have a significant ability to adapt their skills and use their already gained knowledge in a new situation with different goals or rewards. Such adaptation capability is an important sign of intelligence, but current reinforcement learning agents often perform poorly in similar situations. In this work, we propose a framework to learn a generalizable policy that can efficiently adapt to an unseen task where different tasks only differ in their reward function. Our approach is based on two key components: (a) successor features, a representation scheme that makes it possible to immediately compute the value of a policy on any task, and (b) Robust policy gradient, a generalization of standard policy gradient theorem to find a generalizable policy that can work well on a set of tasks. Putting these two together leads to an approach that integrates naturally into the RL framework and can be applied to all Actor-Critic methods without the need for much change in the original algorithm implementation. We provide our approach in a firm theoretical ground and present experiments that show it successfully promotes transfer in A2C and PPO methods in a sequence of tasks in the Linear Quadratic Regulator environment.

  / pdf / code (soon)

• Shrinkage-based policy gradient using James-Stein estimator for Reinforcement Learning
  Ehsan Mehralian, Amir-massoud Farahmand
  Technical Report, 2019.
  

• Expansive Representation Aggregation for Learningto Detect Equivalent Questions
  Ehsan Mehralian, Farzaneh Mirzazadeh
  Submitted to AAAI, 2018.

• Homograph Disambiguation System Using Semi-supervised Learning
  Ehsan Mehralian
  BS Thesis, 2018.
  

Projects

• Differentially Private Reinforcement Learning
  Studied the effect of differential privacy mechanisms in the generalizability of an RL agent. Introduced a differentially private DQN method that makes the reward functions indistinguishable and proposed a technique to transfer knowledge between tasks without privacy concern.
  
  

• More Structured Sparsity For Convolutional Neural Networks
  Proposed a method to find the best kernel size in each layer of CNNs. Using this method, we compressed VGG network so that the final pruned network can be retrained 26% faster with only 0.2 % accuracy drop on CIFAR-10 dataset.
  
  

Teaching

• Winter 2020, Fall 2019: Introduction to Machine Learning (CSC 311)
  
• Summer 2020, Summer 2019: Algorithm Design, Analysis & Complexity (CSC 373)
  
• Winter 2019, Fall 2018: Mathematical Expression and Reasoning (CSC 165)