Be Considerate: Avoiding Negative Side Effects in Reinforcement Learning
We consider the problem of avoiding negative side effects on other agents, similarly to in our AAAI 2022 paper, but in the context of reinforcement learning.
AAMAS 2022
We consider the problem of avoiding negative side effects on other agents, similarly to in our AAAI 2022 paper, but in the context of reinforcement learning.
AAMAS 2022
Planning to Avoid Side Effects
AI systems may cause negative side effects because their given objectives don't capture everything that they should not do. We consider how to avoid side effects in the context of symbolic planning, including by finding plans that don't interfere with possible goals or plans of other agents.
AAAI 2022
Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning
This is the expanded version of our ICML 2018 paper that introduced reward machines, which give structured representations of reward functions. This paper uses a slightly different definition and introduces the CRM algorithm, a simpler variant of the QRM algorithm from the original paper.
JAIR, Volume 73, 2022
Avoiding Negative Side Effects by Considering Others
This is a preliminary version of the paper "Be Considerate: Avoiding Negative Side Effects in Reinforcement Learning" above.
NeurIPS 2021 Workshop on Safe and Robust Control of Uncertain Systems
Planning to Avoid Side Effects (Preliminary Report)
This is a preliminary version of our paper "Planning to Avoid Side Effects" above, with some different definitions.
IJCAI 2021 Workshop on Robust and Reliable Autonomy in the Wild (R2AW)
Explaining the Plans of Agents via Theory of Mind
We explore using epistemic planning to resolve discrepancies between agents' beliefs about the validity of plans.
ICAPS 2021 Workshop on Explainable AI Planning (XAIP)
Representing Plausible Beliefs about States, Actions, and Processes
This thesis deals with the topic of modelling an agent’s beliefs about a dynamic world in a way that allows for changes in beliefs, including retracting of beliefs. It elaborates on work from the KR 2018 and KR 2020 papers below, and also has material regarding beliefs about environmental processes.
PhD thesis, University of Toronto, 2021
FL-AT: A Formal Language-Automaton Transmogrifier
This work implements a system to translate formal languages into reward machines, following the proposal in our IJCAI 2019 paper.
ICAPS 2020 system demo
Changing Beliefs about Domain Dynamics in the Situation Calculus
We build on the approach from our KR 2018 paper, to model changing beliefs about domain dynamics, such as action effects.
KR 2020
Towards the Role of Theory of Mind in Explanation
We provide an account of explanation in terms of the beliefs of agents and the mechanisms by which agents revise their beliefs. The account allows for explanations to refer to beliefs.
EXTRAAMAS 2020
Epistemic Plan Recognition
In the task of plan recognition, an observer infers the plan and goal of an actor. We introduce the notion of epistemic plan recognition, which uses epistemic logic to model the observer in a plan recognition setting, represent agent beliefs, and allow for the recognition of epistemic goals.
AAMAS 2020
Learning Reward Machines for Partially Observable Reinforcement Learning
In order to address reinforcement learning for (some) partially observable problems, we use discrete optimization to find a form of finite state machine that summarizes the agent's history.
NeurIPS 2019
LTL and Beyond: Formal Languages for Reward Function Specification in Reinforcement Learning
This paper describes how to convert specifications of reward functions written in LTL and other formal languages into reward machines, and how to apply automated reward-shaping to reward machines.
IJCAI 2019
Searching for Markovian Subproblems to Address Partially Observable Reinforcement Learning
This is a preliminary version of the paper "Learning Reward Machines for Partially Observable Reinforcement Learning" above.
RLDM 2019
Specifying Plausibility Levels for Iterated Belief Change in the Situation Calculus
This paper describes a qualitative model of plausibility based on counting the extensions of certain predicates.
KR 2018
Using Reward Machines for High-Level Task Specification and Decomposition in Reinforcement Learning
We introduce reward machines -- a form of automaton that gives a structured description of a reward function. This structure can be exploited by reinforcement learning algorithms to learn faster (analogously to how the structure of formulas was used in the AAMAS 2018 paper below).
ICML 2018
Teaching Multiple Tasks to an RL Agent using LTL
By defining tasks using linear temporal logic (LTL) formulas, we're able to speed up learning how to complete the tasks by exploiting the structure of the formulas.
AAMAS 2018
Advice-Based Exploration in Model-Based Reinforcement Learning
Linear temporal logic (LTL) formulas and a heuristic are used to guide exploration during reinforcement learning. Note that the slides have embedded videos that may not play on some systems.
Canadian AI 2018
Towards Representing What Readers of Fiction Believe
We use a temporal modal logic to describe a reader's beliefs about the reading process. We also discuss some ideas on how to model how a reader "carries over" real-world knowledge into fictional stories.
Commonsense 2017
Using Advice in Model-Based Reinforcement Learning
This is a preliminary version of the paper "Advice-Based Exploration in Model-Based Reinforcement Learning" above.
RLDM 2017
Resource-bounded inference with three-valued neighborhood semantics
This is an expanded report based on "Towards Tractable Inference for Resource-Bounded Agents".
MSc paper, University of Toronto, 2015
Towards Tractable Inference for Resource-Bounded Agents
This paper, written during my master's program, considers a formal model of belief that was meant to avoid attributing unlimited reasoning power to agents.
Commonsense 2015
Independence of Tabulation-Based Hash Classes
This theory paper about properties of hash functions resulted from my undergraduate research in theoretical computer science.
LATIN 2012
