Toryn Qwyllyn Klassen

Planning with Epistemic Preferences


We investigate automated planning with epistemic preferences, i.e., soft goals over the knowledge or beliefs of agents.
Toryn Q. Klassen, Christian Muise, and Sheila A. McIlraith
KR 2023
Paper thumbnail

Workshop version:

Towards Human-Aware AI via Planning with Epistemic Preferences


This version was completed after the KR paper, and has some additional planning-with-preferences references.
Toryn Q. Klassen, Christian Muise, and Sheila A. McIlraith
ICAPS 2023 Workshop on Human-Aware Explainable Planning (HAXP 2023)

Epistemic Side Effects: An AI Safety Problem


We introduce the notion of epistemic side effects, potentially undesirable changes made to knowledge or beliefs by an AI system in pursuit of an underspecified objective, and describe a way to sometimes avoid them in reinforcement learning.
Toryn Q. Klassen, Parand Alizadeh Alamdari, and Sheila A. McIlraith
AAMAS 2023 (Blue Sky Ideas Track)
Paper thumbnail

Workshop version:

Epistemic Side Effects & Avoiding Them (Sometimes)


Toryn Q. Klassen, Parand Alizadeh Alamdari, and Sheila A. McIlraith
2022 NeurIPS ML Safety Workshop

Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machines


We investigate using reward machines for reinforcement learning when there is uncertainty about what propositions are true.
Andrew Li, Zizhao Chen, Pashootan Vaezipoor, Toryn Q. Klassen, Rodrigo Toro Icarte, and Sheila A. McIlraith
Deep RL Workshop, NeurIPS 2022
Paper thumbnail

Learning to Follow Instructions in Text-Based Games


We improve the performance of a reinforcement learning system for playing text-based games by converting in-game instructions into Linear Temporal Logic (LTL).
Mathieu Tuli, Andrew Li, Pashootan Vaezipoor, Toryn Q. Klassen, Scott Sanner, and Sheila A. McIlraith
NeurIPS 2022
Paper thumbnail

Workshop version:

Resolving Misconceptions about the Plans of Agents via Theory of Mind


We explore using epistemic planning to resolve discrepancies between agents' beliefs about the validity of plans
Maayan Shvo, Toryn Q. Klassen, and Sheila A. McIlraith
ICAPS 2022
Paper thumbnail

Workshop version:

Explaining the Plans of Agents via Theory of Mind


Maayan Shvo, Toryn Q. Klassen, and Sheila A. McIlraith
ICAPS 2021 Workshop on Explainable AI Planning (XAIP)

An AI Safety Threat from Learned Planning Models


In this position paper, we consider the threat in symbolic planning of plans having undesirable side effects as result of being found for underspecified objectives. We discuss how the risk may be amplified by the use of learned planning models, but also how the learned models may provide features that help ameliorate the risk.
Toryn Q. Klassen, Sheila A. McIlraith, and Christian Muise
ICAPS 2022 Workshop on Reliable Data-Driven Planning and Scheduling
Paper thumbnail

Be Considerate: Avoiding Negative Side Effects in Reinforcement Learning


We consider the problem of avoiding negative side effects on other agents, similarly to in our AAAI 2022 paper, but in the context of reinforcement learning.
Parand Alizadeh Alamdari, Toryn Q. Klassen, Rodrigo Toro Icarte, and Sheila A. McIlraith
AAMAS 2022
Paper thumbnail

Planning to Avoid Side Effects


AI systems may cause negative side effects because their given objectives don't capture everything that they should not do. We consider how to avoid side effects in the context of symbolic planning, including by finding plans that don't interfere with possible goals or plans of other agents.
Toryn Q. Klassen, Sheila A. McIlraith, Christian Muise, and Jarvis Xu
AAAI 2022
Paper thumbnail

Workshop version:

Planning to Avoid Side Effects (Preliminary Report)


This version has some different definitions. The description of the connection to soft goals may be clearer here.
Toryn Q. Klassen and Sheila A. McIlraith
IJCAI 2021 Workshop on Robust and Reliable Autonomy in the Wild (R2AW)

Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning


This is the expanded version of our ICML 2018 paper that introduced reward machines, which give structured representations of reward functions. This paper uses a slightly different definition and introduces the CRM algorithm, a simpler variant of the QRM algorithm from the original paper.
Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith
JAIR, Volume 73, 2022
Paper thumbnail

Representing Plausible Beliefs about States, Actions, and Processes


This thesis deals with the topic of modelling an agent’s beliefs about a dynamic world in a way that allows for changes in beliefs, including retracting of beliefs. It elaborates on work from the KR 2018 and KR 2020 papers below, and also covers knowing how to achieve goals in the presence of environmental processes.
Toryn Q. Klassen
PhD thesis, University of Toronto, 2021
Paper thumbnail

FL-AT: A Formal Language-Automaton Transmogrifier


This work implements a system to translate formal languages into reward machines, following the proposal in our IJCAI 2019 paper.
Jaime Middleton, Toryn Q. Klassen, Jorge Baier, and Sheila A. McIlraith
ICAPS 2020 system demo
Paper thumbnail

Changing Beliefs about Domain Dynamics in the Situation Calculus


We build on the approach from our KR 2018 paper, to model changing beliefs about domain dynamics, such as action effects.
Toryn Q. Klassen, Sheila A. McIlraith, and Hector J. Levesque
KR 2020
Paper thumbnail

Towards the Role of Theory of Mind in Explanation


We provide an account of explanation in terms of the beliefs of agents and the mechanisms by which agents revise their beliefs. The account allows for explanations to refer to beliefs.
Maayan Shvo, Toryn Q. Klassen, and Sheila A. McIlraith
Workshop on EXplanable TRansparent Autonomous Agents and Multi-Agent Systems (EXTRAAMAS 2020)
Paper thumbnail

Epistemic Plan Recognition


In the task of plan recognition, an observer infers the plan and goal of an actor. We introduce the notion of epistemic plan recognition, which uses epistemic logic to model the observer in a plan recognition setting, represent agent beliefs, and allow for the recognition of epistemic goals.
Maayan Shvo, Toryn Q. Klassen, Shirin Sohrabi, and Sheila A. McIlraith
AAMAS 2020
Paper thumbnail

Learning Reward Machines for Partially Observable Reinforcement Learning


In order to address reinforcement learning for (some) partially observable problems, we use discrete optimization to find a form of finite state machine that summarizes the agent's history.
Rodrigo Toro Icarte, Ethan Waldie, Toryn Q. Klassen, Richard Valenzano, Margarita P. Castro, and Sheila A. McIlraith
NeurIPS 2019
Paper thumbnail

Workshop version:

Searching for Markovian Subproblems to Address Partially Observable Reinforcement Learning


Rodrigo Toro Icarte, Ethan Waldie, Toryn Q. Klassen, Richard Valenzano, Margarita P. Castro, and Sheila A. McIlraith
RLDM 2019

LTL and Beyond: Formal Languages for Reward Function Specification in Reinforcement Learning


This paper describes how to convert specifications of reward functions written in LTL and other formal languages into reward machines, and how to apply automated reward-shaping to reward machines.
Alberto Camacho, Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith
IJCAI 2019
Paper thumbnail

Specifying Plausibility Levels for Iterated Belief Change in the Situation Calculus


This paper describes a qualitative model of plausibility based on counting the extensions of certain predicates.
Toryn Q. Klassen, Sheila A. McIlraith, and Hector J. Levesque
KR 2018
Paper thumbnail

Using Reward Machines for High-Level Task Specification and Decomposition in Reinforcement Learning


We introduce reward machines -- a form of automaton that gives a structured description of a reward function. This structure can be exploited by reinforcement learning algorithms to learn faster (analogously to how the structure of formulas was used in the AAMAS 2018 paper below).
Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith
ICML 2018
Paper thumbnail

Teaching Multiple Tasks to an RL Agent using LTL


By defining tasks using linear temporal logic (LTL) formulas, we're able to speed up learning how to complete the tasks by exploiting the structure of the formulas.
Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith
AAMAS 2018
Paper thumbnail

Advice-Based Exploration in Model-Based Reinforcement Learning


Linear temporal logic (LTL) formulas and a heuristic are used to guide exploration during reinforcement learning. Note that the slides have embedded videos that may not play on some systems.
Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith
Canadian AI 2018
Paper thumbnail

Workshop version:

Using Advice in Model-Based Reinforcement Learning


Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith
RLDM 2017

Towards Representing What Readers of Fiction Believe


We use a temporal modal logic to describe a reader's beliefs about the reading process. We also discuss some ideas on how to model how a reader "carries over" real-world knowledge into fictional stories.
Toryn Q. Klassen, Hector J. Levesque, and Sheila A. McIlraith
Commonsense 2017
Paper thumbnail

Resource-bounded inference with three-valued neighborhood semantics


This is an expanded report based on "Towards Tractable Inference for Resource-Bounded Agents".
Toryn Q. Klassen
MSc paper, University of Toronto, 2015
Paper thumbnail

Towards Tractable Inference for Resource-Bounded Agents


This paper, written during my master's program, considers a formal model of belief that was meant to avoid attributing unlimited reasoning power to agents.
Toryn Q. Klassen, Sheila A. McIlraith, and Hector J. Levesque
Commonsense 2015
Paper thumbnail

Independence of Tabulation-Based Hash Classes


This theory paper about properties of hash functions resulted from my undergraduate research in theoretical computer science.
Toryn Q. Klassen and Philipp Woelfel
LATIN 2012
Paper thumbnail