Ian Berlot-Attwell

Research

My research interest is in how discrete and recombinable concepts and skills can emerge in learning systems; my current line of work focuses on library learning, i.e., tool creation, in a world-modelling context. Given a system (typically an LLM) that solves a stream of tasks (e.g., by generating Python code), the question is whether we can learn reusable and composable tools (e.g., well designed, non-trivial, reusable Python functions). More broadly, I am interested in the larger problem of library learning for program synthesis, and how hierarchical skills can be built up over time.

Other research interests include world modelling, multi-modal representations of text and vision, and model evaluation. I have done research measuring compositional generalization to new combinations of textual and visual primitives, as well as research into the evaluation of generative dialogue agents.

Selected Publications

Is This LLM Library Learning? Evaluation Must Account For Compute and Behaviour> [paper, code, experiment data]
Ian Berlot-Attwell, Tobias Sesterhenn, Frank Rudzicz, Xujie Si
Accepted Oral, EACL 2026

Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisions [paper]
Stéphane Aroca-Ouellette*, Ian Berlot-Attwell*, Panagiotis Lymperopoulos, Abhiramon Rajasekharan, Tongqi Zhu, Herin Kang, Kaheer Suleman, Sam Pasupalak
*Equal contribution
Accepted to the World Modeling Workshop at Mila, 2026

Attribute Diversity Determines the Systematicity Gap in VQA [paper]
Ian Berlot-Attwell, A. Michael Carrell, Kumar Krishna Agrawal, Yash Sharma, Naomi Saphra
Accepted to EMNLP 2024

Relevance in Dialogue: Is Less More? An Empirical Comparison of Existing Metrics, and a Novel Simple Metric [paper, code]
Ian Berlot-Attwell, Frank Rudzicz
Published: ConvAI Workshop at ACL 2022
Awarded best PhD Oral Presentation at UofT Graduate Language Research Day 2021

Exploring Text Specific and Blackbox Fairness Algorithms in Multimodal Clinical NLP [paper, code]
John Chen, Ian Berlot-Attwell, Xindi Wang, Safwan Hossain, Frank Rudzicz
Published: ClinicalNLP Workshop at EMNLP 2020
Awarded Best Short Paper at ClinicalNLP

Other works

CASSANDRA: Programmatic and Probabilistic Learning and Inference for Stochastic World Modeling [paper]
Panagiotis Lymperopoulos, Abhiramon Rajasekharan, Ian Berlot-Attwell, Stéphane Aroca-Ouellette, Kaheer Suleman
Accepted to the World Modeling Workshop at Mila, 2026

LLM Library Learning Fails: A LEGO-Prover Case Study [paper]
Ian Berlot-Attwell, Frank Rudzicz, Xujie Si
arXiv preprint 2025

NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation [paper, code]
Kaustubh Dhole et al.
Published: NEJLT Vol. 9 No. 1, 2023

Neuro-Symbolic VQA: A review from the perspective of AGI desiderata [paper]
Ian Berlot-Attwell
arXiv preprint 2021

On the Use of Linguistic Features for the Evaluation of Generative Dialogue Systems [paper]
Ian Berlot-Attwell, Frank Rudzicz
arXiv preprint 2021