PhD Researcher · University of Toronto

Sheza Munir

Responsible and Safe AI

I am a first-year PhD student in Computer Science at the University of Toronto, advised by Dr. Ishtiaque Ahmed. My research sits at the intersection of NLP and sociotechnical systems — I study data annotation as a human practice, examining how annotator expertise, lived experience, and disagreement shape the models we build.

Before Toronto, I completed my Master's at the University of Michigan, where I worked with Dr. Lu Wang on LLM factuality evaluation. My work spans annotation pipelines, fairness, and safety in AI — with a conviction that subjectivity and conflict are signal, not noise.

Sheza Munir
Data Annotation as Sociotechnical Practice

How annotator identity, expertise, and lived experience influence what gets labeled — and what gets erased — in training data.

Subjectivity & Disagreement in AI

Treating label conflict as meaningful signal. Building aggregation and reasoning frameworks for high-disagreement, socially sensitive tasks.

LLM Factuality & Evaluation

Benchmarking and probing the factual reliability of large language models, with a focus on long-form generation and hallucination triggers.

AI Safety & Fairness

Ethical reasoning frameworks for AI systems. Deepfake detection and robustness in low-resource language settings.

2026
FAccT 2026
The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation
Sheza Munir, Benjamin Mah, Krisha Kalsi, Shivani Kapania, Julian Posada, Edith Law, Ding Wang, Syed Ishtiaque Ahmed
2026
CAI 2026
A Fair, Multi-Perspective, Ethical Reasoning Framework
Sheza Munir, Ahanaf Rodoshi, Sumin Lee, Feiran Chang, Xujie Si, Syed Ishtiaque Ahmed
2026
ICLR 2026
ExpertLongBench: Benchmarking Language Models on Expert-Level Long-Form Generation Tasks with Structured Checklists
Jie Ruan, Inderjeet Nair, Shuyang Cao, Amy Liu, Sheza Munir, et al., Lu Wang
2025
ACL 2025
FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation
Farima Fatahi Bayat, Lechen Zhang, Sheza Munir, Lu Wang
2025
EMNLP 2025
Verifact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts
Xin Liu, Lechen Zhang, Sheza Munir, Yiyang Gu, Lu Wang
2024
ACL Findings 2024
Deepfake Defense: Constructing and Evaluating a Specialized Urdu Deepfake Audio Dataset
Sheza Munir, Wassay Sajjad, Mukeet Raza, Emaan Abbas, Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza
See full publication list and citation metrics
Google Scholar · Sheza Munir
Google Scholar ↗
2025 – Present
PhD in Computer Science
University of Toronto — The Third Space Lab
Studying data annotation as a sociotechnical process. Designing annotation and task-routing frameworks that treat subjectivity and conflict as signal. Advised by Dr. Syed Ishtiaque Ahmed.
2025 – Present
Teaching Assistant — CSC 300: Computers and Society
University of Toronto
2024 – 2025
Research Assistant — LAUNCH Lab
University of Michigan, Ann Arbor · Advisor: Dr. Lu Wang
Developed VERIFY, a factuality evaluation pipeline for LLMs, and FACTBENCH — a 1,000-prompt benchmark for identifying hallucination triggers. Paper accepted at ACL 2025.
2023 – 2025
Master of Health Informatics (MHI)
University of Michigan — Data Science & Machine Learning focus
Graduate Student Instructor for SI 504: Git, Shell, and Servers.
2023 – 2025
Graduate Student Instructor — SI 504
University of Michigan
Managed course content and mentored 300+ students in version control and shell scripting.
2021 – 2023
Research Assistant & Lab Head — CSALT
Lahore University of Management Sciences (LUMS)
Pioneered a 35,000-audio Urdu deepfake detection dataset using deep learning. Managed a team of 33 students across data acquisition, annotation, and model training. Paper accepted at ACL 2024.
2019 – 2023
BSc in Computer Science
Lahore University of Management Sciences (LUMS)
Dean's Honor List.
Outstanding Graduate Student Instructor Award — Nominee
SI 504: Git, Shell and Servers · University of Michigan
2024
Winner — Health Tech Pitch Competition
Conference: Health Data for Pakistan · Aga Khan University · Migraines Prediction Project
2023
100% Merit Scholarship
University of Michigan · Full tuition for 4 semesters of Masters
2023–25
Student as Co-Researcher Grant (ScR)
LUMS · Awarded for the Migraines Prediction Project
2022
Best Project — Aarzu Physical Therapy App
LUMS Collaborative Project Showcase · Featured Fall 2021 & Spring 2022
2022
International Chemistry Olympiad — Honourable Mention
51st IChO · Represented Pakistan · Top finish among 80 countries
2019
Dean's Honor List — Rank 2 of 242
LUMS · 100% merit scholarship awardee · 2019, 2020, 2021, 2022
2019–23
Leadership & Community
Social Coordinator — CS Graduate Society
University of Toronto · Organizes book club, cookie breaks, and community events for the CS grad community
Google Developer Student Club Lead
GDSC LUMS · 2021–2022 · Organized tech talks, camps, workshops, and hackathons
VP Innovation & Design Society
LUMS · 2021–2022 · Led UI/UX design sittings, talks, and competitions focused on accessible tech
Selected Projects
Aarzu — Physical Therapy App
mHealth app for low-income, low-literate users at Pakistan Society for Rehabilitation of the Disabled. 100% task-completion with doctors, 84.4% with patients. Best Project, Fall 2021.
Migraines Prediction Engine
LSTM models on wearable time-series data (heart rate, EDA, stress) for 5 patients. Predicted migraines one day in advance with 88% accuracy. ScR Grant winner.
MLH Open Source Fellowship
Major League Hacking · Summer 2022 · Built unit testing and search-engine failure backup systems in Python with 15 international interns
I'm always open to new collaborations, conversations about responsible AI, and connecting with researchers working on fairness, annotation, and AI safety.
Location Toronto, Canada
Twitter @sheza_munir
GitHub ShezaMunir
LinkedIn shezamunir