My work is on helping machines analyze and understand videos, particularly TV series and films. As all stories revolve around characters, we lay a strong focus on analyzing them, answering typical questions through popular vision tasks — detection, clustering, identification — to break the complex video into understandable meta-data (e.g., who appears when).
I work particularly in the area of enhancing such video meta-data along with rich textual descriptions in the form of plot synopses (example) or books (for film-adaptations) to perform semantic tasks such as story-based retrieval, or weak label mining. Inspired by XKCD, we also worked on visualization of the storyline through character interactions.
If you have any questions, please feel free to contact me.
Co-organizing a workshop at ICCV on video and language (MovieQA + LSMDC).
Joined the machine learning group at University of Toronto in October 2016.