TV series and films are often based on book adaptations. We aim to jointly analyze books and their adaptations, specifically related to finding scene-level differences or describing video segments.

Read more about it in our CVPR 2015 paper...

Inspired by XKCD we present an automatic way to generate charts which capture interactions between characters. The charts can be used to provide a quick glimpse of the episode, or facilitate smart video browsing and story-based retrieval.

Read more about it in our CVPR 2014 paper...

Searching through large collections of structured video data such as TV series and movies is a very challenging problem. In this project, we use natural language descriptions such as plot synopses to bridge the gap between videos and storylines thus allowing to query for meaningful story events.

Read more about it in our ICMR 2014 paper...

Person identification is a very popular problem in the field of multimedia analysis. In this area, we worked on multiple tasks. One of the first was to move from face recognition towards full person recognition and specially even when the face is unseen thus increasing coverage and improving id performance. Read more about it in our CVPR 2012 paper...

We also worked on using labeled and unlabeled data along with constraints to learn better semi-supervised person models. Read more about it in our CVPR 2013 paper...

Finally, scaling up the problem of learning and identifying across lots of video data is another interesting task. Unlike before, we present results on an entire season of a TV series constituting 15+ hours of video. Read more about it in our AVSS 2014 paper...

We spent some time on evaluating the extraction of weak labels using subtitles and transcripts and were able to use an energy function based model to improve performance. Read more about it in our FG 2015 paper...

Mini projects for various courses during the duration of my undergraduate and graduate studies. A short description for each can be found here.