Use the growing number of video adaptations from novels to films to find an alignment between the two forms of storytelling. Find differences, use vivid descriptions from the book to describe video shots.

alignment cues between books and movies

Paper and Poster

Book2Movie: Aligning Video scenes with Book chapters
Makarand Tapaswi, Martin Baeuml and Rainer Stiefelhagen
IEEE Computer Vision and Pattern Recognition (CVPR Poster), Boston, June 2015
[paper] [supp] [extended abstract]


  • Align two modes of story-telling, books and their video adaptations
  • Efficient solution through shortest path based approach, which disregards all assumptions about monotonic structure of the adaptations
  • Allows to find high-level differences (was the scene part of the book?) between the adaptations
  • Evaluation on the first movie of Harry Potter series, and first season of Game of Thrones


Game of Thrones season/book 1 and Harry Potter series movie/book 1
This package contains the ground truth alignment between book chapters and parts of the corresponding video. We also include the face tracks obtained from both videos for further work in person identification and the shot boundaries to reduce video processing overhead.


At a glimpse

A snapshot of all multimedia related projects can be found here. This is the link you will come to from any of our papers too!

Multimedia Resources

I hope to collect links at one place to make it easy to browse the wonderful works.

CV, MM Papers on the Web

Meta-resources for Computer Vision conferences CVPapers, ACM Multimedia MM papers, and Multimedia Information Retrieval papers MIR.