Rent3D: Floor-Plan Priors for Monocular Layout Estimation

The goal of this work is to enable a 3D ``virtual-tour'' of an apartment given a small set of monocular images of different rooms, as well as a 2D floor plan. We frame the problem as inference in a Markov Random Field which reasons about the layout of each room and its relative pose (3D rotation and translation) within the full apartment. This gives us accurate camera pose in the apartment for each image. What sets us apart from past work in layout estimation is the use of floor plans as a source of prior knowledge, as well as localization of each image within a bigger space (apartment). In particular, we exploit the floor plan to impose aspect ratio constraints across the layouts of different rooms, as well as to extract semantic information, e.g., the location of windows which are marked in floor plans. We show that this information can significantly help in resolving the challenging room-apartment alignment problem. We also derive an efficient exact inference algorithm which takes only a few ms per apartment. This is due to the fact that we exploit integral geometry as well as our new bounds on the aspect ratio of rooms which allow us to carve the space, significantly reducing the number of physically possible configurations. We demonstrate the effectiveness of our approach on a new dataset which contains over 200 apartments.

Results

People

Chenxi Liu, Alexander Schwing, Kaustav Kundu, Raquel Urtasun, Sanja Fidler

Data

Rent3D dataset:

Rental ad data for 215 apartments

# apartments	# of images	avg. # rooms per apt	avg. # walls per apt	avg. # windows per apt	avg. # doors per apt
Statistics for the Rent3D Dataset
`215`	`1570`	`6`	`31`	`6`	`9`

Floor-plans have annotations for:

Room types
Walls (represented with lines and dimension in real world)
Windows
Doors

Each photo of a room has the following ground-truth:

Scene type (e.g., kitchen, bedroom, outdoor)
Room layout
Windows
Doors

Ground-truth alignment of each photo within floor-plan

Contact

For questions regarding the data please contact Kaustav Kundu and Sanja Fidler.

Relevant Publications

If you use the data please cite the following publication:

Rent3D: Floor-Plan Priors for Monocular Layout Estimation (oral presentation)

Chenxi Liu*, Alex Schwing*, Kaustav Kundu, Raquel Urtasun, Sanja Fidler

In Computer Vision and Pattern Recognition (CVPR), Boston, 2015

* Denotes equal contribution

Paper Abstract Suppl. Mat. Bibtex

@inproceedings{ApartmentsCVPR15,
title = {Rent3D: Floor-Plan Priors for Monocular Layout Estimation},
author = {Chenxi Liu and Alex Schwing and Kaustav Kundu and Raquel Urtasun and Sanja Fidler},
booktitle = {CVPR},
year = {2015}}

The goal of this paper is to enable a 3D "virtual-tour" of an apartment given a small set of monocular images of different rooms, as well as a 2D floor plan. We frame the problem as the one of inference in a Markov random field which reasons about the layout of each room and its relative pose (3D rotation and translation) within the full apartment. This gives us information, for example, about in which room the picture was taken. What sets us apart from past work in layout estimation is the use of floor plans as a source of prior knowledge. In particular, we exploit the floor plan to impose aspect ratio constraints across the layouts of different rooms, as well as to extract semantic information, e.g., the location of windows which are labeled in floor plans. We show that this information can significantly help in resolving the challenging room-apartment alignment problem. We also derive an efficient exact inference algorithm which takes only a few ms per apartment. This is due to the fact that we exploit integral geometry as well as our new bounds on the aspect ratio of rooms which allow us to carve the space, reducing significantly the number of physically possible configurations. We demonstrate the effectiveness of our approach in a new dataset which contains over 200 apartments.