Abstract: The goal of low-level vision is to estimate an underlying scene, given an observed image. Real-world scenes (eg, albedos or shapes) can be very complex, conventionally requiring high dimensional representations which are hard to estimate and store. We propose a low-dimensional representation, called a scene recipe, that relies on the image itself to describe the complex scene configurations.

Shape recipes are an example: these are the regression coefficients that predict the bandpassed shape from image data. We describe the benefits of this representation, and show two uses: (1) we improve stereo shape estimates by learning shape recipes at low resolution and applying them at high resolution; (2) Shape recipes implicitly contain information about lighting and materials and we show they can be used for material segmentation.

Joint work with Antonio Torralba.