Fall 2008 Talk Descriptions

Abstract: Advances in biology and nanotechnology require the development of increasingly complex strategies for manipulating micro-nanometer-sized objects, the lack of which is a key barrier to such areas as high-throughput genetic research and rapid prototyping of high-performance nano devices, to name just a few. Leveraging technologies of microelectromechanical systems (MEMS) and robotics, we are developing fundamental technologies, MEMS tools, and micro-nanorobotic systems to autonomously manipulate and characterize biological cells and nanomaterials. In micro-nanomanipulation, two most important sensing modalities are optical or electron microscopy vision and micro-nanoNewton force. Besides introducing our research activities in biology and nanotechnology, I will discuss opportunities and interesting challenges of computer vision microscopy in micro-nanomanipulation.

Bio: Yu Sun is an Assistant Professor of the Dept. of Mechanical and Industrial Engineering with joint appointments in the Inst. of Biomaterials and Biomedical Engineering and the Dept. of Electrical and Computer Engineering at the University of Toronto.

Sun received an M.S. degree in electrical engineering in 2001 and a Ph.D. degree in mechanical engineering in 2003, both from the University of Minnesota. His degrees from China were a B.S. degree in electrical engineering from the Dalian University of Technology in 1996 and an M.S. degree in electrical engineering from the Inst. of Automation, Chinese Academy of Sciences in Beijing in 1999. Prior to joining the faculty of Toronto in 2004, Sun held a Research Scientist position at the Swiss Federal Institute of Technology (ETH-Zürich).

Sun has published over 100 articles and is an inventor of 8 US/PCT patents (granted or pending) on micro-nano systems and devices. He is the Canada Research Chair in Micro and Nano Engineering Systems. He was a recipient of Premiere’s Early Researcher Award for research in “MEMS-assisted micro and nanomanipulation of biological cells and nanomaterials” in 2006 and MRI Innovations Award in 2008. He has also won best paper awards and was nominated for best paper awards at major robotics and automation conferences.

Abstract: Computational Photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics+vision, electronics, art, and online sharing in social networks. The field is evolving through three phases. The first phase was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view or depth of field. I call this Epsilon Photography. Due to limited capabilities of a camera, the scene is sampled via multiple photos, each captured by epsilon variation of the camera parameters. It corresponds to the low-level vision: estimating pixels and pixel features. The second phase is building tools that go beyond capabilities of this super-camera. I call this Coded Photography. The goal here is to reversibly encode information about the scene in a single photograph (or a very few photographs) so that the corresponding decoding allows powerful decomposition of the image into light fields, motion deblurred images, global/direct illumination components or distinction between geometric versus material discontinuities. This corresponds to the mid-level vision: segmentation, organization, inferring shapes, materials and edges. The third phase will be about going beyond the radiometric quantities and challenging the notion that a camera should mimic a single-chambered human eye. Instead of recovering physical parameters, the goal will be to capture the visual essence of the scene and analyze the perceptually critical components. I call this Essence Photography and it may loosely resemble depiction of the world after high level vision processing. It will spawn new forms of visual artistic expression and communication.

In this talk, I will focus on Coded Photography. 'Less is more' in Coded Photography. By blocking light over time or space, we can preserve more details about the scene in the recorded single photograph.

1. Coded Exposure: By blocking light in time, by fluttering the shutter open and closed in a carefully chosen binary sequence, we can preserve high spatial frequencies of fast moving objects to support high quality motion deblurring. 2. Coded Aperture Optical Heterodyning: By blocking light near the sensor with a sinusoidal grating mask, we can record 4D light field on a 2D sensor. And by blocking light with a mask at the aperture, we can extend the depth of field and achieve full resolution digital refocussing. 3. Coded Illumination: By observing blocked light at silhouettes, a multi-flash camera can locate depth discontinuities in challenging scenes without depth recovery. 4. Coded Sensors: By sensing intensities with lateral inhibition, a 'Gradient Camera' can record large as well as subtle changes in intensity to recover a high-dynamic range image. 5. Coded Spectrum: By blocking parts of a 'rainbow', we can create cameras with digitally programmable wavelength profile.

I will show several applications and describe emerging techniques to recover scene parameters from coded photographs.

Recent joint work with Jack Tumblin, Amit Agrawal, Ashok Veeraraghavan and Ankit Mohan

Bio: Ramesh Raskar joined the Media Lab in spring 2008 as head of the Camera Culture research group. The group focuses on developing tools to help us capture and share the visual experience. This research involves developing novel cameras with unusual optical elements, programmable illumination, digital wavelength control, and femtosecond analysis of light transport, as well as tools to decompose pixels into perceptually meaningful components. Raskar's research also involves creating a universal platform for the sharing and consumption of visual media.

Raskar received his PhD from the University of North Carolina at Chapel Hill, where he introduced "Shader Lamps," a novel method for seamlessly merging synthetic elements into the real world using projector-camera based spatial augmented reality. In 2004, Raskar received the TR100 Award from Technology Review, which recognizes top young innovators under the age of 35, and in 2003, the Global Indus Technovator Award, instituted at MIT to recognize the top 20 Indian technology innovators worldwide. He holds 30 US patents and has received four Mitsubishi Electric Invention Awards. He is currently co-authoring a book on Computational Photography.

Abstract: Simulating the appearance of complex scenes faithfully and efficiently is a fundamental challenge in graphics. My research develops algorithms that scale to complex illumination and scenes, by exploiting limitations in human perception. In this talk I will describe two complementary research goals. First, we need to define image fidelity: when is a rendered image good enough? We introduce a new appearance-based measure of image fidelity called visual equivalence[SIG07,SIG08] that goes beyond pixel accuracy to capture what graphics practitioners care about: preserving appearance in complex scenes.

Second, we design perceptually-based algorithms that scale to complex scenes and complex illumination effects, like motion blur, depth-of-field, participating media, and subsurface scattering. I will describe our work on lightcuts for scalable final rendering [SIG05,SIG06,EG08] and matrix row-column sampling for lighting preview [SIG07,EGSR08]. These approaches are complementary in performance and can be applied across a range of applications such as cinematic relighting, production rendering, games, cultural heritage, and ecommerce, among others.

Bio: Kavita Bala is an Assistant Professor in the Computer Science Department and Program of Computer Graphics at Cornell University. She received her S.M. and Ph.D. from the Massachusetts Institute of Technology (MIT), and her B.Tech. from the Indian Institute of Technology (IIT, Bombay). Bala leads research projects in scalable rendering, perceptually-based rendering, interactive global illumination, parallelization, and texture synthesis. She has co-authored the graduate-level textbook "Advanced Global Illumination" (A K Peters publisher, second edition). In 2005 she co-chaired the Eurographics Symposium on Rendering (EGSR). Bala has received the NSF CAREER award, Cornell's College of Engineering James and Mary Tien Excellence in Teaching Award, and Cornell's Affinito-Stewart award.

Abstract: We investigate problems of reconstructing a 3D surface from various geometric constraints in a single image. The first part will deal with planarity constraints. Planarity is important for human perception and has applications in single view modeling and structured light. We propose a method for solving a network of intersecting planar curves based on a characterization of the trivial subspace of the system. We also demonstrate a combination of planarity with surface smoothness for parallel planar curves. The second part will deal with the integration of a smooth surface under constraints that have discrete ambiguities at the local level. Our work is a based on the mathematical formulation of Zhu and Shi for in/out reversal of surface patches in shape from shading. We show that a similar formulation applies to problems with ambiguous normals, e.g. shape from texture. These problems, which involve both continuous and discrete variables, can be transformed into entirely discrete optimization problems. Approximate solutions can be obtained from a semidefinite programming (SDP) relaxation. Finally, we apply several heuristics to improve the rounding phase of SDP embeddings.

Abstract: Light field photography is a technique for recording light intensity as a function of position and direction in a 3D scene. Unlike conventional photographs, light fields permit manipulation of viewpoint and focus after the imagery has been recorded. At Stanford we have built a number of devices for capturing light fields, including (1) an array of 128 synchronized video cameras, (2) a handheld camera in which a microlens array has been inserted between the main lens and sensor plane, and (3) a microscope in which a similar microlens array has been inserted at the intermediate image plane.

The third device permits us to capture light fields of microscopic biological (or industrial) objects in a single snapshot. Although diffraction limits the product of spatial and angular resolution in these light fields, we can nevertheless produce useful perspective flyarounds and 3D focal stacks from them. Since microscopes are inherently orthographic devices, perspective flyarounds represent a new way to look at microscopic specimens. Focal stacks are not new, but manual techniques for capturing them are time-consuming and hence not applicable to moving or light-sensitive specimens. Applying 3D deconvolution to these focal stacks, we can produce a set of cross sections, which can be visualized using volume rendering. Ours is the first technology (of which we are aware) that can produce volumetric models from a single photograph.

In this talk, I will describe a prototype light field microscope and show perspective views, focal stacks, and reconstructed volumes for a variety of biological specimens. I will also survey some promising directions for this technology. For example, by introducing a second microlens array and a video projector, we can control the light field arriving at a specimen as well as the light field leaving it. Potential applications of this idea include microscope scatterometry - measuring reflectance as a function of incident and reflected angle, and "designer illumination" - illuminating one part of a microscopic object sswhile avoiding illuminating another.

Biographical sketch:

Marc Levoy is a Professor of Computer Science and (jointly) Electrical Engineering at Stanford University. He received a Bachelor's and Master's in Architecture from Cornell University in 1976 and 1978, and a PhD in Computer Science from the University of North Carolina at Chapel Hill in 1989. In the 1970's he worked on computer animation, developing an early computer-assisted cartoon animation system. This system was used by Hanna-Barbera Productions from 1983 until 1996 to produce The Flintstones, Scooby Doo, and other shows. In the 1980's he worked on volume rendering, a family of techniques for displaying sampled three-dimensional functions, for example computed tomography (CT) or magnetic resonance (MR) data. In the 1990's he worked on technology and algorithms for digitizing three-dimensional objects. This led to the Digital Michelangelo Project, in which he and a team of researchers spent a year in Italy digitizing the statues of Michelangelo using laser scanners. His current interests include light field sensing and display, computational photography, and applications of computer graphics in microscopy and biology. Awards: Charles Goodwin Sands Medal for best undergraduate thesis (1976), National Science Foundation Presidential Young Investigator (1991), ACM SIGGRAPH Computer Graphics Achievement Award (1996), ACM Fellow (2007). Recent professional service: Papers Chair of SIGGRAPH 2007.

Send questions or comments about this page to
Page last modified on Wednesday, September 17, 2008