In this assignment you will implement and experiment with an image blending technique based on the Pyramid-Based Blending algorithm to be discussed in class.
The goal of this assignment is four-fold:
- to understand their construction using 1D convolutions
- to appreciate their usefulness for image editing & manipulation, by decomposing images at multiple levels of detail
While the due date for the assignment is 2 weeks away, it is strongly advised that you read the paper and go through the supplied code in the next 2-3 days, and then begin coding as soon as possible. As usual, plan to spend some time looking at the code already supplied (methods, classes, etc), as you will have to depend on it for your own implementation.
Pyramid-based image blending was introduced by Burt & Adelson in 1983 as an effective way to create seamless mosaics. Most recently, a variation of this technique has been used in AutoStitch, a tool for automatically stitching a set of photos into a large image panorama. Pyramid image blending works by blending the Laplacian pyramids of two input photos:
using the Gaussian pyramid of a "mask" image as the alpha matte:
The result of this blend is a new Laplacian pyramid from which we can reconstruct a full-resolution, blended version of the input photos. In the example above, the blended photo is impossible to capture with a traditional camera in one shot, as it has two objects in focus, one on the foreground and one on the background, but not the objects in between:
Since the actual blending step is trivial (it's just standard alpha matting) the most important computations are (1) computation of the Laplacian pyramid of an image and (2) reconstruction of the Gaussian pyramid of an image given its Laplacian pyramid. These computations, in turn, involve two basic functions: the REDUCE() and the EXPAND() function discussed (or to be discussed) in class. These two functions are mainly what you have to implement for this part of the assignment. You need to read and understand Burt and Adelson's original paper on the Laplacian and Gaussian pyramids. You should read up to, but not including, the section entitled "Entropy". The actual blending algorithm was introduced in a follow-up paper on image mosaics. The only section from that paper you should look at is a very brief Section 3.2 (Splining regions of arbitrary shape), which lists the 3 steps of the blending algorithm. For your purposes, you can ignore the rest of that paper.
As usual, I am providing you with a full implementation of the blending tool in executable form (see the menu options under "Blending"), along with a suite of test images, so that you can run it yourself and see how it works (see the executables partA/bin/VisComp_full.exe and partA/bin/viscomp_full in the assignment zipfile). The program should work for a variety of image formats (tif, jpeg, etc), and the binary masks you can use are identical to those you used for A2 (eg. they can be in .bmp format). In addition to the basic code, I am providing you with a "packing" routine in the source code that allows you to visualize a photo's entire pyramid in a single image for debugging purposes, as in the examples above.
The supplied program can be run in interactive mode (ie. UI windows are displayed and user works through menus and/or keyboard accelerators), in batch mode (ie. user supplies a set of command-line options specifying the input and output images names) or both (ie. some or all command-line options are specified and, after images are loaded, the UI windows are displayed as well). Type 'viscomp -help' to see the command line options.
I suggest you use the above order in tackling these tasks. Specifically:
The helper code is packaged into the tarfile blend.tar.gz. The following sequence of commands will add files to your existing CS320 directory under your home directory on CDF and will unpack the code:
> cd ~
> tar xvfz blend.tar.gz
> rm blend.tar.gz
This will create the directory ~/CS320/Assignments/Blend along with files and subdirectories needed for the assignment. All the code that you turn in should be in those directories as well, exactly as specified in the details below. The relevant pieces of the code are in the directory src/pyramid.
The goal of this part of the assignment is to implement the three core routines for Gauss/Laplacian pyramid construction (the pyramid.reduce(), pyramid.expand() methods) as well as the pyramid::blend() routine for image blending. The first three routines take a single image as input and they assume that the image is square and has 2^N+1 rows/columns. The supplied source code pads images that do not conform to this constraint with zeros, so you can safely assume that any image passed to these routines does have these dimensions. Overall, there is much less code to go over, compared to Assignment 2, since the technique itself is very compact and has few steps.
to blend two images:
viscomp -no_gui -blending -bsource0 <Source0> -bsource1 <Source1> -bmask <Mask> -bblend <Blended>
to save the gaussian pyramid and/or laplacian pyramids of these image, supply the additional options:
-bgpyr0 -bgpyr1 -bgpyrm -bgpyrb
to output the gaussian pyramids of source0, source1, mask and blended image, respectively, and
-blpyr0 -blpyr1 -blpyrb
to save their laplacian pyramids of source0, source1, and the blended image.
See PartB/README.txt for details.
Once you are done with the above, edit the file 320/Assignments/Blend/CHECKLIST.txt (also available here) to specify which components of the assignment you have completed, along with notes about parts that you were not able to complete, if any.
Pack up your portion of the code with the following commands:
> cd ~/CS320/Assignments
> tar cvfz assign3.tar.gz Blend/CHECKLIST Blend/partB/{WRITTEN,*.jpg} Blend/partA/bin/viscomp Blend/partA/src/{Makefile,ADDITIONS} Blend/partA/src/pyramid Blend/partA/src/morphing Blend/partC
Finally, you should use CDF's assignment submission system to submit your assignment:
> submit -c csc320h -a Assign3 assign3.tar.gz
Note that the system has been configured (1) to accept only files whose name is assign3.tar.gz and (2) to not accept submissions that are more than 4 days late. Just do 'man submit' at the unix prompt to get more info on this process.
In evaluating your assignment, the first thing we will look for are the files CHECKLIST and WRITTEN.
Then we will make sure that your code compiles (just by typing "make" in the partA/src directory).
Then we will run your code on test examples in both the interactive and the non-interactive modes.
Finally, we will look at your code. It must be well commented: if the TA has doubts about the correctness of a specific implementational feature and there is not enough documentation/comments for the TA to decide, you will lose points.