CSC320 Visual Computing Assignment 4

Announcements

• Please read the Questions and Answers Section at the bottom!!
• I have posted some starter code in the QandA

Requirements

Assignment #4 will introduce you to a fun and widely-used application of visual computing: creating a video in which the photo of an object or person is morphed into another photo of the (same or different) object or person. It closely follows an assignment given at a CMU computer graphics course. As in that assignment, your goal will be to produce a "morph" animation of your face into another student's face. Each of you will generate one second of animation, which we will put onto a video tape. The final video will be a metamorphosis through each of the faces in the class. Here is the video that the CMU students produced. Here is your chance to create your own special effects by morphing your portrait into that of another person or object!

• Part 0.A: Find a partner for the project You are supposed to work on this assignment in pairs. You should find a partner as soon as possible and discuss how you should divide up the work.
• Part 0.B: Create the directories for your new assignment. Download and unpack the tarfile assign4.tar.gz > cd ~/CS320 > tar xvfz assign4.tar.gz All the code that you turn in should be under the assign4 directory, as specified in the details below.
• Part 1: Getting familiar with the Field Warping tool

  The main component of the morphing tool is an image warping algorithm called Field Warping. Included in the tarfile is a Windows application called WinWarp that allows you to warp images using this algorithm. Here is what you can do to warp an image:

  • Run the Win32 application (on Windows) ~/CS320/assign4/field-warp/WinWarp
  • Select Image->Open Image... from the menu and open an image from the directory ~/CS320/assign4/images/
    Note that the program only opens images in the Targe format (extension .tga). The images in the images/ directory are supplied in 4 formats---Targa, Tiff, GIF and Vista so that you can use them with both WinWarp and with your A4 code.
  • Two windows will open up, one called Source and one called Destination.
  • Click and drag the right mouse button in the Source image. This will create a directed line segment whose direction is indicated by an arrow. After releasing the button, a directed line segment also appears in the destination image.
  • By clicking and dragging the right mouse button again, you can create additional line segments
  • Now left-click at the endpoint of a line segment in the destination image to drag the endpoint.
  • To compute the field warp, select Image->Preview from the menu or Image->Warp for a higher-quality rendering of the warped image.
  • You can also control whether or not antialiasing is used by choosing Options->Anti-Alias from the menu.
• Part 2: Getting familiar with the details of the method

  You will be implementing the method as described in the paper by Beier & Neely (in html , in pdf) which was covered in class and tutorial.

  Read through the paper to understand the overall method and the context. As in Assignments #2 and #3, an important skill that you will need to develop while reading a research paper such as this one is to focus only on those parts of the paper that you must fully understand to complete the assignment---you can ignore parts of the paper that require background knowledge you don't have, or you can ask me or the TAs to explain those parts of the paper in more detail.

  • Read Sections 1 & 2 for background and motivation behind the work
  • The Sections you are asked to fully understand and implement are Sections 3.2 and 3.3 (except for the 'Ghost-busting' sub-section of 3.3).
• Part 3: The Beier-Neely Algorithm (70 points)

  The heart of your project will be to implement the Beier-Neely field warping algorithm, which takes as input two source images, S1 and S2, a set of corresponding lines in these images, and a warp parameter warpfraction, and creates a new image D that corresponds to an intermediate morph between the two images according to the t parameter (i.e. D=S1 if warpfraction=0 and D=S2 if warpfraction=1).

  Your implementation should follow closely the description in the paper. It is suggested that you begin with the simplest version of the algorithm and add features/capabilities as you go along. In particular, you are asked to implement the following:

  • Step 1--Field warping from a single line pair (25 points): Given a line in S1 and a corresponding line in S2, warp the image S1 so that the pixels on the specified line in S1 get mapped to the line in S2. You should warp the image using the Backward-Mapping warp algorithm outlined in the paper and described in class.
    
  • Step 2--Field warping from multiple line pairs (15 points): Extend the algorithm to allow image warps specified by multiple line pairs.
  • Step 3--Creating in-between field warps from two images (10 points): Create intermediate warps of the two source images from a collection of corresponding line pairs and a warpfraction value as follows:

    a. Let PQi1 be the i-th line in S1 and PQi2 be the i-th line in S2.

    b. Compute the position of the line, PQiD, in the destination image by linearly interpolating PQi1 and PQi2 according to warpfraction.

    c. Warp S1 using the field warping algorithm and the line pairs (PQi1,PQiD)

    d. Warp S2 using the field warping algorithm and the line pairs (PQi2,PQiD)

  • Step 4--Creating the intermediate morphs by adding a cross-dissolve step (5 points): Cross-dissolve the warped images computed in steps 3c and 3d to get the intermediate morph. Use the warpfraction parameter to blend the pixel colors of the two warped images.
  • Step 5--Anti-aliased field warping (15 points): Add an anti-aliasing capability to your morphing algorithm so that no aliasing artifacts or jaggies appear in the destination image. You are free to choose a specific anti-aliasing method, among those discussed in class.

  Note that Beier & Neely's paper has a typo on p.37 (2nd column) in the pseudo-code for the multiple line algorithm: the displacement calculation should be Di=Xi'-X, (not Di=Xi'-Xi).

• Part 4: Extending your code to enable unattended morphing (10 points)

  From the command line, you provide two images, two .cmd files that contain a list of line segments, and any additional parameters you may want to use to control morphing (e.g., number of frames to generate, parameters that control anti-aliasing, etc). Here is an example of what the command line invocation should look like (run from within the assign4 directory):

  > java Morph -mS1 images/fabio2.gif -mS2 images/fernanda.gif -msrcl lines1.cmd -mdestl lines2.cmd -frames 10 -mD f
  • fabio2.gif is the input source image; this is an input parameter
  • fernanda.gif is the input destination image; this is also an input parameter; these two parameters tell the program that it should morph fabio2.gif into fernanda.gif
  • lines1.cmd is the file containing a list of line segments over fabio2.gif that will be used to control the field morph, as in the supplied WinWarp field morphing application.
  • lines2.cmd is the file containing a list of line segments over fernanda.gif that will be used to control the field morph, as in the supplied WinWarp field morphing application. IMPORTANT: the number of lines in the files lines1.cmd and lines2.cmd should be the same.
  • 10 is the number of images to be generated by your morphing program.
  • f is the "base" filename of the images to be genered by your morphing program.

  When invoked in this way, the program should generate the following 10 files:

  f.00.png (this image is identical to fabio2.gif) f.01.png ... f.08.png (the above images correspond to the intermediate morphs) f.09.png (this image is identical to fernanda.gif)

  To add these command-line options, you will need to write code to process command line arguments. There are libraries around to process command line arguments.

  IMPORTANT: Do not deviate from these specs in any way, as we will use the command line to test your program.

• Part 5: Extending your interface to enable morphing (10 points)

  Your task is to create a program called "Morph.java" that is run from the command line and that has the following general functions:

  • When the program is invoked, it should bring up a GUI that will be your main interface with the Morph program.
  • At the very minumum, the interface should include two panels in which all drawing operations will be performed, and which should display the end-results of the morphing algorithm. These may be brought up inside the same JFrame or in different JFrames.
  • The interface should provide (at a minimum) the following functions:
    • Selecting two "source images", i.e., color image files (jpg, gif, png etc.) that will be morphed into each other. Use the JFileChooser as was done in a3. Once the images are selected, they should be displayed in the panels. The interface should appear similar to this picture:

      

    • Buttons, menus, or dialog boxes that control the parameters of the morph algorithm. At a very minimum, the interface should allow one to enable/disable anti-aliasing and to set the number of intermediate morphs.
    • An interface for drawing lines in the source and destination images and for interactively re-positioning their endpoints. More specifically, a user should be able to use the mouse button to draw lines in either of the two images. Once the line is specified in one image, it should be immediately drawn in the other image as well, as in the WinWarp interface:

      

    • The interface should allow for the lines' endpoints to be interactively re-positioned in either of the two images to better fit the image's contours. In the example below, the line endpoints in the right image are re-positioned to better fit the subject's mouth:

      

    • Be prepared to draw at least 10-30 line pairs between the images to get good morphs. This means that your line drawing/re-positioning interface should not be very slow! You can reduce the size of your window to speed things up:

      

    • Your interface should allow saving of the line pairs into two .cmd files (see assignment 1), one for each image, that you can re-load later on.
    • Once the line pairs have been specified interactively or loaded from a lines file, the interface should allow the user to specify a single value for the warpfraction parameter.
    • With the warpfraction parameter defined, the user should be able to press a "Warp" button (or to select an option from a menu) in order to compute a single intermediate morph that corresponds to that parameter.
    • When operating in this mode, your interface should allow a user to view the results of each stage of the process. Users should be able to display both warped images as well as the final result. Note: The idea here is different from the one discussed in class, that is, each image is separately warped (to obtain two complete warped images). These are then cross dissolved to get the final image.
• Part 6: Creating your portion of the class video (10 points)

  We will set up a time (right before or right after a tutorial) when you will all have your picture taken in a "standardized" fashion. When your picture is taken, you will be assigned a number---you should use your morphing application to create a 1-second transition from your portrait to the portrait of the student with the next-highest number (last student gets to morph with student #1). Note that one second of video is equivalent to 30 individual frames.

  It is important to note that the visual quality of the resulting morph is highly dependent on the placement of the lines that control the morph. You will get between 0 and 10 points depending on the visual quality of the morph you generate. Be sure to use line segments to delineate the nose, eyes, lips, hairline, chin and overall silhouette of in each of the two portrait photos.

  Since this part of the assignment requires generating a total of 30 images for the video, the portraits we will use should have a fixed resolution 640x480 pixels. The images you generate should have this resolution as well.

  You should follow the procedure below when generating your images. Failure to do so will cause loss of points for this part of the assignment:

  • At time of picture taking you will be assigned a two-digit ID, XX. Suppose also that YY=XX+1 (or YY=00 if you are the last student in the list).
  • Once captured, your picture will be made available on this website (check back here later).
  • You should use your interface in the interactive mode in order to decide where to place the lines in the two source portraits (XX.v and YY.v). Once you are satisfied that the lines give you a visually pleasing result (e.g., the intermediate morphs look like real faces, not as two overlapping portraits of different people), save them in the files ~/CS320/assign4/images/lines.XX.cmd (for the lines over image XX.v) ~/CS320/assign4/images/lines.YY.cmd (for the lines over image YY.v)
  • To create your portion of the video execute the following commands:
    > mv ~/CS320/assign4/00 ~/CS320/assign4/XX > cp DIRNAME/portrait.XX.jpg ~/CS320/assign4/images/XX.jpg > cp DIRNAME/portrait.YY.jpg ~/CS320/assign4/images/YY.jpg > cd ~/CS320/assign4/ > java Morph -mS1 images/XX.jpg -mS2 images/YY.jpg -msrcl images/lines.XX.cmd -mdestl images/lines.YY.cmd -frames 30 -mD XX/morph
  • This should create 30 images:
    ~/CS320/assign4/XX/morph.00.png ~/CS320/assign4/XX/morph.01.png .... ~/CS320/assign4/XX/morph.29.png

  If you are working with a partner, you should perform the above procedure once for EACH partner. This means that if your partner's id is ZZ you must have two directories called ~/CS320/assign4/XX and ~/CS320/assign4/ZZ with the first containing the 30 frames for the morph from XX to YY and the second directory containing the 30 images for the morph from ZZ to (ZZ+1).

• Part 7: Testing your code

  Among others, we will be using the images in the directory ~/CS320/assign4/images to test your implementation. The images are provided in .gif format for use in your code as well as in Targe (.tga) format for use with WinWarp and in Tiff (.tif) format for image viewing with gimp or other tools:

  • The image grid.gif contains a grid that you can use to test your warping code. Just load the image and a lines pair file that you created and apply your field warping code on it---the warped image you obtain will show how the pixels in the original image are re-positioned by your warping implementation.
  • The images checker1.{v,tif,gif} and checker2.{v,tif,gif} contain checkerboard patterns that we will use to see whether your code creates images that have aliasing artifacts.
  • The file also contains two face images that you can use to test your implementation.
• Part 8: What to turn in
  • In directory ~/CS320/assign4, place a file called README.txt that briefly describes the components/steps you implemented, how they differ from the Baier/Neely implementation (if any), what features you omitted/added (if you have implemented additional functionalities that merit more discussion, you can expand your writeup).
  • The thirty images you generated with your morphing code, in the appropriate directory, as described in Part 6 above.
  • The lines file for that morph, placed in the appropriate directory.
  • IMPORTANT: make sure that you use the naming conventions described in Part 6 for your images, as we will use a script to combine images into a single class video.
  • All your source code and an executable version of your program, called Morph.java. Placed in the ~/CS320/assign4/ directory.
• Part 9: Packing Everything Up and Turning It In

  Once you are done with the above, pack up all your code with the following commands:

  > cd ~/CS320 > tar cvfz assign4.tar.gz assign4

  To unpack your code, we will follow the procedure outlined above, i.e., we will run 'tar xvfz x.tar.gz' at the unix prompt, rm *.class, javac *.java Java Morph with and without command line arguments. I cannot stress the importance of making sure that your code compiles and runs correctly when unpacked! If it does not, we will have no way of knowing whether your code was ever correct!!!

  Finally, you should use the web based submit to submit your assignment:

  Note that the system has been configured (1) to accept only files whose name is assign4.tar.gz and (2)  to not accept submissions that are more than 4 days late.

Documentation

References

• The Java tutorial, Java2D api.
• Java2d tutorial
• Image observing without a UI
• Some image processing applets

Questions and Answers

Question:

How do I write an image?

Answer:

Here is a small stub that contains all the ideas... import java.awt.image.*; import javax.imageio.ImageIO; import java.io.File; import java.io.IOException; public void save(){ File file = new File("hulkOut.png"); try { ImageIO.write(dest.getImagePanel().getImage(), "png" , file); } catch (IOException e) { e.printStackTrace(); } }

Question:

Can I have some starter code?

Answer:

OK, here is MorphGUI.java, code which generates the following GUI... You will still need to add code to load images into s1 and s2 as well as hookup event handlers and figure out how to gather parameters from the forms. There are two basic approaches:

1. Fire events each time a user manipulates a parameter. Save the appropriate parameter in an object. When the user wants to compute something, just pass the object.
2. When the user actually wants to compute something, gather all parameters from all controls and then compute it.

Question:

Where are the class pictures?

Answer:

here

Question:

What does the command file look like?

Answer:

As in assignment 1, except we really only care about lines. You should have a list of lines for each image. For Morph to work properly, there should be the same number in each cmd file. D. Save all objects currently-displayed on the GL canvas, along with their colors, to a .cmd file (10 points) File specifications: - One line per object on the screen - Lines should strictly adhere to the following format: first character on each line is the type of object, followed by a space; the remainder of the line is a space-separated sequence of integers - For polygon objects, the format should be p <N> <row1> <col1> ... <rowN> <colN> where N is the number of polygon vertices - For circle objects: c <center-row> <center-col> <radius> - For rectangles: r <row1> <col1> <row2> <col2> - For lines: l <row1> <col1> <row2> <col2> - For a point: p <row> <col> - Each drawing command may be preceded by a line: C <r> <g> <b> indicating that the drawing command(s) that follow used <r> <g> <b> as their drawing color

Question:

What about the numberws we are supposed to be assigned for naming of files?

Answer:

I have posted numbers on the set of class pictures. The final set will be up shortly.

Question:

The submit link was not working!!

Answer:

It is now.

Question:

I could not submit...

Answer:

I (Arnold) ran out of quota. I have cleaned up my space a bit. Tomorrow, I will ask Sue to increase my quota so that the remaining submissions can come in. Please do not panic if you can not submit until sometime tomorrow morning. Try submitting tomorrow.

Question:

What if I (the student) run out of quota when trying to submit?

Answer:

When logged into a linux box, you can move everything to /tmp and create your piece of the movie and tar up and submit from there. Please remember to remove your files from /tmp otherwise someone else may be tempted to copy your work.

Question:

How about that submit problem...

Answer:

It has been fixed, please see the note by the submit link above. Here is what I have in the submits so far. Please check your file sizes credit.erin> date; ls -l */*/* Wed Apr 7 20:01:35 EDT 2004 -rw------- 1 arnold 32812484 Apr 5 01:15 Atwi_MajedIsmail/a4/assign4.tar.gz -rw------- 1 arnold 0 Apr 5 02:03 Aziz_Maroquine/a4/assign4.tar.gz -rw------- 1 arnold 31916764 Apr 5 13:17 Balaura_AdrianAlexander/a4/assign4.tar.gz -rw------- 1 arnold 14965924 Apr 5 08:13 Banh_Jason/a4/assign4.tar.gz -rw------- 1 arnold 14844197 Apr 6 16:33 Butt_OmairNawaz/a4/assign4.tar.gz -rw------- 1 arnold 23953408 Apr 5 08:03 Byrd_NelvinAndrew/a4/assign4.tar.gz -rw------- 1 arnold 13728232 Apr 5 17:26 Chow_AndyYau/a4/assign4.tar.gz -rw------- 1 arnold 1383247 Apr 5 06:13 Groff_Blair/a4/assign4.tar.gz -rw------- 1 arnold 28365311 Apr 5 14:13 Gueorguiev_StefanGueorguiev/a4/assign4.tar.gz -rw------- 1 arnold 12619088 Apr 5 12:26 Huynh_JohnQuocChuong/a4/assign4.tar.gz -rw------- 1 arnold 22948590 Apr 5 16:29 Kostalas_Tasso/a4/assign4.tar.gz -rw------- 1 arnold 0 Apr 5 08:50 Krolo_Ivan/a4/assign4.tar.gz -rw------- 1 arnold 22610555 Apr 5 07:45 Manuh_Chris/a4/assign4.tar.gz -rw------- 1 arnold 31942162 Apr 5 00:48 McCash_Drew/a4/assign4.tar.gz -rw------- 1 arnold 27473303 Apr 5 05:25 McCrae_JamesPalmer/a4/assign4.tar.gz -rw------- 1 arnold 27060413 Apr 5 13:45 Perrier_Annie/a4/assign4.tar.gz -rw------- 1 arnold 191 Apr 5 00:26 Student_Joe/a4/assign4.tar.gz -rw------- 1 arnold 16567096 Apr 5 05:33 Tathgar_Gagandeep/a4/assign4.tar.gz -rw------- 1 arnold 31568233 Apr 5 07:37 Tran_Ramy/a4/assign4.tar.gz credit.erin>