CSC320 Visual Computing Assignment 3 Development Notes

Development Notes

1. Write class SChannels. SChannels holds array/arrays of shorts. Each of R,G,B has its own array (you may want to allow an arbitrary number of channels). Each pixel has 3 shorts associated with it, one for R, one for G, one for B. Note: It is not yet clear if this is the best structure for the solution. In the end I may swap this out for a better approach.
2. Add code to ImagePanel so that it can return one of these SChannels for its associated image.
   • The BufferedImage in the ImagePanel is of type ARGB. This means that if you get a pixel bits 0-7 are B bits 8-15 are G bits 16-23 are R bits 24-32 are A Need to understand the bit shift operators, for this and for the inverse operation. Of course, could have also used multiply/divide by powers of 2.
   • I also wrote code to ask an imagePanel to return the largest 2ⁱ by 2ⁱ square the image would support. Later maybe I will allow the user to slide such a square over the image to specify the part of the image they want to play with.
3. Write code to turn an SChannel into an image. Make sure that both the code to grab an SChannel and to create an image from an SChannel works. Try it out on an image.
   • Carefull here, when turning an SChannel back into an image, the BufferedImage is of type ARGB, that is, a single int codes ARGB as described above. You must set the alpha component. Now I did not save the alpha component in my SChannel, so I have to explicitly put it back in this method.
   • I found it usefull to have a test button on my JFrame so that I could execute some code on demand. Here is the source for the button (inbetween old AppFrame code) getContentPane().add(imagePanel, BorderLayout.CENTER); // New code -------------------------------------------- JPanel p=new JPanel(); JButton b=new JButton("Test"); p.add(b); getContentPane().add(p, BorderLayout.WEST); b.addActionListener(new ActionListener(){ public void actionPerformed(ActionEvent e){ SChannels c=imagePanel.getSChannelsLargestSquare(); imagePanel.setImage(c); pack(); // resize the window for the new image } }); // End new code ---------------------------------------- fileChooser = new JFileChooser();
   • Tested all of this on the Hulk image and on the mona lisa image.
     
4. Either create a Haar utility class or give the SChannels the ability to compute the transform themselves. I think I will try this with an external Haar utility class. The transform function will take a 2-d array of shorts (essentially a single channel) and return a new 2-d array of shorts.
5. Code the Haar transform as specified above. This will be recursive code. The hint is that it should work on a specified sub array. Modify it so that it works on a SChannel.
   • This can be implemented either with or without juggling pixels. If you do not juggle pixels, you can save some space and time. The problem with not juggling pixels is that the result is hard to read and manipulate. I chose to juggle pixels. That is, the results are as was described in the class notes. I reimplemented the transform (simplified the code I used to create the week 4 tutorial).
   • The simplest way was to declare a temporary array and then compute the transform at the current level into the temporary array. I copy the temporary array back to the right part of the original array and then recursively compute the Haar transform on the first 1/4 of the array.
   • I tested this...see inverse below.
6. Write code to invert a Haar transform, test both to see if they work. Try to pick data so that the transform and it's inverse can be comparable (the same).
   • An issue with the inverse, roundoff error creeps in, this may cause inverted pixels to be outside the normal range of pixel values. When computing the inverse of the transform, I needed to fix the pixels in the currently computed average image. That is, make sure that if it is below 0, then I set it to 0, if it is above 255, then I set it to 255.
   • Tested Haar transform and inverse on the week 4 tutorial example my Haar.java main... public static void main(String [] args){ short [][] a={ {16, 12, 4, 8}, {8, 12, 4, 16}, {4, 16, 8, 12}, {12, 8, 16, 4} }; System.out.println("Original"); printArray(a); transform(a.length,a); System.out.println("Transform"); printArray(a); inverse(a.length,a); System.out.println("Inverse"); printArray(a); } and the output Original 16 12 4 8 8 12 4 16 4 16 8 12 12 8 16 4 Transform 10 1 0 -4 0 1 -2 2 2 -2 2 2 0 0 -4 -4 Inverse 16 12 4 8 8 12 4 16 4 16 8 12 12 8 16 4
   • Also tried this on some images. To do this, I added the following to my ImagePanel class public void haarTileImage(){ SChannels c=getSChannelsLargestSquare(); System.out.println("sChannels"); System.out.println(c); Haar.transform(c); System.out.println("Transform"); System.out.println(c); Haar.inverse(c); System.out.println("Inverse"); System.out.println(c); setImage(c); } Now since looking at the output on large images is useless, I created a smaller image and ran the transform on it. I am using and the even smaller image .
     The results of executing my code on is in 6.txt.
   • To Test all this code, I modified my AppFrame, you can see the code below... public AppFrame(String title){ super(title); setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); setJMenuBar(createMenuBar()); imagePanel=new ImagePanel(); getContentPane().add(imagePanel, BorderLayout.CENTER); // Testing panel JPanel p=new JPanel(); JButton b=new JButton("Channels Transform"); b.addActionListener(new ActionListener(){ public void actionPerformed(ActionEvent e){ SChannels c=imagePanel.getSChannelsLargestSquare(); imagePanel.setImage(c); pack(); // resize the window for the new image } }); p.add(b); b=new JButton("Haar transform"); b.addActionListener(new ActionListener(){ public void actionPerformed(ActionEvent e){ imagePanel.haarTileImage(); pack(); // resize the window for the new image } }); p.add(b); getContentPane().add(p, BorderLayout.NORTH); // End Testing panel fileChooser = new JFileChooser(); pack(); }
7. Write code to zero small (in absolute value) detail coefficients. Test this both on specified arrays and on input images. What is the effect? What did you expect the effect to be?
   • Note: Zeroing takes place on the Haar transformed image.
   • One must be carefull here, this is essentially image compression. The question is, when should a detail coefficient be zeroed? Well, there is not really one right answer for this, but the following is the approach I took... Actually, a set of 3 detail coefficients will be zeroed or not. Consider a set of 4 values a b c d with a, the average coefficient and b, c, d the corresponding details. I zero b,c,d provided that (Math.abs(b)+Math.abs(c)+Math.abs(d))<threshold The problem is that, if we do this only on a single channel, then the color will change. So this above notion needs to be modified to take into account 'little change' across all channels (r,g,b).
   • I added the public static void detailThreshold(SChannels c, int threshold){ method to my Haar class. It was actually tricky to get this correct. To start, I suggest you work on applying the detailThreshold only to a set of details at a particular resolution. You will not need to generate the the average pixels for the specified resolution, just find all the detail pixels at a particular resolution. Next apply your method to all resolutions (each a power of 2). For a quick demo, I modified my haarTile method of ImagePanel as follows: public void haarTileImage(){ SChannels c=getSChannelsLargestSquare(); // System.out.println("sChannels"); // System.out.println(c); Haar.transform(c); // System.out.println("Transform"); // System.out.println(c); Haar.detailThreshold(c,50); // System.out.println("Detail Threshold"); // System.out.println(c); Haar.inverse(c); // System.out.println("Inverse"); // System.out.println(c); setImage(c); } I applied this to mona lisa and the result is .
   • This is starting to have the right effect. The problem is that there are lots of small details. In the end, we will want tiles of a specified range of sizes.
   • Note: The Haar transform of this image has lots of 0 detail coefficients. This means that we can efficiently store this image (represent the repeated 0 coefficients in some nice way).
8. Come up with a few options regarding adding the 'realistic effect' to your existing application.
   • I will have to come back to this...
   • Had to figure out how to paste an image into a BufferedImage (the bimg held by the Image Panel). I added the following to ImagePanel as proof of concept... public void pasteImage(){ BufferedImage b=ImageLoader.loadImage("greenFrogPart.gif"); Graphics2D g=bimg.createGraphics(); g.drawImage(b,null, 10,20); } I created a new ImageLoader class (since it seems that I will be doing a lot of this...) import javax.swing.*; import java.awt.*; import java.awt.image.*; public class ImageLoader extends JPanel { private static ImageLoader il=new ImageLoader(); /** * @param fileName the name of a file containing an image JPEG, GIF etc. * @return a BufferedImage holding the corresponding image, null if unable to load the image */ public static BufferedImage loadImage(String fileName){ return il.load(fileName); } /** * @param fileName the name of a file containing an image JPEG, GIF etc. * @return a BufferedImage holding the corresponding image, null if unable to load the image */ private BufferedImage load(String fileName){ // Algorithm: Load the image into an image icon // create an appropriate BufferedImage (same size as the given image) // paint the image icon into the buffered image ImageIcon imgi = new ImageIcon(fileName); Image img = imgi.getImage(); int width = img.getWidth(this); int height = img.getHeight(this); if(width==-1 || height==-1) return null; BufferedImage bimage = new BufferedImage(width, height,BufferedImage.TYPE_INT_ARGB); Graphics2D bimageContext = bimage.createGraphics(); bimageContext.drawImage(img, 0, 0, null); return bimage; } } Also, I rewrote the ImagePanel method below to take advantage of the new ImageLoader public boolean setImage(String fileName){ setImage(ImageLoader.loadImage(fileName)); return true; }
9. Modify your Haar transform and inverse so that it can be used to tile a picture with a given range of tile sizes. Look only at a certain part of the transform, you will get what you want. May need to write a method which takes an image in one resolution and produces an image of another resolution from it. Of course, you can do this strictly inside short arrays.
   • I modified the detailThreshold method so that it will zero all details at sufficiently high resolution. This has the effect of limiting the smallest tile in the image. The result of this effect can be seen in . Is this the approach I will use in general? I have to think about it a bit.
10. If you have any interesting pictures, I would like to get a copy to put up a gallery for other students to see. Please let me know if you want me to leave your name off.
11. You are on your own now!!