Readings

• On the course Web page look at:
  • Unofficial exam time
    Arts and Science Examination TimeTable
• In the text read
  • Chapter 12, 16

Chapter 13

List searching: linear vs binary search & efficiency of searching.

Consider a list of objects, stored in an array

• with a key field and
• some way of ordering the list using the key field. i.e. some method to compare relative position.
• Let n be the number of elements in a list.

Sorted versus Unsorted lists

• an unsorted list forces linear searching. This is
  okay as long as n isn't too large.
  But what if n > 2¹⁰⁰
• a sorted list allows us to exploit a binary search

Linear search for key

Starting at the first element, look at each
element in the list until key found or end of list.

5	2	6	1	3	4	7	value
0	1	2	3	4	5	6	index

boolean found = false;
int i;
for (i=0;i<n;i++) {
 if (key == list[i].key()) {found=true;break;}
}

Binary search for key

Starting in the middle of the list, compare with key if the key is not greater then discard the second half otherwise discard the first half continue until key is found or less than 2
elements are left.

							# comparisions	# elements
			.				1	1
	.				.		2	2
.		.		.		.	3	4
1	2	3	4	5	6	7	value	7 total
0	1	2	3	4	5	6	index

  boolean found = false;
  boolean done = false;
  int first = 0;
  int last = n;
  int mid;
  while (! found and ! done) {
    mid = (first + last)/2;
    if (key == list[mid].key()) {found=true;}
    else if (first > last) {mid=-1;done=true;}
    else if (key < list[mid].key()) {
       last=mid-1;}
    else {
       first=mid+1;}}

Counting comparisons

Assume each key is equally likely
to be searched for

                Best  Average       Worst

 linear         1    (n+1)/2          n

 binary         1    |log2(n)|  |log2(n)+1|
                     `-     -'  `-       -'

BIG-O notation & asymptotic behaviour

We want a way of comparing algorithmic efficiency which gives an upper bound function depending on the ``size'' of the input. Given a timing function
f: N+ -> N+,
BIG-O measures the worst case performance of an algorithm. i.e. searching for the last element in an unsorted list is O(n).

Since some computers are faster than others, we need a way of measuring the relative efficiency of algorithms that is independent of the computer that it is run on.

Since different statements within a program take different amounts of time to execute, We also want to ignore how long it takes for a statement to be executed.

BIG-O notation is a measure of how fast a function grows when n gets large.

In particular, when comparing searching algorithms we want to know how much time algorithms will take when n get large.
Suppose we want to compare two algorithms which run in time f(n) and g(n) respectively. Consider their plots.

|         y = f(n) /   _-  y = g(n)
|                 /  _-
|                / _-
|               /_-
|             _/-
|           _-/|
|         _- / |
|       _-  /  |
|     _-   /   |
`--------------|-------------------->
              n0                   n

f(n) grows faster than g(n) and at some point n0 is larger than g for all larger n.

Algorithmic Complexity

For some function f, we say a searching algorithm runs in O(f(n)) time if the worst case behaviour doesn't take more than C*f(n) seconds to perform for large enough n, where C is a constant that is independent of n. So O(f(n)) can be thought to be a measurement of performance, independent of the amount of time it takes to execute any one statement on the computer. More formally we say

Definition:

We say g(n) is in O(f(n)) if there are constants C and n0 so that g(n) <= C f(n) for all n > n0.

O(f(n)) is a family of functions with representative worst complexity f(n).

Algorithmic Complexity

Let f : N -> N⁺ then we can show that C*f(n) in O(f(n)) for any positive constant C.

We can prove that multiplying f by a constant or adding a constant doesn't change things since in the definition I can choose a larger constant if I want.

O(f(n)) = O(c*f(n)) = O(c*f(n)+d)

We say O(f(n)) > O(g(n)) if there are no constants C and n0 such that f(n) <= C*g(n) for all n > n0.

O(log₂(n)) < O(n) < O(n²) = O(n²+n+1)

Prove O(n²) = O(n²+n+1)

Proof: Assuming n > 1 implies

 1 < n <  n2 implies

 n2 + n + 1  <    n2 + n2 + n2
             =  3 n2

Let n0 = 1 and C = 3 then for all n > 1 ... Using this idea this algorithm runs in O(n²) time.

for i 1 to n
   for j 1 to n
       S             O(n2)

for i 1 to n
       S             O(n)

O(n²) + O(n) = O(n²)

Hashing

• Hashing is another technique for storing and finding information
• A hash function produces an integer hash code for any given item
• The hash code is scaled to a hash index relative to the size of the hash table
• The item is stored in hash table at the hash index
• Collisions may occur, resulting in a list of values at a particular index
• Therefore a small search may be necessary
• Because hashing is based on calculations, it is simple and efficient
• Storing and searching involve the same amount of effort

Chapter 12

What is Recursion?

• Something that is defined by referring to itself.
• can be circular, but then it isn't much use and results in infinite recursion.
• needs at least one base case to avoid infinite recursion, perhaps more.

i.e. Definition of a list

a LIST is a: number
       or a: number , LIST

Why use Recursion?

• some problems are easier to solve
  recursively
• you are given a recursive definition and you want to know whether something
  ``is a'' LIST instance.
• hides implementation details.

a PALINDROME is a: empty_string
             is a: letter
             or a: letter PALINDROME letter

String PALINDROME(String s)
  if s.length <= 1 result true
  else if s.charAt(0) == s.charAt(s.length-1)
    result PALINDROME(s.substr(1,s.length-1)

Iterative solution to Reverse_Numbers

import java.io.*;

class Reverse_Numbers {

  public static void reverse
                     (DataInputStream stdin, int index)
                     throws IOException {

    int[] numbers = new int[index];
    for (int i = 0; i < numbers.length; i++) {
       numbers[i] = Integer.parseInt (stdin.readLine());
    }

    for (int i = numbers.length-1; i >= 0; i--)
       System.out.println (numbers[i]);
}

 public static void main (String[] args)
                     throws IOException {

    static final int number = 4;
    DataInputStream stdin = new DataInputStream(System.in);

    reverse(stdin, number);

 }  // method main

}  // class Reverse_Numbers

Recursion - example

import java.io.*;

class Reverse_NumbersR {

  public static void reverse
                     (DataInputStream stdin, int index)
                     throws IOException {
    if (index > 0) {
      int number = Integer.parseInt (stdin.readLine());
      reverse(stdin, index-1);
      System.out.println (number);
    }
  }

  public static void main (String[] args)
                           throws IOException {

    static final int number = 4;
    DataInputStream stdin = new DataInputStream(System.in);

    reverse(stdin, number);

 }  // method main

}  // class Reverse_NumbersR

Recursive reverse trace

reverse(stdin, 4)

Recursive Binary Search

  public int search(Object[] list, Object key,
                    int first, int last) {
      int i = (first + last)/2;
      if (first > last)              return -1;
      else if (key == list[i].key()) return i;
      else if (key < list[i].key())
        return search(list,key,first,i-1);
      else // key > list[i].key()
        return search(list,key,i+1,last); }

Non-Recursive Binary Search

  public int search(Object[] list, Object key,
                    int first, int last) {
  int first = 0;
  int last = n;
  int i;
  while (true) {
    i = (first + last)/2;
    if (first > last) {i=-1;break;}
    else if (key == list[i].key()) {break;}
    else if (key < list[i].key()) {
       last=i-1;}
    else {
       first=i+1;}
  }
  return i;}

• Lecture 11