Lecture 08: Iterative Correctness

2025-07-09

Table of contents

• Correctness for Iterative Algorithms
• Correctness of merge

• link to other slides
• with scribbles from lecture

Recap

• An algorithm is correct if the precondition implies the postcondition. I.e. if I gave the algorithm valid inputs, I should get the expected output.
• Proving correctness of recursive algorithms is done by induction on the input size.

Correctness for Iterative Algorithms

Iterative Algorithms

Iterative Algorithms are algorithms with a for loop or a while loop in them.

Conventions

i = 0
while i < N:
    # some more code here...
    i += 1

• “After the \(k\)th iteration” refers to the point in the execution of the program just before the loop condition is evaluated for the \(k+1\)st time.

• “Before the \(k+1\)th iteration” is the exact same thing as “after the \(k\)th iteration”

• For example, after the \(k\)th iteration, the value of \(i\) is \(k\)

• After the 0th iteration refers to the point just before the first time the loop condition is evaluated.

A multiplication algorithm for natural numbers

• Input: \((x,y)\)
• Precondition: \(x, y \in \mathbb{N}\)
• Postcondition: return \(xy\).

def mult(x, y):
    i = 0 
    total = 0
    while i < x:
        total = total + y
        i = i + 1

Proof

Solution

Assume the precondition that \(x, y \in \mathbb{N}\), we’ll show that on input \(x, y\), \(\texttt{mult}(x, y)\) return \(xy\).

Let \(\texttt{total}_n\) and \(i_n\) be the value of the variables \(\texttt{total}\) and \(i\) after the \(n\)th iteration.

Let \(P(n)\) be the following predicate: after the \(n\)th iteration,

1. \(i = n\), and

2. \(\texttt{total} = n y\)

We’ll start by showing \(\forall n \in \mathbb{N}. P(n)\).

Note. If there is no \(n\)th iteration, then \(P(n)\) is vacuously true.

By induction on \(n\).

Base Case. We’ll start with \(P(0)\). After the \(0\)th iteration (before the first iteration), we have \(i = 0\), an \(\texttt{total} = 0\), so the base case holds.

Inductive Step. Let \(k \in \mathbb{N}\) be a natural number and suppose \(P(k)\) is true, we’ll show \(P(k+1)\). If there was no \(k+1\)th iteration, then \(P(k+1)\) is vacuously true, so suppose the \(k+1\) iteration ran. Then we have \[ \texttt{total}_{k+1} = \texttt{total}_k + y = ky + y = (k+1)y, \]

and \[ i_{k+1} = i_k + 1 = k + 1. \]

This completes the induction, and hence \(\forall n. P(n)\).

Then, since \(x \in \mathbb{N}\), we have \(P(0), P(1),..., P(x)\), by a. we have that the loop condition passes after the \(i\)th iteration for all \(i < x\), and fails after the \(x\)th iteration at which point we return the value of \(\texttt{total}\) after the \(x\)th iteration which by b. is equal to \(xy\).

Convention

Use subscripts to denote the value of a variable after iteration \(i\). E.g., \(\texttt{total}_i\) is the value of the variable \(\texttt{total}\) after iteration \(i\).

General Strategy

Define a loop invariant - some property that is true at the end of every iteration. Note that it can depend on the iteration number. Call it, for example, \(P(n)\). Another common one is \(P(i)\) if you use \(i\) as the iteration counter.

Prove the following:

Initialization. Show that the loop invariant is true at the start of the loop if the precondition holds.

Maintenance. Show that if the loop invariant is true at the start of any iteration, it is also true at the start of the next iteration.

Termination. Show that the loop terminates and that when the loop terminates, the loop invariant applied to the last iteration implies the postcondition.

Tip: Since the value of variables in code can change at each iteration, it is useful to use the convention in the previous slide to refer to the value of a variable after a certain iteration.

Runtime?

def mult(x, y):
    i = 0 
    total = 0
    while i < x:
        total = total + y
        i = i + 1

Let’s say \(x\) and \(y\) are both \(n\)-digit numbers and it takes time \(O(n)\) time to add two \(n\) digit numbers.

What is the worst-case time complexity of \(\texttt{mult}\) in terms of \(n\), the number of digits?

Solution

Since \(y\) is a \(n\) digit number, it can be as large as \(999...99\) (\(n\)-times), which is equal to \(10^n - 1 = \Theta(10^n)\). Thus, the loop runs for \(O(10^n)\) iterations!

The eventual result has as many \(2n\) digits. Thus, each addition takes time \(O(2n) = O(n)\). In total, the running time is \(\Theta(n10^n)\).

This is terrible. For reference, Grade School Multiplication gets \(O(n^2)\), and Karatsuba’s Algorithm from last week gets \(O(n^{1.59})\). The best-known algorithm for multiplying has runtime \(O(n \log (n))\). By the way, this fast algorithm was just discovered in 2019 and published in 2021!

for Loops

for loops are another type of loop. You can think of loops as while loops with an appropriate loop condition. For example

for i in range(N):
    # some code here...```  

is equivalent to

i = 0
while i < N:    
    # some code here...
    i += 1

Termination

Termination can usually be proved as a consequence of the loop invariant.

Usually, the argument will go something like this.

• By contradiction, suppose the loop didn’t terminate. Then it reaches iteration \(N\) (where \(N\) is some value you chose, big enough to derive a contradiction).

• Then the loop invariant \(P(N)\) implies that the value of some variables is something. This implies the loop condition will be false in the next iteration, which is a contradiction.

If you’re more precise, you can often find the exact number of iterations using the Loop Invariant. That looks something like

• Claim: The loop exits after the \(N\)th iteration

• Let \(i < N\), \(P(i)\) implies that the loop condition is true.

• Furthermore \(P(N)\) implies that the loop condition is false. Therefore, the loop exists after the \(N\)th iteration.

Mystery algorithm

What does the following algorithm do?

Precondition: \(x, y \in \mathbb{N}\), \(y > 0\).

def mystery(x, y):
    val = 0
    c = 0
    while val < x:
        val = val + y
        c = c + 1
    return c

Postcondition:

Solution

Returns \(\left\lceil x/y \right\rceil\).

Proof of Correctness

Loop Invariant. \(P(n)\) is the following predicate. After the \(n\)th iteration

1. \(c = n\)

2. \(\texttt{val}= ny\)

We’ll show \(\forall n \in \mathbb{N}. P(n)\)

Solution

Initialization. For \(n = 0\), \(c\) and \(\texttt{val}\) are both initialized to be \(0\), so \(c = 0\), and \(\texttt{val}= 0 \cdot y = 0\). Thus, the base case holds.

Maintenance. Suppose \(P(k)\), we’ll show that \(P(k+1)\) also holds. If the \(k+1\)th iteration did not run, \(P(k+1)\) is vacuously true. Suppose the \(k+1\)th iteration runs. Then, the variables are updated as follows

• \(c_{k+1} = c_k + 1 = k+1\), and

• \(\texttt{val}_{k+1} = \texttt{val}_k + y = ky + y = (k+1)y\)

this completes the induction.

Termination. We claim that the loop terminates after at most \(n=\left\lceil x /y \right\rceil\) iterations. Indeed, \(P(n)\).b implies that after the \(n\)th iteration, \(\texttt{val}\) is equal to \(\left\lceil x/y \right\rceil \cdot y \geq x\). Thus, the loop terminates.

Let \(n\) be the last iteration that runs so that we return \(c_n = n\). Since the \(n\) was the last iteration that runs by the loop condition, we have

\[ \begin{align*} \texttt{val}_{n-1} < x \leq \texttt{val}_{n} \implies (n-1)y < x \leq ny \implies n-1 < x/y \leq n. \end{align*} \]

In other words, \(n\) is the first integer greater than or equal to \(x/y\), i.e., \(\left\lceil x/y \right\rceil\).

Convention

If the predicate \(P(n)\) has multiple parts like

1. ...

2. ...

Use \(P(n)\).a, \(P(n)\).b,... to refer to specific parts of the predicate.

Variations

• One after the other. Prove the correctness of each loop in sequence.

• Nested loops. “inside out”. Decompose (or imagine) the inner loop as a separate function. Prove the correctness of that function as a lemma, and then prove the correctness of the outer loop. We will see an example in the tutorial.

Another way to prove termination: descending sequence

Another way to prove termination is to define a descending sequence of natural numbers, \(a_1,a_2,...\) indexed by the iteration number.

By the WOP, this sequence must be finite; otherwise, the set \(\{a_1,a_2,...\}\) has no minimal element!

Example

How can we define a descending sequence of natural numbers for this algorithm?

def mystery(x, y):
    val = 0
    c = 0
    while val < x:
        val = val + y
        c = c + 1
    return c

Solution

Idea: \(a_n = x + y - \texttt{val}_n\). Where \(\texttt{val}_n\) is the value of \(\texttt{val}\) at the start of iteration \(n\).

Descending Sequence

Solution

Claim. \(a_n \in \mathbb{N}\) and is decreasing.

By induction. The base case holds from the precondition.

Assuming the claim is true at the start of iteration \(k\), we’ll show that it is also true at the start of iteration \(k+1\). We have \[ \begin{align*} a_{k+1} & = x + y - \texttt{val}_{k+1} \\ & = x + y - \texttt{val}_{k} - y \\ & = a_k - y \\ & < a_k & (y > 0) \end{align*} \]

Furthermore, canceling the \(y\)s in the second line, we get \(a_{k+1} = x - \texttt{val}_k\), which is greater than \(0\) since the while check passes. This combined with the fact that \(y \in \mathbb{N}\) and \(a_k \in \mathbb{N}\) implies that \(a_{k+1} \in \mathbb{N}\).

Thus, \(a_n\) is indeed a decreasing sequence of natural numbers, and the algorithm terminates. (Then argue again that the LI implies the postcondition after the loop ends.)

Proofs of termination: as a part of the LI vs. descending sequence

Most of the time, the LI will imply termination, saving you from having to do another induction proof. I prefer this method.

However, it is easier to define a descending sequence of natural numbers in some cases - we’ll see some examples in the tutorial.

Correctness of merge

Merge

def merge(x, y):
    l = []
    while len(x) > 0 or len(y) > 0:
        if len(x) > 0 and len(y) > 0:
            if y[0] <= x[0]:
                l.append(y.pop(0)) # 1.
            else:
                l.append(x.pop(0)) # 2.
        elif len(x) == 0:   
            l.append(y.pop(0)) # 3.
        else:
            l.append(x.pop(0)) # 4.
    return l

Correctness of Merge

• Precondition: \(x\) and \(y\) are sorted lists.

• Postcondition: A sorted list containing the elements from \(x\) and \(y\).

Counters

For a list \(l\) of natural numbers, let \(\texttt{Counter}(l)\) be a mapping of the elements of \(l\) to the number of times they appear. Ways to think about this

• collections.Counter

• \(\texttt{Counter}(l)\) can be thought of as a multiset (an unordered collection of objects where the same object can appear multiple times)

• \(\texttt{Counter}(l):\mathbb{N}\to \mathbb{N}\) where \(\texttt{Counter}(l)(x)\) is the number of times \(x\) appears in \(l\).

Counters

We can use Counters to express the pre and postconditions more formally.

Precondition. \(x\) and \(y\) are sorted lists of natural numbers.

Postcondition. Returns a sorted list \(l\) such that \(\texttt{Counter}(l) = \texttt{Counter}(x + y)\), note that the \(+\) here is concatenation of lists. This means the returned list is sorted and contains all the elements in \(x\) and \(y\) with the correct frequencies.

Correctness of \(\texttt{merge}\)

Loop Invariant.

\(P(n):\) After the \(n\)th iteration,

1. \((a \in x_n + y_n \land b \in l_n) \implies a \geq b\).

2. \(\texttt{Counter}(x_n+y_n+l_n) = \texttt{Counter}(x_0 + y_0)\).

3. \(\texttt{len}(l_n) = n\).

4. \(x_n, y_n, l_n\) are all sorted.

\(P(n):\) After the \(n\)th iteration,

1. \((a \in x_n + y_n \land b \in l_n) \implies a \geq b\).

2. \(\texttt{Counter}(x_n+y_n+l_n) = \texttt{Counter}(x_0 + y_0)\).

3. \(\texttt{len}(l_n) = n\).

4. \(x_n, y_n, l_n\) are all sorted.

Solution

Initialization. We’ll show \(P(0)\):

1. is vacuously true since \(l_0\) is empty

2. \(\texttt{Counter}(x_0 + y_0 + l_0) = \texttt{Counter}(x_0 + y_0 + []) = \texttt{Counter}(x_0 + y_0)\)

3. \(\texttt{len}(l_n) = \texttt{len}([]) = 0\).

4. \(x_0, y_0\) are sorted by the precondition, and \(l_0\) is vacuously sorted.

Maintenance. Let \(k \in \mathbb{N}\) be any natural number and suppose \(P(k)\). We’ll show that \(P(k+1)\). There are several cases marked by the numbers in the comments. Let’s start with case 1.

The variables are updated as follows.

• \(l_{k+1} = l_k + [y_k[0]]\).

• \(y_{k+1} = y_k[1:]\).

• \(x_{k+1} = x_k\).

In this case, \(x\) and \(y\) are both non-empty and \(y[0] \leq x_0\). We’ll show \(P(k+1)\)

Maintenance. (a.) We need to show that \(a \in x_{k+1} + y_{k+1} \land b \in l_{k+1} \implies a \geq b\). Since \(y_k[0]\) is the only element that moved, by \(P(k)\).a, it suffices to consider when \(b = y_k[0]\). I.e., we need to show that \(y_k[0]\) is minimal in \(x_k + y_k\).

Since \(y_k\) and \(x_k\) are sorted by \(P(k)\).d, \(y_k[0]\) is minimal in \(y_k\), since \(y_k[0] \leq x_k[0]\), and \(x_k\) is sorted, \(y_k[0]\) is also minimal in \(x_k\).

Maintenance. (b.) We have \[ \begin{align*} \texttt{Counter}(x_{k+1} + y_{k+1} + l_{k+1}) & = \texttt{Counter}(x_{k} + y_{k}[1:] + l_{k} + [y_k[0]]) \\ & = \texttt{Counter}(x_{k} + y_{k} + l_{k}) \\ & = \texttt{Counter}(x_0 + y_0). \end{align*} \]

The second line holds because Counteris unordered, and the third line holds because of \(P(k)\).b

Maintenance. (c.) \(\texttt{len}(l_{k+1}) = \texttt{len}(l_{k}) + 1 = k+1\)

Maintenance. (d.) By \(P(k)\).d we have \(x_k, y_k\), and \(l_k\) are all sorted.

• \(x_{k+1} = x_k\), and so is still sorted.

• \(y_{k+1}\) is a sublist of \(y_k\) and so is also sorted.

• \(l_{k+1} = l_{k} + [y_k[0]]\) is sorted because \(P(k)\).a implies that \(\forall b \in l_k\), \(y_k \geq b\).

The other cases are similar; I’ll leave this to you.

Termination. Let \(n = \texttt{len}(x_0 + y_0)\). I claim that the algorithm terminates after \(n\) iterations. By \(P(n)\).c, we have \(\texttt{len}(l_n) = \texttt{len}(x) + \texttt{len}(y)\), by \(P(n)\).b, we have \(\texttt{Counter}(x_n + y_n + l_n) = \texttt{Counter}(x_0 + y_0)\), since \(\texttt{len}(l_n) = \texttt{len}(x_0 + y_0)\), \(x_n\) and \(y_n\) must be empty and thus the loop condition fails. Thus, the algorithm terminates.

Now let \(n\) be the last iteration that runs. Since the next iteration did not run, we have \(x_n\) and \(y_n\) are both empty. By \(P(n).b\), we have \(\texttt{Counter}(x_n + y_n + l_n) = \texttt{Counter}(l_n) = \texttt{Counter}(x_0 + y_0)\). Furthermore, by \(P(n).d\), \(l_n\) is sorted as required.

Loop Invariants

• It’s normal for the loop invariant to have many parts!

• If you’re trying to prove a loop invariant and you get stuck and wish some other property holds, try adding what you need as part of the loop invariant.

• For example, it’s common for part 4. of a loop invariant to imply part 1. of the loop invariant.

Summary - Correctness of Algorithms

• If the algorithm is recursive, prove correctness directly by induction.

• For algorithms with loops, prove the correctness of the loop by defining a Loop Invariant, proving the Loop Invariant, and showing that the Loop Invariant holding at the end of the algorithm implies the postcondition.