What is induction used for
Induction is used to prove statements of the following form: \[\forall n \in \mathbb{N}. (P(n)).\]
Note that \(P\) is a predicate on the natural numbers. I.e., for any natural number \(n\), \(P(n)\) is either true or false. For example, \(P(n)\) might be
The sum of the first \(n\) odd numbers is \(n^2\).
\((12^n - 1)\) is divisible by 11.
Trees with \(n\) vertices have \(n-1\) edges.
Induction is super useful when analyzing the correctness and runtime of algorithms.
Induction
![]()
Induction
\[(P(0) \land \forall k \in \mathbb{N}.(P(k) \implies P(k+1))) \implies \forall n \in \mathbb{N}. (P(n))\]
Induction
\[(P(0) \land \forall k \in \mathbb{N}.(P(k) \implies P(k+1))) \implies \forall n \in \mathbb{N}. (P(n))\]
“If I can show that the first domino falls, and I can show that for any domino, if that domino falls, the next one falls, every domino falls”.
Here, the \(k\)th domino falling is analogous to \(P(k)\) being true.
Induction template
Suppose we want to prove \(\forall n \in \mathbb{N}. (P(n))\). Here is the template:
By induction.
Base case. [Prove \(P(0)\) is true]
Inductive step. Let \(k \in \mathbb{N}\) be an arbitrary natural number, and assume \(P(k)\). We’ll show \(P(k+1)\). [Prove \(P(k+1)\) assuming \(P(k)\)]. This completes the induction.
Flexibility
There is some flexibility in the proof by induction template. For example, sometimes, we want to prove a statement is true for all \(n \geq 1\), so the base case starts at 1 instead of 0. Sometimes, for the inductive step, we need to show that if the previous two dominoes fall, then the next one will also fall. In this case, we would need to prove two base cases. These will come up in the examples, and you will see why we can do this next week.
You should be okay as long as you showed enough so that “all the dominoes fall”.
Examples of Proofs by Induction
Binary Trees
![]()
Perfect binary trees
![]()
In a perfect binary tree, all leaves are at the same level, and every other vertex has two children and one parent (except for the root, which does not have a parent).
Number of vertices of perfect binary trees
![]()
How many vertices does a perfect binary tree of height \(n\) have?
\(\forall n \in \mathbb{N}. (2^0 + 2^1 +...+2^n = 2^{n+1} - 1)\)
Solution
By induction.
Base case. For the base case, we need to check \(2^0 = 2^{0+1} -1\). This holds since they are both equal to 1.
Inductive step. Let \(k \in \mathbb{N}\) be an arbitrary natural number, and assume \(\sum_{i=0}^k2^i = 2^{k+1} - 1\). We’ll show \(\sum_{i=0}^{k + 1}2^i = 2^{k+2} - 1\). We have \[
\begin{align*}
\sum_{i=0}^{k + 1}2^i & = 2^{k+1} + \sum_{i=0}^{k}2^i \\
& = 2^{k+1} + 2^{k+1} - 1 \\
& = 2^{k+2} - 1,
\end{align*}
\] as required.
Another way to do it…
![]()
We’ll see next time.
Sum of first \(n\) odd numbers is the \(n\)th square
![]()
Sum of first \(n\) odd numbers is the \(n\)th square
Solution
We want to prove \(\forall n \in \mathbb{N}, n \geq 1. (\sum_{i=1}^n 2i-1 = n^2)\).
Base case. \(2\cdot 1 - 1 = 1 = 1^2,\) so the base case holds. Note that we prove \(P(1)\) instead of \(P(0)\) since the statement we are trying to prove starts at 1.
Inductive step. Let \(k \in \mathbb{N}, k \geq 1\) be an arbitrary natural number, and assume \(\sum_{i=1}^k 2i-1 = k^2\). We’ll show \(\sum_{i=1}^{k + 1}2i-1 = (k+1)^2\).
\[
\begin{align*}
\sum_{i=1}^{k + 1}2i-1 & = 2(k+1) - 1 + \sum_{i=1}^{k}2i-1 \\
& = 2k+1 + k^2 \\
& = (k+1)^2
\end{align*}
\]
as required.
\(\forall n \in \mathbb{N}. (n^3 - n + 3)\) is divisible by \(3\)
Solution
By induction.
Base case. For the base case, we have \(0^3 - 0 + 3 = 3\) which is divisible by \(3\).
Inductive step. Let \(k \in \mathbb{N}\) by any natural number and assume \(k^3 - k + 3\) is divisible by \(3\), i.e. \(k^3 - k + 3 = 3p\) for some \(p \in \mathbb{N}\). We will show \((k+1)^3 - (k + 1) + 3\) is also divisible by \(3\). We have \[
\begin{align*}
(k+1)^3 - (k + 1) + 3 & = k^3 + 3k^2 + 3k + 1 - k - 1 + 3 \\
& = (k^3 - k + 3) + 3k^2 + 3k + 1 - 1 \\
& = 3p + 3k^2 + 3k \\
& = 3(p + k^2 + k)
\end{align*},
\]
as required.
\(\forall n \in \mathbb{N}. \text{ the units digit of } 7^n\) is 1, 3, 7, or 9
Solution
By induction.
Base case. \(7^0 = 1\), so the base case holds.
Inductive step. Let \(k \in \mathbb{N}\) be an arbitrary natural number, and assume the units digit of \(7^k\) is either \(1, 3, 7\) or \(9\). There are several cases.
- If it was \(1\), \(7^{k+1}\) has units digit \(7\).
- If it was \(3\), \(7^{k+1}\) has units digit \(1\).
- If it was \(7\), \(7^{k+1}\) has units digit \(9\).
- If it was \(9\), \(7^{k+1}\) has units digit \(3\).
Thus, the inductive step holds, and we are done.
\(\forall n \in \mathbb{N}. (n^2 \leq 2^n)\)
False! E.g., for \(n = 3\), the LHS is 9 and the RHS is 8.
\(\forall n \in \mathbb{N}, n \geq 4.(n^2 \leq 2^n)\)
![]()
\(\forall n \in \mathbb{N}, n \geq 4.(n^2 \leq 2^n)\)
Solution
By induction.
\(4^2 = 16\) and \(2^4 = 16\), so the base case holds.
Let \(k \in \mathbb{N}\) be an arbitrary natural number with \(k \geq 4\), and assume \(k^2 \leq 2^{k}\). We’ll show \((k+1)^2 \leq 2^{k+1}\). We have
\[
\begin{align*}
(k+1)^2 & = k^2 + 2k + 1 \\
& \leq k^2 + (k-2)k + 1 & (k-2 \geq 2) \\
& = k^2 + k^2 - 2k + 1 \\
& \leq 2k^2 & (-2k + 1 \leq 0) \\
& \leq 2\cdot 2^k & (\text{IH}) \\
& = 2^{k+1},
\end{align*}
\]
completing the induction.
All birds have the same color - “proof”
Claim. \(\forall n \in \mathbb{N}\), a set of \(n\) birds will all have the same color.
Base Case. For \(n = 0\), the claim is vacuously true.
Inductive step. Let \(k \in \mathbb{N}\) be any number and assume a set of \(k\) birds will all have the same color. Let \(S\) be a set of \(k + 1\) birds. We’ll show that all the birds in \(S\) have the same color.
![]()
Remove an arbitrary bird \(b_1\). ![]()
The remaining \(k\) birds must have the same colour by the inductive hypothesis. ![]()
Add back the removed bird and now remove a different bird \(b_2\). ![]()
By the same reasoning, the remaining \(k\) birds must have the same color. Therefore, \(b_1\) has the same colour as birds which have not been removed, which in turn have the same colour as \(b_2\).
![]()
What went wrong?
Solution
There is an implicit assumption here that there is another bird other than \(b_1\) and \(b_2\)! I.e., for this to hold, we need \(|S|\geq 3\)! In particular, it does not hold for \(k = 1\) (\(|S| = 2\)).
Takeaway
Induction can be tricky! Make sure your inductive step does not assume anything more than what you claim! For example, in the \(n^2 \leq 2^n\) example, we could assume \(k \geq 4\) since we were restricting to the case where \(k \geq 4\), but we couldn’t do the same for the “all birds have the same color” example.
Complete Induction
Complete induction is another way to prove statements of the form \(\forall n \in \mathbb{N}. (P(n))\).
Another way to get all the dominoes to fall
If I want to show \(\forall n \in \mathbb{N}. (P(n))\), it suffices to prove
“If I can show that the first domino falls, and I can show that for any domino, if that domino falls and all previous dominoes fall, that the next one also falls, every domino falls”.
Complete Induction template
Suppose we want to prove \(\forall n \in \mathbb{N}. (P(n))\). Here is the template:
By complete induction.
Base case. [Prove \(P(0)\) is true]
Inductive step. Let \(k \in \mathbb{N}\) be an arbitrary natural number, and assume for every \(i \in \mathbb{N}\) with \(i \leq k\), \(P(i)\) holds. We’ll show \(P(k+1)\). [Prove \(P(k+1)\) using this assumption]
Chocolate
![]()
Chocolate - Attempted proof by regular induction
Claim: Let \(n \in \mathbb{N}, n \geq 1\) be any natural number. A chocolate bar with \(n\) individual pieces requires \(n-1\) breaks to split the bar into individual pieces.
Let \(P(n)\) be the predicate: A bar of chocolate composed of \(n\) individual pieces requires \(n-1\) breaks.
Base case. For \(n=1\), the chocolate bar is already a single piece of chocolate and thus requires \(1 - 1 = 0\) breaks.
Inductive step. Let \(k \in \mathbb{N}, k \geq 1\) be an arbitrary natural number at least 1, and assume \(P(k)\) is true. We need to show \(P(k+1)\) is true. Let \(B\) be a bar of chocolate with \(k+1\) pieces, and pick a way to break the chocolate. We are left with two blocks of chocolate, of sizes \(a\) and \(b\), respectively, where \(a + b = k + 1\).
Chocolate - Proof by Complete Induction
Solution
Let \(P(n)\) be the predicate: A bar of chocolate composed of \(n\) individual pieces requires \(n-1\) breaks.
Base case. For \(n=1\), the chocolate bar is already a single piece of chocolate and thus requires \(1 - 1 = 0\) breaks.
Inductive step. Let \(k \in \mathbb{N}, k \geq 1\) be an arbitrary natural number at least 1, and assume \(P(i)\) is true for all \(i \leq k\). We need to show \(P(k+1)\) is true. Let \(B\) be a bar of chocolate with \(k+1\) pieces, and select any way to break the bar. We are left with two blocks of chocolate of size \(a\) and \(b\) respectively, where \(a + b = k + 1\), \(a \leq k\), and \(b \leq k\). Applying the inductive hypothesis to each of these blocks, we have that the two blocks require \(a-1\) and \(b-1\) breaks respectively, so the total number of breaks (including the initial) is \(a + b - 2 + 1 = k\), as required.
Here’s the picture:
![]()
Number of Edges in a Tree
As a reminder, a tree is a graph \(G = (V, E)\) that is both acyclic (has no cycles) and connected (every pair of vertices is connected by some path). What was our conjecture from last time?
![]()
Solution
A tree with \(|V|\) vertices has \(|V| - 1\) edges.
A tree has \(|V|\) - 1 edges
![]()
Why would I ever use regular induction if I can use complete induction?
You can always use complete induction if you wish! The inductive hypothesis is stronger (I.e., you get to assume \(P(0)\land...\land P(k)\) instead of just \(P(k)\)), but still lets you prove the same statement: \(\forall n \in \mathbb{N}. (P(n))\).
That being said, some mistakes are easier to make when using complete induction, and sometimes regular induction is easier to work with.
Induction and Algorithms
Proof by induction is an incredibly powerful technique that will allow us to prove strong guarantees about the runtime and correctness of algorithms.
I.e., Induction lets us prove \(\forall n \in \mathbb{N}. (P(n))\) consider what this means when
\(P(n)\) is: Algorithm X is correct on inputs of size \(n\).
\(P(n)\) is: Algorithm X is correct if the for loop runs for at most \(n\) iterations.
\(P(n)\) is: The runtime of Algorithm X is \(\Theta(n^2)\).
..etc.
A note about style
Sometimes you might find it easier to define a predicate in the following way: “Let \(P(n)\) be the predicate...” for example, in the chocolate example. This approach allows you to refer to the predicate easily. For example, defining \(P\) allows you to say “Assume \(P(k)\) is true...”
Other times, you might find it easier to directly work with the predicate without giving it a name, for example, in the divisibility example. This approach makes stating the inductive hypothesis more troublesome, but it reminds the reader of exactly what you’re trying to prove.
Both are fine!
Additional Notes
Induction and recursive algorithms are closely linked. Think of how! We will explore this in future classes.
Induction is a challenging concept to grasp. In particular, it takes a little bit of faith to believe that simply showing a base case and an inductive step allows us to prove a statement is true for all natural numbers. It’s good to keep the domino analogy in mind, i.e., when writing your proofs, ask - ‘did I show all the dominoes fall?’
Although intuition is important, at the end of the day, remember that the base case and inductive step are both mathematical statements that you need to prove. I.e., you should approach proving \(\forall k \in \mathbb{N}. (P(k) \implies P(k+1))\) like you would approach proving any other FOL statement.
