<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">% Course notes for the current or upcoming lectures. After each lecture, I will
% move the material I presented from this file into a separate record just for
% that lecture.

\documentclass{article}

\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{changepage}  % For adjustwidth environment
\usepackage{colortbl}  % Provides the  \arrayrulecolor command.
\usepackage{environ}
\usepackage{framed}  % For leftbar environment
\usepackage[hidelinks]{hyperref}  % For \autoref
\usepackage{mathtools}  % For \shortintertext
\usepackage{parskip}  % Don't indent paragraphs; skip instead.
\usepackage{varwidth}
\usepackage{url}
\usepackage{xcolor}


% Stuff that should already be on slides.
\NewEnviron{onslide}{{

    \newdimen\slidewidth
    \slidewidth=\linewidth
    % Without this, I get errors beginning "Overfull \hbox (6.79999pt too
    % wide)"
    \advance\slidewidth by -7pt

    \medskip

    \fbox{\parbox{\slidewidth}{\BODY}}

    \medskip

  }}

% Stuff I should write or draw during lecture.
\NewEnviron{writelive}{\vspace{1ex}\begin{leftbar}{\BODY}\end{leftbar}}

\newenvironment{indentone}{\begin{adjustwidth}{2em}{0em}}{\end{adjustwidth}}


% Binary and unary all-uppercase text operators. \mathop gives too little space
% around the binary ones; e.g. \(P \op{AND} Q\) needs lots of space between
% those capital letters.
\newcommand\op[1]{{\ \mathrm{#1}\ }}
\newcommand\uop[1]{{\mathrm{#1}}}

\newcommand\AND{{\op{AND}}}
\newcommand\IFF{{\op{IFF}}}
\newcommand\IMPLIES{{\op{IMPLIES}}}
\newcommand\NOT{{\uop{NOT}}}
\newcommand\OR{{\op{OR}}}
\newcommand\XOR{{\op{XOR}}}

\newcommand\FALSE{{\mathrm{F}}}
\newcommand\TRUE{{\mathrm{T}}}

\DeclareMathOperator\concat{+\!\!+}
\DeclareMathOperator\Bkt{Bkt}
\DeclareMathOperator\leftT{left}
\DeclareMathOperator\rightT{right}

\newcommand\stringLit[1]{{\texttt{"#1"}}}

\newcommand\ComplexNums{{\mathbb{C}}}
\newcommand\Ints{{\mathbb{Z}}}
\newcommand\Nats{{\mathbb{N}}}
\newcommand\Reals{{\mathbb{R}}}
\newcommand\Rationals{{\mathbb{Q}}}

\newcommand\lbkt{{\texttt{[}}}
\newcommand\rbkt{{\texttt{]}}}

\newcommand\powerset{{\mathcal{P}}}

\newcommand\optional[1]{\textcolor[rgb]{0.5,0.5,0.3}{#1}}

% Warning: this resets the colour to black using \arrayrulecolor.
\newcommand{\greyline}{\arrayrulecolor{black!20}\hline\arrayrulecolor{black}}

\newcommand\Todo[1]{{\fbox{\begin{varwidth}{\linewidth}TODO: {#1}\end{varwidth}}}}

% Don't put theorem text in italics, because in this document, italics is for
% descriptions of things I should write or have on slides.
\theoremstyle{definition}
\newtheorem{theorem}{Theorem}
\newtheorem*{theorem*}{Theorem}
\newtheorem*{definition*}{Definition}
\newtheorem*{lemma*}{Lemma}

\begin{document}

This week, the lecture was only 50 minutes long.

These are \emph{rough notes} based on my own notes I prepared for the Week~7
lecture. They go with the annotated lecture slides you can see on the ``More''
page of the course website. Although they are rough, I'm providing them here in
case they help you remember what we did in lecture.

\section{Legend}


\begin{onslide}
Text that was already on the slides is in a box, like this.

\emph{Descriptions of things that were already on the slides are formatted like
  this.}
\end{onslide}

\begin{writelive}
Text I planned to write during lecture is formatted like this, with a line to
the left.

\emph{Descriptions of other things I planned to draw or write or do are
  formatted like this.}
\end{writelive}

Anything else is text I planned to say, or things I planned to ask.

\section{Big Oh notation}

\subsection{Definition of Big Oh}

\begin{onslide}
\textbf{Big \(O\) Notation}

Let \( \mathcal {F} \) be the set of all functions from \( \Nats \) to
\( \Reals^+ \).

For any \( f \in \mathcal {F} \), let
\[ O(f) = \{ g \in \mathcal {F} \mid \exists c \in \Reals^+ .\&gt; \exists b \in
\Nats .\&gt; \forall n \in \Nats .\&gt; (n \ge b \IMPLIES g(n) \le c f(n)) \} \]
\end{onslide}

In other words:

\begin{writelive}
\( f \in O(g) \) means for all sufficiently large \( n \), \( g(n) \) is at most
a constant factor times \( f(n) \).

\emph{(Draw example: graph with \( 3 n \) vs.\ \( n^2 \). Draw a line where we
  can put \(b\). We can take \( c = 1 \).)}
\end{writelive}

\subsection{Properties of Big Oh}

\begin{onslide}
\textbf{Properties of big \(O\)}

Summary on ``Further reading'' page of course website.

\begin{itemize}
\item Constant factors don't matter:

If \( d &gt; 0 \) is a constant, then \( d f(n) \in O( f(n) ) \) and
\( f(n) \in O( d f(n) ) \).
\end{itemize}
\end{onslide}

\begin{writelive}
\(5 n^2 \in O( n^2 )\)
\end{writelive}

\begin{onslide}
\begin{itemize}
\item Low order terms don't matter:

If \( \lim_{ n \to \infty } \frac {h(n)} {g(n)} = 0 \) then
\( g(n) + h(n) \in O( g(n) ) \).
\end{itemize}
\end{onslide}

\begin{writelive}
\( n^3 + n^2 + n \in O( n^3 ) \)
\end{writelive}

\begin{onslide}
\begin{itemize}
\item Transitivity:

If \( f(n) \in O( g(n) ) \) and \( g(n) \in O( h(n) ) \) then \( f(n) \in O( h(n) )\)
\item Summation rules:

If \( f_1 (n) \in O( g_1 (n) ) \) and \( f_2 (n) \in O( g_2 (n) ) \) then
\( f_1 (n) + f_2 (n) \in O( g_1 (n) + g_2 (n) ) \).

\( \max \{ f (n) , g (n) \} \in O( f (n) + g (n) ) \)

\( f (n) + g (n) \in O( \max \{ f (n) , g (n) \} ) \)
\item Product rule:

If \( f_1 (n) \in O( g_1 (n) ) \) and \( f_2 (n) \in O( g_2 (n) ) \) then
\( f_1 (n) f_2 (n) \in O( g_1 (n) g_2 (n) ) \)
\end{itemize}
\end{onslide}

\begin{writelive}
\( 3 \cdot (2^n + 1) \cdot (n^3 + n + 2) \in O( 2^n \cdot n^3 ) \)
\end{writelive}

\begin{onslide}
Exercise: prove some of these.
\end{onslide}

\begin{onslide}
\textbf{Exponents and logarithms}

\begin{itemize}
\item Exponents matter:

If \( a \le b \) then \( n^a \in O( n^b ) \).

If \( a &gt; b \) then \( n^a \not\in O( n^b ) \).
\end{itemize}
\end{onslide}

\begin{writelive}
\( n^2 \in O( n^{ 2.5 } ) \)

\( n^3 \not\in O( n ) \)
\end{writelive}

\begin{onslide}
\begin{itemize}
\item Bases of exponents matter:

If \( 1 &lt; a \le b \) then \( a^n \in O( b^n ) \).

If \( 1 &lt; b &lt; a \) then \( a^n \not\in O( b^n ) \).
\end{itemize}
\end{onslide}

\begin{writelive}
\( 2^n \in O( 3^n ) \)

\( 3^n \not\in O( 2^n ) \)
\end{writelive}

\begin{onslide}
\begin{itemize}
\item Bases of logarithms don't matter:

For all \( a , b &gt; 1 \), \( \log_a (n) \in O( \log_b (n) ) \)
\end{itemize}
\end{onslide}

\begin{writelive}
\( \log_2 (n) \in O( \log_{10} (n) \)
\end{writelive}

\begin{onslide}
\begin{itemize}
\item Exponential functions grow faster than polynomial functions:

\( \forall b &gt; 1 \) and \( a \in \Reals \), \( n^a \in O( b^n ) \) but
\( b^n \not\in O( n^a ) \).
\end{itemize}
\end{onslide}

\begin{writelive}
\( n^{100} \in O( 2^n ) \)

\( 2^n \not\in O( n^{100} ) \)
\end{writelive}

\begin{onslide}
\begin{itemize}
\item Polynomial functions grow faster than polylogarithmic functions:

For all \(a, b &gt; 0 \), \( (\log n)^a \in O( n^b ) \) but
\( n^b \not\in O( (\log n)^a ) \).
\end{itemize}
\end{onslide}

\begin{writelive}
\( ( \log n )^3 \in O( n ) \)

\( n \not\in O( ( \log n )^3 ) \)
\end{writelive}

\subsection{Definition of big \(\Omega\) and big \(\Theta\)}

\begin{onslide}
For any \( f \in \mathcal {F} \),
\[ O(f) = \{ g \in \mathcal {F} \mid \exists c \in \Reals^+ .\, \exists b \in
\Nats .\, \forall n \in \Nats .\, (n \ge b \IMPLIES g(n) \le c f(n)) \} \]
\end{onslide}

\begin{writelive}
\[ \Omega(f) = \{ g \in \mathcal {F} \mid \exists c \in \Reals^+ .\, \exists b \in
\Nats .\, \forall n \in \Nats .\, (n \ge b \IMPLIES g(n) \ge c f(n)) \} \]
\begin{flalign*}
  \Theta(f)
  =&amp; O(f) \cap \Omega(f) &amp; \\
  =&amp; \{ g \in \mathcal {F} \mid \exists c_1 \in \Reals^+ .\, \exists c_2 \in \Reals^+ .\, \exists b \in \Nats
  .\, \forall n \in \Nats .\, \\
  &amp; (n \ge b \IMPLIES c_1 f(n) \le g(n) \le c_2 f(n)) \}
\end{flalign*}
\end{writelive}

\section{Solving recurrences}

\begin{onslide}
\textbf{Solving recurrences}

Define \( T : \Nats \to \Nats \) by:
\begin{itemize}
\item \( T (0) = 3 \)
\item For \( n \in \Nats \), \( T (n + 1) = 2 T (n) + 1 \)
\end{itemize}

Find a \textbf{closed form} for \(T (n)\).
\end{onslide}

\begin{writelive}
\emph{(Arrow to the equations.)} Recurrence relation

\emph{(Arrow to ``closed form''.)} Simple expression with no recursion.

\begin{tabular}{c|c}
  n &amp; \(T(n)\) \\
  \hline
  0 &amp; 3 \\
  1 &amp; 7 \\
  2 &amp; 15 \\
  3 &amp; 31 \\
  4 &amp; 63
\end{tabular}

Guess: \( T (n) = 4 \cdot 2^n - 1 \)

Verify: prove it (usually by induction)
\end{writelive}

\begin{onslide}
\textbf{Solving a recurrence} means finding a closed form: writing the same
function as a simple expression with no recursion.

Methods for solving recurrences:
\begin{enumerate}
\item Guess and verify
\item Repeated substitution and verify
\item Special techniques for ``divide and conquer'' recurrences / Master Theorem
\item Transformations
\item Solving linear recurrences using characteristic polynomials
\end{enumerate}
\end{onslide}

We just did guess and verify.

\begin{onslide}
\textbf{Repeated substitution and verify}

Define \(M : \Ints^+ \to \Nats\) by
\begin{itemize}
\item \( M(1) = c \)
\item For \(n &gt; 1\), \( M(n) = M( \lceil n/2 \rceil ) + M( \lfloor n/2 \rfloor ) + dn\)
\end{itemize}
where \(c, d \in \Nats\) are constants.
\end{onslide}

\begin{writelive}
Assume \(n\) is a power of 2.

\( M(n) = 2 M( n/2 ) + dn \)

\begin{flalign*}
M(n)
=&amp; 2 M( n/2 ) + d n &amp; \\
=&amp; 2 [ 2 M( n/2^2 ) + dn/2 ] + dn \\
=&amp; 2 [ 2 [ 2 M( n/2^3 ) + dn/2^2 ] + dn/2 ] + dn \\
\vdots&amp; \quad \text{find the pattern} \\
=&amp; 2^k M( n/2^k ) + kdn
\end{flalign*}
When \( k = \log_2 n \), get
\( M(n) = n M( 1 ) + dn \log_2 n = cn + dn \log_2 n \).

Now, prove this is correct using induction. For \(k \in \Nats\), let \(P (k)\) =
``\( M(2^k) = c 2^k + d k 2^k \)''.

What if \(n\) not a power of 2?
\end{writelive}

\begin{onslide}
\emph{(Repeat definition of \(M\).)}

\begin{theorem}
\( M(n) \in O( n \log n ) \)
\end{theorem}
\end{onslide}

We ended the lecture here. (I explained that if we ignore \(n\) that are not
powers of \(2\), the theorem is not hard: we have already proved
\( M(n) = nc + dn \log n \) in that case, and we can use properties of big
\(O\).)

\end{document}
</pre></body></html>