CSC336 Numerical Methods
Course information for current students:
 20231214: A3 has been marked and marks are available on MarkUs.
Since there are no classes anymore,
I write here a few notes.
 20231208:
The final is comprehensive, with more focus
on the topics not covered in the midterm.
See file feprep.txt for some more information.
I have also posted some practice problems.
 20231205:
Please complete the course evaluations.
Course evaluations are important for the university, the department,
the instructor, as well as for fellow students.
And course evaluations become more reliable, when the participation is high.
Please take a few minutes of your time to complete them.
 20231205:
One more slot of office hours for A3:
Wednesday, Dec 6, 2023, 34 PM.
Office hours for exam:
Tuesday, Dec 12, 2023, 4:305:30 PM,
Friday, Dec 15, 2023, 24 PM.
If these do not fit, send me email.
 20231204: A2 has been marked and marks are available on MarkUs.
I will discuss part of the solutions in class tomorrow.
 20231120: Assignment 3 has been posted:
due Thursday, December 7, 2023, ***6PM***.
You may also need
testnl.m.
Extra and regular office hours :
[Tuesday, Nov 21, 2023, 4:305:30 PM],
[Monday, Nov 27, 2023, 4:305:30 PM],
[Tuesday, Nov 28, 2023, 4:305:30 PM],
[Friday, Dec 1, 2023, 3:004:00 PM],
[Monday, Dec 4, 2023, 4:305:30 PM],
[Tuesday, Dec 5, 2023, 4:305:30 PM].
 20231030: The midterm has been marked and marks are available
through the
CDF (teach.cs) student secure site.
These marks are NOT available on MarkUs, but include those of MarkUs.
Please notify me, if there are any discrepancies.
In class, tomorrow, I will discuss parts of the solutions,
and return the exams.
 20231030:
Office hours and extra office hours for A2:
Tue 31102023 4:305:30 PM,
Fri 3112023 3:304:30 PM,
Mon 6112023 4:305:30 PM,
Tue 7112023 4:305:30 PM,
Fri 10112023 3:304:30 PM,
Mon 13112023 4:305:30 PM,
Tue 14112023 4:305:30 PM.
Note that some of these are during the Fall break (610 November 2023),
when we do not have classes, but I am available for office hours.
 20231016: The midterm (Tue 24102023, 6 PM) will cover all of
Computer arithmetic and Direct methods for solving linear systems.
All material up to GE/LU/pivoting is included.
The midterm does NOT include norms and condition numbers of matrices.
See more info here.
Office hours and extra office hours:
Friday, 20 Oct 2023, 14:0015:30 PM,
Monday, 23 Oct 2023, 16:3017:30,
Tuesday 24 Oct 2023, 16:0017:30,
If none of the above fits you or you need to meet remotely
through Zoom, please send me email ahead of time.
 20231015: Marks for A1 are released on MarkUs.
Solutions will be discussed in next class.
They will not be posted, so please come to class.
 20231014: Assignment 2 has been posted:
Due Wed Nov 15, 2023, at ***6 PM***.
MarkUs will open a few days before the submission deadline.
 20231003: Regarding A1 generally:
mathematically equivalent means exactly equal (no remainder).
For the output: keep given formats and given headers
(the fprintf statements before the loop that outputs indications
what each column corresponds to).
Make sure output is aligned (to match equivalent order digits.)
 20231002: In Q2 of A1, the fourdecimaldigitsprecision
floatingpoint arithmetic is with rounding (proper if it matters).
 20231001:
MarkUs
is now open for A1, Due Wed Oct 4, 2023, at ***6 PM***.
 20230920:
Extra office hours :
Monday, Sept 25, 2023, 4:305:30 PM,
Friday, Sept 29, 2023, 3:304:30 PM,
Monday, Oct 2, 2023, 4:305:30 PM,
besides the regular ones
Tuesday, Sept 26, 2023, 4:305:30 PM,
Tuesday, Oct 3, 2023, 4:305:30 PM.
 20230919: Assignment 1 has been posted
on the course website:
Due Wed Oct 4, 2023, at ***6 PM***.
MarkUs will open a few days before the submission deadline.
To access the assignment, or the notes, you will need to type
your CDF (teaching labs) username (same as UTorId) as loginname,
and last 5 digits of your student number as password.
This password (for accessing the website) cannot be reset,
and it is only for accessing the website.
If you registered for the course today, please wait a day,
to check access to the website.
Please note that to access the servers/computers,
the password is different.
There is an initial password set by the system (usually the student number),
and then you can set it to whatever you want.

The course starts on Tuesday, September 12, 2023, 69 PM, in BA 1170.
Note that we start at 6:10 PM.

No classes
the week 610 November 2023 (Fall break / Reading week  A&S calendar).

Bulletin board for csc336 Fall 2023

Important note on the use of bulletin boards:
No parts of or whole answers to the assignment/exam problems
should be posted to the boards (or anywhere else),
even after the assignment is due.
Questions and answers (even written by you) should never
be shared with anyone or anywhere.
Any violation of this rule will bring trouble to the poster.
Please use judgement before posting.
Any questions posted on the bulletin boards should be general enough
and should not reveal intermediate or final results (correct or wrong).
If unsure, ask by email to instructor.

All assignments and exams are to be done individually by each student.
See the course outline
about academic integrity and additional information.
 Here is a latex example file,
with associated files
spyalt.eps, spyblock.eps,
trochoid1.m,
assign.bib, to compile correctly,
and output 336a1.pdf.
You can use it for assignments, but you may also use you own latex template.
Please always use font size 12 and linespread 1.1, as shown in the sample file.
Do NOT use dark background in any page or figure.
See the course outline for more details on presentation.

CDF/Teaching Labs:
Please see http://www.cdf.toronto.edu
or http://www.teach.cs.toronto.edu
(they are the same), especially the
Resources, Intro to new students tab, and the
Using Labs, Remote Access Server tab.

MATLAB, etc remotely on cdf:
If you want to run matlab and other applications remotely
on teach.cs (CDF), and you are using Windows or Mac OS
on your computer/laptop, you will need the socalled
X forwarding on your computer/laptop.
X forwarding allows you to run any application, including matlab,
remotely on teach.cs (CDF) and the output/display/plots/etc
be shown on your computer/laptop.
See
https://www.teach.cs.toronto.edu/using_cdf/x2go.html
for a way to have X forwarding on your computer/laptop.
The basic Matlab is free for students.
See https://www.mathworks.com/academia/tahportal/universityoftoronto676468.html
So you can also install it on your computer/laptop,
and run it locally.
You can also consider free (perpetually) alternatives to Matlab,
such as Octave.

To use matlab on cdf, through the linux shell, you need to login via ssh
(on an xterm or other terminal in unix, linux, mac, or cygwin,
and on putty in windows,
or through some free X forwarding application such as mobaxterm, or X2Go)
with a command such as
ssh X user@cdf.toronto.edu
or
ssh l user X f wolf.cdf.toronto.edu xterm
where ``user'' is your cdf username, then, once on cdf, run /usr/local/bin/matlab
or /usr/local/bin/matlab nodesktop
Within matlab, you may want to go to a certain directory, say ~/matlab,
and for this you can use the unix shell command cd ~/matlab
within matlab. You may also want to have a startup.m file
in that directory, to always run some standard commands
(e.g. format compact) every time you start matlab.

The textbook for this course is Michael Heath's,
Scientific Computing: an introductory survey,
SIAM 2018 (Revised Second Edition).
This is the same book used for CSC336 Spring 2023.
https://my.siam.org/Store/Product/viewproduct/?ProductId=30156464
The list price is approximately $94.00 USD, but
there is also a 30% discount for SIAM members
(students are able to take advantage of this if they signup
for a free SIAM Student membership).
https://www.siam.org/Membership/JoinSIAM/IndividualMembers/Student
This book was also published by McGrawHill Inc in the past,
but because the price was quite higher,
we ordered custommade copies through the UT bookstore,
which were cheaper.
This year we did not order those copies,
as the SIAM price is reasonable.
However, if you find those copies from past students,
they are equivalent (not identical, but equivalent) to the SIAM edition.
This book covers most of what is needed in the course.
Textbook web page (see educational modules)
Material to be covered in the course
(with textbook sections in parentheses)
20230912 (3 hours)
0. What is Scientific Computing? (1.1, 1.2.1)
1. Computer arithmetic; data and computational errors
1.1 (Human) Representation of nonnegative integers
 Algorithm for converting base b integers to decimal
 Algorithm for converting decimal integers to base b
1.2 (Human) Representation of reals
 Algorithm for converting base b fractions to decimal
 Algorithm for converting decimal fractions to base b
1.3 Computer representation of numbers (1.3.17)
 floatingpoint numbers, mantissa, exponent, normalized mantissa,
significant digits, overflow, underflow, range of representable numbers,
representable numbers, chopping, rounding
 The IEEE Standard
1.4 Roundoff error (1.3.5)
1.5 Absolute and relative errors (1.2.2)
1.6 Computer arithmetic  saturation in addition (1.3.8)
1.7 Machine epsilon (1.3.5)
1.8 Error propagation in simple arithmetic calculations (1.3.89)
 Multiplication, division, addition/subtraction
 catastrophic cancellation
1.9 Error propagation in computation: conditioning of problems (1.2.6)
 condition number of function
1.10 Error propagation in computation: stability of algorithms (1.2.7)
1.11 Forward and backward errors (1.2.35)
Propagated data error
Truncation (discretization) and rounding errors, computational error
Total error
1.12 Taylor series
20230919 (3 hours)
1.13 O(n^b) and O(h^a) notations
Tutorial on matlab
Tutorial on computer arithmetic
20230926 (3 hours)
2. Direct methods for solving square linear systems
2.1 Vectors and matrices  review of terminology and properties
2.2 Solving lower triangular linear systems [2.4.2]
Forward substitution (f/s)
2.3 Solving upper triangular linear systems [2.4.2]
Back substitution (b/s)
2.4 Equivalent linear systems  row operations [2.4.1]
2.5 An example of solving a linear system by GE and b/s
2.6 Gaussian elimination (GE) [2.4.3, 2.4.4, 2.4.67]
2.7 LU factorization [2.4.3, 2.4.4, 2.4.7]
elementary Gauss (elimination) transformation matrices
2.8 Symmetric and symmetric positive definite matrices [2.5.1, 2.5.2]
LDL^T and Choleski factorizations
2.9 Banded matrices [2.5.3]
Banded LU/GE and b/f/s
2.10 Computing the inverse of a matrix [2.4.7]
Properties of inverse
[more on elementary Gauss (elimination) transformation matrices]
20231003 (3 hours)
Properties of inverse (end)
Tutorial 3 (matrices, operation counts, GE/LU, inverse)
2.11 GE with partial pivoting [2.4.56]
breakdown or instability of GE, interchanges of rows, columns
types of pivoting
elementary Gauss transformation matrices; elementary permutation matrices
example
2.12 Scaled partial pivoting [2.4.10]
2.13 Complete pivoting
2.14 Effect of pivoting to special matrices  symmetry and bandedness
2.15 MATLAB and solution of linear systems
2.16 Mathematical software [2.7]
20231010 (3 hours)
Tutorial 4 (GE/LU pivoting)
2.17 Inner products
2.18 Norms [2.3]
2.19 Vector norms [2.3.1]
2.20 Matrix norms [2.3.2]
2.21 Condition number of a matrix [2.3.35]
Other interpretations of the condition number of a matrix
Tutorial 5 (norms and condition numbers of matrices), Q14
20231017 (3 hours)
Discussion on A1
Tutorial 5 (norms and condition numbers of matrices), Q510
3. Nonlinear equations and systems [5.1, 5.2]
roots, multiplicity
3.1 Fixed points and roots of functions [5.2]
systolic or contractive functions
3.2 Existence and uniqueness of root and fixed point [5.2]
Theorems 1, 2, 3, 4 with proofs
20231024 (3 hours)
midterm
Theorems 1, 2, 3, 4 with proofs (end)
3.3 Numerical methods for solving nonlinear equations [5.3, 5.4, 5.5]
Nonlinear solvers, residual, stopping criteria
3.4 Convergence rate of iterative methods [5.4]
3.5 The bisection method [5.5]
20231031 (3 hours)
3.6 Fixedpoint (functional) iteration methods [5.5]
Convergence of fixedpoint iteration (Th 4b, Th 5)
Rate of convergence of fixedpoint iteration (Th 6 with proof)
3.7 Newton's method (NewtonRaphson method) [5.5]
Rate of convergence of Newton's method
3.8 The secant method [5.5]
3.9 Numerical methods for solving nonlinear systems of equations [5.6]
Newton's method
3.10 Overview of iterative methods for nonlinear equations and systems
Fall break
20231114 (3 hours)
Tutorial 6 (nonlinear equations)
4 Interpolation
4.1 Approximation and interpolation  Introduction [7.1]
4.2 Polynomial approximation  Weierstrass theorem
4.3 Evaluating a polynomial  Horner's rule (nested multiplication) [7.3.1]
4.4 Polynomial interpolation using monomial basis functions [7.3.1]
4.5 Polynomial interpolation using Lagrange basis functions [7.3.2]
20231121 (3 hours)
4.6 Existence and uniqueness of polynomial interpolant [7.2]
4.7 Polynomial interpolation using Newton's basis functions
and the Divided Differences Table [7.3.3]
4.8 Comparison of the three bases
4.9 Error of the polynomial interpolant [7.3.5]
Tutorial 7 (polynomial interpolation)
4.10 Pitfalls of polynomial interpolation [7.3.5]
4.11 Piecewise polynomials and splines [7.4]
4.12 Linear spline interpolation (Lagrange form) [7.4.2]
Error in linear spline interpolation
20231128 (3 hours)
4.13 Cubic spline interpolation  choice of endconditions [7.4.2]
Error in cubic spline interpolation
4.14 Spline interpolation in MATLAB
4.15 Construction of a clamped cubic spline interpolant
4.16 Piecewise cubic Hermite interpolation
4.17 Overview of spline and pp interpolating methods
Tutorial 8 (piecewise polynomial and spline interpolation)
20231205 (2 hours)
extra note on Q4 of Tutorial 8
A2 discussion
summary
Notes and handouts:
Note on use of notes:
Notes will be available when the course starts.
While it may be convenient to study for the course by reading the notes,
it should be made clear that the notes are not there to substitute
the textbook or any relevant book. Notes are always more condensed
and give less overall information than books.
Notes with math notation, etc, are difficult to read online.
It may be preferable for some of you to print out the 4page style notes
on paper (preferably doublesided).
Access to the data below requires that
you type in your CDF (teaching labs) username
(same as UTorId)
and last 5 digits of your student number as password.
This password (for accessing the website) cannot be reset.
Lecture notes
 Computer Arithmetic and Errors,
1page,
[photo: hexadecimal, IEEE exponents],
[photo: IEEE ranges, correct in r digits],
[photo: condition number of function],
[photo: condition number of function, b/w and f/w errors]
 Vectors and matrices  terminology, properties,
1page,
f/b/s (forward and back substitutions) and GE,
1page,
LU factorisation on general and special matrices, inverse,
1page,
[photo: extra office hours, elementary permutation, LU factorization],
[photo: spd LDL^T, Choleski, banded f/b/s],
GE/LU with pivoting (partial, scaled partial, complete),
1page,
[extra: GE with pivoting, scaling],
[photo: misc A1, complete pivoting],
Norms and condition numbers of matrices,
1page,
[photo tut04q5: Choleski fact, lect: RCOND warning note: warning is of the form RCOND = 1e15],
[photo: Eucl. norm of orthogonal mat],

Nonlinear equations and systems,
1page,
see also hand written notes and
more hand written notes
[photo: nonlinear system, multiplicity, contraction, f(x) = 0 and x = g(x)],
[photo: IVT, MVT],
[photo: applicability of Thm 1, 2, 3],
[photo: applicability of Thm 4, residual, error],
[photo: order of convergence of sequence,
order of convergence of discretization method],
[photo: bisection],
[photo: rate of convergence of fixed point iteration, Newton's error],

Interpolation 1 (monomials),
1page,
[photo: evaluate interpolant on data points]
Interpolation 2 (Lagrange, Newton's, error, pitfalls),
1page,
[photo: multiple polynomial interpolants, order of convergence of sequence and order of convergence of discretization method],
Piecewise polynomial interpolation, splines,
1page,
[photo: discretization and rounding errors],
[photo: general spline interpolation, cubic endconditions]

Summary,
1page
Tutorial notes
 Tutorial on MATLAB, 1page
[photo: (tutmat) matrices, (tut01q2) sum geom series, (tut01q6) roots of quadratic],
 Tutorial 2, 1page,
[photo: (tutmat) matrices, (tut01q2) sum geom series, tut01q6 roots of quadratic],
[photo: diff (tut01q6) of cosines, (tut01q2) sum .aaa...],
[photo: diff of cosines, (tut01q8) exp(x)],
 Tutorial 3, 1page,
 Tutorial 4, 1page,
[photo: (tut04q4) misc spd, A^{1/2}],
[photo: (tut04q5) Choleski fact, (lect): RCOND warning note: warning is of the form RCOND = 1e15],
 Tutorial 5, 1page,
[photo: (tut05q2, q3, q4) condnum vs det, inv 2x2],
A1:
q1,
q1, q4,
q2,
q3,
 Tutorial 6, 1page,
[photo: q3],
[photo: q8]
 Tutorial 7, 1page,
 Tutorial 8, 1page,
[photo: q1, q4],
[photo: q4],
[photo: q4 extra note],
A2: [A2q3],
[A2q1],
[A2q1],
[A2q2],
[A2q2]
Assignments
Other