Course information for current students:
Material covered in lectures:
The sections mentioned correspond to textbooks found in the reference list.
no name designation = Yousef Saad, Iterative Methods for Sparse Linear Systems
HaYo = L. A. Hageman and D. M. Young, Applied Iterative Methods
GoVL = Gene Golub and Charles Van Loan, Matrix computations
Ortega2 = J. M. Ortega, Matrix theory: a second course
VL = Charles Van Loan, Computational Frameworks for the Fast Fourier Transform
Ortega = J. M. Ortega, Parallel and Vector Solution of Linear Systems
Briggs = A multigrid tutorial, SIAM
Hackb = W. Hackbusch, Iterative Solution of Large Sparse Systems of Equations
10-9-07 2 hours, handouts: course outline, notes pgs 1-14
1. Introduction
1.1 Scope of the course
1.2 Vectors and matrices [1.1-3, 1.6-7, 1.11, 1.13.1, HaYo 1.2, GoVL 2.1, 2.7]
1.3 Eigenvalues and eigenvectors [1.2, 1.8-9, 1.11, HaYo 1.3]
13-9-07 2 hours, handouts: notes pgs 17-41
1.4 Norms and inner products [1.4-5, 1.13.2, HaYo 1.4, GoVL 2.2-3]
1.5 Block matrices -- Partitioned matrices [1.3, HaYo 1.5, GoVL 1.3, 4.5]
1.6 One-dimensional Boundary value problems -- A model problem
1.7 Two-dimensional Boundary value problems -- A model problem [2.1-2, HaYo 1.6-7]
17-9-07 2 hours, handouts: notes pgs 45-49, 53-55
1.8 Stencils [2.2]
1.9 Tensor products of matrices [GoVL 4.5, Ortega2 6.3, VL 1.1.10]
1.10 Finding eigenvalues/vectors of tridiagonal matrices
with constant coefficients along the diagonal
For some of the above topics see also Ortega2 Chapters 1, 2 and 3.
20-9-07 2 hours, handouts: notes pgs 57-85
2. Direct methods for solving linear systems
2.1 LU factorisation [GoVL 3.2]
2.2 Back and forward substitutions [GoVL 3.1]
2.3 Symmetric matrices [GoVL 3.2]
2.4 Banded matrices [GoVL 3.2]
2.5 Pivoting [GoVL 3.4]
2.6 A mathematical description of the GE/LU algorithm [GoVL 3.2.5, 3.4,
Ortega2 pgs 19-22]
2.7 Sparse matrices and storage schemes [3.4-6]
24-9-07 2 hours, handouts: notes pgs 89-104
2.8 Adjacency graphs [3.2]
3. Iterative methods for solving linear systems
3.1 General
3.2 Jacobi, Gauss-Seidel, SOR and SSOR methods [4.1, HaYo 2.3, GoVL 10.1]
3.3 Block iterative methods [4.1.1]
27-9-07 2 hours, handouts: assignment 1
3.4 Convergence of vectors and matrices
3.5 Convergence of iterative linear solvers [1.8.4, 4.2.1]
3.6 Rate of convergence of iterative methods [4.2.1, HaYo 2.2]
Three "definitions" of rate/order of convergence
3.7 Convergence of basic iterative methods on special matrices
Comparison of Jacobi and GS
Diagonally dominant matrices [4.2.3]
SPD matrices [4.2.4]
Spectral radius of SOR
1-10-07 2 hours, handouts: notes 105-112
Consistently ordered matrices - Optimal w for SOR [4.2.5]
[For the above topics, also see Young, Iter. Sol. of Large Lin. Sys.]
A note on the use of the terms "rate" and "order" of convergence
3.8 Preconditioning [4.1.2, 10.1-4, 10.6]
Incomplete Factorisation preconditioning
Block diagonal preconditioning
SSOR preconditioning
4-10-07 2 hours, handouts: notes 113-117
3.9 Symmetrisable and extrapolated methods [HaYo 2.2]
3.10 Polynomial acceleration of iterative methods [HaYo 3.1-2]
3.11 Comparison of various direct and iterative methods for the model 2D BVP
and the 5-point-star matrix -- Computational issues [~ HaYo 2.4]
11-10-07 2 hours, handouts: notes 121-132, 133-151
4. Chebyshev (Tchebyshev) acceleration [HaYo 4.1-2, Saad 6.11.1, 12.3.2]
4.1 Chebyshev polynomials
4.2 Chebyshev acceleration
QR Least-squares problems and orthogonal transformations [1.7]
Overdetermined systems
The normal equations for overdetermined systems
Underdetermined systems
The normal equations for underdetermined systems
18-10-07 2 hours
[some questions on assignment 1]
Orthogonal transformations
Householder reflections
QR factorisation
Solving linear systems with QR
22-10-07 2 hours, handouts: assignment 2
discussion on assignment 1
25-10-07 2 hours, midterm
1-11-07 2 hours, handouts: notes 225-243
discussion on midtern
Gram-Schmidt orthogonalization
Solving linear systems with Gram-Schmidt
Review of factorizations
FR Fourier solvers (VL 1.1, 4.4-5)
Discrete Fourier Transform (DFT) Matrix
Discrete Fourier Transform (DFT)
Fast Fourier Transform (FFT) -- various transforms
The standard FFT algorithm
Fourier representation of functions -- Fourier Spectrum
Solving one-dimensional BVPs using FFTs
8-11-07 2 hours
Tensor products of matrices (revisited)
Tensor product form of discrete two-dimensional BVPs
Diagonalization and block-diagonalization of matrices in tensor product form
Solving two-dimensional BVPs using tensor products and FFTs
2D-FFT and 1D-FFT algorithms
15-11-07 2 hours, handouts: notes 153-168
6. Minimisation methods -- Conjugate gradient acceleration
6.1 Introduction
6.2 The steepest descent method [5.3.1, HaYo 7.2, GoVL 10.2.1]
Properties of SD
prove: if the error at some iteration is an eigenvector of A,
then CD converges in the next iteration
prove: (r^(k+1), r^(k)) = 0
prove: ||e^(k+1)||_{A^{1/2}} <= alpha ||e^(k)||_{A^{1/2}}
alpha = (kappa-1)/(kappa+1)
6.3 The family of Conjugate Direction (CD) methods [HaYo 7.2, GoVL 10.2.2-3]
explain: how the stepsize lambda_k is chosen and why A-orthogonality helps
prove: (d^(k), r^(0)) = (d^(k), r^(k))
22-11-07 2 hours, handouts: assignment 3, notes 169-172, 173-186
course evaluations
6.4 The Conjugate Gradient method [6.7, HaYo 7.2-4, GoVL 10.2.4,6,7]
A case of CD method
A three-term recurrence relation for CG
The computationally efficient CG algorithm
Properties of CG
prove: (dk, rk+1) = 0, (dk, rk) = (rk, rk), (rj, rk) = 0, (dj, Adk) = 0
prove: the equivalence of the alternative defs for alphak and rk
6.5 The preconditioned CG (PCG) method [9.2, GoVL10.3.1, HaYo 7.4]
7. Iterative methods for general (including non-symmetric) systems
7.1 Introduction - Krylov subspace methods [6.1, 6.2, 6.3, 6.3.1]
7.2 Generalized Minimal Residual (GMRES) method
29-11-07 2 hours
7.3 Restarted Generalized Minimal Residual method (GMRES(m)) [6.5]
7.4 Convergence of GMRES [6.11.4]
7.5 Full Orthogonalization Method (FOM) [6.4]
7.6 Conjugate Residual (CR) method [6.8]
7.7 Other methods (GCR, Orthomin, Orthodir) [6.9]
7.8 Bi-orthogonal bases and related methods (BiCG, QMR, CGS, BiCGStab, TFQMR)
[7.1, 7.3, 7.4]
7.9 Methods related to the normal equations [8.1, 8.3]
6-12-07 2 hours, handouts: notes 189-208
8. Methods motivated by PDEs
8.1 The multigrid method [Briggs, Hackb 10.x]
Motivation for the two-level method
The multi-level method
Multigrid preconditioning
Extension and restriction operators for the 1D model BVP
Convergence of MG on the 1D model BVP
V-cycle and full multigrid
Extension and restriction operators for the 2D model BVP
Computational procedures and complexity
Parallel Computation
To be done next:
Selected notes:
1a=def_4.ps,
1b=eig_4.ps,
1c=norm_4.ps,
1d=block_4.ps,
1e=bvp_4.ps,
1g=tensor_4.ps,
1h=teig_4.ps,
To access these notes use the username you gave me in class, and as password the last 5 digits of the student id number you have me in class.