CSC2321S Matrix Calculations

Spring 2022 Bulletin board for csc2321 spring 2022 -- course outline -- MarkUs

Course information for current students:

Selected notes:
Course information sheet

Material to be covered in lectures:
The sections mentioned correspond to textbooks found in the reference list.

no name designation = Yousef Saad, Iterative Methods for Sparse Linear Systems
HaYo = L. A. Hageman and D. M. Young, Applied Iterative Methods
GoVL = Gene Golub and Charles Van Loan, Matrix computations
Ortega2 = J. M. Ortega, Matrix theory: a second course
VL = Charles Van Loan, Computational Frameworks for the Fast Fourier Transform
Ortega = J. M. Ortega, Parallel and Vector Solution of Linear Systems
Briggs = William L. Briggs, A multigrid tutorial, SIAM
Hackb = W. Hackbusch, Iterative Solution of Large Sparse Systems of Equations

2022-01-14 2 hrs 1. Introduction 1.1 Scope of the course 1.2 Vectors and matrices [1.1-3, 1.6-7, 1.11, 1.13.1, HaYo 1.2, GoVL 2.1, 2.7] 1.3 Eigenvalues and eigenvectors [1.2, 1.8-9, 1.11, HaYo 1.3] 1.4 Norms and inner products [1.4-5, 1.13.2, HaYo 1.4, GoVL 2.2-3] Condition number of a matrix 2022-01-21 2 hrs 1.5 Block matrices -- Partitioned matrices [1.3, HaYo 1.5, GoVL 1.3, 4.5] 1.6 One-dimensional boundary value problems -- A model problem 1.7 Two-dimensional boundary value problems -- A model problem [2.1-2, HaYo 1.6-7] 1.8 Stencils [2.2] 1.9 Tensor products of matrices [GoVL 4.5, Ortega2 6.3, VL 1.1.10] 2022-01-28 2 hrs 1.10 Finding eigenvalues/vectors of tridiagonal matrices with constant coefficients along the diagonal 2. Direct methods for solving linear systems 2.1 Gauss elimination, LU factorisation [GoVL 3.2] 2.2 Back and forward substitutions, solution of a linear system [GoVL 3.1] 2.3 Symmetric matrices, LDL^T decomposition, Choleski decomposition [GoVL 3.2] 2.4 Banded matrices, banded storage, 5-pt-star matrix [GoVL 3.2] 2.5 Pivoting, row, column, complete, symmetric, banded matrices [GoVL 3.4] 2.6 A mathematical description of the GE/LU algorithm [GoVL 3.2.5, 3.4, Ortega2 pgs 19-22] - Elementary Gauss transformations, no pivoting 2022-02-04 2 hrs - Elementary permutation matrices, row pivoting, complete pivoting 2.7 Sparse matrices and storage schemes [3.4-6] Yale, profile, Purdue storage schemes 2.8 Adjacency graphs [3.2] 3. Iterative methods for solving linear systems 3.1 General 3.2 Jacobi, Gauss-Seidel, SOR and SSOR methods [4.1, HaYo 2.3, GoVL 10.1] 3.3 Block iterative methods [4.1.1] 3.4 Convergence of vectors and matrices 3.5 Convergence of iterative linear solvers [1.8.4, 4.2.1] 3.6 Rate of convergence of iterative methods [4.2.1, HaYo 2.2] 2022-02-11 2 hrs Three "definitions" of rate/order of convergence 3.7 Convergence of basic iterative methods on special matrices Comparison of Jacobi and GS Diagonally dominant matrices [4.2.3] SPD matrices [4.2.4] Spectral radius of SOR Consistently ordered matrices - Optimal w for SOR [4.2.5] [For the above topics, also see Young, Iter. Sol. of Large Lin. Sys.] 3.8 Preconditioning [4.1.2, 10.1-4, 10.6] Incomplete Factorisation preconditioning Block diagonal preconditioning SSOR preconditioning 3.9 Symmetrisable and extrapolated methods [HaYo 2.2] 2022-02-18 2 hrs 3.10 Polynomial acceleration of iterative methods [HaYo 3.1-2] 3.11 Comparison of various direct and iterative methods for the model 2D BVP and the 5-point-star matrix -- Computational issues [~ HaYo 2.4] A1 discussion FR Fourier solvers (VL 1.1, 4.4-5) Discrete Fourier Transform (DFT) Matrix Discrete Fourier Transform (DFT) Fast Fourier Transform (FFT) -- various transforms The standard FFT algorithm Solving one-dimensional BVPs using FFTs 2022-03-04 1.5 hrs midterm 2022-03-11 2 hrs midterm discussion (1 hr) Tensor product form of discrete two-dimensional BVPs Solving two-dimensional BVPs using tensor products and FFTs - Diagonalization of matrices in tensor product form 2D-FFT algorithm for two-dimensional BVPs 2022-03-18 2 hrs 4. Minimisation methods -- Conjugate gradient acceleration 4.1 Introduction Gradient, Hessian, Directional derivative Descent direction, steepest descent direction 4.2 The steepest descent method [5.3.1, HaYo 7.2, GoVL 10.2.1] Properties of SD prove: if the error at some iteration is an eigenvector of A, then CD converges in the next iteration prove: (r^(k+1), r^(k)) = 0 prove: ||e^(k+1)||_{A^{1/2}} <= alpha ||e^(k)||_{A^{1/2}} alpha = (kappa-1)/(kappa+1) 4.3 The family of Conjugate Direction (CD) methods [HaYo 7.2, GoVL 10.2.2-3] explain: how the stepsize lambda_k is chosen and why A-orthogonality helps prove: (d^(k), r^(0)) = (d^(k), r^(k)) The Gram-Schmidt orthogonalization method 2022-03-25 2 hrs A type of QR factorization 4.4 The Conjugate Gradient method [6.7, HaYo 7.2-4, GoVL 10.2.4,6,7] A case of CD method A three-term recurrence relation for CG The computationally efficient CG algorithm Properties of CG prove: (dk, rk+1) = 0, (dk, rk) = (rk, rk), (rj, rk) = 0, (dj, Adk) = 0 prove: the equivalence of the alternative defs for alphak and rk 4.5 The preconditioned CG (PCG) method [9.2, GoVL10.3.1, HaYo 7.4] 5. Iterative methods for general (including non-symmetric) systems 5.1 Introduction - Krylov subspace methods [6.1, 6.2, 6.3, 6.3.1] 5.2 Generalized Minimal Residual (GMRES) method 2022-04-01 2 hrs 5.2 Generalized Minimal Residual (GMRES) method (end) 5.3 Restarted Generalized Minimal Residual method (GMRES(m)) [6.5] 5.4 Convergence of GMRES [6.11.4] 5.5 Minimal Residual Method (MRES or MinRes) [5.3.2] 5.6 Full Orthogonalization Method (FOM) [6.4] 5.7 Conjugate Residual (CR) method [6.8] 5.8 Other methods (GCR, Orthomin, Orthodir) [6.9] 5.9 Bi-orthogonal bases and related methods (BiCG, QMR, CGS, BiCGStab, TFQMR) [7.1, 7.3, 7.4] 5.10 Methods related to the normal equations [8.1, 8.3] 2022-04-08 2 hrs MG The multigrid method [Briggs, Hackb 10.x] Motivation for the two-level method The multi-level method Multigrid preconditioning Extension and restriction operators for the 1D model BVP Convergence of MG on the 1D model BVP V-cycle and full multigrid Extension and restriction operators for the 2D model BVP Computational procedures and complexity

To access these notes use the username for your teaching lab (cdf) account, which ought to be the same as your UTorId, and as password the last 5 digits of your student id number. This password for accesing the notes cannot be reset.

Lecture notes

Assignments Recordings (links and passwords in quercus)