Course information for current students:
Selected notes:
Course information sheet
Material to be covered in lectures:
The sections mentioned correspond to textbooks found in the reference list.
no name designation = Yousef Saad, Iterative Methods for Sparse Linear Systems HaYo = L. A. Hageman and D. M. Young, Applied Iterative Methods GoVL = Gene Golub and Charles Van Loan, Matrix computations Ortega2 = J. M. Ortega, Matrix theory: a second course VL = Charles Van Loan, Computational Frameworks for the Fast Fourier Transform Ortega = J. M. Ortega, Parallel and Vector Solution of Linear Systems Briggs = William L. Briggs, A multigrid tutorial, SIAM Hackb = W. Hackbusch, Iterative Solution of Large Sparse Systems of Equations
2023-09-13 2 hrs 1. Introduction 1.1 Scope of the course 1.2 Vectors and matrices [1.1-3, 1.6-7, 1.11, 1.13.1, HaYo 1.2, GoVL 2.1, 2.7] 1.3 Eigenvalues and eigenvectors [1.2, 1.8-9, 1.11, HaYo 1.3] 1.4 Norms and inner products [1.4-5, 1.13.2, HaYo 1.4, GoVL 2.2-3] 2023-09-20 2 hrs Condition number of a matrix 1.5 Block matrices -- Partitioned matrices [1.3, HaYo 1.5, GoVL 1.3, 4.5] 1.6 One-dimensional boundary value problems -- A model problem 1.7 Two-dimensional boundary value problems -- A model problem [2.1-2, HaYo 1.6-7] 1.8 Stencils [2.2] 1.9 Tensor products of matrices [GoVL 4.5, Ortega2 6.3, VL 1.1.10] 2023-09-27 2 hrs 1.9 Tensor products of matrices [GoVL 4.5, Ortega2 6.3, VL 1.1.10] end 1.10 Finding eigenvalues/vectors of tridiagonal matrices with constant coefficients along the diagonal 2. Direct methods for solving linear systems 2.1 Gauss elimination, LU factorisation [GoVL 3.2] 2.2 Back and forward substitutions, solution of a linear system [GoVL 3.1] 2.3 Symmetric matrices, LDL^T decomposition, Choleski decomposition [GoVL 3.2] 2.4 Banded matrices, banded storage, 5-pt-star matrix [GoVL 3.2] 2.5 Pivoting, row, column, complete, symmetric, banded matrices [GoVL 3.4] 2.6 A mathematical description of the GE/LU algorithm [GoVL 3.2.5, 3.4, Ortega2 pgs 19-22] - Elementary Gauss transformations, no pivoting - Elementary permutation matrices, row pivoting, complete pivoting 2023-10-04 2 hrs 2.7 Sparse matrices and storage schemes [3.4-6] Yale, profile, Purdue storage schemes 2.8 Adjacency graphs [3.2] 3. Iterative methods for solving linear systems 3.1 General 3.2 Jacobi, Gauss-Seidel, SOR and SSOR methods [4.1, HaYo 2.3, GoVL 10.1] 3.3 Block iterative methods [4.1.1] 3.4 Convergence of vectors and matrices 3.5 Convergence of iterative linear solvers [1.8.4, 4.2.1] 2023-10-11 2 hrs 3.6 Rate of convergence of iterative methods [4.2.1, HaYo 2.2] Three "definitions" of rate/order of convergence 3.7 Convergence of basic iterative methods on special matrices Comparison of Jacobi and GS Diagonally dominant matrices [4.2.3] SPD matrices [4.2.4] Spectral radius of SOR Consistently ordered matrices - Optimal w for SOR [4.2.5] [For the above topics, also see Young, Iter. Sol. of Large Lin. Sys.] 2023-10-18 2 hrs 3.8 Preconditioning [4.1.2, 10.1-4, 10.6] Incomplete Factorisation preconditioning Block diagonal preconditioning SSOR preconditioning 3.9 Symmetrisable and extrapolated methods [HaYo 2.2] 3.10 Polynomial acceleration of iterative methods [HaYo 3.1-2] 2023-10-25 2 hrs 3.10 Polynomial acceleration of iterative methods [HaYo 3.1-2] (end) 3.11 Comparison of various direct and iterative methods for the model 2D BVP and the 5-point-star matrix -- Computational issues [~ HaYo 2.4] A1 discussion FR Fourier solvers (VL 1.1, 4.4-5) Discrete Fourier Transform (DFT) Matrix Discrete Fourier Transform (DFT) Fast Fourier Transform (FFT) -- various transforms The standard FFT algorithm 2023-11-01 2 hrs Solving one-dimensional BVPs using FFTs Tensor product form of discrete two-dimensional BVPs Solving two-dimensional BVPs using tensor products and FFTs - Diagonalization of matrices in tensor product form 2D-FFT algorithm for two-dimensional BVPs - Block-diagonalization of matrices in tensor product form 1D-FFT algorithm for two-dimensional BVPs - Other boundary conditions, variable coefs, etc 4. Minimisation methods -- Conjugate gradient acceleration 4.1 Introduction Gradient, Hessian, Directional derivative Descent direction, steepest descent direction 4.2 The steepest descent method [5.3.1, HaYo 7.2, GoVL 10.2.1] Properties of SD 2023-11-08 2 hrs prove: if the error at some iteration is an eigenvector of A, then CD converges in the next iteration prove: (r^(k+1), r^(k)) = 0 prove: ||e^(k+1)||_{A^{1/2}} <= alpha ||e^(k)||_{A^{1/2}} alpha = (kappa-1)/(kappa+1) 4.3 The family of Conjugate Direction (CD) methods [HaYo 7.2, GoVL 10.2.2-3] explain: how the stepsize lambda_k is chosen and why A-orthogonality helps prove: (d^(k), r^(0)) = (d^(k), r^(k)) The Gram-Schmidt orthogonalization method A type of QR factorization 4.4 The Conjugate Gradient method [6.7, HaYo 7.2-4, GoVL 10.2.4,6,7] A case of CD method A three-term recurrence relation for CG The computationally efficient CG algorithm Properties of CG Comparison of SD and CG 2023-11-15 2 hrs prove: (dk, rk+1) = 0, (dk, rk) = (rk, rk), (rj, rk) = 0, (dj, Adk) = 0 prove: the equivalence of the alternative defs for alphak and rk 4.5 The preconditioned CG (PCG) method [9.2, GoVL10.3.1, HaYo 7.4] DD Domain decomposition - Schur complement methods * Domain decomposition in 1D - ordering - arrowhead matrix * General banded matrix - ordering - arrowhead matrix * Schur complement - capacitance - Gauss transform system * Solving the arrowhead system * Domain decomposition in 2D * The use of CG in solving the Schur complement system [13.4] * Domain decomposition and preconditioning 2023-11-22 2 hrs 5. Iterative methods for general (including non-symmetric) systems 5.1 Introduction - Krylov subspace methods [6.1, 6.2, 6.3, 6.3.1] 5.2 Generalized Minimal Residual (GMRES) method 5.3 Restarted Generalized Minimal Residual method (GMRES(m)) [6.5] 5.4 Convergence of GMRES [6.11.4] 5.5 Minimal Residual Method (MRES or MinRes) [5.3.2] 5.6 Full Orthogonalization Method (FOM) [6.4] 5.7 Conjugate Residual (CR) method [6.8] 5.8 Other methods (GCR, Orthomin, Orthodir) [6.9] 5.9 Bi-orthogonal bases and related methods (BiCG, QMR, CGS, BiCGStab, TFQMR) [7.1, 7.3, 7.4] 5.10 Methods related to the normal equations [8.1, 8.3] 2023-11-29 2 hrs MG The multigrid method [Briggs, Hackb 10.x] Motivation for the two-level method The multi-level method Multigrid preconditioning Extension and restriction operators for the 1D model BVP Convergence of MG on the 1D model BVP V-cycle and full multigrid Extension and restriction operators for the 2D model BVP Computational procedures and complexity
Access to the data below requires that you type in your CDF (teaching labs) username (same as UTorId) and last 5 digits of your student number as password. This password (for accessing the website) cannot be reset.
Lecture notes
