### CSC2321F Matrix Calculations

Aims
• Review the basic concepts in the numerical solution of linear systems.
• Introduce state-of-the-art developments in numerical linear algebra / PDEs.
• Develop and study efficient linear solvers, with focus on solvers for large sparse linear systems.
• Implement the above solvers as software.
• Use existing software and higher level environments.
• Analyze the performance of methods and software.
Outline
• Introduction
• Motivation
• Vectors and matrices
• Eigenvalues and eigenvectors
• Norms and inner products
• Block matrices
• Boundary value problems (1-dim) and stencils
• Boundary value problems (2-dim) and stencils
• Stencils and sparsity patterns
• Tensor products of matrices
• Direct methods for solving linear systems
• Gauss elimination, LU factorisation, back and forward substitutions
• Symmetric matrices, symmetric positive definite matrices, Cholesky factorisation
• Banded matrices
• Pivoting
• Sparse matrix storage schemes
• Adjacency graphs and irreducibility
• Iterative methods for solving linear systems
• Introduction
• Richardson, Jacobi, Gauss-Seidel, SOR and SSOR methods
• Block methods
• Convergence of matrices and vectors
• Convergence of iterative methods
• Rate of convergence of iterative methods
• Convergence theorems: Comparison of Jacobi and GS, Diagonal dominant matrices, SPD matrices, Spectral radius of SOR, Consistently ordered matrices - Optimal w for SOR
• Rates of convergence of basic iterative methods on the model problem, Computational issues
• Preconditioning
• Symmetrisable and extrapolated methods
• Polynomial acceleration
• Chebyshev acceleration
• Chebyshev polynomials
• Chebyshev acceleration
• Conjugate gradient acceleration
• The steepest descent method
• The family of Conjugate Direction methods
• The Conjugate Gradient method
• A three-term recurrence relation for CG
• The preconditioned CG method
• Methods for red-black partitioned matrices
• Red-black partitioned matrices
• Chebyshev and CG methods
• Cyclic semi-iterative and CG methods
• Reduced system semi-iterative and CG methods
• Gauss-Seidel and related methods
• The red-black ordering for the 5-point-star matrix
• Multicolor ordering and parallel computation
• Partial Differential Equations
• Schur complement method, arrowhead matrix, application to the 1D BVP
• The use of CG for the solution of the Schur complement system
• Schur complement method, arrowhead matrix, application to the 2D BVP
• Schwarz alternating (splitting) method, preconditioning
• Multigrid method, two- and multi-level method, preconditioning, extension and restriction operators, convergence, V-cycle and full MG
• Fast Fourier Transform methods, application to the 1D BVP
• FFT methods for the 2D BVP; diagonalization and block-diagonalization
• Interpolation
• Deboor decomposition
• Iterative methods for general (including non-symmetric) systems
• Introduction - Krylov subspace methods
• Generalized Minimal Residual (GMRES) method
• Restarted Generalized Minimal Residual method (GMRES(m))
• Convergence of GMRES
• Full Orthogonalization Method (FOM)
• Conjugate Residual (CR) method
• Other methods (GCR, Orthomin, Orthodir)
• Bi-orthogonal bases and related methods (BiCG, QMR, CGS, BiCGStab, TFQMR)

References
 Numerical Linear Algebra L. A. Hageman and D. M. Young Applied Iterative Methods Academic Press 1981 R. S. Varga Matrix iterative analysis Prentice Hall 1962 D. M. Young Iterative Solution of Large Linear Systems Academic Press 1971 J. M. Ortega Matrix theory: a second course Plenum Press 1987 Gilbert W. Stewart Introduction to matrix computations Academic Press 1973 William W. Hager Applied Numerical Linear Algebra Prentice Hall 1988 Gene Golub and Charles Van Loan Matrix computations John Hopkins Univ. Press 1996 Advanced Methods Yousef Saad Iterative Methods for Sparse Linear Systems SIAM 2003 (PWS 1996) Wolfgang Hackbusch Iterative Solution of Large Sparse Systems of Equations Springer Verlag 1994 William L. Briggs A multigrid tutorial SIAM 2000 Charles Van Loan Computational Frameworks for the Fast Fourier Transform SIAM 1992 O. Axelsson and V. A. Barker Finite element solution of boundary value problems Academic Press 1984 Parallel Computing James M. Ortega Introduction to Parallel and Vector Solution of Linear Systems Plenum Press 1988 Eric F. Van de Velde Concurrent Scientific Computing Number 16 in Texts in Applied Mathematics Springer Verlag 1994 Gene H. Golub and J. M. Ortega Scientific computing: an introduction with parallel computing Academic Press 1993 General Numerical Analysis S. D. Conte and Carl de Boor Elementary Numerical Analysis McGraw-Hill Inc. David Kincaid and Ward Cheney Numerical Analysis Brooks/Cole 2002 (1996) Michael Heath Scientific Computing: an introductory survey McGraw-Hill Inc. 2003 Richard L. Burden and J. Douglas Faires Numerical Analysis Brooks/Cole 2001 (1997) L. W. Johnson and R. D. Riess Numerical Analysis Addison Wesley Partial Differential Equations Arieh Iserles A first course in the Numerical Analysis of Partial Differential Equations Cambridge University Press 1996 Michael A. Celia and William G. Gray Numerical methods for differential equations Prentice Hall 1992 William F. Ames Numerical Methods for Partial Differential Equations Academic Press 1977 3rd edition (or 2nd edition) (or Thomas Nelson & Sons) C. A. Hall and T. A. Porsching Numerical Analysis of Partial Differential Equations Prentice Hall 1990 John C. Strikwerda Finite Difference schemes and Partial Differential Equations Wadsworth and Brooks/Cole 1989 John R. Rice and R. F. Boisvert Solving Elliptic Problems with ELLPACK Springer Verlag 1985 Splines P. M. Prenter Splines and Variational Methods John Wiley & Sons 1975 Carl de Boor A Practical Guide to Splines Springer Verlag 1978

• Calculus: Taylor series, Rolle's theorem, mean value theorem, graphs of functions, continuity, convergence, de l' Hospital's rule, partial differentiation, etc.
• Numerical Linear Algebra (included in CSC350/336): some knowledge of direct and iterative methods for solving linear systems. Fluency in matrix and vector manipulation, both algebraic and algorithmic.
• Interpolation (included in CSC351/436): spline interpolation.
• Partial Differential Equations: minimal knowledge of PDEs.
• Theory of Computer Algorithms: minimal knowledge of computer algorithms, data structures and computational complexity.
• Programming: while most programming is in MATLAB, proficiency in some conventional programming language, such as FORTRAN or C, especially the handling of matrices, vectors, indices, etc., is assumed; knowledge of MATLAB.

Schedule for Fall 2019
 Lectures Tuesday 1-3 Room BA 2179 Office Hours Monday 3:30-4:30 (other hours by appointment) Room BA 4226

Tentative marking scheme for Fall 2019
 Problem set 1 25% Term test 25% Problem set 2 25% Problem set 3 25%
Most assignments include substantial computer work.