### CSC456-2306S High-Performance Scientific Computing

Aims
• Introduce the basic concepts in parallel computation and state-of-the-art scientific computing.
• Formulate parallel numerical methods.
• Introduce selected advanced numerical methods.
• Implement the above methods on specific parallel architectures.
• Study the performance of methods and machines.
Outline
• Introduction
• Motivation for parallel computing
• Parallel architectures, communication complexity
• Concepts: granularity, degree of parallelism, speedup, efficiency, load balancing, data ready time
• Simple examples: adding two vectors of size n, summing up n numbers, inner product of two vectors, matrix-vector multiplication, total exchange
• Measuring and studying speedup and efficiency -- Amdahl's law -- Scaled speedup and efficiency
• Linear systems: Direct methods
• Gauss elimination, LU factorisation
• Fine grain LU factorisation, Data flow algorithm
• Symmetric and symmetric positive definite matrices, Cholesky decomposition
• Triangular systems, Back substitution
• Multiple right side vectors
• Banded systems, LU factorisation, Triangular banded systems, Pivoting in banded systems
• Tridiagonal systems, Cyclic reduction
• Banded Systems, Partitioning methods
• Introduction to Boundary Value Problems (BVPs)
• One dimensional model BVP -- simple finite difference method
• Two dimensional model BVP -- simple finite difference method
• Linear systems: Iterative methods
• Introduction
• Jacobi method, application to the 2D BVP
• Conjugate gradient method, application to the 2D BVP
• Preconditioning
• Preconditioned conjugate gradient method, application to the 2D BVP
• Gauss Seidel, SOR and SSOR methods
• Red-Black colouring, the GS, SOR and SSOR methods for the 2D BVP
• Multicolouring, the GS, SOR and SSOR methods for the 9-point-star stencil
• Asynchronous iterations
• Block iterative methods
• Partial Differential Equations
• Schur complement method, arrowhead matrix, application to the 1D BVP
• The use of CG for the solution of the Schur complement system
• Schur complement method, arrowhead matrix, application to the 2D BVP
• Schwarz alternating (splitting) method, preconditioning
• Multigrid method
• Fast Fourier Transform methods, application to the 1D BVP
• Tensor products of matrices
• FFT methods for the 2D BVP
• Interpolation
• Deboor decomposition

References
 Parallel Computing James M. Ortega Introduction to Parallel and Vector Solution of Linear Systems Plenum Press 1988 Ian Foster Designing and Building Parallel Programs Addison Wesley 1995 George Em Karniadakis and Robert M. Kirby II Parallel Scientific Computing in C++ and MPI A Seamless Approach to Parallel Algorithms and their Implementation Cambridge 2003 J. M. Bahi, S. Contassot-Vivier and R. Couturier Parallel Iterative Algorithms: from sequential to grid computing Chapman & Hall/CRC 2007 William Gropp, Ewing Lusk and Anthony Skjellum Using MPI: portable parallel programming with the message-passing interface MIT Press 2014 see also Using MPI - web page Michael J. Quinn Parallel Programming in C with MPI and OpenMP McGraw Hill 2004 Jianping Zhu Solving Partial Differential Equations on Parallel Computers World Scientific 1994 Ananth Grama, Anshul Gupta, George Karypis and Vipin Kumar Introduction to Parallel Computing: Design and Analysis of Algorithms Addison Wesley 2003 Jeffrey D. Ullman Computational Aspects of VLSI Computer Science Press 1984 Dimitri P. Bertsekas and John N. Tsitsiklis Parallel and Distributed Computation; Numerical Methods Prentice Hall 1989 see also Parallel and Distributed Computation - web page Numerical Linear Algebra William W. Hager Applied Numerical Linear Algebra Prentice Hall 1988 Gene Golub and Charles Van Loan Matrix computations John Hopkins University Press 1996 General Numerical Analysis Uri Ascher and Chen Greif A first course in Numerical Methods SIAM 2011 (e-book on library) Samuel D. Conte and Carl de Boor Elementary Numerical Analysis SIAM 2018 (also McGraw-Hill Inc.) David Kincaid and Ward Cheney Numerical Analysis Brooks/Cole Michael Heath Scientific Computing: an introductory survey McGraw-Hill Inc. Richard L. Burden and J. Douglas Faires Numerical Analysis Brooks/Cole L. W. Johnson and R. D. Riess Numerical Analysis Addison Wesley Partial Differential Equations Michael A. Celia and William G. Gray Numerical methods for differential equations Prentice Hall 1992 William F. Ames Numerical Methods for Partial Differential Equations Academic Press 1977 3rd edition (or 2nd edition) (or Thomas Nelson & Sons) John C. Strikwerda Finite Difference schemes and Partial Differential Equations Wadsworth and Brooks/Cole 1989 Splines P. M. Prenter Splines and Variational Methods John Wiley & Sons 1975 Carl de Boor A Practical Guide to Splines Springer-Verlag 1978 Advanced Methods Yousef Saad Iterative Methods for Sparse Linear Systems PWS 1996 or SIAM 2003 Wolfgang Hackbusch Iterative Solution of Large Sparse Systems of Equations Springer Verlag 1994 William L. Briggs A multigrid tutorial SIAM 2000

• Elementary calculus: Taylor series, Rolle's theorem, mean value theorem, graphs of functions, continuity, convergence, de l' Hospital's rule, partial derivatives, etc.
• Numerical Linear Algebra (included in CSC336): knowledge of direct methods for solving linear systems. Fluency in matrix and vector manipulation, both algebraic and algorithmic.
• Interpolation (included in CSC336/CSC436): some knowledge on interpolation.
• Partial Differential Equations: minimal knowledge of PDEs.
• Theory of Computer Algorithms: some knowledge on computer algorithms, data structures and computational complexity.
• Programming: proficiency in some conventional programming language, preferably C/C++ or FORTRAN.

Schedule for Winter 2021
 Lectures Wednesday 1-3PM Room Virtual Tutorial Thursday 3-4PM Room Virtual Office Hours Monday 3:30-4:30PM Room Virtual
Tutorials may be used for lectures.

Tentative marking scheme for Winter 2021
 Problem set 0 8% Problem set 1 12% Problem set 2 20% Problem set 3 20% Term test 1 20% Term test 2 20%
Most of the assignments include substantial computer work.