CSC456-2306F High-Performance Scientific Computing

The first meeting of the course is Thursday, September 8, 2022, 2-3 PM.

• Aims
• Outline
• References
• Prerequisites
• Schedule for Fall 2022
• Marking scheme for Fall 2022
• Announcements for current students (this is the site for currently registered students)

• Introduce the basic concepts in parallel computation and state-of-the-art scientific computing.
• Formulate parallel numerical methods.
• Introduce selected advanced numerical methods.
• Implement the above methods on specific parallel architectures.
• Study the performance of methods and machines.

• Introduction
  • Motivation for parallel computing
  • Parallel architectures, communication complexity
  • Concepts: granularity, degree of parallelism, speedup, efficiency, load balancing, data ready time
  • Simple examples: adding two vectors of size n, summing up n numbers, inner product of two vectors, matrix-vector multiplication, total exchange
  • Measuring and studying speedup and efficiency -- Amdahl's law -- Scaled speedup and efficiency
• Linear systems: Direct methods
  • Gauss elimination, LU factorisation
  • Fine grain LU factorisation, Data flow algorithm
  • Symmetric and symmetric positive definite matrices, Cholesky decomposition
  • Triangular systems, Back substitution
  • Multiple right side vectors
  • Banded systems, LU factorisation, Triangular banded systems, Pivoting in banded systems
  • Tridiagonal systems, Cyclic reduction
  • Banded Systems, Partitioning methods
• Introduction to Boundary Value Problems (BVPs)
  • One dimensional model BVP -- simple finite difference method
  • Two dimensional model BVP -- simple finite difference method
• Linear systems: Iterative methods
  • Introduction
  • Jacobi method, application to the 2D BVP
  • Conjugate gradient method, application to the 2D BVP
  • Preconditioning
  • Preconditioned conjugate gradient method, application to the 2D BVP
  • Gauss Seidel, SOR and SSOR methods
  • Red-Black colouring, the GS, SOR and SSOR methods for the 2D BVP
  • Multicolouring, the GS, SOR and SSOR methods for the 9-point-star stencil
  • Asynchronous iterations
  • Block iterative methods
• Partial Differential Equations
  • Schur complement method, arrowhead matrix, application to the 1D BVP
  • The use of CG for the solution of the Schur complement system
  • Schur complement method, arrowhead matrix, application to the 2D BVP
  • Schwarz alternating (splitting) method, preconditioning
  • Multigrid method
  • Fast Fourier Transform methods, application to the 1D BVP
  • Tensor products of matrices
  • FFT methods for the 2D BVP
• Interpolation
  • Deboor decomposition

References

Parallel Computing
James M. Ortega	Introduction to Parallel and Vector Solution of Linear Systems	Plenum Press 1988
Ian Foster	Designing and Building Parallel Programs	Addison Wesley 1995
George Em Karniadakis and Robert M. Kirby II	Parallel Scientific Computing in C++ and MPI A Seamless Approach to Parallel Algorithms and their Implementation	Cambridge 2003
J. M. Bahi, S. Contassot-Vivier and R. Couturier	Parallel Iterative Algorithms: from sequential to grid computing	Chapman & Hall/CRC 2007
William Gropp, Ewing Lusk and Anthony Skjellum	Using MPI: portable parallel programming with the message-passing interface	MIT Press 2014 see also Using MPI - web page
Michael J. Quinn	Parallel Programming in C with MPI and OpenMP	McGraw Hill 2004
Jianping Zhu	Solving Partial Differential Equations on Parallel Computers	World Scientific 1994
Ananth Grama, Anshul Gupta, George Karypis and Vipin Kumar	Introduction to Parallel Computing: Design and Analysis of Algorithms	Addison Wesley 2003
Jeffrey D. Ullman	Computational Aspects of VLSI	Computer Science Press 1984
Dimitri P. Bertsekas and John N. Tsitsiklis	Parallel and Distributed Computation; Numerical Methods	Prentice Hall 1989 see also Parallel and Distributed Computation - web page
Numerical Linear Algebra
William W. Hager	Applied Numerical Linear Algebra	Prentice Hall 1988
Gene Golub and Charles Van Loan	Matrix computations	John Hopkins University Press 1996
General Numerical Analysis
Uri Ascher and Chen Greif	A first course in Numerical Methods	SIAM 2011 (e-book on library)
Samuel D. Conte and Carl de Boor	Elementary Numerical Analysis	SIAM 2018 (also McGraw-Hill Inc.)
David Kincaid and Ward Cheney	Numerical Analysis	Brooks/Cole
Michael Heath	Scientific Computing: an introductory survey	McGraw-Hill Inc.
Richard L. Burden and J. Douglas Faires	Numerical Analysis	Brooks/Cole
L. W. Johnson and R. D. Riess	Numerical Analysis	Addison Wesley
Partial Differential Equations
Michael A. Celia and William G. Gray	Numerical methods for differential equations	Prentice Hall 1992
William F. Ames	Numerical Methods for Partial Differential Equations	Academic Press 1977 3rd edition (or 2nd edition) (or Thomas Nelson & Sons)
John C. Strikwerda	Finite Difference schemes and Partial Differential Equations	Wadsworth and Brooks/Cole 1989
Splines
P. M. Prenter	Splines and Variational Methods	John Wiley & Sons 1975
Carl de Boor	A Practical Guide to Splines	Springer-Verlag 1978
Advanced Methods
Yousef Saad	Iterative Methods for Sparse Linear Systems	PWS 1996 or SIAM 2003
Wolfgang Hackbusch	Iterative Solution of Large Sparse Systems of Equations	Springer Verlag 1994
William L. Briggs	A multigrid tutorial	SIAM 2000

Prerequisites

• Elementary calculus: Taylor series, Rolle's theorem, mean value theorem, graphs of functions, continuity, convergence, de l' Hospital's rule, partial derivatives, etc.
• Numerical Linear Algebra (included in CSC336): knowledge of direct methods for solving linear systems. Fluency in matrix and vector manipulation, both algebraic and algorithmic.
• Interpolation (included in CSC336/CSC436): some knowledge on interpolation.
• Partial Differential Equations: minimal knowledge of PDEs.
• Theory of Computer Algorithms: some knowledge on computer algorithms, data structures and computational complexity.
• Programming: proficiency in some conventional programming language, preferably C/C++ or FORTRAN.

Schedule for Fall 2022

Lectures	Tuesday 1-3PM	Room BA 1230
Tutorial	Thursday 2-3PM	Room BA 2165
Office Hours	Wedensday 3:30-4:30PM	Room BA 4226

Tutorials may be used for lectures.

Tentative marking scheme for Fall 2022

Problem set 1	20%
Term test	30%
Problem set 2	25%
Problem set 3	25%

The assignments include substantial computer work.