CSC456-2306F High-Performance Scientific Computing

• Aims
• Outline
• References
• Prerequisites
• Schedule for Fall 2006
• Marking scheme for Fall 2006
• Announcements for current students

• Introduce the basic concepts in parallel computation and state-of-the-art scientific computing.
• Formulate parallel numerical methods.
• Introduce selected advanced numerical methods.
• Implement the above methods on specific parallel architectures.
• Study the performance of methods and machines.

• Introduction
  • Motivation for parallel computing
  • Parallel architectures, communication complexity
  • Concepts: granularity, degree of parallelism, speedup, efficiency, load balancing, data ready time
  • Simple examples: adding two vectors of size n, summing up n numbers, inner product of two vectors, matrix-vector multiplication, total exchange
  • Measuring and studying speedup and efficiency -- Amdahl's law -- Scaled speedup and efficiency
• Linear systems: Direct methods
  • Gauss elimination, LU factorisation
  • Fine grain LU factorisation, Data flow algorithm
  • Symmetric and symmetric positive definite matrices, Cholesky decomposition
  • Triangular systems, Back substitution
  • Multiple right side vectors
  • Banded systems, LU factorisation, Triangular banded systems, Pivoting in banded systems
  • Tridiagonal systems, Cyclic reduction
  • Banded Systems, Partitioning methods
• Introduction to Boundary Value Problems (BVPs)
  • One dimensional model BVP -- simple finite difference method
  • Two dimensional model BVP -- simple finite difference method
• Linear systems: Iterative methods
  • Introduction
  • Jacobi method, application to the 2D BVP
  • Conjugate gradient method, application to the 2D BVP
  • Preconditioning
  • Preconditioned conjugate gradient method, application to the 2D BVP
  • Gauss Seidel, SOR and SSOR methods
  • Red-Black colouring, the GS, SOR and SSOR methods for the 2D BVP
  • Multicolouring, the GS, SOR and SSOR methods for the 9-point-star stencil
  • Asynchronous iterations
  • Block iterative methods
• Partial Differential Equations
  • Schur complement method, arrowhead matrix, application to the 1D BVP
  • The use of CG for the solution of the Schur complement system
  • Schur complement method, arrowhead matrix, application to the 2D BVP
  • Schwarz alternating (splitting) method, preconditioning
  • Multigrid method
  • Fast Fourier Transform methods, application to the 1D BVP
  • Tensor products of matrices
  • FFT methods for the 2D BVP
• Interpolation
  • Deboor decomposition

References

Parallel Computing
James M. Ortega	Introduction to Parallel and Vector Solution of Linear Systems	Plenum Press 1988
Jianping Zhu	Solving Partial Differential Equations on Parallel Computers	World Scientific 1994
Gene H. Golub and J. M. Ortega	Scientific computing: an introduction with parallel computing	Academic Press 1993
Ian Foster	Designing and Building Parallel Programs	Addison Wesley 1995
Vipin Kumar, Ananth Grama, Anshul Gupta and George Karypis	Introduction to Parallel Computing: Design and Analysis of Algorithms	Benjamin / Cummings 1994
Eric F. Van de Velde	Concurrent Scientific Computing Number 16 in Texts in Applied Mathematics	Springer-Verlag, New York 1994.
Dimitri P. Bertsekas and John N. Tsitsiklis	Parallel and Distributed Computation; Numerical Methods	Prentice Hall 1989
G. Fox, M. Johnson, G. Lyzenga, S. Otto, J. Salmon and D. Walker	Solving Problems on Concurrent Processors; General Techniques and Regular Problems	Prentice Hall 1989
Jeffrey D. Ullman	Computational Aspects of VLSI	Computer Science Press 1984
K. Hwang and Briggs	Computer Architecture and Parallel Processing	McGraw Hill 1984
S. Lakshmivarahan and Sudarshan K. Dhall	Analysis and Design of Parallel Algorithms: Arithmetic and Matrix Problems	McGraw-Hill 1990
Numerical Linear Algebra
William W. Hager	Applied Numerical Linear Algebra	Prentice Hall 1988
Gene Golub and Charles Van Loan	Matrix computations	John Hopkins University Press 1996
General Numerical Analysis
S. D. Conte and Carl de Boor	Elementary Numerical Analysis	McGraw-Hill Inc.
David Kincaid and Ward Cheney	Numerical Analysis	Brooks/Cole 2002
Michael Heath	Scientific Computing: an introductory survey	McGraw-Hill Inc. 1997
Richard L. Burden and J. Douglas Faires	Numerical Analysis	Brooks/Cole 1997, 6th edition
L. W. Johnson and R. D. Riess	Numerical Analysis	Addison Wesley
Partial Differential Equations
Michael A. Celia and William G. Gray	Numerical methods for differential equations	Prentice Hall 1992
William F. Ames	Numerical Methods for Partial Differential Equations	Academic Press 1977 3rd edition (or 2nd edition) (or Thomas Nelson & Sons)
John C. Strikwerda	Finite Difference schemes and Partial Differential Equations	Wadsworth and Brooks/Cole 1989
John R. Rice and R. F. Boisvert	Solving Elliptic Problems with ELLPACK	Springer-Verlag, New York 1985
Splines
P. M. Prenter	Splines and Variational Methods	John Wiley & Sons 1975
Carl de Boor	A Practical Guide to Splines	Springer-Verlag 1978
Advanced Methods
Yousef Saad	Iterative Methods for Sparse Linear Systems	PWS 1996 or SIAM 2003
Wolfgang Hackbusch	Iterative Solution of Large Sparse Systems of Equations	Springer Verlag 1994
William L. Briggs	A multigrid tutorial	SIAM 2000
Gene Golub and Charles Van Loan	Matrix computations	John Hopkins Univ. Press 1996

Prerequisites

• Elementary calculus: Taylor series, Rolle's theorem, mean value theorem, graphs of functions, continuity, convergence, de l' Hospital's rule, etc.
• Numerical Linear Algebra (included in CSC350): rough knowledge of direct and iterative methods for solving linear systems. Fluency in matrix and vector manipulation, both algebraic and algorithmic.
• Interpolation (included in CSC351): spline interpolation.
• Partial Differential Equations: minimal knowledge of PDEs.
• Theory of Computer Algorithms: minimal knowledge of computer algorithms, data structures and computational complexity.
• Programming: proficiency in some conventional programming language, preferably FORTRAN or C.

Schedule for Fall 2006

Lectures	Wednesday 4-6pm	Room BA4010
Tutorial	Monday 5-6pm	Room BA4010
Office Hours	Tuesday 1:30-2:30pm	Room BA 4226

Tutorials may be used for lectures.

Tentative marking scheme for Fall 2006

Problem set 1	20%
Problem set 2	20%
Problem set 3	25%
Term test 1	20%
Term test 2	15%

Most of the assignments include substantial computer work.