- Aims
- Outline
- References
- Prerequisites
- Schedule for Fall 2022
- Marking scheme for Fall 2022
- Announcements for current students
(
**this is the site for currently registered students**)

- Introduce the basic concepts in parallel computation and state-of-the-art scientific computing.
- Formulate parallel numerical methods.
- Introduce selected advanced numerical methods.
- Implement the above methods on specific parallel architectures.
- Study the performance of methods and machines.

- Introduction
- Motivation for parallel computing
- Parallel architectures, communication complexity
- Concepts: granularity, degree of parallelism, speedup, efficiency, load balancing, data ready time
- Simple examples: adding two vectors of size n, summing up n numbers, inner product of two vectors, matrix-vector multiplication, total exchange
- Measuring and studying speedup and efficiency -- Amdahl's law -- Scaled speedup and efficiency

- Linear systems: Direct methods
- Gauss elimination, LU factorisation
- Fine grain LU factorisation, Data flow algorithm
- Symmetric and symmetric positive definite matrices, Cholesky decomposition
- Triangular systems, Back substitution
- Multiple right side vectors
- Banded systems, LU factorisation, Triangular banded systems, Pivoting in banded systems
- Tridiagonal systems, Cyclic reduction
- Banded Systems, Partitioning methods

- Introduction to Boundary Value Problems (BVPs)
- One dimensional model BVP -- simple finite difference method
- Two dimensional model BVP -- simple finite difference method

- Linear systems: Iterative methods
- Introduction
- Jacobi method, application to the 2D BVP
- Conjugate gradient method, application to the 2D BVP
- Preconditioning
- Preconditioned conjugate gradient method, application to the 2D BVP
- Gauss Seidel, SOR and SSOR methods
- Red-Black colouring, the GS, SOR and SSOR methods for the 2D BVP
- Multicolouring, the GS, SOR and SSOR methods for the 9-point-star stencil
- Asynchronous iterations
- Block iterative methods

- Partial Differential Equations
- Schur complement method, arrowhead matrix, application to the 1D BVP
- The use of CG for the solution of the Schur complement system
- Schur complement method, arrowhead matrix, application to the 2D BVP
- Schwarz alternating (splitting) method, preconditioning
- Multigrid method
- Fast Fourier Transform methods, application to the 1D BVP
- Tensor products of matrices
- FFT methods for the 2D BVP

- Interpolation
- Deboor decomposition

Parallel Computing |
||

James M. Ortega | Introduction to Parallel and Vector Solution of Linear Systems | Plenum Press 1988 |

Ian Foster | Designing and Building Parallel Programs | Addison Wesley 1995 |

George Em Karniadakis and Robert M. Kirby II | Parallel Scientific Computing in C++ and MPI A Seamless Approach to Parallel Algorithms and their Implementation | Cambridge 2003 |

J. M. Bahi, S. Contassot-Vivier and R. Couturier | Parallel Iterative Algorithms: from sequential to grid computing | Chapman & Hall/CRC 2007 |

William Gropp, Ewing Lusk and Anthony Skjellum | Using MPI: portable parallel programming with the message-passing interface | MIT Press 2014
see also Using MPI - web page |

Michael J. Quinn | Parallel Programming in C with MPI and OpenMP | McGraw Hill 2004 |

Jianping Zhu | Solving Partial Differential Equations on Parallel Computers | World Scientific 1994 |

Ananth Grama, Anshul Gupta, George Karypis and Vipin Kumar | Introduction to Parallel Computing: Design and Analysis of Algorithms | Addison Wesley 2003 |

Jeffrey D. Ullman | Computational Aspects of VLSI | Computer Science Press 1984 |

Dimitri P. Bertsekas and John N. Tsitsiklis | Parallel and Distributed Computation; Numerical Methods | Prentice Hall 1989
see also Parallel and Distributed Computation - web page |

Numerical Linear Algebra |
||

William W. Hager | Applied Numerical Linear Algebra | Prentice Hall 1988 |

Gene Golub and Charles Van Loan | Matrix computations | John Hopkins University Press 1996 |

General Numerical Analysis |
||

Uri Ascher and Chen Greif | A first course in Numerical Methods | SIAM 2011 (e-book on library) |

Samuel D. Conte and Carl de Boor | Elementary Numerical Analysis | SIAM 2018 (also McGraw-Hill Inc.) |

David Kincaid and Ward Cheney | Numerical Analysis | Brooks/Cole |

Michael Heath | Scientific Computing: an introductory survey | McGraw-Hill Inc. |

Richard L. Burden and J. Douglas Faires | Numerical Analysis | Brooks/Cole |

L. W. Johnson and R. D. Riess | Numerical Analysis | Addison Wesley |

Partial Differential Equations |
||

Michael A. Celia and William G. Gray | Numerical methods for differential equations | Prentice Hall 1992 |

William F. Ames | Numerical Methods for Partial Differential Equations | Academic Press 1977 3rd edition (or 2nd edition) (or Thomas Nelson & Sons) |

John C. Strikwerda | Finite Difference schemes and Partial Differential Equations | Wadsworth and Brooks/Cole 1989 |

Splines |
||

P. M. Prenter | Splines and Variational Methods | John Wiley & Sons 1975 |

Carl de Boor | A Practical Guide to Splines | Springer-Verlag 1978 |

Advanced Methods |
||

Yousef Saad | Iterative Methods for Sparse Linear Systems | PWS 1996 or SIAM 2003 |

Wolfgang Hackbusch | Iterative Solution of Large Sparse Systems of Equations | Springer Verlag 1994 |

William L. Briggs | A multigrid tutorial | SIAM 2000 |

- Elementary calculus: Taylor series, Rolle's theorem, mean value theorem, graphs of functions, continuity, convergence, de l' Hospital's rule, partial derivatives, etc.
- Numerical Linear Algebra (included in CSC336): knowledge of direct methods for solving linear systems. Fluency in matrix and vector manipulation, both algebraic and algorithmic.
- Interpolation (included in CSC336/CSC436): some knowledge on interpolation.
- Partial Differential Equations: minimal knowledge of PDEs.
- Theory of Computer Algorithms: some knowledge on computer algorithms, data structures and computational complexity.
- Programming: proficiency in some conventional programming language, preferably C/C++ or FORTRAN.

Lectures | Tuesday 1-3PM | Room BA 1230 |

Tutorial | Thursday 2-3PM | Room BA 2165 |

Office Hours | Wedensday 3:30-4:30PM | Room BA 4226 |

** Tentative marking scheme for Fall 2022 **

Problem set 1 | 20% |

Term test | 30% |

Problem set 2 | 25% |

Problem set 3 | 25% |