MPI is an interface or a set of library routines that are built on top of several common developing environments applicable to scientific computing such as FORTRAN, C and C++. For the reason of familiarity and ease of implementation on my part, we will proceed with C.
There are several implementations of MPI that can be used. It seems that mpich implementation by Mathematics and Computer Science (MCS) Division at Argonne National Laboratory of U.S. Department of Energy is widely used, and the setup is used with this particular implementation in the setup of our nodes used to implement the heat flow solver. In here, we will examine a few of essential library routines used in the programs for this project, and get into their syntax, as well as their semantics.
Nodes or processes that is spawned to run pieces of the entire program will need to be patched together (otherwise, we can just write serial programs, which in themselves are interesting, but not quite helpful in breaking up the problem into smaller pieces and solve them concurrently.) With C programming language with the running environment of UNIX or UNIX like operating systems, processes are used, so most of the time, the same piece of source code is used for all processes. so they will have the variables with the same names, and so forth, except they will be getting their own copy. Now, these processes eventually need to communicate to pass on boundary information between processes as well as coming up with a tangible result to be expressed at the end of the program execution. For more implementation specific issues, they are discussed in Chapter 4.
The following subsections will provide a few MPI routines that I have used in implementing parallel algorithms.