Parallel Computing

MatlabMPI

• Code: MatlabMPI was originally written by Dr. Jeremy Kepner at MIT's Lincoln Laboratory. I have worked on it further to improve the MPI_Bcast function, and include a new function, MPI_Reduce. You can download my version of MatlabMPI here.
• Examples: Here is an example of K-means clustering implemented using MatlabMPI. This example uses as data the test set from the MNIST digit dataset. You can download the necessary .mat file here: mnist_test.zip
  
  A more sophisticated example using MPI for Multiple Cause Vector Quantization is available here: mcvq_mpi.m.
• Using MatlabMPI on CSLAB: need to include acdirmin=0.1 as an NFS mount parameter for the drive used to store messages
• Tips:
  • Program starts, but nothing happens: Perhaps one of the workers has encountered an error. Check the *.out text files in the MatMPI directory to see if this is the case. If there are no MatMPI/*.out files, then there was a problem starting your processes. This can happen if you're trying to start a worker on a machine you have never ssh'ed to before. Manually ssh to each of the machines, and accept the machine's host key.
  • Trouble Seeing the Output: When invoking your script via eval(MPI_Run('yourscript', N, machines)) make sure that the first machine listed is the one you're working at, and that machines{1} == system('hostname') exactly or the output won't be shown. For example, if your workstation is boltzmann.ai, using 'boltzmann' or 'botlzmann.cs' or 'localhost' as the machine name will cause problems.
  • MPI_Run script name: Make sure the string passed to MPI_Run contains only the name of the script to run-- no leading spaces, no ; etc or things won't work correctly. For example, MPI_RUN(' K=3; my_script; quit', ...) is not valid.
  • Profiling: To profile your program add the following to the start of your script
    

    % PROFILING ON
profile off; profile clear; profile on -timer real;
disp('profiling is on');

    and the following right before MPI_FINALIZE is called add

    % PROFILING OFF
profile off; profsave(profile('info'),num2str(my_rank));

    After running the program there will be a directory named #/, one for each of the processors, containing an HTML version of the profile results.
  • Debugging: Errors encountered by the various worker processes are written to the files MatMPI/*.out. It is very important to use fprintf(1,'message\n') instead of disp('message') when printing status messages in your program. When calling disp (unlike fprintf) the output buffer doesn't seem to get flushed after each call, so messages are not immediately written to the output files.

David Ross Last Updated: 2008-04-20 . . .