Other reading/resources #

There are many excellent textbooks and tutorials on parallel programming. Here are some I like, with brief commentary.

• Victor Eijkhout, Introduction to High Performance Computing. This is a gentle, and quite comprehensive, introduction to parallel computing. The author has a separate book which contains more details on OpenMP and MPI programming models.

• Georg Hager & Gerhard Wellein, Introduction to High Performance Computing for Scientists and Engineers, also available through the Durham library. This book is slightly heavier-going and has a focus on code optimisation and performance-modelling from the off. An annotated bibliography is provided by Georg Hager.

• Bill Gropp, one of the authors of the MPI standard, has a course on designing and building applications for extreme scale systems. He covers performance models as well, and has a lot of good advice.

• Matt Knepley’s notes on computational science also have much good advice, with a focus on the use of PETSc providing parallel data structures and solvers.

• The Lawrence Livermore high performance computing centre, has some nice tutorials. Including one introducing OpenMP and one introductory, and one advanced MPI tutorial.

• The UK’s supercomputing service ARCHER, ran regular training events over its lifetime. Much of their training material is online, including some videos of previous courses.