Introduction to High Performance Computing

High-performance computing refers to a specialized use and programming of (parallel) supercomputers, computer clusters, and everything from software to hardware to speed up computations. The course will discuss topics related to fundamental concepts and techniques in parallel computation structuring and design, architectures of high-performance computing systems, accessing high performance computing resources, optimization techniques for data access, developing applications for those resources, and executing developed applications.
The contents of the course provide a balance of theoretical and practical aspects in high performance computing. In this course the students will learn how to write faster code that is highly optimized for modern multi-core processors and clusters, using modern software development tools, performance profilers, specialized algorithms, parallelization strategies, and advanced parallel programming constructs in OpenMP and MPI. The practical part of this course is also covered in the lab through group projects in setting up a cluster using an open-source toolkit, exercises and practical assignments.

The aims of this course are:

• To introduce the fundamentals of High Performance Computing architecture.
• To describe high performance computing in the context of scientific computing.
• To explain the concepts of parallel processing as it pertains to high performance computing.
• To develop and execute parallel programs on high performance computing resources using parallel programming paradigms such as multi-core computing via OpenMP, Multi-computer technologies (cluster computing via MPI).

Upon the successful completion of this course, students should be able to:

• have an awareness of the modern field of computational science and engineering and of the impact of high performance computing on science and industry.
• get some practices in the various paradigms of high performance computing and their potential for performance and programmability.
• be capable of writing algorithms that yield good performance on high performance architectures, and to be able to estimate and evaluate their performance.
• apply parallel programs on different hardware architectures and software environments.

Textbooks:

1. High Performance Computing: Modern Principles and Practices, Thomas Sterling, Matthew Anderson, Maciej Brod, 2018

References:

1. High Performance Computing for Dummies, by Douglas Eadline, 2009
2. Introduction to High Performance Computing for Scientists and Engineers, by Georg Hager and Gerhard Wellein, 2011.
3. High Performance Python, Micha Gorelick and Ian Ozsvald, 2014
4. Introduction to High Performance Scientific Computing, D. L. Chopp, 2019