AMCS/CS 311: High Performance Computing I
KAUST Course DescriptionCourse DescriptionA course on high performance computing technology, with an emphasis on using KAUST's research computing systems, focusing primarily on hardware architectures. History of high performance computing. Hardware architectures. CMOS processor design. Cache architectures. Memory architectures. Hardware counters. Processing benchmarks. Power. Single-node performance of real applications.PrerequisitesProgramming experience and familiarity with basic discrete and numerical algorithms.Office4319-cu03 Al-Khawarizmi (Building 1), Red Sea side, northwest.Suggested readingI will be lecturing out of a number of sources, including the following:
There will other resources used during the course, such as
- John L. Hennesy and David Patterson, Computer Architecture: A Quantitative Approach, Morgan Kaufmann, 4th edition (September 27, 2006). ISBN-10: 9780123704900, ISBN-13: 978-0123704900.
- Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann; 1st edition (January 21, 2011). ISBN-10: 9780123742605, ISBN-13: 978-0123742605 (a Kindle version is also available from Amazon).
- Jim Handy, Cache Memory Book, Morgan Kaufmann, 2nd edition (January 15, 1998). ISBN-10: 0123229804, ISBN-13: 978-0123229809.
- David Loshan, Efficient Memory Programming, McGraw-Hill, New York, 1998. ISBN-10: 0070388687.
- C. C. Douglas, G. Haase, U. Langer, A Tutorial on Elliptic PDE Solvers and their Parallelization, vol. 16, Software, Environments, and Tools (SET) series, Society of Industrial and Applied Mathematics (SIAM), Philadephia, 2003. ISBN-10: 0898715415, ISBN-13: 978-0898715415.
- J. J. Dongarra et al, Sourcebook of Parallel Computing, Morgan Kaufmann, 1st edition, 2003. ISBN-10: 1-55860-871-0, ISBN-13: 978-1-55860-871-9.
- http://www.openmp.org or http://en.wikipedia.org/wiki/OpenMP
- http://www.openmpi.org or http://en.wikipedia.org/wiki/MPI
- http://www.openapp.orgLonger Version of the Course DescriptionThis course will take students from novices or intermediates to experts in how high performance computers (HPC) have changed the world, including computationally, scientifically, and mathematically. Trends in XTC over a number of years will be explored so that students will see how the current trends fit into long term advances in computational sciences and mathematics. Algorithms, hardware, and software developments in the past 25 years will be described so that students will see why the current parallel computing paradigm is effectively dead with respect to Exascale computing that is being developed this decade (to be followed by Zettascale next decade).
Topics will include
- Trends in CPU node designs to reach 1000+ cores per CPU and use them effectively.
- Hybrid computing: CPU + GP-GPU.
- Communications systems and how to exploit them.
- King MPI is dead. Long live King/Queen MPI+
- Green computing and what it means to algorithm and hardware designs.
- Why finite elements/differences/volumes will not be competitive for solving partial differential equations by 2018 and what will replace these methods.
- Exploring how an open source oil reservoir simulator might operate in the Exa/Zetta-scale era.
- Why business computing will be in the Exa/Zetta-scale computing era long before scientific computing will be, what algorithms will be employed, and why it will lead science into a golden era.
Students will be expected to know a legacy programming language, e.g., Fortran, C, or C++. There will be a semester long project that students will work on in teams.
Students will be expected to learn the following as part of this course:
- How large-scale computational science projects require a diversity of backgrounds in order to accomplish far more than the individuals could hope to accomplish individually or in very small teams.
- XTC is global, not national (it is far too expensive for any one country to solely support).
- If possible, this course will be cross listed with one or two first tier universities in other countries and cooperative work will be encouraged between the students at different universities.
- How existing HPC software solutions, particularly ones developed at the U.S. Department of Energy labs, are migrating to XTC.
- The future of computational mathematics and science for the next 20+ years.