CS159: Introduction to Parallel Processing
Description
A combination hardware architecture and software development class focused on multi-threaded, parallel processing algorithms and techniques. Overview of high-performanceparallel processing hardware architectures ranging from on-chip Instruction-Level Parallelism to multi-core microprocessor chips to large distributed supercomputing systems including Clusters, Grids, and Clouds. Discussion and hands-on exercises in a broad range of various parallel programming paradigms and languages such as Pthreads, MPI, OpenMP, Map-Reduce Hadoop, CUDA and OpenCL. The class focus will be on understanding the fundamental concepts associated with the design and analysis of parallel processing systems. Special emphasis will be placed on avoiding the unique non-deterministic software defects that can arise in parallel processing systems including race conditions and deadlocks. The class will also provide an overview of current parallel software development toolkits including debuggers and performance profilers.
Course Documents
Documents
- Syllabus [DOC]
- Term Project [PDF]
- Parallel Processing is a Hardware Solution [PDF]
- Parallel Processing is a Software Challenge - and Opportunity [PDF]
- Hardware Parallelism [PDF]
- Deadlock [PDF]
- Race Conditions, Semaphores [PDF]
- Amdahl's Law [PDF]
- Overview of MPI vs. OpenMP [PDF]
- OpenMP-Pitfalls-WrapUp [PDF]
- Review & Conclusions [PDF]
- OpenMP Pocket Guide Summary [PDF]