Parallel Programming
When: Winter semester 2017-18
CAMPUS #: 17ws-23895
Contacts: Prof. Paolo Bientinesi, Dr. William McDoniel
Language: English
Office hours
- Prof. Bientinesi: Tuesdays, 11am-1pm, by appointment.
AICES R432 (Rogowski Building - Schinkelstrasse 2)
Prerequisites
Knowledge of the C programming language. You are supposed to know
- the basic datatypes; how to define and use data structures (structs)
- how to operate on pointers and memory addresses
- how to allocate and free dynamic memory (malloc, free)
- how to declare and call (recursive) functions
- how to pass arguments to functions "by value" and "by address"
- how to use library functions; this means including headers, using the provided data structures, calling functions, ...
Topics
- Principles of parallel performance
- Parallel architectures
- Patterns of parallel programming
- Processes and threads
- Open MP
- Execution model
- Work sharing
- Loop-level parallelism
- Synchronization
- Message Passing Interface
- Collective communication
- Point-to-Point communication
- Derived data types
Lectures & Exercises
- Mondays 12:15 - 13:00 (lecture) 1080|140 (R 140), Schinkelstraße 1, Templergraben 51
- Mondays 16:00 - 17:30 (exercises) 2090|120 (FT), Melatener Straße 23
- Wednesdays 12:15 - 13:45 (lecture) 1080|140 (R 140), Schinkelstraße 1, Templergraben 51
Lectures
- 11.10 - First lecture. Presentation [PDF].
Introduction to PP, overview of the semester [PDF].
Architectures, pipelining, throughput [PDF]. - 16.10 - Dependencies [PDF].
- 18.10 - Timings [PDF] [code].
- 23.10 - OpenMP, part 1. Parallel regions [PDF] [code].
- 25.10 - Speedup, strong & weak scalability [PDF] [code].
- 30.10 - OpenMP, part 2 [PDF] [code]. [Exercise for Monday 06.11.17]
- 06.11 - Ahmdal's law [PDF] [code].
- 08.11 - OpenMP, part 3. [Synchronization] [Work Sharing 1] [code].
- 13.11 - OpenMP, part 4: Work-sharing & synchronization cont'd. [slides] [code].
- 15.11 - OpenMP, part 4: Work-sharing & synchronization cont'd. [slides] [code].
- 20.11 - OpenMP, part 5: Nested parallelism [slides] [code].
- 22.11 - Double buffering [Reference: Sec 4.1 & 4.2], Granularity [Reference: Sec 4.3] [NO slides!].
- 27.11 - OpenMP, part 6: Tasks [slides] [code].
- 29.11 - MPI, part 1: Introduction, "minimal MPI" [slides] [code].
- 04.12 - MPI, part 2: first program [code].
- 06.12 - OpenMP, part 7: Vectorization [slides] [code].
- 11.12 - MPI, part 3: Collective communication [slides] [code].
- 13.12 - Sample exam: performance, OpenMP
- 18.12 - TIS-100
- 20.12 - MPI, part 4: Collective communication [slides] [code].
- 08.01 - MPI, part 5: Collective communication [slides].
- 10.01 - MPI, part 6: Lower bounds [slides] - updated!.
- 15.01 - MPI, part 7: Point-to-point communication [slides] [code].
- 17.01 - MPI, part 8: Point-to-point communication [slides] [code].
- 22.01 - MPI, part 9: Send modes [slides] [code].
- 24.01 - MPI, part 10: Data types [slides] [code].
- 29.01 - MPI, part 11: Communicators [slides] [code].
- 31.01 - Exercises [code].
Exercise sessions
- 23.10 - Intro to RWTH's cluster. OpenMP & MPI basics.
- 30.10 - Speedup, efficiency, strong & weak scalability.
- 06.11 - OpenMP [Exercise] [code].
- 13.11 - OpenMP [Exercise]
- 20.11 - OpenMP Work-Sharing [Exercise]
- 27.11 - OpenMP Nested Parallelism [Exercise]
- 04.12 - OpenMP Wrap-Up [Exercise]
- 11.12 - MPI Collectives - Part 1 [code].
- 18.12 - Test exam
- 08.01 - MPI Collectives - Part 2 [code].
- 15.01 - MPI: Matrix-vector multiply [slides].
- 22.01 - MPI: Point-to-point communication [slides] [code].
- 29.01 - MPI: Exercises [code] [exercises] [HW3-exercise1.c]
Exams
- 07.02.2018, 13:00 - 16:00.
- 26.03.2018, 10:30 - 13:30. [Cheat Sheet]