Parallel Programming
When: Winter semester 2018-19
CAMPUS #: 18ws-23895
Contact: Prof. Paolo Bientinesi, Markus Hoehnerbach
Language: English
Office hours
- Prof. Bientinesi: Tuesdays, 11am-1pm, by appointment.
AICES R432 (Rogowski Building - Schinkelstrasse 2)
Prerequisites
Knowledge of the C programming language. You are supposed to know
- the basic datatypes; how to define and use data structures (structs)
- how to operate on pointers and memory addresses
- how to allocate and free dynamic memory (malloc, free)
- how to declare and call (recursive) functions
- how to pass arguments to functions "by value" and "by address"
- how to use library functions; this means including headers, using the provided data structures, calling functions, ...
Topics
- Principles of parallel performance
- Parallel architectures
- Patterns of parallel programming
- Processes and threads
- Open MP
- Execution model
- Work sharing
- Loop-level parallelism
- Synchronization
- Message Passing Interface
- Collective communication
- Point-to-Point communication
- Derived data types
Textbook
- Parallel Programing in Science and Engineering, by Victor Eijkhout [Repository] [HTML]
Lectures
- Mondays 16:00 - 16:45 HKW 5 (1132|603)
- Fridays 16:30 - 18:00 1080|005 (R 5), Schinkelstraße 1, Templergraben 51
Exercises
- Mondays 16:45 - 18:15 HKW 5 (1132|603)
Extra Lectures
- Mondays 12:15 - 14:30 GRS 001, Schinkelstraße
- 29.10 - Performance metrics [PDF]
- 05.11 - MPI
- 07.01 - MPI
- 21.01
Lectures
- 12.10 - Presentation [PDF]. Introduction [PDF].
- 15.10 - Double lecture. Architectures [PDF], Dependencies [PDF].
- 19.10 - Performance metrics [PDF]
- 22.10 - Guest lecture
- 26.10 - Scalability, Amdahl's law [PDF] [Files]
- 29.10 - Double lecture [PDF]
- 02.11 - Processes & Threads. Shared memory vs. distributed memory [PDF]
- 05.11 - MPI (pt.1): Intro [PDF]
- 06.11 - 13:00: Guest lecture @ Rogowski Building, room 115. Heinrich Bockhorst (Intel): "One-Sided Communication, MPI on Threads, Overlap of Communication and Computation. Old MPI topics - New Answers?" [PDF]
09.11- rescheduled12.11- rescheduled16.11- rescheduled- 19.11 - MPI (pt.2): Collective Communication
- 23.11 - MPI (pt.3): Collective Communication [PDF][Files]
- 26.11 - MPI (pt.4): Lower Bounds [PDF]
- 30.11 - MPI (pt.5): Point-to-point Communication [PDF][Files]
- 03.12 - OpenMP (pt. 1) Intro [PDF][Files]
- 07.12 - OpenMP (pt. 2) Synchronization [PDF][Files]
- 10.12 - OpenMP (pt. 3) Worksharing (for) [PDF][Files]
- 14.12 - OpenMP (pt. 4) Worksharing (sections, single) [PDF][Files]
- 17.12 - OpenMP (pt. 5) Nested Parallelism [PDF][Files]
- 21.12 - OpenMP (pt. 6) Tasks [PDF][Files]
- 07.01 - MPI (pt.6): Communication-computation Overlap [Exercises]
- 08.01 - 09:00: Guest Lecture @ Rogowski Building, room 115. Torsten Hoefler (ETH): "MPI Remote Memory Access Programming and Scientific Benchmarking of Parallel Codes" [PDF]
- 11.01 - MPI (pt.7): Point-to-point Communication [PDF]
- 14.01 - MPI (pt.8): Datatypes. Communicators [PDF][Files]
18.01- 21.01 - Exercises: Implementation of MPI collectives
25.01
Exercise sessions
- 15.10 → Lecture
- 22.10 - Timings [Files]
- 29.10 - Performance [PDF]
- 05.11 - Performance [Homework]
- 19.11 - MPI [Files]
- 26.11 - MPI [Homework]
- 10.12 - OpenMP [Homework][Files]
- 17.12 - OpenMP [Homework][Files]
- 07.01 - Comp-comm Overlap [Exercises]
- 14.01 - [MPI exercises]
- 21.01 - OpenMP [Homework][Files]
Exams
- 06.02.2018, 13:00 - 16:00, HKW 1 (1132|203)
- 06.03.2018, 10:30 - 13:30, VT (1100|U101)