EE 599-003 - Cluster/Multi-Core Computing
EE 699-003 - Cluster/Multi-Core Computing

Preliminary ABET-style Syllabus

EE 599 - Cluster/Multi-Core Computing: 3 Credits

EE 699 - Cluster/Multi-Core Computing: 3 Credits

Cluster/Multi-Core Computing is about the many flavors of MIMD-parallel computing architectures that now dominate high-performance computing. It used to be that 4 processors were taken seriously as a parallel supercomputer. Now, there are cell phones with 4 cores and Sequoia, an IBM BlueGene/Q system that was at the top of the Top500 supercomputers list in June 2012 (but is #2 as of November 2012!), was the first machine to have over a million cores: it has 1,572,864 cores.

MIMD (Multiple Instruction, Multiple Data) programming models, and MIMD hardware, can be viewed as divided into two main classes based on how the essentially independent processing elements (PEs) interact. In shared memory models, PEs interact by sharing access to data structures in memory. What has been stored in a memory location by one PE can be examined by another PE. In distributed memory models, PEs interact by explicitly invoking function/subroutine calls that implement communication operations between PEs. For example, one PE might call a function to send a message to another PE which calls another function to recieve it. Either type of programming model can be implemented on either type of hardware model, although shared memory models generally lead to less efficient implementations on distributed memory hardware.

In order to ensure that students get a stronger programming experience, this course will focus on the programming models first and for most of the course. Students will use C-based language/library systems to write their programs. For distributed memory, the focus will be on MPI -- which is highly portable and efficient on most MIMD hardware. For shared memory, a number of different programming models will be used. The dominant shared-memory programming standard in the high-performance computing world is OpenMP, which is essentially a set of language extensions. However, we also will cover a variety of other shared-memory programming models that are in common use.

Each student will have hands-on experience programming multi-core systems and cluster supercomputers for assigned problems. Not just the programming models, but also key algorithms will be discussed in detail. In addition, each graduate student (EE 699) will be expected to solve a modest MIMD programming problem that they individually will propose, write-up, code, and present to the class.

After covering the programming models, the course will focus on the evolution of MIMD architecture and hardware. MIMD architecture is in most ways quite mature, but there are quite a few issues that directly and dramatically impact performance. For distributed memory systems, we will discuss issues of network technology and topology, OS interface, and system management; for shared memory systems, we will focus more on cache issues and OS scheduling problems.

TEXTBOOK:

None required - Course notes

Each of the programming models has extensive collections of supporting materials freely available online, and we will be making heavy use of some of those materials in addition to notes developed here.

COORDINATOR:

Dr. Henry G. Dietz, Professor

GOALS:

The goal of this course is to enable students to make effective use of MIMD-parallel computers. Further, students will understand the relevant architectural issues.

Goals will be tested primarily by the programming projects, but also by two in-class exams. The first exam will test understanding of distributed-memory issues and MPI, while the second will focus on shared-memory issues. Each test will account for about 20-25% of your course grade, with projects making the rest of your grade.

PREREQUISITE:

For EE599: Undergraduate standing and fluency in C or a similar programming language

For EE699: Graduate standing and fluency in C or a similar programming language

TOPICS:

1. Introduction
2. Distributed-memory programming, algorithms, and MPI
3. Shared memory programming, algorithms, OpenMP and other environments
4. MIMD architecture and performance tuning
5. Projects
6. Graduate project presentations and discussions

OUTCOMES:

1. Understand and be proficient in using MPI, the standard message-passing interface library and its supporting environment.
2. Understand and have experience using several shared-memory programming environments, including OpenMP.
3. Understand the operation and evolution of MIMD architecture, from multi-core processors to cluster supercomputers.
4. Understand key algorithms and issues for performance tuning of programs running on MIMD hardware.

COMPUTER USAGE:

Students will perform open-ended experiments using MIMD computing to solve engineering problems.

DESIGN CONTENT:

Students will design parallel approaches for their problems suitable for execution on MIMD systems. Implementation of these approaches will largely reuse code provided to them, with only a small amount of new coding required. The design content, which is the bulk of the effort in the projects, is the casting of a problem in a form that can be efficiently accelerated by MIMD parallel processing.

CLASS SCHEDULE:

Lecture 3 hours per week.

Engineering Science: 1 Credit (33%)

Engineering Design: 2 Credits (67%)

• An ability to apply knowledge of mathematics, science, and engineering.
• An ability to design and conduct experiments, as well as to analyze and interpret data.
• An ability to design a system, component, or process to meet desired needs.
• An ability to identify, formulate, and solve engineering problems.
• An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice.

PREPARED BY: Henry G. Dietz; last update: January 17, 2012