slide 23 of 32

Aggregate Functions
(Collective Communication)

• Processors interact as a group
• Simplest is barrier synchronization
  Each processor waits until all have arrived
• Data is submitted by each processor, aggregate functions returned to each
  • Associative reductions (e.g., sum)
  • Scans/parallel prefix
• Low latency and bandwidth
• Used mostly to control execution