Question

Given matrices of order M (MxM), the time complexity of the sequential multiplication time can be
defined as T1= cM3 units of time, where c is a hardware-dependant constant.
Assume that a MPI parallel code requires Tn=cM3/n + dM2/n units of time to complete on a
n-node configuration, where d is a constant determined by the MPI implementation used. The
second term in Tn accounts for the total message passing overhead experienced.
Answer the following questions assuming the MPI parallel code has no sequential bottleneck, is used
for a matrix of order 256 (M=256), and executed on a 4096-node cluster (n=4096) with a
hardware configuration with constant c=1. For parts a) and b) assume an implementation of MPI
(e.g MPICH-2) with d=4.
[35 marks]

a) Using the canonical Amdahl’s law, calculate the speedup of the parallel code on n nodes.

b) What is the efficiency of the n-node cluster

c) Assume now a MPI implementation which incorporates the MPI-3 RMA interface and has a
d=0.
i. What is the new speedup and efficiency on n nodes?

Answer 1