One interesting answer I saw on Stack Overflow.
Someone asked how he can receive the msg send by `MPI_Bcast`.
The answer is
For MPI collective communications, everyone has to particpate; everyone has to call the Bcast, or the Allreduce, or what have you. (That’s why the Bcast routine has a parameter that specifies the “root”, or who is doing the sending; if only the sender called bcast, you wouldn’t need this.) Everyone calls the broadcast, including the receivers; the receviers don’t just post a receive.
The reason for this is that the collective operations can involve everyone in the communication, so that you state what you want to happen (everyone gets one processes’ data) rather than how it happens (eg, root processor loops over all other ranks and does a send), so that there is scope for optimizing the communication patterns (eg, a tree-based hierarchical communication that takes
log(P)steps rather than
Psteps for P processes).
Good point for how to understand collective operations.