在上一篇中我们介绍了 mpi4py 中的非阻塞集合通信方法,下面我们将介绍 mpi4py 中的近邻集合通信方法,另一个 MPI-3 引进的新特性。
近邻集合通信(neighborhood collective communication) 是定义在拓扑通信子之上的一类集合通信操作,不同于一般集合通信操作中通信子上的所有进程都会参与,近邻集合通信只要求拓扑通信子上的某个进程和其直接邻居进程之间进行集合通信,其它进程可以不用参与,因此近邻集合通信具有非常好的可扩展性。
近邻集合操作也有阻塞和非阻塞两个版本。
方法接口
下面给出 mpi4py 中近邻集合通信的方法接口。
MPI.Topocomm.neighbor_allgather(self, sendobj)
近邻全收集操作,对应 MPI.Comm.allgather。以小写字母开头的方法,可以传递任意可被 pickle 的 Python 对象 sendobj
,为阻塞通信方法。
MPI.Topocomm.neighbor_alltoall(self, sendobj)
近邻全发散操作,对应 MPI.Comm.alltoall。以小写字母开头的方法,可以传递任意可被 pickle 的 Python 对象 sendobj
,为阻塞通信方法。
MPI.Topocomm.Neighbor_allgather(self, sendbuf, recvbuf)
近邻阻塞全收集操作,对应 MPI.Comm.Allgather,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Neighbor_allgatherv(self, sendbuf, recvbuf)
近邻阻塞向量全收集操作,对应 MPI.Comm.Allgatherv,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Neighbor_alltoall(self, sendbuf, recvbuf)
近邻阻塞全发散操作,对应 MPI.Comm.Alltoall,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Neighbor_alltoallv(self, sendbuf, recvbuf)
近邻阻塞向量全发散操作,对应 MPI.Comm.Alltoallv,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Neighbor_alltoallw(self, sendbuf, recvbuf)
近邻阻塞向量全发散操作,对应 MPI.Comm.Alltoallw,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Ineighbor_allgather(self, sendbuf, recvbuf)
近邻非阻塞全收集操作,对应 MPI.Comm.Iallgather,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Ineighbor_allgatherv(self, sendbuf, recvbuf)
近邻非阻塞向量全收集操作,对应 MPI.Comm.Iallgatherv,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Ineighbor_alltoall(self, sendbuf, recvbuf)
近邻非阻塞全发散操作,对应 MPI.Comm.Ialltoall,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Ineighbor_alltoallv(self, sendbuf, recvbuf)
近邻非阻塞向量全发散操作,对应 MPI.Comm.Ialltoallv,只适用于传递具有缓冲区接口的对象。
MPI.Topocomm.Ineighbor_alltoallw(self, sendbuf, recvbuf)
近邻非阻塞向量全发散操作,对应 MPI.Comm.Ialltoallw,只适用于传递具有缓冲区接口的对象。
由以上方法接口可以看出,近邻集合通信最基本最主要的操作只有两个:MPI.Topocomm.Neighbor_allgather 和 MPI.Topocomm.Neighbor_alltoall,其它的要么是其向量版本,要么是对应的非阻塞版本。Neighbor_allgather 会发送一份相同的数据给其所有直接邻居,并从这些邻居处接收一份数据;Neighbor_alltoall 会给其每个直接邻居发送一份不同的数据,并从这些邻居处接收一份数据。以 3 × 3 周期性的笛卡尔拓扑为例,下图显示了 Neighbor_allgather 的发送和接收关系。
3 × 3 周期性的笛卡尔拓扑 Neighbor_allgather 发送和接收关系例程
以上图中的 3 × 3 笛卡尔拓扑为例,下面给出近邻集合操作的使用例程。
# neighbor.py
"""
Demonstrates the usage of neighborhood collective communication.
Run this with 9 processes like:
$ mpiexec -n 9 python neighbor.py
"""
import numpy as np
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
dims = [3, 3]
# -----------------------------------------------------------------
# neighbor_allgather with periodic boundary
periods = [True, True]
cart_comm = comm.Create_cart(dims, periods)
recv_obj = cart_comm.neighbor_allgather(rank)
print 'neighbor_allgather (periodic): rank %d has %s' % (rank, recv_obj)
# -----------------------------------------------------------------
# neighbor_allgather with non-periodic boundary
periods = [False, False]
cart_comm = comm.Create_cart(dims, periods)
recv_obj = cart_comm.neighbor_allgather(rank)
print 'neighbor_allgather (non-periodic): rank %d has %s' % (rank, recv_obj)
# -----------------------------------------------------------------
# neighbor_alltoall with periodic boundary
periods = [True, True]
cart_comm = comm.Create_cart(dims, periods)
recv_obj = cart_comm.neighbor_alltoall(['a', 'b', 'c', 'd'])
print 'neighbor_alltoall (periodic): rank %d has %s' % (rank, recv_obj)
# -----------------------------------------------------------------
# Neighbor_allgather with periodic boundary
periods = [True, True]
cart_comm = comm.Create_cart(dims, periods)
send_buf = np.array([rank], dtype='i')
recv_buf = np.full((4,), -1, dtype='i') # initialize with all -1
cart_comm.Neighbor_allgather(send_buf, recv_buf)
print 'Neighbor_allgather (periodic): rank %d has %s' % (rank, recv_buf)
# -----------------------------------------------------------------
# Ineighbor_allgather with non-periodic boundary
periods = [False, False]
cart_comm = comm.Create_cart(dims, periods)
send_buf = np.array([rank], dtype='i')
recv_buf = np.full((4,), -1, dtype='i')
req = cart_comm.Ineighbor_allgather(send_buf, recv_buf)
req.Wait()
print 'Ineighbor_allgather (non-periodic): rank %d has %s' % (rank, recv_buf)
运行结果如下:
$ mpiexec -n 9 python neighbor.py
neighbor_allgather (periodic): rank 7 has [4, 1, 6, 8]
neighbor_allgather (non-periodic): rank 7 has [4, None, 6, 8]
neighbor_alltoall (periodic): rank 7 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 7 has [4 1 6 8]
Ineighbor_allgather (non-periodic): rank 7 has [ 4 -1 6 8]
neighbor_allgather (periodic): rank 8 has [5, 2, 7, 6]
neighbor_allgather (non-periodic): rank 8 has [5, None, 7, None]
neighbor_alltoall (periodic): rank 8 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 8 has [5 2 7 6]
Ineighbor_allgather (non-periodic): rank 8 has [ 5 -1 7 -1]
neighbor_allgather (periodic): rank 0 has [6, 3, 2, 1]
neighbor_allgather (non-periodic): rank 0 has [None, 3, None, 1]
neighbor_alltoall (periodic): rank 0 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 0 has [6 3 2 1]
Ineighbor_allgather (non-periodic): rank 0 has [-1 3 -1 1]
neighbor_allgather (periodic): rank 1 has [7, 4, 0, 2]
neighbor_allgather (non-periodic): rank 1 has [None, 4, 0, 2]
neighbor_alltoall (periodic): rank 1 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 1 has [7 4 0 2]
Ineighbor_allgather (non-periodic): rank 1 has [-1 4 0 2]
neighbor_allgather (periodic): rank 2 has [8, 5, 1, 0]
neighbor_allgather (non-periodic): rank 2 has [None, 5, 1, None]
neighbor_alltoall (periodic): rank 2 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 2 has [8 5 1 0]
Ineighbor_allgather (non-periodic): rank 2 has [-1 5 1 -1]
neighbor_allgather (periodic): rank 3 has [0, 6, 5, 4]
neighbor_allgather (non-periodic): rank 3 has [0, 6, None, 4]
neighbor_alltoall (periodic): rank 3 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 3 has [0 6 5 4]
Ineighbor_allgather (non-periodic): rank 3 has [ 0 6 -1 4]
neighbor_allgather (periodic): rank 4 has [1, 7, 3, 5]
neighbor_allgather (non-periodic): rank 4 has [1, 7, 3, 5]
neighbor_alltoall (periodic): rank 4 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 4 has [1 7 3 5]
Ineighbor_allgather (non-periodic): rank 4 has [1 7 3 5]
neighbor_allgather (periodic): rank 5 has [2, 8, 4, 3]
neighbor_allgather (non-periodic): rank 5 has [2, 8, 4, None]
neighbor_alltoall (periodic): rank 5 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 5 has [2 8 4 3]
Ineighbor_allgather (non-periodic): rank 5 has [ 2 8 4 -1]
neighbor_allgather (periodic): rank 6 has [3, 0, 8, 7]
neighbor_allgather (non-periodic): rank 6 has [3, None, None, 7]
neighbor_alltoall (periodic): rank 6 has ['b', 'a', 'd', 'c']
Neighbor_allgather (periodic): rank 6 has [3 0 8 7]
Ineighbor_allgather (non-periodic): rank 6 has [ 3 -1 -1 7]
由以上的输出结果可以看出,对笛卡尔拓扑,近邻集合操作某进程从其直接邻居处接收数据的顺序为:按照笛卡尔网格的维度顺序从小到大接收(如果拓扑为周期性的,会考虑其对应的周期性)。比如对进程 4,首先接收其上面行的数据 1, 然后其下面行的数据 7, 然后其左边列的数据 3, 最后其右边列的数据5。如果拓扑为非周期性的,则不会向其不存在的边界邻居发送数据,也不会从其不存在的边界邻居接收数据。
以上介绍了 mpi4py 中的近邻集合通信方法,在下一篇中我们将介绍 mpi4py 中的非阻塞通信子复制和组集合通信子创建方法。
网友评论