parallel algorithms and tasks specifically • Message Passing Interface (MPI): 3 decades of distributed-memory parallel programming • MPI teaches design patterns to detect and handle recoverable errors ## What Unified divergent interfaces circa 1991 – MPI 1.0 released 1994 – 3.1 in 2015; 4.0 pending • Millions-way parallelism • Stable: '90s code works ## MPI • Modest hardware requirements • Cooperates Cooperates with other programming models – e.g., for GPUs ## MPI's parallelism model ## • P “processes” – Fixed location & number – Like parallel for_each over 0, 1, ..., P-1 ## • Do NOT share memory

Calculate dataauto values = algorithm(1.0f, 3, data);// Send data to proc. 1 MPI\_Send(values.data(), values.size(), MPI\_FLOAT, 1, 0, MPI\_COMM\_WORLD);// Data is now sent.|// Allocate space for datastd::vec datastd::vector<float>recv\_values(num\_values);// Receive data from proc. 0 MPI\_Recv(recv\_values.data(), num\_values, MPI\_FLOAT, 0, 0, MPI\_COMM\_WORLD);// Data is now in\`recv\_values\`| ## Communication Mechanisms Calculate dataauto values = algorithm(1.0f, 3, data);// Send data to proc. 1MPI\_Send(values.data(),values.size(),MPI\_FLOAT, 1,0, MPI\_COMM\_WORLD);// Data is now sent.|// Allocate space for datastd::vect

Calculate dataauto values = algorithm(1.0f, 3, data);// Send data to proc. 1MPI\_Send(values.data(),values.size(),MPI\_FLOAT, 1,0, MPI\_COMM\_WORLD);// Data is now sent.|// Allocate space for datastd::vect datastd::vector<float>recv\_values(num\_values);// Receive data from proc. 0MPI\_Recv(recv\_values.data(),num\_values,MPI\_FLOAT, 0,0, MPI\_COMM\_WORLD);// Data is now in/\`recv\_values\`| ## How do I program Calculate dataauto values = algorithm(1.0f, 3, data);// Send data to proc. 1MPI\_Send(values.data(),values.size(),MPI\_FLOAT, 1,0, MPI\_COMM\_WORLD);// Data is now sent.|// Allocate space for datastd::vect

在高性能计算领域，MPI是重要的分布式计算模型。MPI是一种基于消息传递的并行编程技术，其定义了一组具有可移植性的编程接口。通过MPI编程模型，程序员能编写基于消息通信的应用程序，而这个应用程序能在不同的节点上启动并协调工作，它们需访问共享存储，而Ceph FS正好提供共享存储的访问。 各个厂商或组织遵循这些标准实现自己的MPI软件包，典型的实现包含开放源码的MPICH、OpenMPI、LAM-MPI以及商业实现Intel 业实现Intel MPI。其中，OpenMPI实现了MPI-1.2和MPI-2.0的通信规范，并支持TCP和RDMA（Remote Direct Memory Access），是常用的MPI实现库。CentOS 7.2默认带有OpenMPI软件包，安装过程如下。 ## （1） 下载OpenMPI [root@compute-node1 ~]# cd cephfs [root@compute-node1]# software/openmpi-1.10.1/run.sh。 ## （3） 蒙特卡洛法计算PI值示例代码 #include #include #include "mpi.h" #include #include using namespace std; long long int M(long long int mynumber

are an example of such an abstraction. On the other hand, packages like MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. 4. GPU computing: The Julia GPU compiler provides domains at once. Julia’s implementation of message passing is different from other environments such as MPI $ ^{1} $ . Communication in Julia is generally “one-sided”, meaning that the programmer needs to explicitly for multi-homed hosts. As an example of a non-TCP/IP transport, an implementation may choose to use MPI, in which case –worker must NOT be specified. Instead, newly launched workers should call init_worker(cookie)

are an example of such an abstraction. On the other hand, packages like MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. ## 4. GPU computing: The Julia GPU compiler provides once. Julia's implementation of message passing is different from other environments such as MPI $ ^{1} $ . Communication in Julia is generally "one-sided", meaning that the programmer needs for multi-homed hosts. As an example of a non-TCP/IP transport, an implementation may choose to use MPI, in which case --worker must NOT be specified. Instead, newly launched workers should call init_worker(cookie)

are an example of such an abstraction. On the other hand, packages like MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. ## 4. GPU computing: The Julia GPU compiler provides once. Julia's implementation of message passing is different from other environments such as MPI $ ^{1} $ . Communication in Julia is generally "one-sided", meaning that the programmer needs for multi-homed hosts. As an example of a non-TCP/IP transport, an implementation may choose to use MPI, in which case --worker must NOT be specified. Instead, newly launched workers should call init_worker(cookie)

are an example of such an abstraction. On the other hand, packages like MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. ## 4. GPU computing: The Julia GPU compiler provides once. Julia's implementation of message passing is different from other environments such as MPI $ ^{1} $ . Communication in Julia is generally "one-sided", meaning that the programmer needs for multi-homed hosts. As an example of a non-TCP/IP transport, an implementation may choose to use MPI, in which case --worker must NOT be specified. Instead, newly launched workers should call init_worker(cookie)

are an example of such an abstraction. On the other hand, packages like MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. ## 4. GPU computing: The Julia GPU compiler provides once. Julia's implementation of message passing is different from other environments such as MPI $ ^{1} $ . Communication in Julia is generally "one-sided", meaning that the programmer needs for multi-homed hosts. As an example of a non-TCP/IP transport, an implementation may choose to use MPI, in which case --worker must NOT be specified. Instead, newly launched workers should call init_worker(cookie)

are an example of such an abstraction. On the other hand, packages like MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. 4. GPU computing: The Julia GPU compiler provides domains at once. Julia’s implementation of message passing is different from other environments such as MPI $ ^{1} $ . Communication in Julia is generally “one-sided”, meaning that the programmer needs to explicitly for multi-homed hosts. As an example of a non-TCP/IP transport, an implementation may choose to use MPI, in which case –worker must NOT be specified. Instead, newly launched workers should call init_worker(cookie)