MPI

Purpose

Guide agents through MPI (Message Passing Interface) programming: point-to-point and collective communication, non-blocking operations, subcommunicators, MPI+OpenMP hybrid patterns, process launching with mpirun, debugging techniques, MPI-IO, and common performance issues.

When to Use

• Parallelizing across multiple nodes or sockets
• Implementing distributed algorithms (matrix decompose, FFT)
• Combining MPI process parallelism with OpenMP thread parallelism
• Running HPC jobs with Slurm/PBS + mpirun
• Debugging deadlocks and message mismatches
• Parallel file I/O with MPI-IO

Workflow

1. Minimal MPI program