MLIP Performance Benchmarking

Goal

Evaluate and compare the inference speed (latency) and memory consumption of various foundation MLIP models to determine their suitability for different simulation scales and timescales.

Benchmark Script

The benchmark_mlips.py script measures performance by running short MD simulations on NaCl supercells of varying sizes.

Usage

Run the script within the appropriate conda environment for the models being tested. The script automatically skips models not supported by the current environment.

Multi-Environment Benchmarking

Because different MLIPs require isolated Conda environments (e.g., mace-agent, matgl-agent, fairchem-agent), the benchmark results are built incrementally.

1. Run the script in each environment: The script gracefully skips models whose libraries are missing while preserving and updating the central speed_benchmark.yaml file.
2. Consolidate: Run the script in any environment (that has matplotlib) with the --only_plot flag to generate the combined graphs from the accumulated total data.