scikit-learn - Explainability & Interpretability

In scientific research, a model's "why" is as important as its "what". This guide focuses on tools that reveal the decision-making process of machine learning models, ensuring they are scientifically valid and not just overfitting on artifacts.

When to Use

• Validating that a model uses physically meaningful features (e.g., in drug discovery).
• Identifying biases or "shortcuts" the model has learned from the training data.
• Explaining individual predictions to non-experts (Local explanations).
• Ranking the global impact of variables on a complex system (Global explanations).
• Scientific auditing and regulatory compliance.

Core Principles

1. Model-Specific vs. Model-Agnostic

• Model-Specific: Tools like feature_importances_ in Random Forests. Fast but tied to one architecture.
• Model-Agnostic: Tools like SHAP or Permutation Importance. Work on any model (SVM, MLP, etc.) but are more compute-intensive.