Task	Data Size	Interpretability Need	Recommended
Classification (small)	< 10K	High	Logistic Regression, Decision Tree
Classification (medium)	10K-100K	Medium	Random Forest, XGBoost
Classification (large)	> 100K	Low OK	Neural Network, XGBoost
Regression (linear)	Any	High	Linear/Ridge/Lasso
Regression (nonlinear)	Medium+	Medium	Random Forest, Gradient Boosting
Clustering	Any	Medium	K-Means, DBSCAN, Hierarchical
Dimensionality reduction	Any	Medium	PCA, t-SNE, UMAP
Anomaly detection	Any	Medium	Isolation Forest, LOF
Time series	Any	Varies	ARIMA, Prophet, LSTM

Standard Pipeline

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV
from sklearn.preprocessing import StandardScaler, LabelEncoder
from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline

# 1. Preprocessing
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

# 2. Pipeline with scaling
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('model', RandomForestClassifier(random_state=42))
])

# 3. Cross-validation
scores = cross_val_score(pipe, X_train, y_train, cv=5, scoring='roc_auc')
print(f"CV AUC: {scores.mean():.3f} ± {scores.std():.3f}")

# 4. Hyperparameter tuning
param_grid = {
    'model__n_estimators': [100, 300, 500],
    'model__max_depth': [5, 10, None],
    'model__min_samples_leaf': [1, 5, 10]
}
grid = GridSearchCV(pipe, param_grid, cv=5, scoring='roc_auc', n_jobs=-1)
grid.fit(X_train, y_train)

# 5. Evaluation
y_pred = grid.predict(X_test)
print(classification_report(y_test, y_pred))
print(f"Test AUC: {roc_auc_score(y_test, grid.predict_proba(X_test)[:,1]):.3f}")

Feature Importance & Explainability

# Built-in importance (tree models)
importances = grid.best_estimator_.named_steps['model'].feature_importances_
feat_imp = pd.Series(importances, index=X.columns).sort_values(ascending=False)

# SHAP values (model-agnostic)
# pip install shap
import shap
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
shap.summary_plot(shap_values, X_test)

Unsupervised Learning

from sklearn.cluster import KMeans, DBSCAN
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE

# PCA
pca = PCA(n_components=0.95)  # retain 95% variance
X_pca = pca.fit_transform(X_scaled)
print(f"Components: {pca.n_components_}, Explained variance: {pca.explained_variance_ratio_.cumsum()[-1]:.3f}")

# K-Means with elbow method
inertias = [KMeans(n_clusters=k, random_state=42).fit(X_scaled).inertia_ for k in range(2, 11)]

# t-SNE visualization
X_tsne = TSNE(n_components=2, random_state=42, perplexity=30).fit_transform(X_scaled)

Reporting ML Results in Papers

Always include:

Dataset description (size, features, class balance)
Preprocessing steps
Model selection rationale
Cross-validation strategy (k-fold, stratified, leave-one-out)
Hyperparameter search space and method
Multiple metrics (accuracy, precision, recall, F1, AUC)
Comparison with baselines
Feature importance / model interpretation
Confidence intervals or statistical tests on performance
Code/data availability statement

Tips

Always use stratified splits for imbalanced data
Report multiple metrics, not just accuracy
Compare against simple baselines (majority class, mean prediction)
Use nested CV for unbiased performance estimation
Check for data leakage (especially with time series)
Document random seeds for reproducibility

ml-pipeline

ML Pipeline

Pipeline Overview

Model Selection Guide