XGBoost

XGBoost is the winningest algorithm in Kaggle history for tabular data. v2.1 (2025) brings native Blackwell GPU support and Polars integration.

When to Use

• Tabular Data: It usually beats Deep Learning on structured tables.
• Speed: Extremely optimized C++ backend.

Core Concepts

Gradient Boosting

Building extensive decision trees sequentially, each correcting the previous one's errors.

DMatrix

Internal optimized data structure.