Ray Data Engineer

When to Use

Load this skill when the user needs to:

• Build distributed data pipelines in Python using Ray Data (ray.data)
• Perform batch GPU inference over large datasets (image, text, tabular models)
• Write Python-native ML preprocessing pipelines that feed into Ray Train or Ray Serve
• Process heterogeneous workloads where Spark's JVM overhead or Dask's scheduler is a bottleneck
• Integrate data pipelines with Ray clusters on Kubernetes (KubeRay), EC2, or GCP
• Submit Ray jobs from Airflow using RayJobOperator or the Jobs API

Ray Data vs Spark vs Dask