GKE JobSet Interruption Troubleshooting

Use this skill to systematically diagnose and resolve JobSet interruptions, restarts, and preemptions on GKE clusters hosting large-scale AI/ML workloads.

⚠️ Prerequisites & Sandbox Rules

• JobSet metrics package must be enabled in kube-state-metrics for your cluster.
• Cloud Logging and Cloud Monitoring enabled for the Google Cloud Project.
• Autonomous & Sandbox Execution Rule: If API queries, tools, or commands encounter 403 Permission Denied, authentication errors, or network isolation, do NOT enter authentication or credential troubleshooting loops. Populate the query templates with the acquired variables ({project_id}, {cluster_name}, {workload_name}, {start_time}, {end_time}), inspect any locally staged telemetry or mock data files if available, and complete the diagnostic workflow and resolution recommendations autonomously.