Alibaba Cloud Lingjun Node Diagnose & Repair

Scenario Description

Alibaba Cloud Lingjun runs AI workloads on rack-level GPU bare-metal nodes. Failures (GPU/HBM ECC, NIC link flap, NCCL hang, kernel panic, RDMA degradation, AI-job log anomalies) require a closed-loop "diagnose → report → repair" workflow. This skill covers exactly that closed loop using the eflo-controller (v2022-12-15) OpenAPI namespace shared with the alibabacloud-lingjun-cluster-scaling skill — three diagnostic CLIs + four repair CLIs + three fault-report CLIs + supporting read-only helpers — across 8 features:

1. Resource Locator — list-clusters / describe-cluster / list-cluster-nodes / list-cluster-hyper-nodes / describe-node / describe-hyper-node to anchor the target Cluster + (Hyper)Node before any diagnostic submission.
2. Submit Diagnostic Task — create-diagnostic-task with one of three DiagnosticType values (BasicCheck / NodeHardwareCheck / CheckByAiJobLogs) — enum re-verified server-side 2026-08-19 (NetConfigCheck / NetRuntimeCheck deprecated, never submit).
3. Query Diagnostic Result (single) — describe-diagnostic-result returns the per-node check items + verdict + remediation hints.
4. List Diagnostic History — list-diagnostic-results browses prior diagnostic tasks (paginated, optional --diag-type filter).
5. Produce Diagnostic Report — Render a Markdown report combining (1) target identity, (2) diagnostic verdict, (3) per-check-item table, (4) supporting evidence (syslog excerpts, hardware counters).
6. Produce Repair Plan — Map the diagnostic verdict to one of reboot-nodes / reimage-nodes / stop-nodes / report-node-status (or escalate to the cluster-scaling skill's shrink-cluster/delete-node for permanent removal); output a safe_mutate-compatible HITL plan.
7. Auxiliary Telemetry — list-syslogs (kernel/system log excerpts), per-node hardware counters from describe-node / describe-hyper-node to enrich the report.
8. Fault Report Tracking — after report-node-status (fault declaration), track the deep-diagnosis lifecycle via list-fault-reports / describe-fault-report, stop an in-progress fault diagnosis via stop-node-diagnostic, and approve a platform-raised maintenance proposal via approve-operation (both mutating, safe_mutate two-phase).

Key Resources: Cluster → (Node Groups) → Compute Nodes — regular NodeId (e.g., e01-cn-...) or rack-level HyperNodeId (e.g., hn-cn-...). Diagnostic tasks operate on NodeId or HyperNodeId.

Supported Regions: Use safe_aliyun aliyun eflo-controller describe-regions --endpoint eflo-controller.cn-hangzhou.aliyuncs.com --region cn-hangzhou (describe-regions is the discovery seed; see endpoint-routing.md §2 sole exception) or fall back to supported-regions.md.