Oracle Cloud Incident Runbook

Overview

Self-service runbook for when OCI instances go down and the status page stays green. OCI's status page has a history of not acknowledging outages in real time (London Jan 2026 — 502s and instances disappearing for 10 minutes with no status update). OCI Support response times average 4+ hours for Sev-1 tickets. This runbook gives you health probes, automated instance recovery, cross-AD failover, and cross-region failover — all executable without waiting on Oracle.

Purpose: Detect OCI service degradation independently, recover instances automatically, and fail over to alternate availability domains or regions when the primary is impacted.

Prerequisites

• OCI CLI installed and configured — ~/.oci/config validated (see oraclecloud-install-auth)
• Python 3.8+ with the OCI SDK — pip install oci
• Pre-configured resources: at least one compute instance, a VCN with subnets in multiple ADs
• IAM policies: manage instances, manage volumes, inspect work-requests in the target compartment
• Boot volume backups enabled (recovery depends on having a recent backup)

Instructions

Step 1: Independent Health Probes