Skill: Enterprise Lakehouse FinOps & Cost Management

Description

Enables the AI agent to analyze, attribute, and reduce cloud costs associated with Lakehouse architecture (Storage, Compute, and API operations).

Context

Lakehouses can quickly become expensive due to stale snapshots, orphaned files, excessive API calls (e.g., S3 PUT/GET), and unoptimized clusters. This skill teaches the agent how to implement strict FinOps practices across Databricks, AWS EMR, and Snowflake.

Instructions

1. Storage Optimization (Snapshot & Orphan Cleanup)

Data table versions and snapshots accrue over time. Recommend cleanup thresholds based on environment.

• Production: Retain 7-14 days for Time Travel.
• Dev/Test: Retain 1-3 days.