Skill: Enterprise Lakehouse Table Inspection & Discovery

Description

Provides the agent with methodologies to discover data lake assets, connect to Enterprise catalogs (Unity Catalog, AWS Glue, Polaris), and inspect Delta Lake, Apache Iceberg, and Apache Hudi tables.

Context

Engineers need deep metadata without heavy compute. The agent must seamlessly connect to the configured metadata catalog and use lightweight Python libraries to retrieve schemas, partition layouts, and snapshot histories.

Instructions

1. Catalog Connection

Always ask the user for their preferred catalog or check their environment variables (AWS_PROFILE, DATABRICKS_HOST).

Iceberg via REST/Polaris:

from pyiceberg.catalog import load_catalog