Federate Lakehouse Catalog via Cross-cloud Lakehouse

This skill describes how to set up a federated catalog in BigQuery to query remote catalogs like Databricks Unity Catalog or AWS Glue Data Catalog data in AWS over the public internet.

Prerequisites

• For Databricks: Databricks Workspace URL and OAuth Service Principal (Client ID and Secret) with read access.
• For AWS Glue: AWS Administrator access to create IAM roles and permissions policies.
• Active Google Cloud project with administrative access to create lakehouse resources, and secrets in the case of Databricks.

Procedure

Step 1: Information Gathering and Region Selection