Cross-Catalog & Cross-Schema Querying
This feature enables you to combine and analyze data from different schemas and catalogs, providing a comprehensive view of your data landscape.
To perform cross-schema queries, you need to fully qualify the path by providing the catalog name, schema name, and table name.
Usage Examples:
Cross-Schema Query: This query retrieves columns
column1
andcolumn2
from the tabletable_name
within the specifiedschema_name
andcatalog_name
.
Cross-Catalog Query: This query retrieves columns
column1
andcolumn2
from the tabletable_name
within the specifiedschema_name
andcatalog_name
.
Limitations:
Slight Increase in Parsing Time: Due to the additional complexity introduced by cross-schema and cross-catalog querying, there may be a slight increase in parsing time for initial queries. However, subsequent queries benefit from query plan caching and performance optimizations.
Fully Qualified Path Requirement: When performing cross-schema or cross-catalog queries, it is essential to provide the entire path, including the catalog name, schema name, and table name.
Examples:
Hive Catalog:
<catalog_name>.<db_name>.<table_name>
catalog_name
- name of catalog specified during catalog creation in e6dataGlue Catalog:
<catalog_name>.<db_name>.<table_name> catalog_name
- name of catalog specified during catalog creation in e6dataUnity Metastore:
<unity_catalog_name>.<db_name>.<table_name> unity_catalog_name
- name of catalog which is created under Unity in Databricks
Same Account and Region for Cross-Catalog Queries: To query tables across catalogs, the catalogs must reside within the same account and region. This limitation ensures data security and optimal performance.
Last updated