Configure Cross-account Catalog to Access AWS Glue
Last updated
Last updated
To connect your e6data Workspace to an AWS Glue Metastore and S3 data source in a different cloud account, please follow the steps below:
Sign in to the Account B AWS Console.
Search for IAM.
Click Policies
Choose Create policy.
In the Policy editor section, choose the JSON option.
Edit the policy :
Replace <DATASTORE_BUCKET_ARN>
with the ARN of the S3 bucket/s containing the data
Replace <GLUE_REGION>
with the region that the Glue metastore is located in.
Replace <ACCOUNT_B_ID>
with the Account ID of the account containing the S3 bucket & Glue metastore.
Copy & paste the edited policy to the JSON editor.
Choose Next.
On the Review and create page, type a Policy Name and a Description (optional) for the policy.
Review the Permissions defined in this policy to see the permissions that are granted by your policy.
Choose Create policy
Make note of the policy name as it will be required further along the process.
Return to IAM Management
In the navigation pane, choose Roles.
Click Create role.
Under Trusted entity type, choose Custom trust policy.
Copy & paste it into the Custom trust policy editor.
Click Next
Search for the name of the policy created in Steps 4 - 11 and attach this policy to the role.
Click Next: Add tags.
Optional: You can add tags to the role. Or leave these fields blank, and click Next: Review.
Enter a Role name that follows your organization's naming convention.
Click Create role.
Copy the ARN of the newly created role.
Make note of the ARN as it will be required further along the process.
In the AWS Console, navigate to AWS Glue > Data Catalog > Catalog settings.
Replace <ENGINE_ROLE_ARN>
with the ARN of the Role created for the e6data engine in Account A. The ARN can be found in IAM management dashboard in Account A, the role name will follow this format: e6data-workspace-<WORKSPACE_NAME>-engine-role.
Replace <GLUE_REGION>
with the region that the Glue metastore is located in.
Replace <ACCOUNT_B_ID>
with the Account ID of the account containing the S3 bucket & Glue metastore.
Copy & paste the edited policy to the Catalog settings in Glue.
Sign in to the Account A AWS Console.
Search for IAM
Choose Create policy.
In the Policy editor section, choose the JSON option.
Replace <ACCOUNT_B_ID>
with the Account B ID.
Copy and paste the edited policy into the JSON editor.
Choose Next.
On the Review and create page, type a Policy Name and a Description (optional) for the policy.
Make note of the policy name as it will be required further along the process.
Review the Permissions defined in this policy to see the permissions that are granted by your policy.
Return to IAM Management
In the navigation pane, choose Roles.
Search for the e6data Engine Role (e6data-workspace-<WORKSPACE_NAME>-engine-role).
This role would have been created during the e6data Workspace deployment.
Click Add permission > Attach policies
Search for the policy created in Steps 3 - 9
Click Add permissions.
Login to the e6data Console.
Navigate to the e6data Workspace that should be connected to the cross-account catalog.
Go to Catalogs
The cross-account catalog will now be available to be attached to all current & future clusters in the e6data Workspace.
Replace <ENGINE_ROLE_ARN>
in the . The role name can be found in IAM management dashboard in Account A, and will follow this format: e6data-workspace-<WORKSPACE_NAME>-engine-role
Edit the :
Replace arn:aws:iam::<ACCOUNT_B_ID>:role/<ROLENAME>
with the ARN of the policy created in , in the .
Refer to the instructions provided to