Workspace Creation

This page outlines the porcess of creating a workspace for connecting to e6data serverless compute

Workspace Creation

1. Log in to the e6data Console.

2. Navigate to Workspaces from the left navigation bar.

3. Click Create Workspace.

4. Select Serverless as the workspace type.

5. Select the Version from the available options.

6. Choose the preferred region.

7. Click Create.

The workspace creation process takes approximately 10 minutes. Once the status changes to Active, proceed to the next step.

Configuring Cloud Resources for Workspace

1. Within the created workspace, navigate to the Connectivity panel.

2. Click on Cloud Resource and then Create.

3. Enter the following details:

   1. External ID: Auto-generated ID for cross-account role validation.

   2. Cloud Resource Name: Name for the CloudFormation template.

   3. Version: Select the latest version.

   4. Cross Account Role ARN: To be obtained from CloudFormation stack outputs.

4. Click on the provided link to run the CloudFormation script, this action redirects you to the AWS CloudFormation page in the AWS console

5. In the AWS console, validate the Stack Name with the Cloud Resource Name provided earlier.

6. Provide the following parameters:

   1. BucketNames: List of S3 buckets containing data to be queried.

   2. e6dataAccountId: The E6Data AWS account ID for cross-account access.

   3. ExternalId: External ID for additional validation.

   4. VPCEndpointId: ID of the VPC Endpoint for secure data access.

7. Acknowledge that AWS CloudFormation might create IAM resources with custom names.

8. Click Create Stack. The process completes in 2-3 minutes.

9. Once the status is CREATE_COMPLETE, navigate to the Outputs section and copy the RoleArn.

10. Return to the e6data console, paste the Cross Account Role ARN, and click Create.

IAM Policies for Serverless Mode Attach this policy to the role associated with the cloud resources created using the CloudFormation template.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectTagging",
                "s3:GetObjectVersion",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::/"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Action": [
                "glue:GetDatabase*",
                "glue:GetTable*",
                "glue:GetPartitions",
                "glue:CreateTable",
                "glue:UpdateTable",
                "glue:DeleteTable"
            ],
            "Resource": "*",
            "Effect": "Allow"
        },
        {
            "Action": [
                "cloudformation:CreateChangeSet",
                "cloudformation:DescribeChangeSet",
                "cloudformation:DescribeStacks"
            ],
            "Resource": "<CLOUDFORMATION_STACK_ARN>",
            "Effect": "Allow"
        }
    ]
}

This policy grants the necessary permissions for the e6data engine role to:

• List the contents of the S3 bucket (s3:ListBucket).

The following permissions enable users to interact with objects within the S3 bucket

• s3:PutObject – Upload new objects to the bucket.

• s3:GetObject – Retrieve stored objects for processing.

• s3:GetObjectTagging – View metadata tags associated with an object.

• s3:GetObjectVersion – Access previous versions of objects if versioning is enabled.

• s3:PutObjectTagging – Assign metadata tags to objects for categorization and searchability

Accessing and Managing Glue Data Catalog

With the provided Glue permissions, users can interact with the AWS Glue Data Catalog, which is essential for organizing and querying structured datasets:

• glue:GetDatabase* – Retrieve details of available databases within AWS Glue.

• glue:GetTable* – Access metadata about tables in the Glue Data Catalog.

• glue:GetPartitions – Fetch partition information to optimize query performance.

• glue:CreateTable – Enables the creation of views generated by E6Data.

• glue:UpdateTable – Enables updates to views generated by E6Data.

• glue:DeleteTable – Enables the deletion of views generated by E6Data.