LogoLogo
  • Welcome to e6data
  • Introduction to e6data
    • Concepts
    • Architecture
      • e6data in VPC Deployment Model
      • Connect to e6data serverless compute
  • Get Started
  • Sign Up
  • Setup
    • AWS Setup
      • In VPC Deployment (AWS)
        • Prerequisite Infrastructure
        • Infrastructure & Permissions for e6data
        • Setup Kubernetes Components
        • Setup using Terraform in AWS
          • Update a AWS Terraform for your Workspace
        • AWS PrivateLink and e6data
        • VPC Peering | e6data on AWS
      • Connect to e6data serverless compute (AWS)
        • Workspace Creation
        • Catalog Creation
          • Glue Metastore
          • Hive Metastore
          • Unity Catalog
        • Cluster Creation
    • GCP Setup
      • In VPC Deployment (GCP)
        • Prerequisite Infrastructure
        • Infrastructure & Permissions for e6data
        • Setup Kubernetes Components
        • Setup using Terraform in GCP
        • Update a GCP Terraform for your Workspace
      • Connect to e6data serverless compute (GCP)
    • Azure Setup
      • Prerequisite Infrastructure
      • Infrastructure & Permissions for e6data
      • Setup Kubernetes Components
      • Setup using Terraform in AZURE
        • Update a AZURE Terraform for your Workspace
  • Workspaces
    • Create Workspaces
    • Enable/Disable Workspaces
    • Update a Workspace
    • Delete a Workspace
  • Catalogs
    • Create Catalogs
      • Hive Metastore
        • Connect to a Hive Metastore
        • Edit a Hive Metastore Connection
        • Delete a Hive Metastore Connection
      • Glue Metastore
        • Connect to a Glue Metastore
        • Edit a Glue Metastore Connection
        • Delete a Glue Metastore Connection
      • Unity Catalog
        • Connect to Unity Catalog
        • Edit Unity Catalog
        • Delete Unity Catalog
      • Cross-account Catalog Access
        • Configure Cross-account Catalog to Access AWS Hive Metastore
        • Configure Cross-account Catalog to Access Unity Catalog
        • Configure Cross-account Catalog to Access AWS Glue
        • Configure Cross-account Catalog to Access GCP Hive Metastore
    • Manage Catalogs
    • Privileges
      • Access Control
      • Column Masking
      • Row Filter
  • Clusters
    • Edit & Delete Clusters
    • Suspend & Resume Clusters
    • Cluster Size
    • Load Based Sizing
    • Auto Suspension
    • Query Timeout
    • Monitoring
    • Connection Info
  • Pools
    • Delete Pools
  • Query Editor
    • Editor Pane
    • Results Pane
    • Schema Explorer
    • Data Preview
  • Notebook
    • Editor Pane
    • Results Pane
    • Schema Explorer
    • Data Preview
  • Query History
    • Query Count API
  • Connectivity
    • IP Sets
    • Endpoints
    • Cloud Resources
    • Network Firewall
  • Access Control
    • Users
    • Groups
    • Roles
      • Permissions
      • Policies
    • Single Sign-On (SSO)
      • AWS SSO
      • Okta
      • Microsoft My Apps-SSO
      • Icons for IdP
    • Service Accounts
    • Multi-Factor Authentication (Beta)
  • Usage and Cost Management
  • Audit Log
  • User Settings
    • Profile
    • Personal Access Tokens (PAT)
  • Advanced Features
    • Cross-Catalog & Cross-Schema Querying
  • Supported Data Types
  • SQL Command Reference
    • Query Syntax
      • General functions
    • Aggregate Functions
    • Mathematical Functions & Operators
      • Arithematic Operators
      • Rounding and Truncation Functions
      • Exponential and Root Functions
      • Trigonometric Functions
      • Logarithmic Functions
    • String Functions
    • Date-Time Functions
      • Constant Functions
      • Conversion Functions
      • Date Truncate Function
      • Addition and Subtraction Functions
      • Extraction Functions
      • Format Functions
      • Timezone Functions
    • Conditional Expressions
    • Conversion Functions
    • Window Functions
    • Comparison Operators & Functions
    • Logical Operators
    • Statistical Functions
    • Bitwise Functions
    • Array Functions
    • Regular Expression Functions
    • Generate Functions
    • Cardinality Estimation Functions
    • JSON Functions
    • Checksum Functions
    • Unload Function (Copy into)
    • Struct Functions
  • Equivalent Functions & Operators
  • Connectors & Drivers
    • DBeaver
    • DbVisualiser
    • Apache Superset
    • Jupyter Notebook
    • Tableau Cloud
    • Tableau Desktop
    • Power BI
    • Metabase
    • Zeppelin
    • Python Connector
      • Code Samples
    • JDBC Driver
      • Code Samples
      • API Support
    • Configure Cluster Ingress
      • ALB Ingress in Kubernetes
      • GCE Ingress in Kubernetes
      • Ingress-Nginx in Kubernetes
  • Security & Trust
    • Best Practices
      • AWS Best Practices
    • Features & Responsibilities Matrix
    • Data Protection Addendum(DPA)
  • Tutorials and Best Practices
    • How to configure HIVE metastore if you don't have one?
    • How-To Videos
  • Known Limitations
    • SQL Limitations
    • Other Limitations
    • Restart Triggers
    • Cloud Provider Limitations
  • Error Codes
    • General Errors
    • User Account Errors
    • Workspace Errors
    • Catalog Errors
    • Cluster Errors
    • Data Governance Errors
    • Query History Errors
    • Query Editor Errors
    • Pool Errors
    • Connectivity Errors
  • Terms & Condition
  • Privacy Policy
    • Cookie Policy
  • FAQs
    • Workspace Setup
    • Security
    • Catalog Privileges
  • Services Utilised for e6data Deployment
    • AWS supported regions
    • GCP supported regions
    • AZURE supported regions
  • Release Notes & Updates
    • 6th Sept 2024
    • 6th June 2024
    • 18th April 2024
    • 9th April 2024
    • 30th March 2024
    • 16th March 2024
    • 14th March 2024
    • 12th March 2024
    • 2nd March 2024
    • 10th February 2024
    • 3rd February 2024
    • 17th January 2024
    • 9th January 2024
    • 3rd January 2024
    • 18th December 2023
    • 12th December 2023
    • 9th December 2023
    • 4th December 2023
    • 27th November 2023
    • 8th September 2023
    • 4th September 2023
    • 26th August 2023
    • 21st August 2023
    • 19th July 2023
    • 23rd May 2023
    • 5th May 2023
    • 28th April 2023
    • 19th April 2023
    • 15th April 2023
    • 10th April 2023
    • 30th March 2023
Powered by GitBook
On this page
  • Create a VPC, Subnets & other VPC Resources
  • Create GKE Cluster & Default Node Pool
  • Create GKE Cluster
  1. Setup
  2. GCP Setup
  3. In VPC Deployment (GCP)

Prerequisite Infrastructure

PreviousIn VPC Deployment (GCP)NextInfrastructure & Permissions for e6data

Last updated 2 months ago

The following components are required prior to setting up the infrastructure needed by e6data. These are commonly present in most cloud environments, but if any are not present, please follow the linked guides below to create them.

Create a VPC, Subnets & other VPC Resources

Optional, only required if a VPC is not already present to create a GKE Cluster or to install e6data in a new VPC.

For custom-mode VPC networks, first create a network and then establish the desired subnets within a region. You are not required to specify subnets for all regions immediately or at all, but you cannot create instances in a region without defined subnets. Lastly, define the firewall rules for your network.

To create a custom-mode VPC network with only IPv4 subnets, follow these steps.

  1. In the Google Cloud console, go to the VPC networks page.

  2. Click Create VPC network.

  3. Enter a Name for the network.

  4. Choose Custom for the Subnet creation mode.

  5. In the New subnet section, specify the following configuration parameters for a subnet:

    1. Provide a Name for the subnet.

    2. Select a Region.

    3. Enter an IP address range. This is the for the subnet.

      If you select a range that is not an RFC 1918 address, confirm that the range doesn't conflict with an existing configuration. For more information, see .

    4. To define a secondary range for the subnet, click Create secondary IP range.

      If you select a range that is not an RFC 1918 address, confirm that the range doesn't conflict with an existing configuration. For more information, see .

    5. Private Google Access: Choose whether to enable for the subnet when you create it or later by editing it.

    6. Flow logs: Choose whether to enable for the subnet when you create it or later by editing it.

    7. Click Done.

  6. To add more subnets, click Add Subnet and repeat the previous steps. You can also to the network after you have created the network.

  7. In the Firewall rules section, select zero or more predefined in the IPv4 firewall rules. The rules address common use cases for connectivity to instances.

    If you don't select any predefined rules, you can after you create the network.

    Each predefined rule name starts with the name of the VPC network that you are creating, NETWORK. In the IPv4 firewall rules tab, the predefined ingress firewall rule named NETWORK-allow-custom is editable. On the right side of the row that contains the rule, click Edit to select subnets, add additional IPv4 ranges, and specify protocols and ports.

    The NETWORK-allow-custom firewall rule is not automatically updated if you later add additional subnets. If you need firewall rules for the new subnets, you must update the firewall configuration to add the rules.

  8. Choose the Dynamic routing mode for the VPC network.

    For more information, see . You can later.

  9. Maximum transmission unit (MTU): Choose whether the network has an MTU of 1460 (default), 1500, or 8896. Before setting the MTU to a value higher than 1460, review .

  10. Click Create.

Please make note of the VPC Region, it will be required when creating the Workspace in the e6data Console.

Create GKE Cluster & Default Node Pool

Optional, only required if a GKE Cluster is not present or to install e6data in a new GKE Cluster.

Create GKE Cluster

To get started with setting up a Google Kubernetes Engine (GKE) cluster along with a default node pool, please follow the comprehensive documentation provided by GCP:

For high availability, it is advisable to utilize a regional cluster.

Please make note of the GKE Cluster Name, it will be required when creating the Workspace in the e6data Console.

Go to VPC networks
primary IPv4 range
IPv4 subnet ranges
IPv4 subnet ranges
Private Google Access
VPC flow logs
add more subnets
firewall rules
create your own firewall rules
dynamic routing mode
change the dynamic routing mode
Maximum transmission unit
Virtual Private Cloud (VPC)
Google Kubernetes Engine (GKE) cluster
Create GKE Cluster
Creating a regional cluster | Google Kubernetes Engine (GKE) | Google Cloud