High-level Architecture
This page provides a high-level overview of e6 query engine architecture. It introduces the key constructs of e6data query engine and describes the deployment models we support.
Platform layers
The platform is organized into two planes and your data:
Control Plane
The e6data Console where you manage your account, workspaces, billing, and global access control. This is what you see when you sign in at the e6data URL. The Control Plane is shared across all your workspaces - it doesn't process queries or touch your data.
Organization and account
Tenancy, membership, and billing
Workspace management
Create, open, disable, and delete workspaces
Global access control
Users, groups, roles, and SSO
Releases and monitoring
Version management and platform telemetry
Compute Plane
The workspace-scoped environment where catalogs, clusters, and queries live. You enter a Compute Plane by opening a workspace from the Control Plane. Each workspace has its own Compute Plane, which includes the SQL Editor, the catalogs you've connected, the clusters you've created, Run History, Access Control, Service Accounts, and Settings.
Catalogs
Connections to your metastores (Catalogs)
Clusters
The compute that runs queries (Clusters)
SQL Editor
Write and run queries in the browser (Query editor)
Run History
A searchable log of every query (Query history)
Settings
Access control, version upgrades, observability, and connectivity
Your data
Stays in your cloud account. e6data reads data directly from S3, ADLS Gen2, or your Iceberg/Hive metastore. It never copies your data into a separate store.
Core objects
Account
An e6data account is the top-level entity for handling all your workloads - billing, support, and tenant membership are scoped to the account.
Tenant
Within an account, a tenant is your organization. Multiple users belong to one tenant, and resources (workspaces, catalogs, clusters) belong to a tenant.
Workspace
A workspace is an isolated environment for a team or project. It has its own Compute Plane, catalogs, clusters, and access control. You can have one workspace or many - separate workspaces for prod vs. dev, or per business unit, are both common patterns.
When you create a workspace, you choose a Workspace Type (Serverless or In Your VPC), a Cloud Provider (AWS or Azure), and a Region where your data resides.
Cluster
A cluster is the compute that runs your queries inside a workspace. Each cluster has a Cluster Runtime (CRT) version, a Cluster Size (for example, XS, S, M), and optional Autoscaling, QPS-based scaling, and Auto-Suspension configuration. Clusters can be connected to one or more catalogs.
Catalog
A catalog connects e6data to your metastore so it can discover databases, tables, and columns. Supported catalog types: Hive Metastore, AWS Glue, Databricks Unity, Microsoft Fabric (OneLake), Apache Polaris, Iceberg REST Catalog, and AWS S3 Tables.
Querying
SQL Editor
The native SQL console inside the Compute Plane where you write and run queries.
Query history
A searchable log of all queries run in a workspace, with timing, status, user, and query text. Useful for audit, debugging, and cost attribution.
e6data Deployment Flavors
e6data offers two deployment models. In both, the Control Plane (workspace management, access control, monitoring, releases) is hosted by e6data, and your data always stays in your cloud account - only the Compute Plane differs.
Compute Plane (clusters, planners, executors)
Runs in e6data's cloud
Runs in your Kubernetes cluster (EKS, AKS)
Catalog metadata, query history
e6data
Your cluster + your cloud storage
Your data
Your cloud account, never moved
Your cloud account, never moved
Query traffic
To e6data's managed compute
Stays entirely within your network
You manage
Nothing - fully managed
The Kubernetes cluster, IAM, and storage lifecycle
Serverless Model
e6data fully manages the Compute Plane in its own cloud, eliminating the need to operate Kubernetes or infrastructure. Secure, short-lived cloud identity federation provides access to your storage without storing long-lived credentials.
Choose Serverless when you want the fastest start, don't need to host compute yourself, and want e6data to handle scaling and upgrades. Less ideal when compliance mandates compute inside your VPC, you have strict data-locality needs, or you're air-gapped.
Set up: AWS Serverless · Azure Serverless.
In Your VPC - Cloud prem Model
The Compute Plane runs within your Kubernetes cluster, ensuring query traffic remains inside your private network. You manage the underlying infrastructure and cloud IAM, while e6data manages the platform software, releases, and workspace lifecycle.
Choose In Your VPC when security or compliance requires compute inside your VPC, you're in a regulated industry, or your data sources have no public routing. Less ideal when you want the fastest start or don't have a platform team to run the cluster.
Set up: AWS In-VPC / CloudPrem · Azure In-VPC / CloudPrem.
Last updated
Was this helpful?