> For the complete documentation index, see [llms.txt](https://docs.e6data.com/query-engine/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.e6data.com/query-engine/guides/deployment/aws-in-vpc/configure-registry-vpc-eks-networking.md).

# Configure registry, VPC, EKS, and networking

Build the AWS infrastructure (VPC, IAM, EKS, networking, ECR access) and install the Kubernetes platform components for an In-VPC e6data deployment.

This page builds the AWS infrastructure for an In-VPC deployment and installs the Kubernetes platform components that the e6data workspace runs on. Run these steps with the AWS CLI, `kubectl`, and Helm.

## Infrastructure requirements

Review these requirements before you begin. They apply whether you create a new VPC (Part 1, Step 1) or reuse an existing one.

### VPC options

You can either create a new VPC or use an existing one:

| Option           | When to use                                                                                                               |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **New VPC**      | Recommended for an isolated e6data deployment. Follow Step 1 in Part 1 below.                                             |
| **Existing VPC** | Use when e6data must share a network with other workloads. Skip Step 1 and confirm your VPC meets the requirements below. |

### VPC and subnet CIDR ranges

| Resource            | Minimum                    | Recommended                | Notes                                           |
| ------------------- | -------------------------- | -------------------------- | ----------------------------------------------- |
| **VPC CIDR**        | /20 (4,096 IPs)            | /16 (65,536 IPs)           | Must accommodate EKS nodes, pods, and services. |
| **Private subnets** | /22 per subnet (1,024 IPs) | /20 per subnet (4,096 IPs) | e6data engine nodes run here.                   |
| **Public subnets**  | /24 per subnet (256 IPs)   | /24 per subnet             | NAT Gateway and load balancers only.            |

Each e6data engine pod consumes one IP address. For large workloads with 100+ concurrent pods, make sure private subnets have enough IP space.

### Availability zones

| Requirement         | Minimum  | Recommended |
| ------------------- | -------- | ----------- |
| **Number of AZs**   | 2        | 3           |
| **Private subnets** | 1 per AZ | 1 per AZ    |
| **Public subnets**  | 1 per AZ | 1 per AZ    |

e6data spreads workloads across AZs for high availability. Tag subnets so EKS, the load balancer controller, and Karpenter can discover them:

```bash
# Private subnets (internal load balancers)
kubernetes.io/role/internal-elb = 1
# Public subnets (internet-facing load balancers)
kubernetes.io/role/elb = 1
# All subnets (EKS auto-discovery)
kubernetes.io/cluster/<cluster-name> = shared
# Private subnets (Karpenter node provisioning)
karpenter.sh/discovery = <cluster-name>-eks
```

### Service quotas

Verify these quotas before deploying. Request increases early - they can take 24–48 hours.

| Service | Quota                       | Minimum required |
| ------- | --------------------------- | ---------------- |
| EC2     | Running On-Demand instances | 100 vCPUs        |
| EC2     | Running Spot instances      | 500 vCPUs        |
| EC2     | EC2-VPC Elastic IPs         | 5                |
| EKS     | Clusters per Region         | 1+ available     |
| VPC     | NAT Gateways per AZ         | 1+ available     |
| VPC     | VPCs per Region             | 1+ available     |

### Instance types

| Component    | Instance families              | Architecture     | Notes                                        |
| ------------ | ------------------------------ | ---------------- | -------------------------------------------- |
| System nodes | t3, m5                         | x86\_64 (amd64)  | EKS managed node group for system pods.      |
| e6-operator  | t3, t4g                        | amd64 or arm64   | Operator controller pods.                    |
| Engine nodes | r6g, r6gd, r7g, r7gd, m7g, c7g | arm64 (Graviton) | Query engine pods, provisioned by Karpenter. |

The e6data engine is optimized for AWS Graviton (arm64) instances for better price-performance. Confirm your Region has capacity for the r6g/r7g families.

### VPC CNI configuration

The Amazon VPC CNI plugin manages pod networking:

| Setting                    | Value | Rationale                                                        |
| -------------------------- | ----- | ---------------------------------------------------------------- |
| `WARM_IP_TARGET`           | 5     | Pre-allocates IPs for faster pod startup.                        |
| `MINIMUM_IP_TARGET`        | 10    | Keeps a minimum number of IPs available per node.                |
| `ENABLE_PREFIX_DELEGATION` | true  | Increases IP capacity per node (recommended for large clusters). |

For clusters with 100+ pods, enable prefix delegation to avoid IP exhaustion. It assigns /28 prefixes (16 IPs) instead of individual IPs, allowing up to 110 pods per node on most instance types:

```bash
kubectl set env daemonset aws-node -n kube-system ENABLE_PREFIX_DELEGATION=true
kubectl set env daemonset aws-node -n kube-system WARM_PREFIX_TARGET=1
```

### Existing VPC checklist

If you reuse an existing VPC, verify:

* [ ] DNS hostnames enabled (`enableDnsHostnames: true`).
* [ ] DNS resolution enabled (`enableDnsSupport: true`).
* [ ] Private subnets route to a NAT Gateway for internet access.
* [ ] Private subnets tagged `kubernetes.io/role/internal-elb = 1`.
* [ ] Public subnets tagged `kubernetes.io/role/elb = 1`.
* [ ] Private subnets tagged `karpenter.sh/discovery = <cluster-name>-eks`.
* [ ] Cluster security group tagged `karpenter.sh/discovery = <cluster-name>-eks` (after EKS creation).
* [ ] Sufficient IP space in private subnets (minimum /22 per subnet).

## Part 1 - AWS infrastructure

### Step 1: Create the VPC

```bash
# Set your variables
export CLUSTER_NAME="e6data-cluster"
export REGION="us-east-1"
export VPC_CIDR="10.0.0.0/16"

# Create VPC
VPC_ID=$(aws ec2 create-vpc \
  --cidr-block $VPC_CIDR \
  --tag-specifications "ResourceType=vpc,Tags=[{Key=Name,Value=${CLUSTER_NAME}-vpc}]" \
  --query 'Vpc.VpcId' --output text --region $REGION)

echo "VPC_ID=$VPC_ID"

# Enable DNS hostnames
aws ec2 modify-vpc-attribute --vpc-id $VPC_ID --enable-dns-hostnames --region $REGION

# Create Internet Gateway
IGW_ID=$(aws ec2 create-internet-gateway \
  --tag-specifications "ResourceType=internet-gateway,Tags=[{Key=Name,Value=${CLUSTER_NAME}-igw}]" \
  --query 'InternetGateway.InternetGatewayId' --output text --region $REGION)

aws ec2 attach-internet-gateway --vpc-id $VPC_ID --internet-gateway-id $IGW_ID --region $REGION

# Get availability zones
AZS=$(aws ec2 describe-availability-zones --region $REGION --query 'AvailabilityZones[0:3].ZoneName' --output text)
AZ1=$(echo $AZS | awk '{print $1}')
AZ2=$(echo $AZS | awk '{print $2}')
AZ3=$(echo $AZS | awk '{print $3}')

# Create public subnets
PUBLIC_SUBNET_1=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.1.0/24 \
  --availability-zone $AZ1 \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=Name,Value=${CLUSTER_NAME}-public-1},{Key=kubernetes.io/role/elb,Value=1}]" \
  --query 'Subnet.SubnetId' --output text --region $REGION)

PUBLIC_SUBNET_2=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.2.0/24 \
  --availability-zone $AZ2 \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=Name,Value=${CLUSTER_NAME}-public-2},{Key=kubernetes.io/role/elb,Value=1}]" \
  --query 'Subnet.SubnetId' --output text --region $REGION)

# Create private subnets
PRIVATE_SUBNET_1=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.10.0/24 \
  --availability-zone $AZ1 \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=Name,Value=${CLUSTER_NAME}-private-1},{Key=kubernetes.io/role/internal-elb,Value=1}]" \
  --query 'Subnet.SubnetId' --output text --region $REGION)

PRIVATE_SUBNET_2=$(aws ec2 create-subnet --vpc-id $VPC_ID --cidr-block 10.0.11.0/24 \
  --availability-zone $AZ2 \
  --tag-specifications "ResourceType=subnet,Tags=[{Key=Name,Value=${CLUSTER_NAME}-private-2},{Key=kubernetes.io/role/internal-elb,Value=1}]" \
  --query 'Subnet.SubnetId' --output text --region $REGION)

# Create Elastic IP for NAT Gateway
EIP_ALLOC=$(aws ec2 allocate-address --domain vpc \
  --tag-specifications "ResourceType=elastic-ip,Tags=[{Key=Name,Value=${CLUSTER_NAME}-nat-eip}]" \
  --query 'AllocationId' --output text --region $REGION)

# Create NAT Gateway
NAT_GW=$(aws ec2 create-nat-gateway \
  --subnet-id $PUBLIC_SUBNET_1 \
  --allocation-id $EIP_ALLOC \
  --tag-specifications "ResourceType=natgateway,Tags=[{Key=Name,Value=${CLUSTER_NAME}-nat}]" \
  --query 'NatGateway.NatGatewayId' --output text --region $REGION)

echo "Waiting for NAT Gateway to become available..."
aws ec2 wait nat-gateway-available --nat-gateway-ids $NAT_GW --region $REGION

# Create route tables
PUBLIC_RT=$(aws ec2 create-route-table --vpc-id $VPC_ID \
  --tag-specifications "ResourceType=route-table,Tags=[{Key=Name,Value=${CLUSTER_NAME}-public-rt}]" \
  --query 'RouteTable.RouteTableId' --output text --region $REGION)

aws ec2 create-route --route-table-id $PUBLIC_RT --destination-cidr-block 0.0.0.0/0 \
  --gateway-id $IGW_ID --region $REGION

aws ec2 associate-route-table --route-table-id $PUBLIC_RT --subnet-id $PUBLIC_SUBNET_1 --region $REGION
aws ec2 associate-route-table --route-table-id $PUBLIC_RT --subnet-id $PUBLIC_SUBNET_2 --region $REGION

PRIVATE_RT=$(aws ec2 create-route-table --vpc-id $VPC_ID \
  --tag-specifications "ResourceType=route-table,Tags=[{Key=Name,Value=${CLUSTER_NAME}-private-rt}]" \
  --query 'RouteTable.RouteTableId' --output text --region $REGION)

aws ec2 create-route --route-table-id $PRIVATE_RT --destination-cidr-block 0.0.0.0/0 \
  --nat-gateway-id $NAT_GW --region $REGION

aws ec2 associate-route-table --route-table-id $PRIVATE_RT --subnet-id $PRIVATE_SUBNET_1 --region $REGION
aws ec2 associate-route-table --route-table-id $PRIVATE_RT --subnet-id $PRIVATE_SUBNET_2 --region $REGION

# Create S3 VPC Endpoint (saves NAT costs for S3 traffic)
aws ec2 create-vpc-endpoint \
  --vpc-id $VPC_ID \
  --service-name com.amazonaws.${REGION}.s3 \
  --route-table-ids $PRIVATE_RT \
  --tag-specifications "ResourceType=vpc-endpoint,Tags=[{Key=Name,Value=${CLUSTER_NAME}-s3-endpoint}]" \
  --region $REGION

echo "VPC setup complete!"
echo "VPC_ID=$VPC_ID"
echo "PUBLIC_SUBNETS=$PUBLIC_SUBNET_1,$PUBLIC_SUBNET_2"
echo "PRIVATE_SUBNETS=$PRIVATE_SUBNET_1,$PRIVATE_SUBNET_2"

# Tag private subnets for Karpenter discovery
aws ec2 create-tags --resources $PRIVATE_SUBNET_1 $PRIVATE_SUBNET_2 \
  --tags Key=karpenter.sh/discovery,Value=${CLUSTER_NAME}-eks \
  --region $REGION

# Note: the cluster security group is tagged for Karpenter after the EKS cluster is created (Step 3).
```

### Step 2: Create IAM roles

**EKS cluster role:**

```bash
cat > eks-cluster-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "eks.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

aws iam create-role \
  --role-name ${CLUSTER_NAME}-eks-cluster-role \
  --assume-role-policy-document file://eks-cluster-trust-policy.json

aws iam attach-role-policy \
  --role-name ${CLUSTER_NAME}-eks-cluster-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKSClusterPolicy
```

**EKS node role:**

```bash
cat > eks-node-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ec2.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

aws iam create-role \
  --role-name ${CLUSTER_NAME}-eks-node-role \
  --assume-role-policy-document file://eks-node-trust-policy.json

aws iam attach-role-policy --role-name ${CLUSTER_NAME}-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
aws iam attach-role-policy --role-name ${CLUSTER_NAME}-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
aws iam attach-role-policy --role-name ${CLUSTER_NAME}-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
aws iam attach-role-policy --role-name ${CLUSTER_NAME}-eks-node-role \
  --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore

aws iam create-instance-profile --instance-profile-name ${CLUSTER_NAME}-eks-node-profile
aws iam add-role-to-instance-profile \
  --instance-profile-name ${CLUSTER_NAME}-eks-node-profile \
  --role-name ${CLUSTER_NAME}-eks-node-role
```

**Karpenter role:**

```bash
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

cat > karpenter-trust-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}
EOF

aws iam create-role \
  --role-name ${CLUSTER_NAME}-karpenter-role \
  --assume-role-policy-document file://karpenter-trust-policy.json

cat > karpenter-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Karpenter",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateFleet", "ec2:CreateLaunchTemplate", "ec2:CreateTags",
        "ec2:DeleteLaunchTemplate", "ec2:DescribeAvailabilityZones",
        "ec2:DescribeImages", "ec2:DescribeInstances",
        "ec2:DescribeInstanceTypeOfferings", "ec2:DescribeInstanceTypes",
        "ec2:DescribeLaunchTemplates", "ec2:DescribeSecurityGroups",
        "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets",
        "ec2:RunInstances", "ec2:TerminateInstances",
        "iam:PassRole", "pricing:GetProducts", "ssm:GetParameter"
      ],
      "Resource": "*"
    },
    {
      "Sid": "SQS",
      "Effect": "Allow",
      "Action": ["sqs:DeleteMessage", "sqs:GetQueueAttributes", "sqs:GetQueueUrl", "sqs:ReceiveMessage"],
      "Resource": "arn:aws:sqs:${REGION}:${AWS_ACCOUNT_ID}:${CLUSTER_NAME}-karpenter"
    }
  ]
}
EOF

aws iam put-role-policy \
  --role-name ${CLUSTER_NAME}-karpenter-role \
  --policy-name KarpenterPolicy \
  --policy-document file://karpenter-policy.json

aws iam create-instance-profile --instance-profile-name ${CLUSTER_NAME}-karpenter
aws iam add-role-to-instance-profile \
  --instance-profile-name ${CLUSTER_NAME}-karpenter \
  --role-name ${CLUSTER_NAME}-eks-node-role
```

**ALB Controller role:**

```bash
aws iam create-role \
  --role-name ${CLUSTER_NAME}-alb-controller-role \
  --assume-role-policy-document file://karpenter-trust-policy.json

curl -o alb-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json

aws iam put-role-policy \
  --role-name ${CLUSTER_NAME}-alb-controller-role \
  --policy-name ALBControllerPolicy \
  --policy-document file://alb-policy.json
```

**EBS CSI driver role:**

The EBS CSI driver addon uses EKS Pod Identity for its IAM permissions:

```bash
cat > ebs-csi-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}
EOF

aws iam create-role \
  --role-name ${CLUSTER_NAME}-ebs-csi-driver-role \
  --assume-role-policy-document file://ebs-csi-trust-policy.json

aws iam attach-role-policy \
  --role-name ${CLUSTER_NAME}-ebs-csi-driver-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy
```

### Step 3: Create the EKS cluster

```bash
aws eks create-cluster \
  --name ${CLUSTER_NAME}-eks \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-eks-cluster-role \
  --resources-vpc-config subnetIds=${PRIVATE_SUBNET_1},${PRIVATE_SUBNET_2},endpointPublicAccess=true,endpointPrivateAccess=true \
  --access-config authenticationMode=API_AND_CONFIG_MAP \
  --kubernetes-version 1.31 \
  --region $REGION

echo "Waiting for EKS cluster to become active (this takes ~10-15 minutes)..."
aws eks wait cluster-active --name ${CLUSTER_NAME}-eks --region $REGION

aws eks update-kubeconfig --name ${CLUSTER_NAME}-eks --region $REGION

CLUSTER_ENDPOINT=$(aws eks describe-cluster --name ${CLUSTER_NAME}-eks \
  --query 'cluster.endpoint' --output text --region $REGION)
OIDC_ISSUER=$(aws eks describe-cluster --name ${CLUSTER_NAME}-eks \
  --query 'cluster.identity.oidc.issuer' --output text --region $REGION)

# Tag the cluster security group for Karpenter discovery
CLUSTER_SG=$(aws eks describe-cluster --name ${CLUSTER_NAME}-eks \
  --query 'cluster.resourcesVpcConfig.clusterSecurityGroupId' --output text --region $REGION)

aws ec2 create-tags --resources $CLUSTER_SG \
  --tags Key=karpenter.sh/discovery,Value=${CLUSTER_NAME}-eks \
  --region $REGION
```

### Step 4: Create the OIDC provider and Pod Identity add-ons

```bash
eksctl utils associate-iam-oidc-provider \
  --cluster ${CLUSTER_NAME}-eks --region $REGION --approve

aws eks create-addon --cluster-name ${CLUSTER_NAME}-eks --addon-name eks-pod-identity-agent --region $REGION
aws eks create-addon --cluster-name ${CLUSTER_NAME}-eks --addon-name vpc-cni --region $REGION

# Install the EBS CSI driver with a Pod Identity association so it can manage EBS volumes
aws eks create-addon --cluster-name ${CLUSTER_NAME}-eks --addon-name aws-ebs-csi-driver \
  --pod-identity-associations "serviceAccount=ebs-csi-controller-sa,roleArn=arn:aws:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-ebs-csi-driver-role" \
  --region $REGION
```

### Step 5: Create the system node group

```bash
aws eks create-nodegroup \
  --cluster-name ${CLUSTER_NAME}-eks \
  --nodegroup-name system-nodes \
  --scaling-config minSize=2,maxSize=4,desiredSize=2 \
  --instance-types t3.medium \
  --node-role arn:aws:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-eks-node-role \
  --subnets $PRIVATE_SUBNET_1 $PRIVATE_SUBNET_2 \
  --labels role=system \
  --region $REGION

aws eks wait nodegroup-active \
  --cluster-name ${CLUSTER_NAME}-eks --nodegroup-name system-nodes --region $REGION
```

### Step 6: Create the S3 metadata bucket

```bash
export WORKSPACE_NAME="my-workspace"  # Replace with your workspace name

# Note: us-east-1 is S3's default region and rejects LocationConstraint
if [ "$REGION" = "us-east-1" ]; then
  aws s3api create-bucket \
    --bucket e6-${WORKSPACE_NAME}-metadata \
    --region $REGION
else
  aws s3api create-bucket \
    --bucket e6-${WORKSPACE_NAME}-metadata \
    --region $REGION \
    --create-bucket-configuration LocationConstraint=$REGION
fi

aws s3api put-public-access-block \
  --bucket e6-${WORKSPACE_NAME}-metadata \
  --public-access-block-configuration \
  "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

aws s3api put-bucket-versioning \
  --bucket e6-${WORKSPACE_NAME}-metadata \
  --versioning-configuration Status=Enabled

aws s3api put-bucket-encryption \
  --bucket e6-${WORKSPACE_NAME}-metadata \
  --server-side-encryption-configuration '{
    "Rules": [{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]
  }'

cat > bucket-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "EnforceSSL",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata",
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata/*"
      ],
      "Condition": { "Bool": {"aws:SecureTransport": "false"} }
    }
  ]
}
EOF

aws s3api put-bucket-policy \
  --bucket e6-${WORKSPACE_NAME}-metadata \
  --policy file://bucket-policy.json
```

{% hint style="warning" %}
Do not enable S3 Object Lock on this bucket - it can break metadata operations.
{% endhint %}

### Step 7: Create workspace IAM roles

Create the **engine**, **console**, and **app** roles. All three trust `pods.eks.amazonaws.com`:

```bash
cat > engine-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}
EOF

# Engine role (query engine and Laminar)
aws iam create-role --role-name ${WORKSPACE_NAME}-engine-role \
  --assume-role-policy-document file://engine-trust-policy.json

cat > engine-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3MetadataAccess",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket", "s3:AbortMultipartUpload"],
      "Resource": [
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata",
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata/*"
      ]
    },
    {
      "Sid": "CrossAccountAccess",
      "Effect": "Allow",
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Resource": "*"
    }
  ]
}
EOF

aws iam put-role-policy --role-name ${WORKSPACE_NAME}-engine-role \
  --policy-name EnginePolicy --policy-document file://engine-policy.json

# Console role (UI backend)
aws iam create-role --role-name ${WORKSPACE_NAME}-console-role \
  --assume-role-policy-document file://engine-trust-policy.json

cat > console-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3MetadataAccess",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket", "s3:AbortMultipartUpload"],
      "Resource": [
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata",
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata/*"
      ]
    },
    {
      "Sid": "VPCEndpointDiscovery",
      "Effect": "Allow",
      "Action": ["ec2:DescribeVpcEndpoints", "ec2:DescribeInstances"],
      "Resource": "*"
    },
    {
      "Sid": "EKSPodIdentityLookup",
      "Effect": "Allow",
      "Action": ["eks:DescribeCluster", "eks:ListPodIdentityAssociations", "eks:DescribePodIdentityAssociation"],
      "Resource": [
        "arn:aws:eks:${REGION}:${AWS_ACCOUNT_ID}:cluster/${CLUSTER_NAME}-eks",
        "arn:aws:eks:${REGION}:${AWS_ACCOUNT_ID}:podidentityassociation/${CLUSTER_NAME}-eks/*"
      ]
    }
  ]
}
EOF

aws iam put-role-policy --role-name ${WORKSPACE_NAME}-console-role \
  --policy-name ConsolePolicy --policy-document file://console-policy.json

# App role (Copilot and Metriq)
aws iam create-role --role-name ${WORKSPACE_NAME}-app-role \
  --assume-role-policy-document file://engine-trust-policy.json

cat > app-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "S3MetadataAccess",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket", "s3:AbortMultipartUpload"],
      "Resource": [
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata",
        "arn:aws:s3:::e6-${WORKSPACE_NAME}-metadata/*"
      ]
    },
    {
      "Sid": "SecretsManagerAccess",
      "Effect": "Allow",
      "Action": ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret"],
      "Resource": "arn:aws:secretsmanager:${REGION}:${AWS_ACCOUNT_ID}:secret:copilot-metriq-secret-*"
    }
  ]
}
EOF

aws iam put-role-policy --role-name ${WORKSPACE_NAME}-app-role \
  --policy-name AppPolicy --policy-document file://app-policy.json
```

### Step 8: Create Pod Identity associations

```bash
kubectl create namespace $WORKSPACE_NAME

aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace $WORKSPACE_NAME --service-account ${WORKSPACE_NAME}-engine \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${WORKSPACE_NAME}-engine-role --region $REGION

aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace $WORKSPACE_NAME --service-account ${WORKSPACE_NAME}-console \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${WORKSPACE_NAME}-console-role --region $REGION

aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace $WORKSPACE_NAME --service-account ${WORKSPACE_NAME}-copilot \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${WORKSPACE_NAME}-app-role --region $REGION

aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace $WORKSPACE_NAME --service-account ${WORKSPACE_NAME}-metriq \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${WORKSPACE_NAME}-app-role --region $REGION

aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace $WORKSPACE_NAME --service-account ${WORKSPACE_NAME}-laminar \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${WORKSPACE_NAME}-engine-role --region $REGION
```

### Step 9: Create the Karpenter SQS queue

```bash
aws sqs create-queue --queue-name ${CLUSTER_NAME}-karpenter --region $REGION

QUEUE_ARN=$(aws sqs get-queue-attributes \
  --queue-url https://sqs.${REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${CLUSTER_NAME}-karpenter \
  --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

cat > sqs-policy.json << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"Service": "events.amazonaws.com"},
      "Action": "sqs:SendMessage",
      "Resource": "${QUEUE_ARN}"
    }
  ]
}
EOF

aws sqs set-queue-attributes \
  --queue-url https://sqs.${REGION}.amazonaws.com/${AWS_ACCOUNT_ID}/${CLUSTER_NAME}-karpenter \
  --attributes Policy="$(cat sqs-policy.json | jq -c .)"
```

## Part 2 - Kubernetes platform components

{% hint style="info" %}
e6data uses the `helm template | kubectl apply --server-side` pattern instead of `helm install` to avoid Helm's 1 MB release Secret limit and ensure clean field-manager handling during upgrades.
{% endhint %}

### Step 1: Install Karpenter CRDs

```bash
export KARPENTER_VERSION="1.8.1"
tmpdir=$(mktemp -d)

helm template karpenter-crd \
  oci://public.ecr.aws/karpenter/karpenter-crd \
  --version "${KARPENTER_VERSION}" --include-crds \
  > "${tmpdir}/karpenter-crds.yaml"

kubectl apply --server-side --force-conflicts -f "${tmpdir}/karpenter-crds.yaml"
kubectl wait --for=condition=Established crd/nodepools.karpenter.sh crd/nodeclaims.karpenter.sh --timeout=60s
kubectl get crds | grep karpenter
```

### Step 2: Install the Karpenter controller

```bash
CLUSTER_ENDPOINT=$(aws eks describe-cluster --name ${CLUSTER_NAME}-eks \
  --query 'cluster.endpoint' --output text --region $REGION)

cat > karpenter-values.yaml << EOF
settings:
  clusterName: ${CLUSTER_NAME}-eks
  clusterEndpoint: ${CLUSTER_ENDPOINT}
  interruptionQueue: ${CLUSTER_NAME}-karpenter
replicas: 2
controller:
  resources:
    requests: { cpu: 200m, memory: 256Mi }
    limits: { cpu: 1, memory: 1Gi }
EOF

helm template karpenter \
  oci://public.ecr.aws/karpenter/karpenter \
  --version "${KARPENTER_VERSION}" --namespace kube-system --skip-crds \
  -f karpenter-values.yaml \
  | kubectl apply --server-side --force-conflicts -f -

# Link the karpenter service account to the IAM role created in Part 1, Step 2
aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace kube-system --service-account karpenter \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-karpenter-role --region $REGION

# Restart Karpenter so it picks up the new credentials
kubectl rollout restart deployment/karpenter -n kube-system

kubectl rollout status deployment/karpenter -n kube-system --timeout=120s
kubectl get pods -n kube-system -l app.kubernetes.io/name=karpenter
```

### Step 3: Install the AWS Load Balancer Controller

```bash
export ALB_CONTROLLER_VERSION="1.8.1"

helm repo add eks https://aws.github.io/eks-charts
helm repo update eks

cat > alb-values.yaml << EOF
clusterName: ${CLUSTER_NAME}-eks
region: ${REGION}
vpcId: ${VPC_ID}
EOF

helm upgrade --install aws-load-balancer-controller \
  eks/aws-load-balancer-controller \
  --version "${ALB_CONTROLLER_VERSION}" --namespace kube-system \
  -f alb-values.yaml

# Link the aws-load-balancer-controller service account to the IAM role created in Part 1, Step 2
aws eks create-pod-identity-association --cluster-name ${CLUSTER_NAME}-eks \
  --namespace kube-system --service-account aws-load-balancer-controller \
  --role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${CLUSTER_NAME}-alb-controller-role --region $REGION

# Restart the controller so it picks up the new credentials
kubectl rollout restart deployment/aws-load-balancer-controller -n kube-system

kubectl rollout status deployment/aws-load-balancer-controller -n kube-system --timeout=120s
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
```

### Step 4: Create the e6-operator NodePool

```bash
cat << EOF | kubectl apply -f -
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: e6operator
spec:
  amiSelectorTerms:
    - alias: al2023@latest
  instanceProfile: ${CLUSTER_NAME}-karpenter
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: ${CLUSTER_NAME}-eks
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: ${CLUSTER_NAME}-eks
  tags:
    Name: ${CLUSTER_NAME}-e6operator
EOF

cat << EOF | kubectl apply -f -
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: e6operator
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: e6operator
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values: [t3.medium, t3.large, t4g.medium, t4g.large]
        - key: topology.kubernetes.io/zone
          operator: In
          values: ["${AZ1}", "${AZ2}"]
      taints:
        - key: e6operator
          value: "true"
          effect: NoSchedule
  limits:
    cpu: 100
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 30s
EOF

kubectl get nodepools.karpenter.sh e6operator
kubectl get ec2nodeclasses.karpenter.k8s.aws e6operator
```

### Step 5: Install cert-manager

```bash
export CERT_MANAGER_VERSION="v1.19.2"

helm upgrade --install cert-manager \
  oci://quay.io/jetstack/charts/cert-manager \
  --version "${CERT_MANAGER_VERSION}" --namespace cert-manager --create-namespace \
  --set crds.enabled=true --set webhook.timeoutSeconds=4 --timeout 10m --wait

kubectl rollout status deployment/cert-manager -n cert-manager --timeout=120s
kubectl rollout status deployment/cert-manager-webhook -n cert-manager --timeout=120s
kubectl rollout status deployment/cert-manager-cainjector -n cert-manager --timeout=120s
kubectl get pods -n cert-manager
```

### Step 6: Install the e6-operator CRDs

e6data provides the CRD files. Apply them with server-side apply:

```bash
export E6_OPERATOR_CRDS_PATH="./e6-operator-crds"

kubectl apply --server-side --force-conflicts -f "${E6_OPERATOR_CRDS_PATH}/"
kubectl wait --for=condition=Established crd/namespaceconfigs.e6data.io --timeout=120s
kubectl get crds | grep e6data.io
```

All CRDs from the provided archive are installed - expect 20+ resources with the `e6data.io` suffix, including:

```
e6catalogs.e6data.io
metadataservices.e6data.io
monitoringservices.e6data.io
namespaceconfigs.e6data.io
queryrouters.e6data.io
queryservices.e6data.io
```

### Step 7: Create the gp3 StorageClass

```bash
cat << 'EOF' | kubectl apply --server-side --force-conflicts -f -
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
  fsType: ext4
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
EOF

kubectl get storageclass gp3
```

### Step 8: Install the e6-operator controller

e6data provides the operator Helm chart. Install it with `helm template | kubectl apply`:

```bash
export E6_OPERATOR_IMAGE_REPO="<ECR_REPO_PROVIDED_BY_E6DATA>"
export E6_OPERATOR_IMAGE_TAG="<VERSION_PROVIDED_BY_E6DATA>"
export E6_OPERATOR_CHART="./e6-operator-chart"

kubectl create namespace e6operator --dry-run=client -o yaml | kubectl apply -f -

cat > operator-values.yaml << EOF
image:
  repository: ${E6_OPERATOR_IMAGE_REPO}
  tag: ${E6_OPERATOR_IMAGE_TAG}
  pullPolicy: IfNotPresent
replicaCount: 2
resources:
  requests: { cpu: 200m, memory: 256Mi }
  limits: { cpu: 1, memory: 1Gi }
logLevel: info
tolerations:
  - key: e6operator
    operator: Equal
    value: "true"
    effect: NoSchedule
EOF

helm template e6operator "${E6_OPERATOR_CHART}" \
  --namespace e6operator -f operator-values.yaml \
  | kubectl apply --server-side --force-conflicts -f -

kubectl rollout status deployment/e6operator -n e6operator --timeout=900s
kubectl get pods -n e6operator
```

## Next

Continue to [Deploy workspace and e6data](/query-engine/guides/deployment/aws-in-vpc/deploy-workspace-and-e6data.md).


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.e6data.com/query-engine/guides/deployment/aws-in-vpc/configure-registry-vpc-eks-networking.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.