AWS PrivateLink and e6data

This topic describes configuring AWS PrivateLink to securely connect the e6data to your AWS account.

What is AWS PrivateLink?

AWS PrivateLink is an AWS service that facilitates the creation of private VPC endpoints, enabling direct and secure connectivity between your AWS VPCs and the e6data VPC without requiring traversal through the public Internet. The connectivity is for AWS VPCs in the same AWS region.

Why do we need AWS Privatelink?

Leveraging AWS Endpoint services and Endpoints, this integration ensures the utmost confidentiality and integrity of your data.

What are the prerequisites for configuring AWS PrivateLink?

Create an IAM OIDC Identity Provider

Creating an OIDC (OpenID Connect) provider for your EKS cluster is crucial in this context because it directly relates to providing secure access for e6data clusters to interact with data buckets within your AWS account. e6data uses OIDC for more secure access as it provides the least privilege & credential isolation. To create an OIDC (OpenID Connect) provider for your EKS cluster, please refer to the documentation Creating an IAM OIDC provider for your cluster - Amazon EKS.

How to establish a Private Link Between the Customer Account and the e6data Client Account?

Step 1: Update Security Group Settings

Update the security group associated with your EKS cluster. Allow inbound traffic on port 443 from e6data's Virtual Private Cloud (VPC) CIDR range. Please reach out to the e6data administrator to obtain the specific CIDR range information.

Step 2: Setting Up Target Group

Establish a target group for the Network LoadBalancer. The target group should correspond to the private IP addresses of the Elastic Network Interfaces (ENIs) allocated during the creation of your EKS cluster.

To create a Target Group, please refer to the following documentation: Creating Target Group.

Parameters to specify in the first step of creating a target group (Specify group details)

Parameters to specify in the second step of creating a target group (Register targets)

Steps to get the private IP's of the NIC's of the EKS cluster

The network interfaces (NICs) with the description "AMAZON EKS <EKS_CLUSTER_NAME>" will be the two NICs associated with your EKS cluster.

You can get the private IPs using the aws cli command:

aws ec2 describe-network-interfaces --filters "Name=description,Values=Amazon EKS <EKS_CLUSTER_NAME>" --query 'NetworkInterfaces[].{ID:NetworkInterfaceId,PrivateIP:PrivateIpAddress,Status:Status}' --output text

Step 3: Create an Internal Network Load Balancer (NLB)

• Create an internal Network Load Balancer (NLB) within your Virtual Private Cloud (VPC). This NLB will manage network traffic within the VPC, ensuring it remains isolated from public internet access.

• To create an internal network load balancer, please refer to the following documentation: Creating an Internal NLB.

To configure your load balancer and listener

List of parameters to specify

Step 4: Create an Endpoint Service for NLB

To enhance data security and maintain compliance, set up endpoint services for the NLB. This will enable secure access to your data processing service from other AWS accounts, all while avoiding exposure to the public internet. To create an endpoint service, please refer to the following documentation: Creating an Endpoint Service

• Once the endpoint service is created, You need to add “arn:aws:iam::<e6data_account_id>:root” in the allowed principals.

Step 5: Create Amazon S3 VPC Endpoint:

To enhance security AWS provides the option to access S3 buckets privately within your Amazon Virtual Private Cloud (VPC) using VPC endpoints.

You can access Amazon S3 from your VPC using gateway VPC endpoints. After you create the gateway endpoint, you can add it as a target in your route table for traffic destined from your VPC to Amazon S3.

To create S3 VPC Endpoint, please refer to the following documentation: Creating S3 VPC Endpoint.

Step 6: Update the e6data S3 Bucket Policy:

To update bucket policies to control access to buckets from specific endpoints.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::<e6data_s3_bucket_name>/*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceVpce": "<VPC_endpoint_ID>"
                }
            }
        }
    ]
}

Step 7: Validating the Endpoint Service and Connection:

After setting up the Private Link:

• Please log in to your AWS account and proceed to the "Endpoint Services" section.

• This endpoint service has been configured to permit access to the e6data AWS account ID as an allowed principal.

• A connection will be successfully created with the e6data control plane to facilitate secure communication