Read S3 using EKS Pod Identity

How-to guideApplies to: Unified engine deployments

Read an S3 prefix using a named runtime credential, without saving access keys.

Prepare the EKS identity

Install the EKS Pod Identity Agent and associate the credential-service Pod’s Kubernetes service account with an IAM role that can read the input prefix. The Pod must receive the container credential endpoint and authorization-token file used by EKS. Creating the SQL credential does not create this association. The credential service resolves this named identity; remote workers must be able to reach that service. Worker access to engine result/table storage is separate and still needs its own runtime permissions. Use an existing S3 bucket containing Parquet files for the TPC-DS SF1 item table. The examples use s3://recipe-data/tpcds-sf1/item/ in us-east-2. Replace the bucket, path, and Region with your real values. The identity needs s3:ListBucket on the bucket and s3:GetObject on that prefix. Add kms:Decrypt when its objects use a customer-managed KMS key. Keep the view and query in the same SQL session. Follow the AWS Pod Identity association setup before running the SQL.

Create the credential and read data

CREATE CREDENTIAL recipe_pod_reader USING aws
OPTIONS (
  'authentication' = 'pod_identity',
  'region' = 'us-east-2'
);

CREATE OR REPLACE TEMPORARY VIEW recipe_pod_items
USING parquet
OPTIONS (
  path 's3://recipe-data/tpcds-sf1/item/',
  credential_reference 'recipe_pod_reader',
  region 'us-east-2',
  endpoint 'https://s3.us-east-2.amazonaws.com',
  allow_http 'false'
);

SELECT i_item_sk, i_item_id
FROM recipe_pod_items
ORDER BY i_item_sk
LIMIT 5;

The SELECT must return item rows. Creating the credential alone does not call S3 or prove that the Pod has a working identity. When finished:

DROP VIEW recipe_pod_items;
DROP CREDENTIAL recipe_pod_reader;

This leaves the AWS role and service-account association in place.

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Read S3 using EKS Pod Identity

Prepare the EKS identity

Create the credential and read data

​Prepare the EKS identity

​Create the credential and read data

Prepare the EKS identity

Create the credential and read data