Read S3 using IRSA

How-to guideApplies to: Unified engine deployments

Read an S3 prefix using a named runtime credential, without saving access keys.

Prepare the EKS identity

IRSA lets a Kubernetes service account use an AWS IAM role through a projected web identity token. Configure the EKS OpenID Connect (OIDC) provider, IAM role trust policy, and service-account annotation outside SQL. The credential-service Pod must receive AWS_ROLE_ARN and a readable AWS_WEB_IDENTITY_TOKEN_FILE for that association. Remote workers must be able to reach the credential service. Worker permissions for engine result/table storage are separate from this named input credential. Use an existing S3 bucket containing Parquet files for the TPC-DS SF1 item table. The examples use s3://recipe-data/tpcds-sf1/item/ in us-east-2. Replace the bucket, path, and Region with your real values. The identity needs s3:ListBucket on the bucket and s3:GetObject on that prefix. Add kms:Decrypt when its objects use a customer-managed KMS key. Keep the view and query in the same SQL session. Follow the AWS IRSA setup before running the SQL.

Create the credential and read data

CREATE CREDENTIAL recipe_irsa_reader USING aws
OPTIONS (
  'authentication' = 'irsa',
  'region' = 'us-east-2'
);

CREATE OR REPLACE TEMPORARY VIEW recipe_irsa_items
USING parquet
OPTIONS (
  path 's3://recipe-data/tpcds-sf1/item/',
  credential_reference 'recipe_irsa_reader',
  region 'us-east-2',
  endpoint 'https://s3.us-east-2.amazonaws.com',
  allow_http 'false'
);

SELECT i_item_sk, i_item_id
FROM recipe_irsa_items
ORDER BY i_item_sk
LIMIT 5;

The SELECT must return item rows. Creating the credential alone does not call S3 or prove that the Pod has a working identity. With a token file mounted at a custom location, replace the CREATE statement’s options with explicit role_arn and token_file values. Both must be supplied together. The file must be readable on the credential-service host. When finished:

DROP VIEW recipe_irsa_items;
DROP CREDENTIAL recipe_irsa_reader;

This leaves the AWS role and service-account association in place.

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Prepare the EKS identity

Create the credential and read data

​Prepare the EKS identity

​Create the credential and read data

Prepare the EKS identity

Create the credential and read data