Read S3 using saved AWS keys

How-to guideApplies to: Unified engine deployments

Create a named credential from secret references, then bind that credential to a Parquet view.

Have keys and a data prefix ready

Use an existing S3 bucket containing Parquet files for the TPC-DS SF1 item table. The examples use s3://recipe-data/tpcds-sf1/item/ in us-east-2. Replace the bucket, path, and Region with your real values. The identity needs s3:ListBucket on the bucket and s3:GetObject on that prefix. Add kms:Decrypt when its objects use a customer-managed KMS key. Keep the view and query in the same SQL session. Obtain the key pair from AWS outside SQL. This recipe does not create IAM users or access keys. For temporary credentials, also obtain their session token and keep all three values from the same session. Local secrets need the encrypted secret store.

Save the key references

CREATE SECRET recipe_aws_access_key AS '<access-key-id>';
CREATE SECRET recipe_aws_secret_key AS '<secret-access-key>';

CREATE CREDENTIAL recipe_saved_reader USING aws
OPTIONS ('authentication' = 'saved')
SECRETS (
  'access_key_id' = 'recipe_aws_access_key',
  'secret_access_key' = 'recipe_aws_secret_key'
);

For temporary keys, add their session token before running the data query:

CREATE SECRET recipe_aws_session_token AS '<session-token>';
ALTER CREDENTIAL recipe_saved_reader
SET SECRETS ('session_token' = 'recipe_aws_session_token');

Omit this optional block for long-lived keys. A saved session token is not automatically renewed. Replace all session values together when they expire.

Read using the named credential

CREATE OR REPLACE TEMPORARY VIEW recipe_saved_items
USING parquet
OPTIONS (
  path 's3://recipe-data/tpcds-sf1/item/',
  credential_reference 'recipe_saved_reader',
  region 'us-east-2',
  endpoint 'https://s3.us-east-2.amazonaws.com',
  allow_http 'false'
);

SELECT i_item_sk, i_item_id
FROM recipe_saved_items
ORDER BY i_item_sk
LIMIT 5;

Expect up to five item rows. The credential selects the keys; the view’s Region selects the S3 endpoint. Saved AWS credentials do not accept a Region option. When finished:

DROP VIEW recipe_saved_items;
DROP CREDENTIAL recipe_saved_reader;
DROP SECRET IF EXISTS recipe_aws_session_token;
DROP SECRET recipe_aws_secret_key;
DROP SECRET recipe_aws_access_key;

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Read S3 using saved AWS keys

Have keys and a data prefix ready

Save the key references

Read using the named credential

​Have keys and a data prefix ready

​Save the key references

​Read using the named credential

Have keys and a data prefix ready

Save the key references

Read using the named credential