Read S3 Parquet using temporary AWS keys

How-to guideApplies to: Unified engine deployments

Use an explicit saved AWS key set for one Parquet file source.

Before you start

Use an existing nonempty Parquet dataset and a configured S3 file-source root covering its prefix. A credential does not authorize an arbitrary storage root. The identity needs list/read access to the bucket and actual files, plus KMS decrypt permission when needed. Use the region and HTTPS endpoint for that bucket. Keep the temporary view and its SELECT in the same Workbench workspace. Obtain all three values from the same unexpired AWS session. The engine does not renew saved temporary keys. Local secret values require the encrypted secret store.

Values to collect

Input bucket/prefix, endpoint, region, and matching key values.

Run the SQL

Replace every <placeholder> before running. Use unused names for the resources below. Run the statements in order in Workbench.

CREATE SECRET recipe_temporary_aws_keys_key AS '<access-key-id>';
CREATE SECRET recipe_temporary_aws_keys_secret AS '<secret-access-key>';
CREATE SECRET recipe_temporary_aws_keys_session AS '<session-token>';

CREATE CREDENTIAL recipe_temporary_aws_keys_reader USING aws
OPTIONS (
  'authentication' = 'saved'
)
SECRETS ('access_key_id' = 'recipe_temporary_aws_keys_key', 'secret_access_key' = 'recipe_temporary_aws_keys_secret', 'session_token' = 'recipe_temporary_aws_keys_session');

CREATE OR REPLACE TEMPORARY VIEW recipe_temporary_aws_keys
USING parquet
OPTIONS (
  path 's3://<bucket>/<parquet-prefix>/',
  credential_reference 'recipe_temporary_aws_keys_reader',
  region '<aws-region>',
  endpoint '<s3-endpoint>',
  allow_http 'false'
);

SELECT * FROM recipe_temporary_aws_keys LIMIT 5;

Check the result

The SELECT should return rows from the chosen dataset. For temporary AWS keys, it also verifies the matching session token.

Clean up

Remove only the local view, credential, and secrets.

DROP VIEW recipe_temporary_aws_keys;
DROP CREDENTIAL recipe_temporary_aws_keys_reader;
DROP SECRET recipe_temporary_aws_keys_key;
DROP SECRET recipe_temporary_aws_keys_secret;
DROP SECRET recipe_temporary_aws_keys_session;

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Read S3 Parquet using temporary AWS keys

Before you start

Values to collect

Run the SQL

Check the result

Clean up

​Before you start

​Values to collect

​Run the SQL

​Check the result

​Clean up

Before you start

Values to collect

Run the SQL

Check the result

Clean up