Switch from saved keys to runtime identity

How-to guideApplies to: Unified engine deployments

Change how a credential obtains its AWS identity while keeping the name already used by a file view.

Prepare both methods

Use an existing S3 bucket containing Parquet files for the TPC-DS SF1 item table. The examples use s3://recipe-data/tpcds-sf1/item/ in us-east-2. Replace the bucket, path, and Region with your real values. The identity needs s3:ListBucket on the bucket and s3:GetObject on that prefix. Add kms:Decrypt when its objects use a customer-managed KMS key. Keep the view and query in the same SQL session. Have valid saved AWS keys and an EKS Pod Identity with equivalent read access. Complete the Pod Identity setup on the credential-service host first. Use unused recipe names and keep the local encrypted secret store available.

Start with saved keys

CREATE SECRET recipe_switch_access_key AS '<access-key-id>';
CREATE SECRET recipe_switch_secret_key AS '<secret-access-key>';

CREATE CREDENTIAL recipe_switch_reader USING aws
OPTIONS ('authentication' = 'saved')
SECRETS (
  'access_key_id' = 'recipe_switch_access_key',
  'secret_access_key' = 'recipe_switch_secret_key'
);

CREATE OR REPLACE TEMPORARY VIEW recipe_switch_items
USING parquet
OPTIONS (
  path 's3://recipe-data/tpcds-sf1/item/',
  credential_reference 'recipe_switch_reader',
  region 'us-east-2',
  endpoint 'https://s3.us-east-2.amazonaws.com',
  allow_http 'false'
);

SELECT i_item_sk, i_item_id
FROM recipe_switch_items
ORDER BY i_item_sk
LIMIT 5;

This starting example uses long-lived keys. If using temporary keys, also add their session-token secret as shown in Saved AWS keys.

Change the definition in one statement

ALTER CREDENTIAL recipe_switch_reader
SET OPTIONS ('authentication' = 'pod_identity', 'region' = 'us-east-2')
UNSET SECRETS ('access_key_id', 'secret_access_key');

DESCRIBE CREDENTIAL recipe_switch_reader;
SELECT i_item_sk, i_item_id FROM recipe_switch_items ORDER BY i_item_sk LIMIT 5;

Remove the old method’s secret-reference fields in the same ALTER. Pod Identity does not accept saved-key fields. If the starting definition has session_token or the saved expires_at_unix_millis option, remove those in that ALTER too. The view still names recipe_switch_reader and does not need to be recreated. A fresh SELECT checks the new access path. Already running work can retain credentials it acquired before the change. After the new method works, remove the unused local key values:

DROP SECRET recipe_switch_secret_key;
DROP SECRET recipe_switch_access_key;

These drops do not revoke the AWS keys. Revoke them through AWS when no other consumer needs them. To finish the exercise:

DROP VIEW recipe_switch_items;
DROP CREDENTIAL recipe_switch_reader;

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Switch from saved keys to runtime identity

Prepare both methods

Start with saved keys

Change the definition in one statement

​Prepare both methods

​Start with saved keys

​Change the definition in one statement

Prepare both methods

Start with saved keys

Change the definition in one statement