Use AWS Secrets Manager

How-to guideApplies to: Unified engine deployments

Use a local secret name for a client secret held in AWS. The local resource contains a reference; the value stays managed by the external provider.

Prepare the external source

Have an administrator create Secrets Manager secret /laminar/recipes/polaris-client-secret containing the raw Polaris client secret as its SecretString. A JSON wrapper is not selected automatically. The engine’s secret-service identity needs secretsmanager:GetSecretValue and, for a customer-managed key, kms:Decrypt. Configure engine provider alias recipe_sm with type aws-secrets-manager. Set its optional region and profile. Supply the AWS SDK identity through the secret-service process environment or its workload identity. For Pod Identity, the SDK uses the Pod’s container credential settings. For IRSA, it uses the role ARN and mounted web identity token file. Grant this identity access to the secret. Load this provider configuration before running the SQL. See Secrets configuration. Use an existing Polaris catalog with a retail warehouse and the SF1 item table in namespace tpcds_sf1. Replace the example URLs, client ID, warehouse, namespace, and Region with your values. The Polaris client needs access to the warehouse and table. These examples request PRINCIPAL_ROLE:ALL; use the scope granted to your client. For the final SELECT, Polaris must supply usable temporary S3 credentials for the table. Otherwise attach a separate storage credential as shown in Separate catalog and storage credentials. A successful login or table listing alone does not prove S3 access. AWS documents the required permissions in GetSecretValue.

Create the reference and query data

CREATE SECRET recipe_remote_client_secret
USING 'recipe_sm:///laminar/recipes/polaris-client-secret';

DESCRIBE SECRET recipe_remote_client_secret;

CREATE CREDENTIAL recipe_remote_oauth USING oauth2
OPTIONS (
  'authentication' = 'client_credentials',
  'client_id' = '<client-id>',
  'token_endpoint' = 'https://polaris.example.com/api/catalog/v1/oauth/tokens'
)
SECRETS ('client_secret' = 'recipe_remote_client_secret');

CREATE CATALOG recipe_remote_retail USING `iceberg-rest`
OPTIONS (
  'catalog.type' = 'polaris',
  'uri' = 'https://polaris.example.com/api/catalog',
  'warehouse' = 'retail',
  'auth.type' = 'oauth2',
  'oauth2-server-uri' = 'https://polaris.example.com/api/catalog/v1/oauth/tokens',
  'scope' = 'PRINCIPAL_ROLE:ALL',
  's3.region' = 'us-east-2',
  's3.endpoint' = 'https://s3.us-east-2.amazonaws.com',
  's3.allow-http' = 'false'
)
CREDENTIALS (
  'auth' = 'recipe_remote_oauth'
);

SELECT i_item_sk, i_item_id
FROM recipe_remote_retail.tpcds_sf1.item
ORDER BY i_item_sk
LIMIT 5;

The reference uses the configured alias, not an automatically registered AWS provider name. A successful SELECT confirms the reference can resolve, OAuth can obtain a token, and the table files can be read.

Remove the local resources

DROP CATALOG recipe_remote_retail;
DROP CREDENTIAL recipe_remote_oauth;
DROP SECRET recipe_remote_client_secret;

This does not delete the AWS secret or parameter.

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Use AWS Secrets Manager

Prepare the external source

Create the reference and query data

Remove the local resources

​Prepare the external source

​Create the reference and query data

​Remove the local resources

Prepare the external source

Create the reference and query data

Remove the local resources