Cross Account Access

How-to guideApplies to: AWS

Configure cross-account access separately for the MSK source, Glue Schema Registry and the Iceberg sink’s S3 warehouse. Each section follows the same sequence: create a role, attach resource permissions, configure trust, authorize role assumption, and configure the connector. Replace every angle-bracket placeholder before saving a policy. Ask your e6data contact for the exact ingestion workload role ARN that your roles must trust. If your deployment uses more than one workload role, include each supplied ARN in the trust policy. The Source role ARN and Registry role ARN fields take IAM role ARNs. MSK cluster, topic, consumer-group and Glue resource ARNs belong in the permissions policies. The S3 sink takes its writer role ARN in client.assume-role.arn.

Your DevOps team configures the roles and resource permissions in your AWS account. Your e6data contact configures permission for the ingestion workload to assume those roles. Both sides must be in place before the connector can use a role.

MSK

Create an IAM role in the account that owns your MSK cluster. Use it as the Kafka source role.

1. Attach resource permissions

Attach this identity policy to the source role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ConnectToSourceCluster",
      "Effect": "Allow",
      "Action": ["kafka-cluster:Connect"],
      "Resource": ["<msk-cluster-arn>"]
    },
    {
      "Sid": "ReadSourceTopics",
      "Effect": "Allow",
      "Action": [
        "kafka-cluster:DescribeTopic",
        "kafka-cluster:ReadData"
      ],
      "Resource": ["<msk-topic-arn>"]
    },
    {
      "Sid": "UseSourceConsumerGroups",
      "Effect": "Allow",
      "Action": [
        "kafka-cluster:DescribeGroup",
        "kafka-cluster:AlterGroup"
      ],
      "Resource": ["<msk-consumer-group-arn>"]
    }
  ]
}

Use the exact cluster ARN. Build the topic and consumer-group ARNs from the same cluster name and UUID:

arn:aws:kafka:<region>:<account-id>:cluster/<cluster-name>/<cluster-uuid>
arn:aws:kafka:<region>:<account-id>:topic/<cluster-name>/<cluster-uuid>/<topic-name>
arn:aws:kafka:<region>:<account-id>:group/<cluster-name>/<cluster-uuid>/<group-id>

Add an ARN for each topic or group the pipeline uses. Set config.type.source_config.group_id to the group ID you authorize. These permissions allow consumption without granting topic creation or message production.

2. Configure role trust

Set this trust policy on the source role, using the ingestion workload role ARN supplied by e6data:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowIngestionEngineToAssumeRole",
      "Effect": "Allow",
      "Principal": {
        "AWS": ["<e6-ingestion-workload-role-arn>"]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

3. Authorize role assumption

Share the source role ARN with your e6data contact. The ingestion workload needs sts:AssumeRole permission on that exact role. e6data applies a policy like this to the ingestion workload identity. It includes the separate source and registry roles used in this guide; only the roles your deployment uses are included.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AssumeCustomerIngestionRoles",
      "Effect": "Allow",
      "Action": ["sts:AssumeRole"],
      "Resource": [
        "<source-iam-role-arn>",
        "<registry-iam-role-arn>"
      ]
    }
  ]
}

4. Configure the connector

In the console, select AWS MSK IAM authentication and enter the source role in Source role ARN. Set the region to the MSK cluster’s region. For API or YAML configuration, use these settings inside the Kafka source’s configuration object. Set group_id to a consumer group authorized by the policy in step 1.

{
  "bootstrap_servers": "<msk-iam-bootstrap-brokers>",
  "authentication": {
    "type": "aws_msk_iam",
    "aws_msk_iam_config": {
      "region": "<msk-region>",
      "role_arn": "<source-iam-role-arn>"
    }
  },
  "topic": "<topic-name>",
  "type": {
    "source_config": {
      "offset": "earliest",
      "group_id": "<group-id>"
    }
  }
}

Network access to the brokers must be set up separately from IAM permissions.

Glue Schema Registry

Create an IAM role in the account that owns your Glue registry. Use it as the registry role. To reuse the MSK source role instead, see Use one role for MSK and Glue.

1. Attach resource permissions

Attach this identity policy to the registry role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CheckSchemaRegistry",
      "Effect": "Allow",
      "Action": ["glue:GetRegistry"],
      "Resource": ["<glue-registry-arn>"]
    },
    {
      "Sid": "ReadSchemaVersions",
      "Effect": "Allow",
      "Action": ["glue:GetSchemaVersion"],
      "Resource": [
        "<glue-registry-arn>",
        "<glue-schema-arn>"
      ]
    }
  ]
}

Copy the registry and schema ARNs from AWS Glue, and include every schema the source topic uses. Update the policy when you add a schema outside the authorized resources. GetRegistry is used for connection testing; GetSchemaVersion reads the registered schemas. These permissions don’t create or register schemas.

2. Configure role trust

Set this trust policy on the registry role, using the same ingestion workload role ARN supplied by e6data:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowIngestionEngineToAssumeRole",
      "Effect": "Allow",
      "Principal": {
        "AWS": ["<e6-ingestion-workload-role-arn>"]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

3. Authorize role assumption

Share the registry role ARN with your e6data contact. The ingestion workload needs sts:AssumeRole permission on that exact role, as shown in the MSK section.

4. Configure the connector

In the console, select Glue as the schema registry provider. Enter Glue region, Registry name and Registry role ARN. This Kafka source configuration shows MSK and Glue with separate roles. Replace the broker, topic, group, region, role and registry values for your deployment.

{
  "bootstrap_servers": "<msk-iam-bootstrap-brokers>",
  "authentication": {
    "type": "aws_msk_iam",
    "aws_msk_iam_config": {
      "region": "<msk-region>",
      "role_arn": "<source-iam-role-arn>"
    }
  },
  "topic": "<topic-name>",
  "type": {
    "source_config": {
      "offset": "earliest",
      "group_id": "<group-id>"
    }
  },
  "multi_schema": true,
  "schema_registry_provider": "glue",
  "schema_registry": {
    "region": "<glue-region>",
    "registry_name": "<registry-name>",
    "role_arn": "<registry-iam-role-arn>"
  }
}

Omit the source table’s schema for multi-schema ingestion. The source and registry roles may be in different accounts. The region in each block must match its service.

Use one role for MSK and Glue

Attach both the MSK and Glue permissions policies above to the source role. Keep its ARN in the MSK authentication settings, remove schema_registry.role_arn, and set:

{
  "schema_registry_provider": "glue",
  "schema_registry": {
    "region": "<glue-region>",
    "registry_name": "<registry-name>",
    "use_source_role": true
  }
}

In the console, select Use source role for registry. A separate registry role ARN and source-role reuse can’t be enabled together. Reuse requires AWS MSK IAM authentication and a configured source role ARN. If role assumption fails, the connection fails rather than switching identities. With no role ARN or reuse setting, the connector uses the deployment’s default AWS credentials. These source and registry permissions don’t grant access to the Iceberg catalog or object storage; configure those separately for the sink. Broker network access must also be set up separately from IAM permissions.

S3

Create an IAM role in the account that owns your Iceberg warehouse bucket. Use it as a dedicated S3 writer role for the sink.

1. Attach resource permissions

Attach this identity policy to the writer role. Replace <bucket-name> and <warehouse-prefix> with your bucket and warehouse path. Use the prefix without leading or trailing slashes. The object permissions cover every destination table under that warehouse prefix.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadBucketLocation",
      "Effect": "Allow",
      "Action": ["s3:GetBucketLocation"],
      "Resource": ["arn:aws:s3:::<bucket-name>"]
    },
    {
      "Sid": "ListWarehouse",
      "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": ["arn:aws:s3:::<bucket-name>"],
      "Condition": {
        "StringLike": {
          "s3:prefix": ["<warehouse-prefix>", "<warehouse-prefix>/*"]
        }
      }
    },
    {
      "Sid": "ReadAndWriteWarehouseObjects",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": ["arn:aws:s3:::<bucket-name>/<warehouse-prefix>/*"]
    }
  ]
}

2. Configure role trust

Set this trust policy on the writer role, using the ingestion workload role ARN supplied by e6data. Ask your e6data contact for the workspace-specific external ID, and use the same value in both the trust policy and the sink settings.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "<e6-ingestion-workload-role-arn>"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<external-id-agreed-with-e6data>"
        }
      }
    }
  ]
}

3. Authorize role assumption

Share the writer role ARN with your e6data contact. The ingestion workload needs sts:AssumeRole permission on that exact role, and the worker needs its workload identity to call STS.

4. Configure the connector

Set these values under the Iceberg sink’s config.storageOptions. Use the bucket’s region and the external ID agreed with e6data. The parameters are described in the Iceberg connector’s storage options.

storageOptions:
  s3.region: us-east-1
  client.assume-role.arn: arn:aws:iam::<customer-account-id>:role/<iceberg-writer-role>
  client.assume-role.external-id: <external-id-agreed-with-e6data>
  client.assume-role.session-name: laminar-iceberg

The sink uses this writer role for S3 access. It doesn’t reuse the Kafka source role automatically. The external ID settings shown here apply to the S3 sink; the MSK and Glue configurations above don’t have an external ID setting. When the writer role and bucket are in the same customer account, the role’s identity policy can grant S3 access without a separate cross-account bucket allow, subject to existing bucket policies and other restrictions. If the bucket uses a customer-managed KMS key, also grant the writer role the key permissions needed to read and write encrypted objects. Configure Iceberg catalog permissions separately.

Ingest Engine

Get started

Connectors

Streaming SQL

Operate

Reference

Concepts

MSK

1. Attach resource permissions

2. Configure role trust

3. Authorize role assumption

4. Configure the connector

Glue Schema Registry

1. Attach resource permissions

2. Configure role trust

3. Authorize role assumption

4. Configure the connector

Use one role for MSK and Glue

S3

1. Attach resource permissions

2. Configure role trust

3. Authorize role assumption

4. Configure the connector

​MSK

​1. Attach resource permissions

​2. Configure role trust

​3. Authorize role assumption

​4. Configure the connector

​Glue Schema Registry

​1. Attach resource permissions

​2. Configure role trust

​3. Authorize role assumption

​4. Configure the connector

​Use one role for MSK and Glue

​S3

​1. Attach resource permissions

​2. Configure role trust

​3. Authorize role assumption

​4. Configure the connector

MSK

1. Attach resource permissions

2. Configure role trust

3. Authorize role assumption

4. Configure the connector

Glue Schema Registry

1. Attach resource permissions

2. Configure role trust

3. Authorize role assumption

4. Configure the connector

Use one role for MSK and Glue

S3

1. Attach resource permissions

2. Configure role trust

3. Authorize role assumption

4. Configure the connector