> ## Documentation Index
> Fetch the complete documentation index at: https://docs.e6data.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Kafka connections

> kafka_connections

ReferenceApplies to: Unified engine deployments

`[kafka_connections]`

## Settings

### `kafka_connections`

Defines named Kafka connections that source definitions can reuse.

* **Default:** `{}`
* **Type:** map

> **Example**
>
> A connection named `events` contains the broker addresses and limits for your events cluster. Adding the connection does not start a workload.

### `<name>.bootstrap_servers`

Lists the Kafka brokers a client first contacts to discover the cluster.

* **Default:** `""`
* **Type:** string

> **Example**
>
> `"broker-a:9092,broker-b:9092"` gives the client two starting brokers. The client can then discover other brokers in the same cluster.

### `<name>.properties`

Supplies non-secret connection settings using librdkafka property names.

* **Default:** `{}`
* **Type:** map

> **Example**
>
> A property such as `security.protocol = "SASL_SSL"` selects the broker connection mode. Put actual secret values behind secret references instead.

### `<name>.secret_properties`

Maps Kafka client property names to references that resolve secret values.

* **Default:** `{}`
* **Type:** map

> **Example**
>
> Map a password property to a saved secret reference so the client resolves its value when opening the connection.

### `<name>.timeout_ms`

Limits how long one external command or Kafka network operation may take, in milliseconds.

* **Default:** `30000`
* **Type:** integer (milliseconds); positive

> **Example**
>
> `30000` allows up to 30 seconds for that operation. It is not the deadline for an entire workload.

### `<name>.max_partitions`

Limits how many partitions one Kafka source may contain.

* **Default:** `1024`
* **Type:** integer; positive

> **Example**
>
> `1024` admits up to 1024 partitions. A source spanning more partitions needs its limit reviewed before execution.

### `<name>.max_offsets_per_batch`

Limits how many Kafka offsets are admitted into one source batch.

* **Default:** `100000`
* **Type:** integer; positive

> **Example**
>
> `100000` bounds the offset range admitted per batch. Later offsets remain work for later batches.

### `<name>.max_buffered_bytes`

Limits how many bytes this component keeps in its data buffer.

* **Default:** `67108864`
* **Type:** integer (bytes); positive

> **Example**
>
> `33554432` gives the buffer 32 MiB. This is one component’s buffer budget, not the whole worker’s memory limit.

### `<name>.legacy_generation`

Supplies an operator-controlled topic identity for older Kafka brokers without topic IDs.

* **Default:** Unset
* **Type:** string; optional

> **Example**
>
> Use a fixed generation only when topic replacement is forbidden. Do not reuse it to make a newly replaced topic look like the old topic.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.