ReferenceApplies to: Unified engine deployments
[committer]
Storage I/O
io_limits.max_batch_rows
Limits how many rows one connector read returns at once.
- Default:
65536
- Type: integer; positive
Example
65536 caps a connector batch at 65536 rows. A full scan can still read many batches.
io_limits.max_buffered_bytes
Limits how many bytes this component keeps in its data buffer.
- Default:
33554432
- Type: integer (bytes); positive
Example
33554432 gives the buffer 32 MiB. This is one component’s buffer budget, not the whole worker’s memory limit.
io_limits.max_concurrent_requests
Limits how many data-storage requests this component may make at once.
- Default:
4
- Type: integer; positive
Example
4 allows four simultaneous I/O requests. Raising it can improve throughput but increases pressure on storage and memory.
Settings
batch_size
Limits how many pending write-publication records the committer handles in one pass.
- Default:
100
- Type: integer; positive
Example
With 100, a large backlog of prepared writes is processed in groups. A task is not confirmed committed merely because a worker finished writing files.
Last modified on November 4, 2026
