Ingest Engine is a distributed streaming ingestion engine that runs natively on Kubernetes.
It reads from Kafka, databases (through CDC), HTTP endpoints and cloud storage, transforms the data with SQL, and writes it to Apache Iceberg, Delta Lake and Kafka topics. Processing is real time, with exactly-once delivery. Add it when you want live data in your lakehouse. Query Engine does not need it.Build your first pipeline
Choose your pipeline
What do you want to do?
How it works
Architecture
Controller, workers and pipeline replica sets.
Checkpointing
How exactly-once delivery survives failures.
Watermarks
How event time drives windows and joins.