Before you start

How-to guideApplies to: Unified engine deployments

The examples define a pipeline with SQL. Kafka, the Iceberg catalog, and warehouse storage must already exist.

Required services and access

Requirement	Check before running
Laminar with Streaming SQL enabled	Workbench or your Spark Connect client can reach an instance with the Streaming control service
Streaming workers	Workers can start and reach their configured checkpoint storage
Kafka	Workers can reach the advertised broker addresses and read the example topic
Iceberg catalog	The REST endpoint and selected warehouse are available to the pipeline
Iceberg namespace	`analytics` exists, and the pipeline identity can create and write its example table
Warehouse storage	The pipeline has the object storage access needed to write and publish files

A Unified-only instance reports that the native Streaming control service is not configured. Ask the instance operator to enable Streaming on that deployment.

Check SQL access

SHOW PIPELINES;

The result has a result column containing JSON metadata for each saved pipeline. An empty result is valid when no pipelines exist. This command checks the SQL control path. It does not test Kafka, catalog, or storage access.

Replace the example values

Example value	Replace with
`kafka:9092`	Broker addresses reachable from the workers
`orders`	A test Kafka topic containing JSON order messages
`http://lakekeeper:8181/catalog`	Your Iceberg REST catalog endpoint
`warehouse-id`	The warehouse identifier accepted by that catalog
`analytics`	An existing namespace for the test output
`paid_orders`	An unused Iceberg table name for this exercise

The examples use an unauthenticated Kafka broker and REST catalog. For secured services, add the matching options from Sources and Sinks. Catalog access and object storage access are separate permissions.

Choose names once

The walkthrough registers orders_source, paid_orders_sink, and orders_to_iceberg. Choose unused names in the Streaming registry. If a name is already in use, change it throughout the examples before running them. The pages on sources, sinks, and transformations are alternatives and reference examples. You do not need to run every declaration on every page. A new create request for an existing name reports a conflict.

Use the example workspace

Copy a SQL block into Workbench or your Spark Connect SQL client. Edit connection values, then run the statements in order. Multiple statements run as separate operations. They do not form one transaction. Run STOP PIPELINE when you finish the exercise. A create response confirms acceptance; check the pipeline state before expecting output rows.

Continue

Your first pipeline walks through the complete flow. Use the shared Observability docs for operational visibility.

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Required services and access

Check SQL access

Replace the example values

Choose names once

Use the example workspace

Continue

​Required services and access

​Check SQL access

​Replace the example values

​Choose names once

​Use the example workspace

​Continue

Required services and access

Check SQL access

Replace the example values

Choose names once

Use the example workspace

Continue