> ## Documentation Index
> Fetch the complete documentation index at: https://docs.e6data.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Manage pipelines

> Create, inspect, stop, and resume Streaming pipelines from Workbench or a Spark Connect SQL client. The pipeline continues independently of the SQL session that created it.

How-to guideApplies to: Unified engine deployments

Create, inspect, stop, and resume Streaming pipelines from Workbench or a Spark Connect SQL client. The pipeline continues independently of the SQL session that created it.

## Create and start

`CREATE STREAMING PIPELINE` starts execution by default. The source and sink must already be registered. [Your first pipeline](https://docs.e6data.com/transformation/streaming/first-pipeline) provides the full source, sink, and pipeline script.

A successful SQL operation means the definition was accepted. Workers start asynchronously. Check the pipeline's state before checking output rows.

## List and inspect

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
SHOW PIPELINES;
DESCRIBE PIPELINE orders_to_iceberg;
```

Both commands return a `result` column with JSON metadata. `SHOW` includes pipelines created through the existing pipeline API as well as SQL.

| Field | Meaning |
| - | - |
| `pipeline_id` | Stable native pipeline ID |
| `name` | Name used by the SQL lifecycle commands |
| `job_id` | The pipeline's native job ID |
| `state` | Current execution state, such as `Created`, `Running`, or `Stopped` |
| `stop_mode` | Requested stop behavior; `none` permits running and `checkpoint` requests a checkpoint stop |
| `query` | Saved `INSERT INTO ... SELECT ...` transformation |
| `checkpoint_interval_micros` | Checkpoint interval in microseconds |

The SQL operation can succeed while the pipeline is still starting. Its state and requested stop mode describe different things. For example, a stop request can return `stop_mode: "checkpoint"` while the state is still `Running`.

Use the shared [Observability](https://docs.e6data.com/transformation/operations/observability.md) docs for metrics and service visibility.

## Save without starting

This example reuses the walkthrough's registered connections and creates a separate, paused pipeline.

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
CREATE STREAMING PIPELINE reviewed_orders
WITH (
  parallelism = 2,
  checkpoint_interval = '10 seconds',
  start_paused = true
)
AS INSERT INTO paid_orders_sink
SELECT
  order_id,
  customer_id,
  amount,
  amount * 1.10 AS amount_with_tax
FROM orders_source
WHERE status = 'paid' AND amount > 0;
```

The initial state is `Stopped`, with `stop_mode` set to `checkpoint`. Review its definition before starting it.

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
DESCRIBE PIPELINE reviewed_orders;
START PIPELINE reviewed_orders;
```

This is a second pipeline with its own source progress. Stop the walkthrough pipeline first if both point to the same destination and you do not want duplicate input processing.

## Stop and resume

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
STOP PIPELINE orders_to_iceberg;
DESCRIBE PIPELINE orders_to_iceberg;
```

`STOP` requests a checkpoint stop through the existing Streaming controller. It returns before that stop finishes. Repeat `DESCRIBE` until the state is `Stopped`.

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
START PIPELINE orders_to_iceberg;
DESCRIBE PIPELINE orders_to_iceberg;
```

`START` clears the stop request. The same pipeline resumes through its existing Streaming path and saved checkpoints. It does not create a new pipeline. It also does not prove that the workers have reached `Running` yet.

`START` does not resume jobs whose state is `Failed` or `Finished`. Use the existing native pipeline controls for those jobs. Closing the browser tab or SQL session does not stop any pipeline.

## Retry a request

When a response is lost, retry the original operation with the same session and operation ID. Laminar saves a receipt with the native mutation. A matching retry returns the same resource. Different content with the same identity is rejected.

An old `STOP` retry cannot apply the stop again after a later `START` has been accepted. It returns the same pipeline's current metadata instead.

Running the SQL again as a fresh operation is a new request. Creating an existing name this way reports a name conflict. Creating a new pipeline name creates a separate job, with its own source progress.

## Change or remove a definition

The current SQL commands cover connector registration, pipeline creation, listing, inspection, stop, and start. They do not include `ALTER STREAMING PIPELINE`, `DROP PIPELINE`, or SQL connector deletion.

Use the existing pipeline and table controls for changes and removal. Stop a pipeline before deleting it. Remove connector definitions after their pipeline users are removed. Deleting a definition does not remove the Kafka topic or the Iceberg destination data.

## Clean up the paused example

If you started `reviewed_orders`, stop it and wait for `Stopped` before removing it through the existing controls.

```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
STOP PIPELINE reviewed_orders;
DESCRIBE PIPELINE reviewed_orders;
```

## Continue

[Streaming SQL reference](https://docs.e6data.com/transformation/streaming/sql-reference) lists the command forms, defaults, and validation limits.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.