Skip to main content

Prerequisites

  • Ingest Engine running locally or in your cluster
  • Basic SQL knowledge

Step 1: Create a Source

Define where your data comes from. This example uses a mock source for testing:

Step 2: Create a Destination

Define where results go. For testing, use the preview sink:

Step 3: Write Your Query

Connect source to destination with a transformation:

Step 4: Run the Pipeline

Via the UI: Click “Create Pipeline” and paste your SQL. Via the CLI:
Via the API:

What’s Happening?

  1. Mock source generates fake events continuously
  2. TUMBLE window groups events into 1-minute buckets
  3. Aggregations calculate stats for each window
  4. Preview sink displays results in the UI

Try These Modifications

Filter Events

Add a WHERE clause to process only specific event types:

Add Grouping

Group results by an additional dimension:

Change Window Size

Adjust the window duration for different granularity:

Real Data Sources

Replace the mock source with real connectors: Kafka:
Kinesis:

Continue

Continue with DDL for the complete connection-table and view syntax.
Last modified on October 1, 2026