Prerequisites
- Ingest Engine running locally or in your cluster
- Basic SQL knowledge
Step 1: Create a Source
Define where your data comes from. This example uses a mock source for testing:
Step 2: Create a Destination
Define where results go. For testing, use the preview sink:
Step 3: Write Your Query
Connect source to destination with a transformation:
Step 4: Run the Pipeline
Via the UI: Click “Create Pipeline” and paste your SQL.
Via the CLI:
Via the API:
What’s Happening?
- Mock source generates fake events continuously
- TUMBLE window groups events into 1-minute buckets
- Aggregations calculate stats for each window
- Preview sink displays results in the UI
Try These Modifications
Filter Events
Add a WHERE clause to process only specific event types:
Add Grouping
Group results by an additional dimension:
Change Window Size
Adjust the window duration for different granularity:
Real Data Sources
Replace the mock source with real connectors:
Kafka:
Kinesis:
Continue
Continue with DDL for the complete connection-table and view syntax.Last modified on October 1, 2026