Quick start
Build your first streaming SQL pipeline in 5 minutes.
Prerequisites
e6 Ingestion Engine running locally or in your cluster
Basic SQL knowledge
Step 1: Create a Source
Define where your data comes from. This example uses a mock source for testing:
CREATE TABLE events (
event_id VARCHAR,
user_id VARCHAR,
event_type VARCHAR,
page VARCHAR,
session_id VARCHAR,
timestamp TIMESTAMP
) WITH (
connector = 'mock',
rows_per_second = '100'
);Step 2: Create a Destination
Define where results go. For testing, use the preview sink:
Step 3: Write Your Query
Connect source to destination with a transformation:
2024-01-15 10:00:00
2024-01-15 10:01:00
6
3
2024-01-15 10:01:00
2024-01-15 10:02:00
8
4
Step 4: Run the Pipeline
Via the UI: Click "Create Pipeline" and paste your SQL.
Via the CLI:
Via the API:
What's Happening?
Mock source generates fake events continuously
TUMBLE window groups events into 1-minute buckets
Aggregations calculate stats for each window
Preview sink displays results in the UI
Try These Modifications
Filter Events
Add a WHERE clause to process only specific event types:
10:00:00
10:01:00
2
2
Add Grouping
Group results by an additional dimension:
page_view
10:00:00
10:01:00
3
click
10:00:00
10:01:00
2
purchase
10:00:00
10:01:00
1
Change Window Size
Adjust the window duration for different granularity:
Real Data Sources
Replace the mock source with real connectors:
Kafka:
Kinesis:
Continue
Continue with DDL for the complete connection-table and view syntax.
Last updated
Was this helpful?

