apiVersion: laminar.stream/v1alpha1
kind: LaminarTable
metadata:
name: my-table
namespace: e6data
spec:
clusterRef: e6data
connector: kafka
config:
name: my_table
# ... connector config ...
schema: # <-- schema goes here
format:
json: {}
fields:
- field_name: user_id
field_type:
type:
primitive: Int64
nullable: false
Fields
Thefields array defines the structure of each record.
Primitive Types
| Type | Description | SQL Equivalent |
|---|---|---|
Bool | Boolean | BOOLEAN |
Int32 | 32-bit signed integer | INTEGER |
Int64 | 64-bit signed integer | BIGINT |
UInt32 | 32-bit unsigned integer | - |
UInt64 | 64-bit unsigned integer | - |
F32 | 32-bit floating point | FLOAT |
F64 | 64-bit floating point | DOUBLE |
String / Utf8 | UTF-8 string | VARCHAR / TEXT |
Bytes | Binary data | BYTEA |
Json | JSON data | JSON |
Date32 | Date (days since epoch) | DATE |
DateTime | RFC3339 datetime | TIMESTAMP |
UnixMillis | Unix timestamp (milliseconds) | TIMESTAMP |
UnixMicros | Unix timestamp (microseconds) | TIMESTAMP |
UnixNanos | Unix timestamp (nanoseconds) | TIMESTAMP |
Field Definition
Each field requiresfield_name, field_type, and nullable:
fields:
- field_name: user_id
field_type:
type:
primitive: Int64
nullable: false
- field_name: email
field_type:
type:
primitive: Utf8
nullable: true
Struct (Nested Objects)
Use thestruct type for nested objects:
fields:
- field_name: address
field_type:
type:
struct:
fields:
- field_name: street
field_type:
type:
primitive: Utf8
nullable: true
- field_name: city
field_type:
type:
primitive: Utf8
nullable: false
- field_name: zip
field_type:
type:
primitive: Utf8
nullable: true
nullable: true
List (Arrays)
Use thelist type for arrays:
fields:
- field_name: tags
field_type:
type:
list:
field_name: item
field_type:
type:
primitive: Utf8
nullable: false
nullable: true
fields:
- field_name: items
field_type:
type:
list:
field_name: item
field_type:
type:
struct:
fields:
- field_name: sku
field_type:
type:
primitive: Utf8
nullable: false
- field_name: quantity
field_type:
type:
primitive: Int32
nullable: false
nullable: false
nullable: false
Primary Keys
For CDC source tables, specify primary keys to identify unique rows:schema:
format:
json:
debezium: true
primary_keys:
- id
fields:
- field_name: id
field_type:
type:
primitive: Int64
nullable: false
- field_name: name
field_type:
type:
primitive: Utf8
nullable: false
Complete Example
A full LaminarTable with a detailed schema:apiVersion: laminar.stream/v1alpha1
kind: LaminarTable
metadata:
name: order-events
namespace: e6data
spec:
clusterRef: e6data
connector: kafka
config:
name: order_events
connection_profile_id: my-kafka
config:
topic: order-events
type:
source_config:
offset: earliest
schema:
format:
json:
timestamp_format: rfc3339
bad_data:
drop: {}
fields:
- field_name: order_id
field_type:
type:
primitive: Utf8
nullable: false
- field_name: customer
field_type:
type:
struct:
fields:
- field_name: id
field_type:
type:
primitive: Int64
nullable: false
- field_name: name
field_type:
type:
primitive: Utf8
nullable: false
- field_name: email
field_type:
type:
primitive: Utf8
nullable: true
nullable: false
- field_name: items
field_type:
type:
list:
field_name: item
field_type:
type:
struct:
fields:
- field_name: sku
field_type:
type:
primitive: Utf8
nullable: false
- field_name: quantity
field_type:
type:
primitive: Int32
nullable: false
- field_name: price
field_type:
type:
primitive: F64
nullable: false
nullable: false
nullable: false
- field_name: total
field_type:
type:
primitive: F64
nullable: false
- field_name: event_time
field_type:
type:
primitive: DateTime
nullable: false