Ingest Engine metrics
Controller
| Metric | Type | Labels | Description |
|---|---|---|---|
laminar_controller_active_pipelines | Gauge | - | Number of running pipelines |
laminar_controller_registered_nodes | Gauge | - | Number of registered worker nodes |
laminar_controller_registered_slots | Gauge | - | Total slot count across all workers |
laminar_controller_free_slots | Gauge | - | Available (unused) slot count |
laminar_controller_compaction_tuples_in | Gauge | - | Tuples being considered for compaction |
laminar_controller_compaction_tuples_out | Gauge | - | Tuples after compaction |
Node
| Metric | Type | Labels | Description |
|---|---|---|---|
laminar_node_running_workers | Gauge | - | Number of workers managed by this node |
Worker - Task-Level
Common labels:node_id, subtask_idx, operator_name
| Metric | Type | Description |
|---|---|---|
laminar_worker_messages_recv | Counter | Messages received by subtask |
laminar_worker_messages_sent | Counter | Messages sent by subtask |
laminar_worker_bytes_recv | Counter | Bytes received by subtask |
laminar_worker_bytes_sent | Counter | Bytes sent by subtask |
laminar_worker_batches_recv | Counter | Batches received by subtask |
laminar_worker_batches_sent | Counter | Batches sent by subtask |
laminar_worker_deserialization_errors | Counter | Deserialization errors |
Worker - Source
Common labels:job_id, node_id, subtask_idx, operator_name
| Metric | Type | Extra Labels | Description |
|---|---|---|---|
laminar_worker_consumer_lag | Gauge | - | Consumer lag across all partitions |
laminar_worker_source_lag_messages | Gauge | partition | Lag per partition |
laminar_worker_source_errors_total | Counter | error_type | Source connection/read errors |
laminar_worker_source_status | Gauge | - | Connection status (0=disconnected, 1=connected) |
laminar_worker_source_messages_read_total | Counter | - | Messages read from external source |
laminar_worker_source_batches_read_total | Counter | - | Batches read from external source |
laminar_worker_source_bytes_read_total | Counter | - | Bytes read from external source |
Worker - Sink
Common labels:job_id, node_id, subtask_idx, operator_name
| Metric | Type | Description |
|---|---|---|
laminar_worker_sink_buffer_records | Gauge | Records buffered in sink |
laminar_worker_sink_messages_written_total | Counter | Messages written to external sink |
laminar_worker_sink_batches_written_total | Counter | Batches written to external sink |
laminar_worker_sink_bytes_written_total | Counter | Bytes written to external sink |
Worker - Pipeline Latency
| Metric | Type | Labels | Description |
|---|---|---|---|
laminar_worker_barrier_latency_seconds | Histogram | job_id | End-to-end barrier latency (source to sink) |
laminar_worker_operator_barrier_latency_seconds | Histogram | job_id, node_id, subtask_idx, operator_name | Per-operator barrier processing time (debug level) |
laminar_worker_operator_errors_total | Counter | job_id, node_id, subtask_idx, operator_name, operator_type, error_type | Errors per operator |
Worker - Operator Pressure
Common labels:job_id, node_id, subtask_idx, operator_id
| Metric | Type | Description |
|---|---|---|
laminar_worker_operator_inflight_messages | Gauge | In-flight messages at checkpoint boundary |
laminar_worker_operator_inflight_bytes | Gauge | In-flight bytes at checkpoint boundary |
laminar_worker_operator_out_queue_partitions | Gauge | Outbound queue partitions |
laminar_worker_operator_out_queue_capacity | Gauge | Total outbound queue capacity |
laminar_worker_operator_out_queue_remaining | Gauge | Remaining outbound queue slots |
laminar_worker_operator_out_queue_queued | Gauge | Outbound queued messages |
laminar_worker_operator_out_queue_queued_bytes | Gauge | Outbound queued bytes |
laminar_worker_operator_out_queue_saturated | Gauge | Whether outbound queue is saturated (1=yes, 0=no) |
laminar_worker_operator_out_queue_utilization_ratio | Gauge | Queue utilization from 0.0 to 1.0 |
Worker - TX Queues
Common labels:node_id, subtask_idx, operator_name, next_node, next_node_idx
| Metric | Type | Description |
|---|---|---|
laminar_worker_tx_queue_size | Gauge | Size of a tx queue |
laminar_worker_tx_queue_rem | Gauge | Remaining space in a tx queue |
laminar_worker_tx_bytes | Gauge | Bytes queued in a tx queue |
Worker - State
| Metric | Type | Labels | Description |
|---|---|---|---|
laminar_worker_current_files | Gauge | node_id, task_id | Parquet files in checkpoint |
laminar_worker_table_size_keys | Gauge | node_id, task_id, table_char | Keys in state table |
HTTP Source
Common labels:job_id, pipeline_id, subtask_idx, operator_name
| Metric | Type | Extra Labels | Description |
|---|---|---|---|
laminar_worker_http_source_requests_total | Counter | status | HTTP requests received by response status |
laminar_worker_http_source_bytes_received_total | Counter | - | Bytes received via HTTP POST |
laminar_worker_http_source_auth_failures_total | Counter | - | Requests rejected due to auth failure (401) |
laminar_worker_http_source_deserialization_errors_total | Counter | - | Deserialization errors |
laminar_worker_http_source_flushes_total | Counter | trigger | Buffer flushes by trigger type |
laminar_worker_http_source_flush_seconds | Histogram | trigger | Time spent flushing buffer |
laminar_worker_http_source_flush_errors_total | Counter | trigger | Buffer flush errors |
laminar_worker_http_source_buffer_messages | Gauge | - | Messages in channel buffer |
laminar_worker_http_source_server_up | Gauge | - | Server status (0=down, 1=up) |
laminar_worker_http_source_request_size_bytes | Histogram | - | HTTP request body size distribution |
laminar_worker_http_source_queue_wait_seconds | Histogram | - | Time from accept to dequeue |
laminar_worker_http_source_deserialize_seconds | Histogram | - | Deserialization time per request |
laminar_worker_http_source_concurrency_rejections_total | Counter | - | Requests rejected at concurrency limit |
laminar_worker_http_source_events_dropped_total | Counter | - | Events dropped (buffer full or concurrency limit) |
laminar_worker_http_source_bytes_rejected_total | Counter | reason | Bytes in rejected requests |
Iceberg Sink
Common labels:pipeline_id, sink_type, namespace, table
| Metric | Type | Extra Labels | Description |
|---|---|---|---|
iceberg_sink_rows_written_total | Counter | - | Rows written to Iceberg table |
iceberg_sink_bytes_written_total | Counter | - | Bytes written to Iceberg table |
iceberg_sink_batches_received_total | Counter | - | Batches received by sink |
iceberg_sink_data_files_created_total | Counter | - | Data files created |
iceberg_sink_delete_files_created_total | Counter | - | Equality delete files created |
iceberg_sink_commits_total | Counter | status | Commits to Iceberg catalog (success/failure) |
iceberg_sink_commit_duration_seconds | Histogram | - | Time to commit to catalog |
iceberg_sink_checkpoint_duration_seconds | Histogram | - | Time to checkpoint sink |
iceberg_sink_seconds_since_last_commit | Gauge | - | Seconds since last successful commit |
iceberg_sink_positional_deletes_total | Counter | - | In-flight PK cancellations |
iceberg_sink_equality_deletes_total | Counter | - | Rows written to delete files |
iceberg_sink_write_duration_seconds | Histogram | phase | Time in write path phases |
iceberg_sink_commits_dropped_total | Counter | operation | Dropped commits |
iceberg_sink_catalog_operation_duration_seconds | Histogram | operation | Catalog operation duration |
iceberg_sink_catalog_operation_errors_total | Counter | operation | Failed catalog operations |
Storage
| Metric | Type | Labels | Description |
|---|---|---|---|
laminar_storage_ops_total | Counter | op, backend | Object store operations |
laminar_storage_ops_errors_total | Counter | op, backend | Object store operation errors |
laminar_storage_ops_retries_total | Counter | op, backend | Object store retry attempts |
laminar_storage_ops_duration_seconds | Histogram | op, backend | Object store operation latency |
laminar_storage_bytes_total | Counter | direction, backend | Bytes transferred |
Buffer Service
| Metric | Type | Labels | Description |
|---|---|---|---|
laminar_buffer_ingest_requests_total | Counter | status | Ingest requests by status |
laminar_buffer_ingest_bytes_total | Counter | - | Compressed bytes received |
laminar_buffer_channel_depth | Gauge | - | Chunks queued in channel |
laminar_buffer_flush_files_total | Counter | - | Files flushed to storage |
laminar_buffer_flush_bytes_total | Counter | - | Bytes flushed to storage |
laminar_buffer_flush_duration_seconds | Histogram | - | Time to close and upload a file |
laminar_buffer_flush_errors_total | Counter | - | Flush errors (after retries exhausted) |
laminar_buffer_flush_retries_total | Counter | - | Flush retry attempts |
gRPC
| Metric | Type | Labels | Description |
|---|---|---|---|
grpc_request_counter | Counter | - | Total gRPC requests |
Kubernetes Operator Metrics
| Metric | Type | Labels | Description |
|---|---|---|---|
reconcile_duration_seconds | Histogram | controller | Duration of reconcile loops |
reconcile_total | Counter | controller, result | Total reconcile attempts |
reconcile_errors_total | Counter | controller, error_type | Total reconcile errors |
component_ready | Gauge | controller, name, namespace, component | Component readiness (0/1) |
requeue_total | Counter | controller, reason | Total requeues |
manifest_apply_total | Counter | controller, kind, result | Manifest apply attempts |
manifest_apply_duration_seconds | Histogram | controller, kind | Manifest apply duration |
cleanup_duration_seconds | Histogram | controller | Cleanup operation duration |