Sensors

How-to guideApplies to: Unified engine deployments

Wait for an external condition before starting dependent work. A sensor saves its deadline, check history summary, next check time, and last observation. It does not hold a SQL compute worker between checks.

Choose a condition

Condition	Ready when	Setup
Object exists	A named object is present.	A URL within configured readable storage.
HTTP status or job state	The expected HTTP status matches, and an optional JSON state matches.	An accessible endpoint and any secret reference.
Workload run	One exact workload run succeeds.	Its workload ID and run ID.

A sensor is an ordering step. It does not load an object’s data or pass the HTTP response as rows. Give dependent tasks their own SQL and storage access. In the task editor, choose Wait to configure a sensor. Select its condition under External wait, set its timeout and check interval, then connect dependent tasks with ordering-only dependencies.

1. Wait for a known run

First create and run Your first workload. Record that workload’s ID and its successful run ID. Open this example and replace both placeholder IDs in wait_for_orders before saving. They refer to the earlier workload, not the new workload being created.

{
  "name": "Learn - wait for another run",
  "tasks": [
    {
      "name": "wait_for_orders",
      "sql": "",
      "settings": {
        "current_catalog": "default",
        "namespace": [
          "default"
        ],
        "time_zone": "UTC",
        "case_sensitive": false,
        "ansi_enabled": true
      },
      "control": {
        "sensor": {
          "condition": {
            "kind": "workload_run",
            "workload_id": "11111111-1111-4111-8111-111111111111",
            "run_id": "22222222-2222-4222-8222-222222222222"
          },
          "timeout_seconds": 3600,
          "poll_interval_seconds": 30
        }
      }
    },
    {
      "name": "summarize",
      "sql": "WITH orders(id, category, region, amount) AS (\n  VALUES\n    (1, 'Books', 'North', 20),\n    (2, 'Games', 'South', 30),\n    (3, 'Games', 'North', 15),\n    (4, 'Books', 'North', 40),\n    (5, 'Garden', 'South', 25),\n    (6, 'Garden', 'North', 10)\n)\nSELECT COUNT(*) AS order_count, SUM(amount) AS sales FROM orders;",
      "settings": {
        "current_catalog": "default",
        "namespace": [
          "default"
        ],
        "time_zone": "UTC",
        "case_sensitive": false,
        "ansi_enabled": true
      }
    }
  ],
  "dependencies": [
    {
      "upstream": "wait_for_orders",
      "downstream": "summarize",
      "input_name": null
    }
  ],
  "options": {
    "execution_target": "DuckDb"
  },
  "policy": "Manual"
}

Validate, save as Manual, and start the new workload. The sensor should observe the earlier run’s success, then allow summarize to execute. Expect six orders and sales of 140. Repeat using an active upstream run to observe waiting. The sensor checks that exact run, not whichever run becomes latest later. A failed, skipped, or cancelled upstream run fails the condition. A sensor cannot wait for its own run.

2. Wait for an object

Choose Object exists and provide a configured storage URL. A task’s control can contain:

{
  "sensor": {
    "condition": {
      "kind": "object_exists",
      "url": "s3://your-configured-bucket/orders/2026-01-01/_READY"
    },
    "timeout_seconds": 3600,
    "poll_interval_seconds": 30
  }
}

Replace the URL with a location your instance is configured to read. The sensor checks the named object. It does not prove that every file in the directory is complete. Use a producer’s completion marker when that is the contract. A missing object is pending. An unconfigured storage location or unresolved access fails the check. Creating a credential alone does not authorize an arbitrary storage root.

3. Wait for an HTTP job state

Choose HTTP job status. Check status 200 and the /status field in a JSON response:

{
  "sensor": {
    "condition": {
      "kind": "http",
      "url": "https://jobs.example.com/jobs/your-job-id",
      "expected_status": 200,
      "secret": "job_status_token",
      "json_pointer": "/status",
      "expected_value": "completed"
    },
    "timeout_seconds": 1800,
    "poll_interval_seconds": 15
  }
}

Use a real endpoint and a saved bearer-token secret. With only an expected status, a matching status is enough. With a JSON pointer, the selected value must also match. JSON state responses are bounded to 64 KiB. The native host must permit external HTTP. It uses bounded HTTP checks and does not follow redirects. Authentication rejection fails the check. A nonmatching status or unfinished state remains pending until the deadline.

Understand time and failure

Timeout is 1 second to 31 days. Poll interval must be positive and no greater than timeout; its omitted API default is 30 seconds. The timeout starts when the sensor begins waiting after its predecessors succeed. A restart does not reset the deadline. A pending observation schedules another check. An invalid configuration or failed condition can end the sensor early. Expiry fails the task and prevents dependent tasks from proceeding normally. Inspect deadline, next check, number of checks, and last observation in task workflow details. Use Notifications for sensor_timeout events.

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Choose a condition

1. Wait for a known run

2. Wait for an object

3. Wait for an HTTP job state

Understand time and failure

​Choose a condition

​1. Wait for a known run

​2. Wait for an object

​3. Wait for an HTTP job state

​Understand time and failure

Choose a condition

1. Wait for a known run

2. Wait for an object

3. Wait for an HTTP job state

Understand time and failure