# e6data Docs: Transformation Engine > Product guides and references for e6data Query Engine, Transformation Engine, Ingest Engine, AgentFlow and the Platform they share. ## Transformation Engine ### Transformation Engine - [Transformation Engine](https://docs.e6data.com/transformation/index.md): Start with Learn Laminar for practical examples. Use Workloads to build and schedule task workflows, Streaming to define continuous pipelines with SQL, and the SQL reference to loo ### Learn Laminar - [Learn Laminar](https://docs.e6data.com/transformation/learn/index.md): Build something small, inspect its result, then change one thing. These paths take you from a first query to a scheduled workload. #### SQL basics - [Run your first analysis](https://docs.e6data.com/transformation/learn/analysis.md): Query six orders, filter them by region, group sales by category, and save the analysis. You will finish with three category totals and a query you can reopen. - [Query your first dataset](https://docs.e6data.com/transformation/learn/query-data.md): Read, shape, and sort a small dataset defined inside the query. - [Filter and classify data](https://docs.e6data.com/transformation/learn/filter-data.md): Keep the rows you need and label them with clear rules. #### Transform data - [Handle missing values](https://docs.e6data.com/transformation/learn/missing-values.md): Understand NULL, supply defaults, and count missing values correctly. - [Summarize data](https://docs.e6data.com/transformation/learn/summarize-data.md): Calculate totals and compare groups without creating a table. - [Clean text and convert types](https://docs.e6data.com/transformation/learn/clean-text.md): Normalize text, extract fields, and convert values safely. - [Work with dates and time](https://docs.e6data.com/transformation/learn/dates-and-time.md): Use fixed dates so every run has a clear expected result. - [Read JSON and nested values](https://docs.e6data.com/transformation/learn/json-and-nested.md): Extract useful scalar values from JSON, arrays, and structs. #### Advanced queries - [Combine datasets](https://docs.e6data.com/transformation/learn/combine-data.md): Match customers and orders, and find records with no match. - [Build queries in steps](https://docs.e6data.com/transformation/learn/query-steps.md): Use named steps and subqueries to make an analysis easier to read. - [Compare lists](https://docs.e6data.com/transformation/learn/compare-lists.md): Combine, deduplicate, and compare two small lists. - [Rank and compare rows](https://docs.e6data.com/transformation/learn/rank-and-compare.md): Use window functions to keep detail rows while comparing related values. - [Solve complete analysis problems](https://docs.e6data.com/transformation/learn/analysis-problems.md): Combine the earlier lessons to answer practical questions with small inputs. #### MERGE - [Understand MERGE matches](https://docs.e6data.com/transformation/learn/merge-match.md): Match existing and incoming customers, then update an existing customer and - [Choose which rows MERGE updates](https://docs.e6data.com/transformation/learn/merge-conditions.md): Update selected fields and apply conditions without changing the meaning of a - [Combine MERGE actions](https://docs.e6data.com/transformation/learn/merge-actions.md): Use update, insert, and delete clauses together and observe clause order. - [Synchronise a complete source snapshot](https://docs.e6data.com/transformation/learn/merge-snapshot.md): Mark or remove target customers that are absent from a complete source snapshot. - [Prepare a reliable MERGE source](https://docs.e6data.com/transformation/learn/merge-source.md): Choose one source row per key, match composite keys, and handle NULL keys - [Make repeated MERGE batches predictable](https://docs.e6data.com/transformation/learn/merge-repeat.md): Compare a version-guarded merge with an additive update when a batch is - [Apply customer change events](https://docs.e6data.com/transformation/learn/merge-events.md): Compare physical deletion with a stored deletion marker and reject an older - [Run MERGE incrementally](https://docs.e6data.com/transformation/learn/merge-incremental.md): Append two source batches and apply only each new snapshot interval to an #### Connect data - [Secrets](https://docs.e6data.com/transformation/learn/secrets.md): Create a secret with SQL, inspect its metadata, and reference it from a credential. This page covers encrypted local values and every external secret provider available in Laminar. - [Credentials](https://docs.e6data.com/transformation/learn/credentials.md): Use SQL to define how Laminar authenticates to a service. This page covers all five credential providers and all 31 supported authentication methods. Choose the method that matches - [Connect your data](https://docs.e6data.com/transformation/learn/connect.md): Connect an existing catalog service, inspect a table, and run a query that reads its files. A catalog lists tables and their schemas. Table storage holds the actual data. #### More guides - [Build a dashboard](https://docs.e6data.com/transformation/learn/dashboard.md): Create a table and bar chart for category sales. Add a region filter and verify that changing it changes both widgets. - [Build a notebook](https://docs.e6data.com/transformation/learn/notebook.md): Build a SQL notebook with a setup cell, an analysis cell, and a short explanation. Run it for two regions and inspect the recorded results. - [Automate an analysis](https://docs.e6data.com/transformation/learn/automate.md): Create a two-task SQL workload. The first task groups sample orders. The second writes those results to a private table. Verify the output, then add a schedule. - [Process new data](https://docs.e6data.com/transformation/learn/incremental.md): Copy an initial source batch, append one new order, and run again. Verify that the output grows by one row because the workload keeps its source position. ### Workloads - [Workloads](https://docs.e6data.com/transformation/workloads/index.md): Turn a query into a repeatable workflow. Define tasks, connect their dependencies, choose compute, and decide when each run should happen. #### Get started - [Your first workload](https://docs.e6data.com/transformation/workloads/first-workload.md): Create a workload that counts six orders and calculates total sales. Then run the same saved definition again and compare the two run identities. - [Build a multi-task workflow](https://docs.e6data.com/transformation/workloads/multi-task-workflow.md): Read six orders once, summarize them in two parallel paths, and combine both summaries. This example introduces a task graph without external data. #### Build workflows - [Tasks and dependencies](https://docs.e6data.com/transformation/workloads/tasks-and-dependencies.md): Use tasks to give each step a clear purpose. Connect them when one step needs another step's successful completion or returned rows. - [Parameters and query context](https://docs.e6data.com/transformation/workloads/parameters.md): Keep one workflow definition and supply different typed values for each run. Set the initial catalog, namespace, SQL time zone, and name rules once for all tasks. - [Compute and execution limits](https://docs.e6data.com/transformation/workloads/compute.md): Choose where task attempts execute and how much work can proceed together. A graph defines dependencies; compute settings determine whether ready tasks have capacity to run. - [Publish a table result](https://docs.e6data.com/transformation/workloads/publish-results.md): Turn the inline orders summary into a durable Iceberg table. Verify the commit, query the output, and repeat the run without adding duplicate summary rows. - [Incremental workloads](https://docs.e6data.com/transformation/workloads/incremental.md): Process changes from a tracked source table using saved progress. This is different from filtering rows by a time window or running a continuous streaming pipeline. #### Schedule workflows - [Schedules and time zones](https://docs.e6data.com/transformation/workloads/schedules.md): A schedule decides when run requests become due. It does not execute SQL itself or guarantee immediate worker availability. - [Overlap, missed runs, and queues](https://docs.e6data.com/transformation/workloads/concurrency.md): Schedules can produce work faster than it finishes. Choose what should happen when a run is still active or the scheduler finds overdue occurrences. - [Data windows](https://docs.e6data.com/transformation/workloads/data-windows.md): Give each run an explicit period of data to process. A data window records the intended interval even if execution starts late or the run is repeated later. - [Backfills](https://docs.e6data.com/transformation/workloads/backfills.md): Process a chosen range of historical schedule windows. Preview the windows before starting, decide which existing runs make a window eligible, and limit concurrent execution. #### Control execution - [Inspect runs and attempts](https://docs.e6data.com/transformation/workloads/runs.md): Follow the recorded execution rather than assuming the current editor represents what an earlier run used. A run connects the saved revision, parameter values, schedule trigger, ta - [Retry policies](https://docs.e6data.com/transformation/workloads/retries.md): Retry an eligible failed task within the same run. The controller owns the retry decision and saves the policy and delay so a restart does not create a new retry budget. - [Selective reruns](https://docs.e6data.com/transformation/workloads/reruns.md): Repeat selected tasks from a completed run while reusing eligible successful work. A selective rerun creates a new linked run; the source run stays unchanged. - [Sensors](https://docs.e6data.com/transformation/workloads/sensors.md): Wait for an external condition before starting dependent work. A sensor saves its deadline, check history summary, next check time, and last observation. It does not hold a SQL com - [Notifications](https://docs.e6data.com/transformation/workloads/notifications.md): Send workload events to an HTTP receiver or Slack webhook. Delivery is saved separately from task execution, so a destination outage does not change a completed run's outcome. - [Troubleshooting](https://docs.e6data.com/transformation/workloads/troubleshooting.md): Start with the run identity and the exact phase or error. Separate scheduling, waiting, execution, publication, and notification delivery before changing the definition. #### Reference - [Results and table commits](https://docs.e6data.com/transformation/workloads/results-and-commits.md): A query result and a confirmed table write are different outputs. Some tasks return rows, some publish a table, and some only wait or perform an action. - [Definition and API reference](https://docs.e6data.com/transformation/workloads/reference.md): Use the same workload model as the editor when creating or managing workflows through HTTP. Routes below are relative to your engine's HTTP origin and include the /api/v1 prefix. ### SQL reference #### Getting started - [SQL reference](https://docs.e6data.com/transformation/sql/index.md): Look up Unified SQL syntax, types, functions, and statement behavior. SQL examples can be copied into your Unified SQL client. - [Example data](https://docs.e6data.com/transformation/sql/example-data.md): Live table examples use Hub's Lakekeeper Iceberg REST catalog and MinIO storage. Hub runs separately from the engine through Docker Compose. Follow Hub setup to start the services, - [SQL overview](https://docs.e6data.com/transformation/sql/overview.md): Unified accepts Spark-style SQL with documented engine and connector restrictions. The same language is available through Workbench, SQL notebook cells, and live reference examples #### Query syntax - [Queries](https://docs.e6data.com/transformation/sql/queries.md): SELECT returns expressions from a source. A source can be a table, view, subquery, inline VALUES relation, or a supported table function. A SELECT without FROM evaluates scalar exp - [Joins](https://docs.e6data.com/transformation/sql/joins.md): Joins combine relations using keys or predicates. Qualify shared column names with source aliases. Matching duplicate keys can multiply result rows; inspect key uniqueness before s - [Aggregations](https://docs.e6data.com/transformation/sql/aggregations.md): Aggregate functions reduce rows to a value per group. Every selected expression must be grouped or aggregated, apart from supported constant expressions. - [CTEs and subqueries](https://docs.e6data.com/transformation/sql/subqueries.md): A common table expression names an intermediate query within one statement. A subquery places a query inside another query. Neither automatically persists its result as a table. - [Set operations](https://docs.e6data.com/transformation/sql/set-operations.md): Set operations combine compatible result shapes. Inputs must have the same number of columns and compatible types. Use aliases in the first input to name the result. - [Window queries](https://docs.e6data.com/transformation/sql/windows.md): Window functions compute values over related rows while retaining a result row for each input row. The OVER clause defines the partition, ordering, and optional frame. - [Operators and nulls](https://docs.e6data.com/transformation/sql/operators.md): Operators combine values into expressions. Use parentheses to make precedence explicit, especially when mixing AND with OR or combining numeric operations. #### Data types - [Types and expressions](https://docs.e6data.com/transformation/sql/expressions.md): Expressions produce values from literals, columns, operators, functions, casts, and subqueries. The engine resolves types before execution. Sources and destinations can support a n #### Functions - [Function inventory](https://docs.e6data.com/transformation/sql/functions.md): Functions are grouped by purpose below. The inventory on this page is generated from the exact function registry compiled into Unified and includes canonical names, aliases, syntax - [String functions](https://docs.e6data.com/transformation/sql/functions-string.md): String functions operate on text expressions. The common functions below return NULL for a NULL required argument unless the row documents a different behavior. Character counts an - [Numeric functions](https://docs.e6data.com/transformation/sql/functions-numeric.md): Numeric functions resolve an overload from the input types. Cast an input explicitly when an example needs floating-point behavior. Most required null arguments produce NULL. Inval - [Date and time functions](https://docs.e6data.com/transformation/sql/functions-datetime.md): Temporal operations depend on the input type and session time zone. DATE is a calendar value; TIMESTAMP and TIMESTAMP_NTZ have different zone semantics. The examples use date_dim a - [Conditional functions](https://docs.e6data.com/transformation/sql/functions-conditional.md): Conditional expressions choose or normalize values. Their result branches must resolve to compatible types. Explicit casts are useful when mixing numeric, text, or temporal argumen - [JSON functions](https://docs.e6data.com/transformation/sql/functions-json.md): Unified provides json_extract_string(json_text, path) for extracting a value from JSON text. It returns STRING or NULL. - [Arrays, maps, and structs](https://docs.e6data.com/transformation/sql/functions-nested.md): Nested values make it possible to carry related values in one expression. Input element and field types are resolved during planning, and backend support can be narrower than the f - [Aggregate functions](https://docs.e6data.com/transformation/sql/functions-aggregate.md): Aggregate functions compute a value for a group. They can also appear as aggregate window functions where supported. Use GROUP BY to define groups and HAVING to filter them. - [Window functions](https://docs.e6data.com/transformation/sql/functions-window.md): Window functions require OVER to define the relevant rows. Their ordering is independent of the query's final presentation order. See window queries(windows.md) for frame syntax. #### Catalogs, tables, and views - [Statements](https://docs.e6data.com/transformation/sql/statements.md): Use statements to inspect objects, define tables and views, and change data. Commands run in a session; a script stops at its first failed statement. Connector capabilities determi - [Tables and sources](https://docs.e6data.com/transformation/sql/tables-and-sources.md): A catalog supplies object metadata. A source connector reads rows and a sink connector publishes writes. Their supported types, write modes, and maintenance operations can differ. - [TBLPROPERTIES](https://docs.e6data.com/transformation/sql/tblproperties.md): TBLPROPERTIES saves named settings on a table. Use it to record custom metadata, choose an Iceberg format version at creation, or control supported options for later writes. #### Writing data - [Writing data](https://docs.e6data.com/transformation/sql/writes.md): Prepare a private write namespace as described in Example data(example-data.md). Examples use demo.examples for its private namespace. Each script first recreates its own small tab #### Table maintenance - [Table maintenance](https://docs.e6data.com/transformation/sql/maintenance/index.md): Table maintenance improves the physical layout of an existing table. Use it to combine small data files, combine positional-delete files, or reorganize manifest metadata. Start by - [Inspect and prepare a table](https://docs.e6data.com/transformation/sql/maintenance/inspect.md): Inspect the table before choosing maintenance. Keep a small set of before-and-after measurements so you can explain what changed. - [Compact data files](https://docs.e6data.com/transformation/sql/maintenance/compact-data.md): Data compaction reads eligible data files and writes a new layout. Bin packing combines small files. Sorting orders records by chosen columns within the rewritten output. Z-order p - [Compact delete files](https://docs.e6data.com/transformation/sql/maintenance/compact-deletes.md): Iceberg format v2 can record row changes in positional-delete files. Each delete entry identifies a row position in a data file. Delete compaction combines eligible delete files wh - [Optimize manifests](https://docs.e6data.com/transformation/sql/maintenance/manifests.md): An Iceberg manifest lists data files or delete files and their metadata. Manifest optimization rewrites that metadata layout. Use it when file metadata is fragmented across many ma - [Maintenance SQL reference](https://docs.e6data.com/transformation/sql/maintenance/reference.md): CALL submits one table maintenance operation. The target catalog and connector must support its action. The current native Iceberg provider supports data compaction, positional-del ### HTTP API - [HTTP API](https://docs.e6data.com/transformation/http/index.md): Use HTTP to query data, save workloads, run notebooks, refresh dashboards, manage catalogs, and inspect execution. The same engine services handle requests from the console and fro #### Get started - [Make your first request](https://docs.e6data.com/transformation/http/first-request.md): Read capabilities and compute choices before creating resources. This example needs an accessible engine and no external data. - [Addresses and access](https://docs.e6data.com/transformation/http/access.md): Use the HTTP address and API prefix of the instance you intend to operate. The native engine routes use /api/v1. An embedded deployment can mount Unified under another prefix, such - [Request fields and types](https://docs.e6data.com/transformation/http/requests.md): Each reference page lists its method, path, parameters, JSON body, response, and error behavior. The server remains authoritative for permissions and semantic rules. - [Request identity and revisions](https://docs.e6data.com/transformation/http/request-identity.md): Identify one intended operation with the identity field that API expects. Keep that identity and request content when recovering a lost response. - [Asynchronous execution and cancellation](https://docs.e6data.com/transformation/http/async.md): Many execute APIs accept work before it finishes. HTTP 202 means acceptance, not successful SQL, table publication, or notification delivery. - [Pagination and Arrow results](https://docs.e6data.com/transformation/http/results.md): List endpoints and query-result endpoints have different paging units. Read each operation's query parameters and response cursor before fetching another page. - [Errors and recovery](https://docs.e6data.com/transformation/http/errors.md): Read the status and error code together. A failed HTTP exchange does not always prove that a mutation was rejected. - [Events and streaming responses](https://docs.e6data.com/transformation/http/events.md): Some APIs return a bounded event page. Notebook cell events and assistant turns also expose server-sent event streams. These are observation APIs; listening does not itself cancel #### Complete examples - [Query inline orders](https://docs.e6data.com/transformation/http/examples/query-orders.md): Use only HTTP to count and summarize inline data. No files, catalog connection, or credentials are needed. - [Create and run a workload](https://docs.e6data.com/transformation/http/examples/workload.md): Save a Manual SQL workload, read its revision, start a run, and inspect its tasks. This example uses inline orders and needs only a running engine and available compute. - [Browse a catalog](https://docs.e6data.com/transformation/http/examples/catalog.md): Discover existing catalogs and relations without executing a data query. - [Save a query document](https://docs.e6data.com/transformation/http/examples/saved-query.md): Save reusable SQL without executing it. The workspace service must be enabled. - [Run a notebook](https://docs.e6data.com/transformation/http/examples/notebook.md): Create an inline SQL notebook, open its SQL kernel session, execute a cell, read outputs, and close the session. The notebook service and SQL kernel must be enabled. - [Refresh a dashboard](https://docs.e6data.com/transformation/http/examples/dashboard.md): Create a dashboard with one inline SQL dataset, request a refresh, and inspect its snapshot. The dashboard service and query compute must be enabled. - [Backfill historical windows](https://docs.e6data.com/transformation/http/examples/backfill.md): Preview historical schedule windows before creating runs. Use a saved cron or interval batch workload with data windows enabled and no incremental cursor. - [Publish and maintain a table](https://docs.e6data.com/transformation/http/examples/publish-maintain.md): Use SQL over HTTP to create a private Iceberg table, publish inline rows, and inspect it. Then use the supported maintenance statements for that table. - [Investigate a run](https://docs.e6data.com/transformation/http/examples/inspect-run.md): Inspect an existing workload execution without starting replacement work. Enter its workload ID and run ID under Example values. - [Use assistant conversations and events](https://docs.e6data.com/transformation/http/examples/assistant.md): Create a conversation, submit one read-scope turn, watch its events, and read the final result. The assistant must be configured and its provider available. - [SQL-managed features over HTTP](https://docs.e6data.com/transformation/http/sql-managed-features.md): Use the session operation API for features whose public definition is SQL. The HTTP transport carries the SQL statement; the SQL reference defines its syntax. - [Define a streaming pipeline over HTTP](https://docs.e6data.com/transformation/http/examples/streaming.md): Register a Kafka source, register an Iceberg sink, and create a continuous pipeline through SQL session operations. Execution uses the native streaming path. - [Compute, configuration, and observability](https://docs.e6data.com/transformation/http/administration.md): Use read APIs to understand the instance before changing shared settings. Administrative HTTP actions use the deployment's existing access rules. #### API reference - [API reference](https://docs.e6data.com/transformation/http/reference/index.md): Choose an operation by resource. Read parameters beside request and response examples. Copy requests into your own client. ##### Observability - [Activity runs](https://docs.e6data.com/transformation/http/reference/activity-runs.md): activity runs. GET /api/v1/activity/{kind}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Audit history](https://docs.e6data.com/transformation/http/reference/audit-history.md): audit history. GET /api/v1/audit. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Global operation history](https://docs.e6data.com/transformation/http/reference/global-operation-history.md): global operation history. GET /api/v1/operations. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Dashboard metrics](https://docs.e6data.com/transformation/http/reference/dashboard-metrics.md): dashboard metrics. POST /api/v1/telemetry/metrics/dashboard. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Prometheus query](https://docs.e6data.com/transformation/http/reference/prometheus-query.md): Proxy a supported Prometheus API path. Query values remain URL query parameters for both GET and POST. - [Prometheus query post](https://docs.e6data.com/transformation/http/reference/prometheus-query-post.md): Proxy a supported Prometheus API path. Query values remain URL query parameters for both GET and POST. - [Telemetry sql](https://docs.e6data.com/transformation/http/reference/telemetry-sql.md): telemetry sql. POST /api/v1/telemetry/sql. Read local Greptime with one bounded SELECT query. Select metrics, logs, or traces. Writes, external files, and other databases are rejec - [Telemetry status](https://docs.e6data.com/transformation/http/reference/telemetry-status.md): telemetry status. GET /api/v1/telemetry/status. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Telemetry traces](https://docs.e6data.com/transformation/http/reference/telemetry-traces.md): telemetry traces. GET /api/v1/telemetry/traces. Read bounded spans from the latest 15 minutes. Trace IDs accept hexadecimal or UUID form. ##### Assistant - [Assistant capabilities](https://docs.e6data.com/transformation/http/reference/assistant-capabilities.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant list conversations](https://docs.e6data.com/transformation/http/reference/assistant-list-conversations.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant create conversation](https://docs.e6data.com/transformation/http/reference/assistant-create-conversation.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant get conversation](https://docs.e6data.com/transformation/http/reference/assistant-get-conversation.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant submit turn](https://docs.e6data.com/transformation/http/reference/assistant-submit-turn.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant get turn](https://docs.e6data.com/transformation/http/reference/assistant-get-turn.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant approve action](https://docs.e6data.com/transformation/http/reference/assistant-approve-action.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant cancel turn](https://docs.e6data.com/transformation/http/reference/assistant-cancel-turn.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant events](https://docs.e6data.com/transformation/http/reference/assistant-events.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Assistant event stream](https://docs.e6data.com/transformation/http/reference/assistant-event-stream.md): Watch assistant events as server-sent events. Last-Event-ID or query.after resumes after a sequence. Stop watching does not cancel the turn. ##### Configuration and capabilities - [Capabilities](https://docs.e6data.com/transformation/http/reference/capabilities.md): capabilities. GET /api/v1/capabilities. Discover configured roles, execution targets, notebook and workspace availability. - [Get configuration](https://docs.e6data.com/transformation/http/reference/get-configuration.md): get configuration. GET /api/v1/config. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Save configuration](https://docs.e6data.com/transformation/http/reference/save-configuration.md): save configuration. PUT /api/v1/config. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Validate configuration](https://docs.e6data.com/transformation/http/reference/validate-configuration.md): validate configuration. POST /api/v1/config/validate. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get feature flags](https://docs.e6data.com/transformation/http/reference/get-feature-flags.md): get feature flags. GET /api/v1/feature-flags. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Update feature flag](https://docs.e6data.com/transformation/http/reference/update-feature-flag.md): update feature flag. PUT /api/v1/feature-flags. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Search resources](https://docs.e6data.com/transformation/http/reference/search-resources.md): search resources. GET /api/v1/search/{kind}. Uses the same payload, validation, revision checks and response as the unified HTTP API. ##### Catalogs and tables - [List catalogs](https://docs.e6data.com/transformation/http/reference/list-catalogs.md): list catalogs. GET /api/v1/catalogs. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create catalog](https://docs.e6data.com/transformation/http/reference/create-catalog.md): create catalog. POST /api/v1/catalogs. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Validate catalog connection](https://docs.e6data.com/transformation/http/reference/validate-catalog-connection.md): validate catalog connection. POST /api/v1/catalogs/validate. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Delete catalog](https://docs.e6data.com/transformation/http/reference/delete-catalog.md): delete catalog. DELETE /api/v1/catalogs/{catalog_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get catalog](https://docs.e6data.com/transformation/http/reference/get-catalog.md): get catalog. GET /api/v1/catalogs/{catalog_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Update catalog](https://docs.e6data.com/transformation/http/reference/update-catalog.md): update catalog. PATCH /api/v1/catalogs/{catalog_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Catalog capabilities](https://docs.e6data.com/transformation/http/reference/catalog-capabilities.md): catalog capabilities. GET /api/v1/catalogs/{catalog_id}/capabilities. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List namespaces](https://docs.e6data.com/transformation/http/reference/list-namespaces.md): list namespaces. GET /api/v1/catalogs/{catalog_id}/namespaces. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Patch catalog](https://docs.e6data.com/transformation/http/reference/patch-catalog.md): patch catalog. PATCH /api/v1/catalogs/{catalog_id}/options. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List relations](https://docs.e6data.com/transformation/http/reference/list-relations.md): list relations. GET /api/v1/catalogs/{catalog_id}/relations. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get relation](https://docs.e6data.com/transformation/http/reference/get-relation.md): get relation. GET /api/v1/catalogs/{catalog_id}/relations/{name}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Relation capabilities](https://docs.e6data.com/transformation/http/reference/relation-capabilities.md): relation capabilities. GET /api/v1/catalogs/{catalog_id}/relations/{name}/capabilities. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Set table format version](https://docs.e6data.com/transformation/http/reference/set-table-format-version.md): set table format version. POST /api/v1/catalogs/{catalog_id}/relations/{name}/format-version. Uses the same payload, validation, revision checks and response as the unified HTTP AP - [Set table sort order](https://docs.e6data.com/transformation/http/reference/set-table-sort-order.md): set table sort order. POST /api/v1/catalogs/{catalog_id}/relations/{name}/sort-order. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Search relations](https://docs.e6data.com/transformation/http/reference/search-relations.md): search relations. GET /api/v1/catalogs/{catalog_id}/search. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Register table](https://docs.e6data.com/transformation/http/reference/register-table.md): register table. POST /api/v1/catalogs/{catalog_id}/tables/register. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Inspect table](https://docs.e6data.com/transformation/http/reference/inspect-table.md): inspect table. GET /api/v1/catalogs/{catalog_id}/tables/{name}/inspection. Inspect Iceberg snapshots, history, files or summary at an optional pinned snapshot. Pass namespace as an - [Rename view](https://docs.e6data.com/transformation/http/reference/rename-view.md): rename view. POST /api/v1/catalogs/{catalog_id}/views/{name}/rename. Uses the same payload, validation, revision checks and response as the unified HTTP API. ##### Dashboards - [List dashboards](https://docs.e6data.com/transformation/http/reference/list-dashboards.md): list dashboards. GET /api/v1/dashboards. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create dashboard](https://docs.e6data.com/transformation/http/reference/create-dashboard.md): create dashboard. POST /api/v1/dashboards. Save the dedicated dashboard definition: SQL datasets, parameter definitions, widgets, layout, schedule and retention. Preserve dataset a - [Preview dashboard dataset](https://docs.e6data.com/transformation/http/reference/preview-dashboard-dataset.md): preview dashboard dataset. POST /api/v1/dashboards/preview. Execute a read-only dashboard dataset preview with parameters. Returns a session and operation identity; poll operation_ - [Delete dashboard](https://docs.e6data.com/transformation/http/reference/delete-dashboard.md): delete dashboard. DELETE /api/v1/dashboards/{dashboard_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get dashboard](https://docs.e6data.com/transformation/http/reference/get-dashboard.md): get dashboard. GET /api/v1/dashboards/{dashboard_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Save dashboard](https://docs.e6data.com/transformation/http/reference/save-dashboard.md): save dashboard. PUT /api/v1/dashboards/{dashboard_id}. Save the dedicated dashboard definition: SQL datasets, parameter definitions, widgets, layout, schedule and retention. Preser - [Activate dashboard](https://docs.e6data.com/transformation/http/reference/activate-dashboard.md): activate dashboard. POST /api/v1/dashboards/{dashboard_id}/activate. Enable scheduled dashboard refreshes using the current revision. This can start engine work. - [Pause dashboard](https://docs.e6data.com/transformation/http/reference/pause-dashboard.md): pause dashboard. POST /api/v1/dashboards/{dashboard_id}/pause. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Refresh dashboard](https://docs.e6data.com/transformation/http/reference/refresh-dashboard.md): refresh dashboard. POST /api/v1/dashboards/{dashboard_id}/refresh. Start a dashboard snapshot refresh. Reuse submission_id after a lost response and poll dashboard_snapshot. Read e - [Resume dashboard](https://docs.e6data.com/transformation/http/reference/resume-dashboard.md): resume dashboard. POST /api/v1/dashboards/{dashboard_id}/resume. Enable scheduled dashboard refreshes using the current revision. This can start engine work. - [Dashboard history](https://docs.e6data.com/transformation/http/reference/dashboard-history.md): dashboard history. GET /api/v1/dashboards/{dashboard_id}/snapshots. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Dashboard snapshot](https://docs.e6data.com/transformation/http/reference/dashboard-snapshot.md): dashboard snapshot. GET /api/v1/dashboards/{dashboard_id}/snapshots/{refresh_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Cancel dashboard refresh](https://docs.e6data.com/transformation/http/reference/cancel-dashboard-refresh.md): cancel dashboard refresh. POST /api/v1/dashboards/{dashboard_id}/snapshots/{refresh_id}/cancel. Uses the same payload, validation, revision checks and response as the unified HTTP - [Dashboard dataset results](https://docs.e6data.com/transformation/http/reference/dashboard-dataset-results.md): Read a completed result page as base64 Arrow IPC. Query limit counts batches, not rows; the HTTP default is one batch. Use continuation_token for the next page. HTTP does not accep ##### Compute and workers - [Execution options](https://docs.e6data.com/transformation/http/reference/execution-options.md): execution options. GET /api/v1/execution-options. List configured provisioner names, compute profiles, instance defaults, and known resource limits. No credentials are returned. - [List workers](https://docs.e6data.com/transformation/http/reference/list-workers.md): list workers. GET /api/v1/workers. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Update worker admission](https://docs.e6data.com/transformation/http/reference/update-worker-admission.md): update worker admission. POST /api/v1/workers/admission. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Worker capacity](https://docs.e6data.com/transformation/http/reference/worker-capacity.md): worker capacity. GET /api/v1/workers/capacity. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create deployment profile](https://docs.e6data.com/transformation/http/reference/create-deployment-profile.md): Change shared compute capacity. CPU and memory choices affect new work; review existing profiles and limits first. - [Delete deployment profile](https://docs.e6data.com/transformation/http/reference/delete-deployment-profile.md): delete deployment profile. DELETE /api/v1/workers/profiles/{name}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Update deployment profile](https://docs.e6data.com/transformation/http/reference/update-deployment-profile.md): Change shared compute capacity. CPU and memory choices affect new work; review existing profiles and limits first. - [Change deployment profile state](https://docs.e6data.com/transformation/http/reference/change-deployment-profile-state.md): change deployment profile state. POST /api/v1/workers/profiles/{name}/state. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Worker details](https://docs.e6data.com/transformation/http/reference/worker-details.md): worker details. GET /api/v1/workers/{allocation_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Retire deployment worker](https://docs.e6data.com/transformation/http/reference/retire-deployment-worker.md): retire deployment worker. POST /api/v1/workers/{allocation_id}/retire. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Retry worker provisioning](https://docs.e6data.com/transformation/http/reference/retry-worker-provisioning.md): retry worker provisioning. POST /api/v1/workers/{allocation_id}/retry-provisioning. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Worker tasks](https://docs.e6data.com/transformation/http/reference/worker-tasks.md): worker tasks. GET /api/v1/workers/{allocation_id}/tasks. Uses the same payload, validation, revision checks and response as the unified HTTP API. ##### Notebooks - [List notebooks](https://docs.e6data.com/transformation/http/reference/list-notebooks.md): list notebooks. GET /api/v1/notebooks. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create notebook](https://docs.e6data.com/transformation/http/reference/create-notebook.md): create notebook. POST /api/v1/notebooks. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Import notebook](https://docs.e6data.com/transformation/http/reference/import-notebook.md): import notebook. POST /api/v1/notebooks/import. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List notebook summaries](https://docs.e6data.com/transformation/http/reference/list-notebook-summaries.md): list notebook summaries. GET /api/v1/notebooks/summaries. Prefer this lightweight listing to list_notebooks when document bodies and outputs are unnecessary. - [Delete notebook](https://docs.e6data.com/transformation/http/reference/delete-notebook.md): delete notebook. DELETE /api/v1/notebooks/{notebook_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get notebook](https://docs.e6data.com/transformation/http/reference/get-notebook.md): get notebook. GET /api/v1/notebooks/{notebook_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Save notebook](https://docs.e6data.com/transformation/http/reference/save-notebook.md): save notebook. PUT /api/v1/notebooks/{notebook_id}. Save a complete ipynb document using its current revision. Include existing settings to preserve execution settings. - [Export notebook](https://docs.e6data.com/transformation/http/reference/export-notebook.md): export notebook. GET /api/v1/notebooks/{notebook_id}/export. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Notebook revisions](https://docs.e6data.com/transformation/http/reference/notebook-revisions.md): notebook revisions. GET /api/v1/notebooks/{notebook_id}/revisions. Read saved notebook revision history or one exact revision without changing the active document. - [Notebook revision](https://docs.e6data.com/transformation/http/reference/notebook-revision.md): notebook revision. GET /api/v1/notebooks/{notebook_id}/revisions/{revision}. Read saved notebook revision history or one exact revision without changing the active document. - [Notebook run summaries](https://docs.e6data.com/transformation/http/reference/notebook-run-summaries.md): notebook run summaries. GET /api/v1/notebooks/{notebook_id}/run-summaries. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List notebook runs](https://docs.e6data.com/transformation/http/reference/list-notebook-runs.md): list notebook runs. GET /api/v1/notebooks/{notebook_id}/runs. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Submit notebook run](https://docs.e6data.com/transformation/http/reference/submit-notebook-run.md): submit notebook run. POST /api/v1/notebooks/{notebook_id}/runs. Run one cell (cell_id) or all code cells sequentially. Python runs trusted code in the notebook kernel with spark av - [Get notebook run](https://docs.e6data.com/transformation/http/reference/get-notebook-run.md): get notebook run. GET /api/v1/notebooks/{notebook_id}/runs/{run_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Notebook events](https://docs.e6data.com/transformation/http/reference/notebook-events.md): Use the connected instance's access context. Read identifiers and current state before submitting a mutation. - [Notebook outputs](https://docs.e6data.com/transformation/http/reference/notebook-outputs.md): notebook outputs. GET /api/v1/notebooks/{notebook_id}/runs/{run_id}/cells/{execution_id}/outputs. Read retained cell output events, including sequence IDs. execution_id is a cell e ##### Sessions and SQL - [List notebook sessions](https://docs.e6data.com/transformation/http/reference/list-notebook-sessions.md): list notebook sessions. GET /api/v1/notebooks/{notebook_id}/sessions. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Open notebook session](https://docs.e6data.com/transformation/http/reference/open-notebook-session.md): open notebook session. POST /api/v1/notebooks/{notebook_id}/sessions. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Close notebook session](https://docs.e6data.com/transformation/http/reference/close-notebook-session.md): close notebook session. DELETE /api/v1/notebooks/{notebook_id}/sessions/{session_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get notebook session](https://docs.e6data.com/transformation/http/reference/get-notebook-session.md): get notebook session. GET /api/v1/notebooks/{notebook_id}/sessions/{session_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Interrupt notebook session](https://docs.e6data.com/transformation/http/reference/interrupt-notebook-session.md): interrupt notebook session. POST /api/v1/notebooks/{notebook_id}/sessions/{session_id}/interrupt. Interrupt the notebook kernel and request cancellation of active engine operations - [List session summaries](https://docs.e6data.com/transformation/http/reference/list-session-summaries.md): list session summaries. GET /api/v1/sessions. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Open session](https://docs.e6data.com/transformation/http/reference/open-session.md): open session. POST /api/v1/sessions. Create a session owned by the configured HTTP owner. Creation is not idempotent; retain the returned session_id. - [Close session](https://docs.e6data.com/transformation/http/reference/close-session.md): close session. DELETE /api/v1/sessions/{session_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Read session](https://docs.e6data.com/transformation/http/reference/read-session.md): read session. GET /api/v1/sessions/{session_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Analyze](https://docs.e6data.com/transformation/http/reference/analyze.md): analyze. POST /api/v1/sessions/{session_id}/analyze. Inspect SQL schema or explain its plan without submitting an execution. - [Get config](https://docs.e6data.com/transformation/http/reference/get-config.md): get config. GET /api/v1/sessions/{session_id}/config. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Set config](https://docs.e6data.com/transformation/http/reference/set-config.md): set config. PATCH /api/v1/sessions/{session_id}/config. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Unset config](https://docs.e6data.com/transformation/http/reference/unset-config.md): unset config. DELETE /api/v1/sessions/{session_id}/config/{key}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List operations](https://docs.e6data.com/transformation/http/reference/list-operations.md): list operations. GET /api/v1/sessions/{session_id}/operations. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Submit operation](https://docs.e6data.com/transformation/http/reference/submit-operation.md): submit operation. POST /api/v1/sessions/{session_id}/operations. Execute SQL in an existing session. SQL may read or modify data. Returns operation identity/status, not completed r - [Operation status](https://docs.e6data.com/transformation/http/reference/operation-status.md): operation status. GET /api/v1/sessions/{session_id}/operations/{operation_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Operation attempts](https://docs.e6data.com/transformation/http/reference/operation-attempts.md): operation attempts. GET /api/v1/sessions/{session_id}/operations/{operation_id}/attempts. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get operation attempt](https://docs.e6data.com/transformation/http/reference/get-operation-attempt.md): get operation attempt. GET /api/v1/sessions/{session_id}/operations/{operation_id}/attempts/{attempt_id}. Uses the same payload, validation, revision checks and response as the uni - [Cancel operation](https://docs.e6data.com/transformation/http/reference/cancel-operation.md): cancel operation. POST /api/v1/sessions/{session_id}/operations/{operation_id}/cancel. Request cancellation of this session operation. Repeated calls are safe. Poll operation_statu - [Operation results](https://docs.e6data.com/transformation/http/reference/operation-results.md): Read a completed result page as base64 Arrow IPC. Query limit counts batches, not rows; the HTTP default is one batch. Use continuation_token for the next page. HTTP does not accep - [Operation metadata](https://docs.e6data.com/transformation/http/reference/operation-metadata.md): operation metadata. GET /api/v1/sessions/{session_id}/operations/{operation_id}/results/metadata. Uses the same payload, validation, revision checks and response as the unified HTT - [Operation xray](https://docs.e6data.com/transformation/http/reference/operation-xray.md): operation xray. GET /api/v1/sessions/{session_id}/operations/{operation_id}/xray. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Session summary](https://docs.e6data.com/transformation/http/reference/session-summary.md): session summary. GET /api/v1/sessions/{session_id}/summary. Uses the same payload, validation, revision checks and response as the unified HTTP API. ##### Workloads - [Global run summaries](https://docs.e6data.com/transformation/http/reference/global-run-summaries.md): global run summaries. GET /api/v1/runs. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List workloads](https://docs.e6data.com/transformation/http/reference/list-workloads.md): list workloads. GET /api/v1/workloads. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create workload](https://docs.e6data.com/transformation/http/reference/create-workload.md): create workload. POST /api/v1/workloads. Save SQL, batch Python and PySpark streaming tasks, dependencies, schedule and execution options. Python defaults to Batch; set python_mode - [Validate workload](https://docs.e6data.com/transformation/http/reference/validate-workload.md): validate workload. POST /api/v1/workloads/validate. Validate a workload. Batch Python is not executed. Explicit Streaming Python runs as trusted code to capture its streaming plan, - [Delete workload](https://docs.e6data.com/transformation/http/reference/delete-workload.md): delete workload. DELETE /api/v1/workloads/{workload_id}. Delete a workload from the saved workload list and block new runs. Stop streams and finish or cancel queued and active runs - [Get workload](https://docs.e6data.com/transformation/http/reference/get-workload.md): get workload. GET /api/v1/workloads/{workload_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Update workload](https://docs.e6data.com/transformation/http/reference/update-workload.md): update workload. PUT /api/v1/workloads/{workload_id}. Save SQL, batch Python and PySpark streaming tasks, dependencies, schedule and execution options. Python defaults to Batch; se - [List backfills](https://docs.e6data.com/transformation/http/reference/list-backfills.md): list backfills. GET /api/v1/workloads/{workload_id}/backfills. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create backfill](https://docs.e6data.com/transformation/http/reference/create-backfill.md): create backfill. POST /api/v1/workloads/{workload_id}/backfills. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Preview backfill](https://docs.e6data.com/transformation/http/reference/preview-backfill.md): preview backfill. POST /api/v1/workloads/{workload_id}/backfills/preview. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get backfill](https://docs.e6data.com/transformation/http/reference/get-backfill.md): get backfill. GET /api/v1/workloads/{workload_id}/backfills/{backfill_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Change backfill](https://docs.e6data.com/transformation/http/reference/change-backfill.md): change backfill. POST /api/v1/workloads/{workload_id}/backfills/{backfill_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Pause workload](https://docs.e6data.com/transformation/http/reference/pause-workload.md): pause workload. POST /api/v1/workloads/{workload_id}/pause. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Resume workload](https://docs.e6data.com/transformation/http/reference/resume-workload.md): resume workload. POST /api/v1/workloads/{workload_id}/resume. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Workload run summaries](https://docs.e6data.com/transformation/http/reference/workload-run-summaries.md): workload run summaries. GET /api/v1/workloads/{workload_id}/run-summaries. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List runs](https://docs.e6data.com/transformation/http/reference/list-runs.md): list runs. GET /api/v1/workloads/{workload_id}/runs. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Start run](https://docs.e6data.com/transformation/http/reference/start-run.md): start run. POST /api/v1/workloads/{workload_id}/runs. Submit a workload run; poll get_run and list_tasks. Reuse submission_id on retry. Assistant calls must include expected_revisi - [Get run](https://docs.e6data.com/transformation/http/reference/get-run.md): get run. GET /api/v1/workloads/{workload_id}/runs/{run_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Cancel run](https://docs.e6data.com/transformation/http/reference/cancel-run.md): cancel run. POST /api/v1/workloads/{workload_id}/runs/{run_id}/cancel. Request cancellation of this run without changing its schedule. Repeated calls are safe. Poll get_run until t - [Notification history](https://docs.e6data.com/transformation/http/reference/notification-history.md): notification history. GET /api/v1/workloads/{workload_id}/runs/{run_id}/notifications. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Retry notification](https://docs.e6data.com/transformation/http/reference/retry-notification.md): retry notification. POST /api/v1/workloads/{workload_id}/runs/{run_id}/notifications/retry. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Run profile](https://docs.e6data.com/transformation/http/reference/run-profile.md): run profile. GET /api/v1/workloads/{workload_id}/runs/{run_id}/profile. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Compare run profiles](https://docs.e6data.com/transformation/http/reference/compare-run-profiles.md): compare run profiles. GET /api/v1/workloads/{workload_id}/runs/{run_id}/profile/comparison. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Record profile evidence](https://docs.e6data.com/transformation/http/reference/record-profile-evidence.md): record profile evidence. PUT /api/v1/workloads/{workload_id}/runs/{run_id}/profile/evidence. Uses the same payload, validation, revision checks and response as the unified HTTP API - [Start rerun](https://docs.e6data.com/transformation/http/reference/start-rerun.md): start rerun. POST /api/v1/workloads/{workload_id}/runs/{run_id}/rerun. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Preview rerun](https://docs.e6data.com/transformation/http/reference/preview-rerun.md): preview rerun. POST /api/v1/workloads/{workload_id}/runs/{run_id}/rerun/preview. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List tasks](https://docs.e6data.com/transformation/http/reference/list-tasks.md): list tasks. GET /api/v1/workloads/{workload_id}/runs/{run_id}/tasks. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get task](https://docs.e6data.com/transformation/http/reference/get-task.md): get task. GET /api/v1/workloads/{workload_id}/runs/{run_id}/tasks/{task_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [List attempts](https://docs.e6data.com/transformation/http/reference/list-attempts.md): list attempts. GET /api/v1/workloads/{workload_id}/runs/{run_id}/tasks/{task_id}/attempts. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get attempt](https://docs.e6data.com/transformation/http/reference/get-attempt.md): get attempt. GET /api/v1/workloads/{workload_id}/runs/{run_id}/tasks/{task_id}/attempts/{attempt_id}. Uses the same payload, validation, revision checks and response as the unified - [Task profile](https://docs.e6data.com/transformation/http/reference/task-profile.md): task profile. GET /api/v1/workloads/{workload_id}/runs/{run_id}/tasks/{task_id}/profile. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Task results](https://docs.e6data.com/transformation/http/reference/task-results.md): Read a completed result page as base64 Arrow IPC. Query limit counts batches, not rows; the HTTP default is one batch. Use continuation_token for the next page. HTTP does not accep - [Task metadata](https://docs.e6data.com/transformation/http/reference/task-metadata.md): task metadata. GET /api/v1/workloads/{workload_id}/runs/{run_id}/tasks/{task_id}/results/metadata. Uses the same payload, validation, revision checks and response as the unified HT ##### Workspace documents - [List workspace documents](https://docs.e6data.com/transformation/http/reference/list-workspace-documents.md): list workspace documents. GET /api/v1/workspace/documents. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Create workspace document](https://docs.e6data.com/transformation/http/reference/create-workspace-document.md): create workspace document. POST /api/v1/workspace/documents. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Delete workspace document](https://docs.e6data.com/transformation/http/reference/delete-workspace-document.md): delete workspace document. DELETE /api/v1/workspace/documents/{document_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Get workspace document](https://docs.e6data.com/transformation/http/reference/get-workspace-document.md): get workspace document. GET /api/v1/workspace/documents/{document_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. - [Save workspace document](https://docs.e6data.com/transformation/http/reference/save-workspace-document.md): save workspace document. PUT /api/v1/workspace/documents/{document_id}. Uses the same payload, validation, revision checks and response as the unified HTTP API. ### Recipes - [Recipes](https://docs.e6data.com/transformation/recipes/overview.md): Choose the exact system and authentication method you need. Each recipe connects #### Catalog connections ##### Apache Polaris - [Connect to Apache Polaris using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/polaris/oauth.md): Register Apache Polaris in Unified and read an existing table with OAuth client - [Connect to Apache Polaris using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/polaris/bearer.md): Register Apache Polaris in Unified and read an existing table with a bearer - [Connect to Apache Polaris using OAuth and separate S3 credentials](https://docs.e6data.com/transformation/recipes/catalogs/polaris/separate-storage.md): Register Apache Polaris in Unified and read an existing table with OAuth and ##### Lakekeeper - [Connect to Lakekeeper using no authentication](https://docs.e6data.com/transformation/recipes/catalogs/lakekeeper/none.md): Register Lakekeeper in Unified and read an existing table with no - [Connect to Lakekeeper using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/lakekeeper/bearer.md): Register Lakekeeper in Unified and read an existing table with a bearer token. - [Connect to Lakekeeper using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/lakekeeper/oauth.md): Register Lakekeeper in Unified and read an existing table with OAuth client ##### Project Nessie - [Connect to Project Nessie using no authentication](https://docs.e6data.com/transformation/recipes/catalogs/nessie/none.md): Register Project Nessie in Unified and read an existing table with no - [Connect to Project Nessie using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/nessie/bearer.md): Register Project Nessie in Unified and read an existing table with a bearer - [Connect to Project Nessie using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/nessie/oauth.md): Register Project Nessie in Unified and read an existing table with OAuth client ##### AWS Glue native - [Connect to AWS Glue native using the runtime AWS identity](https://docs.e6data.com/transformation/recipes/catalogs/glue-native/runtime.md): Register AWS Glue native and read a table with the selected AWS identity. - [Connect to AWS Glue native using an assumed role](https://docs.e6data.com/transformation/recipes/catalogs/glue-native/assume-role.md): Register AWS Glue native and read a table with the selected AWS identity. - [Connect to AWS Glue native in another AWS account](https://docs.e6data.com/transformation/recipes/catalogs/glue-native/cross-account.md): Register AWS Glue native and read a table with the selected AWS identity. ##### AWS Glue Iceberg REST - [Connect to AWS Glue Iceberg REST using the runtime AWS identity](https://docs.e6data.com/transformation/recipes/catalogs/glue-rest/runtime.md): Register AWS Glue Iceberg REST and read a table with the selected AWS identity. - [Connect to AWS Glue Iceberg REST using an assumed role](https://docs.e6data.com/transformation/recipes/catalogs/glue-rest/assume-role.md): Register AWS Glue Iceberg REST and read a table with the selected AWS identity. ##### Amazon S3 Tables - [Connect to Amazon S3 Tables using the runtime AWS identity](https://docs.e6data.com/transformation/recipes/catalogs/s3-tables/runtime.md): Register Amazon S3 Tables and read a table with the selected AWS identity. - [Connect to Amazon S3 Tables using an assumed role](https://docs.e6data.com/transformation/recipes/catalogs/s3-tables/assume-role.md): Register Amazon S3 Tables and read a table with the selected AWS identity. ##### Unity Catalog native API - [Connect to the native Unity Catalog API using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/unity-native/bearer.md): Use the native Unity Catalog provider and a saved token to read an existing ##### Unity Catalog Iceberg REST - [Connect to Unity Catalog Iceberg REST using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/unity-rest/oauth.md): Register Unity Catalog Iceberg REST in Unified and read an existing table with - [Connect to Unity Catalog Iceberg REST using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/unity-rest/bearer.md): Register Unity Catalog Iceberg REST in Unified and read an existing table with a ##### Snowflake Open Catalog - [Connect to Snowflake Open Catalog using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/open-catalog/oauth.md): Register Snowflake Open Catalog in Unified and read an existing table with OAuth - [Connect to Snowflake Open Catalog using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/open-catalog/bearer.md): Register Snowflake Open Catalog in Unified and read an existing table with a ##### Snowflake Horizon - [Connect to Snowflake Horizon using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/horizon/bearer.md): Register Snowflake Horizon in Unified and read an existing table with a bearer - [Connect to Snowflake Horizon using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/horizon/oauth.md): Register Snowflake Horizon in Unified and read an existing table with OAuth ##### Google BigLake - [Connect to Google BigLake using application default credentials](https://docs.e6data.com/transformation/recipes/catalogs/biglake/application-default.md): Use a Google runtime identity to authenticate to the BigLake Iceberg REST API. ##### Microsoft OneLake - [Connect to Microsoft OneLake using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/onelake/bearer.md): Register Microsoft OneLake in Unified and read an existing table with a bearer - [Connect to Microsoft OneLake using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/onelake/oauth.md): Register Microsoft OneLake in Unified and read an existing table with OAuth ##### Other Iceberg REST services - [Connect to an Iceberg REST catalog using no authentication](https://docs.e6data.com/transformation/recipes/catalogs/iceberg-rest/none.md): Register an Iceberg REST catalog in Unified and read an existing table with no - [Connect to an Iceberg REST catalog using a bearer token](https://docs.e6data.com/transformation/recipes/catalogs/iceberg-rest/bearer.md): Register an Iceberg REST catalog in Unified and read an existing table with a - [Connect to an Iceberg REST catalog using OAuth client credentials](https://docs.e6data.com/transformation/recipes/catalogs/iceberg-rest/oauth.md): Register an Iceberg REST catalog in Unified and read an existing table with - [Connect to an Iceberg REST catalog using AWS request signing](https://docs.e6data.com/transformation/recipes/catalogs/iceberg-rest/sigv4.md): Register an Iceberg REST catalog in Unified and read an existing table with AWS #### File access - [Read S3 Parquet using an EC2 instance role](https://docs.e6data.com/transformation/recipes/files/ec2-instance-role.md): Create the selected AWS credential, bind it to a file view, and read Parquet - [Read S3 Parquet using an ECS task identity](https://docs.e6data.com/transformation/recipes/files/ecs-task-role.md): Create the selected AWS credential, bind it to a file view, and read Parquet - [Read S3 Parquet using a named AWS profile](https://docs.e6data.com/transformation/recipes/files/aws-profile.md): Create the selected AWS credential, bind it to a file view, and read Parquet - [Read S3 Parquet using an assumed role with an external ID](https://docs.e6data.com/transformation/recipes/files/assume-role-external-id.md): Create the selected AWS credential, bind it to a file view, and read Parquet - [Read S3 Parquet using temporary AWS keys](https://docs.e6data.com/transformation/recipes/files/temporary-aws-keys.md): Use an explicit saved AWS key set for one Parquet file source. - [Read MinIO Parquet using saved keys](https://docs.e6data.com/transformation/recipes/files/minio-parquet.md): Use an explicit saved AWS key set for one Parquet file source. - [Read CSV with a header and explicit column types](https://docs.e6data.com/transformation/recipes/files/csv-header.md): Register a temporary file view and query the selected input. - [Read TSV with an explicit schema](https://docs.e6data.com/transformation/recipes/files/tsv-schema.md): Register a temporary file view and query the selected input. - [Read JSON files from a configured location](https://docs.e6data.com/transformation/recipes/files/json-files.md): Register a temporary file view and query the selected input. - [Read matching CSV files from a folder](https://docs.e6data.com/transformation/recipes/files/folder-filter.md): Register a temporary file view and query the selected input. #### Cloud identities ##### AWS - [Read S3 using an AWS web identity token file](https://docs.e6data.com/transformation/recipes/identities/aws-web-identity.md): Exchange a runtime token file for an AWS role and read a Parquet dataset. ##### Google Cloud - [Connect to BigLake using a service-account document](https://docs.e6data.com/transformation/recipes/identities/gcp-service-account.md): Use this Google identity method for one BigLake catalog connection. - [Connect to BigLake using Google workload identity federation](https://docs.e6data.com/transformation/recipes/identities/gcp-workload-identity.md): Use this Google identity method for one BigLake catalog connection. - [Connect to BigLake by impersonating a service account](https://docs.e6data.com/transformation/recipes/identities/gcp-impersonation.md): Use this Google identity method for one BigLake catalog connection. ##### Azure - [Connect to OneLake using a managed identity](https://docs.e6data.com/transformation/recipes/identities/azure-managed-identity.md): Use this Azure identity to read an existing OneLake table. - [Connect to OneLake using a user-assigned managed identity](https://docs.e6data.com/transformation/recipes/identities/azure-user-assigned-identity.md): Use this Azure identity to read an existing OneLake table. - [Connect to OneLake using Azure workload identity](https://docs.e6data.com/transformation/recipes/identities/azure-workload-identity.md): Use this Azure identity to read an existing OneLake table. - [Connect to OneLake using an Azure client secret](https://docs.e6data.com/transformation/recipes/identities/azure-client-secret.md): Use this Azure identity to read an existing OneLake table. - [Connect to OneLake using an Azure client certificate](https://docs.e6data.com/transformation/recipes/identities/azure-client-certificate.md): Use this Azure identity to read an existing OneLake table. #### Secrets - [Use Google Secret Manager for a saved credential](https://docs.e6data.com/transformation/recipes/secrets/google-secret-manager.md): Create a named reference and use its value through a matching credential. - [Pin a Google Secret Manager secret version](https://docs.e6data.com/transformation/recipes/secrets/google-secret-manager-pinned.md): Create a named reference and use its value through a matching credential. - [Use Azure Key Vault for a saved credential](https://docs.e6data.com/transformation/recipes/secrets/azure-key-vault.md): Create a named reference and use its value through a matching credential. - [Pin an Azure Key Vault secret version](https://docs.e6data.com/transformation/recipes/secrets/azure-key-vault-pinned.md): Create a named reference and use its value through a matching credential. - [Use HashiCorp Vault KV v2 for a saved credential](https://docs.e6data.com/transformation/recipes/secrets/vault.md): Create a named reference and use its value through a matching credential. - [Pin a HashiCorp Vault KV v2 secret version](https://docs.e6data.com/transformation/recipes/secrets/vault-pinned.md): Create a named reference and use its value through a matching credential. - [Use an environment-secret mapping for a saved credential](https://docs.e6data.com/transformation/recipes/secrets/environment.md): Create a named reference and use its value through a matching credential. - [Use a file-secret mapping for a saved credential](https://docs.e6data.com/transformation/recipes/secrets/file.md): Create a named reference and use its value through a matching credential. - [Pin an AWS Secrets Manager secret version](https://docs.e6data.com/transformation/recipes/secrets/aws-secrets-manager-pinned.md): Create a new reference fixed to one existing remote secret version. - [Pin an AWS Parameter Store secret version](https://docs.e6data.com/transformation/recipes/secrets/aws-parameter-store-pinned.md): Create a new reference fixed to one existing remote secret version. - [Store a secret and use it](https://docs.e6data.com/transformation/recipes/secrets/store-and-use.md): Keep a bearer token in a named secret. A credential refers to that secret, and - [Use AWS Secrets Manager](https://docs.e6data.com/transformation/recipes/secrets/aws-secrets-manager.md): Use a local secret name for a client secret held in AWS. The local resource - [Use AWS Parameter Store](https://docs.e6data.com/transformation/recipes/secrets/aws-parameter-store.md): Use a local secret name for a client secret held in AWS. The local resource - [Pin an external secret version](https://docs.e6data.com/transformation/recipes/secrets/pin-version.md): Keep a credential on a chosen external value instead of following the current - [Rotate a secret](https://docs.e6data.com/transformation/recipes/secrets/rotate.md): Replace an OAuth client secret while keeping the credential and catalog names - [Move a local secret to an external provider](https://docs.e6data.com/transformation/recipes/secrets/move-to-external.md): Keep a local name that consumers already use, but fetch its value from AWS - [Remove access resources after use](https://docs.e6data.com/transformation/recipes/secrets/cleanup.md): Remove a connection before removing the credential and secret that it uses. #### Credentials - [Read S3 using saved AWS keys](https://docs.e6data.com/transformation/recipes/credentials/aws-saved-keys.md): Create a named credential from secret references, then bind that credential to - [Use the AWS credential chain or a profile](https://docs.e6data.com/transformation/recipes/credentials/aws-runtime.md): Use an identity already available on the engine host. SQL saves the choice of - [Read S3 using EKS Pod Identity](https://docs.e6data.com/transformation/recipes/credentials/eks-pod-identity.md): Read an S3 prefix using a named runtime credential, without saving access keys. - [Read S3 using IRSA](https://docs.e6data.com/transformation/recipes/credentials/eks-irsa.md): Read an S3 prefix using a named runtime credential, without saving access keys. - [Assume an AWS role](https://docs.e6data.com/transformation/recipes/credentials/assume-role.md): Use one named AWS identity to obtain temporary credentials for another IAM role. - [Use a bearer token](https://docs.e6data.com/transformation/recipes/credentials/bearer-token.md): Send an existing access token to an Iceberg REST catalog. A bearer credential - [Use OAuth client credentials](https://docs.e6data.com/transformation/recipes/credentials/oauth-client-credentials.md): Give the engine a client ID and a client-secret reference. It exchanges them for - [Separate catalog and storage credentials](https://docs.e6data.com/transformation/recipes/credentials/separate-catalog-storage.md): Log in to Polaris with OAuth, then read its S3 table files using an assumed AWS - [Use multiple AWS identities](https://docs.e6data.com/transformation/recipes/credentials/multiple-aws-identities.md): Read two S3 prefixes in one SQL session, using a different named AWS role for - [Switch from saved keys to runtime identity](https://docs.e6data.com/transformation/recipes/credentials/switch-to-runtime.md): Change how a credential obtains its AWS identity while keeping the name already ### Operations - [Observability](https://docs.e6data.com/transformation/operations/observability.md): Open Monitoring → Observability to inspect retained metrics. The page reads recent metrics from controller memory and loads older samples from saved history when needed. It does no #### Configuration - [Configuration](https://docs.e6data.com/transformation/operations/configuration/index.md): The configuration file tells Unified which services to start, where to store data, and how to run work. Laminar reads this TOML file and checks its values before starting those ser - [Storage](https://docs.e6data.com/transformation/operations/configuration/storage.md): Unified uses one storage_root for its durable state. Use an S3 URL for remote workers, or an absolute local directory for a local deployment. - [Networking and security](https://docs.e6data.com/transformation/operations/configuration/networking-security.md): Separate client access from internal service and worker traffic. Every advertised address must be reachable from the process that uses it. A listener can bind all interfaces while - [Resources and capacity](https://docs.e6data.com/transformation/operations/configuration/resources.md): Size the services and task workers separately. Services keep planning and durable execution state. Workers need memory for source buffers, the Native Engine, result buffers, and pr ##### Services and features - [Mode and network](https://docs.e6data.com/transformation/operations/configuration/deployment.md): network - [Availability](https://docs.e6data.com/transformation/operations/configuration/availability.md): availability - [Assistant](https://docs.e6data.com/transformation/operations/configuration/agent.md): agent - [Feature flags](https://docs.e6data.com/transformation/operations/configuration/feature-flags.md): feature_flags - [Workspace](https://docs.e6data.com/transformation/operations/configuration/workspace.md): workspace - [Dashboards](https://docs.e6data.com/transformation/operations/configuration/dashboards.md): dashboards - [UI files](https://docs.e6data.com/transformation/operations/configuration/ui-path.md): ui_path ##### Storage and sources - [Database](https://docs.e6data.com/transformation/operations/configuration/database.md): database - [Storage root](https://docs.e6data.com/transformation/operations/configuration/storage-root.md): storage_root - [Table storage](https://docs.e6data.com/transformation/operations/configuration/table-storage.md): table_storage - [Result storage](https://docs.e6data.com/transformation/operations/configuration/result-storage.md): result_storage - [Result object size](https://docs.e6data.com/transformation/operations/configuration/max-result-object-bytes.md): max_result_object_bytes - [Iceberg](https://docs.e6data.com/transformation/operations/configuration/iceberg.md): iceberg - [File sources](https://docs.e6data.com/transformation/operations/configuration/file-sources.md): file_sources - [File source limits](https://docs.e6data.com/transformation/operations/configuration/file-source-limits.md): file_source_limits - [Delta source limits](https://docs.e6data.com/transformation/operations/configuration/delta-source-limits.md): delta_source_limits - [Kafka connections](https://docs.e6data.com/transformation/operations/configuration/kafka-connections.md): kafka_connections - [Credentials](https://docs.e6data.com/transformation/operations/configuration/credentials.md): credentials - [Secrets](https://docs.e6data.com/transformation/operations/configuration/secrets.md): secrets - [Catalog client](https://docs.e6data.com/transformation/operations/configuration/catalog-client.md): catalog_client ##### Workers and execution - [Workers](https://docs.e6data.com/transformation/operations/configuration/compute.md): workers - [Worker listener](https://docs.e6data.com/transformation/operations/configuration/worker-listen-address.md): network.workers - [Worker connections](https://docs.e6data.com/transformation/operations/configuration/worker-connection.md): workers.runtime.connection - [Submission defaults](https://docs.e6data.com/transformation/operations/configuration/submission-defaults.md): submission_defaults - [Controller](https://docs.e6data.com/transformation/operations/configuration/controller.md): controller - [Scheduler](https://docs.e6data.com/transformation/operations/configuration/scheduler.md): scheduler - [Committer](https://docs.e6data.com/transformation/operations/configuration/committer.md): committer - [Provisioner batch size](https://docs.e6data.com/transformation/operations/configuration/provisioner-batch-size.md): workers.batch_size - [Gateway](https://docs.e6data.com/transformation/operations/configuration/gateway.md): gateway - [Executor](https://docs.e6data.com/transformation/operations/configuration/executor.md): workers.runtime.executor - [Native Engine](https://docs.e6data.com/transformation/operations/configuration/duckdb.md): workers.runtime.duckdb - [Table maintenance](https://docs.e6data.com/transformation/operations/configuration/maintenance.md): maintenance - [Runtime](https://docs.e6data.com/transformation/operations/configuration/runtime.md): runtime - [Internal transport](https://docs.e6data.com/transformation/operations/configuration/transport.md): transport ##### Clients and notebooks - [HTTP API](https://docs.e6data.com/transformation/operations/configuration/http-api.md): http_api - [Spark Connect](https://docs.e6data.com/transformation/operations/configuration/spark-connect.md): spark_connect - [SQL sessions](https://docs.e6data.com/transformation/operations/configuration/session.md): session - [Driver limits](https://docs.e6data.com/transformation/operations/configuration/driver-limits.md): driver_limits - [Result read limits](https://docs.e6data.com/transformation/operations/configuration/result-read-limits.md): result_read_limits - [Notebooks](https://docs.e6data.com/transformation/operations/configuration/notebooks.md): notebooks ##### Telemetry - [Telemetry](https://docs.e6data.com/transformation/operations/configuration/telemetry.md): Configure Unified telemetry under telemetry. Workers report logs, metrics, #### More guides - [Operations](https://docs.e6data.com/transformation/operations/index.md): Use Operations to inspect engine activity and configure Unified services. ## OpenAPI Specs - [transformation](/openapi/transformation.json)