Tables and sources

ReferenceApplies to: Unified engine deployments

A catalog supplies object metadata. A source connector reads rows and a sink connector publishes writes. Their supported types, write modes, and maintenance operations can differ.

Table names

table, namespace.table, and catalog.namespace.table use increasingly explicit context. Unqualified names use the session’s current catalog and namespace. Examples use readable names such as demo.tpcds.item. Replace these names with your catalog and namespace.

SELECT i_item_sk, i_category FROM demo.tpcds.item ORDER BY i_item_sk LIMIT 5;

Create and populate a table

Prepare the write lab first. This script recreates only its item_sample table, copies five source rows, and reads the result. Running it again replaces that small copy.

SET spark.sql.caseSensitive = true;
DROP TABLE IF EXISTS demo.examples.item_sample;
CREATE TABLE demo.examples.item_sample (i_item_sk BIGINT, i_item_id STRING, i_category STRING) USING iceberg;
INSERT INTO demo.examples.item_sample
SELECT i_item_sk, i_item_id, i_category
FROM demo.tpcds.item WHERE i_item_sk <= 5;
SELECT * FROM demo.examples.item_sample ORDER BY i_item_sk;

The example separates the schema definition from the insert. CREATE TABLE AS SELECT (CTAS) also needs the catalog to support staged creation and publication. The Iceberg REST connector requests this support from the configured server. CLUSTERED BY is rejected; supported bucket partition transforms use a different grammar. New Iceberg tables default to format v3. Add TBLPROPERTIES ('format-version' = '2') to CREATE TABLE or CTAS when native maintenance is needed. For an existing table, ALTER TABLE name SET TBLPROPERTIES ('format-version' = '3') requests a format upgrade. Format versions cannot be downgraded or unset. Native table maintenance is enabled only for v2 tables. Reads and writes support v3.

Explicit schemas

This script recreates a small empty table before inserting two rows. SQL query types and sink types can differ: a type accepted in SELECT may still be rejected by an Iceberg writer.

SET spark.sql.caseSensitive = true;
DROP TABLE IF EXISTS demo.examples.notes;
CREATE TABLE demo.examples.notes (note_id BIGINT, note_text STRING) USING iceberg;
INSERT INTO demo.examples.notes VALUES (1, 'First note'), (2, 'Second note');
SELECT * FROM demo.examples.notes ORDER BY note_id;

Alter a table

For setting, inspecting, changing, and removing saved properties, see the dedicated TBLPROPERTIES page. This example recreates a lab table, adds a nullable column, and changes a table property. Catalogs can accept a narrower set of schema changes than the SQL grammar.

SET spark.sql.caseSensitive = true;
DROP TABLE IF EXISTS demo.examples.schema_demo;
CREATE TABLE demo.examples.schema_demo (item_key BIGINT) USING iceberg;
ALTER TABLE demo.examples.schema_demo ADD COLUMNS (category STRING);
ALTER TABLE demo.examples.schema_demo SET TBLPROPERTIES ('docs.topic' = 'schema');
DESCRIBE TABLE demo.examples.schema_demo;

Temporary views

A local temporary view belongs to its session. The following view reads the original SF1 item table. Use the same session and temporary view for the next example.

CREATE OR REPLACE TEMPORARY VIEW docs_items AS
SELECT i_item_sk, i_item_id, i_category FROM demo.tpcds.item WHERE i_item_sk <= 10;
SELECT * FROM docs_items ORDER BY i_item_sk;

Query-backed persistent and global temporary views have separate catalog and scope requirements. Do not assume that a temporary view is visible in another session.

Drop a table

DROP removes the named table according to the catalog’s rules. IF EXISTS tolerates an absent name. This complete example creates and drops only a lab table.

SET spark.sql.caseSensitive = true;
CREATE TABLE IF NOT EXISTS demo.examples.drop_demo (item_key BIGINT) USING iceberg;
DROP TABLE IF EXISTS demo.examples.drop_demo;
SELECT 'Lab table removed' AS status;

File sources

CSV, TSV, JSON, and Parquet can be read through a local temporary file view with USING and an OPTIONS path. A typed column list is optional where schema inference is supported. CSV options include header and delimiter; folder filtering can use pathGlobFilter. Supported local and object-store locations require configured storage access.

Element	Meaning
`CREATE OR REPLACE TEMPORARY VIEW name`	Session-local file view
`USING csv`, `tsv`, `json`, or `parquet`	File format
`OPTIONS (path '…')`	Actual accessible location
`header 'true'`, `delimiter ','`	CSV parsing options
`pathGlobFilter '*.csv'`	File selection filter

File views must be local temporary views; persistent/global temporary file views and IF NOT EXISTS on this form are rejected. Folder reads are bounded. The live examples use catalog-backed SF1 tables so every input already exists.

CSV options

Use USING csv with an OPTIONS path. Set header 'true' when the first row contains column names. Use delimiter to choose the field separator and pathGlobFilter '*.csv' to select files. An explicit column list keeps the types stable across input files.

Parquet options

Use USING parquet with an OPTIONS path to read Parquet files. A pathGlobFilter '*.parquet' filter selects matching files in a folder. These are read options. See Iceberg write options for compression, file sizes, and other settings used when writing Parquet.

Writes and publication

A write needs the corresponding connector capability. Row-level mutations also need target row identity and an allowed complete target read. Prepared data or delete files become visible only after the committer and catalog confirm publication. Inspect the final operation and confirmed write outcome. A read can observe a catalog snapshot newer than an earlier read. A script is a sequence of operations, not a multi-statement transaction. See Writing data.

Transformation Engine

Learn Laminar

Workloads

SQL reference

HTTP API

Recipes

Operations

Table names

Create and populate a table

Explicit schemas

Alter a table

Temporary views

Drop a table

File sources

CSV options

Parquet options

Writes and publication

​Table names

​Create and populate a table

​Explicit schemas

​Alter a table

​Temporary views

​Drop a table

​File sources

​CSV options

​Parquet options

​Writes and publication

Table names

Create and populate a table

Explicit schemas

Alter a table

Temporary views

Drop a table

File sources

CSV options

Parquet options

Writes and publication