> ## Documentation Index
> Fetch the complete documentation index at: https://docs.e6data.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Limitations

> What the e6data PySpark compatibility layer does not support, and how unsupported operations behave.

The `e6-spark-compat` library covers the PySpark DataFrame API and a large set of SQL functions, but it isn't a complete drop-in for every Spark feature. Know these boundaries before migrating a workload. For what *is* supported, see [Supported APIs and compatibility notes](/query/pyspark/supported-apis).

## Read-oriented engine

e6data is a query engine for reading data. The compatibility layer translates DataFrame operations into SQL that runs on e6data - it is not a general-purpose Spark runtime.

* **No RDD API.** Only the DataFrame and SQL APIs are supported, not low-level RDD operations.
* **No UDFs / arbitrary Python execution.** Operations must translate to SQL; arbitrary Python or JVM UDFs that run inside Spark executors aren't supported.
* **No Spark-managed cluster or streaming.** There's no `SparkContext`-style cluster management or Structured Streaming; the session connects to an e6data cluster.

## Partial API coverage

Some APIs are supported only in part:

* **Window functions and complex types** are fully available in standard execution but only a **subset** runs on the native execution path.
* **Geospatial functions** are available via the Sedona-compatible layer in standard execution but not on the native path.
* **Cache hints** (`cache()`, `persist()`, `coalesce()`) are accepted as pass-through hints rather than Spark-style materialization.

When a query uses a feature the native path doesn't support, e6data **falls back to the standard path automatically** for that query - no error, no missed result.

## Write operations

Write support depends on the target catalog and table format; e6data is read-only for some catalog types. Confirm write support for your catalog before relying on it.

## See also

* [PySpark compatibility](/query/pyspark)
* [Supported APIs and compatibility notes](/query/pyspark/supported-apis)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.