Release notes

Feature Previews & Documentation Updates

• Unified query interface across AWS, Azure, GCP, and on-prem systems
• Hybrid cluster setup with Main + Ancillary Clusters
• Federated SQL engine ensures compliance and reduces data movement
• Support for both centralized and federated metastores

• Required permissions
• IAM roles
• Trust configuration steps

Improved Integrations with Polaris, Iceberg, Hudi, and Delta

• Table Formats: Apache Iceberg, Apache Hudi, and Delta Lake
• Catalog Systems: Apache Polaris

• You can now create, edit, and delete Polaris catalog connections directly in the UI.
• Full support for exploring Iceberg tables registered under Polaris, including schema discovery and partitioned queries.
• Improved metadata synchronization and catalog validation during connection setup.

• Iceberg tables can now be accessed seamlessly through supported external catalogs such as AWS Glue, Polaris, and Snowflake Open Catalog.
• Support for both partitioned and non-partitioned Iceberg tables.
• Enhanced handling of schema evolution and metadata-aware querying.
• Improved catalog management UI for editing and managing Iceberg integrations.

• Added support for reading Delta Lake tables through catalogs like AWS Glue and Unity Catalog.
• Metadata visibility improvements for schema and file stats.
• Cleaner user interface for catalog selection and validation.
• Table browsing and querying experience aligned with other table formats.

• Access Hudi tables via supported external catalogs: AWS Glue and Apache Hive
• Support for Copy-on-Write (CoW) table types
• Query modes: Read-Optimized, Incremental, and Real-Time
• Supports time travel, schema evolution, and upserts/inserts/deletes
• Access to commit metadata for incremental processing
• Improved integration with external metastores for smoother table discovery and querying.

• Enhanced Data Analyst role

​

Enhanced Data Analyst role

Overview

Key Features of the enhanced “Data Analyst” Role:

• Programmatic Access:
  Users with the Data Analyst role can execute queries programmatically, allowing them to integrate query execution into automated processes, scripts, or APIs for streamlined data analysis and reporting.
• Query Editor Export:
  Users are granted access to the query editor and can download query results. This functionality allows data analysts to run queries and export the resulting data for offline analysis or further use, ensuring flexible access to data.
• Query Editor View Only:
  Users can execute queries using the query editor but are restricted from downloading query results. This is ideal for scenarios where data needs to be analyzed on-platform without the risk of sensitive data being exported.

• Multi-Factor Authentication (MFA)
• Service Accounts

​

Multi-Factor Authentication (MFA)

Overview:

Key Features:

• Enhanced Security: Reduces the risk of unauthorized access by requiring multiple forms of authentication.

​

Service Accounts

Overview:

Key Features:

• Programmatic Access: Allows scripts, applications, or automated processes to interact with the platform securely.
• RBAC Integration: Assign specific roles to control the scope of actions a service account can perform.
• Simplified Management: Eliminate the need to manage individual user invitations and access rights for each team member.
• Persistent Access: Maintain continuous access regardless of team member changes.

• Improved Connectivity and Security Enhancements
• Streamlined Query Editing Experience: Enhanced Handling of Large SQL Queries
• Enhanced Query History Management: Faster Downloads and Improved Filtering

​

Improved Connectivity and Security Enhancements

• Reduced IAM Permissions: We’ve streamlined the IAM permissions required for users in their workspace, specifically targeting connectivity resources. For more information, please refer to this documentation.
• Enhanced ACM Certificate Validation: We’ve bolstered the validation process for ACM (Amazon Certificate Manager) certificates during TLS (Transport Layer Security) and gateway connectivity. This enhancement ensures stronger encryption and heightened security for your connections.

​

Streamlined Query Editing Experience: Enhanced Handling of Large SQL Queries

• Formatting Support: Users can now format their SQL queries within a specified limit directly in the query editor. This feature enhances readability and organization, even for extensive query texts.
• Non-disruptive Formatting: Formatting large queries may require some time, but worry not! We’ve implemented a progress bar to keep you informed. Plus, the page remains responsive throughout the process, ensuring uninterrupted editing without any hanging or blocking.
• Supportive Messaging: If a query exceeds the formatting limit, users will receive a message indicating that formatting is not possible due to the query’s size. However, this does not disrupt your querying experience; you can continue editing your SQL query seamlessly.

​

Enhanced Query History Management: Faster Downloads and Improved Filtering

• Faster Download Times:
  • We’ve optimized the download process for query history, especially when dealing with a large number of queries. Now, you’ll experience reduced download times, ensuring quicker access to your query details.
  • To keep you informed about the download progress, we’ve added a progress bar. This feature displays the percentage of completion, allowing you to track the download status effortlessly.
• Enhanced Filter for Status:
  • We’ve revamped the filter for query status, making it more user-friendly. Instead of typing and searching, you can now select the status directly from a dropdown menu. This improvement simplifies the filtering process, enhancing your overall user experience.

• Understanding Schema Behavior in Schema Explorer

​

Understanding Schema Behavior in Schema Explorer

• Catalog Privileges Supporting Column Masking and Row Filter
  • Column Masking
  • Row Filtering
• Query History Query Count API
• Support for Deletion Vector in Iceberg Tables
• New Functions Support

​

Catalog Privileges Supporting Column Masking and Row Filter

Column Masking

Row Filtering

​

Query History Query Count API

​

Support for Deletion Vector in Iceberg Tables

Limitations:

• Applying filters on columns from which data has been deleted in complex data types is not currently supported.

​

New Functions Support

• STRPOS: This function enables you to find the position of a substring within a string. It returns the position of the first occurrence of the substring within the string. If the substring is not found, it returns zero.
• LENGTH: The LENGTH function allows you to determine the length of a string.
• STARTSWITH: The STARTSWITH function simplifies substring search operations by returning TRUE if the first expression starts with the second expression.

• Enhanced Query History Export CSV Functionality

​

Enhanced Query History Export CSV Functionality

1. Introducing the DataExport Role: Enhanced Query Result Download Capability
2. Enhanced Resource Selection Dropdown in Catalog Privileges
3. Introducing Client Perceived Time in Query History

​

Introducing the DataExport Role: Enhanced Query Result Download Capability

​

Enhanced Resource Selection Dropdown in Catalog Privileges

​

Introducing Client Perceived Time in Query History

1. Enhanced Cluster Connection Info and Connectivity
2. Catalog refresh allowed for Data Analyst roles

​

Enhanced Cluster Connection Info and Connectivity

• Reforming connectivity types: In the latest update, the process for creating connections in the cluster tab has been relocated to endpoints and renamed Direct Connectivity. Additionally, the previous Direct Connectivity option under endpoints has been rebranded as TLS Connectivity. Notably, Direct Connectivity does not require DNS mapping. Please refer to the documentation for more information. We recommend using direct connectivity exclusively during Proof of Concept (POC) mode.
• Restructuring Association of clusters and IP sets: This restructuring enables users to create/edit endpoints and associate clusters and IP Sets with them seamlessly in one place.
• Revamping the Cluster Connection Page: The page has been redesigned to streamline the collection of connection information according to the chosen connectivity type. Please refer to the documentation for more information.

​

Catalog refresh allowed for Data Analyst roles

1. Catalog Privileges
2. Support for Liquid Clustering
3. Support for Unnest Operator for Primitive Types
4. Impersonation Feature for Metabase BI Tool only via Apache Ranger
5. Supporting New Functions

​

Catalog Privileges

Key Features:

• Granular Access Control: Define access privileges at the catalog, schema, table, and column levels, precisely controlling who can access what data.
• User-Friendly Interface: Easily manage access privileges directly through the intuitive interface of e6.
• Transparent Query History: Gain insight into denied access attempts through the transparent query history feature.
• Future Enhancements: Anticipate upcoming enhancements, including support for row filtering and column masking.
• Role-Based Access: Empower users with the Data Admin role to create and manage access privileges, ensuring centralized control and oversight.
• Automatic Access Provisioning: Experience seamless access provisioning as any new user added to the workspace automatically inherits access to all data assets.

Limitations:

• Column masking and row filtering functionalities are scheduled for future releases and unavailable in the current version.
• Changes to access privileges may take 60 seconds to become effective after saving.

​

Support for Liquid Clustering

​

Support for Unnest Operator for Primitive Types

​

Impersonation Feature for Metabase BI Tool only via Apache Ranger

• A new parameter has been added which identifies the session user in the Metabase BI tool.
• Enables clear attribution of queries to the correct user, aiding data governance and security efforts.
• To utilize this feature, users must employ the latest engine alongside the e6data Clojure driver. (Please contact e6data support for Clojure driver)

​

Supporting New Function

• ENDSWITH - Returns TRUE if the first expression ends with second expression, simplifying substring search operations.
• ASCII - Return the ASCII code for the first character of a given string.
• MD5 - Returns a 32-character hex-encoded string.
• REPEAT - Generate a string by duplicating the input by number of repetitions specified.
• SIGN - The SIGN (SIGNUM) function returns the signum value of its argument.
• TO_TIMESTAMP_NTZ - Returns timestamp with UTC timezone (timestamp with no time zone).

​

Gateway Connectivity

• Connectivity
• Notebook

Connectivity

1. Persistent connectivity: Even if you disable and then re-enable the workspace, the connection information remains constant, ensuring uninterrupted connectivity between clusters and BI tools.
2. Client-based connection information: Once connectivity is enabled for your clusters, you can retrieve the connection information for the supported clients. This connection information is required for external connectivity with the clients.

• Update Cross-Account IAM permissions
• ALB controller helm chart upgrade
• e6data helm chart upgrade

Notebook (Paragraph based Editor)

Query Editor

Multi Result Tab

1. Efficient Result Navigation: Access results for different queries seamlessly through dedicated tabs in the left navigation panel of the result pane.
2. User-Friendly Interface: Simply click on the desired tab to view and download the results for a specific query.

Copy and View Result Cell Data

Error Handling during query failure

Catalog

Catalog Refresh at the schema level

Enhanced error handling: download log file

Retry option during catalog failure

Catalog operations speed up through optimization

Cluster

Optimizing Workload Management with Load-Based Sizing

Cluster Monitoring

Ability to add new users/tokens without needing a cluster restart

Schema explorer

Schema Reload Option

Query Editor result pane

Export Result CSV

Access Control

Query History View Role

Query optimizations & improvements

• In-memory caching optimizations
  While explicitly caching data in executors, we have seen a few issues faced by users. We have now revamped our explicit caching methodology to better utilize executor memory. This new methodology not only ensures that there are lesser failures during caching process but also improves performance of the queries run on the cached data.
• Pruning of non-partition columns and optimizations
  We have improved our pruning ability with the refactoring of our pruning methodology. We have also introduced the ability to prune non-partition columns which will help in faster data skipping. Thus, resulting in faster query execution times.
• Introducing CTE reuse
  When multiple CTEs are present in a query, the planner now optimizes their execution, promoting reuse across the query. It reduces redundant computation, leading to quicker query execution times.
• We have fixed a series of bugs to reduce query failures during heavy load on systems. Major bug fixes as stated below
  • When running a large set of concurrent queries (>100QPS), queries used to fail due to socket timeout issues. There was a bug in our communication layer between internal components that was causing this issue. It has now been fixed
  • When running a large set of concurrent queries (>100QPS), queries would fail with a Planner IP mismatch error. There was a bug with how we were mapping the planner to query requests via our network router. This is now fixed
  • We have identified and fixed a few more bugs that were causing query failures

New SQL Functions support

• GENERATE_TIMESTAMP_ARRAY:
  • Generates an array of timestamps based on specified parameters, allowing you to easily create time series data for analysis.
• GENERATE_DATE_ARRAY:
  • Similar to GENERATE_TIMESTAMP_ARRAY, this function generates an array of dates, simplifying date-related operations in your queries.
• REGEXP_COUNT:
  • Counts the number of occurrences of a specified regular expression pattern within a given string, enhancing your ability to perform complex string manipulations.
• REGEXP_LIKE:
  • Enables pattern matching using regular expressions within your queries, providing a more versatile way to filter and analyze textual data.
• EXCEPT:
  • Introduces the EXCEPT statement, allowing you to specify the names of one or more columns to exclude from the result. All matching column names are omitted from the output. This operation is particularly useful for tailoring your query results by excluding specific columns.
• REGEXP_EXTRACT_ALL
  • Extracts all occurrences of a regular expression pattern from a string, providing a comprehensive view of all matching substrings.
• REGEXP_CONTAINS
  • Checks if a string contains a specified regular expression pattern, offering a flexible and efficient way to identify pattern presence.
• SUBSCRIPT OPERATOR - alias offset/safe offset:
  • Retrieves an element from an array at a specified index or a default value if the index is out of bounds, ensuring safe and controlled access to array elements.
• ELEMENT_AT
  • Retrieves an element from an array at a specified index or a default value if the index is out of bounds, providing a convenient way to access array elements.
• SIZE
  • Returns the number of elements in an array, offering a quick and straightforward way to determine the size of arrays.
• LOCATE
  • Returns the position of the first occurrence of a substring in a string, simplifying substring search operations.
• CONTAINS_SUBSTR
  • Checks if a string contains a specified substring, providing a straightforward way to determine the presence of substrings in a string.
• INSTR
  • Returns the position of the first occurrence of a substring in a string, enhancing substring search capabilities.
• SOUNDEX
  • Computes a phonetic representation of a string, enabling fuzzy string matching based on similar sounds.
• PARSE_DATE
  • Parses a string into a date, ensuring accurate conversion of date-related information in your queries.
• PARSE_DATETIME
  • Parses a string into a datetime, facilitating precise conversion of date and time information in your queries.
• PARSE_TIMESTAMP
  • Parses a string into a timestamp, providing a reliable way to convert string representations into timestamp data for analysis.

Pip Package versions

JDBC Driver versions

• Platform Enhancement
  • Catalog Auto Refresh

​

Platform Enhancement

Catalog Auto Refresh

• Backend Enhancements
  • Security of Monitoring Components
• Platform Enhancements
  • Enhanced Access Control

​

Backend Enhancements

Security of Monitoring Components

​

Platform Enhancements

Enhanced Access Control

• Platform Enhancements
  • Find and Replace Feature in Query Editor

​

Platform Enhancements

Find and Replace Feature in Query Editor

• RegExp Search
• Case-Sensitive Search
• Whole Word Search
• Search in Selection.

• Plugin Enhancements
  • Performance Improvements in the e6data Python Connector
• Platform Enhancements
  • Performance Improvements in the Query Editor
• Engine Enhancements
  • Added Support for Deletion Vectors
  • Introduced DROP CACHE Command
  • New SQL Functions Supported

​

Plugin Enhancements

Performance Improvements in the e6data Python Connector

​

Platform Enhancements

Performance Improvements in the Query Editor

​

Engine Enhancements

Added Support for Deletion Vectors

Limitations

• Querying deleted rows with an IN clause may result in an error.
  • Example: SELECT * FROM "table_with_del_vector" WHERE id IN ('ABCXYZ');
• While using deletion vectors performance testing was conducted for up to 10 million deleted records. For a higher number of deleted records, performance may degrade from a few milliseconds up to 1 second. Users may experience slower query execution times when utilizing deletion vectors. Further optimizations are in the pipeline to resolve this.

Introduced DROP CACHE Command

Syntax

Limitations

• Only full/complete table cache drops are currently supported.
• In certain scenarios, the drop cache operation may not be successful. Repeated execution of DROP CACHE can lead to a state where subsequent cache commands fail. A cluster restart may be required to mitigate this situation.
• The current syntax does not gracefully handle scenarios where the DROP CACHE command is issued on a table that doesn’t exist. The error message incorrectly states “unsupported syntax,” which may be misleading. Future updates will enhance error messaging for non-existent tables in the DROP CACHE command.

New SQL Functions Supported

• WEEK: Extract the week component from a given date or timestamp.
• CONTAINS_SUBSTR: Checks if a string contains a specified substring and returns a boolean value.
• INSTR: Returns the position of the first occurrence of a substring within a string.
• LEFT: Returns a specified number of characters from the beginning (left) of a string.
• REGEXP_CONTAINS: Checks if a string matches a regular expression pattern and returns a boolean value.
• REGEXP_REPLACE: Replaces substrings matching a regular expression pattern with a specified replacement string.
• SOUNDEX: Returns a phonetic representation (SOUNDEX code) of a string, which can be used to identify similar-sounding words.

• Platform Enhancements
  • Improvements to Query Editor session handling

​

Platform Enhancements

Improvements to Query Editor session handling

• UI Enhancements

​

Usage Notes

​

UI Enhancements

Query Editor

Component Creation

• Changes to User Roles and Privileges
• Refresh Catalog

​

Usage Notes

​

New Features

Changes to User Roles and Privileges

• WorkspaceManager - Provides privileges to enable, disable, create, and update assigned workspaces; and to view catalogs & clusters in assigned workspaces.

Refresh Catalog

• Enable & Disable Workspaces
• UI Enhancements

​

Usage Notes

​

Enable/Disable Workspaces

1. Disabling a Workspace terminates all resources in the workspace node group/node pool to save costs.
2. Enabling a Workspace spins up the required resources again.

​

UI Enhancements

• The Schema Explorer in the Query Editor can now be expanded or collapsed to allow users to optimize screen real estate.
• The refresh button in the Cluster Metrics page has been improved, allowing users to set an auto-refresh interval for the dashboard.

• Adding Pod Metrics to Cluster Metrics
• Ability to Update Workspaces
• Adding option to edit Query History View
• UI Enhancements

​

Usage Notes

​

Adding Pod Metrics to Cluster Metrics

​

Ability to Update Workspaces

​

Adding option to edit Query History View

​

UI Enhancements

• Query History
• Cluster External Connectivity

• Separate Query Editor Viewer Role
• Query Resume in Query Editor
• Customizing Query Editor View
• Adding the Ability to Edit Catalogs
• Adding Ability to Provision External Access to Clusters

​

Usage Notes

​

Separate Query Editor Viewer Role

​

Query Resume in Query Editor

​

Customizing Query History View

​

Adding the Ability to Edit Catalogs

​

Adding Ability to Provision External Access to Clusters

• Platform Release Notes
  • Query Editor Enhancements
    • Schema Search
    • Enhanced Query Run Button
    • Visual Enhancements
  • UI Enhancements

​

Usage Notes

​

Platform Release Notes

Query Editor Enhancements

Schema Search

Enhanced Query Run Button

Visual Enhancements

UI Enhancements

• The top navigation bar has been restructured across the e6data console.
• Hovering over the left navigation sidebar will expand and collapse it.

• Platform Release Notes
  • WorkspaceAdmin Role
  • Multi-Catalog Support
• Engine Release Notes
  • SQL Optimizations
• Bug Fixes
• Known Limitations

​

Usage Notes

• Suspend and resume active Clusters

​

Platform Release Notes

WorkspaceAdmin Role

Multi-Catalog Support

​

Engine Release Notes

SQL Optimizations

Support for the collect_list Function

• The collect_list function enables you to create an ordered list of values from a specified column within a group or window.
• It is particularly useful when you need to gather and process multiple values associated with a particular grouping key or window frame.

Cross-Schema & Cross-Catalog Querying

• This feature lets you combine and analyze data from different schemas and catalogs, providing a comprehensive view of your data landscape.
• To perform cross-schema queries, you need to fully qualify the path by providing the catalog name, schema name, and table name.
  • Hive Catalog:
    <catalog_name>.<db_name>.<table_name>
catalog_name - name of catalog specified during catalog creation in e6data
  • Glue Catalog:
    <catalog_name>.<db_name>.<table_name>
catalog_name - name of catalog specified during catalog creation in e6data
  • Unity Metastore:
    <unity_catalog_name>.<db_name>.<table_name>
unity_catalog_name - name of catalog which is created under Unity in Databricks

​

Bug Fixes & Improvements

• You can now seamlessly query and analyze data stored in multi-level partitioned columns.
• We utilize the partitioning information to optimize query execution plans. This optimization ensures that queries involving multi-level partitioned columns are processed efficiently, resulting in improved performance.

​

Known Limitations

1. Cross-schema & cross-catalog querying have a few limitations, as listed below:
   1. Slight Increase in Parsing Time: Due to the additional complexity introduced by cross-schema and cross-catalog querying, there may be a slight increase in parsing time for initial queries. However, subsequent queries benefit from query plan caching and performance optimizations
   2. Fully Qualified Path Requirement: When performing cross-schema or cross-catalog queries, it is essential to provide the entire path, including the catalog name, schema name, and table name. For example: <catalog-name>.<schema-name>.<table-name>
   3. Same Account and Region for Cross-Catalog Queries: To query tables across catalogs, the catalogs must reside within the same account and region. This limitation ensures data security and optimal performance.
2. COLLECT_LIST() function has a few limitations, as listed below:
   1. Unsupported Complex Data Types: The COLLECT_LIST function currently does not support complex data type columns as input. It is designed to work with simple data types such as integers, strings, or dates.
   2. Absence of Sorting Capability: The COLLECT_LIST function does not allow for sorting operations, such as using the ORDER BY clause. The resulting list is generated based on the order of appearance of the values in the dataset, without the ability to explicitly control the ordering.
   3. Incompatibility with Distinct Operation: The COLLECT_LIST function cannot be used in conjunction with the DISTINCT operation. It is intended to gather all values within a specified grouping or windowing context, without eliminating duplicates or applying distinctness.
   4. Unsupported Union Operation with Window Functionality: The COLLECT_LIST function cannot be directly combined with the UNION operation when used within a window function. Unioning the results of COLLECT_LIST across different window partitions is currently not supported.
   5. Null Values Omitted from the List: When using the COLLECT_LIST function, null values are automatically excluded from the resulting list. The function gathers and returns non-null values only.
3. Please take note of the following restrictions when working with complex data types:
   1. Struct Limitations:
      1. Nested Structs: We do not support nested struct types, such as an array of structs. Querying the entire struct or individual leaf fields is supported, but querying intermediate levels within a struct is not.
      2. Array Inside a Struct and Struct inside a struct: Arrays and structs can be used within a struct and are supported.
      3. Standard Operations and Functions: Standard operations and functions can be applied to primitive struct types, allowing you to manipulate and analyze the data. However, operations, like joins, group by, order by, and union all involving structs, are only supported for leaf nodes.
   2. Array Limitations:
      1. Non-Primitive Arrays: Non-primitive arrays, including arrays of arrays, arrays of structs, and nested complex types within arrays, are not supported.
4. Queries that have the “Nested With clause having a limit” pattern will face parsing issues. Example of a type of that query:

WITH cte2 AS( WITH cte AS
(
           SELECT     d_year,
                      Sum(ss_net_profit) AS ss_net_profit
           FROM       store_sales ss
           INNER JOIN date_dim dd
           ON         ss.ss_sold_date_sk=dd.d_date_sk
           GROUP BY   1
           ORDER BY   1,
                      2)
SELECT   *
FROM     cte
ORDER BY 1 limit 2)
SELECT   d_year,
         avg(ss_net_profit)
FROM     cte2
GROUP BY 1
ORDER BY 1

• Platform Release Notes
  • Tab Management in Query Editor
  • Improvements to Automatically Added Clusters Tags
• Engine Release Notes
  • SQL Optimizations
• Bug Fixes
• Known Limitations

​

Usage Notes

• Update all Workspaces
• Suspend and resume all Clusters

​

Platform Release Notes

Tab Management in Query Editor

Improvements to Automatically Added Workspace & Clusters Tags

​

Engine Release Notes

SQL Optimizations

• Better handling of CTEs
  • When the same Common Table Expression (CTE) is used multiple times in a query, the engine now processes it in an optimized fashion to avoid duplicate processing.
• Optimizations for RIGHT JOIN and LEFT JOIN operators
  • Queries containing RIGHT JOIN operations now leverage more cores when possible to process queries more efficiently. Additionally, we have optimized some LEFT JOINs to behave as RIGHT JOINs, which triggers this optimization.
• COUNT DISTINCT optimization for high cardinality cases
  • If a COUNT DISTINCT operation is performed on a table with a large number of unique values, the engine now processes queries in a much more optimized manner.
• Improvements in high cardinality aggregations
  • If aggregations have many unique group values, the engine now processes queries more optimally by using all available cores.
• Improvements in dynamic filters and filter push-down
  • The engine now pushes down more complex filter types, reducing the data processed during table scans. We have also added more complex filter conditions to the list of filters we can push down. Example functions: DATE_TRUNC, DATETIME, UPPER, and LOWER

​

Bug Fixes

• DATE_DIFF correctness fixes:
  • Issue: Timestamps were considered during DATE_DIFF operations
  • Solution: Only dates are now accepted as a parameter and the difference will be provided in days by default.
• Complete list of tables from Catalog:
  • Issue: If the number of tables exceeded 100, the 101st table and after were not visible in the Schema Explorer. And it was arranged in alphabetical order
  • Solution: All tables are now listed in the Schema Explorer
• Column listing of delta tables
  • Issue: The columns for delta tables were not getting listed in the Schema explorer
  • Solution: The columns for delta tables are now getting listed on the schema explorer

​

Known Limitations

1. Tables with partitioned columns are currently not visible in the Schema Explorer but it is available for querying purposes.
2. Filter pushdown for queries with multiple CTEs is currently not happening due to a known bug.
3. Usage of STDDEV_SAMP function leads to row count mismatch issue.

• Data Preview
• Query Editor UI Improvements

​

Usage Notes

​

Platform Release Notes

Data Preview

Query Editor UI Improvements

• IP Allowlisting for External Access

​

Usage Notes

• Update Workspaces

​

Platform Release Notes

IP Allowlisting for External Access

• Cross-account Catalog Access Support
• Execution Planner Enhancements
• Query Editor Enhancements

​

Usage Notes

• Update Workspaces

​

Platform Release Notes

Cross-account Catalog Support

• Hive Metastores
• AWS Glue Metastores

Execution Planner Enhancements

Query Editor Enhancements

• query results
• query status
• result row count
• query cost

• Auto Save in Query Editor
• SSO Login from e6data Portal
• SSO via Google IdP

​

Usage Notes

• Update Workspaces

​

Platform Release Notes

Auto Save in Query Editor

SSO Login from e6data Portal

SSO via Google IdP

• Single Sign-On (SSO) via AWS SSO, Okta & SAML 2.0
• Support for AWS S3 Gateway Endpoints
• Dynamic Filtering with Date and Timestamp Columns
• Parquet Page-level Data Skipping/Pruning
• Distribution Enhancements

​

Usage Notes

• Update Workspaces
• Update Clusters to v0.6.12

​

Platform Release Notes

Single Sign-On (SSO) via AWS SSO, Okta & SAML 2.0

• AWS SSO
• Okta
• Other SAML 2.0 compliant IdPs

Support for AWS S3 Gateway Endpoints

​

Engine Release Notes

Dynamic Filtering with Date and Timestamp Columns

Parquet Page-level Data Skipping/Pruning

Distribution Enhancements

• Table Scans
• Filters
• Joins between Small Tables
• Aggregations

• Queries with Windowing functions
• Joins between Big Tables