5th May 2023

Summary

The 5th May 2023 release of e6data includes the following features & enhancements:

• Platform Release Notes

  • Tab Management in Query Editor

  • Improvements to Automatically Added Clusters Tags

• Engine Release Notes

  • SQL Optimizations

• Bug Fixes

• Known Limitations

Usage Notes

To get access to all features from this release:

• Update all Workspaces

• Suspend and resume all Clusters

Platform Release Notes

Tab Management in Query Editor

Users will now be able to drag and move tabs to change their order in the Query Editor. Each user's tab order will be auto-saved for future sessions.

More information: Tab Management

Improvements to Automatically Added Workspace & Clusters Tags

A tag containing the email address of the user who creates a Workspace or Cluster will be added automatically. This will help map usage and costs back to a specific user.

More information:

• Workspace Tags

Engine Release Notes

SQL Optimizations

• Better handling of CTEs

  • When the same Common Table Expression (CTE) is used multiple times in a query, the engine now processes it in an optimized fashion to avoid duplicate processing.

• Optimizations for RIGHT JOIN and LEFT JOIN operators

  • Queries containing RIGHT JOIN operations now leverage more cores when possible to process queries more efficiently. Additionally, we have optimized some LEFT JOINs to behave as RIGHT JOINs, which triggers this optimization.

• COUNT DISTINCT optimization for high cardinality cases

  • If a COUNT DISTINCT operation is performed on a table with a large number of unique values, the engine now processes queries in a much more optimized manner.

• Improvements in high cardinality aggregations

  • If aggregations have many unique group values, the engine now processes queries more optimally by using all available cores.

• Improvements in dynamic filters and filter push-down

  • The engine now pushes down more complex filter types, reducing the data processed during table scans. We have also added more complex filter conditions to the list of filters we can push down. Example functions: DATE_TRUNC, DATETIME, UPPER, and LOWER

Bug Fixes

• DATE_DIFF correctness fixes:

  • Issue: Timestamps were considered during DATE_DIFF operations

  • Solution: Only dates are now accepted as a parameter and the difference will be provided in days by default.

• Complete list of tables from Catalog:

  • Issue: If the number of tables exceeded 100, the 101st table and after were not visible in the Schema Explorer. And it was arranged in alphabetical order

  • Solution: All tables are now listed in the Schema Explorer

• Column listing of delta tables

  • Issue: The columns for delta tables were not getting listed in the Schema explorer

  • Solution: The columns for delta tables are now getting listed on the schema explorer

Know Limitations

1. Tables with partitioned columns are currently not visible in the Schema Explorer but it is available for querying purposes.

2. Filter pushdown for queries with multiple CTEs is currently not happening due to a known bug.

3. Usage of STDDEV_SAMP function leads to row count mismatch issue.