9th December 2023
Plugin, platform, engine enhancements.
Last updated
Plugin, platform, engine enhancements.
Last updated
The 9th December 2023 release of e6data includes the following features & enhancements:
Version 2.0.0 of the has been released. The new version provides a significant boost in query performance.
The latest Python connector has been implemented in the Query Editor which has improved query performance.
The e6data engine now supports reading deletion vectors for enhanced storage optimization in Delta Lake tables. Deletion vectors allow DELETE operations to mark existing rows as removed without rewriting the entire Parquet file, improving performance and efficiency.
Querying deleted rows with an IN clause may result in an error.
Example: SELECT * FROM "table_with_del_vector" WHERE id IN ('ABCXYZ');
While using deletion vectors performance testing was conducted for up to 10 million deleted records. For a higher number of deleted records, performance may degrade from a few milliseconds up to 1 second. Users may experience slower query execution times when utilizing deletion vectors. Further optimizations are in the pipeline to resolve this.
Introduced the DROP CACHE command, which enables users to clear all files cached in-memory by the engine using the specified query.
DROP CACHE <select * from table query>
Only full/complete table cache drops are currently supported.
In certain scenarios, the drop cache operation may not be successful. Repeated execution of DROP CACHE can lead to a state where subsequent cache commands fail. A cluster restart may be required to mitigate this situation.
The current syntax does not gracefully handle scenarios where the DROP CACHE command is issued on a table that doesn't exist. The error message incorrectly states "unsupported syntax," which may be misleading. Future updates will enhance error messaging for non-existent tables in the DROP CACHE command.
The following functions have been added to enhance query capabilities:
WEEK: Extract the week component from a given date or timestamp.
CONTAINS_SUBSTR: Checks if a string contains a specified substring and returns a boolean value.
INSTR: Returns the position of the first occurrence of a substring within a string.
LEFT: Returns a specified number of characters from the beginning (left) of a string.
REGEXP_CONTAINS: Checks if a string matches a regular expression pattern and returns a boolean value.
REGEXP_REPLACE: Replaces substrings matching a regular expression pattern with a specified replacement string.
SOUNDEX: Returns a phonetic representation (SOUNDEX code) of a string, which can be used to identify similar-sounding words.