Jethro Server Release Notes

Jethro Server Release Notes

Version 3.8.0  |  Version 3.8.1 

Version 3.7.0 

Version 3.6.0  |  Version 3.6.1

Version 3.5.0  |  Version 3.5.1

Version 3.4.8  Version 3.4.7  Version 3.4.6   |  Version 3.4.5  | Version 3.4.4  | Version 3.4.3  |  Version 3.4.2  |  Version 3.4.1  |  Version 3.4.0

Version 3.3.0  |  Version 3.2.0  |  Version 3.1.0  |  Version 3.0.0  

Version 2.1.0  |  Version 2.0.0 

Version 1.6.0  |  Version 1.5.0  |  Version 1.4.0  |  Version 1.3.0  |  Version 1.2.0  |  Version 1.1.0 

Version 3.8.1

RELEASE DATE: Jul, 2023

Improvements

JD-6443 Improve TRACE message to monitor cubes scenarios
JD-6444 Cubes Generation Optimization
JD-6448 Tableau connector integration - minor compliance fix

Bug Fixes

JD-6010 Composed string function reach size limitations
JD-6092 Server fail to handle corrupted compressed map
JD-6442 Connector load failed due to log4j issue
JD-6447 Server resource manager non-optimal calculation may cause query rejection

Version 3.8.0

RELEASE DATE: Jun, 2022

New Features

JD-6426 Implement SHOW PRIMARY KEY and PK ODBC API
JD-6423 SPLIT function

Improvements

JD-6439 Loader - allow remove profile file and and add schema name to report file name. Set via parameters
JD-6434 Connector: use latest log4j (2.17.0) - mitigation for log4j vulnerability CVE-2021-44228
JD-6419 Handle pass through ODBC syntax at server level via host substitute fie (Tableau 2020.3)

Bug Fixes

JD-6440 Fix scheduled load lock corruption. The scheduled load service stop when encountering lock corruption and some scheduled jobs were not executed
JD-6433 Fix cube generation issue for queries with escaped single quote \('\) inside string literal that failed due to show cubes syntax error. Query sent to show cubes has non-escaped single quote
JD-6432 Fix inconsistent results: when join query has non-standard case expression that return 0 or 1 instead of true/false
JD-6427 Fix wrong result: day function over trunk function return wrong value
JD-6425 Fix case were count distinct query internally fails but keeps running in a loop filling logs with error message
JD-6422 Minor fix for JethroAdmin collect logs - correct handling of symbolic links in log folders
JD-6421 Fix nested Case When with internal Boolean expression that return numeric value (0 or 1) fails with "Double conversion for result of Case-When clause was requested"
JD-6420 Fix wrong results: group by show -1 values in more than one group following issue with fix JD-6386 in version 3.6.1
JD-6418 Fix server core - case with string range condition

Version 3.7.0

- Cubes cleanup by usage important improvements
- Support Tableau 2020.3
- Important bug fixes

RELEASE DATE: May, 2021

New Features

JD-6409 Support Having expression with non-aggregate group by expression (Tableau 2020.3)
JD-6410 Support function over group by expression in target list. Currently fails with: "Expression not in GROUP BY key" (Tableau 2020.3)

Improvements

JD-6398 Cubes cleanup by usage to drop unused rebuild cubes - cubes rebuild/update to keep cubes creation date
JD-6404 Notify in log on rows duplication in join
JD-6414 Support centos/rhel 8 - handle libgcrupt.so.11
JD-6416 Support Boolean expression that return numeric value (1 or 0) instead of true/false (Tableau 2020.3)

Bug Fixes

JD-6391 Server terminated when deleting from local cache
JD-6392 Server core - left join and complex if expression in where referencing to right table (outer to inner join optimization issue)
JD-6394 Bad decimal values fetch from Parquet via connector for high precision decimal column (38,10)
JD-6395 Repeated input stream error messages in log due to invalid adaptive entries causing server memory to inflate until server is killed
JD-6399 Cubes/adaptive usage info may be incomplete/inaccurate in some cases (new usage info is ignored)
JD-6400 Flatten query cause segmentation fault
JD-6401 Wrong value loaded to table when input data contains very long text (>4096) + 4096 max string size no imposed when using default (simple) tokenizer
JD-6402 Wrong results: join is wrongly removed by optimizer
JD-6408 Query with IN expression inside GB function fails with "expression is not in GROUP BY" error (Tableau 2020.3)
JD-6412 Join of sub-query with join key being literal fails with request execution failed (Join column wasn't found in left table) (Tableau 2020.3)
JD-6415 Group by on curdate() fails (Tableau 2020.3)
JD-6417 Aggregation query with having count(1)>1 and cross join returns large results set - should return single line (Tableau 2020.3)

Version 3.6.1

RELEASE DATE: Aug, 2020

Improvements

JD-6379 Jethro folders and files permissions tightening
JD-6381 Add GPG key to rpm
JD-6383 Support query text substitute (beta)
JD-6385 Cube maintenance: monitor adaptive entries usage and auto drop unused cubes and cache entries

Bug Fixes

JD-6351 Reset keys initial hash default
JD-6371 Server failure caused by query with cross join
JD-6372 Multiple "cancel event initiated" and "reached limit event" in log. Issue with query cancellation
JD-6373 Possible wrong result when flatten optimization applied on left join and right table is a sub-query with join (following 3.6 JD-6355)
JD-6374 Server failure on show cube query following duplicate alias error when query contains special characters
JD-6375 Concurrent loads fail due to lock issue - exception raised while trying to unlock lock
JD-6378 Parquet file read failed with: libsnappyjava.so: failed to map segment from shared object
JD-6386 Duplicated rows fetched on select * from join in a specific scenario
JD-6388 Possible wrong results when flatten optimization applied on join of multi table sub-query (following 3.6 JD-6355)
JD-6389 Query fails when filter comparing timestamp to date before 1/1/1970

Version 3.6.0

RELEASE DATE: Feb, 2020

New Features

JD-6282 Implement Full Outer Join

Improvements

JD-6341 Support Coalesce function
JD-6348 Reduce query init time - cache large schemas in deserialized format
JD-6352 Support Implicit Cross Join
JD-6355 Flatten query with join in sub query and outer query
JD-6357 Add queries statistics log message for simple queries stats analysis
JD-6363 Show local cache - add log to monitor progress

Bug Fixes

JD-6283 IfNull function fails if both columns are NULL
JD-6316 Query using Cubes fails with: ("internal error: exception raised" - "could not find the virtual object of the virtual column")
JD-6337 Timestamps functions error on NULL
JD-6338 Missing implicit group by null on aggregation with having
JD-6339 Hadoop URI issues
JD-6344 Query fails: aggregation refers to joined sub-query aggregation
JD-6345 Incomplete TRUE/FALSE literals handling in some expressions
JD-6350 Fail to write to adaptive for query with sub-query/view ("Can not write adaptive, 2 or more different expressions are equal")
JD-6351 Memory/files footprint of server is high when schema contain large number of low cardinality string columns

Version 3.5.1

RELEASE DATE: Oct, 2019

Improvements

JD-6320 New function: current_user()
JD-6334 Show external data source columns to return Jethro types (required from JM)
JD-6322 NUMERICCOUNT function
JD-6323 SHOW TABLE COLUMNS - skip calculation of very high cardinality distinct when metadata not available
JD-6324 show statistics: add show data sources and show external tables
JD-6325 Improve status checks for IO operations

Bug Fixes

JD-6292 Server failure following INSERT INTO VALUES with NULL::STRING literal
JD-6301 Server failure at cubes match
JD-6319 Query joining managed table with external table fails
JD-6326 Reduce maint service IO operation during garbage collection and cubes scan cycles
JD-6327 Cube gets where condition following cube update due to rewrite that pushes ON condition into WHERE clause but does not remove it from ON condition
JD-6329 Connector: inconsistent results from Parquet decimal data type
JD-6330 Accuracy issue: query with left join where right table is a sub-query with filter
JD-6336 Connector: fail to read Parquet binary columns as strings

Version 3.5.0

RELEASE DATE: Feb, 2019

- Merge tables
- Various improvements
- Bug fixes

Upgrade Instructions

Not special upgrade instruction from 3.4.x are required.

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

New Features

JD-6278 MERGE TABLE - quick vertical merge of two or more tables into a new table

Improvements

JD-6247 Support preview for orc and parquet files
JD-6286 Allow LDAP SSL without certificate
JD-6288 Concurrent query init time optimization: cache schema and global config, reduce keys refresh frequency
JD-6290 Add support from HDP3 client environment
JD-6303 SHOW FULL TABLES - support wildcard semantics instead of table name (=all tables)
JD-6305 Partial support ORACLE as generic JDBC data source
JD-6306 Parameters default values changes

Bug Fixes

JD-6313 Inconsistent results when querying union joined as right table with empty results set on union sub-select
JD-6279 Select DATE data type from Parquet/ORC fails or fetch wrong values
JD-6285 Altering value of query.timeout is ignored
JD-6289 Total query time in plan for very fast queries shows wrong value (1537328463.426310)
JD-6291 Server core following query failure - cast of timestamp column to string in where clause
JD-6294 Accuracy issue - query with where expressions with case pushed down to sub-query
JD-6300 Query queueing issues
JD-6307 Accuracy issue - case and cross join
JD-6308 Keys dat files corrupted / missing while metadata indicated keys file is complete. Server core when reading keys file
JD-6309 Accuracy issue following partition overwrite when value in replaced partition not longer exists in the table
JD-6310 External table filter rewrite issues

Version 3.4.8

RELEASE DATE: Jul 8, 2018

- Functionality required for Hive 3 integration  
- Bug fixes

Upgrade Instructions

IMPORTANT!!

Read carefully the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before and after starting the upgrade process.

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

Improvements

JD-5359 Allow using OFFSET even if ORDER BY not specified
JD-5467 SHOW FUNCTIONS - new show command
JD-6070 Enable preemptive timeout cancellation by default

Bug Fixes

JD-5734 The false expression "<> NULL" in WHERE clause cause server failure
JD-6096 Loader fails with core dump on exit
JD-6124 Various SHOW CREATE TABLE issues with external table attributes 
JD-6169 Query accuracy - when if function branch is Qlik function and if function term is negative (1=0)
JD-6200 Server failure can occur when query is being canceled while query in pending in queue
JD-6203 Maint log error - fail to delete OBSOLETE cube dat file. This is valid scenario - error should not be logged
JD-6248 Cast from INTEGER to Integer failed error received when fetching from external table
JD-6249 Monitor will not restarted in case of immediate call to service start of service stop command
JD-6264 Unable to connect to LDAP Posix groups in Oracle Directory Server
JD-6265 Connector fails with "Too many open files" when reading from Parquet/ORC
JD-6270 Fix log printout of SQL statement with LIKE function
JD-6271 CountDistinct container memory leak during cube read

Minor Bug Fixes

JD-6220 spelling and style fixes to ldap log messages

Version 3.4.7

RELEASE DATE: Jun 6, 2018

- Support impersonation for external tables access
- Fix loader performance degradation  
- Bug fixes

Upgrade Instructions

IMPORTANT!!

Read carefully the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before and after starting the upgrade process.

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

New Features

JD-4834 Support impersonation - delegate credentials when accessing external tables
JD-5793 Support metadata and preview from external data sources (for internal use by JM)

Improvements

JD-6180 change MapR version tagging to postfix instead of prefix

Bug Fixes

JD-5940 JethroMonitor log owner set group as user
JD-6136 Enable preemptive timeout cancellation by default
JD-6150 Load performance degradation
JD-6155 BASH complete incompatible with KSH
JD-6159 Collect logs does not output SHOW TABLE MAINT for all tables
JD-6162 Hadoop home setting for mapr is incorrect
JD-6177 Query with count distinct over cube generate internal error
JD-6213 Jethro Server failure following cube check (show cubes) syntax error

Version 3.4.6

RELEASE DATE: May 16, 2018

- DDL commands for show and alter external tables
- Index rebuild
- New string functions (lcase/ucase)
- Bug fixes

Upgrade Instructions

IMPORTANT!!

Read carefully the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before and after starting the upgrade process.

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

New Features

JD-4709 SHOW CREATE TABLE
JD-5779 ALTER EXTERNAL TABLE
JD-5931 Add SQL functions: LCASE() and UCASE()
JD-6021 ALTER INDEX REBUILD

Improvements

JD-5206 Query plan improvements
JD-5219 SHOW PARAMS EXTENDED with parameters metadata
JD-5834 Expand SHOW EXTERNAL TABLES] to show all attributes
JD-5844 SHOW EXTERNAL TABLE COLUMNS table-name]
JD-5865 Support preemptive query cancellation on timeout
JD-5954 New log for GC file deletion

Bug Fixes

JD-5298 Thread sys limit file /etc/security/limits.d/jethrodata got corrupted
JD-5403 show tables in schema ignores wrong command syntax
JD-5893 LDAP authentication fails with Ds that contain special characters
JD-5898 Frequent schema changes cause issue with cube gen and match due to wrong handling of different schema versions on cube match
JD-6037 Create External Data Source and Tables requests hang
JD-6044 Server failure on query executing when push join write disabled by configuration
JD-6045 Empty key dat file is created when loader fails on close
JD-6053 GROUP BY STATISTICS shows garbage rows count for cube candidates exceeded max size in bytes
JD-6095 LDAP group name attribute parameter is ignored on group search. User id attribute is used instead
JD-6096 Load may fail ungracefully (core) during exit
JD-6108 Maint stops sending cubes for check / generate after a cube check run fails
JD-6109 Flatten rewrite generates wrong expression in case of JOIN in outer SQL
JD-6112 Server failure on join query with where expression schema not specified and not default
JD-6117 In the case "cube candidate exceeded max size in bytes", should not write any tooBig statistics

Minor Bug Fixes

JD-5373 "schema <>" is presented on "show join indexes" on non def_schema
JD-5404 Unclear error message when table name is missing on drop scheduled loads command
JD-5720 Failed to commit version dat file for adaptive error should by INFO level message

 

Version 3.4.5

RELEASE DATE: Apr 17, 2018

- Optimizations for queries with large dim tables
- Support partitions when reading from Parquet/ORC
- IO optimizations
- Bug fixes

Upgrade Instructions

IMPORTANT!!

Read carefully the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before and after starting the upgrade process.

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

Improvements

JD-5977 Ext_table: Support partitions when reading from Parquet/ORC
JD-6019 Star no-where / join removal: add configuration for unique dim value threshold and set default to 100M
JD-6032 Optimize IO during Fetch Dispatcher initialization

Bug Fixes

JD-5851 Query failed with "Reached limit" error message
JD-5882 Instance name verification is not imposed in multi part identifier in FROM clause

 

Version 3.4.4

RELEASE DATE: Apr 3, 2018

- Optimizations for queries with joins of flat projections (PBI)
- Optimizations for query init stages
- Monitor slow IO operations in log
- Support column rename
- Important bug fixes

Upgrade Instructions

IMPORTANT!!

Read carefully the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before and after starting the upgrade process.

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

New Features

JD-2857 Optimization: Trim unused columns at rewrite stage
JD-3062 Support ALTER TABLE RENAME COLUMN
JD-5773 Optimization: Support flatten query over joins including joins with subqueries

Improvements

JD-5449 Improve error message in case process fork fails
JD-5067 Collect grouping sizes statistics to be used for cubes generation. Show statistics via: SHOW GROUP BY STATISTICS
JD-5701 Change default for parameter query.in.filter.threshold from 1000 to 2500
JD-5780 Do not add distinct for sub-select in IN clause if all values are unique
JD-5795 Optimize query initialization stage (optimize toSQL function)
JD-5867 Show WARNING in log in simple IO operations (listDir, open) takes too much time

Bug Fixes

JD-4079 ToSQL: Missing alias in after-rewrite query
JD-5575 Cubes not match by original query with specific SUM(case... ) expression causing continues cube generation tries
JD-5722 Cancel of query with UNION takes a very long time to complete
JD-5737 Server failure following UPDATE CUBE request on specific cube scenario
JD-5742 Cancel Query fails due to "CSN node build fail"
JD-5760 Potential wrong cube generation/match following JD-5544 fix (3.4.1)
JD-5830 External table: accuracy issue when running query joining and filtering multiple external tables
JD-5866 Can't grant roll to a group which its name includes hyphen
JD-5874 Adaptive index write in not async and may slow query execution
JD-5900 Cubes generation may hang when storage schema is newer then generate session schema
JD-5904 Accuracy issue in cube scenario for count distinct queries following partition drop
JD-5921 Unspecific error code/message is logged after HDFS error
JD-5938 Potential server failure in case of PLANNER log verbosity is DEBUG
JD-5962 Server consumes significant more memory than necessary upon INSERT INTO/SELECT * from large external table
JD-5990 Support additional Null Safe Join syntax (PowerBI)

Minor Bug Fixes

JD-5564 Log message spelling fix

 

Version 3.4.3

RELEASE DATE: Feb 6, 2018

- Files connector now support Parquet and ORC formats
- Cancel query improvements
- Important bug fixes

Upgrade Instructions

IMPORTANT!!

Read carefully the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before and after starting the upgrade process.

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

New Features

JD-5631 SHOW TABLES] STATS - fast show of table metadata
JD-5730 External table files connector - Support Parquet and ORC files formats
JD-5811 Support new function: Left(string, int

Improvements

JD-5247 Expression in SELECT list may be function of GROUP BY expressions
JD-5735 Optimizer: expend transformation scenarios of OUTER JOINS to INNER JOINS
JD-5756 Optimizer: Allow Push-Join optimization for OUTER LEFT JOIN
JD-5759 Ext TBL: Allow conversion of DECIMAL type from external data source to Jethro numeric data type
JD-5783 Control column names case in result set via config

Bug Fixes

JD-5693 SELECT DISTINCT query fail with 'Request execution failed'
JD-5732 Cancel query: cross join query doesn't responed immediatly to cancellation
JD-5786 Cancel Query: query on external table doesn't responed immediatly to cancellation
JD-5797 SQL statistics should use SHOW TABLES MAINT instead of SHOW ALL TABLES MAINT

 

Version 3.4.2