Version 3.8.0  |  Version 3.8.1 

Version 3.7.0 

Version 3.6.0  |  Version 3.6.1

Version 3.5.0  |  Version 3.5.1

Version 3.4.8  |  Version 3.4.7  | Version 3.4.6   |  Version 3.4.5  | Version 3.4.4  | Version 3.4.3  |  Version 3.4.2  |  Version 3.4.1  |  Version 3.4.0

Version 3.3.0  |  Version 3.2.0  |  Version 3.1.0  |  Version 3.0.0  

Version 2.1.0  |  Version 2.0.0 

Version 1.6.0  |  Version 1.5.0  |  Version 1.4.0  |  Version 1.3.0  |  Version 1.2.0  |  Version 1.1.0 

Version 3.8.1

RELEASE DATE: Jul, 2023

Improvements

JD-6443 Improve TRACE message to monitor cubes scenarios
JD-6444 Cubes Generation Optimization
JD-6448 Tableau connector integration - minor compliance fix

Bug Fixes

JD-6010 Composed string function reach size limitations
JD-6092 Server fail to handle corrupted compressed map
JD-6442 Connector load failed due to log4j issue
JD-6447 Server resource manager non-optimal calculation may cause query rejection

Version 3.8.0

RELEASE DATE: Jun, 2022

New Features

JD-6426 Implement SHOW PRIMARY KEY and PK ODBC API
JD-6423 SPLIT function

Improvements

JD-6439 Loader - allow remove profile file and and add schema name to report file name. Set via parameters
JD-6434 Connector: use latest log4j (2.17.0) - mitigation for log4j vulnerability CVE-2021-44228
JD-6419 Handle pass through ODBC syntax at server level via host substitute fie (Tableau 2020.3)

Bug Fixes

JD-6440 Fix scheduled load lock corruption. The scheduled load service stop when encountering lock corruption and some scheduled jobs were not executed
JD-6433 Fix cube generation issue for queries with escaped single quote \('\) inside string literal that failed due to show cubes syntax error. Query sent to show cubes has non-escaped single quote
JD-6432 Fix inconsistent results: when join query has non-standard case expression that return 0 or 1 instead of true/false
JD-6427 Fix wrong result: day function over trunk function return wrong value
JD-6425 Fix case were count distinct query internally fails but keeps running in a loop filling logs with error message
JD-6422 Minor fix for JethroAdmin collect logs - correct handling of symbolic links in log folders
JD-6421 Fix nested Case When with internal Boolean expression that return numeric value (0 or 1) fails with "Double conversion for result of Case-When clause was requested"
JD-6420 Fix wrong results: group by show -1 values in more than one group following issue with fix JD-6386 in version 3.6.1
JD-6418 Fix server core - case with string range condition

Version 3.7.0

- Cubes cleanup by usage important improvements
- Support Tableau 2020.3
- Important bug fixes

RELEASE DATE: May, 2021

New Features

JD-6409 Support Having expression with non-aggregate group by expression (Tableau 2020.3)
JD-6410 Support function over group by expression in target list. Currently fails with: "Expression not in GROUP BY key" (Tableau 2020.3)

Improvements

JD-6398 Cubes cleanup by usage to drop unused rebuild cubes - cubes rebuild/update to keep cubes creation date
JD-6404 Notify in log on rows duplication in join
JD-6414 Support centos/rhel 8 - handle libgcrupt.so.11
JD-6416 Support Boolean expression that return numeric value (1 or 0) instead of true/false (Tableau 2020.3)

Bug Fixes

JD-6391 Server terminated when deleting from local cache
JD-6392 Server core - left join and complex if expression in where referencing to right table (outer to inner join optimization issue)
JD-6394 Bad decimal values fetch from Parquet via connector for high precision decimal column (38,10)
JD-6395 Repeated input stream error messages in log due to invalid adaptive entries causing server memory to inflate until server is killed
JD-6399 Cubes/adaptive usage info may be incomplete/inaccurate in some cases (new usage info is ignored)
JD-6400 Flatten query cause segmentation fault
JD-6401 Wrong value loaded to table when input data contains very long text (>4096) + 4096 max string size no imposed when using default (simple) tokenizer
JD-6402 Wrong results: join is wrongly removed by optimizer
JD-6408 Query with IN expression inside GB function fails with "expression is not in GROUP BY" error (Tableau 2020.3)
JD-6412 Join of sub-query with join key being literal fails with request execution failed (Join column wasn't found in left table) (Tableau 2020.3)
JD-6415 Group by on curdate() fails (Tableau 2020.3)
JD-6417 Aggregation query with having count(1)>1 and cross join returns large results set - should return single line (Tableau 2020.3)

Version 3.6.1

RELEASE DATE: Aug, 2020

Improvements

JD-6379 Jethro folders and files permissions tightening
JD-6381 Add GPG key to rpm
JD-6383 Support query text substitute (beta)
JD-6385 Cube maintenance: monitor adaptive entries usage and auto drop unused cubes and cache entries

Bug Fixes

JD-6351 Reset keys initial hash default
JD-6371 Server failure caused by query with cross join
JD-6372 Multiple "cancel event initiated" and "reached limit event" in log. Issue with query cancellation
JD-6373 Possible wrong result when flatten optimization applied on left join and right table is a sub-query with join (following 3.6 JD-6355)
JD-6374 Server failure on show cube query following duplicate alias error when query contains special characters
JD-6375 Concurrent loads fail due to lock issue - exception raised while trying to unlock lock
JD-6378 Parquet file read failed with: libsnappyjava.so: failed to map segment from shared object
JD-6386 Duplicated rows fetched on select * from join in a specific scenario
JD-6388 Possible wrong results when flatten optimization applied on join of multi table sub-query (following 3.6 JD-6355)
JD-6389 Query fails when filter comparing timestamp to date before 1/1/1970

Version 3.6.0

RELEASE DATE: Feb, 2020

New Features

JD-6282 Implement Full Outer Join

Improvements

JD-6341 Support Coalesce function
JD-6348 Reduce query init time - cache large schemas in deserialized format
JD-6352 Support Implicit Cross Join
JD-6355 Flatten query with join in sub query and outer query
JD-6357 Add queries statistics log message for simple queries stats analysis
JD-6363 Show local cache - add log to monitor progress

Bug Fixes

JD-6283 IfNull function fails if both columns are NULL
JD-6316 Query using Cubes fails with: ("internal error: exception raised" - "could not find the virtual object of the virtual column")
JD-6337 Timestamps functions error on NULL
JD-6338 Missing implicit group by null on aggregation with having
JD-6339 Hadoop URI issues
JD-6344 Query fails: aggregation refers to joined sub-query aggregation
JD-6345 Incomplete TRUE/FALSE literals handling in some expressions
JD-6350 Fail to write to adaptive for query with sub-query/view ("Can not write adaptive, 2 or more different expressions are equal")
JD-6351 Memory/files footprint of server is high when schema contain large number of low cardinality string columns

Version 3.5.1

RELEASE DATE: Oct, 2019

Improvements

JD-6320 New function: current_user()
JD-6334 Show external data source columns to return Jethro types (required from JM)
JD-6322 NUMERICCOUNT function
JD-6323 SHOW TABLE COLUMNS - skip calculation of very high cardinality distinct when metadata not available
JD-6324 show statistics: add show data sources and show external tables
JD-6325 Improve status checks for IO operations

Bug Fixes

JD-6292 Server failure following INSERT INTO VALUES with NULL::STRING literal
JD-6301 Server failure at cubes match
JD-6319 Query joining managed table with external table fails
JD-6326 Reduce maint service IO operation during garbage collection and cubes scan cycles
JD-6327 Cube gets where condition following cube update due to rewrite that pushes ON condition into WHERE clause but does not remove it from ON condition
JD-6329 Connector: inconsistent results from Parquet decimal data type
JD-6330 Accuracy issue: query with left join where right table is a sub-query with filter
JD-6336 Connector: fail to read Parquet binary columns as strings

Version 3.5.0

RELEASE DATE: Feb, 2019

- Merge tables
- Various improvements
- Bug fixes

Upgrade Instructions

Not special upgrade instruction from 3.4.x are required.

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

New Features

JD-6278 MERGE TABLE - quick vertical merge of two or more tables into a new table

Improvements

JD-6247 Support preview for orc and parquet files
JD-6286 Allow LDAP SSL without certificate
JD-6288 Concurrent query init time optimization: cache schema and global config, reduce keys refresh frequency
JD-6290 Add support from HDP3 client environment
JD-6303 SHOW FULL TABLES - support wildcard semantics instead of table name (=all tables)
JD-6305 Partial support ORACLE as generic JDBC data source
JD-6306 Parameters default values changes

Bug Fixes

JD-6313 Inconsistent results when querying union joined as right table with empty results set on union sub-select
JD-6279 Select DATE data type from Parquet/ORC fails or fetch wrong values
JD-6285 Altering value of query.timeout is ignored
JD-6289 Total query time in plan for very fast queries shows wrong value (1537328463.426310)
JD-6291 Server core following query failure - cast of timestamp column to string in where clause
JD-6294 Accuracy issue - query with where expressions with case pushed down to sub-query
JD-6300 Query queueing issues
JD-6307 Accuracy issue - case and cross join
JD-6308 Keys dat files corrupted / missing while metadata indicated keys file is complete. Server core when reading keys file
JD-6309 Accuracy issue following partition overwrite when value in replaced partition not longer exists in the table
JD-6310 External table filter rewrite issues

Version 3.4.8

RELEASE DATE: Jul 8, 2018

- Functionality required for Hive 3 integration  
- Bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

Improvements

JD-5359 Allow using OFFSET even if ORDER BY not specified
JD-5467 SHOW FUNCTIONS - new show command
JD-6070 Enable preemptive timeout cancellation by default

Bug Fixes

JD-5734 The false expression "<> NULL" in WHERE clause cause server failure
JD-6096 Loader fails with core dump on exit
JD-6124 Various SHOW CREATE TABLE issues with external table attributes 
JD-6169 Query accuracy - when if function branch is Qlik function and if function term is negative (1=0)
JD-6200 Server failure can occur when query is being canceled while query in pending in queue
JD-6203 Maint log error - fail to delete OBSOLETE cube dat file. This is valid scenario - error should not be logged
JD-6248 Cast from INTEGER to Integer failed error received when fetching from external table
JD-6249 Monitor will not restarted in case of immediate call to service start of service stop command
JD-6264 Unable to connect to LDAP Posix groups in Oracle Directory Server
JD-6265 Connector fails with "Too many open files" when reading from Parquet/ORC
JD-6270 Fix log printout of SQL statement with LIKE function
JD-6271 CountDistinct container memory leak during cube read

Minor Bug Fixes

JD-6220 spelling and style fixes to ldap log messages

Version 3.4.7

RELEASE DATE: Jun 6, 2018

- Support impersonation for external tables access
- Fix loader performance degradation  
- Bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

New Features

JD-4834 Support impersonation - delegate credentials when accessing external tables
JD-5793 Support metadata and preview from external data sources (for internal use by JM)

Improvements

JD-6180 change MapR version tagging to postfix instead of prefix

Bug Fixes

JD-5940 JethroMonitor log owner set group as user
JD-6136 Enable preemptive timeout cancellation by default
JD-6150 Load performance degradation
JD-6155 BASH complete incompatible with KSH
JD-6159 Collect logs does not output SHOW TABLE MAINT for all tables
JD-6162 Hadoop home setting for mapr is incorrect
JD-6177 Query with count distinct over cube generate internal error
JD-6213 Jethro Server failure following cube check (show cubes) syntax error

Version 3.4.6

RELEASE DATE: May 16, 2018

- DDL commands for show and alter external tables
- Index rebuild
- New string functions (lcase/ucase)
- Bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.6
Ambari Management Package 1.09

New Features

JD-4709 SHOW CREATE TABLE
JD-5779 ALTER EXTERNAL TABLE
JD-5931 Add SQL functions: LCASE() and UCASE()
JD-6021 ALTER INDEX REBUILD

Improvements

JD-5206 Query plan improvements
JD-5219 SHOW PARAMS EXTENDED with parameters metadata
JD-5834 Expand SHOW EXTERNAL TABLES] to show all attributes
JD-5844 SHOW EXTERNAL TABLE COLUMNS table-name]
JD-5865 Support preemptive query cancellation on timeout
JD-5954 New log for GC file deletion

Bug Fixes

JD-5298 Thread sys limit file /etc/security/limits.d/jethrodata got corrupted
JD-5403 show tables in schema ignores wrong command syntax
JD-5893 LDAP authentication fails with Ds that contain special characters
JD-5898 Frequent schema changes cause issue with cube gen and match due to wrong handling of different schema versions on cube match
JD-6037 Create External Data Source and Tables requests hang
JD-6044 Server failure on query executing when push join write disabled by configuration
JD-6045 Empty key dat file is created when loader fails on close
JD-6053 GROUP BY STATISTICS shows garbage rows count for cube candidates exceeded max size in bytes
JD-6095 LDAP group name attribute parameter is ignored on group search. User id attribute is used instead
JD-6096 Load may fail ungracefully (core) during exit
JD-6108 Maint stops sending cubes for check / generate after a cube check run fails
JD-6109 Flatten rewrite generates wrong expression in case of JOIN in outer SQL
JD-6112 Server failure on join query with where expression schema not specified and not default
JD-6117 In the case "cube candidate exceeded max size in bytes", should not write any tooBig statistics

Minor Bug Fixes

JD-5373 "schema <>" is presented on "show join indexes" on non def_schema
JD-5404 Unclear error message when table name is missing on drop scheduled loads command
JD-5720 Failed to commit version dat file for adaptive error should by INFO level message

Version 3.4.5

RELEASE DATE: Apr 17, 2018

- Optimizations for queries with large dim tables
- Support partitions when reading from Parquet/ORC
- IO optimizations
- Bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

Improvements

JD-5977 Ext_table: Support partitions when reading from Parquet/ORC
JD-6019 Star no-where / join removal: add configuration for unique dim value threshold and set default to 100M
JD-6032 Optimize IO during Fetch Dispatcher initialization

Bug Fixes

JD-5851 Query failed with "Reached limit" error message
JD-5882 Instance name verification is not imposed in multi part identifier in FROM clause

Version 3.4.4

RELEASE DATE: Apr 3, 2018

- Optimizations for queries with joins of flat projections (PBI)
- Optimizations for query init stages
- Monitor slow IO operations in log
- Support column rename
- Important bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

New Features

JD-2857 Optimization: Trim unused columns at rewrite stage
JD-3062 Support ALTER TABLE RENAME COLUMN
JD-5773 Optimization: Support flatten query over joins including joins with subqueries

Improvements

JD-5449 Improve error message in case process fork fails
JD-5067 Collect grouping sizes statistics to be used for cubes generation. Show statistics via: SHOW GROUP BY STATISTICS
JD-5701 Change default for parameter query.in.filter.threshold from 1000 to 2500
JD-5780 Do not add distinct for sub-select in IN clause if all values are unique
JD-5795 Optimize query initialization stage (optimize toSQL function)
JD-5867 Show WARNING in log in simple IO operations (listDir, open) takes too much time

Bug Fixes

JD-4079 ToSQL: Missing alias in after-rewrite query
JD-5575 Cubes not match by original query with specific SUM(case... ) expression causing continues cube generation tries
JD-5722 Cancel of query with UNION takes a very long time to complete
JD-5737 Server failure following UPDATE CUBE request on specific cube scenario
JD-5742 Cancel Query fails due to "CSN node build fail"
JD-5760 Potential wrong cube generation/match following JD-5544 fix (3.4.1)
JD-5830 External table: accuracy issue when running query joining and filtering multiple external tables
JD-5866 Can't grant roll to a group which its name includes hyphen
JD-5874 Adaptive index write in not async and may slow query execution
JD-5900 Cubes generation may hang when storage schema is newer then generate session schema
JD-5904 Accuracy issue in cube scenario for count distinct queries following partition drop
JD-5921 Unspecific error code/message is logged after HDFS error
JD-5938 Potential server failure in case of PLANNER log verbosity is DEBUG
JD-5962 Server consumes significant more memory than necessary upon INSERT INTO/SELECT * from large external table
JD-5990 Support additional Null Safe Join syntax (PowerBI)

Minor Bug Fixes

JD-5564 Log message spelling fix

Version 3.4.3

RELEASE DATE: Feb 6, 2018

- Files connector now support Parquet and ORC formats
- Cancel query improvements
- Important bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

New Features

JD-5631 SHOW TABLES] STATS - fast show of table metadata
JD-5730 External table files connector - Support Parquet and ORC files formats
JD-5811 Support new function: Left(string, int

Improvements

JD-5247 Expression in SELECT list may be function of GROUP BY expressions
JD-5735 Optimizer: expend transformation scenarios of OUTER JOINS to INNER JOINS
JD-5756 Optimizer: Allow Push-Join optimization for OUTER LEFT JOIN
JD-5759 Ext TBL: Allow conversion of DECIMAL type from external data source to Jethro numeric data type
JD-5783 Control column names case in result set via config

Bug Fixes

JD-5693 SELECT DISTINCT query fail with 'Request execution failed'
JD-5732 Cancel query: cross join query doesn't responed immediatly to cancellation
JD-5786 Cancel Query: query on external table doesn't responed immediatly to cancellation
JD-5797 SQL statistics should use SHOW TABLES MAINT instead of SHOW ALL TABLES MAINT

Version 3.4.2

RELEASE DATE: Feb 12, 2018

- Auto renewal/recreation of Kerberos ticket
- Jethro Monitor Service replacing crontab job
- Connector to any external data source via generic JDBC connector
- Cancel query improvements
- Important bug fixes

Upgrade Instructions

Compatibility

Compatible components versions: 

JethroManager 1.5.1
Ambari Management Package 1.09

New Features

JD-5385 Auto renewal/recreation of Kerberos ticket by Jethro
JD-5602 Jethro Monitor Service (replace crontab job)
JD-5638 CREATE EXTERNAL DATA SOURCE to support generic JDBC connector

Improvements

JD-2881 Apply star transformation to join keys of different data types
JD-5606 Backup jd-hadoop-env.sh on version upgrade
JD-5458 Cube match rewrite optimization: remove outer group by when it is redindant to improve query over cube performance
JD-5490 Support partition pruning for string columns
JD-5628 Expand SHOW EXTERNAL DATA SOURCE[S] to show all attributes
JD-5670 Cancel Query flow refactoring in Node execution (internal)
JD-5703 Support SHOW TABLE EXTENDED for specific table

Bug Fixes

JD-5086 EXT_table: query over external table with WHERE and OR on join tables fails
JD-5261 Query fail to find cube the was generated from that query following cube rewrite
JD-5316 Cube with where does not survive rebuild
JD-5334 Cubes .dat files are reported missing in server log when trying to match
JD-5456 Unoptimized multi allocation during client request 
JD-5573 GB function rewrite on query with self-join returns internal error
JD-5580 Query with QLIK function fails with internal error ("could not find the virtual object of the virtual column")
JD-5591 Query with function over timestamp columns that appears more then once in the target list fails on cube reading with internal error ("could not find the virtual object of the virtual column")
JD-5596 Generic low level error messages appears as FATAL in log - should be WARN level
JD-5597 Query fail with "FingerPrint mismatch" error
JD-5626 Join indexes fix utility does not support multi schema
JD-5632 JethroClient core when running query that is killed by another client
JD-5640 Statistics are not collected if jethro password is changed
JD-5654 Cancel Query takes a very long time to complete for query with multiple joins
JD-5674 Queries on external table hang due to mishandling of exception from external data source
JD-5678 Memory Leak in query with count distinct over very high cardinality DOUBLE column
JD-5679 Query hangs when sub-query in where fails
JD-5695 Query hangs when partitioning dim by string columns and the filter (NOT IN) is on this column

Minor Bug Fixes

JD-5535 Self join query without table aliases failed on unclear error message (Request execution failed)
JD-5570 service jethro restart generate "Attempting to gracefully stop Instance" message to the screen

Version 3.4.1

RELEASE DATE: Jan 22, 2018

- Support table move between schemas
- Cubes usage visibility improvements
- Cache query results over cubes
- Important bug fixes

Upgrade Instructions

Compatibility

See 3.4.0 release notes

Improvements

JD-4451 Explicitly show cubes usage in query plan
JD-5014 Show cubes - add creation time and update time
JD-5457 Create results cache entries for queries over cubes for slow queries
JD-5510 ALTER TABLE RENAME support move between schemas

Bug Fixes

JD-4968 Cubes rebuild is not preserving the cube original tag
JD-5394 Cubes generation stop sending requests following failure in cube check request
JD-5459 Cubes generation stop sending requests following failure to send requests to server
JD-5460 Load from external table mapped to file failed is number of columns GT 512
JD-5477 Query fails on internal error ("could not find the virtual object of the virtual column") due to wrong handle of literal expression
JD-5487 Accuracy - where expression with function on string that contain IS NULL ignores null values in column (function to index optimization issue)
JD-5491 Potential wrong results when filter include CASE and condition expressions in CASE contain AND operator
JD-5513 Select from external table reject numeric values of the format "+12345" (fail to convert value to INT/BIGINT)
JD-5544 CAST on literal within SUM aggregation cause cube not to be matched
JD-5547 Cubes generation stop sending requests following rebuild error caused by race condition
JD-5545 Too big assertion when cube generation reach cube size limit is not working since version 3.3.2
JD-5546 JethroServer failure when NULL from subquery passed as target column of IN
JD-5552 Fix for upgrade tool from pre-3.4.0 releases: correct handling of uppercase passwords and increase wait time for server to start

Version 3.4.0

RELEASE DATE: Dec 25, 2017

- Support multiple schemas.
- Fix environment settings for JAVA to simplfy initial setup 
- Bug fixes

Upgrade Instructions

Compatibility

Server version 3.4 is NOT compatible with previous version of ODBC, JDBC and JethroManager. All client components must be upgraded.

Compatible client versions: 

ODBC 3.7
JDBC 3.6
JethroManager 1.5

New Features

JD-1137 Multiple Schames support
JD-4653 Schema related DDL commands: CREATE SCHEMA, DROP SCHEMA, SHOW SCHEMA, USE

Improvements

JD-5197 CREATE EXTERNAL TABLE to allow setting file format spec (delimiter)
JD-5234 Fix jethro CLASSPATH issues
JD-5267 JethroClient: Allow multiple statements when passing request as argument
JD-4569 Improve environment settings for JAVA to support out of the box range of Hadoop distribution and Hive configurations

Bug Fixes

JD-3653 Query fails due to corrupted cube read with "Error getting unit no by serial no". Occurs when cubes group by uses 128bit CSH Hash
JD-4945 Server failure due to invalid casting of null in INSERT INTO
JD-5019 Query with complex timestamp expression fail with error (cast issue)
JD-5131 Cubes queues status message may show identical status line more then once
JD-5138 Rebuild cubes may not be re-generated following intensive schema changes
JD-5139 Rename of system table is not fully blocked
JD-5235 Cubes are ignored and not recreated following internal exception ("Error getting unit no by serial no")
JD-5239 CREATE EXTERNAL DATA SOURCE doesn't trim with extra space after name appends space to the name
JD-5252 Server failure when INSERT INTO value contains invalid byte sequence ('\0')
JD-5284 INSERT INTO: exception in connector CSV parser due to line > 4K silently causes incomplete load
JD-5286 INSERT INTO: loader incorrectly casts special floating point values: nan, infinite
JD-5351 Server failure following intesive autentication try when calling openldap method
JD-5398 Cube update will fail following rename of table and cube will not be updated
JD-5415 SHOW FULL TABLES is missing in collect statistics
JD-5446 Maint push "Starting Garbage Collector Cycle" message every second

Minor Bug Fixes

JD-5186 Fix message typo for: referenital integrity optimization
JD-5340 Schedule loads log contains repated occurences of the messages "The version was null, therefore will not read the file"

Version 3.3.2

RELEASE DATE: Dec 7, 2017

- Queries load balancing: skip busy nodes that are queueing queries. Require upgrade of ODBC/JDBC to version 3.5
- Generate and match cubes for count distinct queries without group by
- Allow long identifier names (128 characters)
- Cube generation efficency: persistent too-big statistics
- Important bug fixes

Upgrade Instructions

New Features

JD-4827 Queries load balancing - skip busy nodes

Improvements

JD-3118 Dynamic aggregation: Count distinct aggregation without group by to match/gen cubes
JD-3995 Persistency for cube Too-Big statistics
JD-4675 Optimizer: aggregation queries are rewritten as Group By NULL to support CND cubes match and improve performance
JD-4794 Include insert into SQL details in loader event system table
JD-4978 Allow identifiers length of up to 128 characters
JD-5152 Minor: improve message on too big cube

Bug Fixes

JD-4728 Orphan loader files of dropped tables are not cleaned by garbage collector 
JD-4812 DROP PARTITION requires range values to be between HIGH to LOW
JD-4868 When LDAP Query result contains more than 1000 entries, it doesn't return the results
JD-4887 Plan presents meaningless time for 'EXTERNAL TABLE FETCH' time
JD-4925 Insert into fails if data contains the special charcter '/001'
JD-5044 Scheduled Load rule for jethro$_loader_events is not properly created on instance creation
JD-5055 Manual cube generation fails when multiple cube generations servers are configured
JD-5111 "Illegal construction of TupleSet. Size = 0" on limit 0 query
JD-5161 Non-helpful error return when insert into fails due to invalid data ("trying to flush footer report section more than once")
JD-5170 JethroMaint garbage collector erasing dat files of valid cubes
JD-5171 JethroServer failure upon retry of a query following cube/adaptive entry not found
JD-5175 When log text is longer then 1024 chars, the Loader is killed
JD-5192 SHOW CUBES operation is very slow
JD-5215 INSERT INTO from an external data source may generate defect records in destination Jethro table
JD-5216 Authorization is checked more then once during parsing - may effect query init time
JD-5238 Temporary files are not removed from tmp folder - GC to remove old tmp files periodicly
JD-5279 JethroClient misinterpt "\N" literal in a line. User unable to define "\N" as NULL in new external table
JD-5322 Server crash when calling open ldap method

Minor Bug Fixes

JD-5260 Improve error message in lo

Version 3.3.1

RELEASE DATE: Nov 18, 2017

- Important improvments and bug Fixes

Improvements

JD-5124 Limit max cube/adaptive entry size in byte via config 
JD-5125 Optimize limit 0 and where 0=1 queries (commonly used by Tableau and other BI tools)

Bug Fixes

JD-5118 Table OVERWRITE or TRUNCATE choose lose of granted table permissions
JD-5119 Schema inflate following intensive schema changes and/or table OVERWRITE operations. This can lead to system wide performance issues.
JD-5147 Loader - wrong logic of rejecting files as having unsupported format (zip) based on filename or path
JD-5148 JethorClient -c create CSV file with wrong timestamp values (following 3.3.0 timestamp range expansion)

Version 3.3.0

RELEASE DATE: Nov 5, 2017

- Extended dates range for TIMESTAMP data type:  01/01/1400 - 31/12/9999
- Bug Fixes

Compatibility

Server version 3.3 is NOT compatible with previous version of ODBC, JDBC and JethroManager. All client components must be upgraded.

Compatible client versions: 

ODBC 3.3
JDBC 3.3
JethroManager 1.3.1

New Features

JD-4703 TIMESTAMP data type now support extended dates range (from 01/01/1400 to 31/12/9999)
JD-4545 Refactor optimizer rewrites - various performance optimizations and improve optimizer flow and logging

Improvements

JD-4924 File connector to support UTF-8 encoding
JD-4956 Collect log to collect schedule loads metadata

Bug Fixes

JD-3598 Adding a new Timestamp column to an existing table fails
JD-4501 CREATE TABLE may takes long time to complete
JD-4714 Column name is partially presented
JD-4856 Accuracy issue when cube is used for COUNT DISTINCT over case function when puki optimization is enabled
JD-4870 Schedule Load- "lsof" is called when drop folder is hdfs
JD-4884 Assume referential integrity rewrite doesn't eliminate sequence of unnecessary JOIN expressions on certain scenarios with QLIK function
JD-4902 Accuracy issue when assume referential integrity is enabled and QLIK function is used
JD-4909 JethroAdmin create-instance error msg fixes
JD-4912 Query with duplicate line of COUNT(DISTINCT (CASE WHEN )) is failing with "Request execution failed"
JD-4914 Scheduled load: incorrect recursive handling of files in subfolders under the drop folder when stating specific file pattern
JD-4926 Cubes are not generated and check test failed when cubes are disabled (dynamic.aggregation.enable=0)
JD-4928 Possible accuracy issue with partitions on timestamp if maximum value of partition has fractional seconds
JD-4946 Jethro Scheduled Load - De-Serialization error due to corrupted loadstate*.xml file
JD-4955 Jethro Schedule Loads service should start by default only on one node in the cluster (loader node)
JD-4964 schedule loads fail with log message 'Exception in Thread 'some name': basic_string::erase'
JD-4965 LIMIT and WHERE clauses are ignored for external tables over files data source
JD-4985 After ping failure monitor may start server too quickly after stop causing server start to fail
JD-4996 Cubes auto generation/update requests are not sent due to rare race condition
JD-4998 Accuracy issue with WHERE year(TS) NOT IN (...). NOT operator not applied
JD-5026 Unable to connect to HIVE External Data Source in kerberized environment due to missing path in classpath
JD-5066 External table from HDFS stack forever when no file is in the hdfs path

Minor Bug Fixes

JD-4987 Change "loaded" to "inserted" in INSERT INTO success message

Version 3.2.0

RELEASE DATE: Oct 4, 2017

Major improvements in cubes generation and maintenances:
- Cubes generation and maintenance can now be scaled to more than one server
- Cubes generation and maintenance reorganized and redundant entries removed
- Cubes status tracing logs significantly improved
- New DDL: SHOW CUBES FROM .. can be used to show existing and potentials cubes for query

Improvements

JD-3575 Cubes generation and maintenance scalability - support multi servers cube gen/maint
JD-3931 Reorganize cubes generation and maintenance queues
JD-3969 Null safe join to support functions in ON expression
JD-4121 Dynamically disable/enable cubes scan and generation background cycle via configuration
JD-4330 Push to cube generation and maintenance queues only if a potential cubes can be created and support new DDL: SHOW CUBES FROM
JD-4791 Improvements for log messages reporting cube generation
JD-4843 Main stability - do not exit maint after failures - additional scenarios not covered in 3.0.5

Bug Fixes

JD-4061 Duplicate and redundant entries are pushed to cube generation/update/regen queue causing queue to inflate
JD-4805 Query does not refer to suitable cube which was generated with RIGHT OUTER JOIN
JD-4832 A redented adaptive entry is created for query that finds a cube match
JD-4836 Jethro uninstall should delete $JETHRO_HOME even if it is not empty
JD-4901 Possible lock corruption due to wrong order of locks for schema-lock
JD-4905 Query with CASE fails with "Request execution failed" due to type-mismatch

Version 3.1.1

RELEASE DATE: Sep 28, 2017

Bug Fixes

JD-4861 Query fails with "ADAPTIVE ERROR FingerPrint mismatch"
JD-4899 Order By ignored if asc and desc order both specified after push join rewrite
JD-4910 File connector fails on hdfs files hadoop-hdfs jar missing in classpath
JD-4917 File connector fails on hdfs files with ArrayIndexOutOfBoundsException error 
JD-4919 Create view is failing if now() function exist in the select statement
JD-4920 Flatten query optimization skips left sub-query in join

Version 3.1.0

RELEASE DATE: Sep 19, 2017

- Support external connector for files
- Various connectors related improvements and fixes
- Additional bug fixes

New Features

JD-4572 External connector for files: allow creating an external table over HDFS/Posix files

Improvements

JD-4553 Connectors - manage/expose version
JD-4871 INSERT INTO should to return message with the number of rows inserted

Bug Fixes

JD-4269 ExtTbl: Proper error should appear if the connection to HIVE failed
JD-4716 ExtTbl: Error message following cast failure (reject) in select from external table is unhelpful
JD-4739 JOIN col type mismatch issues unhelpful error message "Undefined function '='"
JD-4773 Errors related to operations over external tables are not raised to caller
JD-4778 Cannot delete system table using the drop system table utility
JD-4782 Ext_table: REJECT_SAMPLE and REJECT_LIMIT_RATIO limits are not enforced and load passes when it should fail
JD-4793 The connector hangs when hive-jdbc driver is not found in classpath
JD-4797 Insert into should return errors from select/load instead of generic failure message
JD-4835 Scheduled Load mechanism not working as expected with post load file action
JD-4839 Failed to create Hive data source Hive on HDP 2.6.1
JD-4866 Getting error "could not find the virtual object of the virtual column" when reading from a cube
JD-4867 LDAP query fails with "operations error"
JD-4869 Query doesn't generate cube and nothing is reported in the logs
JD-4877 Query with HAVING COUNT(DISTINCT ...) fail with error. Index optimization for Group By is broken for this scenario

Minor Bug Fixes

JD-4461 ExtTbl: Fix "Creating an External table not allowed" message
JD-4717 ExtTbl: Reject count log message spaces fix 
JD-4803 Insufficient logging for distributed stmts (e.g. show active queries) - missing info in cases of failure
JD-4852 SHOW EXTERNAL DATA SOURCES is missing under the SHOW help

Version 3.0.5

RELEASE DATE: Sep 3, 2017

Stability improvements and bug fixes

Improvements

JD-4820 Watchdog reset server in case of severe memory fragmentation
JD-4828 Maint stability - recover and skip errors related to specific operations or objects

Bug Fixes

JD-4821 Server appears hung when execution long queries due to memory cleanup cycle lock following dropping objects (tables, partitions or columns)

Version 3.0.4

RELEASE DATE: Aug 20TH, 2017

- Handle various casting issues related to INSERT INTO
- Loads scheduler issues
- Session files versions purge - fix session files scan overhead that caused delay on every request
- Additional bug fixes

New Features

JD-4163 Support inserting values via INSERT INTO ... VALUES (...)
JD-4704 SHOW EXTERNAL TABLES

Improvements

JD-4218 Allow downcast bigint/double to int/float in INSERT INTO 
JD-4695 Support universal downcast type conversion in INSERT INTO (configured off)
JD-4779 Allow casting integer to float in INSERT INTO

Bug Fixes

JD-4520 Trailing spaces trimming by Tableau in where clause literal results in returning empty set - data type is reported as CHAR instead of VARCHAR
JD-4522 Server failure when executing query on view with UNION ALL that contains literals
JD-4638 Temporary files created during load at instance /tmp folder are not always removed
JD-4684 UNION ALL failure due to missing implicit casting of left to right clause 
JD-4706 INSERT INTO request hang due to closing sockets of forked child process
JD-4725 Loader streamer page files cleanup may deletes temp files on concurrent loads
JD-4726 Loader error information is lost following exception
JD-4732 SHOW LOCAL CACHE return SHOW LOCAL CACHE EXTENDED results and vice versa
JD-4756 Scheduled loaded stop execution load jobs and never recover following HDFS errors
JD-4778 Admin utility cannot delete system table
JD-4783 Potential failure of Jethro Loader Scheduler service if error encountered during meta data read
JD-4785 Potential error in Jethro Loader Scheduler - file not close in case of read failure
JD-4786 Potential minor memory leak when reading a file in Jethro Loader Scheduler
JD-4788 Number of session files reach thousands due to accumulation of previous versions causing significant delay for every query when session is accessed. Previous versions of session files should be deleted by GC

Minor Bugs

JD-4591 Maint is getting stopped due to corrupted lock file without appropriate error message
JD-4610 Unclear error message on SHOW EXTERNAL DATA SOURCES when result is empty
JD-4713 Misspelled error messages

Version 3.0.3

RELEASE DATE: Jul 13TH, 2017

- Support LDAP authentication over SSL/TLS
- Support cubes for optimized union all flat queries
- Expend external connector environments support and improve Hive connector performance
- Scheduled loader can now run concurrent loads
- Performance improvements for queries grouped by datepart functions
- Important bug fixes related to cubes generation and cubes update that significantly improve cubes maintenance efficiency and reduce cube generation/update queue size
- 30+ additional bugs fixes

Improvements

JD-2922 GROUP BY optimization: use functional indexes for all datepart functions and complex timestamp functions expressions
JD-3872 Scheduled loads to support concurrent loads into different tables
JD-4400 Hive connector: performance improvements for fetch from external Hive table
JD-4557 Support LDAP authentication over SSL/TLS
JD-4570 Generate/match cubes for UNION ALL flat left/right queries following GB/AGG push down
JD-4595 External data source connector to support Java 1.7
JD-4621 External data source connector MapR support

Bug Fixes

JD-4005 Loader failure with core when writing the system table
JD-4190 Parameters BIGINT values always casted to INT
JD-4254 Maint resend cube update requests for cubes known to be too big
JD-4276 ExtTbl: show active queries failed while running an external query
JD-4326 Scheduled loads: list of file to load is not refreshed on next try following load fail
JD-4394 Union will fail if one side is integer literal and the other side is integer expression
JD-4397 Missing info in log: cube generation server port number not shown in maint log
JD-4402 Performance degradation in some queries with QLIK function (from version 3.0.0)
JD-4438 ExtTbl: missing supported casting operations in Hive connector
JD-4447 Show active queries and kill queries security deficiencies
JD-4452 ExtTbl: UNION ALL query from external table fails on internal error
JD-4462 Internal Fix for requestId handling
JD-4472 ExtTbl: rejected row count on FILL_NULL is wrong
JD-4476 Internal Fix in thread exception catching
JD-4485 GROUP BY query with hi cardinality count distinct aggregation from left join return wrong results when using partition level count distinct optimization
JD-4493 ExtTbl: UNION ALL query from external table fails with invalid column count error
JD-4505 Jethro Server port is not released following JethroServer restart during INSERT INTO operation
JD-4509 SQL syntax error cause server failure
JD-4511 Internal Fix: wrong assumption of argument of type ColumnCopy
JD-4515 Server failure upon intensive DDL operations
JD-4517 QLIK query over a view or sub-query cause server failure
JD-4529 Insert Into - Loader process is not always terminated when Server process stopped
JD-4539 Cubes are generated for system tables - should be avoided
JD-4558 Failures in delete obsolete cubes cause maint service shutdown
JD-4562 When updated cube is too big the subsequent cube updates commands fail to update the cubes
JD-4567 Server failure when query with count(*) from sub-query fetch the sub-query results from adaptive
JD-4568 Server failure when query where clause include unsupported sub-select = operation
JD-4614 IN/NOT IN will fail when checked against a list with aggregated values
JD-4618 Accuracy issue when reading sub-query from adaptive following specific push down aggregation optimization scenario
JD-4619 Cubes which needs update are sent several times to execution by maint service
JD-4623 Explicit IN subquery down cast behaves abnormally
JD-4626 Comparing literal to IN list cause server failure
JD-4627 Update cube fails when a schema change occur during cube update operation
JD-4640 Cubes generation flow does not detect that a cube was marked as too big and re-execute the cube query
JD-4666 LDAP group lookup fails for ldap user having distinguished name with special characters ("," "\" etc...)

Version 3.0.2

RELEASE DATE: Jun 6TH, 2017

- Improve Hive connector functionality and performance: with native Hive JDBC driver used by the connector advanced Hive JDBC URL syntax is now supported and the connector now works over Kerberized HDFS. Additional performance improvement is gained if using latest Hive JDBC.
- Various security, external table and other bug fixes

Improvements

JD-4482 Connector use native Hive JDBC available from the environment. Effect: connector can run on Kerberos environment and connector performance is improved

Bug Fixes

JD-4007 Query using range Index hangs
JD-4395 Count distinct query fails when matching cube with other aggregation
JD-4416 LDAP group name validation is case sensitive
JD-4419 Views authorization fails when underlying table is not authorized
JD-4426 Upgrade falsely generate additional copy of ssh authorization key
JD-4437 Accuracy issue for self-join queries with function on target list
JD-4440 JDBC packed with rpm should be upgraded to protobuf 2.5 (jdbc 2.0)
JD-4443 Multiple auditing log messages for single audit
JD-4473 External Tables: Server failure following INSERT INTO error when connector fails to run
JD-4475 CLONE of JD-4428 for 3.0.2
JD-4479 INSERT INTO reach deadlock in rare cases if executed following DROP TABLE/COLUMN 
JD-4481 Service fails to start over Hortonwork environment following improper JAVA_HOME declaration
JD-4487 CLONE JD-4486 for 3.0.2
JD-4490 Drop on view set another view to "invalid" when column description is not same as column name
JD-4498 Server failure when receiving query with invalid syntax (brackets around target list)

Version 3.0.1

RELEASE DATE: May 23TH, 2017

Improved Hive connector performance enabling accelerated load from external tables and various connector related bug fixes.

Improvements

JD-4532 Hive connector Performance improvements

Bug Fixes

JD-4430 External Tables: Where fails when casting external table query from different types
JD-4442 External Tables: Reject FILL_NULL count limit is ignored
JD-4453 External Tables: kill query of "insert into" failed to stop the load from hive
JD-4459 External Tables: Insert into failure due to casting issue with "Invalid position specified for group-by column"

Version 3.0.0

RELEASE DATE: May 11TH, 2017

- Enhanced security with complete LDAP authentication and authentication with standard hierarchical permission model.
- External tables mapped to Hive table using Jethro-Hive connector.
- Load data using external table via standard INSERT INTO SELECT command.
- Self Join support.

New Features

JD-3077 Support INSERT INTO SELECT
JD-3632 LDAP authentication (release)
JD-3739 Support EXTERNAL DATA SOURCE and EXTERNAL TABLE
JD-3755 Security: Authorization Support (Permissions, Roles, Groups)
JD-3887 External Connector for HIVE
JD-3963 Security: Auditing

Improvements

JD-1217 Support self join
JD-1275 Sequence based result set column names for unaliased expressions in target list
JD-2782 New NS Component (internal)

Bug Fixes

JD-4333 Queries with literals only are blocked
JD-3211 Qlik Query failes due to duplicate expressions
JD-3435 Add column Process ID (PID) to the loader events system table
JD-3557 Queries with column in order by that is not found in group by should be rejected
JD-3566 Create and drop table not allowed by odbc
JD-4005 Core in Loader when building global UKI
JD-4247 Problem installing Ubuntu package
JD-4251 Timestamp literals from query appear in log as unix timestamp instead of date-time literal
JD-4265 Enable fallen query rewrite by default (flatten.query.rewrite.enable=1)
JD-4319 Multiple consecutive spaces in query literals are trimmed

Version 2.1.4

RELEASE DATE: Jun 6TH, 2017

Improvements

JD-4280 Allow execution of literal only queries

Bug Fixes

JD-4428 Jethro Maint failure following drop of invalid cube in cubes regeneration scenarios
JD-4486 Cubes regeneration is not initiated following subsequent schema changes

Version 2.1.3

RELEASE DATE: May 23TH, 2017

Improvements

JD-3354 Optimization: expand push where support for OUTER JOIN and CROSS JOIN

Bug Fixes

JD-2741 Various issues with literals used as group by dims and/or aggregations over literals (allowed internally following rewrite)
JD-4256 Cubes too-big list is ignored and not used to accelerate future cube generation requests

Version 2.1.2

RELEASE DATE: Apr 12TH, 2017

Optimizations for group by/aggregation queries on top of UNION and bug fixes mostly related to QLIK function.

Improvements

JD-4144 Support cube generate execution hosts "auto" to automatically set it to local host and known server port
JD-4181 Optimization: Push down GROUP BY and/or AGGREGATIONS to flat sub-query with and without UNION ALL

Bug Fixes

JD-3424 Service command report stopped jethro services as "already running" if instance name contained in another running jethro service
JD-3596 Query execution hangs following dynamic joined table switch on complex join with IN sub-query
JD-3814 Query return error following cubes match failure due to duplicated measures
JD-4009 Garbage collector false reports deletion of keys files
JD-4072 Call to SHOW FULL COLUMNS fails with null deredrench error when encountering invalid views
JD-4083 Query find a cube with duplicate measures but the cube is not used by planner (adaptive entry not matched)
JD-4118 Cube match internal error fails query instead of skipping cube match
JD-4142 Qlik set analysis with assume referential integrity using TOTAL and join fails with error 
JD-4147 Qlik set analysis filter expression fail to match literal with upper case letters (since version 2.1.0)
JD-4179 Adaptive cache with QLIK function - Query returns wrong results from adaptive cache in cases query contains duplicate measures
JD-4183 Throttling in high concurrency scenarios and queries delayed when schema altering command are executed
JD-4212 Query with wrong QLIK function syntax cause server failure 
JD-4213 Qlik query fails with "position" exception when duplicate measures encountered

Version 2.1.1

RELEASE DATE: MAR 20TH, 2017

Use tcmalloc allocator and bug fixes.

Improvements

JD-3558 use google allocator – tcmalloc

Bug Fixes

JD-4074 Following schema changes all queries are resent to cube generation queue. This is redundant and cause unnecessary cubes generation operations
JD-4021 Maint service minor memory leak created following JD-3852 fix in 2.1.0
JD-4109 Maint service major memory leak related to adaptive rank file read/update

Version 2.1.0

RELEASE DATE: MAR 3RD, 2017

Support cubes regenerate following tables, cubes operational improvements and bug fixes.

New Features

JD-3851 Regenerate cubes following table overwrite

Improvements

JD-3852 Accelerate adaptive space cleanup by deleting invalid entries before adaptive space is full
JD-3854 Cube related logs message improvements

Bug Fixes

JD-3847 Missing system table load event with current status for failed loads
JD-3853 Invalid de-ser debug message filling log
JD-3921 JDBC\ODBC – Query failed. Reason: Server failed to find session
JD-3928 Cube update cause cube generation server failure
JD-3929 Query with union all and that include repeated literals on one side of the union causes server failure
JD-3944 Server fails to deserialize huge cubes with count distinct fetching in on demand mode
JD-3967 Server failed to find session after 2nd call to SHOW ACTIVE QUERIES
JD-4006 Query with CURRENT_TIMESTAMP sent from Tableau fails with syntax error
JD-4017 when the .dat file is removed from adaptive entry, the size of the entry should be zero in show cubes command

Version 2.0.5

RELEASE DATE: JAN 31TH, 2017

Including LDAP authentication (beta), optimization for union queries, additional minor improvements, bug fixes and

New Features

JD-3632 LDAP Authentication (beta)

Improvements

JD-2855 Rewrite: push where predicate to union all
JD-3668 Add timestamps to dmesg output
JD-3669 Add show version to collect logs sql script
JD-3698 Auto cubes: allow cube to match agg(if…) expressions which are decomposed when there is more than one column in the condition
JD-3742 Send earlier status from JethroLoader to the system table
JD-3813 Add SHOW CUBES to collect logs
JD-3817 Add monitor.log to collect logs
JD-3843 Add crontab list to collect logs

Bug Fixes

JD-3670 Select with count(distinct) over cube can cause server failure
JD-3687 Cube matching stops following exception at reading schema
JD-3704 Mishandling of null values in the JDBC driver
JD-3741 Generate cubes – ORDER BY mistakenly added to the cube expression
JD-3750 Cube generation queue is limited to 1000
JD-3751 Update cubes – ID 0.0.0 is mistakenly printed to log as an indication for every query execution
JD-3752 Update Cubes – Update cube should be executed for each key (the cube) instead of executing the original query
JD-3753 Update Cubes – When changing entry type to CUBE we original query SQL is not updated
JD-3754 CASE with aggregation function expression in condition/results fail with “function not found” exception
JD-3762 Jethro ODBC, wrong vendor name.
JD-3771 JethroAdmin create-instance’ doesn’t start the instance automatically when performed using ssh
JD-3785 JethroMaint service failure with: “Too many open files…” messages
JD-3787 Push where wrong write when NOT operator is above AND – possbile occuracy issue
JD-3789 Select with count(distinct) using per partition UKI optimization can cause server failure
JD-3799 Upgrade ODBC SDK
JD-3805 ODBC minor fixes
JD-3806 Tableau TDC – jethro requires explicit cast of null literals
JD-3831 Loader – $LOAD_START load wrong values
JD-3836 Adaptive is not copied to local cache (adaptive fetch to local cache is enabled as global parameter)
JD-3840 Adjust Jethro ODBC to support known escape sequence for outer join {oj ….}
JD-3849 Significant memory leak overtime when enabling cubes

Version 2.0.4

RELEASE DATE: JAN 4TH, 2017

Instruction for upgrade existing instances to 2.0.4

Version 2.0.4 enables system tables events creation by default. The mandatory requirement for system tables is to have loads scheduler services (JethroLoadsScheduler) run automatically. The service must be started and enabled to run automatically as follows:

1. Edit the service.ini file and set the third parameter for all active instances to yes
   > vi $JETHRO_HOME/instances/services.ini
   –> {instance-name}:port:yes:yes:yes

2. Start the Jethro loads scheduler service:
   > service jethro start {instance-name} loadscheduler

New Features

JD-3593 Enable system table events creation by default

Improvements

JD-2784 Flatten query write: remove sub-queries and merge it into the upper query to reduce multi-level queries overheads and avoid where on external queries (optimization not enabled by default)
JD-3591 Count distinct optimization for high cardinality columns: evaluate count distinct per partition to reduce memory footprint and enable using fast bitmap based evaluation
JD-3697 Show cubes – add status column

Bug Fixes

JD-2873 Push where – not performed if contains IN clause
JD-3173 Fix issues with local cache statistics
JD-3129 Push where – Potential server failure following applying push where optimization
JD-3607 Slow query execution with count distinct
JD-3677 Push where – serve failure following partial push where (one of the where expressions is IN expression)
JD-3691 Accuracy issue: when where expression contains NOT IN and join index defined on the NOT IN column
JD-3700 Adaptive cache – adaptive cache results not being invalidated after dim table append

Version 2.0.3

RELEASE DATE: DEC 16TH, 2016

Bugs fixes and improvements.

Issues list:

Improvements

JD-3329 Utility to reconstruct column index from column data
JD-3622 Expand cubes with where support to allow complex WHERE clauses
JD-3662 Generate cube – additional informative message in jethroserver.log when cube is not generated
JD-3675 Enable hi performance count distinct execution mode via count distinct container for CSN128 and Value Hash group by plans

Bug Fixes

JD-3297 Query with HAVING is not finding a cube match
JD-3404 Cube not found by Cube Generator while cube gen query sent to server find a cube
JD-3597 Query hangs – join with right table is sub-query that returns very big result set
JD-3617 Server failure due to query retry as a result of adaptive fingerprint mismatch exception
JD-3649 Memory leak on cubes generation
JD-3654 Adaptive incremental mode not active if new partition was added
JD-3664 Local cache does not update itself with new events (since 2.0.2)
JD-3681 Memory leak at count distinct queries using count distinct container

Version 2.0.2

RELEASE DATE: NOV 25TH, 2016

Bugs fixes and minor improvements.

Issues list:

Improvements

JD-3401 ODBC: Enhancements for copy TDC file in ODBC installation

Bug Fixes

JD-3458 Jethro server failure during active load while frequent queries sent to the system table (adaptive problem)
JD-3606 Maint service fail and stop due to problem in garbage collection
JD-3608 Maint service fail to initialize in case it fails to commit session configuration
JD-3619 ODBC: fix issue that cause Tableau to fail when disconnect is called without connect

Version 2.0.1

RELEASE DATE: NOV 17TH, 2016

Fixes for 2.0.0 known limitations/bugs.

Issues list:

Improvements

JD-3574 Stop/start of cubes generation/update via set command (no maint restart required)

Bug Fixes

JD-3416 Cubes with avg(if) are not matched
JD-3585 Auto cubes – read from cube accuracy issue when two different aggregations encapsulate an identical CASE/IF expression
JD-3586 SHARED.WHERE WARN Release shared object cannot found a relevant object in map
JD-3587 Auto cubes – read from cube accuracy issue when AVG aggregation encapsulate CASE/IF expression
JD-3588 JethroClient does not exit gracfully after failure (exception handling issue)

Version 2.0.0

RELEASE DATE: NOV 15TH, 2016

Jethro 2.0 introduces auto micro cubes.

Jethro auto micro cubes brings the great performance value of pre-aggregated cubes, combined with the ability to automatically generate and transparently maintain those cube.
Cubes can be useful to accelerate performance for certain set of queries, mainly queries that are not highly filter and would normally scan large parts of the data set. Jethro optimizer combines cubes with indexing and caching technology to ensure interactive queries response time on wide range of use cases.

Issues list:

New Features

JD-2590 Auto Micro Cubes – Match and rewrite dynamic aggregations
JD-2609 Adaptive incremental – Expend support for additional queries types (CSH Hash)
JD-2786 System Tables
JD-2788 System Table for loader events
JD-2919 Auto Micro Cubes – Dynamic aggregation of count distinct queries
JD-2921 Auto Micro Cubes – Support hierarchy levels for timestamp columns
JD-2937 Auto Micro Cubes – Auto generate dynamic aggregations from incoming queries
JD-3110 Auto Micro Cubes – Manual cubes generation via GENERATE CUBES
JD-3111 Auto Micro Cubes – Support cube with where
JD-3112 Auto Micro Cubes – Find match for aggregators that has internal case expression
JD-3118 Auto Micro Cubes – Match count distinct aggregation queries without group by to cubes
JD-3411 Auto Micro Cubes – Allow aggregation queries without group by and with where to generate and match cubes
JD-3345 Auto Micro Cubes – Generate and match cubes for queries without measures
JD-3348 Auto Micro Cubes – SHOW CUBES and related cubes management commands

Improvements

JD-2838 Enable fetch and read adaptive cache files to/from local cache
JD-2839 Add section separators to the jethro loader report file
JD-2878 Add watchdog for JethroMaint
JD-2918 Optimized count distinct for CASE/IF in distinct aggregator
JD-2938 New instances parameters should be set with Yes as default in Service.ini
JD-3043 Count Distinct Optimization: Allow push group by functions and joins to upper query with count distinct using distinct values bitmaps and support adaptive incremental for count distinct
JD-3085 Improve adaptive write efficiency via large write chunks
JD-3183 Adaptive cache read: split each column read according to the size of current TupleSet parameter
JD-3210 Update incremental cubes by maint
JD-3285 Show active queries – add new option to status column: caching

Bug Fixes

JD-2204 Server failure when query is VERY long and includes text after “;”
JD-3137 Memory leak – related to IN with sub-query in outer query
JD-3142 Memory leak of Global UKI
JD-3363 Possible deadlock/leak in buckets merge
JD-3370 Join index not used when where clause contains OR not null
JD-3379 Server failure when query reads all adaptive columns into single buffer
JD-3467 Query failed with error – limit issue
JD-3507 Jethro server failed to execute query with IN literal left operand
JD-3514 Jethro Server failure – query hit partition holder internal error

Version 1.6.11

RELEASE DATE: NOV 9TH, 2016

Bug Fixes

JD-3461 Fix for JethroAdmin list-storage-instances
JD-3544 Query with UNION ALL as subquery stored in adaptive cache failed to update on data load/update
JD-3425 Loader doesn’t reject tokens with blank spaces loaded into numeric columns
JD-3524 Fix server failure in case there is filter with IN expression and a query on the right side of IN has filter of its own
JD-3438 Fix local-conf typos

Version 1.6.10

RELEASE DATE: OCT 12TH, 2016

Improvements

JD-2910 Copy TDC file to the right place upon ODBC installation
JD-3068 SHOW PARAM: present changed parameters modification level including set global and support UNSET command
JD-3239 Add driver version (odbc, jdbc, JethroClient) to JethroServer log
JD-3277 Improve loader performance (streamer)
JD-3282 Collect logs should now collect revisions of schema.xml
JD-3290 Update status codes returned from the ODBC
JD-3298 Loader performance improvement – low cardinality keys
JD-3330 SHOW TABLE COLUMN: reduce memory consumption when there are multiple high cardinality columns
JD-3389 Collect-logs – add instance name and date to log files and improve error handling

Bug Fixes

JD-1506 IN/NULL issue: NOT IN with NULL value in IN list should not return the NULL value
JD-2690 IN/NULL issue: IN expression with list of literal doesn’t ignore NULL in the list
JD-3108 Adaptive cache error in log: “failed to get schema object”. Not an error – should be trace message
JD-3318 ORDER BY return wrong order when sorting positive and negative numbers
JD-3319 UNION ALL – duplicated columns
JD-3320 Session and global parameter files potentially can be corrupted if SET commands issued concurrently
JD-3336 Inconsistent EFS list operation cause wrong read of metadata and valid columns removed by GC. Fix version file validation check
JD-3351 UNION ALL with duplicate literals – duplicated columns when reading from adaptive
JD-3364 Partition size limitation of 4B rows due to file identifier rowID wrap around
JD-3376 Collect logs password is not provided and no appropriate error is issued
JD-3378 Server failure for queries with duplicate aggregators when reading from adaptive in incremental scenario
JD-3381 odbc 32 on windows 7: Cannot create a DSN

Version 1.6.9

RELEASE DATE: SEP 6TH, 2016

New Features

JD-3092 KILL QUERY – interactive kill of long running query

Improvements

JD-3175 Add JDBC support for getTimestamp(calendar) and item MergeColumnHeaderCells methods
JD-3209 support ORDER BY degrees (ordinal number)
JD-3215 Optimization for string functions in where: add optimization for IN (func(col) IN (…))
JD-3280 Support queries selecting literals without FROM clause

Bug Fixes

JD-2663 Wrong result when using cast as “float” with quotes
JD-2769 JethroClient issue error when run in the background
JD-3091 Query cancel does not terminate certain queries
JD-3124 Lock files are not removed
JD-3142 Memory leak of Global UKI
JD-3205 Wrong results: where optimization for string function return treat NULL return value as true
JD-3267 DDL commands fail after drop of dim table (drop table, show tables extended, create join index,…)
JD-3272 Loader and Maint fail following upgrade from 1.4.6 to 1.6.8 (fix leftover files handling)
JD-3276 Server failure during query following partition overwrite

Version 1.6.8

RELEASE DATE: AUG 11TH, 2016

New Features

JD-2787 Scheduled Loads (beta)

Improvements

JD-2930 Add command JethroAdmin list-storage-instances
JD-3016 JethroAdmin collect-logs support password entry
JD-3050 Limit memory factor to avoid huge memory consumption for base parameters
JD-3097 Set instance name max size to 32 characters
JD-3198 Allow Jethro installing with a user other then jethro

Bug Fixes

JD-2998 Server failure if order by column appear in group by inside a function
JD-3011 collect logs does not check instance name is valid – reports success
JD-3071 Illegal query combining Distinct and Order by causes server failure. Should issue an error.
JD-3093 Rare race condition on high concurrency causes all queries to wait on query queuing until time-out exceeded
JD-3114 ODBC: return true max column length for string
JD-3138 Fix threads counter to include all threads including top query thread

Version 1.6.7

RELEASE DATE: JUL 20TH, 2016

Bug Fixes

JD-3104 JethroMaint service stops after DROP PARTITION command is issued. This problem existing only in version 1.6.6 and affect only tables loaded only with version 1.6.6 Loader.

Version 1.6.6

RELEASE DATE: JUL 15TH, 2016

Improvements

JD-2530 Move Loader report log, profiles and rejects files for specific log folder
JD-3020 Allow setting loader job name and include it in loader report file name
JD-3048 TDVT: Support required function: Right(string, int)
JD-3059 Optimize query with group by and count distinct on the same column
JD-3060 Reduce PLANNER time for queries with very high cardinality (improve global UKI access)
JD-3063 Optimize Join performance: dynamically switch left/right tables in join node during query execution
JD-3076 Control and reduce memory footprint for instances with large number of string columns (reduce small initial hash size)
JD-3102 Updated Tableau TDC file

Bug Fixes

JD-2373 Jethro server stops/ when local cache is full and stops again to restart
JD-2961 Local cache won’t replace files after cache space is filled / stop fetching new files even when the is available space
JD-3014 Server failure when receiving query over empty table with function on a string under WHERE
JD-3015 Local cache copy files of tables that were dropped
JD-3022 Local cache metadata files potential unsafe commit
JD-3028 TDVT: issues with Locate and Substring functions taking Floor function as argument
JD-3029 TDVT: TIMESTAMPADD fails when getting interval of type Double
JD-3039 TDVT: Change LOG() to compute LN() and add LOG10() to apply to Tableau standards
JD-3066 TDVT: Wrong results – empty set when using cast of string representing real number to integer
JD-3074 ODBC does not signal exception when it occurs during accumulating results from server
JD-3080 Query on empty table using string functions fail with “Illegal construction of TupleSet. Size = 0” error
JD-3081  JDBC problem with getObject() API: Erroneous NULLs appears in a results set rows
JD-3082 SHOW LOCAL CACHE cause server failure when local cache path not defined
JD-3084 Query with UNION and COUNT DISTINCT fails with “unsupported expression type: 22”
JD-3086 Server failure on rare scenario when executing query on empty tables
JD-3095 SHOW LOCAL CACHE – shows inaccurate statistics
JD-3096 lock ignored as stale though was not actually stale – increase stale time out

Version 1.6.5

RELEASE DATE: JUN 27TH, 2016

Improvements

JD-2464 Ignore range in WHERE that includes all the values
JD-2962 SHOW LOCAL CACHE – make more usable: show cache/touched/storage information per table
JD-2987 Support round with precision
JD-3010 Improve fetch statistics logs
JD-3032 Loader streamer performance: improve rejects handling performance

Bug Fixes

JD-1734 Query CANCEL: fix scenario were query is not stopped when cancelled
JD-2654 Corrupted lock file issue causing locking to fail consistently
JD-2704 Query file if target list in select clause doesn’t include at least one column (for queries with having clause)
JD-2706 Case function can’t handle comparison between BIGINT and DOUBLE
JD-2760 Crash in SHOW LOCAL CACHE EXTENDED
JD-2761 Local cache disk full cause server crash on init
JD-2953 JethroServer – configuration values are transformed to lower case (problem for path values with upper case)
JD-2965 ODBC client: fix scenarios were query is not stopped when cancelled from ODBC
JD-2986 abs function not found
JD-2989 ODBC failed to handle group by boolean (fixed in ODBC 1.7)
JD-2994 ODBC 32bit doesn’t use TDC
JD-3004 Tmp files not deleted from instance-storage/tmp
JD-3024 Query with sub-query that contain tables from the outer query fail (server failure). Related to push where
JD-3053 Query with count distinct on QLIK function fail (server failure)

Version 1.6.4

RELEASE DATE: JUN 9TH, 2016

Improvements

JD-2972 Trim CASE/IF branches with false conditions
JD-2973 JethroAdmin collect log: force instance name and create collect log file name with date and time
JD-2985 Rewrite: additional clean-up for simple math operations in SUM aggregate
JD-2988 Improve adaptive read efficiency
JD-2995 Aggregation optimization: clean cast as date for min/max aggregations to achieve index based aggregation

Bug Fixes

JD-1552 Server crash on group by query following drop partition or failed load if number of unique strings loaded gt number of unique values in the string columns
JD-2933 Join with Union all returns bad results when adaptive is turned on. (result of col1 is duplicated to col2)
JD-2994 Tableau with ODBC 32bit doesn’t use TDC
JD-2996 Wrong results: filter on right table column from left join is ignored following push where optimization (JD-2785)
JD-3000 Server crash on Query that cast BIGINT to STRING
JD-3008 Adaptive read issue
JD-3009 Adaptive errors on WHERE IN query

Version 1.6.3

RELEASE DATE: MAY 26TH, 2016

New Features

JD-2917 Bitmap based GROUP BY COUNT DISTINCT. X5-X100 performance acceleration for many scenarios
JD-2928 Support SQL_TSI_FRAC_SECOND for timestampadd/diff and atan2 (Tableau requirements)

Improvements

JD-2619 Print modified configuration parameters at jethroserver.log start
JD-2840 Write adaptive indexes maps asynchronously. This allow keeping larger adaptive index maps.
JD-2943 Remove star transformation limitation for fact and dim
JD-2949 Fix fetch and execution tuple set sizes calculation to allow more parallel processing
JD-2951 Improve performance for where with functions on string columns

Bug Fixes

JD-2914 Adaptive write for large tuple set is exteriorly slow – need to optimize inefficient implementation
JD-2924 Issue with global configuration. Server fails to init after setting global config
JD-2940 Query failure: UNION ALL inside IN sub-query
JD-2957 TIMESTAMPDIFF – Accuracy issue when swapping the literals inside the function
JD-2958 Wrong expression ID allocation when rewriting aggregation (sum(-C1)) (was Internal Error)
JD-2968 TIMESTAMPADD – microsecond – Accuracy issue when using negative number in TIMESTAMPADD
JD-2975 Remove show local cache from collect-logs (pending show local cache fix)
JD-2980 Shared warn potential issue: release shared object cannot found a relevant object in map

Version 1.6.2

RELEASE DATE: MAY 6TH, 2016

New Features

JD-2669 Loader support hex value input to INT and BIGINT data type

Improvements

JD-2785 Rewrite: push where clause from outer query to inner query (optimization for tableau custom query)
JD-2865 Range Index – rewrite with IN to compute OR of index maps concurrently and to add to adaptive index
JD-2880 Functional Index – rewrite with IN to compute OR of index maps concurrently to add to adaptive index
JD-2903 Join performance: avoid copy of qualification columns during join execution
JD-2913 Performance – optimize GROUP BY over YEAR/MONTH/DAY functions with CAST to DATE

Bug Fixes

JD-2817 Various issues with transform of expression to SQL
JD-2899 Range index – block range index creation on data types other the INT and BIGINT
JD-2915 “CURRENT_TIMESTAMP” with quotes is not identified
JD-2916 Server crash on partition pruning while running now() on hours

Version 1.6.1

RELEASE DATE: APRIL 27TH, 2016

Improvements

JD-2867 Enable adaptive read from sub-queries
JD-2893 Performance – Force to push JOIN if there is where on the DIM and fact doesn’t include aggregations

Bug Fixes

JD-2895 Join index not used with QLIK function creating specific filter
JD-2901 Push join doesn’t push because wrong calculation of potential groups
JD-2904 Truncate table fails after load overwrite finish unsuccessfully
JD-2907 Fail to read from view (adaptive read fail) – getting CHECK failed: (index) lt (size())
JD-2911 Adaptive cache – sub-query results written to adaptive cache if sub-query is a view (should be disabled)

Version 1.6.0

RELEASE DATE: APRIL 20TH, 2016

New Features

JD-2811 Range indexes – optimize range filtering using indexes on ranges of values
JD-2847 Shared Where – cache where results for reuse by other concurrent queries that issues the same where clause

Improvements

JD-2662 Rewrite: Basic cleanup of functions in aggregators
JD-2863 Set ulimit by Installer to support high concurrency
JD-2869 Allow drop of multiple partitions in one operation
JD-2886 Performance – CAST to DATE in GB is not optimized

Bug Fixes

JD-2780 Partitions not pruned if same table appear more then once in the query
JD-2834 Query with left join return empty set when right table is empty
JD-2846 Local cache auto refresh doesn’t fetch new data if server was running before load start and schema was changed
JD-2848 Query with star transformation fails (issue with join remove)
JD-2854 Join column display order – on left join left table columns should be before right columns
JD-2859 Fetch dispatcher – constructor thread leak
JD-2861 JethroData::Lock deadlocks under intensive use
JD-2871 Fetch from column after adding it with add column fails
JD-2875 Pre-query operations are not measured in query total time
JD-2888 JethroLoader – first block size in each partition is less then blocksize

Version 1.5.2

RELEASE DATE: APRIL 4TH, 2016

New Features

JD-2745 Loader: support single digit TIMESTAMP format for month / day / hour / minute / sec
JD-2791 Show version command

Improvements

JD-2770 Loader: Report of success or failure to console
JD-2800 Optimize filter where: avoid processing blocks skipped for push down predict scenarios
JD-2807 Optimize filter where: remove HAVING node from filter where
JD-2819 Change default fetch tuple set size configuration to reduce memory allocation during fetch
JD-2823 set CAP_FAST_METADATA in TDC

Bug Fixes

JD-2461 Wrong results – rare potential error in adaptive delta: if not all partitions updates or if partitions are dropped and added
JD-2675 Cast to VARCHAR fails
JD-2759 Tableau doesnt uses CASE to convert boolean to integer – ODBC failes (due to JD-2726)
JD-2768 Empty set returned from adaptive cache following unexpected client disconnection
JD-2774 Performance: allow group by push function rewrite with count distinct aggregator if column in function already appear in group by
JD-2779 PushDown group by optimization fails on group by functions
JD-2814 Server failure on query with IN expression within function (CASE) inside the WHERE clause
JD-2822 JethroServer does not report fatal initialization errors to console (e.g. port busy)
JD-2825 Query failure (aggregation in CASE function)

Version 1.5.1

RELEASE DATE: MARCH 21TH, 2016

New Features

JD-2551 Add new functions: round(), sign(), sqrt(), space(), ascii(), tan(), cot(), atan(), sin(), asin(), cos(), acos(), degrees(), radians(), pi(), exp(), ln(), log(), replace()

Improvements

JD-2589 Support Redhat/Centos 7.2 (resolve service start/stop issue)
JD-2607 Optimization rewrite: Push multiple joins in group by to upper query
JD-2720 Add support for millisecond period in functions timestampadd() and timestampdiff()

Bug Fixes

JD-2783 Server failure on query IN clause inside aggregator
JD-2790 Results fix for queries with cast(func(..)) target list
JD-2796 Join Index – set index block size in relation to data block size
JD-2798 Now() function return current date without hours/minutes/seconds

Version 1.5.0

RELEASE DATE: MARCH 14TH, 2016

New Features

JD-2757 Support char() function

Improvements

JD-2378 Significantly improve high frequency small load support: data model block sizes is unaffected by load size

Bug Fixes

JD-2740 Union all fails or wrong results with order by or limit clauses
JD-2781 Query stuck when all partitions are pruned and select target list count() aggreagtor

Version 1.4.6

RELEASE DATE: MARCH 13TH, 2016

Improvements

JD-2754 Loader performance improvement – create global UKI only for updated partitions when finalizing

Version 1.4.5

RELEASE DATE: MARCH 8TH, 2016

Improvements

JD-2615 Decrease CROSS JOIN memory usage
JD-2765 Optimization: allow trying star transformation without where optimization for STRING quals if number of keys is small

Bug Fixes

JD-2489 Error when Tableau is connecting to an EMPTY Jethro instance
JD-2531 JDBC causes java.net.SocketException
JD-2575 Loader option LINE TERMINATED BY doesn’t work. Option disabled
JD-2598 CSN node execution time is higher than expected
JD-2660 JDBC – bad error message when no instance is specified in url
JD-2711 Server error: Local cache manager exit
JD-2721 Loader report – improperly printed reject file name
JD-2729 Double slashes in local cache path will prevent auto filling cache
JD-2751 Default value of query.optimizer.grouping.hash.base.size (10m) is too big and can add few seconds to query time due to large mem allocation
JD-2753 local-conf does NOT recognize parameters with capitalized letters and does not report them as wrong
JD-2764 Functional index for date/time optimization doesn’t work when timestamp literal is left of the column identifier
JD-2771 Exception on failure creating new thread not handled properly
JD-2772 Load table in overwrite mode may create wrong join indexes if wrong (non-default) block size settings are used

Version 1.4.4

RELEASE DATE: MARCH 1ST, 2016

Improvements

JD-2604 Add SQL Statistics to JethroAdmin collect-logs

Bug Fixes

JD-2760 crash in SHOW LOCAL CACHE

Version 1.4.3

RELEASE DATE: FEBRUARY 29ND, 2016

New Features

JD-2723 Support char(int or float) – TABLEAU only
JD-2744 Support curdate() function

Improvements

JD-2034 Improve fetch performance: decompression performance analysis and improve
JD-2719 Handle currently ODBC cast syntax for DATE “{ D …} and/or support cast to DATE with DATE keyword
JD-2726 Get from Tableau standard expressions instead of CASE WHEN boolean expression to gain index performance boost
JD-2735 Enable returning table and column identifiers as uppercase strings (configuration based)
JD-2746 Queries with now and date can be saved in adaptive cache since values are computed and saved as literals

Bug Fixes

JD-2734 concat doesn’t support concat of ‘/n’ sent from tableau
JD-2742 Query grouped by function with 2 or more identical aggregators fail
JD-2743 Query grouped by function and literal fails

Version 1.4.2

RELEASE DATE: FEBRUARY 22ND, 2016

Improvements

JD-2684 Support IFNULL function in Jethro SQL
JD-2701 Change NULLIFEQ() name to NULLIF()
JD-2709 Local cache refresh – use event update instead of file open

Bug Fixes

JD-2667 Multiple UNION ALL in one query fail
JD-2673 UNION ALL Query stuck while UNION ALL and JOIN combine on the same expression
JD-2676 UNION ALL Alias is not presented in query results
JD-2686 Unable to use literals/NULL as target column in VIEW
JD-2714 Wrong results related to adaptive sub-query in a specific scenario

Version 1.4.1

RELEASE DATE: FEBRUARY 15TH, 2016

Improvements

JD-2682 Allow join indexes to be create over dimension with no primary key
JD-2670 Loader format function unix_timestamp support fractional milliseconds

Bug Fixes

JD-2612 Query with group by count distinct take very long time
JD-2635 Issues with the distributed lock mechanism
JD-2664 Exception on partition pruning
JD-2672 Local cache refresh new files fails on hdfs
JD-2689 JethroServer creates too many threads causing queries fail
JD-2692 Performance: FI for range expressions (between) not always used
JD-2699 Query/server fail when using function over IN column: F(C1) IN (…)

Version 1.4.0

RELEASE DATE: FEBRUARY 2ND, 2016

Backward Compatibility

Data model conversion from previous versions is required.
Before starting service run: > Jethro U {instance-name} T

New Features

JD-2349 Support UNION ALL

Improvements

JD-2533 Column block encapsulation – store column block metadata and compressed value in a single file and avoid file append

Bug Fixes

JD-2522 Empty set result instead of exception that was raised – not consistent
JD-2637 Remove a warning message in partition pruning
JD-2645 Filter with “IN” does not work when the filter column is a partition key
JD-2653 Maint keeps life-long lock and old lock versions are not removed.
JD-2655 Query with quotes around the datatype in CAST fail
JD-2657 Sub-query adaptive causes simple query with sum and group by to fail. Return unclear error: CHECK failed: (index)  lt (size())

Version 1.3.5

RELEASE DATE: JANUARY 28TH, 2016

Improvements

JD-2634 Optimization for group by function queries: push groups expression to outer queries

Bug Fixes

JD-2595 Slow performance when large joined table key is non-unique and there are multiple rows for each key
JD-2620 JethroServer crash after create/drop tables due to local cache pre-fetch
JD-2633 Performance degradation for distinct queries after after upgrade to 1.3.4 due to change in assume referential integrity
JD-2636 Various issues with assume.referential.integrity version 2

Version 1.3.4

RELEASE DATE: JANUARY 21TH, 2016

Improvements

JD-2491 Optimization: Push Join after GB to upper query (single join)
JD-2525 Assume referential integrity – version 2
JD-2538 Enable adaptive cache for sub-query results
JD-2582 SHOW ALL TABLES/PARTITIONS/COLUMNS
JD-2613 Reduce hash size factor of join node hash

Bug Fixes

JD-2541 JethroLoader Crash when the path to the local cache directory in local-conf.ini is wrong
JD-2577 Service jethro restart – try to bring up a new Maint before the old one went down completely.
JD-2597 Wrong results when using CAST over aggregator in queries optimized with push joins to upper query
JD-2611 Wrong results: GB with CSN 128bit returns
JD-2614 Complex query with cross join and string column crash

Version 1.3.3

RELEASE DATE: DECEMBER 30TH, 2015

Bug Fixes

JD-2584 Block drop partition column with alter table drop column
JD-2593 Loader can crash we using local cache in load

Version 1.3.2

RELEASE DATE: DECEMBER 28TH, 2015

Improvements

JD-2496 Allow more than one <=> operator in JOIN qualification

Bug Fixes

JD-2583 ADD COLUMN fails when table has partitions
JD-2585 Wrong results when using offset gt number of rows in result set

Version 1.3.1

RELEASE DATE: DECEMBER 18TH, 2015

New Features

JD-2555 Support OFFSET clause in LIMIT clause

Improvements

JD-2192 Improve Show Local Cache information

Bug Fixes

JD-2542 Partition pruning – fix minor memory leak
JD-2554 Crash on partition pruning if table appears in more than one sub-query
JD-2572 Loader memory leak – keys pool doesn’t release temp buffer

Version 1.3.0

RELEASE DATE: DECEMBER 12TH, 2015

Backward Compatibility

Data model conversion from previous versions is required.
Before starting service run: > Jethro U {instance-name} K

New Features

JD-1526 Partitions pruning
JD-2362 Support Tableau Date data type usage
JD-2515 Support SQL functions “TIMESTAMPADD” and “TIMESTAMPDIFF”

Improvements

JD-2300 Integrate with libhdfs3 (beta mode)
JD-2354 update ODBC convert capabilities
JD-2379 Keep keys (values and indexes) in multiple files (file per block) and support keys recovery
JD-2398 Trivial Distributed locker based on shared file system
JD-2512 QLIK set analysis support: add support for DISTINCT operator with QLIK function
JD-2526 QLIK set analysis: Add ‘TOTAL! ‘ support
JD-2540 Update ODBC capabilities to reflect supported functions

Bug Fixes

JD-2370 CASE – if first then is NULL then CASE may fail due to type mismatch
JD-2486 Incoherent error message when performing a numeric function (e.g. sum) on a STRING column
JD-2501 Client hangs when connection or queries hangs/return after very long time (actual execution time is small)
JD-2502 libhdfs3 – Loader can’t load data files from HDFS
JD-2510 Monitor – No indication when the Jethro server and Maint were stopped
JD-2511 Monitor – No indication for Maint start in the service.log
JD-2520 QLIK set analysis – use left join instead of inner join
JD-2523 Crash happens when IN clause includes NULL value
JD-2524 Loader streamer creates more data blocks then set in config limit
JD-2535 QLIK set analysis – fix cases when original query doesn’t have target column

Version 1.2.1

RELEASE DATE: NOVEMBER 18TH, 2015

Improvements

JD-2503 Loader Usability – error messages
JD-2506 Improve loader memory control: release keys cached buffers if cache is full and allow using local cache during load

Bug Fixes

JD-2077 Loader: various issues with loader errors logging
JD-2307 Fix loader report: add number of rows removed from overwrite scenarios
JD-2418 Fix misleading message when input file is corrupt: Input row is longer than 10m bytes
JD-2455 Change error message in log file when user is running more than one loader on same table.
JD-2492 Loader report: indicate in report file that rows are being rejected and point to rejects file
JD-2508 Missing configuration in loader report file
JD-2509 Loader memory leak for Timestamp column

Version 1.2.0

RELEASE DATE: NOVEMBER 13TH, 2015

New Features

JD-2356 Primary Key
JD-2197 Join indexes
JD-2469 QLIK set analysis functionality in Jethro SQL (Alpha)

Improvements

JD-2470 Join after Dim GB Optimization – basic scenario (distinct values, no aggregators, single join)
JD-2487 Improve assume referential integrity (drop fact from join when possible)

Bug Fixes

JD-2483 Some aggregations are executed via non-optimized path (full scan) instead of index based aggregation
JD-2499 rpm upgrade removes Jethro monitor job
JD-2500 JOIN invalid qualifier crashes

Version 1.1.4

RELEASE DATE: OCTOBER 27TH, 2015

Bug Fixes

JD-2363 Libhdfs messages captured in logs as ERROR even when it’s informational or warning
JD-2460 Error received on group by query using adaptive delta
JD-2465 Loader test and recovery – didn’t work on functional columns

Version 1.1.3

RELEASE DATE: OCTOBER 14TH, 2015

Improvements

JD-2382 Improve performance of GROUP BY queries with datetime functions on groups
JD-2445 Add date/time FI optimizations for YEAR range and IN

Bug Fixes

JD-2361 wrong results for tpch query: Q19
JD-2434 TPCH Q3b query is causing server failure
JD-2429 Server crash during query with where clause that include column<>”
JD-2430 Memory leak in join queries
JD-2436 Loader does not reject lines with timestemp bigger than max timestamp
JD-2444 Loader issue: unix_timestamp format for timestamp in doesn’t work. User gets a reject: “failed to create unix time”
JD-2363 Do not LOG libhdfs stderr capture as ERROR if it is not
JD-2423 Loader exits with positive status when loader reach maximum rejects
JD-2338 JethroAdmin.sh call service stop/start doesn’t support multi versions installation
JD-2432 jethromonitor.sh does not report instances initialization to /var/log/jethro/monitor.log
JD-2186 loader should set number of threads to 1 in the case of 2 core machines

Version 1.1.2

RELEASE DATE: SEPTEMBER 29TH, 2015

New Features

JD-2359 Cross Join support
JD-2367 Null safe equi join with operator <=>

Improvements

JD-2351 String ranges optimization (use indexes by converting to IN)
JD-2400 Rewrite “JOIN … on A=B or A is null and B is null” to use <=> operator

Bug Fixes

JD-2018 Loader crash at create global uki stage for high cardinality column
JD-2309 JethroServer coredump when local-cache.ini is missing
JD-2318 Corrupt persist.xml file
JD-2327 Query with cross join crash the server
JD-2329 Loader does NOT reject INT with spaces: “12 3”
JD-2368 show local cache – crash if issued before first background update
JD-2369 Loader terminates with error reading from file when creating global UKIs
JD-2370 CASE – if first then is NULL then CASE may fail due to type mismatch
JD-2371 Nested queries – crash because semantic error is identified by NS but not captured and is passed to QueryBuilder
JD-2372 nested queries – join tables are not traversed by semantic for validity
JD-2402 Server stuck on execution

Version 1.1.1

RELEASE DATE: AUGUST 23TH, 2015

Improvements

JD-2353 Support assume referential integrity – mode that eliminate all referenced table from join. phase I : support star schema
JD-2355 allow position reference in GB

Bug Fixes

JD-1711 SELECT with sub-query attributes that aren’t supported should be rejected
JD-1826 SELECT accept unsupported options
JD-2316 CASE mishandles NULL type conversion
JD-2322 wrong result: explicit conversion NULL::float returns 0 in CASE statement
JD-2330 ODBC – update and fix connection properties
JD-2347 IN reject correct aggregation function targets in sub-query due to interim data-type
JD-2365 Loader overwrite option removes functional indexes

Version 1.1.0

RELEASE DATE: AUGUST 11TH, 2015

New Features

JD-1045 Function Indexes Phase 1

Improvements

JD-2344 ODBC driver for 32 bit (Windows)
JD-2343 Optimize queries with repeated expressions in target list
JD-2342 Join optimization: add transformation from outer join to inner join to allow star transformation optimization
JD-2341 Improve JOIN optimization – don’t disable star transformation if another outer join exists after inner join
JD-2340 Adaptive caching for IN with Subquery
JD-2199 Shared adaptive cache
JD-2188 Loader: optional per-column NULL representation

Bug Fixes

JD-2337 Fix merge all filter maps before index full scan
JD-2330 ODBC – update and fix connection properties
JD-2328 Fetch threads in filtered where + partitions
JD-2324 wrong results if GB col is also not last join qualification
JD-2298 Fetch SN performance issue
JD-2286 Datediff function performance is slow
JD-2282 missing if() function for TIMESTAMP causes IF stmt to fail and CASE to crash
JD-2239 wrong qualifier identification if multiple ‘=’ used in join ON clause (was: wrong results on JOIN queries with casting)
JD-2238 Query failed with “request execution failed” (iBasis – flat queries)
JD-2232 Query failed without clear error message. Uses unsupported function within CAST
JD-2231 Query failed with “SemanticException – Cross join is not supported – ON clause requires at least one join condition”
JD-2219 Server memory leaks in query execution
JD-2208 Local-cache.ini is not effecting cache refresh
JD-2177 Wrong result for complex expression that use filter where
JD-2122 Service jethro command does not work well in CentOs 7

Version 1.0.2

RELEASE DATE: JUNE 6TH, 2015

New Features

JD-2217 Support MapR distribution
JD-2073 SHOW PLAN – show detailed execution plan (in log)

Bug Fixes

JD-2214 JethroServer crash after getting cancellation from Client (most likely via pentaho)
JD-2213 Query fails error – join between views
JD-2194 Segfault – server fails on a specific query
JD-2191 Query Cancelled by user. resulted in an internal error and query hanging
JD-2187 JerthroAdmin: incorrect error message on attache instance
JD-2185 Problem with environment setup with HDP (Hortonworks) installation
JD-2180 execution counter – hole of 90% of the time – count distinct
JD-2179 JethroServer crashed or stuck on query
JD-2172 Unsupported version message send due to serialization version mismatch doesn’t provide info required to identify to problem
JD-2145 Loader crashed on deadlock
JD-2099 Schema oid usage isn’t efficient and can also cause lock bugs

Version 1.0.1

RELEASE DATE: MAY 10TH, 2015

New Features

JD-1138 ALTER TABLE ADD/DROP COLUMNS

Improvements

JD-2168 refactor adaptive rank calculation
JD-2005 Dynamic refresh of local cache based on blocks access frequency

Bugs

JD-2178 JethroODBC: ODBC driver should disable catalog support
JD-2176 Jethro initialization failed on test server
JD-2173 when adaptive cache deletes a query, it writes many junk lines in jethro.log
JD-2171 count(*) on columns from sub-query fails ” ‘Scalar’ node build failed”
JD-2117 Adaptive query cache – use FULL if can’t use INCREMENTAL
JD-1950 Query with IN returned wrong result on branch ‘JD-1947_non_opt_build’

Upgrade Instructions

Before upgrade: stop all loads and stop JethroLoadsScheduler service (service jethro stop instance-name] loadscheduler)
Confirm the load scheduler event drop folder is empty (ls storage-root-path]/instance-name]/loads/drop)

After upgrade to 3.4 it is mandatory to run upgrade tool over each existing instance.
Usage:
upgradeTo3.4_MultiSchema <instance> <ip/localhost>:<port> -u <user> -p <pass>

This tool create system schema "sys" and move jethro system tables to "sys" schema

1. Read the version specific upgrade notes which are relevant for your upgrade process, and perform all the steps which are required to be made, before starting the upgrade process.