| Component | ODBC | JDBC | Jethro Manager | Jethro Management package for Ambari |
|---|---|---|---|---|
| Jethro Server 3.4.0 | 3.7 | 3.6 | 1.5 | 1.0.0 |
| Jethro Server 3.3.0 | 3.3 | 3.3 | 1.3.1 | - |
ODBC 3.7.1
Improvements
JD-5084 Add to Tableau TDC: CAP_QUERY_WHERE_FALSE_METADATA='yes' (optimized as of version 3.3.1)
JD-5093 Add to Tableau TDC: CAP_QUERY_TOP_0_METADATA='yes' (optimized as of version 3.3.1)
Bug Fixes
JD-5302 Tableau hangs following session disconnection (ODBC returned status is not identified by Tableau)
Version 3.4.0
RELEASE DATE: Dec 25, 2017
- Support multiple schemas.
- Fix environment settings for JAVA to simplfy initial setup
- Bug fixes
Compatibility
Server version 3.4 is NOT compatible with previous version of ODBC, JDBC and JethroManager. All client components must be upgraded.
Compatible client versions:
ODBC 3.7
JDBC 3.6
JethroManager 1.5
Upgrade Instructions
Before upgrade: stop all loads and stop JethroLoadsScheduler service (service jethro stop instance-name] loadscheduler)
Confirm the load scheduler event drop folder is empty (ls storage-root-path]/instance-name]/loads/drop)
After upgrade to 3.4 it is mandatory to run upgrade tool over each existing instance.
Usage:
upgradeTo3.4_MultiSchema <instance> <ip/localhost>:<port> -u <user> -p <pass>
This tool create system schema "sys" and move jethro system tables to "sys" schema
New Features
JD-1137 Multiple Schames support
JD-4653 Schema related DDL commands: CREATE SCHEMA, DROP SCHEMA, SHOW SCHEMA, USE
Improvements
JD-5197 CREATE EXTERNAL TABLE to allow setting file format spec (delimiter)
JD-5234 Fix jethro CLASSPATH issues
JD-5267 JethroClient: Allow multiple statements when passing request as argument
JD-4569 Improve environment settings for JAVA to support out of the box range of Hadoop distribution and Hive configurations
Bug Fixes
JD-3653 Query fails due to corrupted cube read with "Error getting unit no by serial no". Occurs when cubes group by uses 128bit CSH Hash
JD-4945 Server failure due to invalid casting of null in INSERT INTO
JD-5019 Query with complex timestamp expression fail with error (cast issue)
JD-5131 Cubes queues status message may show identical status line more then once
JD-5138 Rebuild cubes may not be re-generated following intensive schema changes
JD-5139 Rename of system table is not fully blocked
JD-5235 Cubes are ignored and not recreated following internal exception ("Error getting unit no by serial no")
JD-5239 CREATE EXTERNAL DATA SOURCE doesn't trim with extra space after name appends space to the name
JD-5252 Server failure when INSERT INTO value contains invalid byte sequence ('\0')
JD-5284 INSERT INTO: exception in connector CSV parser due to line > 4K silently causes incomplete load
JD-5286 INSERT INTO: loader incorrectly casts special floating point values: nan, infinite
JD-5351 Server failure following intesive autentication try when calling openldap method
JD-5398 Cube update will fail following rename of table and cube will not be updated
JD-5415 SHOW FULL TABLES is missing in collect statistics
JD-5446 Maint push "Starting Garbage Collector Cycle" message every second
Minor Bug Fixes
JD-5186 Fix message typo for: referenital integrity optimization
JD-5340 Schedule loads log contains repated occurences of the messages "The version was null, therefore will not read the file"
ODBC 3.7
New Features
JD-5305 Support ODBC GetTables() & GetColumns() for specific schema/table API as SHOW FULL TABLES in <schema/table>
JDBC 3.6
New Features
JD-5305 Support JDBC GetTables() & GetColumns() for specific schema/table API as SHOW FULL TABLES in <schema/table>
Version 3.3.2
RELEASE DATE: Dec 7, 2017
- Queries load balancing: skip busy nodes that are queueing queries. Require upgrade of ODBC/JDBC to version 3.5
- Generate and match cubes for count distinct queries without group by
- Allow long identifier names (128 characters)
- Cube generation efficency: persistent too-big statistics
- Important bug fixes
Upgrade Instructions
- Before upgrade: stop all loads and stop JethroLoadsScheduler service (service jethro stop instance-name] loadscheduler)
- Confirm the load scheduler event drop folder is empty (ls storage-root-path]/instance-name]/loads/drop)
New Features
JD-4827 Queries load balancing - skip busy nodes
Improvements
JD-3118 Dynamic aggregation: Count distinct aggregation without group by to match/gen cubes
JD-3995 Persistency for cube Too-Big statistics
JD-4675 Optimizer: aggregation queries are rewritten as Group By NULL to support CND cubes match and improve performance
JD-4794 Include insert into SQL details in loader event system table
JD-4978 Allow identifiers length of up to 128 characters
JD-5152 Minor: improve message on too big cube
Bug Fixes
JD-4728 Orphan loader files of dropped tables are not cleaned by garbage collector
JD-4812 DROP PARTITION requires range values to be between HIGH to LOW
JD-4868 When LDAP Query result contains more than 1000 entries, it doesn't return the results
JD-4887 Plan presents meaningless time for 'EXTERNAL TABLE FETCH' time
JD-4925 Insert into fails if data contains the special charcter '/001'
JD-5044 Scheduled Load rule for jethro$_loader_events is not properly created on instance creation
JD-5055 Manual cube generation fails when multiple cube generations servers are configured
JD-5111 "Illegal construction of TupleSet. Size = 0" on limit 0 query
JD-5161 Non-helpful error return when insert into fails due to invalid data ("trying to flush footer report section more than once")
JD-5170 JethroMaint garbage collector erasing dat files of valid cubes
JD-5171 JethroServer failure upon retry of a query following cube/adaptive entry not found
JD-5175 When log text is longer then 1024 chars, the Loader is killed
JD-5192 SHOW CUBES operation is very slow
JD-5215 INSERT INTO from an external data source may generate defect records in destination Jethro table
JD-5216 Authorization is checked more then once during parsing - may effect query init time
JD-5238 Temporary files are not removed from tmp folder - GC to remove old tmp files periodicly
JD-5279 JethroClient misinterpt "\N" literal in a line. User unable to define "\N" as NULL in new external table
JD-5322 Server crash when calling open ldap method
Minor Bug Fixes
JD-5260 Improve error message in lo
ODBC 3.5
New Features
JD-4827 Queries load balancing - skip busy nodes
JDBC 3.5
New Features
JD-4827 Queries load balancing - skip busy nodes
Version 3.3.1
RELEASE DATE: Nov 18, 2017
- Important improvments and bug Fixes
Improvements
JD-5124 Limit max cube/adaptive entry size in byte via config
JD-5125 Optimize limit 0 and where 0=1 queries (commonly used by Tableau and other BI tools)
Bug Fixes
JD-5118 Table OVERWRITE or TRUNCATE choose lose of granted table permissions
JD-5119 Schema inflate following intensive schema changes and/or table OVERWRITE operations. This can lead to system wide performance issues.
JD-5147 Loader - wrong logic of rejecting files as having unsupported format (zip) based on filename or path
JD-5148 JethorClient -c create CSV file with wrong timestamp values (following 3.3.0 timestamp range expansion)
Version 3.3.0
RELEASE DATE: Nov 5, 2017
- Extended dates range for TIMESTAMP data type: 01/01/1400 - 31/12/9999
- Bug Fixes
Compatibility
Server version 3.3 is NOT compatible with previous version of ODBC, JDBC and JethroManager. All client components must be upgraded.
Compatible client versions:
ODBC 3.3
JDBC 3.3
JethroManager 1.3.1
New Features
JD-4703 TIMESTAMP data type now support extended dates range (from 01/01/1400 to 31/12/9999)
JD-4545 Refactor optimizer rewrites - various performance optimizations and improve optimizer flow and logging
Improvements
JD-4924 File connector to support UTF-8 encoding
JD-4956 Collect log to collect schedule loads metadata
Bug Fixes
JD-3598 Adding a new Timestamp column to an existing table fails
JD-4501 CREATE TABLE may takes long time to complete
JD-4714 Column name is partially presented
JD-4856 Accuracy issue when cube is used for COUNT DISTINCT over case function when puki optimization is enabled
JD-4870 Schedule Load- "lsof" is called when drop folder is hdfs
JD-4884 Assume referential integrity rewrite doesn't eliminate sequence of unnecessary JOIN expressions on certain scenarios with QLIK function
JD-4902 Accuracy issue when assume referential integrity is enabled and QLIK function is used
JD-4909 JethroAdmin create-instance error msg fixes
JD-4912 Query with duplicate line of COUNT(DISTINCT (CASE WHEN )) is failing with "Request execution failed"
JD-4914 Scheduled load: incorrect recursive handling of files in subfolders under the drop folder when stating specific file pattern
JD-4926 Cubes are not generated and check test failed when cubes are disabled (dynamic.aggregation.enable=0)
JD-4928 Possible accuracy issue with partitions on timestamp if maximum value of partition has fractional seconds
JD-4946 Jethro Scheduled Load - De-Serialization error due to corrupted loadstate*.xml file
JD-4955 Jethro Schedule Loads service should start by default only on one node in the cluster (loader node)
JD-4964 schedule loads fail with log message 'Exception in Thread 'some name': basic_string::erase'
JD-4965 LIMIT and WHERE clauses are ignored for external tables over files data source
JD-4985 After ping failure monitor may start server too quickly after stop causing server start to fail
JD-4996 Cubes auto generation/update requests are not sent due to rare race condition
JD-4998 Accuracy issue with WHERE year(TS) NOT IN (...). NOT operator not applied
JD-5026 Unable to connect to HIVE External Data Source in kerberized environment due to missing path in classpath
JD-5066 External table from HDFS stack forever when no file is in the hdfs path
Minor Bug Fixes
JD-4987 Change "loaded" to "inserted" in INSERT INTO success message
ODBC 3.3
New Features
JD-4983 Support both old and new timestamps formats depending on server version
Improvements
JD-4977 Allow alias name length of up to 128 characters (previously was 64 characters)
Bug Fixes
JD-4937 Tableau: Connection with multiple servers in DSN-less mode fails. Semicolon seprator (;) breaks connection string parse. Add support for colon (,) seperator for DSN-less connection string
JDBC 3.3
New Features
JD-4983 Support both old and new timestamps formats depending on server version
Version 3.2.0
RELEASE DATE: Oct 4, 2017
Major improvements in cubes generation and maintenances:
- Cubes generation and maintenance can now be scaled to more than one server
- Cubes generation and maintenance reorganized and redundant entries removed
- Cubes status tracing logs significantly improved
- New DDL: SHOW CUBES FROM .. can be used to show existing and potentials cubes for query
Improvements
JD-3575 Cubes generation and maintenance scalability - support multi servers cube gen/maint
JD-3931 Reorganize cubes generation and maintenance queues
JD-3969 Null safe join to support functions in ON expression
JD-4121 Dynamically disable/enable cubes scan and generation background cycle via configuration
JD-4330 Push to cube generation and maintenance queues only if a potential cubes can be created and support new DDL: SHOW CUBES FROM
JD-4791 Improvements for log messages reporting cube generation
JD-4843 Main stability - do not exit maint after failures - additional scenarios not covered in 3.0.5
Bug Fixes
JD-4061 Duplicate and redundant entries are pushed to cube generation/update/regen queue causing queue to inflate
JD-4805 Query does not refer to suitable cube which was generated with RIGHT OUTER JOIN
JD-4832 A redented adaptive entry is created for query that finds a cube match
JD-4836 Jethro uninstall should delete $JETHRO_HOME even if it is not empty
JD-4901 Possible lock corruption due to wrong order of locks for schema-lock
JD-4905 Query with CASE fails with "Request execution failed" due to type-mismatch
Version 3.1.1
RELEASE DATE: Sep 28, 2017
Bug Fixes
JD-4861 Query fails with "ADAPTIVE ERROR FingerPrint mismatch"
JD-4899 Order By ignored if asc and desc order both specified after push join rewrite
JD-4910 File connector fails on hdfs files hadoop-hdfs jar missing in classpath
JD-4917 File connector fails on hdfs files with ArrayIndexOutOfBoundsException error
JD-4919 Create view is failing if now() function exist in the select statement
JD-4920 Flatten query optimization skips left sub-query in join
Version 3.1.0
RELEASE DATE: Sep 19, 2017
- Support external connector for files
- Various connectors related improvements and fixes
- Additional bug fixes
New Features
JD-4572 External connector for files: allow creating an external table over HDFS/Posix files
Improvements
JD-4553 Connectors - manage/expose version
JD-4871 INSERT INTO should to return message with the number of rows inserted
Bug Fixes
JD-4269 ExtTbl: Proper error should appear if the connection to HIVE failed
JD-4716 ExtTbl: Error message following cast failure (reject) in select from external table is unhelpful
JD-4739 JOIN col type mismatch issues unhelpful error message "Undefined function '='"
JD-4773 Errors related to operations over external tables are not raised to caller
JD-4778 Cannot delete system table using the drop system table utility
JD-4782 Ext_table: REJECT_SAMPLE and REJECT_LIMIT_RATIO limits are not enforced and load passes when it should fail
JD-4793 The connector hangs when hive-jdbc driver is not found in classpath
JD-4797 Insert into should return errors from select/load instead of generic failure message
JD-4835 Scheduled Load mechanism not working as expected with post load file action
JD-4839 Failed to create Hive data source Hive on HDP 2.6.1
JD-4866 Getting error "could not find the virtual object of the virtual column" when reading from a cube
JD-4867 LDAP query fails with "operations error"
JD-4869 Query doesn't generate cube and nothing is reported in the logs
JD-4877 Query with HAVING COUNT(DISTINCT ...) fail with error. Index optimization for Group By is broken for this scenario
Minor Bug Fixes
JD-4461 ExtTbl: Fix "Creating an External table not allowed" message
JD-4717 ExtTbl: Reject count log message spaces fix
JD-4803 Insufficient logging for distributed stmts (e.g. show active queries) - missing info in cases of failure
JD-4852 SHOW EXTERNAL DATA SOURCES is missing under the SHOW help
Version 3.0.5
RELEASE DATE: Sep 3, 2017
Stability improvements and bug fixes
Improvements
JD-4820 Watchdog reset server in case of severe memory fragmentation
JD-4828 Maint stability - recover and skip errors related to specific operations or objects
Bug Fixes
JD-4821 Server appears hung when execution long queries due to memory cleanup cycle lock following dropping objects (tables, partitions or columns)
Version 3.0.4
RELEASE DATE: Aug 20TH, 2017
- Handle various casting issues related to INSERT INTO
- Loads scheduler issues
- Session files versions purge - fix session files scan overhead that caused delay on every request
- Additional bug fixes
New Features
JD-4163 Support inserting values via INSERT INTO ... VALUES (...)
JD-4704 SHOW EXTERNAL TABLES
Improvements
JD-4218 Allow downcast bigint/double to int/float in INSERT INTO
JD-4695 Support universal downcast type conversion in INSERT INTO (configured off)
JD-4779 Allow casting integer to float in INSERT INTO
Bug Fixes
JD-4520 Trailing spaces trimming by Tableau in where clause literal results in returning empty set - data type is reported as CHAR instead of VARCHAR
JD-4522 Server failure when executing query on view with UNION ALL that contains literals
JD-4638 Temporary files created during load at instance /tmp folder are not always removed
JD-4684 UNION ALL failure due to missing implicit casting of left to right clause
JD-4706 INSERT INTO request hang due to closing sockets of forked child process
JD-4725 Loader streamer page files cleanup may deletes temp files on concurrent loads
JD-4726 Loader error information is lost following exception
JD-4732 SHOW LOCAL CACHE return SHOW LOCAL CACHE EXTENDED results and vice versa
JD-4756 Scheduled loaded stop execution load jobs and never recover following HDFS errors
JD-4778 Admin utility cannot delete system table
JD-4783 Potential failure of Jethro Loader Scheduler service if error encountered during meta data read
JD-4785 Potential error in Jethro Loader Scheduler - file not close in case of read failure
JD-4786 Potential minor memory leak when reading a file in Jethro Loader Scheduler
JD-4788 Number of session files reach thousands due to accumulation of previous versions causing significant delay for every query when session is accessed. Previous versions of session files should be deleted by GC
Minor Bugs
JD-4591 Maint is getting stopped due to corrupted lock file without appropriate error message
JD-4610 Unclear error message on SHOW EXTERNAL DATA SOURCES when result is empty
JD-4713 Misspelled error messages
ODBC 3.0
Improvements
JD-4151 Allow INSERT INTO via ODBC
JD-4466 Support standard attributes in the ODBC connection string (SERVER, DATABASE)
Bug Fixes
JD-3566 Create and drop table not allowed via ODBC
JD-4020 Expose server version via ODBC API
JD-4495 Drop view not allow via ODBC
JD-4705 Driver version property via SQLgetInfo return SDK version instead of ODBC version
JDBC 2.2
New Features
JD-4574 Improve JDBC logs and logs control via log4j
Bug Fixes
JD-4551 Failing instance authentication on multi connection scenario generated large number of repeated failing request and error messages in jethro.log
JD-4760 JDBC - calling cancel method causes JDBC to fail with null exception
Version 3.0.3
RELEASE DATE: Jul 13TH, 2017
- Support LDAP authentication over SSL/TLS
- Support cubes for optimized union all flat queries
- Expend external connector environments support and improve Hive connector performance
- Scheduled loader can now run concurrent loads
- Performance improvements for queries grouped by datepart functions
- Important bug fixes related to cubes generation and cubes update that significantly improve cubes maintenance efficiency and reduce cube generation/update queue size
- 30+ additional bugs fixes
Improvements
JD-2922 GROUP BY optimization: use functional indexes for all datepart functions and complex timestamp functions expressions
JD-3872 Scheduled loads to support concurrent loads into different tables
JD-4400 Hive connector: performance improvements for fetch from external Hive table
JD-4557 Support LDAP authentication over SSL/TLS
JD-4570 Generate/match cubes for UNION ALL flat left/right queries following GB/AGG push down
JD-4595 External data source connector to support Java 1.7
JD-4621 External data source connector MapR support
Bug Fixes
JD-4005 Loader failure with core when writing the system table
JD-4190 Parameters BIGINT values always casted to INT
JD-4254 Maint resend cube update requests for cubes known to be too big
JD-4276 ExtTbl: show active queries failed while running an external query
JD-4326 Scheduled loads: list of file to load is not refreshed on next try following load fail
JD-4394 Union will fail if one side is integer literal and the other side is integer expression
JD-4397 Missing info in log: cube generation server port number not shown in maint log
JD-4402 Performance degradation in some queries with QLIK function (from version 3.0.0)
JD-4438 ExtTbl: missing supported casting operations in Hive connector
JD-4447 Show active queries and kill queries security deficiencies
JD-4452 ExtTbl: UNION ALL query from external table fails on internal error
JD-4462 Internal Fix for requestId handling
JD-4472 ExtTbl: rejected row count on FILL_NULL is wrong
JD-4476 Internal Fix in thread exception catching
JD-4485 GROUP BY query with hi cardinality count distinct aggregation from left join return wrong results when using partition level count distinct optimization
JD-4493 ExtTbl: UNION ALL query from external table fails with invalid column count error
JD-4505 Jethro Server port is not released following JethroServer restart during INSERT INTO operation
JD-4509 SQL syntax error cause server failure
JD-4511 Internal Fix: wrong assumption of argument of type ColumnCopy
JD-4515 Server failure upon intensive DDL operations
JD-4517 QLIK query over a view or sub-query cause server failure
JD-4529 Insert Into - Loader process is not always terminated when Server process stopped
JD-4539 Cubes are generated for system tables - should be avoided
JD-4558 Failures in delete obsolete cubes cause maint service shutdown
JD-4562 When updated cube is too big the subsequent cube updates commands fail to update the cubes
JD-4567 Server failure when query with count(*) from sub-query fetch the sub-query results from adaptive
JD-4568 Server failure when query where clause include unsupported sub-select = operation
JD-4614 IN/NOT IN will fail when checked against a list with aggregated values
JD-4618 Accuracy issue when reading sub-query from adaptive following specific push down aggregation optimization scenario
JD-4619 Cubes which needs update are sent several times to execution by maint service
JD-4623 Explicit IN subquery down cast behaves abnormally
JD-4626 Comparing literal to IN list cause server failure
JD-4627 Update cube fails when a schema change occur during cube update operation
JD-4640 Cubes generation flow does not detect that a cube was marked as too big and re-execute the cube query
JD-4666 LDAP group lookup fails for ldap user having distinguished name with special characters ("," "\" etc...)
Version 3.0.2
RELEASE DATE: Jun 6TH, 2017
- Improve Hive connector functionality and performance: with native Hive JDBC driver used by the connector advanced Hive JDBC URL syntax is now supported and the connector now works over Kerberized HDFS. Additional performance improvement is gained if using latest Hive JDBC.
- Various security, external table and other bug fixes
Improvements
JD-4482 Connector use native Hive JDBC available from the environment. Effect: connector can run on Kerberos environment and connector performance is improved
Bug Fixes
JD-4007 Query using range Index hangs
JD-4395 Count distinct query fails when matching cube with other aggregation
JD-4416 LDAP group name validation is case sensitive
JD-4419 Views authorization fails when underlying table is not authorized
JD-4426 Upgrade falsely generate additional copy of ssh authorization key
JD-4437 Accuracy issue for self-join queries with function on target list
JD-4440 JDBC packed with rpm should be upgraded to protobuf 2.5 (jdbc 2.0)
JD-4443 Multiple auditing log messages for single audit
JD-4473 External Tables: Server failure following INSERT INTO error when connector fails to run
JD-4475 CLONE of JD-4428 for 3.0.2
JD-4479 INSERT INTO reach deadlock in rare cases if executed following DROP TABLE/COLUMN
JD-4481 Service fails to start over Hortonwork environment following improper JAVA_HOME declaration
JD-4487 CLONE JD-4486 for 3.0.2
JD-4490 Drop on view set another view to "invalid" when column description is not same as column name
JD-4498 Server failure when receiving query with invalid syntax (brackets around target list)
Version 3.0.1
RELEASE DATE: May 23TH, 2017
Improved Hive connector performance enabling accelerated load from external tables and various connector related bug fixes.
Improvements
JD-4532 Hive connector Performance improvements
Bug Fixes
JD-4430 External Tables: Where fails when casting external table query from different types
JD-4442 External Tables: Reject FILL_NULL count limit is ignored
JD-4453 External Tables: kill query of "insert into" failed to stop the load from hive
JD-4459 External Tables: Insert into failure due to casting issue with "Invalid position specified for group-by column"
Version 3.0.0
RELEASE DATE: May 11TH, 2017
- Enhanced security with complete LDAP authentication and authentication with standard hierarchical permission model.
- External tables mapped to Hive table using Jethro-Hive connector.
- Load data using external table via standard INSERT INTO SELECT command.
- Self Join support.
New Features
JD-3077 Support INSERT INTO SELECT
JD-3632 LDAP authentication (release)
JD-3739 Support EXTERNAL DATA SOURCE and EXTERNAL TABLE
JD-3755 Security: Authorization Support (Permissions, Roles, Groups)
JD-3887 External Connector for HIVE
JD-3963 Security: Auditing
Improvements
JD-1217 Support self join
JD-1275 Sequence based result set column names for unaliased expressions in target list
JD-2782 New NS Component (internal)
Bug Fixes
JD-4333 Queries with literals only are blocked
JD-3211 Qlik Query failes due to duplicate expressions
JD-3435 Add column Process ID (PID) to the loader events system table
JD-3557 Queries with column in order by that is not found in group by should be rejected
JD-3566 Create and drop table not allowed by odbc
JD-4005 Core in Loader when building global UKI
JD-4247 Problem installing Ubuntu package
JD-4251 Timestamp literals from query appear in log as unix timestamp instead of date-time literal
JD-4265 Enable fallen query rewrite by default (flatten.query.rewrite.enable=1)
JD-4319 Multiple consecutive spaces in query literals are trimmed
Version 2.1.4
RELEASE DATE: Jun 6TH, 2017
Improvements
JD-4280 Allow execution of literal only queries
Bug Fixes
JD-4428 Jethro Maint failure following drop of invalid cube in cubes regeneration scenarios
JD-4486 Cubes regeneration is not initiated following subsequent schema changes
Version 2.1.3
RELEASE DATE: May 23TH, 2017
Improvements
JD-3354 Optimization: expand push where support for OUTER JOIN and CROSS JOIN
Bug Fixes
JD-2741 Various issues with literals used as group by dims and/or aggregations over literals (allowed internally following rewrite)
JD-4256 Cubes too-big list is ignored and not used to accelerate future cube generation requests
Version 2.1.2
RELEASE DATE: Apr 12TH, 2017
Optimizations for group by/aggregation queries on top of UNION and bug fixes mostly related to QLIK function.
Improvements
JD-4144 Support cube generate execution hosts "auto" to automatically set it to local host and known server port
JD-4181 Optimization: Push down GROUP BY and/or AGGREGATIONS to flat sub-query with and without UNION ALL
Bug Fixes
JD-3424 Service command report stopped jethro services as "already running" if instance name contained in another running jethro service
JD-3596 Query execution hangs following dynamic joined table switch on complex join with IN sub-query
JD-3814 Query return error following cubes match failure due to duplicated measures
JD-4009 Garbage collector false reports deletion of keys files
JD-4072 Call to SHOW FULL COLUMNS fails with null deredrench error when encountering invalid views
JD-4083 Query find a cube with duplicate measures but the cube is not used by planner (adaptive entry not matched)
JD-4118 Cube match internal error fails query instead of skipping cube match
JD-4142 Qlik set analysis with assume referential integrity using TOTAL and join fails with error
JD-4147 Qlik set analysis filter expression fail to match literal with upper case letters (since version 2.1.0)
JD-4179 Adaptive cache with QLIK function - Query returns wrong results from adaptive cache in cases query contains duplicate measures
JD-4183 Throttling in high concurrency scenarios and queries delayed when schema altering command are executed
JD-4212 Query with wrong QLIK function syntax cause server failure
JD-4213 Qlik query fails with "position" exception when duplicate measures encountered
Version 2.1.1
RELEASE DATE: MAR 20TH, 2017
Use tcmalloc allocator and bug fixes.
Improvements
JD-3558 use google allocator – tcmalloc
Bug Fixes
JD-4074 Following schema changes all queries are resent to cube generation queue. This is redundant and cause unnecessary cubes generation operations
JD-4021 Maint service minor memory leak created following JD-3852 fix in 2.1.0
JD-4109 Maint service major memory leak related to adaptive rank file read/update
Version 2.1.0
RELEASE DATE: MAR 3RD, 2017
Support cubes regenerate following tables, cubes operational improvements and bug fixes.
New Features
JD-3851 Regenerate cubes following table overwrite
Improvements
JD-3852 Accelerate adaptive space cleanup by deleting invalid entries before adaptive space is full
JD-3854 Cube related logs message improvements
Bug Fixes
JD-3847 Missing system table load event with current status for failed loads
JD-3853 Invalid de-ser debug message filling log
JD-3921 JDBC\ODBC – Query failed. Reason: Server failed to find session
JD-3928 Cube update cause cube generation server failure
JD-3929 Query with union all and that include repeated literals on one side of the union causes server failure
JD-3944 Server fails to deserialize huge cubes with count distinct fetching in on demand mode
JD-3967 Server failed to find session after 2nd call to SHOW ACTIVE QUERIES
JD-4006 Query with CURRENT_TIMESTAMP sent from Tableau fails with syntax error
JD-4017 when the .dat file is removed from adaptive entry, the size of the entry should be zero in show cubes command
Version 2.0.5
RELEASE DATE: JAN 31TH, 2017
Including LDAP authentication (beta), optimization for union queries, additional minor improvements, bug fixes and
New Features
JD-3632 LDAP Authentication (beta)
Improvements
JD-2855 Rewrite: push where predicate to union all
JD-3668 Add timestamps to dmesg output
JD-3669 Add show version to collect logs sql script
JD-3698 Auto cubes: allow cube to match agg(if…) expressions which are decomposed when there is more than one column in the condition
JD-3742 Send earlier status from JethroLoader to the system table
JD-3813 Add SHOW CUBES to collect logs
JD-3817 Add monitor.log to collect logs
JD-3843 Add crontab list to collect logs
Bug Fixes
JD-3670 Select with count(distinct) over cube can cause server failure
JD-3687 Cube matching stops following exception at reading schema
JD-3704 Mishandling of null values in the JDBC driver
JD-3741 Generate cubes – ORDER BY mistakenly added to the cube expression
JD-3750 Cube generation queue is limited to 1000
JD-3751 Update cubes – ID 0.0.0 is mistakenly printed to log as an indication for every query execution
JD-3752 Update Cubes – Update cube should be executed for each key (the cube) instead of executing the original query
JD-3753 Update Cubes – When changing entry type to CUBE we original query SQL is not updated
JD-3754 CASE with aggregation function expression in condition/results fail with “function not found” exception
JD-3762 Jethro ODBC, wrong vendor name.
JD-3771 JethroAdmin create-instance’ doesn’t start the instance automatically when performed using ssh
JD-3785 JethroMaint service failure with: “Too many open files…” messages
JD-3787 Push where wrong write when NOT operator is above AND – possbile occuracy issue
JD-3789 Select with count(distinct) using per partition UKI optimization can cause server failure
JD-3799 Upgrade ODBC SDK
JD-3805 ODBC minor fixes
JD-3806 Tableau TDC – jethro requires explicit cast of null literals
JD-3831 Loader – $LOAD_START load wrong values
JD-3836 Adaptive is not copied to local cache (adaptive fetch to local cache is enabled as global parameter)
JD-3840 Adjust Jethro ODBC to support known escape sequence for outer join {oj ….}
JD-3849 Significant memory leak overtime when enabling cubes
Version 2.0.4
RELEASE DATE: JAN 4TH, 2017
Instruction for upgrade existing instances to 2.0.4
Version 2.0.4 enables system tables events creation by default. The mandatory requirement for system tables is to have loads scheduler services (JethroLoadsScheduler) run automatically. The service must be started and enabled to run automatically as follows:
1. Edit the service.ini file and set the third parameter for all active instances to yes
> vi $JETHRO_HOME/instances/services.ini
–> {instance-name}:port:yes:yes:yes
2. Start the Jethro loads scheduler service:
> service jethro start {instance-name} loadscheduler
New Features
JD-3593 Enable system table events creation by default
Improvements
JD-2784 Flatten query write: remove sub-queries and merge it into the upper query to reduce multi-level queries overheads and avoid where on external queries (optimization not enabled by default)
JD-3591 Count distinct optimization for high cardinality columns: evaluate count distinct per partition to reduce memory footprint and enable using fast bitmap based evaluation
JD-3697 Show cubes – add status column
Bug Fixes
JD-2873 Push where – not performed if contains IN clause
JD-3173 Fix issues with local cache statistics
JD-3129 Push where – Potential server failure following applying push where optimization
JD-3607 Slow query execution with count distinct
JD-3677 Push where – serve failure following partial push where (one of the where expressions is IN expression)
JD-3691 Accuracy issue: when where expression contains NOT IN and join index defined on the NOT IN column
JD-3700 Adaptive cache – adaptive cache results not being invalidated after dim table append
Version 2.0.3
RELEASE DATE: DEC 16TH, 2016
Bugs fixes and improvements.
Issues list:
Improvements
JD-3329 Utility to reconstruct column index from column data
JD-3622 Expand cubes with where support to allow complex WHERE clauses
JD-3662 Generate cube – additional informative message in jethroserver.log when cube is not generated
JD-3675 Enable hi performance count distinct execution mode via count distinct container for CSN128 and Value Hash group by plans
Bug Fixes
JD-3297 Query with HAVING is not finding a cube match
JD-3404 Cube not found by Cube Generator while cube gen query sent to server find a cube
JD-3597 Query hangs – join with right table is sub-query that returns very big result set
JD-3617 Server failure due to query retry as a result of adaptive fingerprint mismatch exception
JD-3649 Memory leak on cubes generation
JD-3654 Adaptive incremental mode not active if new partition was added
JD-3664 Local cache does not update itself with new events (since 2.0.2)
JD-3681 Memory leak at count distinct queries using count distinct container
Version 2.0.2
RELEASE DATE: NOV 25TH, 2016
Bugs fixes and minor improvements.
Issues list:
Improvements
JD-3401 ODBC: Enhancements for copy TDC file in ODBC installation
Bug Fixes
JD-3458 Jethro server failure during active load while frequent queries sent to the system table (adaptive problem)
JD-3606 Maint service fail and stop due to problem in garbage collection
JD-3608 Maint service fail to initialize in case it fails to commit session configuration
JD-3619 ODBC: fix issue that cause Tableau to fail when disconnect is called without connect
Version 2.0.1
RELEASE DATE: NOV 17TH, 2016
Fixes for 2.0.0 known limitations/bugs.
Issues list:
Improvements
JD-3574 Stop/start of cubes generation/update via set command (no maint restart required)
Bug Fixes
JD-3416 Cubes with avg(if) are not matched
JD-3585 Auto cubes – read from cube accuracy issue when two different aggregations encapsulate an identical CASE/IF expression
JD-3586 SHARED.WHERE WARN Release shared object cannot found a relevant object in map
JD-3587 Auto cubes – read from cube accuracy issue when AVG aggregation encapsulate CASE/IF expression
JD-3588 JethroClient does not exit gracfully after failure (exception handling issue)
Version 2.0.0
RELEASE DATE: NOV 15TH, 2016
Jethro 2.0 introduces auto micro cubes.
Jethro auto micro cubes brings the great performance value of pre-aggregated cubes, combined with the ability to automatically generate and transparently maintain those cube.
Cubes can be useful to accelerate performance for certain set of queries, mainly queries that are not highly filter and would normally scan large parts of the data set. Jethro optimizer combines cubes with indexing and caching technology to ensure interactive queries response time on wide range of use cases.
Issues list:
New Features
JD-2590 Auto Micro Cubes – Match and rewrite dynamic aggregations
JD-2609 Adaptive incremental – Expend support for additional queries types (CSH Hash)
JD-2786 System Tables
JD-2788 System Table for loader events
JD-2919 Auto Micro Cubes – Dynamic aggregation of count distinct queries
JD-2921 Auto Micro Cubes – Support hierarchy levels for timestamp columns
JD-2937 Auto Micro Cubes – Auto generate dynamic aggregations from incoming queries
JD-3110 Auto Micro Cubes – Manual cubes generation via GENERATE CUBES
JD-3111 Auto Micro Cubes – Support cube with where
JD-3112 Auto Micro Cubes – Find match for aggregators that has internal case expression
JD-3118 Auto Micro Cubes – Match count distinct aggregation queries without group by to cubes
JD-3411 Auto Micro Cubes – Allow aggregation queries without group by and with where to generate and match cubes
JD-3345 Auto Micro Cubes – Generate and match cubes for queries without measures
JD-3348 Auto Micro Cubes – SHOW CUBES and related cubes management commands
Improvements
JD-2838 Enable fetch and read adaptive cache files to/from local cache
JD-2839 Add section separators to the jethro loader report file
JD-2878 Add watchdog for JethroMaint
JD-2918 Optimized count distinct for CASE/IF in distinct aggregator
JD-2938 New instances parameters should be set with Yes as default in Service.ini
JD-3043 Count Distinct Optimization: Allow push group by functions and joins to upper query with count distinct using distinct values bitmaps and support adaptive incremental for count distinct
JD-3085 Improve adaptive write efficiency via large write chunks
JD-3183 Adaptive cache read: split each column read according to the size of current TupleSet parameter
JD-3210 Update incremental cubes by maint
JD-3285 Show active queries – add new option to status column: caching
Bug Fixes
JD-2204 Server failure when query is VERY long and includes text after “;”
JD-3137 Memory leak – related to IN with sub-query in outer query
JD-3142 Memory leak of Global UKI
JD-3363 Possible deadlock/leak in buckets merge
JD-3370 Join index not used when where clause contains OR not null
JD-3379 Server failure when query reads all adaptive columns into single buffer
JD-3467 Query failed with error – limit issue
JD-3507 Jethro server failed to execute query with IN literal left operand
JD-3514 Jethro Server failure – query hit partition holder internal error
Version 1.6.11
RELEASE DATE: NOV 9TH, 2016
Bug Fixes
JD-3461 Fix for JethroAdmin list-storage-instances
JD-3544 Query with UNION ALL as subquery stored in adaptive cache failed to update on data load/update
JD-3425 Loader doesn’t reject tokens with blank spaces loaded into numeric columns
JD-3524 Fix server failure in case there is filter with IN expression and a query on the right side of IN has filter of its own
JD-3438 Fix local-conf typos
Version 1.6.10
RELEASE DATE: OCT 12TH, 2016
Improvements
JD-2910 Copy TDC file to the right place upon ODBC installation
JD-3068 SHOW PARAM: present changed parameters modification level including set global and support UNSET command
JD-3239 Add driver version (odbc, jdbc, JethroClient) to JethroServer log
JD-3277 Improve loader performance (streamer)
JD-3282 Collect logs should now collect revisions of schema.xml
JD-3290 Update status codes returned from the ODBC
JD-3298 Loader performance improvement – low cardinality keys
JD-3330 SHOW TABLE COLUMN: reduce memory consumption when there are multiple high cardinality columns
JD-3389 Collect-logs – add instance name and date to log files and improve error handling
Bug Fixes
JD-1506 IN/NULL issue: NOT IN with NULL value in IN list should not return the NULL value
JD-2690 IN/NULL issue: IN expression with list of literal doesn’t ignore NULL in the list
JD-3108 Adaptive cache error in log: “failed to get schema object”. Not an error – should be trace message
JD-3318 ORDER BY return wrong order when sorting positive and negative numbers
JD-3319 UNION ALL – duplicated columns
JD-3320 Session and global parameter files potentially can be corrupted if SET commands issued concurrently
JD-3336 Inconsistent EFS list operation cause wrong read of metadata and valid columns removed by GC. Fix version file validation check
JD-3351 UNION ALL with duplicate literals – duplicated columns when reading from adaptive
JD-3364 Partition size limitation of 4B rows due to file identifier rowID wrap around
JD-3376 Collect logs password is not provided and no appropriate error is issued
JD-3378 Server failure for queries with duplicate aggregators when reading from adaptive in incremental scenario
JD-3381 odbc 32 on windows 7: Cannot create a DSN
Version 1.6.9
RELEASE DATE: SEP 6TH, 2016
New Features
JD-3092 KILL QUERY – interactive kill of long running query
Improvements
JD-3175 Add JDBC support for getTimestamp(calendar) and item MergeColumnHeaderCells methods
JD-3209 support ORDER BY degrees (ordinal number)
JD-3215 Optimization for string functions in where: add optimization for IN (func(col) IN (…))
JD-3280 Support queries selecting literals without FROM clause
Bug Fixes
JD-2663 Wrong result when using cast as “float” with quotes
JD-2769 JethroClient issue error when run in the background
JD-3091 Query cancel does not terminate certain queries
JD-3124 Lock files are not removed
JD-3142 Memory leak of Global UKI
JD-3205 Wrong results: where optimization for string function return treat NULL return value as true
JD-3267 DDL commands fail after drop of dim table (drop table, show tables extended, create join index,…)
JD-3272 Loader and Maint fail following upgrade from 1.4.6 to 1.6.8 (fix leftover files handling)
JD-3276 Server failure during query following partition overwrite
Version 1.6.8
RELEASE DATE: AUG 11TH, 2016
New Features
JD-2787 Scheduled Loads (beta)
Improvements
JD-2930 Add command JethroAdmin list-storage-instances
JD-3016 JethroAdmin collect-logs support password entry
JD-3050 Limit memory factor to avoid huge memory consumption for base parameters
JD-3097 Set instance name max size to 32 characters
JD-3198 Allow Jethro installing with a user other then jethro
Bug Fixes
JD-2998 Server failure if order by column appear in group by inside a function
JD-3011 collect logs does not check instance name is valid – reports success
JD-3071 Illegal query combining Distinct and Order by causes server failure. Should issue an error.
JD-3093 Rare race condition on high concurrency causes all queries to wait on query queuing until time-out exceeded
JD-3114 ODBC: return true max column length for string
JD-3138 Fix threads counter to include all threads including top query thread
Version 1.6.7
RELEASE DATE: JUL 20TH, 2016
Bug Fixes
JD-3104 JethroMaint service stops after DROP PARTITION command is issued. This problem existing only in version 1.6.6 and affect only tables loaded only with version 1.6.6 Loader.
Version 1.6.6
RELEASE DATE: JUL 15TH, 2016
Improvements
JD-2530 Move Loader report log, profiles and rejects files for specific log folder
JD-3020 Allow setting loader job name and include it in loader report file name
JD-3048 TDVT: Support required function: Right(string, int)
JD-3059 Optimize query with group by and count distinct on the same column
JD-3060 Reduce PLANNER time for queries with very high cardinality (improve global UKI access)
JD-3063 Optimize Join performance: dynamically switch left/right tables in join node during query execution
JD-3076 Control and reduce memory footprint for instances with large number of string columns (reduce small initial hash size)
JD-3102 Updated Tableau TDC file
Bug Fixes
JD-2373 Jethro server stops/ when local cache is full and stops again to restart
JD-2961 Local cache won’t replace files after cache space is filled / stop fetching new files even when the is available space
JD-3014 Server failure when receiving query over empty table with function on a string under WHERE
JD-3015 Local cache copy files of tables that were dropped
JD-3022 Local cache metadata files potential unsafe commit
JD-3028 TDVT: issues with Locate and Substring functions taking Floor function as argument
JD-3029 TDVT: TIMESTAMPADD fails when getting interval of type Double
JD-3039 TDVT: Change LOG() to compute LN() and add LOG10() to apply to Tableau standards
JD-3066 TDVT: Wrong results – empty set when using cast of string representing real number to integer
JD-3074 ODBC does not signal exception when it occurs during accumulating results from server
JD-3080 Query on empty table using string functions fail with “Illegal construction of TupleSet. Size = 0” error
JD-3081 JDBC problem with getObject() API: Erroneous NULLs appears in a results set rows
JD-3082 SHOW LOCAL CACHE cause server failure when local cache path not defined
JD-3084 Query with UNION and COUNT DISTINCT fails with “unsupported expression type: 22”
JD-3086 Server failure on rare scenario when executing query on empty tables
JD-3095 SHOW LOCAL CACHE – shows inaccurate statistics
JD-3096 lock ignored as stale though was not actually stale – increase stale time out
Version 1.6.5
RELEASE DATE: JUN 27TH, 2016
Improvements
JD-2464 Ignore range in WHERE that includes all the values
JD-2962 SHOW LOCAL CACHE – make more usable: show cache/touched/storage information per table
JD-2987 Support round with precision
JD-3010 Improve fetch statistics logs
JD-3032 Loader streamer performance: improve rejects handling performance
Bug Fixes
JD-1734 Query CANCEL: fix scenario were query is not stopped when cancelled
JD-2654 Corrupted lock file issue causing locking to fail consistently
JD-2704 Query file if target list in select clause doesn’t include at least one column (for queries with having clause)
JD-2706 Case function can’t handle comparison between BIGINT and DOUBLE
JD-2760 Crash in SHOW LOCAL CACHE EXTENDED
JD-2761 Local cache disk full cause server crash on init
JD-2953 JethroServer – configuration values are transformed to lower case (problem for path values with upper case)
JD-2965 ODBC client: fix scenarios were query is not stopped when cancelled from ODBC
JD-2986 abs function not found
JD-2989 ODBC failed to handle group by boolean (fixed in ODBC 1.7)
JD-2994 ODBC 32bit doesn’t use TDC
JD-3004 Tmp files not deleted from instance-storage/tmp
JD-3024 Query with sub-query that contain tables from the outer query fail (server failure). Related to push where
JD-3053 Query with count distinct on QLIK function fail (server failure)
Version 1.6.4
RELEASE DATE: JUN 9TH, 2016
Improvements
JD-2972 Trim CASE/IF branches with false conditions
JD-2973 JethroAdmin collect log: force instance name and create collect log file name with date and time
JD-2985 Rewrite: additional clean-up for simple math operations in SUM aggregate
JD-2988 Improve adaptive read efficiency
JD-2995 Aggregation optimization: clean cast as date for min/max aggregations to achieve index based aggregation
Bug Fixes
JD-1552 Server crash on group by query following drop partition or failed load if number of unique strings loaded gt number of unique values in the string columns
JD-2933 Join with Union all returns bad results when adaptive is turned on. (result of col1 is duplicated to col2)
JD-2994 Tableau with ODBC 32bit doesn’t use TDC
JD-2996 Wrong results: filter on right table column from left join is ignored following push where optimization (JD-2785)
JD-3000 Server crash on Query that cast BIGINT to STRING
JD-3008 Adaptive read issue
JD-3009 Adaptive errors on WHERE IN query
Version 1.6.3
RELEASE DATE: MAY 26TH, 2016
New Features
JD-2917 Bitmap based GROUP BY COUNT DISTINCT. X5-X100 performance acceleration for many scenarios
JD-2928 Support SQL_TSI_FRAC_SECOND for timestampadd/diff and atan2 (Tableau requirements)
Improvements
JD-2619 Print modified configuration parameters at jethroserver.log start
JD-2840 Write adaptive indexes maps asynchronously. This allow keeping larger adaptive index maps.
JD-2943 Remove star transformation limitation for fact and dim
JD-2949 Fix fetch and execution tuple set sizes calculation to allow more parallel processing
JD-2951 Improve performance for where with functions on string columns
Bug Fixes
JD-2914 Adaptive write for large tuple set is exteriorly slow – need to optimize inefficient implementation
JD-2924 Issue with global configuration. Server fails to init after setting global config
JD-2940 Query failure: UNION ALL inside IN sub-query
JD-2957 TIMESTAMPDIFF – Accuracy issue when swapping the literals inside the function
JD-2958 Wrong expression ID allocation when rewriting aggregation (sum(-C1)) (was Internal Error)
JD-2968 TIMESTAMPADD – microsecond – Accuracy issue when using negative number in TIMESTAMPADD
JD-2975 Remove show local cache from collect-logs (pending show local cache fix)
JD-2980 Shared warn potential issue: release shared object cannot found a relevant object in map
Version 1.6.2
RELEASE DATE: MAY 6TH, 2016
New Features
JD-2669 Loader support hex value input to INT and BIGINT data type
Improvements
JD-2785 Rewrite: push where clause from outer query to inner query (optimization for tableau custom query)
JD-2865 Range Index – rewrite with IN to compute OR of index maps concurrently and to add to adaptive index
JD-2880 Functional Index – rewrite with IN to compute OR of index maps concurrently to add to adaptive index
JD-2903 Join performance: avoid copy of qualification columns during join execution
JD-2913 Performance – optimize GROUP BY over YEAR/MONTH/DAY functions with CAST to DATE
Bug Fixes
JD-2817 Various issues with transform of expression to SQL
JD-2899 Range index – block range index creation on data types other the INT and BIGINT
JD-2915 “CURRENT_TIMESTAMP” with quotes is not identified
JD-2916 Server crash on partition pruning while running now() on hours
Version 1.6.1
RELEASE DATE: APRIL 27TH, 2016
Improvements
JD-2867 Enable adaptive read from sub-queries
JD-2893 Performance – Force to push JOIN if there is where on the DIM and fact doesn’t include aggregations
Bug Fixes
JD-2895 Join index not used with QLIK function creating specific filter
JD-2901 Push join doesn’t push because wrong calculation of potential groups
JD-2904 Truncate table fails after load overwrite finish unsuccessfully
JD-2907 Fail to read from view (adaptive read fail) – getting CHECK failed: (index) lt (size())
JD-2911 Adaptive cache – sub-query results written to adaptive cache if sub-query is a view (should be disabled)
Version 1.6.0
RELEASE DATE: APRIL 20TH, 2016
New Features
JD-2811 Range indexes – optimize range filtering using indexes on ranges of values
JD-2847 Shared Where – cache where results for reuse by other concurrent queries that issues the same where clause
Improvements
JD-2662 Rewrite: Basic cleanup of functions in aggregators
JD-2863 Set ulimit by Installer to support high concurrency
JD-2869 Allow drop of multiple partitions in one operation
JD-2886 Performance – CAST to DATE in GB is not optimized
Bug Fixes
JD-2780 Partitions not pruned if same table appear more then once in the query
JD-2834 Query with left join return empty set when right table is empty
JD-2846 Local cache auto refresh doesn’t fetch new data if server was running before load start and schema was changed
JD-2848 Query with star transformation fails (issue with join remove)
JD-2854 Join column display order – on left join left table columns should be before right columns
JD-2859 Fetch dispatcher – constructor thread leak
JD-2861 JethroData::Lock deadlocks under intensive use
JD-2871 Fetch from column after adding it with add column fails
JD-2875 Pre-query operations are not measured in query total time
JD-2888 JethroLoader – first block size in each partition is less then blocksize
Version 1.5.2
RELEASE DATE: APRIL 4TH, 2016
New Features
JD-2745 Loader: support single digit TIMESTAMP format for month / day / hour / minute / sec
JD-2791 Show version command
Improvements
JD-2770 Loader: Report of success or failure to console
JD-2800 Optimize filter where: avoid processing blocks skipped for push down predict scenarios
JD-2807 Optimize filter where: remove HAVING node from filter where
JD-2819 Change default fetch tuple set size configuration to reduce memory allocation during fetch
JD-2823 set CAP_FAST_METADATA in TDC
Bug Fixes
JD-2461 Wrong results – rare potential error in adaptive delta: if not all partitions updates or if partitions are dropped and added
JD-2675 Cast to VARCHAR fails
JD-2759 Tableau doesnt uses CASE to convert boolean to integer – ODBC failes (due to JD-2726)
JD-2768 Empty set returned from adaptive cache following unexpected client disconnection
JD-2774 Performance: allow group by push function rewrite with count distinct aggregator if column in function already appear in group by
JD-2779 PushDown group by optimization fails on group by functions
JD-2814 Server failure on query with IN expression within function (CASE) inside the WHERE clause
JD-2822 JethroServer does not report fatal initialization errors to console (e.g. port busy)
JD-2825 Query failure (aggregation in CASE function)
Version 1.5.1
RELEASE DATE: MARCH 21TH, 2016
New Features
JD-2551 Add new functions: round(), sign(), sqrt(), space(), ascii(), tan(), cot(), atan(), sin(), asin(), cos(), acos(), degrees(), radians(), pi(), exp(), ln(), log(), replace()
Improvements
JD-2589 Support Redhat/Centos 7.2 (resolve service start/stop issue)
JD-2607 Optimization rewrite: Push multiple joins in group by to upper query
JD-2720 Add support for millisecond period in functions timestampadd() and timestampdiff()
Bug Fixes
JD-2783 Server failure on query IN clause inside aggregator
JD-2790 Results fix for queries with cast(func(..)) target list
JD-2796 Join Index – set index block size in relation to data block size
JD-2798 Now() function return current date without hours/minutes/seconds
Version 1.5.0
RELEASE DATE: MARCH 14TH, 2016
New Features
JD-2757 Support char() function
Improvements
JD-2378 Significantly improve high frequency small load support: data model block sizes is unaffected by load size
Bug Fixes
JD-2740 Union all fails or wrong results with order by or limit clauses
JD-2781 Query stuck when all partitions are pruned and select target list count() aggreagtor
Version 1.4.6
RELEASE DATE: MARCH 13TH, 2016
Improvements
JD-2754 Loader performance improvement – create global UKI only for updated partitions when finalizing
Version 1.4.5
RELEASE DATE: MARCH 8TH, 2016
Improvements
JD-2615 Decrease CROSS JOIN memory usage
JD-2765 Optimization: allow trying star transformation without where optimization for STRING quals if number of keys is small
Bug Fixes
JD-2489 Error when Tableau is connecting to an EMPTY Jethro instance
JD-2531 JDBC causes java.net.SocketException
JD-2575 Loader option LINE TERMINATED BY doesn’t work. Option disabled
JD-2598 CSN node execution time is higher than expected
JD-2660 JDBC – bad error message when no instance is specified in url
JD-2711 Server error: Local cache manager exit
JD-2721 Loader report – improperly printed reject file name
JD-2729 Double slashes in local cache path will prevent auto filling cache
JD-2751 Default value of query.optimizer.grouping.hash.base.size (10m) is too big and can add few seconds to query time due to large mem allocation
JD-2753 local-conf does NOT recognize parameters with capitalized letters and does not report them as wrong
JD-2764 Functional index for date/time optimization doesn’t work when timestamp literal is left of the column identifier
JD-2771 Exception on failure creating new thread not handled properly
JD-2772 Load table in overwrite mode may create wrong join indexes if wrong (non-default) block size settings are used
Version 1.4.4
RELEASE DATE: MARCH 1ST, 2016
Improvements
JD-2604 Add SQL Statistics to JethroAdmin collect-logs
Bug Fixes
JD-2760 crash in SHOW LOCAL CACHE
Version 1.4.3
RELEASE DATE: FEBRUARY 29ND, 2016
New Features
JD-2723 Support char(int or float) – TABLEAU only
JD-2744 Support curdate() function
Improvements
JD-2034 Improve fetch performance: decompression performance analysis and improve
JD-2719 Handle currently ODBC cast syntax for DATE “{ D …} and/or support cast to DATE with DATE keyword
JD-2726 Get from Tableau standard expressions instead of CASE WHEN boolean expression to gain index performance boost
JD-2735 Enable returning table and column identifiers as uppercase strings (configuration based)
JD-2746 Queries with now and date can be saved in adaptive cache since values are computed and saved as literals
Bug Fixes
JD-2734 concat doesn’t support concat of ‘/n’ sent from tableau
JD-2742 Query grouped by function with 2 or more identical aggregators fail
JD-2743 Query grouped by function and literal fails
Version 1.4.2
RELEASE DATE: FEBRUARY 22ND, 2016
Improvements
JD-2684 Support IFNULL function in Jethro SQL
JD-2701 Change NULLIFEQ() name to NULLIF()
JD-2709 Local cache refresh – use event update instead of file open
Bug Fixes
JD-2667 Multiple UNION ALL in one query fail
JD-2673 UNION ALL Query stuck while UNION ALL and JOIN combine on the same expression
JD-2676 UNION ALL Alias is not presented in query results
JD-2686 Unable to use literals/NULL as target column in VIEW
JD-2714 Wrong results related to adaptive sub-query in a specific scenario
Version 1.4.1
RELEASE DATE: FEBRUARY 15TH, 2016
Improvements
JD-2682 Allow join indexes to be create over dimension with no primary key
JD-2670 Loader format function unix_timestamp support fractional milliseconds
Bug Fixes
JD-2612 Query with group by count distinct take very long time
JD-2635 Issues with the distributed lock mechanism
JD-2664 Exception on partition pruning
JD-2672 Local cache refresh new files fails on hdfs
JD-2689 JethroServer creates too many threads causing queries fail
JD-2692 Performance: FI for range expressions (between) not always used
JD-2699 Query/server fail when using function over IN column: F(C1) IN (…)
Version 1.4.0
RELEASE DATE: FEBRUARY 2ND, 2016
Backward Compatibility
Data model conversion from previous versions is required.
Before starting service run: > Jethro U {instance-name} T
New Features
JD-2349 Support UNION ALL
Improvements
JD-2533 Column block encapsulation – store column block metadata and compressed value in a single file and avoid file append
Bug Fixes
JD-2522 Empty set result instead of exception that was raised – not consistent
JD-2637 Remove a warning message in partition pruning
JD-2645 Filter with “IN” does not work when the filter column is a partition key
JD-2653 Maint keeps life-long lock and old lock versions are not removed.
JD-2655 Query with quotes around the datatype in CAST fail
JD-2657 Sub-query adaptive causes simple query with sum and group by to fail. Return unclear error: CHECK failed: (index) lt (size())
Version 1.3.5
RELEASE DATE: JANUARY 28TH, 2016
Improvements
JD-2634 Optimization for group by function queries: push groups expression to outer queries
Bug Fixes
JD-2595 Slow performance when large joined table key is non-unique and there are multiple rows for each key
JD-2620 JethroServer crash after create/drop tables due to local cache pre-fetch
JD-2633 Performance degradation for distinct queries after after upgrade to 1.3.4 due to change in assume referential integrity
JD-2636 Various issues with assume.referential.integrity version 2
Version 1.3.4
RELEASE DATE: JANUARY 21TH, 2016
Improvements
JD-2491 Optimization: Push Join after GB to upper query (single join)
JD-2525 Assume referential integrity – version 2
JD-2538 Enable adaptive cache for sub-query results
JD-2582 SHOW ALL TABLES/PARTITIONS/COLUMNS
JD-2613 Reduce hash size factor of join node hash
Bug Fixes
JD-2541 JethroLoader Crash when the path to the local cache directory in local-conf.ini is wrong
JD-2577 Service jethro restart – try to bring up a new Maint before the old one went down completely.
JD-2597 Wrong results when using CAST over aggregator in queries optimized with push joins to upper query
JD-2611 Wrong results: GB with CSN 128bit returns
JD-2614 Complex query with cross join and string column crash
Version 1.3.3
RELEASE DATE: DECEMBER 30TH, 2015
Bug Fixes
JD-2584 Block drop partition column with alter table drop column
JD-2593 Loader can crash we using local cache in load
Version 1.3.2
RELEASE DATE: DECEMBER 28TH, 2015
Improvements
JD-2496 Allow more than one <=> operator in JOIN qualification
Bug Fixes
JD-2583 ADD COLUMN fails when table has partitions
JD-2585 Wrong results when using offset gt number of rows in result set
Version 1.3.1
RELEASE DATE: DECEMBER 18TH, 2015
New Features
JD-2555 Support OFFSET clause in LIMIT clause
Improvements
JD-2192 Improve Show Local Cache information
Bug Fixes
JD-2542 Partition pruning – fix minor memory leak
JD-2554 Crash on partition pruning if table appears in more than one sub-query
JD-2572 Loader memory leak – keys pool doesn’t release temp buffer
Version 1.3.0
RELEASE DATE: DECEMBER 12TH, 2015
Backward Compatibility
Data model conversion from previous versions is required.
Before starting service run: > Jethro U {instance-name} K
New Features
JD-1526 Partitions pruning
JD-2362 Support Tableau Date data type usage
JD-2515 Support SQL functions “TIMESTAMPADD” and “TIMESTAMPDIFF”
Improvements
JD-2300 Integrate with libhdfs3 (beta mode)
JD-2354 update ODBC convert capabilities
JD-2379 Keep keys (values and indexes) in multiple files (file per block) and support keys recovery
JD-2398 Trivial Distributed locker based on shared file system
JD-2512 QLIK set analysis support: add support for DISTINCT operator with QLIK function
JD-2526 QLIK set analysis: Add ‘TOTAL! ‘ support
JD-2540 Update ODBC capabilities to reflect supported functions
Bug Fixes
JD-2370 CASE – if first then is NULL then CASE may fail due to type mismatch
JD-2486 Incoherent error message when performing a numeric function (e.g. sum) on a STRING column
JD-2501 Client hangs when connection or queries hangs/return after very long time (actual execution time is small)
JD-2502 libhdfs3 – Loader can’t load data files from HDFS
JD-2510 Monitor – No indication when the Jethro server and Maint were stopped
JD-2511 Monitor – No indication for Maint start in the service.log
JD-2520 QLIK set analysis – use left join instead of inner join
JD-2523 Crash happens when IN clause includes NULL value
JD-2524 Loader streamer creates more data blocks then set in config limit
JD-2535 QLIK set analysis – fix cases when original query doesn’t have target column
Version 1.2.1
RELEASE DATE: NOVEMBER 18TH, 2015
Improvements
JD-2503 Loader Usability – error messages
JD-2506 Improve loader memory control: release keys cached buffers if cache is full and allow using local cache during load
Bug Fixes
JD-2077 Loader: various issues with loader errors logging
JD-2307 Fix loader report: add number of rows removed from overwrite scenarios
JD-2418 Fix misleading message when input file is corrupt: Input row is longer than 10m bytes
JD-2455 Change error message in log file when user is running more than one loader on same table.
JD-2492 Loader report: indicate in report file that rows are being rejected and point to rejects file
JD-2508 Missing configuration in loader report file
JD-2509 Loader memory leak for Timestamp column
Version 1.2.0
RELEASE DATE: NOVEMBER 13TH, 2015
New Features
JD-2356 Primary Key
JD-2197 Join indexes
JD-2469 QLIK set analysis functionality in Jethro SQL (Alpha)
Improvements
JD-2470 Join after Dim GB Optimization – basic scenario (distinct values, no aggregators, single join)
JD-2487 Improve assume referential integrity (drop fact from join when possible)
Bug Fixes
JD-2483 Some aggregations are executed via non-optimized path (full scan) instead of index based aggregation
JD-2499 rpm upgrade removes Jethro monitor job
JD-2500 JOIN invalid qualifier crashes
Version 1.1.4
RELEASE DATE: OCTOBER 27TH, 2015
Bug Fixes
JD-2363 Libhdfs messages captured in logs as ERROR even when it’s informational or warning
JD-2460 Error received on group by query using adaptive delta
JD-2465 Loader test and recovery – didn’t work on functional columns
Version 1.1.3
RELEASE DATE: OCTOBER 14TH, 2015
Improvements
JD-2382 Improve performance of GROUP BY queries with datetime functions on groups
JD-2445 Add date/time FI optimizations for YEAR range and IN
Bug Fixes
JD-2361 wrong results for tpch query: Q19
JD-2434 TPCH Q3b query is causing server failure
JD-2429 Server crash during query with where clause that include column<>”
JD-2430 Memory leak in join queries
JD-2436 Loader does not reject lines with timestemp bigger than max timestamp
JD-2444 Loader issue: unix_timestamp format for timestamp in doesn’t work. User gets a reject: “failed to create unix time”
JD-2363 Do not LOG libhdfs stderr capture as ERROR if it is not
JD-2423 Loader exits with positive status when loader reach maximum rejects
JD-2338 JethroAdmin.sh call service stop/start doesn’t support multi versions installation
JD-2432 jethromonitor.sh does not report instances initialization to /var/log/jethro/monitor.log
JD-2186 loader should set number of threads to 1 in the case of 2 core machines
Version 1.1.2
RELEASE DATE: SEPTEMBER 29TH, 2015
New Features
JD-2359 Cross Join support
JD-2367 Null safe equi join with operator <=>
Improvements
JD-2351 String ranges optimization (use indexes by converting to IN)
JD-2400 Rewrite “JOIN … on A=B or A is null and B is null” to use <=> operator
Bug Fixes
JD-2018 Loader crash at create global uki stage for high cardinality column
JD-2309 JethroServer coredump when local-cache.ini is missing
JD-2318 Corrupt persist.xml file
JD-2327 Query with cross join crash the server
JD-2329 Loader does NOT reject INT with spaces: “12 3”
JD-2368 show local cache – crash if issued before first background update
JD-2369 Loader terminates with error reading from file when creating global UKIs
JD-2370 CASE – if first then is NULL then CASE may fail due to type mismatch
JD-2371 Nested queries – crash because semantic error is identified by NS but not captured and is passed to QueryBuilder
JD-2372 nested queries – join tables are not traversed by semantic for validity
JD-2402 Server stuck on execution
Version 1.1.1
RELEASE DATE: AUGUST 23TH, 2015
Improvements
JD-2353 Support assume referential integrity – mode that eliminate all referenced table from join. phase I : support star schema
JD-2355 allow position reference in GB
Bug Fixes
JD-1711 SELECT with sub-query attributes that aren’t supported should be rejected
JD-1826 SELECT accept unsupported options
JD-2316 CASE mishandles NULL type conversion
JD-2322 wrong result: explicit conversion NULL::float returns 0 in CASE statement
JD-2330 ODBC – update and fix connection properties
JD-2347 IN reject correct aggregation function targets in sub-query due to interim data-type
JD-2365 Loader overwrite option removes functional indexes
Version 1.1.0
RELEASE DATE: AUGUST 11TH, 2015
New Features
JD-1045 Function Indexes Phase 1
Improvements
JD-2344 ODBC driver for 32 bit (Windows)
JD-2343 Optimize queries with repeated expressions in target list
JD-2342 Join optimization: add transformation from outer join to inner join to allow star transformation optimization
JD-2341 Improve JOIN optimization – don’t disable star transformation if another outer join exists after inner join
JD-2340 Adaptive caching for IN with Subquery
JD-2199 Shared adaptive cache
JD-2188 Loader: optional per-column NULL representation
Bug Fixes
JD-2337 Fix merge all filter maps before index full scan
JD-2330 ODBC – update and fix connection properties
JD-2328 Fetch threads in filtered where + partitions
JD-2324 wrong results if GB col is also not last join qualification
JD-2298 Fetch SN performance issue
JD-2286 Datediff function performance is slow
JD-2282 missing if() function for TIMESTAMP causes IF stmt to fail and CASE to crash
JD-2239 wrong qualifier identification if multiple ‘=’ used in join ON clause (was: wrong results on JOIN queries with casting)
JD-2238 Query failed with “request execution failed” (iBasis – flat queries)
JD-2232 Query failed without clear error message. Uses unsupported function within CAST
JD-2231 Query failed with “SemanticException – Cross join is not supported – ON clause requires at least one join condition”
JD-2219 Server memory leaks in query execution
JD-2208 Local-cache.ini is not effecting cache refresh
JD-2177 Wrong result for complex expression that use filter where
JD-2122 Service jethro command does not work well in CentOs 7
Version 1.0.2
RELEASE DATE: JUNE 6TH, 2015
New Features
JD-2217 Support MapR distribution
JD-2073 SHOW PLAN – show detailed execution plan (in log)
Bug Fixes
JD-2214 JethroServer crash after getting cancellation from Client (most likely via pentaho)
JD-2213 Query fails error – join between views
JD-2194 Segfault – server fails on a specific query
JD-2191 Query Cancelled by user. resulted in an internal error and query hanging
JD-2187 JerthroAdmin: incorrect error message on attache instance
JD-2185 Problem with environment setup with HDP (Hortonworks) installation
JD-2180 execution counter – hole of 90% of the time – count distinct
JD-2179 JethroServer crashed or stuck on query
JD-2172 Unsupported version message send due to serialization version mismatch doesn’t provide info required to identify to problem
JD-2145 Loader crashed on deadlock
JD-2099 Schema oid usage isn’t efficient and can also cause lock bugs
Version 1.0.1
RELEASE DATE: MAY 10TH, 2015
New Features
JD-1138 ALTER TABLE ADD/DROP COLUMNS
Improvements
JD-2168 refactor adaptive rank calculation
JD-2005 Dynamic refresh of local cache based on blocks access frequency
Bugs
JD-2178 JethroODBC: ODBC driver should disable catalog support
JD-2176 Jethro initialization failed on test server
JD-2173 when adaptive cache deletes a query, it writes many junk lines in jethro.log
JD-2171 count(*) on columns from sub-query fails ” ‘Scalar’ node build failed”
JD-2117 Adaptive query cache – use FULL if can’t use INCREMENTAL
JD-1950 Query with IN returned wrong result on branch ‘JD-1947_non_opt_build’