How to resolve "Illegal Hadoop Version: Unknown (expected A.B.* format)"?

How to resolve "Illegal Hadoop Version: Unknown (expected A.B.* format)"?

book

Article ID: KB0076992

calendar_today

Updated On:

Products Versions
TIBCO Data Virtualization 7.0 and higher

Description

Hive data source fails and the log shows the root cause to be:
           Illegal Hadoop Version: Unknown (expected A.B.* format)
as indicated below:

------------
Caused by: org.apache.thrift.transport.TTransportException: Could not create http connection to jdbc:hive2://www.abcdefcompany.com:10001/default;transportMode=http;httpPath=cliservice;hive.server2.thrift.http.path=cliservice;ssl=true;kerberos.krb5.conf=/etc/krb5.conf;principal=hive/_LOCAL@TEST.COM;auth=kerberos;kerberosAuthType=fromSubject. org.apache.http.client.ClientProtocolException
    at org.apache.hive.jdbc.HiveConnection.createHttpTransport(HiveConnection.java:257)
    at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:187)
    ... 59 more
Caused by: org.apache.thrift.transport.TTransportException: org.apache.http.client.ClientProtocolException
    at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:297)
    at org.apache.thrift.transport.THttpClient.flush(THttpClient.java:313)
    at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:65)
    at org.apache.hive.service.cli.thrift.TCLIService$Client.send_OpenSession(TCLIService.java:150)
    at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:142)
    at org.apache.hive.jdbc.HiveConnection.createHttpTransport(HiveConnection.java:249)
    ... 60 more
Caused by: org.apache.http.client.ClientProtocolException
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:909)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
    at org.apache.thrift.transport.THttpClient.flushUsingHttpClient(THttpClient.java:251)
    ... 65 more
Caused by: org.apache.http.HttpException: Illegal Hadoop Version: Unknown (expected A.B.* format)
    at org.apache.hive.jdbc.HttpKerberosRequestInterceptor.process(HttpKerberosRequestInterceptor.java:69)
    at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:109)
    at org.apache.http.protocol.HttpRequestExecutor.preProcess(HttpRequestExecutor.java:176)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:518)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
    ... 67 more
Caused by: java.lang.RuntimeException: Illegal Hadoop Version: Unknown (expected A.B.* format)
    at org.apache.hadoop.hive.shims.ShimLoader.getMajorVersion(ShimLoader.java:159)
    at org.apache.hadoop.hive.shims.ShimLoader.loadShims(ShimLoader.java:134)
    at org.apache.hadoop.hive.shims.ShimLoader.getHadoopThriftAuthBridge(ShimLoader.java:120)
    at org.apache.hive.service.auth.HttpAuthUtils.getKerberosServiceTicket(HttpAuthUtils.java:52)
    at org.apache.hive.jdbc.HttpKerberosRequestInterceptor.process(HttpKerberosRequestInterceptor.java:63)
    ... 71 more
------------

Resolution

This is due to a problem with the "common-version-info.properties" file bundled inside the hadoop-common.jar. As each Hive/Hadoop vendor typically provides their own version of the hadoop commons jar, please contact the vendor and ask them to provide a hadoop-common.jar (along with any accompanying jar files) compatible with the JRE version shipped with the TIBCO Data Virtualization (TDV) installation. The JRE version can be found by running:
        <TDV_INSTALL>/jre/bin/java -version   (on Unix)
        <TDV_INSTALL>\jre\bin\java -version   (on Windows)

For example let's assume that the TDV Install directory is : C:\apps\tdv_80\, then the above command would be :
         C:\apps\tdv_80\jre\bin\java -version

Let us assume that the result of the above command was :
        java version ""1.8.0_192""

Request the vendor to provide jars that work with java version 1.8

 

Issue/Introduction

How to resolve ""Illegal Hadoop Version: Unknown (expected A.B.* format)""