How to resolve the error "java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration" while trying to connect to a Apache Hive Datasource in TIBCO Data Virtualization?

How to resolve the error "java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration" while trying to connect to a Apache Hive Datasource in TIBCO Data Virtualization?

book

Article ID: KB0072460

calendar_today

Updated On:

Products Versions
TIBCO Data Virtualization 8.3 and above

Description

The following error may be encountered when trying to connect to the Apache Hive data source:

User-added image
With the following stack trace:

 com.compositesw.cdms.datasource.DataSourceException: Unable to connect to data source "/shared/ARB/L1 Physical/ds_HPE/HPE_6" at "jdbc:hive2://PD1EZMDFDEV3lxpa.alrajhi.bank:10000/default;ssl=true;sslTrustStore=/usr/tibco/TDV_Server_8.5/conf/server/security/cis_server_truststore.jks".  Make sure the URL information, user name and password are correct.  [datasrc-3961025]                 at com.compositesw.cdms.datasource.PhysicalConnectionPoolManager.createPool(PhysicalConnectionPoolManager.java:114) ~[csserver.jar:?]                 at com.compositesw.cdms.datasource.PhysicalConnectionPoolManager.getPool(PhysicalConnectionPoolManager.java:62) ~[csserver.jar:?]                 at com.compositesw.cdms.ds.jdbc.AbstractRelationalDataSource.getPool(AbstractRelationalDataSource.java:821) ~[csmodule_datasource.jar:?]                 at com.compositesw.cdms.ds.jdbc.AbstractRelationalDataSource.validateConnection(AbstractRelationalDataSource.java:861) ~[csmodule_datasource.jar:?]                 at com.compositesw.cdms.datasource.DefaultDataSourceManager.testConn(DefaultDataSourceManager.java:546) ~[csmodule_datasource.jar:?]                 at com.compositesw.cdms.datasource.DefaultDataSourceManager.testConnectivity(DefaultDataSourceManager.java:481) ~[csmodule_datasource.jar:?]                 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]                 at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]                 at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]                 at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]                 at com.compositesw.common.services.ServiceResolver$1.invoke(ServiceResolver.java:18) ~[cscommon.jar:?]                 at com.sun.proxy.$Proxy23.testConnectivity(Unknown Source) ~[?:?]                 at com.compositesw.cdms.webapi.service.WDataSourceImpl.testConnection(WDataSourceImpl.java:247) ~[cswebapi-server.jar:?]                 ... 31 more Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration                 at org.apache.hadoop.hive.conf.MapRSecurityUtil.buildConfigurationFromDefaultFiles(MapRSecurityUtil.java:136) ~[?:?]                 at org.apache.hadoop.hive.conf.MapRSecurityUtil.getSslProtocolVersion(MapRSecurityUtil.java:174) ~[?:?]                 at org.apache.hive.jdbc.HiveConnection.createUnderlyingTransport(HiveConnection.java:516) ~[?:?]                 at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:545) ~[?:?]

 

Environment

All supported environments

Resolution

The following steps can be used to resolve this issue:

1. The error indicates that the proper class definition was not found because of missing jar files in the /apps/dlm/cis_ds_hive/lib directory.
2. We need to verify that the following jar files are present in this directory:
--------------------------------------------------------
commons-cli-1.2.jar
commons-configuration-1.6.jar
commons-logging-1.2.jar
hadoop-auth-2.7.3.jar
hadoop-common-2.7.3.jar
hadoop-hdfs-2.7.3.jar
hive-exec-2.1.1.jar
hive-jdbc-2.1.1.jar
hive-service-2.1.1.jar
htrace-core-3.1.0-incubating.jar
httpclient-4.4.jar
httpcore-4.4.jar
libfb303-0.9.3.jar
libthrift-0.9.3.jar
log4j-core-2.4.1.jar
--------------------------------------------------------
3. If any of the files are missing, then add them to the directory and restart the TDV Server.

Note: These files can be obtained from the Hive Team.

Issue/Introduction

This article describes the steps that can be used to resolve the NoClassDefFoundError while connecting to the Apache Hive Datasource.