Options

Hive connection error

chasrinichasrini Member Posts: 4 Contributor I
edited December 2018 in Help

Trying to connect to Hive but its failing. Connection being done through kerberos. Below is the error I am getting

 

Exception: eu.radoop.exception.ConnectionException
Message: The following Hadoop configuration properties must be defined, but a connection error prevented the automatic retrieval of these properties: hive.execution.engine, yarn.resourcemanager.scheduler.address, yarn.resourcemanager.resource-tracker.address, yarn.resourcemanager.admin.address, mapreduce.jobhistory.done-dir, mapreduce.jobhistory.intermediate-done-dir, dfs.user.home.dir.prefix.
Stack trace:

eu.radoop.datahandler.HadoopContext.fetchDynamicHadoopSettings(HadoopContext.java:304)
eu.radoop.datahandler.HadoopContext.fetchDynamicHadoopSettings(HadoopContext.java:250)
eu.radoop.datahandler.HadoopContext.getHadoopConfiguration(HadoopContext.java:229)
eu.radoop.datahandler.mapreducehdfs.MapReduceHDFSHandler.getMrhdfsLowLevel(MapReduceHDFSHandler.java:445)
eu.radoop.datahandler.mapreducehdfs.MapReduceHDFSHandler.lambda$testTempPermission$32(MapReduceHDFSHandler.java:1598)
eu.radoop.tools.ExceptionTools.checkOnly(ExceptionTools.java:277)
eu.radoop.datahandler.mapreducehdfs.MapReduceHDFSHandler.testTempPermission(MapReduceHDFSHandler.java:1596)
eu.radoop.RadoopNest.doWork(RadoopNest.java:549)
com.rapidminer.operator.Operator.execute(Operator.java:1025)
com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:812)
com.rapidminer.operator.ExecutionUnit$2.run(ExecutionUnit.java:807)
java.security.AccessController.doPrivileged(Native Method)
com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807)
com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428)
com.rapidminer.operator.Operator.execute(Operator.java:1025)
com.rapidminer.Process.execute(Process.java:1315)
com.rapidminer.Process.run(Process.java:1290)
com.rapidminer.Process.run(Process.java:1181)
com.rapidminer.Process.run(Process.java:1134)
com.rapidminer.Process.run(Process.java:1129)
com.rapidminer.Process.run(Process.java:1119)
com.rapidminer.gui.ProcessThread.run(ProcessThread.java:65)

Cause
Exception: eu.radoop.exception.NoStackTraceOperatorException
Message: HiveQL error. Message: Connection error for Driver "HiveDriverShim (radoop_hive_0.13.0): org.apache.hive.jdbc.HiveDriver@6e9d60e1" with URL "radoop_hive_0.13.0jdbc:hive2://sl73caehmp04.visa.com:10000/default;principal=hive/sl73caehmp04.visa.com@HADOOP.A...
Stack trace:


Cause
Exception: eu.radoop.datahandler.hive.StatementPool$HiveDriverException
Message: Connection error for Driver "HiveDriverShim (radoop_hive_0.13.0): org.apache.hive.jdbc.HiveDriver@6e9d60e1" with URL "radoop_hive_0.13.0jdbc:hive2://sl73caehmp04.visa.com:10000/default;principal=hive/sl73caehmp04.visa.com@HADOOP.ANA.VISA.COM;auth=kerberos;kerberosAuthType=fromSubject;sasl.qop=auth": Could not open client transport with JDBC Uri: jdbc:hive2://sl73caehmp04.visa.com:10000/default;principal=hive/sl73caehmp04.visa.com@HADOOP.ANA.VISA.COM;auth=kerberos;kerberosAuthType=fromSubject;sasl.qop=auth: java.net.ConnectException: Connection timed out: connect
Stack trace:

Tagged:

Answers

  • Options
    jpuentejpuente Employee, Member Posts: 53 RM Product Management

    Hi,

    Are you getting that error in a full test of the connection? Could you please share the full log?

     

    Regards,

    Jesús

  • Options
    carlosponcinellcarlosponcinell Member Posts: 1 Contributor I

    Hi,

    I try to configure RapidMiner with Radoop, but I don´t know how to configure parameters for: "Hive Server Addrees", and Hive "Port".

    Can you help me, or signal where I find it?

    Thanks,

  • Options
    asmahassani08asmahassani08 Member Posts: 31 Contributor II

    hello, 

    please I have this problem when I test radoop , how can I solve this please?

    [Oct 13, 2018 2:18:02 PM]: Connection test for 'hadoop connection' started.
    [Oct 13, 2018 2:18:02 PM]: Removed test Kerberos beacuse Security is not enabled
    [Oct 13, 2018 2:18:02 PM]: Using Radoop version 9.0.1.
    [Oct 13, 2018 2:18:02 PM]: Running 11 tests: [Hive connection, Fetch dynamic settings, Java version, HDFS, MapReduce, Radoop temporary directory, MapReduce staging directory, Spark staging directory, Spark assembly jar existence, UDF jar upload, Create permanent UDFs]
    [Oct 13, 2018 2:18:02 PM]: Running test 1/11: Hive connection
    [Oct 13, 2018 2:18:02 PM]: Hive server 2 connection (localhost:10000) test started.
    [Oct 13, 2018 2:18:06 PM] SEVERE: Test failed: Hive connection
    [Oct 13, 2018 2:18:06 PM]: Cleaning after test: Hive connection
    [Oct 13, 2018 2:18:06 PM]: Total time: 4.265s
    [Oct 13, 2018 2:18:06 PM]: java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
    	at org.apache.hive.jdbc.HiveConnection.<clinit>(HiveConnection.java:109)
    	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
    	at eu.radoop.classloader.HiveDriverShim.connect(HiveDriverShim.java:71)
    	at java.sql.DriverManager.getConnection(DriverManager.java:664)
    	at java.sql.DriverManager.getConnection(DriverManager.java:247)
    	at eu.radoop.datahandler.hive.PooledStatement.lambda$connect$0(PooledStatement.java:214)
    	at eu.radoop.tools.ExceptionTools.check(ExceptionTools.java:301)
    	at eu.radoop.datahandler.hive.PooledStatement.connect(PooledStatement.java:189)
    	at eu.radoop.datahandler.hive.StatementPool.getStatement(StatementPool.java:314)
    	at eu.radoop.datahandler.hive.StatementPool.getStatement(StatementPool.java:181)
    	at eu.radoop.datahandler.hive.HiveHandler.runFastQueryPrivilegedAction(HiveHandler.java:1010)
    	at eu.radoop.datahandler.hive.HiveHandler.runFastQuery(HiveHandler.java:986)
    	at eu.radoop.datahandler.hive.HiveHandler.getSimpleListFast(HiveHandler.java:2575)
    	at eu.radoop.datahandler.hive.HiveHandler.getTableList(HiveHandler.java:2507)
    	at eu.radoop.connections.service.test.connection.TestHiveConnection.call(TestHiveConnection.java:51)
    	at eu.radoop.connections.service.test.connection.TestHiveConnection.call(TestHiveConnection.java:25)
    	at eu.radoop.connections.service.test.RadoopTestContext.lambda$runTest$1(RadoopTestContext.java:279)
    	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:748)
    Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
    	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    	... 21 more

    [Oct 13, 2018 2:18:06 PM] SEVERE: java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
    [Oct 13, 2018 2:18:06 PM] SEVERE: Hive server 2 connection test failed. Please check that the server/daemon runs and is accessible on the address and port you specified.
    [Oct 13, 2018 2:18:06 PM] SEVERE: Test failed: Hive connection
    [Oct 13, 2018 2:18:06 PM] SEVERE: Connection test for 'hadoop connection' failed.
  • Options
    AtekkaAtekka Member Posts: 2 Contributor I
    Hello!

    based on this line from your output : "Hive server 2 connection (localhost:10000) test started."

    Can you let us know what is your config? Maybe you are using tunneling and a remote port is listening for you on localhost:10000 or you manually added this config ?

    Br.,
    Attila
Sign In or Register to comment.