NoClassDefFoundError Could not initialize class org.apache.hadoop.yarn.conf.YarnConfiguration

cazjckcazjck Member Posts: 6 Contributor I
edited December 2018 in Help

Hi,

 I try run Hadoop with single mode on Rapidminer. Howerver, I got issue . I using Rapidminer Studio 7.5

java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.conf.YarnConfiguration
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.rapidminer.pagerank.utilities.HadoopUtilities.runHadoopLocal(HadoopUtilities.java:173)
at com.rapidminer.pagerank.operator.PageRankOperator.doWork(PageRankOperator.java:78)
at com.rapidminer.operator.Operator.execute(Operator.java:1004)
at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812)
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807)
at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428)
at com.rapidminer.operator.Operator.execute(Operator.java:1004)
at com.rapidminer.Process.run(Process.java:1315)
at com.rapidminer.Process.run(Process.java:1179)
at com.rapidminer.Process.run(Process.java:1132)
at com.rapidminer.Process.run(Process.java:1127)
at com.rapidminer.Process.run(Process.java:1117)
at com.rapidminer.gui.ProcessThread.run(ProcessThread.java:65)

In build.gradle

dependencies {
//compile 'com.google.guava:guava:18.0'
compile "org.apache.hadoop:hadoop-hdfs:2.7.3"
compile "org.apache.hadoop:hadoop-client:2.7.3"
compile "org.apache.hadoop:hadoop-mapreduce-client-core:2.7.3"
compile "org.apache.hadoop:hadoop-yarn-common:2.7.3"
compile "org.apache.hadoop:hadoop-mapreduce-client-jobclient:2.7.3"
compile "org.apache.hadoop:hadoop-yarn-client:2.7.3"
compile "org.apache.hadoop:hadoop-yarn:2.7.3"
compile "org.apache.hadoop:hadoop-common:2.7.3"
}

Can everyone help me?

Answers

  • cazjckcazjck Member Posts: 6 Contributor I

    Help me :( .

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    You posted this twice. Doing so will not get your question answered quicker. I will delete the second topic. 

  • Pavithra_RaoPavithra_Rao Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 123 RM Data Scientist

    Hi,

     

    I guess if you provide the full path for the yarn.application.classpath property, the NoClassDefFoundError issue gets resolved.

    YARN configuration XML could be found here: /etc/hadoop/conf/yarn-site.xml

     

    Hope this helps, let us know.

     

    Cheers,

  • cazjckcazjck Member Posts: 6 Contributor I

    Thanks you so much.

    However,  When I run in Eclipse with Hadoop single mode  && cluster mode , it's ok .

    After I install  my extension  , I run extension in Rapidminer, occur exception.

    I don't know, How to add classpath  in  extension of Rapidminer ? Can you provide for me one example ?

    Sorry, I speak english not good.

  • Pavithra_RaoPavithra_Rao Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 123 RM Data Scientist

    Hi,

     

    Could you please check if the Yarn class path is defined in the following XML file:

    YARN configuration XML could be found here: /etc/hadoop/conf/yarn-site.xml

    user@linux$ sudo vi /etc/hadoop/conf/yarn-site.xml
    [ ... ]
    <property>
      <name>yarn.application.classpath</name>
        <value>
          $HADOOP_CONF_DIR,
          $HADOOP_COMMON_HOME/*,
          $HADOOP_COMMON_HOME/lib/*,
          $HADOOP_HDFS_HOME/*,
          $HADOOP_HDFS_HOME/lib/*,
          $HADOOP_MAPRED_HOME/*,
          $HADOOP_MAPRED_HOME/lib/*,
          $YARN_HOME/*,
          $YARN_HOME/lib/*,
          /etc/hadoop/conf,
          /usr/lib/hadoop/*,
          /usr/lib/hadoop/lib,
          /usr/lib/hadoop/lib/*,
          /usr/lib/hadoop-hdfs,
          /usr/lib/hadoop-hdfs/*,
          /usr/lib/hadoop-hdfs/lib/*,
          /usr/lib/hadoop-yarn/*,
          /usr/lib/hadoop-yarn/lib/*,
          /usr/lib/hadoop-mapreduce/*,
          /usr/lib/hadoop-mapreduce/lib/*
        </value>
    </property>

     Cheers,

  • cazjckcazjck Member Posts: 6 Contributor I

    Thanks you.

    But in code initialize Configuration throw exception. You can see some picture below:

     

     

    error.PNG

    error2.PNG

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn
    Why don't you just use the Radoop Extension?
  • cazjckcazjck Member Posts: 6 Contributor I

    I want to create a extension using MapReduce, I don't know use Hive and Spark . :(

Sign In or Register to comment.