The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.
Options

"Radoop Connection: Full Test Error"

kimusu2002kimusu2002 Member Posts: 5 Contributor II
edited June 2019 in Help
Hi,

I have a CDH5 4 Nodes cluster that I want to connect to using radoop and rapidminer. I am getting the error below when running distributed file system upload:
[May 19, 2015 5:33:09 PM]: Distributed file system upload started.
[May 19, 2015 5:37:10 PM] SEVERE: Test data upload to the distributed file system timed out. Please check the NameNode and DataNode services and their logs for traces of errors.

I have opened all necessary ports, for your information.

Thanks in advance for your help.


Below please find the full test log:
[May 19, 2015 5:32:58 PM]: Connection test for 'master.example.com' started.
[May 19, 2015 5:32:58 PM]: Hive server 2 connection (???.???.???.??:10000) test started.
[May 19, 2015 5:32:59 PM]: Hive server 2 connection test succeeded.
[May 19, 2015 5:32:59 PM]: Retrieving required configuration properties...
[May 19, 2015 5:33:00 PM]: Successfully fetched property: yarn.resourcemanager.scheduler.address
[May 19, 2015 5:33:00 PM]: Successfully fetched property: yarn.resourcemanager.resource-tracker.address
[May 19, 2015 5:33:00 PM]: Successfully fetched property: yarn.resourcemanager.admin.address
[May 19, 2015 5:33:00 PM]: Successfully fetched property: yarn.application.classpath
[May 19, 2015 5:33:00 PM]: MapReduce Home added to yarn.application.classpath ($HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*)
[May 19, 2015 5:33:00 PM]: Successfully fetched property: mapreduce.jobhistory.address
[May 19, 2015 5:33:01 PM]: Distributed file system test started.
[May 19, 2015 5:33:01 PM]: Distributed file system test succeeded.
[May 19, 2015 5:33:01 PM]: MapReduce test started.
[May 19, 2015 5:33:02 PM]: MapReduce test succeeded.
[May 19, 2015 5:33:02 PM]: Radoop temporary directory test started.
[May 19, 2015 5:33:02 PM]: Radoop temporary directory test succeeded.
[May 19, 2015 5:33:02 PM]: MapReduce staging directory test started.
[May 19, 2015 5:33:02 PM]: MapReduce staging directory test succeeded.
[May 19, 2015 5:33:02 PM]: Connection test for 'master.example.com' completed successfully.
[May 19, 2015 5:33:07 PM]: --------------------------------------------------
[May 19, 2015 5:33:07 PM]: Connection test for 'master.example.com' started.
[May 19, 2015 5:33:07 PM]: Hive server 2 connection (???.???.???.??:10000) test started.
[May 19, 2015 5:33:07 PM]: Hive server 2 connection test succeeded.
[May 19, 2015 5:33:07 PM]: Retrieving required configuration properties...
[May 19, 2015 5:33:09 PM]: Successfully fetched property: yarn.resourcemanager.scheduler.address
[May 19, 2015 5:33:09 PM]: Successfully fetched property: yarn.resourcemanager.resource-tracker.address
[May 19, 2015 5:33:09 PM]: Successfully fetched property: yarn.resourcemanager.admin.address
[May 19, 2015 5:33:09 PM]: Successfully fetched property: yarn.application.classpath
[May 19, 2015 5:33:09 PM]: MapReduce Home added to yarn.application.classpath ($HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*)
[May 19, 2015 5:33:09 PM]: Successfully fetched property: mapreduce.jobhistory.address
[May 19, 2015 5:33:09 PM]: Distributed file system test started.
[May 19, 2015 5:33:09 PM]: Distributed file system test succeeded.
[May 19, 2015 5:33:09 PM]: MapReduce test started.
[May 19, 2015 5:33:09 PM]: MapReduce test succeeded.
[May 19, 2015 5:33:09 PM]: Radoop temporary directory test started.
[May 19, 2015 5:33:09 PM]: Radoop temporary directory test succeeded.
[May 19, 2015 5:33:09 PM]: MapReduce staging directory test started.
[May 19, 2015 5:33:09 PM]: MapReduce staging directory test succeeded.
[May 19, 2015 5:33:09 PM]: Connection test for 'master.example.com' completed successfully.
[May 19, 2015 5:33:09 PM]: --------------------------------------------------
[May 19, 2015 5:33:09 PM]: Integration test for 'master.example.com' started.
[May 19, 2015 5:33:09 PM]: The test may require several minutes to complete.
[May 19, 2015 5:33:09 PM]: Distributed file system upload started.
[May 19, 2015 5:37:10 PM] SEVERE: Test data upload to the distributed file system timed out. Please check the NameNode and DataNode services and their logs for traces of errors.
Tagged:

Answers

  • Options
    DocMusherDocMusher Member Posts: 333 Unicorn
    Hi,
    I am absolutely a rookie in this topic, but does changing time out and other settings in your Rapidminer preferences change anything, it was successful for me.
    Cheers
    Sven
  • Options
    kimusu2002kimusu2002 Member Posts: 5 Contributor II
    Thanks Sven,

    But as you could see in my post, the the time difference is 4 seconds, and I set a very large time out number of seconds, and it still doesn't work, I think this is a file/directory permission issue, will check that then come post here again
  • Options
    MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,525 RM Data Scientist
    Hi,

    did you have a look on the cloudera manager logs?
    And: I would again recommend to ask our support at support.rapidminer.com, since i think Sven and myself are the only persons here using Radoop

    Cheers,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • Options
    DocMusherDocMusher Member Posts: 333 Unicorn
    Hi,
    Overall, I really like the RADOOP extension. The main reason for that is it provides me the possibility to run processes without squeezing my computer while running. Martin will agree with me that the extra power you get gives you no reason to have the same attention to preprocessing your data. I hope more operators become available within RADOOP.  What is the planning on doing that? Down time is minimal in my set-up. Speed increases subjectively 10 times.
    Cheers
    Sven
  • Options
    MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,525 RM Data Scientist
    If you look on the last 2 releases you can see that we added more predictive power by adding Linear Regression, Logistic Regression and the decision tree.
    So i think this shows that we are activly putting more algorithmic power into radoop.

    We had our radoop team in Dortmund last week. I got some time to talk with them - trust me, there is even more to come :)
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
Sign In or Register to comment.