ANNOUNCEMENT: RAPIDMINER 9.1 HAS JUST BEEN RELEASED!   PLEASE DOWNLOAD AND GIVE FEEDBACK. ENJOY AND HAPPY RAPIDMINING!   -- @sgenzer – Community Manager

Process Pushdown error

asmahassani08asmahassani08 Member Posts: 29 Contributor I
edited December 3 in Help
I'm working on radoop. I'm using the sparkRM to extract association rules. for that, I apply the fpgrowth in the sparkRM operator subprocess.
I have these error : SparkRM: The driver resource request has not yet been granted to the Spark job.
joined the process and the error.

Answers

  • asmahassani08asmahassani08 Member Posts: 29 Contributor I
    I changed the pushdown process to sparkRM operator and have problems: 

    what does it mean: The driver resource request has not yet been granted to the Spark job.
    In rapidminer's logs, I have this  SparkRM (2): Yarn application state of application_1544000400659_0013: ACCEPTED 
    in the yarn :
    ContainersMonitorImpl: Memory usage of ProcessTree container_1544000400659_0013_01_000001 for container-id container_1544000400659_0013_01_000001: 278.2 MB of 2 GB physical memory used; 455.0 MB of 4.2 GB virtual memory used
    the job is accepted and not yet running, what can be the source of the problem?

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager Posts: 1,886  Community Manager
    cc @asimon
  • toth_zsolt_bmetoth_zsolt_bme Member Posts: 1 Contributor I
    Hi,

    this can mean multiple things, my guess is that your cluster doesn't have enough free resources to start the job. You can try to play a bit with the Spark Resource Allocation Policy settings in the Radoop Connection, and the cluster resources, driver memory, executor memory settings in the operator params, e.g. try to decrease the cluster resources%.
    If you have access to the yarn CLI, you can do "yarn logs -applicationId application_1544000400659_0013" to see the full log of the application.


Sign In or Register to comment.