Can you run only one process at the same time after upgrade to 8.1?

kaymankayman Member Posts: 662 Unicorn
edited December 2018 in Help

Hi,

 

I noticed all my processes remain pending if there is already a process running. On the older versions I was able to run processes together so I'm a bit suprised this is no longer possible so I'm wondering what I am doing wrong here.

 

 

Once the running process is finished the next one starts and so on. But since some processes can take quite some time this is really unwanted behaviour, so any advice to get rid of this again is welcome.

Tagged:

Best Answer

  • kaymankayman Member Posts: 662 Unicorn
    Solution Accepted

    Thanks @mschmitz,

     

    Modifying the JA configuration to increase the count and reduce the memory accordingly did the trick.

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist

    Hi,

     

    the number of parallel jobs is now a parameter. You can change this in the JA configs.

     

    Best,
    Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • kaymankayman Member Posts: 662 Unicorn

    Thanks Martin, Is there any documentation about where and how to modify this parameter?

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist

    @kayman,

    sure: https://docs.rapidminer.com/latest/server/administration/queue-management/

    it's a setting of queue, not a Job agent. Sorry for the confusion.

     

    Best,

    Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • kaymankayman Member Posts: 662 Unicorn

    Can I add additional Queues to the default job agent or is it really required to install one for any process you may want to run in parrallel?

     

    Assume I want to run 4 smaller tasks in parrallel, like I sometimes do if I want to download data from different sources at the same time, I would have to create 4 jobagents and ensure I keep all 4 of them always up to date each time I update an extention? That can become pretty problematic to maintain in the end.

     

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist

    Hi @kayman,

     

    nono you can run many jobs per JA. the cool thing is, that you can add another JA on another computer to scale up computing power.

    To quote from the docs:

    Note: A queue can have one or several Job Agents connected to it. Each of these connected Job Agents can execute an arbitrary number of RapidMiner processes concurrently. However, the default value for parallel execution is set to three.
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • kaymankayman Member Posts: 662 Unicorn

    I saw that also, but I'm currently using the bundled JA and I can run only one process at a time instead of the mentioned default 3. As soon as one process is running the rest stays idle until one is finished and I see no real option to add an additional queue without the need to install a new Job agent. So what am I missing?

     

    I would prefer to stick to the default JA as I have no real need yet to scale up and it keeps the maintenance under control

    Since the default is 3, I assume this number can be modified? Maybe I did accidentaly set it to 1 when I did my (headless) install and do I need to 'reset' this in the settings. If so, where is this located?

     

     

Sign In or Register to comment.