RapidMiner

Maven
Maven

Change the execution order of processes on RM Server

Hallo community,

 

I am having a question similar to this one from 2014: Setting-process-priorities

My RM server license is limited to running one process at a time, which is perfectly fine for my purposes. I use the server instance mostly for extensive pre-processing and data gathering tasks. Having said that, some of those processes run for a very long time, due to numerous database querries and the respective process structures (i.e. many loops). Very long means several hours or even a couple of days. I ususally submit several of those processes at a time and just let it run until it is done.

 

However, sometimes I have a situation in which I would like to run a process on the same server instance, that is more important from a timing perspective than the data processing processes. If I send it to the server, it is automatically put at the end of the queue and I was wondering whether there is any way, I can move it up to be at the first spot of the queue. My current solution is aborting all queued processes, submitting the high priority processes and then submitting the whole lot of the low priority processes back to the queue. This seems quite non optimal.

 

I have read the doc on Process Execution Queues, but it does not really do what I was hoping for. Any hint on how I can sort the process execution order on RM Server after the processes have been submitted would be highly appreciated.

 

Thank you.

3 REPLIES
Highlighted
RM Staff
RM Staff
Solution

Re: Change the execution order of processes on RM Server

Hi,

 

you cannot. Queues are by design first in, first out. Re-ordering inside a queue is not possible. You will have to continue using the workaround.

 

Regards,

Marco

_________________________________________________________
Team Lead Software Engineering | RapidMiner GmbH
Community Manager Community Manager
Community Manager
Solution

Re: Change the execution order of processes on RM Server

Hi,

 

you could implement "queueing" yourself using a database table (or whatever works best for you). 

You'd have a meta-process which is queued on the server. It reads the process names ordered by the priority and executes the first one. When it's done, the process is marked in the database. The next iteration will catch the next process and so on.

 

You take the process path from your queue, set a macro, and Execute Process using the path macro. Of course, you won't be able to pass variable parameters (macros). 

 

It's probably not trivial and you'll find some corner cases but it might work better than stopping and re-submitting processes in the long run.

 

Regards,

Balázs

--
Balázs Bárány
Data Scientist, Vienna
https://datascientist.at
Maven
Maven

Re: Change the execution order of processes on RM Server

Hi Balázs,

 

thanks for the creative approach. It does sound a bit daunting, but I will definetly try it out!

Twitter Feed