RapidMiner

Contributor II hughesfleming68
Contributor II

Polling a directory for a file?

Hi,

 

Is there a way to start a processes and then pause until a file is written to a certain directory? Would something like jnotify work from execute script?

 

Thanks,

 

Alex

5 REPLIES
RM Staff
RM Staff

Re: Polling a directory for a file?

Hi,

 

how would you write the file? And of course this is possible with Groovy. 

 

Best,

Martin 

--------------------------------------------------------------------------
Head of Data Science Services at RapidMiner
Contributor II hughesfleming68
Contributor II

Re: Polling a directory for a file?

Hi Martin,

 

The file is written from an external realtime application as a text file. It is triggered from an event if certain conditions are met. The idea is to extract a macro and use that to select a subprocesses. 

 

Thanks. I will see if I can get it to work with Groovy.

 

regards,

 

Alex

 

RM Staff
RM Staff

Re: Polling a directory for a file?

Hi Alex,

 

i think the ideal solution would be a nested operator which checks if the file is locked. If it is locks it waits for x secs, if not it runs the inner process. Right?

 

Best,

Martin

--------------------------------------------------------------------------
Head of Data Science Services at RapidMiner
Highlighted
RM Certified Expert
RM Certified Expert

Re: Polling a directory for a file?

Hi,

 

the new Jackhammer extension has a Handle Exception (Advanced) Operator, that allows retries with a delay if a try subprocess fails. If you simply put that around your File operation, you can have it repeat the inner process until it was successfully.

Of course it's polling and not really a notification, but integrating the native code might be tricky without a real extension. If you don't need hard real time, you can set the delay to a few seconds so that the overhead is minimal.
Anyway I think I will put an operator on our to do list that directly implements that functionality with this library you mentioned. Is coming up again and again.

Greetings,
Sebastian

Old World Computing - Establishing the Future

Professional consulting for your Data Science problems

Contributor II hughesfleming68
Contributor II

Re: Polling a directory for a file?

Hi Sebastian,

 

Thanks for taking the time to respond. I have seen some examples that use jnotify and others that use the watch service API (java.nio.files). I will spend some time with this over the weekend to see if I can get it working outside of Rapidminer first. Please let us know when the new Jackhammer extention is available.

 

A proper polling/notification extension would be terrific. I could imagine a lot uses for something like that.

 

Kind regards,

 

Alex

 

Polls
How can RapidMiner increase participation in our new competitions?
Twitter Feed