Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

Polling a directory for a file?

hughesfleming68hughesfleming68 Member Posts: 323 Unicorn
edited December 2018 in Help

Hi,

 

Is there a way to start a processes and then pause until a file is written to a certain directory? Would something like jnotify work from execute script?

 

Thanks,

 

Alex

Answers

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,527 RM Data Scientist

    Hi,

     

    how would you write the file? And of course this is possible with Groovy. 

     

    Best,

    Martin 

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • hughesfleming68hughesfleming68 Member Posts: 323 Unicorn

    Hi Martin,

     

    The file is written from an external realtime application as a text file. It is triggered from an event if certain conditions are met. The idea is to extract a macro and use that to select a subprocesses. 

     

    Thanks. I will see if I can get it to work with Groovy.

     

    regards,

     

    Alex

     

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,527 RM Data Scientist

    Hi Alex,

     

    i think the ideal solution would be a nested operator which checks if the file is locked. If it is locks it waits for x secs, if not it runs the inner process. Right?

     

    Best,

    Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn

    Hi,

     

    the new Jackhammer extension has a Handle Exception (Advanced) Operator, that allows retries with a delay if a try subprocess fails. If you simply put that around your File operation, you can have it repeat the inner process until it was successfully.

    Of course it's polling and not really a notification, but integrating the native code might be tricky without a real extension. If you don't need hard real time, you can set the delay to a few seconds so that the overhead is minimal.
    Anyway I think I will put an operator on our to do list that directly implements that functionality with this library you mentioned. Is coming up again and again.

    Greetings,
    Sebastian

  • hughesfleming68hughesfleming68 Member Posts: 323 Unicorn

    Hi Sebastian,

     

    Thanks for taking the time to respond. I have seen some examples that use jnotify and others that use the watch service API (java.nio.files). I will spend some time with this over the weekend to see if I can get it working outside of Rapidminer first. Please let us know when the new Jackhammer extention is available.

     

    A proper polling/notification extension would be terrific. I could imagine a lot uses for something like that.

     

    Kind regards,

     

    Alex

     

Sign In or Register to comment.