[SOLVED] Classifying an input from Java Code while Loading model only once

sangihisangihi Member Posts: 5 Contributor II
edited November 2019 in Help
I'm developing an app which uses RapidMiner for classification. I train the classifier time to time (e.g. daily) but I use the classifier in very high rate (250 per sec).
For this purpose, I created two Processes using RM GUI. First one trains the classifier and saves it into a model file while the second one uses it for classification.

In the second process I load the model file which the first process creates. This made it very slow since it seems that the process loads it every time I want to classify an input.
You can see the second process in the following picture:
image

What's the more smart way of doing this? Β 

P.S. I think a solution is to create another process which only loads the created classifier only once and then give it to ApplyModel subprocess as another input. But I didn't find a solution for doing so in Java code.

Answers

  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hey, my suggestion would be to collect the data into larger batches for e.g. each second, instead of firing up a classification process for each single one. That way the impact of slow model loading won't be that hard.

    Furthermore, is there any reason why you store your model in a file system file, instead of using a RapidMiner repository? We usually suggest using Store and Retrieve instead of any of the old Read/Write operators.

    What you could also do is to programatically run your training process and grab the Model object from its output, and then pass the model as the second input to your classification process, the same way as you did with the input data.

    One last note: it's usually easier and more helpful to post the XML of your process instead of a screenshot. See the big red link in my signature for instructions.

    Best, Marius
  • Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,985 RM Engineering
    Hi,

    indeed you should only load the model once (after you created it, you don't even need a process for that) and then create a process which uses the model directly from a process input port. See here.

    Also please note that your application needs to be licensed under the AGPL.

    Edit: Someone was faster ;D

    Regards,
    Marco
  • MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Marco Boeck wrote:
    Edit: Someone was faster ;D
    Yeah, that was me, by 56 seconds :D
  • sangihisangihi Member Posts: 5 Contributor II
    Thank you so much. Using your help, I figured it out :)

    Just out of curiosity:
    indeed you should only load the model once (after you created it, you don't even need a process for that)
    how can I do this without creating a process?
  • Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,985 RM Engineering
    Hi,

    did you check the link I posted? ;)

    RepositoryLocation loc = new RepositoryLocation("//LocalRepository/folder/as/needed/yourData"));
    IOObjectEntry entry = (IOObjectEntry) loc.locateEntry();
    myIOObject = entry.retrieveData(null);
    Regards,
    Marco
Sign In or Register to comment.