Use the Compare Models operator for model lifecycle management and model building

KostasBonikosKostasBonikos Member Posts: 25 Maven
edited November 2018 in Knowledge Base

Either during the prototyping phase of a project or at the later, model lifecycle management stage, there is a need to build models on the same data that employ different algorithms in order to compare the performance of those algorithms agains the data in it's current state and volume.


In the prototyping stage, we need to establish which is the best algorithm for our project.


When managing the process lifecylce, we need to deal with effects of concept drift (, i.e. the variability in importance of certain variables over time and the variability of values which may lead to a model built in the past not performing as well in the future. This may be helped by retraining the algorithm or by finding a different one that performs better with the data as it has evolved.


To achieve the above, we can use the "Compare Models" operator to build models by different algortithms, compare their performance and keep a record of the performance and parameters


model management.png

The best way to learn about this capability is to open the tutorial process which is bundled with the operator. Just drop the Compare Models operator onto the canvas, switch to the Help panel and navigate to the tutorial process. You should see the same process as above.


The data used comes from one of the bundled data sets and has "Play" set as the target variable:

model management 1.png


Then a Multiply operator is used and three different predictive models are built, in this case, using default parameters but those can be optimised if necessary.

The models produced are collected by the Collection operator which passes them to the "Compare Models" operation for Comparison.


model management 2.png


The "Compare Models" operator requires two parameters, a location to save the results and a date format and now we can get to the most interesting part, the output:


model management 3.png


We can obtain models and their performances, neatly arranged under folders that bear the date and time in the format we specified.


There is also a performance record as a data source which can then be imported into a process to be used as input for further actions such as those from Process Control operators undel Utilities.


model management 4.png



  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Hi Kostas,

    Where can I find the Compare Models operator? Thanks!

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    I believe it is a separate extension "Model Management" available for free in the marketplace.

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
  • inspireinspire Member Posts: 1 Newbie
    Unfortunately, this (at least the tutorial program) seems to be broken in the most recent version of rapidminer, it gives an error
    • Message: class com.rapidminer.adaption.belt.IOTable cannot be cast to class com.rapidminer.example.ExampleSet (com.rapidminer.adaption.belt.IOTable and com.rapidminer.example.ExampleSet are in unnamed module of loader 'app')

Sign In or Register to comment.