Options
compare ROCs with different subsets of attributes
Contributor I
Hi,
is it possible to compare ROCs of models with different subsets of attributes? I'm trying to compare the ROCs fo 2 Naive Bayes models, one has the full set of attributes and the other only has a subset. I tried including "Remove Correlated Attributes" / "Select Attributes" in the "Compare ROCs" process but I keep getting errors. "Select Attributes" displays the following error:
Exception: java.lang.ArrayIndexOutOfBoundsException
Message: 1
Stack trace:
com.rapidminer.operator.learner.bayes.SimpleDistributionModel.performPrediction(SimpleDistributionModel.java:447)
com.rapidminer.operator.learner.PredictionModel.apply(PredictionModel.java:76)
com.rapidminer.operator.visualization.ROCBasedComparisonOperator.doWork(ROCBasedComparisonOperator.java:173)
com.rapidminer.operator.Operator.execute(Operator.java:831)
com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:51)
com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:709)
com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:369)
com.rapidminer.operator.Operator.execute(Operator.java:831)
com.rapidminer.Process.run(Process.java:902)
com.rapidminer.Process.run(Process.java:798)
com.rapidminer.Process.run(Process.java:793)
com.rapidminer.Process.run(Process.java:783)
com.rapidminer.gui.ProcessThread.run(ProcessThread.java:63)
Anyone know what I'm doing wrong?
Your help would be greatly appreciated!
is it possible to compare ROCs of models with different subsets of attributes? I'm trying to compare the ROCs fo 2 Naive Bayes models, one has the full set of attributes and the other only has a subset. I tried including "Remove Correlated Attributes" / "Select Attributes" in the "Compare ROCs" process but I keep getting errors. "Select Attributes" displays the following error:
Exception: java.lang.ArrayIndexOutOfBoundsException
Message: 1
Stack trace:
com.rapidminer.operator.learner.bayes.SimpleDistributionModel.performPrediction(SimpleDistributionModel.java:447)
com.rapidminer.operator.learner.PredictionModel.apply(PredictionModel.java:76)
com.rapidminer.operator.visualization.ROCBasedComparisonOperator.doWork(ROCBasedComparisonOperator.java:173)
com.rapidminer.operator.Operator.execute(Operator.java:831)
com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:51)
com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:709)
com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:369)
com.rapidminer.operator.Operator.execute(Operator.java:831)
com.rapidminer.Process.run(Process.java:902)
com.rapidminer.Process.run(Process.java:798)
com.rapidminer.Process.run(Process.java:793)
com.rapidminer.Process.run(Process.java:783)
com.rapidminer.gui.ProcessThread.run(ProcessThread.java:63)
Anyone know what I'm doing wrong?
Your help would be greatly appreciated! 0
Answers
without knowing the exact process setup I can only make a guess: is it possible that you are trying to apply a model which has been trained on the full set of attributes to the example set with less attributes? That is not supposed to work - please train the model on the same example set with the same attributes as the one you apply it to. I admit the the error message could be a bit more expressive, though.
If this does not help you, please post your process here.
Cheers,
Marius
Come see me in class for the answer.