Options

# confidences and probability

I am trying to feed RapidMiner output (predictions and probabilities) into an external program. I understand how to output confidence scores using ExampleSetWriter, but not probabilities. I found a thread in the old SourceForge forum that suggests using PlattScaling to convert confidence scores to probabilities. I looked around a little bit, but could not get this operator to get what I want. How should I go about doing this? How do I find and write the probability that a particular instance belongs to a class?

Thanks,

B

Thanks,

B

0

## Answers

347MavenFor others: Here is the link to the mentioned sf-thread: click.

Regarding your problem:

I understand Platt Scaling this way:

input: the ExampleSet the original model was created with and the created modeloutput:the "corrected" model.Applying the corrected model to your test set results in real probabilities in the confidence column, i.e. confidence(c)=p(y=c|x=current Example).

hope this was helpful

Steffen

PS: This topic is really interesting. Recently I stumbled upon a paper converting scores of multiple-class-classifiers, but unfortunately, I did not have the time to study it yet. Here is the link (PDF):Transforming Classifier Scores into Accurate Multiclass Probability Estimates

3Contributor Iwhile that seems to work, I am having trouble in getting probabilities from the training set with cross-validation. Where exactly should PlattScaling operator be placed in such a scenario?

My cross validation step looks like the following:

-XValidation

--NaiveBayes

--OperatorChain

---ModelApplier

---BinomialClassificationPerformance

---ExampleWriter

---ResultWriter

-ModelWriter

-PerformanceWriter

It seems that PlattScaling should be performed after the learning step and before the ExampleWriter operator. However, by placing it before the ExampleWriter operator, ExampleSet is lost (PlattScaling takes ExampleSet and Model and returns Model). Placing it after the learning step, I somehow get an error in the PerformanceVector operator. The error is that "an object of PerformanceVector type is needed". Without PlattScaling, it works fine. I suppose if PlattScaling had the parameter "keep_example_set", this would work. I would appreciate any feedback.

Thanks

B

347MavenIf you want to create a "platt-scaled" model for each training step, I suggest something like this: (simple copy and paste it to the xml-tab of rm-interface) Hope this is what you wanted to do

greetings

Steffen

3Contributor IB