turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Community Home
- :
- Product Help
- :
- Studio Forum
- :
- confidences and probability

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic to the Top
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

07-26-2008 10:50 PM

07-26-2008 10:50 PM

I am trying to feed RapidMiner output (predictions and probabilities) into an external program. I understand how to output confidence scores using ExampleSetWriter, but not probabilities. I found a thread in the old SourceForge forum that suggests using PlattScaling to convert confidence scores to probabilities. I looked around a little bit, but could not get this operator to get what I want. How should I go about doing this? How do I find and write the probability that a particular instance belongs to a class?

Thanks,

B

Thanks,

B

4 REPLIES

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

07-28-2008 04:25 AM

07-28-2008 04:25 AM

Hello babaluma

For others: Here is the link to the mentioned sf-thread: click.

Regarding your problem:

I understand Platt Scaling this way:

**input**: the ExampleSet the original model was created with and the created model

**output: ** the "corrected" model.

Applying the corrected model to your test set results in real probabilities in the confidence column, i.e. confidence(c)=p(y=c|x=current Example).

hope this was helpful

Steffen

PS: This topic is really interesting. Recently I stumbled upon a paper converting scores of multiple-class-classifiers, but unfortunately, I did not have the time to study it yet. Here is the link (PDF): Transforming Classifier Scores into Accurate Multiclass Probability Estimates

For others: Here is the link to the mentioned sf-thread: click.

Regarding your problem:

I understand Platt Scaling this way:

Applying the corrected model to your test set results in real probabilities in the confidence column, i.e. confidence(c)=p(y=c|x=current Example).

hope this was helpful

Steffen

PS: This topic is really interesting. Recently I stumbled upon a paper converting scores of multiple-class-classifiers, but unfortunately, I did not have the time to study it yet. Here is the link (PDF): Transforming Classifier Scores into Accurate Multiclass Probability Estimates

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

07-28-2008 03:53 PM

07-28-2008 03:53 PM

thanks for the reply.

while that seems to work, I am having trouble in getting probabilities from the training set with cross-validation. Where exactly should PlattScaling operator be placed in such a scenario?

My cross validation step looks like the following:

-XValidation

--NaiveBayes

--OperatorChain

---ModelApplier

---BinomialClassificationPerformance

---ExampleWriter

---ResultWriter

-ModelWriter

-PerformanceWriter

It seems that PlattScaling should be performed after the learning step and before the ExampleWriter operator. However, by placing it before the ExampleWriter operator, ExampleSet is lost (PlattScaling takes ExampleSet and Model and returns Model). Placing it after the learning step, I somehow get an error in the PerformanceVector operator. The error is that "an object of PerformanceVector type is needed". Without PlattScaling, it works fine. I suppose if PlattScaling had the parameter "keep_example_set", this would work. I would appreciate any feedback.

Thanks

B

while that seems to work, I am having trouble in getting probabilities from the training set with cross-validation. Where exactly should PlattScaling operator be placed in such a scenario?

My cross validation step looks like the following:

-XValidation

--NaiveBayes

--OperatorChain

---ModelApplier

---BinomialClassificationPerformance

---ExampleWriter

---ResultWriter

-ModelWriter

-PerformanceWriter

It seems that PlattScaling should be performed after the learning step and before the ExampleWriter operator. However, by placing it before the ExampleWriter operator, ExampleSet is lost (PlattScaling takes ExampleSet and Model and returns Model). Placing it after the learning step, I somehow get an error in the PerformanceVector operator. The error is that "an object of PerformanceVector type is needed". Without PlattScaling, it works fine. I suppose if PlattScaling had the parameter "keep_example_set", this would work. I would appreciate any feedback.

Thanks

B

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

07-28-2008 04:26 PM

07-28-2008 04:26 PM

Hello again

If you want to create a "platt-scaled" model for each training step, I suggest something like this: (simple copy and paste it to the xml-tab of rm-interface)

Hope this is what you wanted to do

greetings

Steffen

If you want to create a "platt-scaled" model for each training step, I suggest something like this: (simple copy and paste it to the xml-tab of rm-interface)

<operator name="Root" class="Process" expanded="yes">

<operator name="XValidation" class="XValidation" expanded="yes">

<operator name="create_scaled_model" class="OperatorChain" expanded="yes">

<operator name="NaiveBayes" class="NaiveBayes">

<parameter key="keep_example_set" value="true"/>

</operator>

<operator name="PlattScaling" class="PlattScaling">

</operator>

</operator>

<operator name="OperatorChain" class="OperatorChain" expanded="yes">

<operator name="ModelApplier" class="ModelApplier">

<list key="application_parameters">

</list>

</operator>

<operator name="BinominalClassificationPerformance" class="BinominalClassificationPerformance">

</operator>

</operator>

</operator>

</operator>

Hope this is what you wanted to do

greetings

Steffen

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

07-29-2008 11:23 PM