Options
[SOLVED] Accuracy performance
Contributor II
Hello,
how can I create accuracy performance, if I have TP, TN, FN, FP values? I created precision performance using BinaryClassificationPerformance, but there isn't accuracy measure. There is also MultiClassificationPerformance class, but it can be created only from ExampleSet (I think). I would like to create accuracy performance from image, so I don't have ExampleSet.
Can somebody help me?
Best,
Václav
how can I create accuracy performance, if I have TP, TN, FN, FP values? I created precision performance using BinaryClassificationPerformance, but there isn't accuracy measure. There is also MultiClassificationPerformance class, but it can be created only from ExampleSet (I think). I would like to create accuracy performance from image, so I don't have ExampleSet.
Can somebody help me?
Best,
Václav
Tagged:
0
Answers
the accuracy is the probability that new examples are classified correctly, so it's #correct_predictions / #total_prediction = (TP + TN) / (TP + TN + FN + FP)
Best regards,
Marius
thank you for answer. I know the formula, but I don't know, how to implement it in RapidMiner so that can be used for optimization.
Example:
I have two binary images - one is reference and one is generated in RM. I need reference image to be same as reference. I have operator which have this two images on the input ports and I need accuracy performance on the output port. So I could use the Optimize Parameters operator to optimize parameters for image generation. How should the code for accuracy looks like if I have TP, TN, FN, FP values?
I hope it's now clearer.
Best,
Václav
unfortunately you are right, to calculate the accuracy you need MultiClassificationPerformance, and that can in fact only be used with examples. So you would have to create an ExampleSet from your confusion matrix to resemble an example set which would create that matrix.
Basically you need only 4 examples, one simulating each of TP, TN, FN, FP with weights corresponding to the value in the confusion matrix.
I.e. for TP = 10, FP = 20, FN = 15, TN = 5 you could create 4 examples:
label, weight, prediction(label)
true, 10, true
false, 20, true
true, 15, false
false, 5, false
Don't forget to assign the weight role to the weight attribute.
Best!
Marius
thank you
Best,
Václav