The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.
Options

[SOLVED] Accuracy performance

StaryVenaStaryVena Member Posts: 126 Contributor II
edited November 2019 in Help
Hello,
how can I create accuracy performance, if I have TP, TN, FN, FP values? I created precision performance using BinaryClassificationPerformance, but there isn't accuracy measure. There is also MultiClassificationPerformance class, but it can be created only from ExampleSet (I think). I would like to create accuracy performance from image, so I don't have ExampleSet.
Can somebody help me?

Best,
Václav

Answers

  • Options
    MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi Václav,

    the accuracy is the probability that new examples are classified correctly, so it's #correct_predictions / #total_prediction = (TP + TN) / (TP + TN + FN + FP)

    Best regards,
    Marius
  • Options
    StaryVenaStaryVena Member Posts: 126 Contributor II
    Hi Marius,
    thank you for answer. I know the formula, but I don't know, how to implement it in RapidMiner so that can be used for optimization.
    Example:
    I have two binary images - one is reference and one is generated in RM. I need reference image to  be same as reference. I have operator which have this two images on the input ports and I need accuracy performance on the output port. So I could use the Optimize Parameters operator to optimize parameters for image generation. How should the code for accuracy looks like if I have TP, TN, FN, FP values?
    I hope it's now clearer.

    Best,
    Václav
  • Options
    MariusHelfMariusHelf RapidMiner Certified Expert, Member Posts: 1,869 Unicorn
    Hi Václav,

    unfortunately you are right, to calculate the accuracy you need MultiClassificationPerformance, and that can in fact only be used with examples. So you would have to create an ExampleSet from your confusion matrix to resemble an example set which would create that matrix.
    Basically you need only 4 examples, one simulating each of  TP, TN, FN, FP with weights corresponding to the value in the confusion matrix.
    I.e. for TP = 10, FP = 20, FN = 15, TN = 5 you could create 4 examples:
    label, weight, prediction(label)
    true, 10, true
    false, 20, true
    true, 15, false
    false, 5, false

    Don't forget to assign the weight role to the weight attribute.

    Best!
    Marius
  • Options
    StaryVenaStaryVena Member Posts: 126 Contributor II
    Hi Marius,
    thank you  :)

    Best,
    Václav
Sign In or Register to comment.