Regarding Kappa Value in Cross Validation

varunm1varunm1 Member Posts: 563   Unicorn
Hello,

Kappa values can be positive or negative depending on the type (level of agreement +ve, level of disagreement -ve). But in cross-validation how the final value of kappa is calculated? Are the negative values converted to positive and then the average kappa of all the folds are taken into consideration or are they are averaged directly?

Thanks,
Varun

Best Answer

Answers

  • varunm1varunm1 Member Posts: 563   Unicorn
    Hello @mschmitz

    Thanks for your response. I see that this kappa (Cohen) formula is combining all confusion matrices into one to get final kappa value in cross-validation. Is my understanding correct? Generally, we take the arithmetic mean of all individual folds, so I was confused how it's working here.

    Thanks,
    Varun
  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,059  RM Data Scientist
    Hi,
    nono. This is basically the formula to get one fold's Kappa. In X-val it is again aggregated.
    BR,
    Martin
    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
    varunm1
  • varunm1varunm1 Member Posts: 563   Unicorn
    @mschmitz Thanks. I got it, so the negative kappa will remain negative while aggregating. 

    Thanks,
    Varun
Sign In or Register to comment.