Regarding Kappa Value in Cross Validation

varunm1varunm1 Moderator, Member Posts: 1,207 Unicorn
Hello,

Kappa values can be positive or negative depending on the type (level of agreement +ve, level of disagreement -ve). But in cross-validation how the final value of kappa is calculated? Are the negative values converted to positive and then the average kappa of all the folds are taken into consideration or are they are averaged directly?

Thanks,
Varun
Regards,
Varun
https://www.varunmandalapu.com/

Be Safe. Follow precautions and Maintain Social Distancing

Best Answer

Answers

  • varunm1varunm1 Moderator, Member Posts: 1,207 Unicorn
    Hello @mschmitz

    Thanks for your response. I see that this kappa (Cohen) formula is combining all confusion matrices into one to get final kappa value in cross-validation. Is my understanding correct? Generally, we take the arithmetic mean of all individual folds, so I was confused how it's working here.

    Thanks,
    Varun
    Regards,
    Varun
    https://www.varunmandalapu.com/

    Be Safe. Follow precautions and Maintain Social Distancing

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist
    Hi,
    nono. This is basically the formula to get one fold's Kappa. In X-val it is again aggregated.
    BR,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • varunm1varunm1 Moderator, Member Posts: 1,207 Unicorn
    @mschmitz Thanks. I got it, so the negative kappa will remain negative while aggregating. 

    Thanks,
    Varun
    Regards,
    Varun
    https://www.varunmandalapu.com/

    Be Safe. Follow precautions and Maintain Social Distancing

Sign In or Register to comment.