🦉 🎤   RapidMiner Wisdom 2020 - CALL FOR SPEAKERS   🦉 🎤

We are inviting all community members to submit proposals to speak at Wisdom 2020 in Boston.


Whether it's a cool RapidMiner trick or a use case implementation, we want to see what you have.
Form link is below and deadline for submissions is November 15. See you in Boston!

CLICK HERE TO GO TO ENTRY FORM

"Maximum theoretical accuracy"

RyujakkRyujakk Member Posts: 17  Maven
edited June 14 in Help
Hello,

I've been wondering, is there a theory on the maximum accuracy one can achieve on a given example set without over-fitting? I'm looking for something that could tell me, "whatever method you use to do regression/classification on this dataset, you'll never achieve over x% accuracy" , or something along those lines.
I tried a quick Google search, but didn't find anything along those lines, so I'm asking the experts now :)

Thanks for any leads on the subject!
Tagged:

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,527   Unicorn
    Hi,
    there is a very popular upper bound for the maximum accuracy you can reach without looking at the data: 100% :)
    Sorry, but without knowing anything about the dependencies between the label and the attributes, you simply cannot make any reasonable assumption about the maximal achievable accuracy. Since you are always looking only at a sample of the underlying statistical distribution, you even cannot say with 100% certainty, if the accuracy of your learned model, estimated by a cross validation, is as high as it will be on new, unseen data.

    Greetings,
      Sebastian
Sign In or Register to comment.