🎉 🎉 RAPIDMINER 9.10 IS OUT!!! 🎉🎉

Download the latest version helping analytics teams accelerate time-to-value for streaming and IIOT use cases.

CLICK HERE TO DOWNLOAD

Training and testing correlation

PapadPapad Computer Science StudentMember Posts: 68  Guru
edited March 2020 in Help
Hello,
When I test a model on known data, as a result I have an accuracy. Is this accuracy always greater than the accuracy I have when I use cross validation where I want to see how my model reacts to unknown data? 
So,  doing the steps in tutorial "Testing a model" , if I have bad accuracy, should I try to find another model or prepare different my data and not go on a cross validation or something else?
Thanks in advance.
Tagged:

Best Answer

  • yyhuangyyhuang Administrator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 355  RM Data Scientist
    edited May 2019 Solution Accepted
    Hi @Papad,

    Most of the time training accuracy is better than the testing accuracy since the model is trained on it and worst case scenario, like k-nn (very small k), the model will try to memorize all the data points in training. 

    We usually ignore the training performances when we optimize the models, cos it would not be helpful for us to understand the predictive capabilities on unseen data. Some exceptions here like detecting outliers, verifying randomness of errors when we compare the predictions and labels for the training set.

    If you had bad accuracy on the testing set, rethink about the validation process and any shifted concept between testing and training data. 

    An insightful post from Ingo can also help you understand why we need cross validation and how to interpret it  (strongly recommend also this)

    Cheers,

    YY

    varunm1sgenzerIngoRMPapad
Sign In or Register to comment.