Is my model good enough?
I'm trying to make a predictive regression model, but having a hard time telling if my model is good enough?
I'm using X-validation and I've read somewhere that you can tell if it's a good fit based on the difference between the training error and the validation error? But how do I get the X-validation to tell me the training error?
Currently my model has a RMSE of about 1,000 and my label has a range from 0 to about 32,000. Out from this I can't really tell if it is any good? Is there another way I can measure if it's a good model?
Oh, and one more thing - I can manage to make my model better if I use a k-NN global anomaly score and remove some of the outliers comming from noise - but i'm afraid that I remove too much information. How can I decide how many outliers I can remove?
Thanks in advance!