Prospective evaluation of model prediction

Casper72

Fairly new to datamining, so this maybe a really dumb question, but I have built a model in RM based on retrospective data that appears to perform very well on both validation and test/holdout data-sets. The model is supposed to predict what will likely be the outcome on some binominal variable one year from now. So I am wondering, when I do know the actual outcome one year from now, how should I ideally evaluate my model performance? I'm thinking there must be some probabilistic uncertainty to be accounted for? Basicly I am trying to compare expected values/probabilities with real values.

varunm1

Hello @Casper72

Are you looking for Chi squared test? This is generally used to find a significant relationship between two same attributes(one with expected values and one with observed values).

Please inform if you need more information.

Casper72

Thank you @varunm1

Brilliant. Should have thought of that.