Parameter Otimization - cant replicate results

pilatuspilatus Member Posts: 1 Contributor I
edited November 2018 in Help
Dear Forum-Members,

I encountered the following problem, that I still can't resolve on my own:

Inside the parameter optimization I put a PCA operator followed by XValidation with Naive Bayes.
This results in an optimal number of principal components and the corresponding performance numbers.
So far, so good.
However, when I construct the same process without optimization and instead configure the PCA with
the optimal number of components taken from step 1, the performance does not match the one from step 1.
Why does this occur?



  • Options
    landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn
    this probably occurs, because the sequence of random numbers used for the splitting in the XValidation is not the same unless you activated local random seed. But if you use a local random seed, you might overfit the number of components to this particular local random seed...

    Anyway to rule out the option, that there's simply an error in the process design, please post your process here.

Sign In or Register to comment.