Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
combine forward with backward selection?
Unicornhi,
is it somehow possible to combine forward and backward selection in an iterative manner? Like doing 2-3 times forward selection, then do 1 backward elimination, do this again several times in a loop, until the moving average performance will not improve anymore in a certain range, like "without increase of at least...%". Then this would be the stopping criterion, is there anything like that in rapidminer?
And would it be possible to compose this function with the already present operators in Rapidminer, or do I need to do this programatically?
0
Answers
I'm not sure if this is a good idea and if it's even feasible. In simplistic terms, with BE you start with all the attributes and then remove the ones that increase performance. With FS you start with one attribute and add more as the performance increases. I think a BE and FS type of setup would defeat the whole purpose of their feature selection startegies.
but could it not be, that if you alternate between e.g 5x backward elimination and then 3x forward elimination, you surpass the greedy algorithm where you can get stuck in local optima and ignore possible global optima or better combinations of features that would result otherwise?
if you remove 5 features and then do FW Selection, is it always the case that you will add the same 5 features that you removed previously? or is there a new combination with better performance possible?