How does Feature Selection - Forward Elimination work in detail?

Muhammed_Fatih_Muhammed_Fatih_ Member Posts: 93 Maven
Hello together, 

I have a question regarding regarding the picking process of Forward Elimination. The documentation of RapidMiner tells us that: "The Forward Selection operator starts with an empty selection of attributes and, in each round, it adds each unused attribute of the given ExampleSet. For each added attribute, the performance is estimated using the inner operators, e.g. a cross-validation. Only the attribute giving the highest increase of performance is added to the selection. Then a new round is started with the modified selection. [...]" 

What does this mean in detail? Assumed I have dataset with 10 attributes from a1,a2, ... to a10. Does Forward Selection sequentially go from a1 to a2 and then to a3 until no increase in performace is reached or what is meant with " Only the attribute giving the highest increase of performance is added to the selection."?

Thank you in advance for your responses! 

Best regards, 

Fatih

Best Answer

Answers

  • Muhammed_Fatih_Muhammed_Fatih_ Member Posts: 93 Maven
    Hi Balazs, 

    thank you for your answer. This helped me! An additional question with regard to the ForwadElimination subprocess. Which classification model would you use within crossvalidation in order to enhance the running time? I am currently conducting a Forwald Elimination based on matrix with 72.000 rows and 9000 attributes. I've chosen SVM as classifier for my bipartite labeling. The model is still running since 2 days. 

    Thank you for your answer! 

    Fatih 
  • BalazsBaranyBalazsBarany Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert Posts: 955 Unicorn
    Hi,

    Naive Bayes is the fastest algorithm, especially with the huge number of attributes.

    However, it is not always correct to use one algorithm for the feature selection and then another final modeling. This approach can work well for you, or fail badly. Still a good idea to try. 

    Regards,

    Balázs
Sign In or Register to comment.