Gradient Boosted Trees: stopped before building all trees

phivuphivu Member Posts: 34 Guru
edited December 2018 in Help

Hi all,

 

I'm using Gradient Boosted Trees to build a binary classifier with ~500 features. I want to increase the number of trees during training to get better feature weights, however, if i set the number of trees > 1000, the training stops before building all trees with no reason (as in the log panel - sceenshot attached - it stopped at 43%). This is not the memory problem as i noticed the memory usage is only 27%. I didn't tick the "early stopping" checkbox.

 

Do you know the true reason behind this, or is there another way to increase the reliability of feature weights? I attached the process here, let me know if you want the data.

Thank you very much for your help!

 

GBT-stopped-in-the-middle.png

Best,
phivu

Best Answer

  • MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,503 RM Data Scientist
    Solution Accepted

    Dear Phivu,

     

    this looks a bit odd. Can you check your rapidminer-studio.log for more information?

     

    On the DS topic: More tries does not increase the reliablity of your weights. GBTs are not bagged, so it's not like RF's. You might simply overtrain more. Than can be the reason why your tree stops. The classes are already seperated.

     

    ~Martin

    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany

Answers

  • phivuphivu Member Posts: 34 Guru

    You're correct, thanks a lot!

Sign In or Register to comment.