Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

AdaBoost vs. BayesianBoosting

cherokeecherokee Member Posts: 82 Maven
edited June 2019 in Help
Hi!

I'm goint o try some boosting for my bachelor's thesis. I haven't yet decided whether using AdaBoost or BayesianBoosting. Actually I don't understand all the differences. What I do understand is that BayesianBoosing can use different fractions of the example set for model fitting and performance estimation. I understand that it is able to reweight examples to ensure equally distributed labels. But what exactly means [tt]allow_marginal_skews[/tt]?

Martin Scholz (the author of the operator) cites in the help text Scholz/2005b. Can anyone give me publication details on (t)his work. I think I would understand the differencen in detail if i read it.

Best regards,
chero
Tagged:

Answers

  • wesselwessel Member Posts: 537 Maven
    Hmm, let me first see if I understand the terminology correctly.

    Bayesian Boosting is like creating all possible models and weighting them according their accuracy?
    (Maybe you also need to weigh them according the model prior).
    Most of the time this is not feasible in practice.

    Adaboost, adaptive boosting, setting the iteration parameter to n, creates n models.
    Models are weighted by their accuracy.
    New training data is created by reweighing examples.
    If examples are correctly classified in the previous iteration their weight goes down,
    if examples are incorrectly classified their weight goes up.
    Freund and Shaffire prove that the error on the training set goes down exponentially fast, using adaboost.

    This paper is really good?
    vorlon.case.edu/~sray/eecs600_fall08/ensembles_survey.ps
    Ensemble Methods in Machine Learning Dietterich filetype:pdf
  • IngoRMIngoRM Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Hi,

    you can find Martin's publication here:

    http://www-ai.cs.uni-dortmund.de/auto?self=$Publication_e9zx9gcx

    Here are his other publications on this and related topics:

    http://www-ai.cs.uni-dortmund.de/PERSONAL/scholz.html

    Cheers,
    Ingo
  • wesselwessel Member Posts: 537 Maven
    This paper is about subgroup detection.

    Not about ensembles or boosting.
  • IngoRMIngoRM Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Yes, but the basic algorithm - and hence also the operator - can be used for both problem types. It's the paper cited by Martin in his implementation comment so I assume it is the one he thinks it is most appropriate even if the title suggests something else. If you find another one of his papers more appropriate: just go ahead and cite it here. They all can be found on the web site stated above.

    Cheers,
    Ingo
  • Stefan_EStefan_E Member Posts: 53 Maven
    Ingo,

    Thanks a lot for the links - was searching the same just over the weekend :-)

    would propose to add these references to the wiki. There is a general lack of documentation where the various algorithms come from... so the wiki would be the natural place growing such a knowledge library.

    Problem is that it appears that the wiki wants to be a duplication of the built-in help. If this is so, you'll end up in a maintenance problem as I expect there is only a one-way conversion path?

    Stefan
  • IngoRMIngoRM Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Yes, you are right. There was actually a discussion about connecting the operator help and the Wiki some time ago:

    http://rapid-i.com/rapidforum/index.php/topic,2013.msg8148.html#msg8148

    But I must admit that I am not sure about the current state of this.

    Cheers,
    Ingo
Sign In or Register to comment.