Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.

Significance testing

lsevellsevel Member Posts: 18 Contributor II
edited November 2018 in Help
Hi all,

I'm interested in comparing the performance of an algorithm to chance with a significance test. I understand there's a t-test operator that may allow me to do this but I'm not sure how exactly. Could anyone offer some advice?

Thanks!

Answers

  • earmijoearmijo Member Posts: 271 Unicorn
    There is an operator T-test. There is an example process that illustrates its use. Click on the operator (Evaluation/Significante) and in the help Tab you'll get a description of the operator and a link to the illustrative process.
  • lsevellsevel Member Posts: 18 Contributor II
    I'm aware of this operator but it assumes a paired t-test and requires two input vectors. Is it necessary to build a 'chance' classifier or is there another way to compare one model to chance?
  • rachel_lomaskyrachel_lomasky Member Posts: 52 Maven

    I also have this question.  I'd like to do a proportion test on rows of a data grid, not model performance vectors.  Is this possible?

     

    Thanks,

    Rachel

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Hello - not sure what you mean by a “proportion test”. Maybe you’re looking for a correlation matrix?

    Scott

  • rachel_lomaskyrachel_lomasky Member Posts: 52 Maven

    What is prop.test in R.  Which I just ended up using as a RapidMiner operator.

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Ah I see. Never heard of prop.test but it looks like basically chi-squared independence / goodness of fit to me. @land does your stats extension do this? Anything in the toolbox, @mschmitz?

    Scott

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn

    Hi,

    we have Chi-Square test but not no direct prop.test equivalent. But seems it is solved by integrating R?

    Greetings,

     Sebastian

  • rachel_lomaskyrachel_lomasky Member Posts: 52 Maven

    Yeah, I'm happy with just using the R function.

Sign In or Register to comment.