Significance testing

lsevellsevel Member Posts: 18 Contributor II
edited November 2018 in Help
Hi all,

I'm interested in comparing the performance of an algorithm to chance with a significance test. I understand there's a t-test operator that may allow me to do this but I'm not sure how exactly. Could anyone offer some advice?

Thanks!

Answers

  • earmijoearmijo Member Posts: 270 Unicorn
    There is an operator T-test. There is an example process that illustrates its use. Click on the operator (Evaluation/Significante) and in the help Tab you'll get a description of the operator and a link to the illustrative process.
  • lsevellsevel Member Posts: 18 Contributor II
    I'm aware of this operator but it assumes a paired t-test and requires two input vectors. Is it necessary to build a 'chance' classifier or is there another way to compare one model to chance?
  • rachel_lomaskyrachel_lomasky Member Posts: 52 Guru

    I also have this question.  I'd like to do a proportion test on rows of a data grid, not model performance vectors.  Is this possible?

     

    Thanks,

    Rachel

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Hello - not sure what you mean by a “proportion test”. Maybe you’re looking for a correlation matrix?

    Scott

  • rachel_lomaskyrachel_lomasky Member Posts: 52 Guru

    What is prop.test in R.  Which I just ended up using as a RapidMiner operator.

  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,959 Community Manager
    Ah I see. Never heard of prop.test but it looks like basically chi-squared independence / goodness of fit to me. @land does your stats extension do this? Anything in the toolbox, @mschmitz?

    Scott

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn

    Hi,

    we have Chi-Square test but not no direct prop.test equivalent. But seems it is solved by integrating R?

    Greetings,

     Sebastian

  • rachel_lomaskyrachel_lomasky Member Posts: 52 Guru

    Yeah, I'm happy with just using the R function.

Sign In or Register to comment.