I'm in the process of assessing Rapid Miner for risk modelling - could someone tell me if the Gini statistic or Komogorff-Smirnov are available for assessing logistic regression performance?
Hi Dorothy, unfortunately rapid miner does not provide this two performance measures, since they do not measure the classification performance directly, rather measuring the correctness of the learner's assumption. Which is to assume that the data is normal distributed in case of the logistic regression, if I remember correctly. But even if an assumption is violated, the performance might be good. Otherwise algorithms like NaiveBayes (independence assumption of attributes) would not be that successfull in practice.
In the data mining community the most widely used method for performance estimation is crossvalidation. It measures the performance directly on the data and hence copes with cases, where data does not fit into any of the (simple) distribution assuptions or learner just do not work on continuous distribution as the powerfull svms.
RapidMiner is suitable for risk modeling in the sense of the algorithms available, and in some ways better than some other tools in which you may be limited to logistic regression only. If you run the model and export the scores you will be able to calculate the Gini from those scores using another tool. R might be able to do this, but I believe it is even possible to do gini using excel.
Contributor I
Answers
unfortunately rapid miner does not provide this two performance measures, since they do not measure the classification performance directly, rather measuring the correctness of the learner's assumption. Which is to assume that the data is normal distributed in case of the logistic regression, if I remember correctly. But even if an assumption is violated, the performance might be good. Otherwise algorithms like NaiveBayes (independence assumption of attributes) would not be that successfull in practice.
In the data mining community the most widely used method for performance estimation is crossvalidation. It measures the performance directly on the data and hence copes with cases, where data does not fit into any of the (simple) distribution assuptions or learner just do not work on continuous distribution as the powerfull svms.
Greetings,
Sebastian