Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Standardization as a preprocessor
Legacy User
Member Posts: 0 Newbie
In addition to normalization as a preprocessing operator, a standardization preprocessing operator would be very handy.
0
Answers
what exactly do you mean with standardization?
Regards,
Tobias
A collection of numeric data is normalized by subtracting the minimum value from all values and dividing by the range of the data. This yields data with a similarly shaped histogram but with all values between 0 and 1.
Standardize -
A collection of numeric data is standardized by subtracting a measure of central location (such as the mean or median) and by dividing by some measure of spread (such as the standard deviation, interquartile range or range). This yields data with a similarly shaped histogram with values centered around 0.
-- It would be useful to be able to specify with a dropdown the the measure of the spread whether it is standard deviation, interquartile range, 5-95% etc, does this make sense? Also the selection of what you call doing a z-transform looks to be mutally exclusive with scaling, there is not a reason for this that I can tell and thus requires two operations.
ok, I got the point. These two tasks are indeed both contained in the [tt]Normalization[/tt] operator where the normalization task is executed if the [tt]z-transform[/tt] parameter is disabled. If it is enabled, the mean is substracted from all values and the difference is divided by the standard deviation of the values. We will think about extending the operator to support other scaling strategies as well for the next release.
Regards,
Tobias
related I'm looking for a method to just subtract the mean without then dividing by the standard deviation.
This is eg. useful for NearestNeighbor learning, in case of measurement offsets: You don't want a z-Normalization there as that would effectively change influence of some parameters to the classification.
Regards,
Stefan
Ingo