Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
reduce No of dimensions
Contributor II
Dear all,
here's what keeps me busy as a newbie to DataMining: my example set contains ~70 k examples of process parameters from 5 machines and a quality label (0/1). each machine parameter has 3-5 variations (3-5 attributes per parameter and machine). Most - but not all - variations referring to the same machine parameter correlate strongly and I was thinking to eliminate "all but one" in such a case, in order to do logistic regression.
When playing arround with weights from correlation matrix, weight by correlation, select by weight, etc. I was only able to eliminate all attributes, which correlate with others, but I want to keep one.
My target is obviously, to identify those attributes, which are influencing the label most.
Maybe I am methodologically totally wrong here, so I appreciate any help.
Thx
ui3o
here's what keeps me busy as a newbie to DataMining: my example set contains ~70 k examples of process parameters from 5 machines and a quality label (0/1). each machine parameter has 3-5 variations (3-5 attributes per parameter and machine). Most - but not all - variations referring to the same machine parameter correlate strongly and I was thinking to eliminate "all but one" in such a case, in order to do logistic regression.
When playing arround with weights from correlation matrix, weight by correlation, select by weight, etc. I was only able to eliminate all attributes, which correlate with others, but I want to keep one.
My target is obviously, to identify those attributes, which are influencing the label most.
Maybe I am methodologically totally wrong here, so I appreciate any help.
Thx
ui3o
0
Answers
for your purpose the operator "Remove Correlated Attributes" could do the job. The possible pitfall can be memory consumption - few months ago I have used the operator "Select by Random" for number of attributes reduction.
I already tested the "Remove Correlated Attributes", but it will remove all and not "all but one". Nice tip with the "Select by Random". You iterate the set of selected attributes? Can you post a sample xml?
Thx for your input.
Greetings,
ui3o