tfidf and entropy
Contributor II
hi i would like to do a feature selection using tfidf and entropy .my question are as below
1. i would like to know do i need to use the generate tfidf or is the tfidf in word vector is enough?
2. for entropy do i use weight by information gain of discretize by entropy
1. i would like to know do i need to use the generate tfidf or is the tfidf in word vector is enough?
2. for entropy do i use weight by information gain of discretize by entropy
1
Best Answer
-
jmergler
Administrator, Moderator, Employee, RapidMiner Certified Analyst, Member, University Professor Posts: 41 
Guru
Hi @rafeena,
I'm not entirely certain I understand the question, but I think I can help.
1. There are many valid ways of getting the TFIDF scores. You do not need to use any particular operator like Generate TFIDF. I think what you are looking for is a data structure with words as attributes and TFIDF scores as values. If you have that, then it does not matter how you got it. If you do not have that, then please share a sample of what you do have so we can better help.
2. Your goal is feature selection, so the Weight by Information Gain operator would work. This will score the attributes. After getting the weights you may consider using the Select by Weights operator.
Please consider sharing your process with sample data, so we can provide more precise help if you need it.
Jeff
2
Answers