"Text similarity"

TwohTwoh Member Posts: 2 Contributor I
edited May 23 in Help
Hello,
In my project i have to analyze similarity between text files and a file which contains list of words.

Does anyone know how to model this? I’ve tried it with 'Cross Distance Operator' but it doesn't work somehow.

Thanks

Answers

  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,529   Unicorn
    Hi, you can do the following:
    Read the word list using a Process Documents Operator with an inner Tokenize operator. Then forward the resulting word list to another Process Documents Operator with an inner Tokenize operator that processes all the text files. You can finally connect both outcoming examplesets with the cross distance operator and select cosine similarity, since it is most appropriate for this.

    Good Luck!

    Greetings,
    Sebastian
  • TwohTwoh Member Posts: 2 Contributor I
    Many thanks, it works. I now have only 0-values in the ‘distance’-row, even if I process two identical files. Do you have any idea why?
  • landland RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,529   Unicorn
    Hi,
    if you compare two identical vectors, a distance of zero is something that I would expect...

    Greetings,
    Sebastian
Sign In or Register to comment.