Problem with text preprocessing

ghamgham Member Posts: 24
edited December 2018 in Help

hi

How do I use the stop word dictionary to abbreviate words and replace them with the original words and delete negative words like not?
Where do I download the stopword dictinary?
Thankful

Answers

  • ghamgham Member Posts: 24

    help me...:smileysad:

  • kypexinkypexin Moderator, RapidMiner Certified Analyst, Member Posts: 280   Unicorn

    Hi @gham

     

    I suggest you to look at 'Text processing' extension for RapidMiner which contains lots of very useful operators for working with texts; among them, there's a group of operators named 'Filter stopwords' which are doing the exact thing you need, for different languages. 

     

    As for downloading the dictionary, I guess there's no 'unified' dictionary as every text processing task would basically need different stoword list, so this part need to be completed manually. Maybe you can just google and download any ready dictionaries / lists someone has built before and shared online.

    yyhuang
  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,226   Unicorn

    If you are using one of the supported languages, there are operators with built-in dictionaries (e.g., English, German, etc.) , so you don't need to download them.  If you are using another language, or if you want a custom stopword dictionary, you will need to create it yourself, or find one on the web and download it.  That's not functionality which is handled inside RapidMiner, but a little bit of web searching will turn up many useful references.

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
    sgenzer
  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,440  Community Manager

    hello @gham Some quick recommendations for you:
    β€’ Post your XML process here in this thread (see https://youtu.be/KkgB5QXWXJ8 and "Read Before Posting" on right when you reply)
    β€’ Attach your dataset if possible (use a fictionalized version if there are privacy concerns)
    β€’ Make sure you have all necessary extensions installed (see https://youtu.be/pjBqG3xtXx4)

    Scott

  • ghamgham Member Posts: 24

    Please help my friend help me out.

     

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761   Unicorn

    @gham did you post your process like what @sgenzer said? You have to do some work here, help us troubleshoot were you're getting stuck. We're not going to buildyour process for you. 

    sgenzer
  • ghamgham Member Posts: 24

    Hello
    How do I remove the negation?

    help me please

    12.JPG

  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,440  Community Manager

    I'm sorry - I have no idea what "remove the negation" means.


    Scott

     

Sign In or Register to comment.