Tokenize operator of Text processing extension stopped working after upgrading to 7.3

aliwebaliweb Member Posts: 13 Contributor II
edited June 2019 in Help

Screen Shot 2016-12-06 at 12.30.32 PM.png


This is a screenshot of the results tab, when putting a breakpoint after the 'Tokenize' operator. The input if a text attribute called 'description' and the output is the same plain attribute instead of the tokenized version.


Rapidminer Studio version 7.3, extension is also up-to-date (7.3.0)


It was working correctly when I was using Studio 7.0 . Is there a way to downgrade? Didn't find older versions on sourcefourge either.

Best Answer

  • Options
    aliwebaliweb Member Posts: 13 Contributor II
    Solution Accepted

    In my process I was reading from a mysql table and the column type in mysql was 'Text', but RapidMiner was seeing it as 'nominal' attribute. Now after using the nominal to text operator, it works.


    p.s: This was not an issue in version 7.0 as I was not using this type convertor operator.



  • Options
    Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    With version 7.3 there's a limit to how much data you can process (10,000 rows). What is your license limit? Could be related to that.

  • Options
    aliwebaliweb Member Posts: 13 Contributor II

    My license has unlimited data rows and in this process I only have 2000 data rows.

  • Options
    zprekopcsakzprekopcsak RapidMiner Certified Expert, Member Posts: 47 Guru


    I have just tested this on 7.3 and I see proper results at the breakpoint. Please check if you have anything meaningful on the input and not just empty text. Also, you can enable the Log panel to look for any potential errors. In case you can reproduce this and can share a demo process and demo data then we can investigate deeper.

    Thanks, Zoltan

Sign In or Register to comment.