RapidMiner

Tokenize operator of Text processing extension stopped working after upgrading to 7.3

SOLVED
Regular Contributor

Tokenize operator of Text processing extension stopped working after upgrading to 7.3

[ Edited ]

Screen Shot 2016-12-06 at 12.30.32 PM.png

 

This is a screenshot of the results tab, when putting a breakpoint after the 'Tokenize' operator. The input if a text attribute called 'description' and the output is the same plain attribute instead of the tokenized version.

 

Rapidminer Studio version 7.3, extension is also up-to-date (7.3.0)

 

It was working correctly when I was using Studio 7.0 . Is there a way to downgrade? Didn't find older versions on sourcefourge either.

See more topics labeled with:

4 REPLIES
Community Manager

Re: Tokenize operator of Text processing extension stopped working after upgrading to 7.3

With version 7.3 there's a limit to how much data you can process (10,000 rows). What is your license limit? Could be related to that.

Regards,
Thomas
LinkedIn: Thomas Ott
Blog: Neural Market Trends
Regular Contributor

Re: Tokenize operator of Text processing extension stopped working after upgrading to 7.3

My license has unlimited data rows and in this process I only have 2000 data rows.

Highlighted
Moderator

Re: Tokenize operator of Text processing extension stopped working after upgrading to 7.3

Hi,

I have just tested this on 7.3 and I see proper results at the breakpoint. Please check if you have anything meaningful on the input and not just empty text. Also, you can enable the Log panel to look for any potential errors. In case you can reproduce this and can share a demo process and demo data then we can investigate deeper.

Thanks, Zoltan

______________________
Zoltan Prekopcsak, VP Engineering
Regular Contributor

Re: Tokenize operator of Text processing extension stopped working after upgrading to 7.3

In my process I was reading from a mysql table and the column type in mysql was 'Text', but RapidMiner was seeing it as 'nominal' attribute. Now after using the nominal to text operator, it works.

 

p.s: This was not an issue in version 7.0 as I was not using this type convertor operator.

Thanks