Options

"Java integration - web mining: Get Page"

wendywendy Member Posts: 7 Contributor II
edited May 2019 in Help
Dear All,

I'm interested in using RM's capability in my own Java program. I want to use the 'Web Mining' extension available in RM in order to retrieve web pages and do further processing afterwards.

Through my program I can initialize RM already. Then I created the 'Get Page' operator and set up all the required parameters. OK, so now my problem is, how can I retrieve the result of 'Get Page' operator? In RM5 I can see that the output of 'Get Page' is a Document. But unfortunately, I cannot find Document even in the javadoc. Below is the my code so far.:

        RapidMiner.init();

       try
       {
           // Load dataset
           Operator op = OperatorService.createOperator("web:extract_html_text_content");
           op.setEnabled(true);
           op.setExpanded(true);
           op.setParameter("random_user_agent", "true");
           op.setParameter("url", "http://news.google.com");

           IOContainer container = op.apply( new IOContainer() );
       }
Can anyone help me on how to obtain the result from 'Get Page' operator?

Thank you!
Tagged:

Answers

  • Options
    Marco_BoeckMarco_Boeck Administrator, Moderator, Employee, Member, University Professor Posts: 1,996 RM Engineering
    Hi,

    two things to note:

    1) To be allowed to use RapidMiner for your project, it needs to be published under the same license as RapidMiner, which is the AGPL.
    2) Document is class from the Text plugin, so you will need to add that as well.

    And we have a Development forum for these questions, so I moved your thread ;)

    Regards,
    Marco
Sign In or Register to comment.