The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.
Options

How to download dynamic information

gortipavangortipavan Member Posts: 1 Contributor I
edited November 2018 in Help
Hi

I have a problem at hand and am wondering if RapidMiner can be used to solve it.

I need to get some information from a website. It is a website that reports scientific information for different lakes. I would like to fetch all the data corresponding to a lake attribute (example, Lake depth). This data is loaded on the website dynamically and is not part of the initial markup (The database is read to dynamically populate certain fields). Is tehre a way I can mine this information using Rapid miner? If so, can someone point me in the right direction?

Thanks

Answers

  • Options
    el_chiefel_chief Member Posts: 63 Contributor II
    scraping websites that use javascript can be difficult.

    i would recommend looking at Selenium, which lets you use say Chrome browser to fetch a web page and create the markup for you, and return that as text.

    http://seleniumhq.org/

    you could also try the watij library which is a java library for scraping sites with dynamic content

    http://watij.com/

    regards,

    neil
Sign In or Register to comment.