Due to recent updates, all users are required to create an Altair One account to login to the RapidMiner community. Click the Register button to create your account using the same email that you have previously used to login to the RapidMiner community. This will ensure that any previously created content will be synced to your Altair One account. Once you login, you will be asked to provide a username that identifies you to other Community users. Email us at Community with questions.
Crawl web operator does not return any results
Contributor I
Hi, i have a problem with Crawl web operator that it doesn't return any result i tryied latest rapidminer and then set up 7.1 but result didn't change and empty result page even I didn't try https url i tried. Could you help me please, where am i making wrong here ?
0
Best Answer
-
kayman
Member Posts: 662 
Unicorn
You didn't apply any crawling rules so basically your operator is just doing nothing even if it reads the content. If you do not state which patterns to follow and which of these to store the system just crawls clueless.
Try the get page operator first and see if you get any result then. This loads the actual url and returns the data. This way you can already validate the connection.
Next define which pages to crawl and store (patterns) with the crawl web operator, or use the get pages and provide a list of url's.
Typically that's a bit more trustworthy as webstructures can be pretty complex for 'blind' crawling, and quite some sites will just kick you of their server if you do this too obviously7
