Options

"Cannot retrieve data from the specified URL" error

cbaslancbaslan Member Posts: 6 Contributor II
edited November 2018 in Help

Hi,

 

I have a process containing a collection of example sets including URLs. I will loop through every example set using loop collection operand. Inside there is a get pages operand and store operand. Total number of websites to be crawled is about 20k from 10 different domains. But when started the process it gives this error and stops. But I know that I have connection and that page causing the error exists. What could be the problem here? Sceenshot is attached. 

 

Thanks

12.png 61.7K
Tagged:

Answers

  • Options
    IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder

    Hard to say.  Could be some Javascript or something to throw it off...

     

    In situations like this, where you loop through thousands of sources and might run into trouble with a few of them, I always recommend to use the operator "Handle Exception" as part or actually around the extraction.  This way the rest of the process and the data collection can still finish even if one source throws the system off (or you are offline for 1 second or...)

     

    Hope this helps,

    Ingo

  • Options
    cbaslancbaslan Member Posts: 6 Contributor II

     Thanks a lot! I will try that and see how it goes. 

Sign In or Register to comment.