Discussions

I've attempted to create a crawl and keep getting errors. I'm trying to scrape for psychic and intuitive guide advisors on sites like yelp.com and etsy.com. I'd appreciate your help with getting this set up.

Is it possible to throttle the request rate?

I have a long list of urls and crawl status says "STARTED" but # urls crawled stays at 0

Hi, Some time ago you changed "logic" and start request billing info even for free plan. OK but when I tried to go to "Billing" page I just see empty page - I tried in different browsers and even from phone. So I thought it is temporary bug but I still experienced such type of issue and cannot understand what are you doing there????

First time using the new interface. My crawl runs fine (EmailCollector.js) - but when I attempt to download the completed file: a - I no longer see the option to download as a csv file b - when I click the download link for json - the json displays in the browser - but no file is downloaded. Thanks

I set up a crawl of Max 10,000 URL's but only loaded a list of 1000 URL's, Depth of 1 and only internal links. Why does it show # of URL's crawled 8340 and crawl is still going?? Thanks

I am trying to crawl yellow pages for Malaysia .. it is able to crawl 1-2 pages at once. However, when I increase number of pages, it gives an error showing Error loading page in few while loading the rest. Can you help me with the reason behind such scenarios ?

The new system won't let me edit any app. The new 80legs implementatin is full of bugs and it is not useful for me. I can wait for the bugs to be fixed but meanwhile I need what I already payed for this month $29 back to my credit card Regards Alejandro

Hello, I've been trying to crawl several websites, including ones I have already succeeded before, but nothing is happening. The crawl just stays on the status "STARTED" for days. Is there a problem that should be fixed soon ?

In some crawls , the results show completed without crawling any URL or showing any JSON file. What are the reasons for such scenarios?