Ask a Question

I'm trying to create a crawl and keep getting an error.

I've attempted to create a crawl and keep getting errors. I'm trying to scrape for psychic and intuitive guide advisors on sites like yelp.com and etsy.com. I'd appreciate your help with getting this set up.

Amazon scraping

Is it possible to crawl Amazon and get buy box prices and other info using a list of ASINs? If possible, how?

Throttling requests?

Is it possible to throttle the request rate?

crawl not starting

I have a long list of urls and crawl status says "STARTED" but # urls crawled stays at 0

Not crawling every page

So I have found that when crawling some URL's it doesn't crawl all the pages? For example anderstore.com only returns 2 pages but there are many more. The crawl level is at 5. Also this URL www.sedgemoorfire.co.uk doesn't return any image links when on the default image crawler. Any ideas on how to fix this?

why am i getting error?

my dashboard says There has been an error No payment information on file. Visit the 80legs portal to reactive your account. and i cant add any urls to url list or execute any jobs.

Crawls Immediately Being Complete - No Results

Hi Ben, I'm also seeing the issue mentioned in the other thread about crawls not working - start a crawl and it's immediately marked as complete. Tried to rerun but same thing. Thanks, Justin

Where do I go to download my results?

The web portal shows my run as "COMPLETED", but I cannot seem to find how to download the results. The /crawls page shows only two actions: "restart crawl" and "cancel crawl". When I click on the name of the crawl, it takes me to a page that shows the stats, but I do not see any download links here either. Please advise.

Won't Add App to Queue

After creating an app, upon pressing the 'Add to Queue' button, I get the error: 'Whoops! Whoops, this is an issue on our end.' I am using the free service and have the following options: Crawl Name educationTrends Depth Level 3 Crawl Size 10000 Crawl only internal links Page Content Text Content, Images, and PDFs selected Meta Information Page Title Thanks Joe

Connect Telegram and Web Page (user and password)

Hi guys!!!! I´m new in this platform. I will like to hear from you and get some advice on the next 2 tasks: Is it possible to get info in real time from this web page https://www.im.center/index.php that requires a user and password? When I say real time I mean that when a user create a post be able to see it and work with it In this link an example of the info I want to get: https://inflexioncomco-my.sharepoint.com/:i:/g/personal/datos_inflexion_com_co/EYY5Z_zhnbdBqWs6pUh4OvkBq8-gREzyMR8H0iojHGkD3Q?e=3wX4ps Is it possible to get real time messages from telegram - https://www.telegram.org/ by using their bot - https://core.telegram.org/bots Finally is it possible to send all this data to elastic search or to create a csv file with all this data in specific format? If its possible can you give me a hand with links or tips on how to configure 80legs to get it done? Finally on the image attach the login web and the info that I will like to get out of it. Best regards Alejandro Holguin M [email protected]