Discussions
Not crawling every page
over 3 years ago by James
So I have found that when crawling some URL's it doesn't crawl all the pages?
For example anderstore.com only returns 2 pages but there are many more.
The crawl level is at 5.
Also this URL www.sedgemoorfire.co.uk doesn't return any image links when on the default image crawler.
Any ideas on how to fix this?