IIS Search Engine Optimization Toolkit
Seo Toolkit doesn't follow my hyperlinks ...
Last post Jul 11, 2019 01:31 PM by SAMFi
Jan 25, 2010 07:11 AM|Joeri P|LINK
When I want to crawl my site with the SEO toolkit, the toolkit doesn't find any link in my website. The result is that he only crawl 1 page ... .
The status code of this 1 crawled page is 200.
Anyone knows what the problem is ? If somebody needs the url for viewing the problem, please give me a sign.
Jan 25, 2010 05:49 PM|CarlosAg|LINK
If you send me the link at microsoft . com ( same user as this post) I'll take a look
Jan 25, 2010 11:51 PM|CarlosAg|LINK
The reason this is happening is because we have a bug in the SEO Toolkit that prevents us from identifying XHTML content types as HTML documents, which means we don't even parse it.
You can see this by looking at the Content Type returned in your page which is: application/xhtml+xml
The good news is that we have fixed this bug in a version that we will be releasing REALLY soon. I will post to this thread when it is available online.
Feb 01, 2010 03:00 AM|CarlosAg|LINK
We have now released a refresh version of the IIS SEO Toolkit that includes the fix for this problem, so please install the latest version and you should stop seeing this problem.
Feb 02, 2010 12:23 AM|Joeri P|LINK
I just updated my version. And it works better now.
But when I crawl a ASP.NET MVC website with a Mobile version on, the SEO toolkit crawl the mobile version instead of the normal website. Is there somewhere a possibility to change this setting ?
Feb 02, 2010 03:15 PM|CarlosAg|LINK
Feb 05, 2010 12:15 AM|Joeri P|LINK
No. The site I check with the SEO toolkit is a ASP.NET MVC website, but when you browse with a mobile device to the site, you will see a mobile version of the site.
So, when I check the site with the SEO toolkit, the toolkit crawls the mobile version of my ASP.NET MVC website, not the normal view.
Feb 05, 2010 12:51 AM|CarlosAg|LINK
Your application must be making that decision and its likely that is making it based on User-Agent, am I right?
I would need you to confirm the exact logic your application is doing to choose the mobile version, since AFAIK there is no built-in logic in MVC for doing anything like this.
Are you using Request.Browser (ie Browser Capabilities) for that detection?
Could you give more details around the app, is it your own code, or an open source app, or?
Feb 05, 2010 01:26 AM|Joeri P|LINK
Yes i'm using the Request.Browser
It's my own code, and if you would, I can mail you more information about the app or about the source code.
Jul 11, 2019 01:31 PM|SAMFi|LINK
I keep running across the same problem across multiple sites (not all but often enough) - I have been looking for a solution and the diagnosis here seems to be correct (of why this is happening): https://weblogs.asp.net/jeffwids/iis7-search-engine-optimization-toolkit-does-not-like-html5-doctype
At the bottom: the estra line above the doctype tag - this isnt an empirical reasearch rather I checked agqainst all the sites I could remeber where the crawl returned only 1 result url - any thoughts on this?