IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?RSS

12 replies

Last post Feb 14, 2020 03:40 PM by jojobar

‹ Previous Thread|Next Thread ›
  • IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Aug 26, 2017 09:49 PM|bing@uncoolcentral.com|LINK

    I can't seem to crawl sites that have TLS 1.0 disabled, e.g. lelabofragrances.com

    ...But I don't know of anOTHER site w/ TLS 1.0 disabled to try, so I can't confirm it's all non-TLS 1.0-friendly sites, or just that one.

    I already have this update, which might allow TLS 1.1/1.2 for IIS SEO Toolkit?
    https://support.microsoft.com/en-us/help/3140245/update-to-enable-tls-1-1-and-tls-1-2-as-a-default-secure-protocols-in

    Any thoughts?

    More to the point: do you know of any other sites w/ TLS 1.0 disabled I could try to crawl?

    Quick way to see which protocols are enabled on a site:

    https://cryptoreport.websecurity.symantec.com/checker/

    Thanks!

    -Dan

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Aug 26, 2017 10:07 PM|bing@uncoolcentral.com|LINK

    UPDATE: found one other site w/ TLS 1.0 disabled. Can't crawl it either.

    community.optymyze.com

    ...So, what's up?

    Is there any way to get IIS crawler to crawl a site w/out TLS 1.0 enabled?

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Mar 13, 2018 12:29 AM|bing@uncoolcentral.com|LINK

    ping

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Mar 17, 2018 09:33 PM|Yesmin|LINK

    Hi, did you find a solution to crawl a site that has TLS 1.0 disabled?

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    May 16, 2018 01:45 PM|mjohnson1287|LINK

    I can confirm that if a site has 1.0, 1.2 and 1.3 enabled it'll work fine, only if 1.0 is disabled will this tool not work. I've installed 4.7.x current .NET and it had no effect. 

    Any thoughts from the peanut gallery? This may be an older tool but nothing out there does as good a technical SEO scan IMO.

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    May 29, 2018 07:57 AM|Menashe Avramov|LINK

    Hi, same issue here, any fix available?

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Sep 27, 2018 09:04 PM|herved|LINK

    Hi,

    I found a way using Fiddler.

    First install it, then go to Rules > Customize Rules, and in the Script editor add the following in the OnBeforeRequest() function at the top:

    if (oSession.HTTPMethodIs("CONNECT") && oSession.HostnameIs("www.yourdomain.com")) {
      oSession["x-OverrideSslProtocols"] = " ssl3;tls1.0;tls1.1;tls1.2";
    }

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Dec 07, 2018 11:41 PM|Benjiiim|LINK

    Can you please explain how this Fiddler configuration is supposed to fix the problem in IIS SEO Toolkit?

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Dec 08, 2018 01:56 AM|lextm|LINK

    Benjiiim

    Can you please explain how this Fiddler configuration is supposed to fix the problem in IIS SEO Toolkit?

    Fiddler works as a proxy, so the network diagram becomes,

    Crawler ---(connection A)---> Fiddler ---(connection B)---> your site

    As the crawler only accepts TLS 1.0, you can configure connection A to support TLS 1.0, while connection B can be TLS 1.1 and above.

    Lex Li
    https://lextudio.com
    ---------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Dec 20, 2018 07:41 PM|jojobar|LINK

    herved

    Hi,

    I found a way using Fiddler.

    First install it, then go to Rules > Customize Rules, and in the Script editor add the following in the OnBeforeRequest() function at the top:

    if (oSession.HTTPMethodIs("CONNECT") && oSession.HostnameIs("www.yourdomain.com")) {
      oSession["x-OverrideSslProtocols"] = " ssl3;tls1.0;tls1.1;tls1.2";
    }

    Tried this it is still not working I am getting an error: 

    The link to 'https://www.....com/' has resulted in an error.
    Details: System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.
    at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
    at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
    at System.Net.TlsStream.ProcessAuthentication(LazyAsyncResult result)
    at System.Net.TlsStream.Write(Byte[] buffer, Int32 offset, Int32 size)
    at System.Net.ConnectStream.WriteHeaders(Boolean async)
    --- End of inner exception stack trace ---
    at System.Net.HttpWebRequest.GetResponse()
    at Microsoft.Web.Management.SEO.Crawler.UrlDownloader.OnGetContent(CrawlerProcessContext context)

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Mar 08, 2019 11:40 PM|kingcaliber|LINK

    any progress to this issue?

    How did you guys solve it?

    I was using the seo tool till August 2018 but now it stopped working and gives me " the url for hyperlink is broken"


  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Jul 11, 2019 01:24 PM|SAMFi|LINK

    Hi Guys

    Hope this is not off topic - the resulting error might be the same but maybe the cause here is different - also unfortunately dont have a solution, rather looking for it myself too..

    Anyway I have come across multiple sites where I get a single row (when crawling with IIS SEO toolkit) and its the "the url for hyperlink is broken" result 

    Always qwondered whats up and came across this: https://weblogs.asp.net/jeffwids/iis7-search-engine-optimization-toolkit-does-not-like-html5-doctype

    And in the comments there: "Actually, it's not the HTML5 tag that's causing SEO grief, it's the extra whitespace before/above the tag that is the problem. ..." <- That seems to be true in all the cases I have come across.

    So would anyone with any IIS SEO TOOLKIT customization experience know how to fix this simple but irritatting issue?

  • Re: IIS SEO Toolkit Not Crawling Sites w/ TLS 1.0 Disabled?

    Feb 14, 2020 03:40 PM|jojobar|LINK

    For those who might have the same issue.

    It is not encouraging that MS has completely abandoned it and also not released the source code. There is no better solution in the market except for a couple of paid solutions that is not as intuitive as this tool.

    I had the same issue and have come up with a solution that works. It involves, using the IIS-SEO dll and then calling the remote server with tls 1.2. I am still using the UI created by Microsoft IIS but planning to replace it with my own when I get time or help to write it.

    Here is the link with instructions: https://github.com/Officeclip/IIS-SEOCrawler

‹ Previous Thread|Next Thread ›