We are excited to announce that the IIS.NET Forums are moving to the new Microsoft Q&A experience. Learn more >

Preventing Exchange Autodiscover from causing floods of 404 errors on our web server [Answered]RSS

8 replies

Last post Jan 22, 2013 06:41 PM by Rovastar

  • Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 21, 2013 09:48 AM|thejoecarroll|LINK

    Our Exchange Autodiscover is configured correctly and working fine for us, but because of the way the Autodiscover feature works (see the answer to this forum discussion: http://community.office365.com/en-us/forums/162/p/42734/145157.aspx#145157) and the fact that our web server responds to ourdomain.com over SSL, ever since we moved to Office 365 last week, our web server has been receiving large numbers of requests for https://ourdomain.com/autodiscover/autodiscover.xml, which, of course, does not exist there. The small amount of additional load on the webserver is not an issue, but we monitor our web app with New Relic, and we keep getting elevated error level alerts because of the high number of 404 (page not found) errors caused by the Autodiscover mechanism checking ourdomain.com first as our potential default SMTP server (which it isn't).

    I've suggested the following solution to our lead developer but want to check first whether it could break Autodiscover for us: we can add a controller and route to our web app to handle requests for https://ourdomain.com/autodiscover/autodiscover.xml by responding with a simple 301 HTTP response status code (permanent redirect) to https://autodiscover.ourdomain.com/autodiscover/autodiscover.xml

    Will this work or cause problems?

    Our web server is a Ubuntu VPS with Apache proxying all requests to Tomcat, the web app being written in Java, so IIS-specific features won't help us, but at first glance, its Application Request Routing feature would be equivalent to what I want to do and I imagine there are more admins here who have dealt with Exchange than there are in the Ubuntu forums :-) Moreover, a Microsoft support rep suggested I consult these forums for advice (see: http://community.office365.com/en-us/forums/160/t/82051.aspx )

    Has anyone else here successfully implemented anything to deal with Autodiscover requests being sent to their web server? Basically, all I want to acheive is to make Outlook etc. go straight to consulting autodiscover.ourdomain.com without generating thousands of errors first on our web server. Any other suggestions are welcome.

    Thanks in advance, and apologies for being slightly off-topic :-)

    Joe

  • Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 21, 2013 09:08 PM|lextm|LINK

    The question to you is that whatever changes you want to make on IIS side, how do you know whether it breaks Office 365 or another app?

    This forum is dedicated to IIS, and your description is not only "slightly off-topic", but "obviously IMHO". Please post to Office 365 forums with your concerns and see what those experts say.

    Lex Li
    Want to have a chat on the issues you meet? Find me at https://booktime.xyz/p/lextm
    ---------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
  • Rovastar Rovastar

    5495 Posts

    MVP

    Moderator

    Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 21, 2013 09:11 PM|Rovastar|LINK

    This is a little off topic but I have seen the frankly annoying

    POST /autodiscover/autodiscover.xml

    appear in logs before for large corperate sites. Incidently when the webserver was not running IIS and without office365 mail to my knownledge but that was not my area.

    Mine were multiple times a second when they occured. Arrrr I thought it was a DOS attack.

    To be honest I just ignored them and never got to the bottom of why they occured.

    For me it is just a 404 and any htttp status codes 400's are client side error. I just presumed it was a misconfigurtion of some email clients (however they did it) that were pointing to this. It wasn't causing a big deal of load just a complete pain when troubleshooting the logs for other issues.

    The solution would be to simply rewrite these requests to the correct place but I have no idea where that is. ARR seems complete overkill unless you need to reroute these request internally.

    But I will be honest I don't know and understand what the autodiscover is.

    For me the key is what and why the client machine calls these - it is not on the webserver. Can you lookup the IP and trace it back and see what is on the client machine?

    (in fact I am curious now and I'll ask the experts there)

    Troubleshoot IIS in style
    https://www.leansentry.com/
  • Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 22, 2013 01:55 AM|thejoecarroll|LINK

    Hi Rovastar,


    and thank you very much for both responding to my off-topic post here as well as popping over to the Office 365 forums to participate in the discussion there.

    I imagine that the autodiscovery process checks the root of your domain first for historical reasons, but I doubt that tweaking modern versions of Outlook to check the autodiscover address first instead would break the feature. I guess, other than constructing a test for it, the only way to know if a 301 redirect will work is to consult an Outlook expert, but one way or another, I'd like to see this behaviour changed since it seems illogical as well as being responsible for considerable log spam on the web server.

    Unfortunately, I don't expect it to change. Yes, this is a client-side problem, but it's part of the default behaviour of the primary email client in business use today and it's not going to go away. The reality is that Outlook follows a process that spams any servers whose hostname is the same as your domain name (and Lync too). So if we accept that Exchange and Outlook are not going to change this behaviour any time soon, then the question remains, what is the best way to handle these spurious but unavoidable requests?

    Here's an explanation of the process, provided by Microsoft support technician Martin Xu:

    The reason is you create CNAME in your host provider and re-point your Autodiscover service to xxx.outlook.com. The testing tool will follow the default process to test the connection. First it will find the original domain to test whether autodiscover can pass, and then will test autodiscover.yourdomain.com. Finally it will check whether there are some CNAME setting for you domain to re-point autodiscover. 
    If you are using xyz@yourdomain.com, the primary SMTP domain is yourdomain.com. It will first check the primary SMTP domain, then check the secondary Autodiscover service URL https://autodiscover.yourdomain.com/autodiscover/autodiscover.xml. For more information about how the Autodiscover Service Work you can refer to: http://technet.microsoft.com/en-us/library/bb124251.aspx#works.

    So, let me rephrase my question to ensure it's on-topic: how do IIS admins deal with the constant barrage of GET and POST requests for /autodiscover/autodiscover.xml to minimise log spam and errors? Have any IIS admins here bothered to reroute these requests using ARR or other methods and are they aware of any official Microsoft-supported solution to this?

    Cheers,
    Joe

  • Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 22, 2013 01:57 AM|thejoecarroll|LINK

    Hi lextm,

    I'll make my question explicitly on-topic: how do IIS admins deal with the unavoidable and constant barrage of GET and POST requests for /autodiscover/autodiscover.xml to their web servers that is generated by Outlook and Lync in order to minimise log spam and errors? Have any IIS admins here bothered to reroute these requests using ARR or other methods and are they aware of any official Microsoft-supported solution to this?

    Joe

  • Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 22, 2013 02:24 AM|thejoecarroll|LINK

    BTW, to give an idea of the scale of the spamming: with just 40 users, most using both a laptop and a smartphone, Outlook, Lync etc. generated 1,226 POST and 410 GET requests in the last 24 hours!

    Note that autodiscover.xml is consulted not just during initial configuration of clients, but regularly to check for updated configurations.

  • Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 22, 2013 03:54 AM|lextm|LINK

    404 is so common that IMHO it is not even a problem. That's all. If all IIS admins are sensitive to 404 of pages that do not exist, I think they will never have any time to do anything else.

    IIS has been designed in a way that handling "floods of 404 errors" is ordinary (unless it becomes DDOS attacks or any other attacks). If you do want something cleaner for your eyes, you might consider ARR hacks as you stated, but whether that has any negative impact is of your own consideration.

    Regards,

    Lex Li
    Want to have a chat on the issues you meet? Find me at https://booktime.xyz/p/lextm
    ---------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
  • Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 22, 2013 04:32 AM|thejoecarroll|LINK

    In our case, the main annoyance (and the reason I noticed this issue) is that these errors are interfering with our New Relic application performance management and monitoring system and constantly generating elevated error level alerts (it's an excellent system, but, AFAIK, it doesn't allow us to filter out 404s from a specific URL from consideration, nor is raising the alert threshold an option; these errors are muddying the water and making it more difficult to recognise genuine issues with our web app). Anyway, I appreciate your point of view, lextm, but it doesn't answer my question, and if you don't think it's a question work asking, please don't feel obliged to answer it Smile

  • Rovastar Rovastar

    5495 Posts

    MVP

    Moderator

    Re: Preventing Exchange Autodiscover from causing floods of 404 errors on our web server

    Jan 22, 2013 06:41 PM|Rovastar|LINK

    Lex,

    The issue with this is a these 404s are like a (D)DOS attack. You can get a *lot* of them.

    Obvioulsy it is not an IIS issue but I am concerned that Exchange/Office 365 appears to be is always forcing a lookup on the root domain when this is wrong/poorly designed.

    As I have seen this in the real world in logs and it has been brought to my attention again I am keen to get a better solution and talking to the Office365 ppl on their forum.

    The solution to stop the 404s for IIS6 (that I presume that is what Joe is using) is to have a simple redirect in IIS so it processes a redirect. 

    http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/6b855a7a-0884-4508-ba95-079f38c77017.mspx?mfr=true

    (Make a seperate directory called autodiscover and redirect all traffic to  https://autodiscover.ourdomain.com/autodiscover/autodiscover.xml)

    The server will still process this and the same small overhead but there will be no 404s. Which I beleive will solve the 404 in the logs issue. However I think the bigger longer term solution is for Office365 not to send them in the first place.

    Troubleshoot IIS in style
    https://www.leansentry.com/