« Previous Next »

Thread: A Server Update Severely degrades Performance

Last post 10-22-2009 4:22 AM by skumar2003. 4 replies.

Average Rating Rate It (5)

RSS

Page 1 of 1 (5 items)

Sort Posts:

  • 10-20-2009, 2:44 PM

    A Server Update Severely degrades Performance

    On the 16th of this month our server (64 bit) was updated (automatically) with multiple server updates. One of more of these updates severely degrades the bandwidth throuput of the machine.

    This machine is connected to a gigabit pipe to the Internet. Typically we see network utilization of about 500-600MB most times. but after these patches it dropped to 4.5-5MB. User were complaining because they were getting the content at the speeds they were used to (obviously). doing bandwidth test we were able to confirm that something was deffinately wrong. But what?

    To cut a long story short, we eventually decided to uninstall the patches that were installed last (5 updates in all) and see if that helps. And it did. It cost us a lot in terms of time, money and customer goodwill, and so I'm posting this here for others.

    we have yet to nail it down to one or more of the updates, but here is the list of updates that were uninstalled. Each one is "Important" and each one requires a re-boot.

    Security Update for Windows Media Format Runtime 11 for Windows Server 2008 x64 Edition (KB954155)

    Security Update for Windows Server 2008 x64 Edition (KB971486)

    Security Update for Windows Server 2008 x64 Edition (KB974571)

    Security Update for Windows Server 2008 x64 Edition (KB975467)

    Security Update for Windows Server 2008 x64 Edition (KB975517)

    Of course if anyone knows which one it is, please let me know.

     

     

  • 10-21-2009, 8:19 AM In reply to

    Re: A Server Update Severely degrades Performance

    I guess the real lesson is to not automatically update production servers without testing...  :)

    If it helps, I updated servers with those same patches, with the exception of the update for Windows Media Format Runtime, with no issues.  I can't say that's the culprit though, it's just not an update appropriate to our environment.

    Jeff

    Look for Wrox's new book Professional IIS 7 in your local bookstore, or order now at Amazon.com
  • 10-21-2009, 8:32 AM In reply to

    Re: A Server Update Severely degrades Performance

    Did you check event logs for any DNS, TCP/IP,Kerberos related errors?

    Also why don't you open Support Ticket with MS Support to investigate this?

    Also have you tried reproducing this issue? I mean on test box?

    ~ Ganesh

  • 10-22-2009, 4:18 AM In reply to

    Re: A Server Update Severely degrades Performance

    What do you test? How can you test? There is no way to go about testing every single functionality of a box after applying one update as a time. I think Microsoft should do all the testing there needs to be which they obviously haven't. And frankly I can't image an "update" that degrades the performance of a box (more like a virus really) get by their testing.By the way Jeff, do you have the IP restriction module installed? Because I think there is a conflict somewhere. Since I couldn't reproduce the issue on our test boxes (there is no way to replicate the hardware and software configuration), we eventually started installing those patches again and test after each patch. Well, as soon as the last patch was installed (The Media Services patch was not installed this time), the bandwidth dropped. So we quickly un-installed the patch again and this time - no luck. The bandwidth didn't go back up. So we un-installed all those patches again and still no luck.What we did find was that any given IP (that's connecting to the box) is restricted to some degree. The box is servicing thousands of request per second, but each IP that's connected seems to be restricted to some percentage of the bandwidth that connection has.

    So we went about uninstalling other software that was recently installed and that was the IP restriction module. Uninstall that and the bandwidth is back up to what it is supposed to be. We weren't using the module yet, but it was installed.

  • 10-22-2009, 4:22 AM In reply to

    Re: A Server Update Severely degrades Performance

    Ganesh,

    There were no events hardware or software that were out of place. Since posting my first post we have tried reproducing the issue but were not able to. The software configuration of the test box is not exactly the same as production (many reason why that is the case).

    I've never opened a support ticket and don't know what's involved and what steps will be required while they are attempting to support (do we have to keep re-booting the server, try this, try that etc.).

Page 1 of 1 (5 items)
Microsoft Communities