View Complete Thread
  • Rovastar Rovastar

    5482 Posts

    MVP

    Moderator

    IIS 6 performance tweak guide (draft)

    Mar 25, 2008 12:07 PM|Rovastar|LINK


     

     

    =====

    Hi folks,
    I am starting to write a performance tweaking guide for IIS 6.0 (I have not used IIS 7 enough yet and it is not in the enterprise much atm).

    I am a contractor and mostly now-a-days (for the past 7 years) I do IIS web support /architect stuff and this has prompted me to write a performance tweaking guide as I am doing performance review off my own back in my latest role and it will be a useful checklist for thing in a new role.

    I found that often the information is all over the place in different MS documentation, blogs, articles, etc and that often they would shy away from placing values in them. Although obviously you should not change the metabase or registry for major changes without doing performance reviews, etc it is nice to have some guidelines of what to expect.
    (I intend increasing this to include detailed monitoring, how to do benchmarking in conjection with this, security, etc for IIS, also one for SQL performance tuning in a web environment and hopefully how to debug IIS hangs and crashes)

    Anyway I have slowly got something together and note a few things.
    a) It is not finished yet.
    b) See a)
    c) It was written initially for a large intranet infrastructure I am working in atm so some specific may remain. I will remove these to more general terms (and I probably have some sentences in there that make little sense….sorry)
    d) The grammar and spelling are probably off. I will go through this in more detail later.
    e) Some of the technical detail may even be off. So please feel free to suggestion corrections.
    f) There are things missing some of them obvious. If they are doing best practice where I am I have not written about it so there is more to add. Also things no things like PHP as they have no PHP.
    g) I apologies as I am working away atm and I cannot upload this to my website so it is a cut and paste job. :( It will be uploaded later.

    Any I would like as much feedback as possible and when it is finished hopefully I will publish it somewhere.

    Let me know what you think

    John

    IIS 6.0 performance tweak guide (draft)

    Standard Resource Toolkits on the servers

    The basic installation of IIS doesn’t come with many useful tools for administration.
     
    Consider installing the following additional software that should be installed on all web servers:

     Microsoft IIS Resource Kit
     
     Microsoft IIS Diagnostic Toolkit (and download the latest versions of the apps that come with these kits e.g. DebugDiag, LogParser)

     Microsoft Sysinternal Suite (stuff like process explorer, etc) http://technet.microsoft.com/en-gb/sysinternals/0e18b180-9b7a-4c49-8120-c47c5a693683.aspx

     Microsoft Network Monitor (latest version)

     A better text editor than standard notepad like notepad++

    Standardise the build of the servers so they are basically the same.

    Hosts


    It is useful to be able to access the sites on your server from the server. This is useful for many reasons detailed error messages, traffic flow through the infrastructure so you know you are on the correct machine, etc.

    Many production environments there may be firewalls and policies in place to stop you doing this if possible you should try and obtain this.

    It may be as simple as trusting your own sites in Internet Explorer so they are allowed through hard admin

    Often in farm environments it is useful to know the actual server that they you are on.

    You could have many servers and the hardware load balancer that controls what traffic comes to which server sometimes isn’t helpful.

    In this case it may be may be useful set up additional host headers and host files references on the production servers in the farms so you can directly access that specific server from its browser.

    Make sure all the web farms files are in synch

    Nightly checks of the webfarm files to make sure they are all in synch with each other.

    We can script this for all the files or schedule tasks with a package like windiff to ensure they are all in synch. Send out mail alerts if they are found to be out of synch.

    Additionally we can modify this so we can work out if staging is in synch with the production server farm. Obviously there will be some changes between the web configuration files but it is still a useful maintenance technique.

    Also check these with the disaster recovery servers to ensure they are all in synch.

    Checking the details of each websites config files (e.g. web.config, etc) and each server metabase file of the farm is also very useful. Obviously these things shouldn’t change but it would be useful to know if they did.

    Luckily, Microsoft are in the process of writing an app for this and other purposes.

    Web Deployment Tool (currently in a useable tech preview stage) has been designed for IIS7 and for migration from IIS6. But it also features a useful looking comparison tool to compare the states of multiple IIS6 servers.

    http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1602

    Application Pool Recycling

    By default an application pool has a recycle period of 1740 minutes or 29 hours. This is a ludicrous setting as it will be recycling application pools in peak times.

    Much better is to recycle the application pools at fixed off peak times. For example - 1:00am.

    For servers in farm environments stagger these recycle times so they are not all the same. E.g. Server A – 1:00am, Server B – 1:10am, Server C – 1:20am, etc

    Additionally you can log when the app pools recycle. There are many reasons for a recycle and by default these are not recorded in the event viewer (very little is recorded by default for us admins). It is useful to have this for all events that occur.

    Complete list:

    AppPoolRecycleTime
    AppPoolRecycleRequests
    AppPoolRecycleSchedule
    AppPoolRecycleMemory
    AppPoolRecycleIsapiUnhealthy
    AppPoolRecycleOnDemand
    AppPoolRecycleConfigChange
    AppPoolRecyclePrivateMemory

    To turn them all on:

    cscript adsutil.vbs Set w3svc/AppPools/DefaultAppPool/LogEventOnRecycle 255

    HTTP Compression

    This can be especially useful for static web content as the compression only has to be done the once and then cached. Dynamic content is compressed on the fly and not cached.

    One of the main areas of usefulness of compression is the saving of the network traffic and thus a potential reduction for your hosting bills.

    Although HTTP compression was previously more useful where there was less broadband and more dialup in years gone by. In today modern environment it is still a useful tool and should not overlooked. Increasingly people are also using mobile, wireless, etc in greater numbers and these will benefit much more with compression.

    One of the previous downsides to HTTP compression years ago was that browsers did not support the HTTP 1.1 standards for this but from, I think IE5.5, they have been setting this ‘on’ as default.

    (The benefits of an Intranet environment is that, in general, you have fewer multiple browsers to be concerned about as you are likely to know what company standard browsers are.)

    Obviously static previously uncompressed content like HTML pages can benefit a lot from compression.

    Although on an intranet the rewards of compression of web content should more than make up for small CPU overhead of producing the compression in the first place.

    The server is not local and is in Ireland so people connecting to the UK intranet will benefit from the reduced amount of traffic. For quickly produced pages on the website the main area of slowdown for the user will the time it take to receive the pages across the network.

    For the infrastructure here you will probably be saving 30% in network traffic overall (plain text files; HTML, ASP, JS, etc (we can configure in the metabase what files we want to compress beyond the default ones) will benefit from 50-60% size reductions for compression, jpegs, and other already compressed formats do not get compressed by default or do we want the extra processing power of compressing them) for a small increase in CPU time which still shouldn’t be more than double the current CPU load (which is very low nearly always below 5%) even maxed out.

    Here is some sample data from Microsoft on the issue:

    Blah blah blah

    Default Documents order

    This is a little anal but it is sadly a common occurrence. Place the actual ‘default content page’ of the website at the top of the list.

    For example if you default page is

    www.mywebsite.com/default.asp

    then don’t have a default documents order of

    Default.aspx
    Default.htm
    Default.asp
    Index.htm

    I know I am being a little anal as this is a small overhead in IIS where it looks first for the other files. As this happens every time, people could be going to them tens of thousands of times a day and it would (slowly) add up.

    Now I don’t know if here all pages are referenced directly with full URLs in which case this is less relevant but still best practice and worthwhile implementing.


    More detailed logging

     By default and IIS site only has the standard default IIS logging turned on.

    There are many more useful fields that we turn on and although these make the logs bigger can but very useful when you can query search through them. Often cookies is not need and is a large

    True you can save some CPU time if you reduce what you log (especially with things like useragent) but the usefulness of these logs more than outweighs the benefit.

    However to get this really useful you have to search it.

    The LogParser tool is very useful however I would recommend going one step further and collating them if possible in a database.

    This give you more scope and makes it very easy to troubleshoot.

    (Although this is more useful as the types of problems are different in a large Internet environment rather than an Intranet one it is still very useful.)

    Some example scenarios that it would have been really handy in the brief time I have been on the servers.

    Troubleshooting a problem where we are investigating an unresponsive IIS server.

    Often a sign of the impending problems might be in the logs might be a collection of 500 errors and failure can happen

    To find out when the errors first occurred we can write queries on the log database for all servers in a particular farm.

    We can narrow the searches down easily to a particular directory, server(s), refers, user, headers, http status codes (and in IIS 6 error sub-codes), etc.

    Then we can see patterns in the 500 errors to see what occurred there.

    This process is very difficult to do at present with flat logs when you are looking over days across a farm of multiple server and if you have to search large files over multiple parameters it nearly impossible.

    There are very few tools for troubleshooting IIS web server problems and many of these have to run in real-time before the error occurs which is never ideal on a production machine (and we have to wait for the error to occur again)

    So collating the IIS logs into a searchable form is a very useful tool.

    Optimising connection timeouts

    The default setting for a site is that a connection stays open for 120 seconds of inactivity before closing and the visitor will have a establish a new connection.

    This is controlled on a per-site basis.

    Website properties  Website tab.

    Http Keep-Alives and connection timeout.

    We are using the best practice of Keep-Alives how the time out period is that of the default. This is generally regarded as being too high. We should reduce this to 60 seconds (or less – many recommend 30 seconds) which is more realistic.

    Caching

    Optimising object cache TTL (time to live)

    By default IIS hold every object in a cache for 30 seconds then it is discarded.

    For a dynamic environment, like webappfarm, this is really not needed so the recommend best practice is that this can be reduced. I would say to 10 seconds.

    Conversely for servers which hold a lot of static connect (the rest) it is recommended that this time is increased. I would say about 60 seconds some say longer.

    This is control via the registry only and is sever wide.

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters

    Create ObjectCacheTTL as a Dword and assign the value (in seconds)

    More Caching

    IIS cache size is, by default, half the physical memory in the box (metabase reg - MemCacheValue). However there is a limit of the maximum size of each file held in the cache (metabase reg – MaxCachedFileSize) this by default is set to 256KB.

    It is worth carefully checking the environment for commonly referenced files to see if any are over the 256KB default maximum. If we have a commonly accessed file that is over the limit then it makes sense to increase this limit.

    Look out for large pictures on common pages and increasing rich media like flash and video clips.

    Caching ASP content

    aspScriptFileCacheSize is the amount of asp pages that by default this is 500. Heavy usage server with a lot of asp content could benefit from an increase in this value however there are various counters to monitor before we need to change the value (and other related values, asp cache to disk, etc). Here Microsoft explains:

    http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/a9171159-c801-4705-b8a9-9eecf58a892f.mspx?mfr=true


    Working set preferences

    By default windows 2003 gives priority to the file system cache rather than the programs or working set.

    It is generally recommended that you change to the working set (programs) rather than the file system.
     
    This is common practice and highly recommend in say, an SQL Server environment, which have even more control of the memory resources in the program. IIS does benefit from this to a less degree hence why they do not turn it on as default as in SQL Server.

    System Properties  Advanced tab  Performance  Advanced Tab  Memory Usage  Adjust performance for Program.

    Additionally going hand-in-hand with this is changing the preferences of the network card to optimise file and print sharing for the network application data.

    Initially, I thought, that as IIS serves files than a large system cache is needed. However IIS is a network file and application server not strictly a file server or an application server and therefore has more of a benefit for network priority.

    Network Connections  Local Area Connection (for each connection)  Properties  File and Printer Sharing  Properties  Optimization  Maximize data throughput for network applications. – instead of file sharing which is default 2003 server setting (again SQL server machines will change the default to network apps)

    Configure more detailed reporting in performance monitor

    Configure Performance monitor to display associated process IDs instead of the incremental numbers (e.g. w3wp_5432 (where PID = 5432) instead of w3wp#1, etc) more details
    http://support.microsoft.com/kb/281884

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfProc\Performance\

    Create ProcessNameFormat as a Dword and assign the value 2

    Configure the admin user settings

    Although a personal preference, there are useful features that you can enable/disable for admin of an IIS server.

    These include:

    Turning off ‘Show Friendly HTTP messages’ in IE

    Create a dedicated web admin config msc file for mmc – include iis, computer management, com, .net components and all things that you need.

    In task manager turn on more columns
    Most useful are:
    ProcessID counter (so we can work out the name (with iisapp.vbs) of what worker process app pools are taking up the most resources) [this was turned off by default in Win2003]

    Virtual Memory

    Trivial this but I have always used 1.5 times (or more) the memory for virtual memory. Most of the IIS best practice document I have seen seems to indict this. I believe 1.5 times is the recommend windows amount for any windows server.


     

    Troubleshoot IIS in style
    https://www.leansentry.com/