IIS 7 and Above
Web Farm Framework
Last post Aug 25, 2016 03:21 PM by LukaszH
May 08, 2012 11:13 AM|nastymatt|LINK
Anyone else noticed their web farm controller server memory creeping up if left alone?
I have two web farms, both the same set up i.e. one control server, 32 servers in farms of 4. inetmgr.exe sits about about 11mb when started. If I leave the farms alone after about 2 weeks the memory used by the process has crept up to around 2.5gb. The
server has 4gb. The only way i can find to reduced the size is to stop the process and restart.
Any one seen this or know of a solution?
IIS 7.5 running in Windows 2008 r2. wff 2.2.
Web Farm Framework
Web Farm Framework 2.1
windows server 2008 r2
May 17, 2012 10:22 AM|nastymatt|LINK
No one? Not even Microsoft? This is a pretty big flaw in WFF to go unnoticed.
Jul 20, 2012 02:42 AM|LukaszH|LINK
Jul 25, 2012 02:28 PM|talldaniel|LINK
Yes, I noticed this. It appears to be caused by people leaving their IIS GUI open, when not in use. Over time it accumulates memory leak until it finally crashes the server.
Solution. Always close the GUI when done.
Aug 07, 2012 03:19 AM|LukaszH|LINK
Nov 19, 2012 10:32 AM|Christar|LINK
I am experiencing the same problem, no GUI opened. Memory used by WFF just gets more and more until server freezes. This freezing of the controller sometimes leads even to a teardown/offline-taking of the farm... Ugly... IF the crashed controller starts
up again, web farm service provisions the servers and opens them on the LB (self written LB plugin similar to the F5 one)
No solution out there yet?
Dec 05, 2012 10:45 AM|sam jongenelen|LINK
I have the same problem.
Leaving the MMC console on definately makes the problem worse; haven't looked at the ARR server for a while, but now it runs using about 3,07Gb! My initial memory usage (on fresh start) is about 26Mb
Not sure how to proceed, anyone tried any hotfixes ? I will test upcoming couple of days if disabling Disk Cache in ARR will help.
For testing purposes, i will leave the MMC open..!
Dec 05, 2012 08:38 PM|Rovastar|LINK
Has anyone raised a support case with Microsoft?
Mar 11, 2013 01:27 PM|LukaszH|LINK
Is it problem with WFF or ARR?
I have a couple of WFF servers and there is no problem with them. I have two ARR servers (one of them is with WFF installed) where the problem with memory leak appears. Nobody logged to server, only normal production work, application and platform provisioning
disabled, caching disabled.
Today I had machine crash because WebFarmService.exe would like to use more than 16 GB of memory (server with 4 GB RAM). Only regular web farm service restart is helping to have it in good conditions...
Event ID 2004
I have memory dump of WebFarmService.exe process when it grows to 3,5 GB. How to check whats inside?
May 17, 2013 06:51 PM|aleksandert1234|LINK
Did you manage to solve this memory leak issue?
May 21, 2013 03:37 AM|sam jongenelen|LINK
No. However, a workaround is to keep the InetMgr application closed...
May 28, 2013 05:48 AM|LukaszH|LINK
For me this workaround is not working. WebFarmFramework is consuming more and more memory even if nobody is logged on server. I've made quick look at my memory dump using DebugDiag tool, and I found that there is problem with native heap which is consuming
a lot of memory:
But what can I do with this informations?
May 30, 2013 08:11 AM|sam jongenelen|LINK
The better question is:
Is this fixed in Server 2012 / IIS8 ??!!
May 30, 2013 07:51 PM|aleksandert1234|LINK
If I understand correctly, you're experiencing this memory leak on primary server where WebFarmService.exe is running, right?
If that's correct, it would be great if you could create a dump file of WebFarmService.exe process when high memory usage is noticed. We're about to use Web Farm Framework on production servers in the near future and I really don't like what I'm seeing in
your post... :-/
Will you be able to publish compressed dump file somewhere? If not, I can set up a temporary ftp site or something similar...
I appreciate your help,
May 30, 2013 07:58 PM|talldaniel|LINK
You might want to reconsider using WFF until MS announces if they are going to upgrade WFF to support IIS 8.0
As far as I know, MsDeploy 3.0 will break WFF, and thus, you cannot use it with IIS8 or with Windows 2012.
I have not validated this to be true by experiment, but I did get it from a reliable source.
May 30, 2013 08:40 PM|aleksandert1234|LINK
I believe you can upgrade to MsDeploy 3.0 after WFF is installed, but I didn't confirm it yet.... will let you know in a day or two.
Regarding WFF and IIS 8.0... I heard some rumors that WFF should support IIS 8.0 in the future. I was told IIS team is totally focused on IIS 8.0 nowadays, but later thay should work on WFF too... Who knows? We'll see :)
May 30, 2013 08:52 PM|talldaniel|LINK
The credible source that told me MSDeploy 3.0 is not compatible with WFF, was a supoort engineer from the MS WFF support team that we used when we filed a support ticket (for another issue) with MS about nine months ago. But I hope you are right.
May 30, 2013 09:02 PM|aleksandert1234|LINK
I hope you're not right, but I'm affraid you are... :)
Btw, what would be a decent alternative to WFF? What are you using today?
May 30, 2013 09:19 PM|talldaniel|LINK
It depends on which features you are referring to. We only use it for replication. What are your planning to use it for? (We arent upgrading to 2012 right away so we have some time.)
May 31, 2013 04:20 AM|LukaszH|LINK
Memory leak is on controller server, primary server and all secondaries are working ok.
My configuration in this deployment:
Memory leak appear on both servers. I'm restarting Web Farm Service every second night.
I have two other deployments, one much bigger, and one for testing purposes. But there I'm not mixing ARR with WFF and it's working fine (on separate servers, two for ARR and one for WFF). Everywhere WFF and ARR configurations are the same.
May 31, 2013 09:24 AM|sam jongenelen|LINK
We are using WFF and ARR in multiple enviroments, with similar setups, and i can say that:
memory leak appears to increase:
This leads me to believe that the problem must be related to:
As a test i disabled automatic Application Provisioning entirely and started using a batch-based Web Deploy script (which manually tells the controller server to provision the applications of the nodes. We deploy manually every 2 weeks (or less) and the
memory leak still persists.
Does anyone have other research results and willing to share? Also, what alternatives are there? We are hosting .NET 4 applications, and the setup seamed like a great idea (and working) but I really don't like rolling restarts on production servers...
May 10, 2014 08:13 AM|Zoltan Lehoczky|LINK
I also experience increasing memory consumption by the w3wp.exe process used by ARR.
Some technical backround: Win Server 2012 R2 Datacenter (Azure Cloud Service), latest ARR and URL Rewrite installed.
What I found causes memory to increase is if a request that is proxied through ARR returns a 404. Strange as it seems a 404 response causes a cca. 1MB permanent increase in the memory consumption of the process.
Also worth to share is that the same web app that contains the rewrite rules employing ARR also hosts a web API endpoint (used for managing the cache). I only experienced the memory leak to happen after adding this API, what in turn needed the addition of
a Global.asax file. However my code is doing nothing on a 404 so I have a hard time getting a grasp on why these two are related, but the fact still is that before adding the API the process used a couple ten MBs of memory and now it uses increasingly more
So this is a bit different than what other here see, but I guess there should be something in common that we don't see yet.
May 10, 2014 04:25 PM|sam jongenelen|LINK
Well, perhaps it is not your application that gives 404, but the Webfarm client/server agent communications (do you have Web Farm Framework running?)
Otherwise, I honestly have no idea too..
Aug 25, 2016 02:39 PM|o0scap|LINK
Aug 25, 2016 03:21 PM|LukaszH|LINK