I am trying to figure out how to use WCAT to to stress testing on a PHP script. I basically want to see how many requests the server can handle with various configuration scenarios, to see what works best for my setup.
This is what I have pieced together so far. I've created a batch file to run
when I run the start_wcat.bat file it opens to command prompts with no errors and seems to proceed running the test. But i don't think its working correctly, the output statistics seem weird, and my data from perfmon seems really low and
only displays during what seems the warmup period.
Isn't 4 requests/sec really really bad? This is a brand new Quad core with 4GB of ram.
I don't know why all the values are 0?
So I know i'm doing something wrong, but I cna't figure it out. I ahve also tried launching WCAT directly, but it has the same results as above.
You seem to be running wcat correctly, though I would recommend you running it on a different machine from the server under stress.
Having said that, you have 2 URL's in your only transaction, every wcat virtual client will sequentially execute the URL's in this transaction. So from the information in this thread the second URL seems to be taking some time(or is stuck) to come back.
Can you remove this URL and see if you can get rid of all 0's that you are seeing?
I did what you said and took out the 2nd request from my scenario.ubr.
My CPU usage while the WCAT test is running is at 100%.
I am still only getting 4/sec in the output and the last output is still all 0's. I have no idea what's going on.
I have downloaded the older WCAT 5.2 and i'm having more luck with it, the UBR config syntax is a different format but I am now able to get actual results with it.. i think. Since the controller and the client both have outputs, which one should I be looking
at? The numbers are close but not the same.
This is my UBR for WCAT 5.2, I am testing a default phpbb3 app.
[Configuration]
WarmupTime 10s
NumClientMachines 1
MaxRecvBuffer 64K
CooldownTime 5s
ThinkTime 0s
NumClientThreads 80
Duration 30s
Comment Test Server
[Performance]
[Script]
SET KeepAlive = TRUE
NEW TRANSACTION
classId = 1
weight = 100
server = "localhost"
NEW REQUEST HTTP
URL = "/forums/index.php"
Hmmm..regards to CPU utilization, which process is consuming the most, w3wp, php-cgi, wcclient or wcctl? You can look it up in task manager.
I'm not sure what index.php is doing, but can you try to stress a simple helloworld.php page with the following script,
<?php echo'hello world' ?>
In your WCAT 5.2 client file you have 80 virtual clients against your 20 from WCAT 6.3, so in theory it should fare 4 times better. Regards to what number you should be looking at, you should be looking at the log file that the controller generates. The
output log file will only contain data for the actual duration of your run, so it will not contain data for warmup or coooldown.
Strange I made a script called test.php with the hello world as you suggest, then re-ran WCAT 5.2
All the requests are "Bad requests", there are no "200 OK" requests for some reason. I tested the test.php in a browser it works fine and its sitting in the sites root.
URL = "/test.php"
As for CPU usage, seems WCAT itself takes a lot, so I will try install the client on another computer on the network once its working properly to get more accurate results.
using a plain test.html file works, but I want to test PHP.
for a plain html file the values are pretty high. (Total 200 OK, 224114, 7470.47)
It would be interesting and helpful if someone else could run the same WCAT test on there server, to compare values. I ran the client from a computer on the local client for these tests.
I guess if WCAT 5.2 is working, there isn't any need to figure out why WCAT 6.3 won't work. Unless there are some improvements in the v6.3 that are worth while.
You shouldn't compare numbers from different hardware because it can churn requests faster or slower depending on it's computing speed. If you plan on doing basic testing you can use WCAT 5.2 but we would recommend using the latest. Here is a sample WCAT
6.3 client file, modify the URL as appropriate,
scenario
{
name = "helloworld";
warmup = 180;
duration = 180;
cooldown = 30;
default
{
setheader
{
name = "Connection";
value = "keep-alive";
}
setheader
{
name = "Accept";
value = "image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, */*";
}
setheader
{
name = "Accept-Language";
value = "en-us";
}
setheader
{
name = "User-Agent";
value = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0";
}
setheader
{
name = "Accept-Encoding";
value = "gzip, deflate";
}
setheader
{
name = "Host";
value = "webcat";
}
version = HTTP11;
statuscode = 200;
close = ka;
}
thanks, that seems to work better. The test seemed to have run and I guess the nice XML output is one reason to use v6.3. The "Requests/sec" result is almost the
same as i got with 5.2 for the helloworld (233/sec).
With the WCAT 5.2 I ran the helloworld on my new server and on my old server. And its concerning that the old server, which is currently in use and running 50 sites, gets high values than my new server. I changed IIS7 to use PHP via ISAPI since thats what
the old server uses, to compare.
New Server (Windows 2008/IIS7, Quad Core X3220, 4GB ram)
<div mce_keep="true">PHP ISAPI</div>
<div mce_keep="true">Total Requests: 6674</div>
<div mce_keep="true">Per Second: 222</div>
Old Server (Windows 2003/IIS6, Dual CPU Xeon 2.4Ghz, 2GB ram)
<div mce_keep="true">PHP ISAPI</div>
<div mce_keep="true">Total Requests: 16885</div>
<div mce_keep="true">Per Second: 562</div>
I thought windows 2008 with IIS7 would result in better performace, it's one of the main reasons I upgraded. The new server is also a fair bit faster hardware. So why does my old server, which is also under load, get better results?
There are lot of variables here - different OS, different hardware. The recommended way of doing this is to install Windows 2003 on your quad core box and get the baseline number, I do understand that this might not be feasible. When trying to gauge performance
impact it is always better to change one thing at a time.
I don't know about PHP ISAPI but running the above helloworld test using FASTCGI gives comparable numbers on both Windows 2003 and 2008 - of course on the same hardware.
canadaka
94 Posts
Help stress testing PHP with WCAT 6.3
Jul 09, 2008 11:29 PM|LINK
I am trying to figure out how to use WCAT to to stress testing on a PHP script. I basically want to see how many requests the server can handle with various configuration scenarios, to see what works best for my setup.
This is what I have pieced together so far. I've created a batch file to run
start_wcat.bat
The 2 config file loaded are as follows:
settings.ubr
scenario.ubr
when I run the start_wcat.bat file it opens to command prompts with no errors and seems to proceed running the test. But i don't think its working correctly, the output statistics seem weird, and my data from perfmon seems really low and only displays during what seems the warmup period.
Isn't 4 requests/sec really really bad? This is a brand new Quad core with 4GB of ram.
I don't know why all the values are 0?
So I know i'm doing something wrong, but I cna't figure it out. I ahve also tried launching WCAT directly, but it has the same results as above.
client.bat
controller.bat
WCAT php
mukhtard
206 Posts
Microsoft
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 12:21 AM|LINK
Hi,
You seem to be running wcat correctly, though I would recommend you running it on a different machine from the server under stress.
Having said that, you have 2 URL's in your only transaction, every wcat virtual client will sequentially execute the URL's in this transaction. So from the information in this thread the second URL seems to be taking some time(or is stuck) to come back. Can you remove this URL and see if you can get rid of all 0's that you are seeing?
Also what is the CPU utilization you are seeing?
Thanks,
Mukhtar Desai
IIS Performance Team
wcat performance
canadaka
94 Posts
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 02:46 AM|LINK
I did what you said and took out the 2nd request from my scenario.ubr.
My CPU usage while the WCAT test is running is at 100%.
I am still only getting 4/sec in the output and the last output is still all 0's. I have no idea what's going on.
I have downloaded the older WCAT 5.2 and i'm having more luck with it, the UBR config syntax is a different format but I am now able to get actual results with it.. i think. Since the controller and the client both have outputs, which one should I be looking at? The numbers are close but not the same.
This is my UBR for WCAT 5.2, I am testing a default phpbb3 app.
mukhtard
206 Posts
Microsoft
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 04:00 AM|LINK
Hmmm..regards to CPU utilization, which process is consuming the most, w3wp, php-cgi, wcclient or wcctl? You can look it up in task manager.
I'm not sure what index.php is doing, but can you try to stress a simple helloworld.php page with the following script,
<?php echo'hello world' ?>
In your WCAT 5.2 client file you have 80 virtual clients against your 20 from WCAT 6.3, so in theory it should fare 4 times better. Regards to what number you should be looking at, you should be looking at the log file that the controller generates. The output log file will only contain data for the actual duration of your run, so it will not contain data for warmup or coooldown.
Thanks,
canadaka
94 Posts
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 04:48 AM|LINK
Strange I made a script called test.php with the hello world as you suggest, then re-ran WCAT 5.2
All the requests are "Bad requests", there are no "200 OK" requests for some reason. I tested the test.php in a browser it works fine and its sitting in the sites root.
URL = "/test.php"
As for CPU usage, seems WCAT itself takes a lot, so I will try install the client on another computer on the network once its working properly to get more accurate results.
using a plain test.html file works, but I want to test PHP.
for a plain html file the values are pretty high. (Total 200 OK, 224114, 7470.47)
Does anyone have a sample UBR file they use?
canadaka
94 Posts
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 05:47 AM|LINK
I seemed to have fixed the bad requests (500) problem. I had to manualy edit the applicationHost.config to add the PHP_FCGI_MAX_REQUESTS variable.
This is the post that helped me: http://forums.iis.net/p/1149114/1868558.aspx#1868558
It would be interesting and helpful if someone else could run the same WCAT test on there server, to compare values. I ran the client from a computer on the local client for these tests.
I guess if WCAT 5.2 is working, there isn't any need to figure out why WCAT 6.3 won't work. Unless there are some improvements in the v6.3 that are worth while.
mukhtard
206 Posts
Microsoft
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 06:31 AM|LINK
You shouldn't compare numbers from different hardware because it can churn requests faster or slower depending on it's computing speed. If you plan on doing basic testing you can use WCAT 5.2 but we would recommend using the latest. Here is a sample WCAT 6.3 client file, modify the URL as appropriate,
scenario
{
name = "helloworld";
warmup = 180;
duration = 180;
cooldown = 30;
default
{
setheader
{
name = "Connection";
value = "keep-alive";
}
setheader
{
name = "Accept";
value = "image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, */*";
}
setheader
{
name = "Accept-Language";
value = "en-us";
}
setheader
{
name = "User-Agent";
value = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0";
}
setheader
{
name = "Accept-Encoding";
value = "gzip, deflate";
}
setheader
{
name = "Host";
value = "webcat";
}
version = HTTP11;
statuscode = 200;
close = ka;
}
transaction
{
id = "helloworld";
weight = 100;
request
{
url = "/helloworld/helloworld.php";
}
}
}
canadaka
94 Posts
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 06:54 AM|LINK
thanks, that seems to work better. The test seemed to have run and I guess the nice XML output is one reason to use v6.3. The "Requests/sec" result is almost the same as i got with 5.2 for the helloworld (233/sec).
With the WCAT 5.2 I ran the helloworld on my new server and on my old server. And its concerning that the old server, which is currently in use and running 50 sites, gets high values than my new server. I changed IIS7 to use PHP via ISAPI since thats what the old server uses, to compare.
New Server (Windows 2008/IIS7, Quad Core X3220, 4GB ram)
Old Server (Windows 2003/IIS6, Dual CPU Xeon 2.4Ghz, 2GB ram)
I thought windows 2008 with IIS7 would result in better performace, it's one of the main reasons I upgraded. The new server is also a fair bit faster hardware. So why does my old server, which is also under load, get better results?
mukhtard
206 Posts
Microsoft
Re: Help stress testing PHP with WCAT 6.3
Jul 10, 2008 07:22 PM|LINK
There are lot of variables here - different OS, different hardware. The recommended way of doing this is to install Windows 2003 on your quad core box and get the baseline number, I do understand that this might not be feasible. When trying to gauge performance impact it is always better to change one thing at a time.
I don't know about PHP ISAPI but running the above helloworld test using FASTCGI gives comparable numbers on both Windows 2003 and 2008 - of course on the same hardware.
ma_khan
869 Posts
MVP
Moderator
Re: Help stress testing PHP with WCAT 6.3
Jan 07, 2010 09:29 PM|LINK
I was just testing PHPBB3 myself so thought you guys should know he results as well:
OS: Windows Server 2008 Enterprise SP1
Configuration: Quadcore, 4 GB RAM
Environment: IIS, ASP.NET and MYSQL
PHPBB3 - PHP without FastCGI:(/forums/index.php)
Request\Sec: 2 :( :( :( :(
PHPBB3 - PHP with FastCGI:(/forums/index.php)
Request\Sec: 19
PHPBB3 - PHP with FastCGI and Output Caching:
Request\Sec: 2638
That's like a 140 time increase... M I doing it right ... :) :D
My .cfg files are as follows: (WCAT 5.2)
Script.cfg
NEW TRANSACTION
classid = 1
NEW REQUEST HTTP
Verb = "GET"
URL = "http://ip/forums/index.php"
Config.cfg
Warmuptime 5s
Duration 30s
Cooldowntime 5s
NumClientMachines 1
NumClientThreads 20
Distribution.cfg
1 50