IIS 5 & IIS 6
Followup to IWA and load-balancing - Manual login not working
Last post Jul 09, 2010 12:15 AM by jimcpl
Jun 29, 2010 05:56 AM|jimcpl|LINK
Awhile ago, I posted about getting IWA working with two IIS servers behind an ACE/VIP:
Thanks to info from the above thread, we were able get that working, and users can "automatically" logged in to the websites, and this works with both Internet Explorer and Firefox.
In order for the automatic login to work, we had to do some configuration in IE (setting the Sites in Local Intranet, and enabling "automatically send username and password"), and in Firefox (using about:config and adding the FQDNs to the trusted-uris parameter).
However, in some cases, our users' browsers have not been configured, and in those cases, they get a popup login window.
For our IE users, they can login using domain\username and password when they get the popup login window, and, in one of our environments (we have two, one production and one for testing), Firefox users can also login manually successfully.
However, in the testing environment, Firefox users cannot login manually. When they try to login with correct domain\username and password (usin Firefox), the popup login window just re-appears.
I've checked all of the things that I can think of thus far, but I haven't been able to determine why, in that testing environment, the manual logins are failing to work.
Can anyone here suggest how we might go about trying to diagnose why these manual logins are failing?
Jun 29, 2010 01:27 PM|tomkmvp|LINK
Post the relevant IIS log file entries.
Jun 29, 2010 02:38 PM|jimcpl|LINK
The IIS log file is showing only 401 errors when I test with a manual login with Firefox (using the domain\username and password).
What I am really puzzled about is that if I access this same URL, but with the trusted-uris set in the FIrefox about:config (i.e., so that it does automatic login), I can access the resource fine.
Why would one (automatic login) work, while the other (manual login) not work?
Jun 29, 2010 04:32 PM|jimcpl|LINK
Jun 29, 2010 04:44 PM|jimcpl|LINK
Here's some additional info:
I checked the Local Security Policy for the IIS both in the production environment (where manual login works) and testing environment (where manual login fails), and both have the same settings for:
Network security: Minimum session security for NTLM SSP based (including secure RPC) clients
So, it looks like the local policy settings are the same for both the working and non-working systems.
I did another test in the non-working environment, using Firefox, and in Event Viewer on the IIS machine, I see:
Event ID: 529
User: NT AUTHORITY\SYSTEM
Reason: Unknow user name or bad password
User Name: iwatestuser1
Logon Type: 3
Logon Process: NtLmSsp
Authentication Package: NTLM
Any ideas why the manual login might be failing?
Jun 29, 2010 06:06 PM|Rovastar|LINK
Like Tom said post the logs what 401 error are you getting 401.x error?
For permissions look at procmon and read up about 401.x errors.
FOr manual login clear all the caches, temp internet files, password, etc on the client machine. Sometimes you get stuff locked in there with an old username/password. Even better geta fresh machien taht has never logged into the site before
Jun 29, 2010 07:29 PM|jimcpl|LINK
Both environments are closed, so I can't get files off of them directly. The logs show "401 2 2148074254" at the end of the lines.
Also, BTW, I checked the Event Viewer on one of the IIS server on our production environment, where the manual login is working, and when I do a successful manual login there, the event viewer looks similar to what I posted earlier, but with a Success instead
of Failed, authentication NTLM, and event id 540.
So, it looks like:
- On production, when do manual login using Firefox, it authenticates using NTLM successfully
- On test, when do manual login using Firefox, it attempts to authenticate using NTLM, but that fails.
Jun 30, 2010 04:56 AM|jimcpl|LINK
I found this article:
It notes that:
"It is also important to consider the error messages you get when you have a conflict in the LMCompatibilityLevel capabilities of different systems. Effectively, the only negotiation in this entire set of protocols is NTLMv2 Session Security. Downlevel systems
will ignore that flag. Therefore, the only error message you get if there is a settings conflict between client and server is that access was denied due to a bad username or password. Neither side to the transaction has the ability to determine that the issue
was due to unsupported authentication protocols. This makes troubleshooting somewhat challenging."
Anyway, I had tried changing the levels in Local Security Policy to see if it'd fix the problem, but still have the same problem (can't login manually).
I only have admin privileges on the IIS machines, so I can't see or change the GPO settings, if there are any, but I was wondering, does anyone know if there are settings at the GPO level, would those be overwriting the settings that I did in Local Security
Policy, and maybe that's why what I did didn't affect anything?
If so, I can try to see if I can get one of our Windows admins to make some changes.
Jul 01, 2010 02:37 AM|jimcpl|LINK
Does ANYONE have any ideas about what to check for to try to resolve this problem?
The next thing that I'm waiting to try is that I'm trying to get the MS AuthDiag tool moved to the IIS machines, so see if that flags anything wrong with our configuration, but, at this point, I'm getting kind of not so hopeful.
The thing that really puzzles me is that, since both the automatic login and the manual login work with Firefox on the production environment, I *KNOW* that it is possible to get both auto and manual login working with Firefox.
I just can't figure out why the manual login doesn't work with Firefox in the test environment.
I'll post back after I try AuthDiag, but if ANYONE has any ideas about what to look for, I'd really appreciate hearing them.
P.S. I found that the 401.2 error code (win32) that I'm seeing in the logs is SEC_E_NO_CREDENTIALS, but don't quite understand WHY I'm getting that, or how to eliminate it :(...
Jul 01, 2010 02:52 AM|jimcpl|LINK
I just found this article:
"Therefore, NTLM authentication cannot be used if an intervening proxy does not support keep-alive connections."
In my case, the IIS servers are load-balanced via a Cisco ACE, and I was wondering whether that counts as "an intervening proxy" in the context of that article?
What I'm thinking is that the keep-alive behavior may be different for the ACE in our production network vs. the test network, which I'm going to check on, but what puzzles me about that "theory" is why would the automatic IWA login work in both networks,
i.e., does the above statement only apply in the case of a manual login?
Jul 01, 2010 04:54 PM|jimcpl|LINK
This is a correction to the earlier post where I provided info about the IIS log file. Here's what I'm seeing:
401 2 2148072454 << I think that this is the initial IIS response to the GET request
401 1 0 << I think that this is the initial 401.1 that causes the popup login window to appear the 1st time
401 1 2148074252 << IIS sends another 401.1, causing the popup to appear again.
<< I think that this is the initial 401.2 that causes the popup login window to appear the 1st time
I think that the "2148072454" is a SEC_E_NO_CREDENTIALS, i.e., the request came into IIS with NO username/password, but I haven't figured out what the translation is for the "2148074252" error.
Does anyone know what that "2148074252" error is?
Jul 01, 2010 05:40 PM|jimcpl|LINK
Sorry for all the posts, but I just noticed something that seems a little odd about the IIS setups.
The actual hostnames for both IIS servers are 16 characters long, e.g., "idmiwa0x12345678" (e.g., seen by typing "hostname" at a command prompt, or looking at right-click My Computer, then Computer Name tab).
But, when I look in the IIS Admin on those machines, at the top under "Internet Information Services (local)", the name that is shown is only 15 characters long, i.e., "idmiwa0x1234567". Also, it looks like the IUSR_ account on these machines is "IUSR_idmiwa0x1234567",
i.e, using the shorter servername, without the 16th character.
Could this difference in the actual hostnames and the other names be causing this problem?
Jul 01, 2010 06:17 PM|tomkmvp|LINK
401 1 0 << I think that this is the initial 401.1 that causes the popup login window to appear the 1st time
Not exactly. What's expected here is a 401 3, meaning the anonymous user account (I'm assuming anonymous user here as you didn't post the fiull log entry as requested) does not have permission to the requested resource.
In your case, the 401 1 means that IIS was unable to authenticate the credentials given either because of a bad username or password.
Jul 01, 2010 06:50 PM|jimcpl|LINK
I really hope that someone here can help me, because I think that I have some new info that may be relevant to the problem.
I got AuthDiag installed, and when I ran it, I got a bunch of messages:
"Service principal name (SPN) for user "domain\iwaservice" not found in Active Directory".
I checked the domain\iwaservice account using setspn -l, and got:
The last one is for the VIP that the Cisco ACE is providing.
So, it looks like the domain\iwaservice account has the SPNs, and I don't quite understand why AuthDiag is showing that:
Now, so I tried "setspn -l idmiwa0112345678" (the 16-character computer name), and that said no such computer, so I tried "setspn -l idmiwa0112345678" (the 15-character computer name), and got:
HOST/idmiwa01xxxxxxx (HOST followed by slash followed by 15-character computer name)
NOTE that when I did the setspn -l on the computer name, it had "idmiwa01xxxxxxx.xxx.xxx" but with "HOST/" in front of it and when I did setspn -l on the domain\iwaservice USER, I also got "idmiwa01xxxxxxx.xxx.xxx", but with "HTTP/" in front of it.
Does that (having the same FQDN twice in SPNs but with different prefixes (HTTP/ vs. HOST/) count as a duplicate SPN???
Should I remove the "HOST/idmiwa01xxxxxxx.xxx.xxx" SPN from the computer object?
If anyone knows, please let me know!!
Jul 01, 2010 11:48 PM|jimcpl|LINK
BTW, I did a test today, where I "isolated" the test configuration to just a single Windows machine/IIS server, i.e., instead of pointing Firefox to the iwa.xxx.xxx hostname, I went directly to one of the IIS servers, idmiwa01xxxxxxx.xxx.xx.
On that IIS server, I have a resource at /simpletestiwaprotected/default.asp, which I configured for Integrated Windows Authentication.
When I did that with Firefox, I got the popup login window, and tried to login, but the popup window re-appeared.
So, it seems like just plain vanilla IWA (minus the load balancing) is, for some reason, not working.
This idmiwaxxxxxxx.xxx.xxx machine was the one that I had run the AuthDiag tool on, and where I got that message about the SPN for "domain\iwatestuser1" not being found in AD.
I don't know if any of this helps :(...
The next thing I want to try is the same isolated/standalone IIS server, but I'm going to switch the App pool identity back to Network Service account, to see if that eliminates the problem.
I think that if it does eliminate the problem then that probably indicates that there may be a problem with the domain user account (domain\iwaservice) that we tried to use for the App pool identity. Does that sound right?
Jul 02, 2010 12:49 AM|jimcpl|LINK
Re. the domain user that we've been using for the app pool identity:
I've encountered info like on this page:
several times. In particular, where it says to give the new domain user the following privileges in Local Security Policy:
- Adjust memory quotas for a process
- Logon as a service
- Replace a process level token
I had checked on that for the domain\iwaservice account we had, and noted that the domain\iwaservice account did NOT have those privileges.
Is it necessary to explicitly give the domain\iwaservice account those local privileges in order for IWA to work? Could that be why the IWA manual logins aren't working?
Jul 02, 2010 04:18 PM|jimcpl|LINK
To followup on my last post: I just tried to switch the app pool identity back to Network Service account. After I did that, IWA stopped working completely, both automatic and manual logins failed.
Now I am really confused :(...
Jul 09, 2010 12:15 AM|jimcpl|LINK
For the record, and in case anyone ever runs acorss this in the future, I've just been able to replicate the problem that I was having (Firefox automatic authentication works, Firefox manual authentication fails).
To replicate the problem, I had to stand up a small test configuration, with a domain controller, a server with IIS, and another machine with Firefox. All 3 machines are WIn2K3 Server.
From what I can tell, depending on how the Local Security Policy on the client machine is setup (Network Security/LMCompatibility), Firefox will, in some combination of the LMCompatibility setting, send a different Type 3 NTLM message when it is configured
for automatic NTLM login vs. when it is not configured for automatic NTLM login (ntlm trusted-uris in about:config).
Then, depending on how the Default Domain Controller Policy (not the GPO) is setup, the Firefox automatic login will still work, but Firefox manual login will fail.
The key to me finding this was looking at sniffer logs. When Firefox is setup for automatic login, the "Authorize: NTLM" header is much longer than when firefox is not setup for automatic NTLM login. As I understand it, with NTLM, the IIS server will do
NTLM pass-through to the DC, i.e., it passes the info from the "Authorize: NTLM" header through to the DC for authentication, so that's probably why the tests with the shorter "Authorize: NTLM" headers failed, whereas the tests with the longer "Authorize:
NTLM" headers worked.
I didn't try to find ALL combinations of policies that would cause this, but the test that I did was enough to prove that this was the problem, to me at least.
I hope that this info helps someone who runs across this in the future, because it's been several weeks of "pulling my hair out" on this one :(...