WebDav Large File Upload HangsRSS

43 replies

Last post Jan 17, 2013 05:51 PM by howardjnl

  • WebDav Large File Upload Hangs

    Nov 02, 2011 08:42 PM|howardjnl|LINK

    I am transferring large files (700MB and larger) from my Vista Pro x64 laptop to my Windows 7 x64 server using IIS 7 and WebDav over HTTPS. I've configured IIS 7 to allow larger file size (4GB). The 700MB transfer takes less than a minute to ALMOST complete, but consistently hangs at 0 seconds remaining and 12MB to 20MB of the file remaining to transfer; it will not complete. Suggestions?

    webdav large file

  • Re: WebDav Large File Upload Hangs

    Nov 03, 2011 12:46 PM|robmcm|LINK

    Based on the timing of one minute before you see the failure, it sounds to me like you're running into the SendReceiveTimeoutInSec threshold for the WebDAV redirector on your client computer, which is set to one minute by default. If you take a look at the "WebDAV Redirector Registry Settings" section of the following walkthrough, that contains the details for that timer:

    http://learn.iis.net/page.aspx/386/using-the-webdav-redirector/

    To make it easier to modify those settings, I wrote a simple user interface for the WebDAV redirector settings that I describe in the following blog post:

    http://blogs.msdn.com/b/robert_mcmurray/archive/2011/10/07/how-to-create-an-html-application-to-configure-your-webdav-redirector-settings.aspx

    Robert McMurray [MSFT]
  • Re: WebDav Large File Upload Hangs

    Nov 03, 2011 05:17 PM|howardjnl|LINK

    I set the SendReceiveTimeoutInSec threshold to 120 seconds on the Vista x64 client, but no change. I then set the FileSizeLimitInBytes up from 50MB to the maximum of 4294967295 bytes; still no change. Not sure how to detect the error or log the problem so as to look into this on either the client or server. I appreciate you taking the time to respond! John
  • Re: WebDav Large File Upload Hangs

    Nov 03, 2011 09:09 PM|robmcm|LINK

    Thanks, John.

    Did you restart the WebClient service on your Vista Computer after you made the registry changes? Those changes are only reflected after the service has been restarted.

    Robert McMurray [MSFT]
  • Re: WebDav Large File Upload Hangs

    Nov 04, 2011 01:08 PM|howardjnl|LINK

    Yes, I tried it before rebooting and after; no change. I notice that the entire 700MB file appears to actually transfer in it's entirety, I can access it on the server AFTER I reboot the Vista client system to clear the connection with IIS. The worker process stays engaged until I shutdown the client. Also interesting: if I try to delete the transferred file on the server from the client WebDav folder, Windows Explorer hangs and I can't clear it unless I crash my client Vista workstation. I can delete a 69MB file I tested with no problem, and transfer it fine, but the much larger file really screws things up.
  • Re: WebDav Large File Upload Hangs

    Nov 04, 2011 02:01 PM|robmcm|LINK

    Ah, then you're probably running into the Request Filtering limits. Here's how you locate and modify those settings:

    1. Open Internet Information Services (IIS) Manager.
    2. In the Connections pane, navitage to your connection, site, or application.
    3. In the Home pane, double-click Request Filtering.
    4. Click Edit Feature Settings... in the Actions pane.

    This is what the dialog looks like:

    You need to change the value of the Maximum allowed content length (Bytes) field.

    Robert McMurray [MSFT]
  • Re: WebDav Large File Upload Hangs

    Nov 05, 2011 01:12 AM|howardjnl|LINK

    At the site, I've already set the value to 4290000000, the maximum size.
  • Re: WebDav Large File Upload Hangs

    Nov 05, 2011 04:49 PM|howardjnl|LINK

    Update: I've tried setting up the app pool to use 5 work processes and then transfer and delete the large test file on the server. Both actions failed before, and both still fail. What's interesting is that 5 worker processed start up simultaneously and all fail to complete in both the transfer and the delete. Still working on this to determine what the precise error is...
  • Re: WebDav Large File Upload Hangs

    Nov 05, 2011 06:13 PM|howardjnl|LINK

    Found an error being logged: unable to access thumbs.db. Set IIS app pool to permit "seeing" hidden files, cleared up that problem. But, still no large file...
  • Re: WebDav Large File Upload Hangs

    Nov 10, 2011 03:53 PM|howardjnl|LINK

    Problem not solved yet, but in my research and testing I've discovered that I had to assign my directory pool identity Read access to each WebDav shared folder to clear up problem with having to connect twice to gain WebDav access.
  • Re: WebDav Large File Upload Hangs

    Nov 14, 2011 05:37 PM|howardjnl|LINK

    Additional info: when I right click on a file from my Vista client on a file in the WebDav folder from my Windows 7 server, the "clock" runs for a minute or more (dependent upon file size) before the clock ends and nothing apparent occurs. Then I right click again on the same file, and immediately get the pop up menu. I can then delete the file. If I do a delete from my Android WebDav client, it immediately deletes the file without pause.
  • Re: WebDav Large File Upload Hangs

    Nov 15, 2011 02:33 PM|howardjnl|LINK

    Hotfix 2593591 of Nov. 10, 2011: might this help with my problem? If so, I don't understand how to apply it. John
  • Re: WebDav Large File Upload Hangs

    Nov 15, 2011 02:48 PM|HCamper|LINK

    Hi,

    What is the Microsoft Support link or  Service Bulletin Article for this 2593591 ?

    Martin

     

     

    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: WebDav Large File Upload Hangs

    Nov 15, 2011 04:12 PM|howardjnl|LINK

    Martin, the link is http://support.microsoft.com/kb/2593591#top. John
  • Re: WebDav Large File Upload Hangs

    Nov 16, 2011 02:10 PM|robmcm|LINK

    Sorry for the delayed response, I was out of town for the past week. ;-]

    That hotfix comes into use when large numbers of properties are set on files on the server; this usually occurs when files are uploaded, and generally occurs with a custom WebDAV client. So it probably won't help your problem with large file uploads.

    I'm not 100% certain, but the delayed right-click when using the Windows WebDAV redirector might be caused by the file being downloaded in order to retrieve properties; I'll check on that.

    Robert McMurray [MSFT]
  • Re: WebDav Large File Upload Hangs

    Nov 16, 2011 02:26 PM|howardjnl|LINK

    Rob, wondering about that, too. Have seen references to thumbs.db and other things that might be used. As mentioned above, don't encounter this problem using my Android client. ALSO, I can upload AND download large files using my Android client.
  • Re: WebDav Large File Upload Hangs

    Nov 16, 2011 02:36 PM|howardjnl|LINK

    Additional Info: Just checked the failed request log on the server. Getting a 404 error relative to accessing a desktop.ini file in a subfolder of the folder I'm working in on the server. The file doesn't exist, and I don't understand why this request is checking in the sub folder either.
  • Re: WebDav Large File Upload Hangs

    Nov 16, 2011 02:59 PM|howardjnl|LINK

    Concerning the 404: I came across a document about Sharepoint discussing WebDav mechanism and it says this is part of the normal connection process, to check for a desktop.ini file, though it's not clear why is looking in a subfolder.
  • Re: WebDav Large File Upload Hangs

    Nov 17, 2011 05:17 PM|howardjnl|LINK

    Too involved to have provided this info: Smaller files, such as 90 MB, do NOT hang up when transferring, right clicking, etc. They just take 1-4 minutes to respond. The larger 700MB file does NOT pop up an error window on my Vista client, it just sets there in "clock spin". A "Cancel" of a transfer just sets there and the folder window goes "unresponsive". If I kill that windows process, Windows Explorer goes haywire and usually hangs up my client system and I'll have to reboot. If I kill the connection from the IIS server end by stopping the directory pool, the folder window on my client will finally finish its "Cancel" gracefully. John
  • Re: WebDav Large File Upload Hangs

    Nov 17, 2011 05:49 PM|howardjnl|LINK

    Additional info about stopping the directory pool: if I have right clicked on the large file in my client's folder view of the WebDav folder, the click just spins. Then when I stop the directory pool on the IIS server, on the client the pop up menu appears in response to the previous right click. If I select delete or properties, a window pops up to say the file is no longer there.
  • Re: WebDav Large File Upload Hangs

    Nov 20, 2011 04:37 PM|howardjnl|LINK

    Including capture of failed requests for 400-499 errors in my WebDav web site, I consistently get failed requests for the file thumbs.db since it is not there. This failed request shows no information in the beginning for username, type of authentication, etc. If I log onto to the target directory and select a medium or large icon view to generate a thumbs.db, I no longer get this error, and the username, authentication type, etc., do show up in the log. ALSO, some directories successfully complete building a thumbs.db, others seem to time out before all files show up with a medium or large icon. Number of files in the directory does not seem to matter as a directory of 30 files 200MB files completed fine but directories of files in the 600-700MB size range do not complete even with only two files in the directory. One file completes with an icon, the other does not. This occurs using either WebDav or plain Windows networking on the same IP Home sub-net. So, I deleted the thumbs.db and tried creating one again but now it won't create a new one. I cleared all queued requests in IIS, cleaned out all temp files for my user on the client system, and tried again. The failed request log then creates a series of logs failed requests, a 404 error looking for folder.jpg, a 404 error looking for folder,gif, a 404 error looking for desktop.ini, a 404 error looking for thumbs.db. I don't know yet if any of this has anything directly to do with the transfer of large files, but it is very strange. I'm really missing something here.
  • Re: WebDav Large File Upload Hangs

    Nov 21, 2011 01:38 AM|howardjnl|LINK

    Accessed the WebDav folder in question from a Windows 7 Pro client: right click on a large file responds immediately. Uploaded a 1.3 GB file without any registry changes to the Web Client. It did freeze at the end as before with the Vista client with a very small amount that supposedly remained to be transferred (39.8 MB in this case) about 60 seconds into the transfer. But the file was actually in the target directory and COMPLETE.
  • Re: WebDav Large File Upload Hangs

    Nov 21, 2011 08:08 PM|howardjnl|LINK

    Just tried Bitkniex WebDav client on my Vista laptop: works like a charm!
  • Re: WebDav Large File Upload Hangs

    Nov 25, 2011 07:29 PM|howardjnl|LINK

    I've tried several commonly suggested "fixes" I've come across in various Microsoft threads to get Windows Webdav to work satisfactorily. They've all failed. Here they are: 1. Remove server from Home Group (client wasn't in one) 2. Disable IPv6 3. Set NIC for: 100MB Full Duplex (No Auto) Disable Large Send Offload (IPv4) Disable Energy Star Disable Large Send Offload (IPv6) (didn't do, not on my NIC) 4.Disable Receive Side Scaling AND Disable Receive Window Auto-Tuning Level Open an elevated command prompt with administrator rights. Type the following commands and press Enter (the second command on RSS is optional): netsh interface tcp set global autotuninglevel=disabled netsh interface tcp set global rss=disabled 5. Set folder options on both server and client to NOT use thumb nails 6. Turn off Remote Differential Compression on Windows 7: (note: can cause some software to fail) Open Control Panel Click on Programs and Features In the Programs and Features window, click on Turn ON/OFF Windows Features When the list populates, click on Remote Differential Compression to deselect it Click OK Reboot See if helps, and if affects any of your software apps. 7. Turn on Indexing Service on both client and server. 8. Nothing helped, returned items 2 thru 7 to original settings.
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 08:08 AM|HCamper|LINK

    Hi,

    You might roll back from Hot Fixes and changes you have made up to this point.

    The registery key "NoRemoteRecursiveEvents registry setting" may not apply for Web Dav.

    Martin

     

     

    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 01:42 PM|howardjnl|LINK

    Martin, you may be in the ballpark, as the progress bar running through the file address bar at the top of the open Webdav'ed folder is suspicious. However, none of the hotfixes you mention "officially" pertain to Vista or Windows 7. Risk to my systems? John
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 01:47 PM|howardjnl|LINK

    BTW, all: Rob has some CAB files from me captured during my transfer problems that he's passed along to some Microsoft web client guys for analysis. John
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 02:47 PM|HCamper|LINK

    Hi,

    I hope this is the correct ballpark :D. 

    The reason for the links is the common information for the Registery changes.

    Do not request the "Hotfixes" or install them unless they are for the indicated Windows Systems.

    The only risk to your system(s) is incorrect Editing of the registery. Before using the Registery Editor use System Restore.

    Then also use export in the Registery Editor for the Key or Branach being changed.

    Save the current Branch and  Key Edit to the Registery Editor Favorites,

    Martin

     

     

    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 03:47 PM|howardjnl|LINK

    Got it. Tried it, and no change. As mentioned above, right-click takes a very long time to respond for a small file, and forever with a large file. Transfer is still interesting as before. For example: I drag a 32MB file into the Webdav folder view and the progress bar moves quickly to the right, then pauses for several minutes doing something. I say "something" because the file actually transferred quickly. If I break the transfer by restarting the app pool in IIS7 on the server, I can access the transferred file just fine from the server's desktop. BTW, if I try to break the session from the client, it won't respond, even if I stop the process from the Task Manager on the client, IIS7 on the server still hangs on to the file.
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 05:15 PM|howardjnl|LINK

    Martin, I've come across an article from May 2011 that described some registry settings to change/add that affect Windows Explorer performance. the four items included the one you mentioned above: Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\stisvc. In the details or right pane, change the value of "Start" to 4 (was 2) Create a new policy for Windows Explorer with a registry value, NoSaveSettings in HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies with 0 dword value data Create a new value, NoNetCrawling in HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Advanced registry key with 1 as dword value data Create a new value, NoRemoteRecursiveEvents in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\Explorer with 1 as dword value data Close the Windows Registry editor and then reboot computer and try using Windows Explorer again Result was to cause failure of Atheros wireless dll to not load, but the network seemed to work as before, but task bar icon erroneously indicated there was no connection. Returned above items to original settings, rebooted, all is back to original state.
  • Re: WebDav Large File Upload Hangs

    Nov 27, 2011 08:01 PM|HCamper|LINK

    Hi Howard,

    Looks like the wrong ball park. I was looking at the UI and Visual parts.

    Best to wait until the report for the failure tests is done.:D.

    Martin

     

     

    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: WebDav Large File Upload Hangs

    Dec 18, 2011 04:52 PM|howardjnl|LINK

    Still working with Rob and Tony at Microsoft, using Process Monitor and Fiddler to sort out what's going on. Delayed investigation by installing Verizon Media Manager on my server which caused a file locking problem, and is a bad product in many other ways. Got it out of the way and am back on track with investigation. While doing this, came across a big performance issue with Webdav access from my Vista client: simply opening a webdav shared folder from my Windows IIS7 server, and doing nothing else, causes a download of all files in the folder being viewed to transfer to the Webdav cache on my client to temp folder "C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore\Tfs_DAV\". Any sub folder I open also begins this transfer of files down to this folder on my client. Trying to find out more about this, why it does it and how it can be disabled and the repercussions if I do so. John
  • Re: WebDav Large File Upload Hangs

    Dec 23, 2011 08:21 AM|VitorArruda|LINK

    Good morning, I am having the same problem. Actually, the windows progress in a copy of a big file takes too much time to get done. It gets stuck when the progress is almost completed. In addition, I have the same problem yours when I right click a larger file. Did you solve your problem? If you need some test, or something I can help, please let me know. Thank you very Much, Vitor.
  • Re: WebDav Large File Upload Hangs

    Dec 23, 2011 04:50 PM|howardjnl|LINK

    Vitor, Here's what I've discovered so far with further testing (uploading a 720MB file to my Win7 IIS7 server) and help from Microsoft: 1. My WebDav client on Vista first copies the file locally to folder \Windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore\Tfs_DAV. Apparently, this is a cache for Webdav transactions. 2. While the file is copied into this cache folder, the green progress bar moves rapidly to the right, but not quite to the end. The copy status window says all but a few MB has completed the transfer. BUT it has not actually been transferred, it was just copied to the cache folder. 3. After a pause of about 10 seconds, the file actually begins to upload to my IIS7 server. The transfer of this 720MB file on my local in-home network takes about 30 minutes. 4. The copy now completes, the green status bar completes, and the copy status window closes. The file is now present on my server. 5. HOWEVER, some kind of synchronization process begins copying the file BACK to the cache folder on my Vista workstation. I'm working with Microsoft guys now to understand this "feature" and hopefully turn it "off". Also, when I right click on a file in the open WebDav folder on my client, the file I right click on is transferred to this cache folder so that my Vista client can determine the file's properties. It looks like WebDav file transactions are all handled through copies of files of files within the local cache. This is really horrible with large files. In this case, it takes about 30 minutes for the right-click pull down menu to appear. There is a hotfix at http://support.microsoft.com/kb/2593591 that may help with this behavior that is applied to the IIS7 server. I asked about it earlier in this thread. The gentleman at Microsoft I've been working with just mentioned it to me and asked if I've applied it. I have not, but he's out on vacation until Jan. 2. I'm going to do some more research on it and will probably try it out today. Hope this helps you. I'll report back on what I find out. Once this behavior is corrected, I'll get back to researching how I may improve the actual data transfer speed between systems. I was working on that, but this caching behavior is in the way of actually working on possible speed improvements. BTW, I downloaded and used Microsoft's Process Monitor to help watch all of this activity on my IIS7 server. John
  • Re: WebDav Large File Upload Hangs

    Dec 23, 2011 05:11 PM|howardjnl|LINK

    Vitor, Sorry, I made a mistake. I already installed the hotfix I mentioned BEFORE I did all of the testing. So, this hotfix has not helped. I won't hear from Microsoft until after Jan. 1. I'll update this thread if I learn anything new. John
  • Re: WebDav Large File Upload Hangs

    Dec 23, 2011 07:18 PM|howardjnl|LINK

    Quick update: I reversed the changes made for the hotfix to confirm it made no difference concerning this issue. I retested and yes, it did nothing to help this problem. John
  • Re: WebDav Large File Upload Hangs

    Dec 24, 2011 06:18 AM|VitorArruda|LINK

    Good morning John,

    Thanks for your feedback, very good your research. I will return of my vacation on Jan, so that I can take your information and mix with I already have. I have a ticket open with Microsoft as well, but it's recent.. they are still getting more information about the problem.

    If I discover something about this case, I let you know.

     Thanks again.

     Vitor.

  • Re: WebDav Large File Upload Hangs

    Dec 30, 2011 02:04 PM|howardjnl|LINK

    Vitor and Martin, The folks at Microsoft have been very helpful to me in sorting out what is going on. The Windows WebDav redirector, working with Windows Explorer, etc., uses the WebDav cache I mentioned above as an intermediate point in its mechanism to transfer, find properties of, delete, etc., files concerning the remote Webdav host. Files transferring to/from the remote host are always copied there. Properties, deletion, renaming, file preview and other such calls always copy the remote file to this cache to perform those activities on the local system. The explains the actions of the progress bar in moving files, the long delays in acting upon large files across the network. Since other webdav clients, such as Bitkinex and my android clients, do not perform some of these Windows specific tasks, such as previews, they respond almost immediately in renaming, deleting, and getting properties of remote files. All of these clients are similar in the actual transfer times of files. Hope this is useful to you guys. Now, back to understanding how the actual transfer times may be improved for Windows to Windows whether it be webdav redirector or Bitkinex based. John
  • Re: WebDav Large File Upload Hangs

    Jan 04, 2012 06:29 AM|TobiSa|LINK

    hey there

    maybe I have some information for you.

    System: Windows SBS 2011 Standard + IIS 7.5 hosted by an ISP  Client: Windows 7 Prof

    I'd the same problems like you. the data copying hangs up in the end of the copy process or it freez ... I could not copy a folder with 30mb or 40mb because the copy job broke down ...

    to copy files larger then 50mb I had set the param "Maximum allowed content length (Bytes)" up to 4gb inside the iis request filtering. after this configuration everething works well. I can copy small files, big files and folders with many many subfolders and a lot of files.

     best regards

    Tobi

  • Re: WebDav Large File Upload Hangs

    Jan 04, 2012 02:06 PM|howardjnl|LINK

    Tobi, i appreciate you joining in this thread. Yes, you'll note that early on in this thread I addressed this transfer file size issue. It is indeed a definite roadblock of not changed. It appears that there are two issues surrounding the apparent slowness and hang of large files. The progress bar at the top of the open WebDav window on the client is misleading. It zooms across rapidly while copying outgoing files into a local cache on the client, then, as the bar sets near the end, it actually performs the transfer. So it's not really a hang I've been experiencing, just a misleading "feature" of Windows. As for the speed of the actual transfer from client to server (or vice versa), a lot of folks are complaning about the slowness of Windows 7 and are suggesting all kinds of to improve this, but I've not been able to measure any improvement resulting from any of them. You can see some of these tweaks earlier in this thread and elsewhere. John
  • Re: WebDav Large File Upload Hangs

    Aug 26, 2012 06:12 AM|StuartDavey|LINK

    Hi John,

    I have the revision history for Webdav.dll on my blog site and this includes a link to the requestable Microsoft Hotfix 2503820 - FIX: Memory leak in the W3wp.exe process for a WebDAV website when many WebDAV clients upload files, download files, or access file directories in WebDAV 7.5

    http://www.aidinit.com/2012/06/%ef%bb%bf%ef%bb%bf%ef%bb%bfwebdav-dll-pre-service-pack-2-revision-history-for-windows-7-sp1-windows-server-2008-r2-sp1-and-windows-small-business-server-2011-sbs-2011/

     Perhaps this may to help address some of the performance issues you are facing with WebDAV

    Hope it helps 

    Stuart

  • Re: WebDav Large File Upload Hangs

    Jan 14, 2013 09:18 AM|darksim905|LINK

    Any updates to this thread? You've become a ghost howardjnl, and your threads and input have been highly useful in helping me understand some of the things that WebDAV is doing behind the shadows. Please don't be one of those people that said they solved their issue yet didn't come back to tell us what did the trick :)

  • Re: WebDav Large File Upload Hangs

    Jan 17, 2013 02:56 PM|JPalm|LINK

    ^Agreed. Experiencing the same issue. Tried the tweaks suggested here to no avail.

  • Re: WebDav Large File Upload Hangs

    Jan 17, 2013 05:51 PM|howardjnl|LINK

    Well, I finally determined two ways to address the problem:

    1. Make the registry size limit change recommended and use Bitkinex instead of Windows based WebDAV, OR

    2. Upgrade to Windows 8.  I took advantage of the $39.99 deal at MS to upgrade to Win 8 Pro and seems to have resolved my problems.  Go figure :-)