Are child requests supported in Integrated Pipeline Mode? : The Official Microsoft IIS Forums

Reply

brettle

Are child requests supported in Integrated Pipeline Mode?

Jul 11, 2007 06:28 PM|LINK

Hi,

The HttpWorkerRequest documentation states:

... your code could create a derived class for the purpose of child-request execution within a given application in order to pass its instance to ProcessRequest. In this case, the derived class should keep a reference to the current HttpWorkerRequest and delegate most of the methods to it.

My NeatUpload component does exactly this and works fine in Classic Pipeline Mode under IIS7. However, in Integrated Pipeline Mode the child request seems to silently fail. I'm logging calls to all of the overrideable .NET 1.1 methods of HttpWorkerRequest and I see the following sequence of calls:

GetFilePath()
SetEndOfSendNotification()
GetHttpVerbName()
GetQueryStringRawBytes()
GetQueryString()
GetServerName()
GetProtocol()
GetLocalPort()
GetUriPath()
GetRemoteAddress()
EndOfRequest()

Note that SendStatus() is never called and neither SendResponseFromFile() nor SendResponseFromMemory() is called. That combined with what I read in an earlier thread on this forum make me think that child requests are not supported in Integrated Pipeline Mode. Is that true?

NeatUpload uses child requests in 2 different ways. First, to extract uploaded files from the entity body a BeginRequest handler 1) creates a child request that delegates most methods to the original request but overrides methods like ReadEntityBody() to do the file extraction, and 2) passes the child request to HttpRuntime.ProcessRequest() and calls CompleteRequest() on the original request. Second, to make the progress bar work in a web garden/farm environment, the upload status is stored in the session and needs to be updated periodically during the upload. Since we don't want to lock the session for the entire upload request, child requests are used to update the status in the session.

If child requests are not supported in Integrated Pipeline Mode, does anyone have any recommendations on alternative ways to manipulate the entity body or update the session without locking it for the entire request? The only possibility I've found for manipulating the entity body is using IHttpRequest::ReadEntityBody and IHttpRequest:SetEntityBody from a native module. The only possibility I've found for updating the session without holding a lock on it involves using methods of ASP.NET 2.0's SessionStateProviderBase. I haven't tried either, so if there are better alternatives or I'm on the wrong track, please let me know.

Thanks,

Dean

Reply

mvolo

629 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 11, 2007 07:19 PM|LINK

Dean,

You are right - unfortunately, the ProcessRequest() call is not supported in Integrated mode. The Integrated mode uses a completely different mechanism for driving request processing (using the IIS integration engine), and so it doesnt support requests for Classic request processing via ProcessRequest.

We missed this one in the list of breaking changes documented here: http://www.iis.net/articles/view.aspx/IIS7/Hosting-Web-Applications/ASP-NET/Upgrading-ASP-NET-1-1-to-IIS7-on-Windows-Vista---L?Page=3.

To access the incoming request entity, you can set a custom Stream implementation on context.Request.Filter. Then, whenever entity is asked for by page/modules, your filter will have the opportunity to see / modify it.

You can also do it in native code using a module that subscribes to the ReadEntity notification.

In terms of updating the session, you may be able to implement it by writing a wrapper provider on top of the existing provider session state providers, but it may be too much work to do for this case.

You can consider creating an application-global hashtable that stores upload state, keyed off by session id or another identifier you store in the session. This will alleviate the need to make separate child requests in order to modify the upload state in the session.

Hope this helps.

Mike Volodarsky
CTO at LeanSentry
Former IIS/ASP.NET PM

Want to become an expert at monitoring and troubleshooting your IIS applications?
See the demo at www.leansentry.com!

Marked as answer by Zhao Ji Ma - MSFT on Jul 17, 2007 09:13 AM

Reply

brettle

8 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 11, 2007 08:13 PM|LINK

mvolo

You are right - unfortunately, the ProcessRequest() call is not supported in Integrated mode.

Thanks for the quick reply, even if it isn't what I wanted to hear...

mvolo

To access the incoming request entity, you can set a custom Stream implementation on context.Request.Filter. Then, whenever entity is asked for by page/modules, your filter will have the opportunity to see / modify it.

A while back I read that accessing Request.Filter, Request.InputStream, and Request.BinaryRead will block until the entire request has been read. That would prevent updating the upload status while the upload is being received. Has that behavior changed? FWIW, all the other upload components I'm aware of either use an ISAPI filter or use reflection to modify non-public fields of the underlying HttpWorkerRequest. I'm not aware of anyone using Request.Filter.

mvolo

In terms of updating the session, ... consider creating an application-global hashtable that stores upload state, keyed off by session id or another identifier you store in the session. This will alleviate the need to make separate child requests in order to modify the upload state in the session.

By "application-global", do you mean the HttpApplicationState? If so, won't that prevent the upload state from being shared across a web garden/farm? Note that web garden/farm support is important because the browser will be updating the progress bar via a separate request, which could go to a different physical server in the farm.

Thanks again for the help,

--Dean

Reply

mvolo

629 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 12, 2007 05:00 AM|LINK

You are right on the money.

2) Request filter will block the original request / tie up a thread. But, it wont prevent you from updating the state inside the filter, as you are reading the request entity in chunks.

Unfortunately I cannot recommend anything that requires private reflection of our classes :(

3) Yes, it would be scoped to the appdomain and not work for a webfarm. You could use a signal file on a SMB share that is updated by the upload code, although that would require additional setup on a web farm beyond an out of proc session state provider that you use today ...

There is another option - you could make a child request using the new Integrated-mode only TransferRequest(), but that would break out of your filter's processing loop and short circuit the parent request - its intended for request redirection, not really child requests. If we provided a TransferRequest() option to make a child request without breaking out / shortcutting the pipeline, then you could accomplish child requests. If other solutions are not adequate, we can try to push for a QFE to do this.

Thanks,

Mike

Mike Volodarsky
CTO at LeanSentry
Former IIS/ASP.NET PM

Want to become an expert at monitoring and troubleshooting your IIS applications?
See the demo at www.leansentry.com!

Reply

brettle

8 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 12, 2007 09:29 AM|LINK

mvolo

2) Request filter will block the original request / tie up a thread. But, it wont prevent you from updating the state inside the filter, as you are reading the request entity in chunks.

Actually, the blocking might not be a problem after all. I can read the entire entity body from the HttpWorkerRequest (updating upload status as I go), filter out the uploaded files, then wrap the remainder of the request in a Stream, and then set that stream as the HttpContext.Request.Filter as the last step. At that point the entire request has been received and my code has done it's work, so would be well.

Unfortunately, when I try to set HttpContext.Request.Filter, I'm getting the following exception:

System.Web.HttpException: The request filter is not valid.

I've tried using a writeable MemoryStream, an unwriteable MemoryStream, and a subclass of MemoryStream that is neither writeable nor seekable. I always get the same error. I can't find any documentation on what is required for a request filter to be "valid". Any ideas?

mvolo

you could make a child request using the new Integrated-mode only TransferRequest(), but that would break out of your filter's processing loop and short circuit the parent request - its intended for request redirection, not really child requests. If we provided a TransferRequest() option to make a child request without breaking out / shortcutting the pipeline, then you could accomplish child requests. If other solutions are not adequate, we can try to push for a QFE to do this.

I can't find any documentation on TransferRequest() beyond the method signatures, but if the child request was guaranteed to go to the same app domain as the parent and I could add an ID to the child request (e.g. in the headers or query string), then I could pass the upload status update from the parent to the child via the HttpApplicationState. That would certainly be easier than trying to wrap the session state provider.

Thanks again for the help,

--Dean

Reply

mvolo

629 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 12, 2007 02:32 PM|LINK

The filter is a chain, which always has to start with the filter currently installed.

So, basically, something like:
Request.Filter = new MyCustomFilter(Request.Filter);

Later, when your filter is asked for data, it will have to read it from the existing filter stream (which is just a wrapper over the entity body being preloaded). If you dont ask for Request.Filter previously to setting it, you'll get that exception.

Just a note on TRQ - it will throw a ThreadAbortException when called. That's what I was alluding to earlier - its not well suited for making multiple child requests at the same parent level. Even if you catch it / cancel the thread abort, the child request wont work until you actually surrender control and let the processing return back to IIS.

Thanks,

Mike

Mike Volodarsky
CTO at LeanSentry
Former IIS/ASP.NET PM

Want to become an expert at monitoring and troubleshooting your IIS applications?
See the demo at www.leansentry.com!

Reply

brettle

8 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 12, 2007 07:15 PM|LINK

mvolo

The filter is a chain, which always has to start with the filter currently installed.

So, basically, something like:
Request.Filter = new MyCustomFilter(Request.Filter);

Later, when your filter is asked for data, it will have to read it from the existing filter stream (which is just a wrapper over the entity body being preloaded). If you dont ask for Request.Filter previously to setting it, you'll get that exception.

Thanks for the explanation. Unfortunately, as soon as I ask for Request.Filter, ASP.NET seems to read the entire request from the HttpWorkerRequest and parse the form fields. It won't return control to my code until it has done that. As a result I don't see any way to use Request.Filter for my purposes. The closest I can come is to do something like:

byte[] filteredEntityBody = GetFilteredEntityBody(workerRequest); // Extracts files and updates upload status

HttpContext.Current.Request.Filter

= new BodyReplacementFilter(filteredEntityBody, HttpContext.Current.Request.Filter /* ignored */);

Unfortunately, that causes ASP.NET to think that there are no form fields in the request. Presumably, when I get Request.Filter ASP.NET is trying to read the entity body from the worker request again and parse the form fields out of it. Since I've already read it, ASP.NET gets nothing. This means that there is no way to use Request.Filter to modify what ASP.NET sees. It can only be used to modify what Request.InputStream returns, which is much less useful. Is that restriction really necessary? Couldn't ASP.NET parse the form fields out of the filtered InputStream instead?

Anyway, unless you have a better idea, it looks like I'm back to needing a native module to do what I want to do.

mvolo

Just a note on TRQ - it will throw a ThreadAbortException when called. That's what I was alluding to earlier - its not well suited for making multiple child requests at the same parent level. Even if you catch it / cancel the thread abort, the child request wont work until you actually surrender control and let the processing return back to IIS.

Understood. I was referring to the possible QFE'd version which I thought you were proposing would not do that. Just to be clear, in order for the fix to be useful to me it, there would be a way to call TransferRequest such that it:

Created and ran a child request in parallel with the current request
Allowed me to pass an ID in the child request (eg. in the query string or header)
Ran the child request in the same app domain as the parent request

What are the chances for such a fix?

Thanks again for the help,

--Dean

Reply

mvolo

629 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 12, 2007 08:48 PM|LINK

Strange - I do not believe asking for the Request.Filter is expected to download the entire entity. Be sure you are asking for Request.Filter, and not Request.InputStream (that one will pre-read). From what I remember, your filter will get a crack at the request entity after its all been preloaded, but before form fields are parsed / the entity is interpreted.

You of course also have to set the filter before any access to Request.Params or Request.Form is made, so that the filter is installed before ASP.NET tries to parse the entity body.

Can you show me a snapshot of code that reproes form parsing before your filter gets to run?

Thanks,

Mike

Mike Volodarsky
CTO at LeanSentry
Former IIS/ASP.NET PM

Want to become an expert at monitoring and troubleshooting your IIS applications?
See the demo at www.leansentry.com!

Reply

brettle

8 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 13, 2007 03:01 AM|LINK

So it turns out that although asking for Request.Filter will not preload the entire entity body, the entire entity body is preloaded before any filter I install can be run. It seems like whenever anything that requires the entity body is needed, the entire entity body is preloaded and then it is passed through the filters.

Below is some code that reproduces the problem. Just use Default.aspx to upload a large file to your server (large enough that it takes a few seconds to upload). The log that is displayed will look something like this:

7/12/2007 5:25:58 PM: Entered Application_BeginRequest
7/12/2007 5:25:58 PM: Getting req.Filter
7/12/2007 5:25:58 PM: Got req.Filter
7/12/2007 5:25:58 PM: Setting req.Filter
7/12/2007 5:25:58 PM: Set req.Filter
7/12/2007 5:26:04 PM: Calling Source.Read(..., 8192)
7/12/2007 5:26:04 PM:   returned 0
...
7/12/2007 5:26:04 PM: Calling Source.Read(..., 8192)
7/12/2007 5:26:04 PM:   returned 0

Note the gap between:

7/12/2007 5:25:58 PM: Set req.Filter

and

7/12/2007 5:26:04 PM: Calling Source.Read(..., 8192)

That's when the upload is actually being received.

Global.asax:

<%@ Application %>
<%@ Import namespace="System.IO" %>

<script runat="server" language="c#">
    public class LoggingFilter : Stream
    {
        public LoggingFilter(Stream source)
        {
            Source = source;
        }

        public static void LogWithTime(string msg)
        {
            string log = HttpContext.Current.Application["TestLog"] as string;
            log = log + System.DateTime.Now + ": " + msg + "\n";
            HttpContext.Current.Application["TestLog"] = log;
        }

        public override int Read(byte[] buf, int offset, int count)
        {
            LogWithTime("Calling Source.Read(..., " + count + ")");
            int bytesRead = Source.Read(buf, offset, count);
            LogWithTime(" returned " + bytesRead);
            return bytesRead;
        }

        public override void Write(byte[] buf, int offset, int count)
        {
            throw new NotSupportedException();
        }

        public override bool CanRead { get { return true; } }
        public override bool CanSeek { get { return false; } }
        public override bool CanWrite { get { return false; } }

        public override long Length { get { return Source.Length; } }
        public override long Position { get { return Source.Position; } set { throw new NotSupportedException(); } }

        public override long Seek(long offset, SeekOrigin direction)
        {
            throw new NotSupportedException();
        }

        public override void SetLength(long length)
        {
            throw new NotSupportedException();
        }

        public override void Close()
        {
            Source.Close();
        }

        public override void Flush()
        {
            Source.Flush();
        }

        private Stream Source;
    }

    protected void Application_BeginRequest(Object sender, EventArgs e)
    {
        LoggingFilter.LogWithTime("Entered Application_BeginRequest");
        HttpRequest req = HttpContext.Current.Request;

        LoggingFilter.LogWithTime("Getting req.Filter");
        Stream origFilter = req.Filter;
        LoggingFilter.LogWithTime("Got req.Filter");

        LoggingFilter.LogWithTime("Setting req.Filter");
        req.Filter = new LoggingFilter(origFilter);
        LoggingFilter.LogWithTime("Set req.Filter");
    }

</script>

Default.aspx:

<%@ Page language="c#" AutoEventWireup="true" %>
<html>
   <head>
       <title>RequestFilterTest</title>
       <script language="c#" runat="server">
       void Button_Clicked(object sender, EventArgs e)
       {
           bodyPre.InnerText = Application["TestLog"] as string;
       }
       </script>
   </head>
   <body>
       <form id="uploadForm" runat="server">
           <input id="inputFile" runat="server" type="file" name="inputFile"/><br/>
           <asp:Button id="submitButton" runat="server" Text="Submit" OnClick="Button_Clicked"/><br/>
           <pre id="bodyPre" runat="server">
           </pre>
       </form>
   </body>
</html>

Any suggestions?

Thanks,

--Dean

Reply

mvolo

629 Posts

Re: Are child requests supported in Integrated Pipeline Mode?

Jul 13, 2007 05:20 PM|LINK

Yeah, I see. We preload the entity, then pass through your filter, then parse the form fields. So while you can extract it there, you cannot really be notified while its being preloaded, especially for a large upload using this approach.

I dont see a reason why the filter couldnt run during the preloading process, but this is not something we can change at this point in the product cycle ... may make a good feature request for the next release.

At this point, I dont see any other way other then private reflection (I havent thought about exactly how), or going with a native module. Let me investigate if restoring ProcessRequest's functionality is something that is feasible, and get back to you.

Mike

Mike Volodarsky
CTO at LeanSentry
Former IIS/ASP.NET PM

Want to become an expert at monitoring and troubleshooting your IIS applications?
See the demo at www.leansentry.com!

Are child requests supported in Integrated Pipeline Mode? RSS

14 replies

brettle

Are child requests supported in Integrated Pipeline Mode?

mvolo

Re: Are child requests supported in Integrated Pipeline Mode?

brettle

Re: Are child requests supported in Integrated Pipeline Mode?

mvolo

mvolo

mvolo

mvolo

Re: Are child requests supported in Integrated Pipeline Mode?

brettle

Re: Are child requests supported in Integrated Pipeline Mode?

mvolo

mvolo

mvolo

Re: Are child requests supported in Integrated Pipeline Mode?

brettle

Re: Are child requests supported in Integrated Pipeline Mode?

mvolo

mvolo

mvolo

Re: Are child requests supported in Integrated Pipeline Mode?

brettle

Re: Are child requests supported in Integrated Pipeline Mode?

mvolo

Re: Are child requests supported in Integrated Pipeline Mode?