Thread: Are child requests supported in Integrated Pipeline Mode?

Last post 05-06-2009 7:48 PM by mbaocha. 14 replies.

Average Rating Rate It (5)Thank you for the rating!

RSS

Page 1 of 1 (15 items)

Sort Posts:

07-11-2007, 2:28 PM

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Hi,

The HttpWorkerRequest documentation states:

... your code could create a derived class for the purpose of child-request execution within a given application in order to pass its instance to ProcessRequest. In this case, the derived class should keep a reference to the current HttpWorkerRequest and delegate most of the methods to it.

My NeatUpload component does exactly this and works fine in Classic Pipeline Mode under IIS7. However, in Integrated Pipeline Mode the child request seems to silently fail. I'm logging calls to all of the overrideable .NET 1.1 methods of HttpWorkerRequest and I see the following sequence of calls:

GetFilePath()
SetEndOfSendNotification()
GetHttpVerbName()
GetQueryStringRawBytes()
GetQueryString()
GetServerName()
GetProtocol()
GetLocalPort()
GetUriPath()
GetRemoteAddress()
EndOfRequest()

Note that SendStatus() is never called and neither SendResponseFromFile() nor SendResponseFromMemory() is called. That combined with what I read in an earlier thread on this forum make me think that child requests are not supported in Integrated Pipeline Mode. Is that true?

NeatUpload uses child requests in 2 different ways. First, to extract uploaded files from the entity body a BeginRequest handler 1) creates a child request that delegates most methods to the original request but overrides methods like ReadEntityBody() to do the file extraction, and 2) passes the child request to HttpRuntime.ProcessRequest() and calls CompleteRequest() on the original request. Second, to make the progress bar work in a web garden/farm environment, the upload status is stored in the session and needs to be updated periodically during the upload. Since we don't want to lock the session for the entire upload request, child requests are used to update the status in the session.

If child requests are not supported in Integrated Pipeline Mode, does anyone have any recommendations on alternative ways to manipulate the entity body or update the session without locking it for the entire request? The only possibility I've found for manipulating the entity body is using IHttpRequest::ReadEntityBody and IHttpRequest:SetEntityBody from a native module. The only possibility I've found for updating the session without holding a lock on it involves using methods of ASP.NET 2.0's SessionStateProviderBase. I haven't tried either, so if there are better alternatives or I'm on the wrong track, please let me know.

Thanks,

Dean

07-11-2007, 3:19 PM In reply to

mvolo
Joined on 09-17-2003, 1:48 PM
Philadelphia, PA
Posts 584

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Dean,

You are right - unfortunately, the ProcessRequest() call is not supported in Integrated mode. The Integrated mode uses a completely different mechanism for driving request processing (using the IIS integration engine), and so it doesnt support requests for Classic request processing via ProcessRequest.

We missed this one in the list of breaking changes documented here: http://www.iis.net/articles/view.aspx/IIS7/Hosting-Web-Applications/ASP-NET/Upgrading-ASP-NET-1-1-to-IIS7-on-Windows-Vista---L?Page=3.

To access the incoming request entity, you can set a custom Stream implementation on context.Request.Filter. Then, whenever entity is asked for by page/modules, your filter will have the opportunity to see / modify it.

You can also do it in native code using a module that subscribes to the ReadEntity notification.

In terms of updating the session, you may be able to implement it by writing a wrapper provider on top of the existing provider session state providers, but it may be too much work to do for this case.

You can consider creating an application-global hashtable that stores upload state, keyed off by session id or another identifier you store in the session. This will alleviate the need to make separate child requests in order to modify the upload state in the session.

Hope this helps.

This posting is provided "AS IS" with no warranties, and confers no rights.

07-11-2007, 4:13 PM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

mvolo:

You are right - unfortunately, the ProcessRequest() call is not supported in Integrated mode.

Thanks for the quick reply, even if it isn't what I wanted to hear...

mvolo:
To access the incoming request entity, you can set a custom Stream implementation on context.Request.Filter. Then, whenever entity is asked for by page/modules, your filter will have the opportunity to see / modify it.

A while back I read that accessing Request.Filter, Request.InputStream, and Request.BinaryRead will block until the entire request has been read. That would prevent updating the upload status while the upload is being received. Has that behavior changed? FWIW, all the other upload components I'm aware of either use an ISAPI filter or use reflection to modify non-public fields of the underlying HttpWorkerRequest. I'm not aware of anyone using Request.Filter.

mvolo:
In terms of updating the session, ... consider creating an application-global hashtable that stores upload state, keyed off by session id or another identifier you store in the session. This will alleviate the need to make separate child requests in order to modify the upload state in the session.

By "application-global", do you mean the HttpApplicationState? If so, won't that prevent the upload state from being shared across a web garden/farm? Note that web garden/farm support is important because the browser will be updating the progress bar via a separate request, which could go to a different physical server in the farm.

Thanks again for the help,

--Dean

07-12-2007, 1:00 AM In reply to

mvolo
Joined on 09-17-2003, 1:48 PM
Philadelphia, PA
Posts 584

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

You are right on the money.

2) Request filter will block the original request / tie up a thread. But, it wont prevent you from updating the state inside the filter, as you are reading the request entity in chunks.

Unfortunately I cannot recommend anything that requires private reflection of our classes :(

3) Yes, it would be scoped to the appdomain and not work for a webfarm. You could use a signal file on a SMB share that is updated by the upload code, although that would require additional setup on a web farm beyond an out of proc session state provider that you use today ...

There is another option - you could make a child request using the new Integrated-mode only TransferRequest(), but that would break out of your filter's processing loop and short circuit the parent request - its intended for request redirection, not really child requests. If we provided a TransferRequest() option to make a child request without breaking out / shortcutting the pipeline, then you could accomplish child requests. If other solutions are not adequate, we can try to push for a QFE to do this.

Thanks,

Mike

This posting is provided "AS IS" with no warranties, and confers no rights.

07-12-2007, 5:29 AM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

mvolo:
2) Request filter will block the original request / tie up a thread. But, it wont prevent you from updating the state inside the filter, as you are reading the request entity in chunks.

Actually, the blocking might not be a problem after all. I can read the entire entity body from the HttpWorkerRequest (updating upload status as I go), filter out the uploaded files, then wrap the remainder of the request in a Stream, and then set that stream as the HttpContext.Request.Filter as the last step. At that point the entire request has been received and my code has done it's work, so would be well.

Unfortunately, when I try to set HttpContext.Request.Filter, I'm getting the following exception:

System.Web.HttpException: The request filter is not valid.

I've tried using a writeable MemoryStream, an unwriteable MemoryStream, and a subclass of MemoryStream that is neither writeable nor seekable. I always get the same error. I can't find any documentation on what is required for a request filter to be "valid". Any ideas?

mvolo:

you could make a child request using the new Integrated-mode only TransferRequest(), but that would break out of your filter's processing loop and short circuit the parent request - its intended for request redirection, not really child requests. If we provided a TransferRequest() option to make a child request without breaking out / shortcutting the pipeline, then you could accomplish child requests. If other solutions are not adequate, we can try to push for a QFE to do this.

I can't find any documentation on TransferRequest() beyond the method signatures, but if the child request was guaranteed to go to the same app domain as the parent and I could add an ID to the child request (e.g. in the headers or query string), then I could pass the upload status update from the parent to the child via the HttpApplicationState. That would certainly be easier than trying to wrap the session state provider.

Thanks again for the help,

--Dean

07-12-2007, 10:32 AM In reply to

mvolo
Joined on 09-17-2003, 1:48 PM
Philadelphia, PA
Posts 584

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

The filter is a chain, which always has to start with the filter currently installed.

So, basically, something like:
Request.Filter = new MyCustomFilter(Request.Filter);

Later, when your filter is asked for data, it will have to read it from the existing filter stream (which is just a wrapper over the entity body being preloaded). If you dont ask for Request.Filter previously to setting it, you'll get that exception.

Just a note on TRQ - it will throw a ThreadAbortException when called. That's what I was alluding to earlier - its not well suited for making multiple child requests at the same parent level. Even if you catch it / cancel the thread abort, the child request wont work until you actually surrender control and let the processing return back to IIS.

Thanks,

Mike

This posting is provided "AS IS" with no warranties, and confers no rights.

07-12-2007, 3:15 PM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

mvolo:
The filter is a chain, which always has to start with the filter currently installed.

So, basically, something like:
Request.Filter = new MyCustomFilter(Request.Filter);

Later, when your filter is asked for data, it will have to read it from the existing filter stream (which is just a wrapper over the entity body being preloaded). If you dont ask for Request.Filter previously to setting it, you'll get that exception.

Thanks for the explanation. Unfortunately, as soon as I ask for Request.Filter, ASP.NET seems to read the entire request from the HttpWorkerRequest and parse the form fields. It won't return control to my code until it has done that. As a result I don't see any way to use Request.Filter for my purposes. The closest I can come is to do something like:

byte[] filteredEntityBody = GetFilteredEntityBody(workerRequest); // Extracts files and updates upload status
HttpContext.Current.Request.Filter
= new BodyReplacementFilter(filteredEntityBody, HttpContext.Current.Request.Filter /* ignored */);

Unfortunately, that causes ASP.NET to think that there are no form fields in the request. Presumably, when I get Request.Filter ASP.NET is trying to read the entity body from the worker request again and parse the form fields out of it. Since I've already read it, ASP.NET gets nothing. This means that there is no way to use Request.Filter to modify what ASP.NET sees. It can only be used to modify what Request.InputStream returns, which is much less useful. Is that restriction really necessary? Couldn't ASP.NET parse the form fields out of the filtered InputStream instead?

Anyway, unless you have a better idea, it looks like I'm back to needing a native module to do what I want to do.

mvolo:

Just a note on TRQ - it will throw a ThreadAbortException when called. That's what I was alluding to earlier - its not well suited for making multiple child requests at the same parent level. Even if you catch it / cancel the thread abort, the child request wont work until you actually surrender control and let the processing return back to IIS.

Understood. I was referring to the possible QFE'd version which I thought you were proposing would not do that. Just to be clear, in order for the fix to be useful to me it, there would be a way to call TransferRequest such that it:

Created and ran a child request in parallel with the current request
Allowed me to pass an ID in the child request (eg. in the query string or header)
Ran the child request in the same app domain as the parent request

What are the chances for such a fix?

Thanks again for the help,

--Dean

07-12-2007, 4:48 PM In reply to

mvolo
Joined on 09-17-2003, 1:48 PM
Philadelphia, PA
Posts 584

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Strange - I do not believe asking for the Request.Filter is expected to download the entire entity. Be sure you are asking for Request.Filter, and not Request.InputStream (that one will pre-read). From what I remember, your filter will get a crack at the request entity after its all been preloaded, but before form fields are parsed / the entity is interpreted.

You of course also have to set the filter before any access to Request.Params or Request.Form is made, so that the filter is installed before ASP.NET tries to parse the entity body.

Can you show me a snapshot of code that reproes form parsing before your filter gets to run?

Thanks,

Mike

This posting is provided "AS IS" with no warranties, and confers no rights.

07-12-2007, 11:01 PM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

So it turns out that although asking for Request.Filter will not preload the entire entity body, the entire entity body is preloaded before any filter I install can be run. It seems like whenever anything that requires the entity body is needed, the entire entity body is preloaded and then it is passed through the filters.

Below is some code that reproduces the problem. Just use Default.aspx to upload a large file to your server (large enough that it takes a few seconds to upload). The log that is displayed will look something like this:

7/12/2007 5:25:58 PM: Entered Application_BeginRequest
7/12/2007 5:25:58 PM: Getting req.Filter
7/12/2007 5:25:58 PM: Got req.Filter
7/12/2007 5:25:58 PM: Setting req.Filter
7/12/2007 5:25:58 PM: Set req.Filter
7/12/2007 5:26:04 PM: Calling Source.Read(..., 8192)
7/12/2007 5:26:04 PM:   returned 0
...
7/12/2007 5:26:04 PM: Calling Source.Read(..., 8192)
7/12/2007 5:26:04 PM:   returned 0

Note the gap between:

7/12/2007 5:25:58 PM: Set req.Filter

and

7/12/2007 5:26:04 PM: Calling Source.Read(..., 8192)

That's when the upload is actually being received.

Global.asax:

<%@ Application %>
<%@ Import namespace="System.IO" %>

<script runat="server" language="c#">
    public class LoggingFilter : Stream
    {
        public LoggingFilter(Stream source)
        {
            Source = source;
        }

        public static void LogWithTime(string msg)
        {
            string log = HttpContext.Current.Application["TestLog"] as string;
            log = log + System.DateTime.Now + ": " + msg + "\n";
            HttpContext.Current.Application["TestLog"] = log;
        }

        public override int Read(byte[] buf, int offset, int count)
        {
            LogWithTime("Calling Source.Read(..., " + count + ")");
            int bytesRead = Source.Read(buf, offset, count);
            LogWithTime(" returned " + bytesRead);
            return bytesRead;
        }

        public override void Write(byte[] buf, int offset, int count)
        {
            throw new NotSupportedException();
        }

        public override bool CanRead { get { return true; } }
        public override bool CanSeek { get { return false; } }
        public override bool CanWrite { get { return false; } }

        public override long Length { get { return Source.Length; } }
        public override long Position { get { return Source.Position; } set { throw new NotSupportedException(); } }

        public override long Seek(long offset, SeekOrigin direction)
        {
            throw new NotSupportedException();
        }

        public override void SetLength(long length)
        {
            throw new NotSupportedException();
        }

        public override void Close()
        {
            Source.Close();
        }

        public override void Flush()
        {
            Source.Flush();
        }

        private Stream Source;
    }

    protected void Application_BeginRequest(Object sender, EventArgs e)
    {
        LoggingFilter.LogWithTime("Entered Application_BeginRequest");
        HttpRequest req = HttpContext.Current.Request;

        LoggingFilter.LogWithTime("Getting req.Filter");
        Stream origFilter = req.Filter;
        LoggingFilter.LogWithTime("Got req.Filter");

        LoggingFilter.LogWithTime("Setting req.Filter");
        req.Filter = new LoggingFilter(origFilter);
        LoggingFilter.LogWithTime("Set req.Filter");
    }

</script>

Default.aspx:

<%@ Page language="c#" AutoEventWireup="true" %>
<html>
   <head>
       <title>RequestFilterTest</title>
       <script language="c#" runat="server">
       void Button_Clicked(object sender, EventArgs e)
       {
           bodyPre.InnerText = Application["TestLog"] as string;
       }
       </script>
   </head>
   <body>
       <form id="uploadForm" runat="server">
           <input id="inputFile" runat="server" type="file" name="inputFile"/><br/>
           <asp:Button id="submitButton" runat="server" Text="Submit" OnClick="Button_Clicked"/><br/>
           <pre id="bodyPre" runat="server">
           </pre>
       </form>
   </body>
</html>

Any suggestions?

Thanks,

--Dean

07-13-2007, 1:20 PM In reply to

mvolo
Joined on 09-17-2003, 1:48 PM
Philadelphia, PA
Posts 584

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Yeah, I see. We preload the entity, then pass through your filter, then parse the form fields. So while you can extract it there, you cannot really be notified while its being preloaded, especially for a large upload using this approach.

I dont see a reason why the filter couldnt run during the preloading process, but this is not something we can change at this point in the product cycle ... may make a good feature request for the next release.

At this point, I dont see any other way other then private reflection (I havent thought about exactly how), or going with a native module. Let me investigate if restoring ProcessRequest's functionality is something that is feasible, and get back to you.

Mike

This posting is provided "AS IS" with no warranties, and confers no rights.

07-13-2007, 2:08 PM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

mvolo:

I dont see a reason why the filter couldnt run during the preloading process, but this is not something we can change at this point in the product cycle ... may make a good feature request for the next release.

Is there anything I need to do to formally make that feature request?

FYI, being able to use Request.Filter would have the additional advantage of working in a medium trust environment. Right now, upload components need to get the current HttpWorkerRequest but that is not permitted under medium trust. The only workaround is to install in the GAC.

mvolo:

At this point, I dont see any other way other then private reflection (I havent thought about exactly how),

Here is an example (from here):

        private void PushRequestToIIS(HttpWorkerRequest request, 
            byte[] textParts)
        {
            BindingFlags bindingFlags = BindingFlags.Instance 
                      | BindingFlags.NonPublic; 
            Type type = request.GetType();
            while ((type != null) && (type.FullName != 
                 "System.Web.Hosting.ISAPIWorkerRequest"))
                type = type.BaseType; 

            if (type != null)
            {
                type.GetField("_contentAvailLength", 
                  bindingFlags).SetValue(request, textParts.Length); 
                type.GetField("_contentTotalLength", 
                  bindingFlags).SetValue(request, textParts.Length);
                type.GetField("_preloadedContent", 
                  bindingFlags).SetValue(request, textParts); 
                type.GetField("_preloadedContentRead", 
                  bindingFlags).SetValue(request, true);
            }
        }

Actually, this points to a fix that you might be more willing to make at this stage. Can you add:

    public virtual void SetEntityBody(byte[] body)
    {
        throw new NotSupportedException();
   }

to HttpWorkerRequest and then override that in ISAPIWorkerRequest to do what you see happening via private reflection above?

mvolo:

or going with a native module.

Correct me if I'm wrong, but wouldn't such a module need to be installed at the server level, instead of the website level? If so, it's probably a non-starter for me, since many of my users are using shared hosting. Not to mention the fact that it would require a major rewrite of my code and maintaining multiple versions... :)

mvolo:

Let me investigate if restoring ProcessRequest's functionality is something that is feasible, and get back to you.

Thanks!

--Dean

11-02-2007, 11:48 PM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Any news on this?

FWIW, I just created a bug report on connect.microsoft.com for this issue so that it doesn't get forgotten:

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=308352

--Dean

11-24-2007, 5:32 AM In reply to

jlchereau
Joined on 03-21-2006, 4:07 PM
Posts 1

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Hello Dean,

Maybe you will want to look at the source code published at http://www.codeplex.com/VelodocXP.

Velodoc is a platform to publish, share, send and transfer large files. Velodoc XP is the open source core. It includes an Http module for uploads and server controls based on ASP.NET Ajax Extensions 1.0.

It can be tried online at http://www.velodoc.net.

Regards,

Jack.

Tags: Ajax IIS 7.0 Integrated Pipeline Mode C#ASP.NET Upload Http Module

11-24-2007, 11:52 AM In reply to

brettle
Joined on 08-30-2005, 11:04 PM
Redwood City, CA
Posts 8

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Hi Jack,

Thanks for the links (and for making your code open source!). Unfortunately, like all other upload modules I'm aware of, Velodoc's UploadHttpModule (specifically UploadHttpModule.RedirectFilteredRequest()) handles uploads by using reflection to change private member variables of the underlying HttpWorkerRequest or HttpRequest. As Mike said above, "I cannot recommend anything that requires private reflection of [MS's] classes". Since those fields are not part of the API, they could theoretically change at anytime, even in a security update. Also, they are only available on particular hosting platforms. For example, Velodoc's reflection code handles Integrated Pipeline Mode as a special case.

BTW, I'm not trying to put down Velodoc. It looks like it supports Integrated Pipeline Mode in the only way that is currently possible. My point is that such an implementation should not be necessary, and would not be necessary if MS supported child requests in Integrated Pipeline Mode.

--Dean

05-06-2009, 7:48 PM In reply to

mbaocha
Joined on 04-21-2009, 8:38 PM
Posts 35

Re: Are child requests supported in Integrated Pipeline Mode?

Reply Contact

Oh jack, i'm greatful. The link was quite helpful.

____________________________________________

Cheap Affordable Web Hosting | Windows Linux PHP ASPX MYSQL Website Hosting | Best Web Design Company

Page 1 of 1 (15 items)