다음을 통해 공유

Uploading large file to IIS 7.5 or 8 using file input element

  • 아티클

I've recently worked on a very interesting file upload issue where my customer was hitting a 2 GB upload limit using Internet Explorer, IIS 7.5 and a simple file upload form with a file input element.
After doing some research and a couple of tests, I was able to build a simple "POC" project showing how to upload up to 4 GB using the following configuration:

  • Internet Explorer 10 client

  • Windows 8 / IIS 8

  • Application configured to run in .Net 4.5 Classic Pipeline

  • Web.Config configured as follows :

     

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.web>
<httpRuntime maxRequestLength="2147483647" />
<httpModules>
<add name="UploadModule" type="UploadModule,UploadModule"/>
</httpModules>
</system.web>
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="4294967295"/>
</requestFiltering>
</security>
</system.webServer>
</configuration>

 

  • simple upload.aspx test page :

     

<!DOCTYPE html>
<html lang="en" xmlns="https://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8" />
<title>Upload Test</title>
</head>
<body>
<form name="frm" action="upload.aspx" enctype="multipart/form-data" method="POST">
<h1>Choose file and click upload</h1>
<input type="file" id="SourceFile_1" name="SourceFile_1" size="40" />
<br />
<div>
<input type="submit" value="Upload" />
</div>
</form>
</body>
</html>

[private void Application_BeginRequest(object theSender, EventArgs theE)]

    {
HttpApplication httpApp = theSender as HttpApplication;
HttpContext context = ((HttpApplication)theSender).Context;
IServiceProvider provider = (IServiceProvider)context;
HttpWorkerRequest httpWorkerReq = (HttpWorkerRequest)provider.GetService(typeof(HttpWorkerRequest));
long receivedBytes=0;
long initialBytes=0;
byte[] buffer = new byte[10 * 1024 * 1024];

        if (httpApp.Request.HttpMethod == "POST")
{
// get the total body length
UInt32 requestLength = (UInt32) httpWorkerReq.GetTotalEntityBodyLength();
// Get the initial bytes loaded
if (httpWorkerReq.GetPreloadedEntityBody() != null)
receivedBytes = httpWorkerReq.GetPreloadedEntityBody().Length;
if (!httpWorkerReq.IsEntireEntityBodyIsPreloaded())
{
// Set the received bytes to initial bytes before start reading
do
{
// Read another set of bytes
initialBytes = httpWorkerReq.ReadEntityBody(buffer, buffer.Length);
// Update the received bytes
receivedBytes += initialBytes;
System.Diagnostics.Trace.WriteLine("#bytes read: " + receivedBytes.ToString());
}
while (initialBytes > 0);
}
System.Diagnostics.Trace.WriteLine("Request Length=" + requestLength.ToString() + " Total bytes read=" + receivedBytes.ToString());
}
}

 

If you use the above settings/pages and upload a large file (nearly 3GB in this example), you should see the following in DebugView :

 

I believe the 4 GB upload barrier using input type=file element is impossible to exceed for the following reasons :

  • there is a 4 GB upload limit in Internet Explorer: https://blogs.msdn.com/b/ieinternals/archive/2011/03/10/wininet-internet-explorer-file-download-and-upload-maximum-size-limits.aspx
    If you try to upload more than 4 GB with IE10 and above sample, IE will simply refuse to upload anything (you won't even see a POST request being sent!)

  • requestFiltering doesn't allow to specify more than 4 GB for maxAllowedContentLength

  • maxRequestLength is expressed in kilobytes and the limit specified above is nearly 2 TB. ASP.NET 2.0 doesn't allow a value greater than 2097151 KB (approx. 2 GB) and trying to set a greater value will fail with the following error :
    "The value for the property 'maxRequestLength' is not valid. The error is: The value must be inside the range 0-2097151"

  • If the application is running under the NET 4.5 Integrated Pipeline, upload will not work above 2G and the following error will be sent by IIS: "HTTP 400.0 – Bad Request ASP.NET detected invalid characters in the URL.".

    Debugging of the error a little bit further shows that its cause is the following stack and exception:

     

    0:034> !clrstack

    0000003a5f72e048 000007fb7e99811c [HelperMethodFrame: 0000003a5f72e048] System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32, IntPtr)
    0000003a5f72e130 000007fb58087ab1 System.Web.Hosting.IIS7WorkerRequest.ReadRequestBasics()
    0000003a5f72e1d0 000007fb5806ee45 System.Web.Hosting.PipelineRuntime.InitializeRequestContext(IntPtr, Int32, System.Web.
    Hosting.IIS7WorkerRequest ByRef, System.Web.HttpContext ByRef)
    0000003a5f72e240 000007fb5806e45f System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32)
    0000003a5f72e3d0 000007fb5806e2e2 System.Web.Hosting.PipelineRuntime.ProcessRequestNotification(IntPtr, IntPtr, IntPtr,Int32)
    0000003a5f72e420 000007fb587cb781 DomainNeutralILStubClass.IL_STUB_ReversePInvoke(Int64, Int64, Int64, Int32)
    0000003a5f72e668 000007fb66959863 [ContextTransitionFrame: 0000003a5f72e668]

    0:034> !dso

    0000003A5F72DEC0 0000003893849348 System.ArithmeticException

    When the integrated pipeline is used, we go though webengine code (webengine4!MgdGetRequestBasics) which doesn't support more than 2 GB content-length and a System.ArithmeticException exception is raised which subsequently cause the HTTP 400 error. With the classic pipeline, we don't use webengine4 but the old ASPNET_ISAPI model and we don't hit the above issue.

If you need to upload more than 4 GB (or 2 GB in case ASP.NET 4.5 integrated pipeline or ASP.NET 2.0), I believe you'll need to use specific client and server code in order to use chunked-encoding and read data using GetBufferlessInputStream.

To perform the upload tests, I've used a quite recent PC (4 processor machine with 16 GB of RAM, 1 GB/s Lan, SSD storage, all the tests were made locally). With such configuration, the upload of 3 GB takes less than 30 seconds. Beyond memory/hardware requirements and impacts (upload of very large file clearly puts a lot of "pressure" on the server machine), my test scenario was not really realistic. In a real world scenario, I should have probably to tweak some settings at various levels (IE, http.sys, IIS, asp.net, etc…etc). Therefore, even if it is technically possible to upload up to 4 GB using above scenario, you may want to consider other means to build a more "robust" upload (using range request for example or using other protocols (FTP, WEBDAV…etc)).

Happy Uploading!

Emmanuel Boersma

Comments

  • Anonymous
    April 07, 2014
    Hi, Thanks for your post. Can you post your upload module here, because I am getting error on context.Request.ContentLength of Upload Module. Thanks for your anticipation.

  • Anonymous
    May 01, 2014
    hey! is it right, that you can't upload > 2gb files with iis 7.5? i have to use iis 8. is that true? thanks in advance.

  • Anonymous
    June 29, 2014
    If you use the above code for file upload, the file being uploaded contains body headers as well. To remove the headers, you may be interested in this blog : www.adhocgeek.com/.../large-file-uploads