Share via

Part 2- demystifying the file upload process in asp.net

  • Article

This is a continuation of the blog post https://blogs.msdn.com/b/rohithrajan/archive/2013/01/22/large-file-uploads-in-asp-net-part-1.aspx

The default option for uploading files in asp.net is using Fileupload controls SaveAs method

https://msdn.microsoft.com/en-us/library/system.web.ui.webcontrols.fileupload.saveas.aspx

when we use the FileUpload control’s save as method,it is internally using a class HttpRawUploadedContent .This is in turn using TempFile helper class .This class creates a temp file for posted data. This temp file stores the uploaded content to a folder called “uploads” inside C:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files 

coming back to HttpRawUploadedContent class,this class contains the uploaded content in byte array field _data.This contains uploaded data (either all of it or part read from temp file)

Let’s have a look at the memory after the upload process has been finished(I have used windbg to look at this.For more details refer this ,this and this)

  • memory usage in IIS 6 +asp.net 2.0

Scenario Tested:

uploading a file of size 450 MB, maxRequestLength is set to 2097151(maximum allowed) , using fileupload control’s saveas method uploadcontrol.SaveAs

https://msdn.microsoft.com/en-us/library/system.web.ui.webcontrols.fileupload.saveas(v=VS.80).aspx

looking at the memory after the uploading a 431 Mb file using windbg

Number of GC Heaps: 1
generation 0 starts at 0x02671fd0
generation 1 starts at 0x02671e00
generation 2 starts at 0x025b1000
ephemeral segment allocation context: (0x02674b18, 0x02675fdc)
segment begin allocated size reserved
000ef780 7a721784 7a74248c 0x00020d08(134,408) 00001000
000e90a0 790d6358 790f5800 0x0001f4a8(128,168) 00001000
025b0000 025b1000 02675fdc 0x000c4fdc(806,876) 00bc2000
Large object heap starts at 0x035b1000
segment begin allocated size reserved
035b0000 035b1000 035ed0c0 0x0003c0c0(245,952) 00fbe000
04730000 04731000 10f31010 0x0c800010(209,715,216) 007fe000
11b70000 11b71000 1e371010 0x0c800010(209,715,216) 007fe000
Total Size 0x1914126c(420,745,836)
------------------------------
GC Heap Size 0x1914126c(420,745,836)

objects in the large heap

!dumpheap -min 85000

  Address MT Size Gen
0x035bd980 0x000dea98 172,976 3 Free
0x04731000 0x79124418 209,715,212 3 System.Byte[]
0x11b71000 0x79124418 209,715,212 3 System.Byte[]

main class used for uploading is System.Web.HttpRawUploadedContent .This class has a field named _data of type System.byte[] which holds the uploaded content in the memory.

!do 0x0265088c
Name: System.Web.HttpRawUploadedContent
MethodTable: 68a8ed20
EEClass: 68a8ecb0
Size: 40(0x28) bytes
GC Generation: 2
(C:\WINDOWS\assembly\GAC_32\System.Web\2.0.0.0__b03f5f7f11d50a3a\System.Web.dll)
Fields:
     MT Field Offset Type VT Attr Value Name
790fed1c 4000fa8 c System.Int32 1 instance 209715200 _fileThreshold
790fed1c 4000fa9 10 System.Int32 1 instance 452644402 _expectedLength
79104f64 4000faa 20 System.Boolean 1 instance 1 _completed
790fed1c 4000fab 14 System.Int32 1 instance 452644402 _length
79124418 4000fac 4 System.Byte[] 0 instance 04731000 _data
68a870e0 4000fad 8 ...dContent+TempFile 0 instance 026501c4 _file
790fed1c 4000fae 18 System.Int32 1 instance 452644196 _chunkOffset
790fed1c 4000faf 1c System.Int32 1 instance 206 _chunkLength
0:000> !do 04731000
Name: System.Byte[]
MethodTable: 79124418
EEClass: 791244d0
Size: 209715212(0xc80000c) bytes
GC Generation: 3
Array: Rank 1, Number of elements 209715200, Type Byte
Element Type: System.Byte
Fields:
None

There are two System.byte[] in the memory.And this is yet to be garbage collected.It will get garbage collected but if you are having lot of large file uploads ,it will definitely create problem in memory.lot of objects in large object heap creates heap fragmentation. This issue is fixed to some extent on .net framework 4.5.You can refer following blog posts for large object heap details

https://msdn.microsoft.com/en-us/magazine/cc534993.aspx

https://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/

  • IIS7+asp.net 3.5 on dual core machine 2 GB

Number of GC Heaps: 2
------------------------------
Heap 0 (01050d00)
generation 0 starts at 0x01ac0050
generation 1 starts at 0x01ac0044
generation 2 starts at 0x01ac0038
ephemeral segment allocation context: none
segment begin allocated size
01ac0000 01ac0038 01bf7024 0x00136fec(1273836)
Large object heap starts at 0x09ac0038
segment begin allocated size
09ac0000 09ac0038 09ad0b78 0x00010b40(68416)
Heap Size 0x147b2c(1342252)
------------------------------
Heap 1 (010519a0)
generation 0 starts at 0x05ac0050
generation 1 starts at 0x05ac0044
generation 2 starts at 0x05ac0038
ephemeral segment allocation context: none
segment begin allocated size
05ac0000 05ac0038 05be6ff4 0x00126fbc(1208252)
Large object heap starts at 0x0bac0038
segment begin allocated size
0bac0000 0bac0038 0bac0048 0x00000010(16)
Heap Size 0x126fcc(1208268)
------------------------------
GC Heap Size 0x26eaf8(2550520)

checking the large objects

!dumpheap -min 85000
------------------------------
Heap 0
Address MT Size
total 0 objects
------------------------------
Heap 1
Address MT Size
total 0 objects
------------------------------
total 0 objects
Statistics:
     MT Count TotalSize Class Name
Total 0 objects

!dumpheap -type System.Web.HttpRawUploadedContent
------------------------------
Heap 0
Address MT Size
01bdbe60 53c76b58 40     
01bf1ea0 53c6119c 20     
total 2 objects
------------------------------
Heap 1
Address MT Size
total 0 objects
------------------------------
total 2 objects
Statistics:
     MT Count TotalSize Class Name
53c6119c 1 20 System.Web.HttpRawUploadedContent+TempFile
53c76b58 1 40 System.Web.HttpRawUploadedContent
Total 2 objects

!do 01bdbe60
Name: System.Web.HttpRawUploadedContent
MethodTable: 53c76b58
EEClass: 53a7426c
Size: 40(0x28) bytes
(C:\Windows\assembly\GAC_32\System.Web\2.0.0.0__b03f5f7f11d50a3a\System.Web.dll)
Fields:
     MT Field Offset Type VT Attr Value Name
54fa2dbc 4001049 c System.Int32 1 instance 81920 _fileThreshold
54fa2dbc 400104a 10 System.Int32 1 instance 843205071 _expectedLength
54f7463c 400104b 20 System.Boolean 1 instance 1 _completed
54fa2dbc 400104c 14 System.Int32 1 instance 843205071 _length
54fa35e0 400104d 4 System.Byte[] 0 instance 01bdbe88 _data
53c6119c 400104e 8 ...dContent+TempFile 0 instance 01bf1ea0 _file
54fa2dbc 400104f 18 System.Int32 1 instance 96 _chunkOffset
54fa2dbc 4001050 1c System.Int32 1 instance 81920 _chunkLength

!do 01bdbe88
Name: System.Byte[]
MethodTable: 54fa35e0
EEClass: 54d5eb8c
Size: 81932(0x1400c) bytes
Array: Rank 1, Number of elements 81920, Type Byte
Element Type: System.Byte
Fields:
None

So on asp.net 3.5 +IIS7 ,we can see a better memory usage.But still one problem,the file gets uploaded to temp files first.

C:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\fileupload\c60361cf\38faed7d\uploads\aqiib-zc.post

!do 01bf1ea0
Name: System.Web.HttpRawUploadedContent+TempFile
MethodTable: 53c6119c
EEClass: 53a681b4
Size: 20(0x14) bytes
(C:\Windows\assembly\GAC_32\System.Web\2.0.0.0__b03f5f7f11d50a3a\System.Web.dll)
Fields:
     MT Field Offset Type VT Attr Value Name
54736e60 4001051 4 ...empFileCollection 0 instance 01bf2838 _tempFiles
54fa0b70 4001052 8 System.String 0 instance 01bf3414 _filename
54f9e86c 4001053 c System.IO.Stream 0 instance 01bf3524 _filestream

!do 01bf3414
Name: System.String
MethodTable: 54fa0b70
EEClass: 54d5d66c
Size: 258(0x102) bytes
(C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: C:\Windows\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\fileupload\c60361cf\38faed7d\uploads\aqiib-zc.post
Fields:
     MT Field Offset Type VT Attr Value Name
54fa2dbc 4000096 4 System.Int32 1 instance 121 m_arrayLength
54fa2dbc 4000097 8 System.Int32 1 instance 120 m_stringLength
54fa1850 4000098 c System.Char 1 instance 43 m_firstChar
54fa0b70 4000099 10 System.String 0 shared static Empty
    >> Domain:Value 0103b168:05ac01d0 01074788:05ac01d0 <<
54fa17a0 400009a 14 System.Char[] 0 shared static WhitespaceChars
    >> Domain:Value 0103b168:05ac095c 01074788:05acae2c <<

So SaveAs is merely copying the file content from temp file to the location you provide.So you can never implement a proper upload progressbar because,by the time you call the saveas method ,it would have put all the content to the temp file.Similarly you cannot use Request.Files in any global module ,because this is internally using the HttpRawUploadedContent class.

The file gets uploaded to temp directory when the System.Web.HttpRequest.GetEntireRawContent() is called.And this is called from FillInFormCollection() which is again called from the getter Request.Form.

 

So if any of the code(even the built in asp.net framework code) refers a Request.Form element,the file gets uploaded to temp directory.

Comments