Powershell and writing files (how fast can you write to a file? )
Hi there,
I’ve been working for some time on a tool similar to PAL from mike lagase in order to automate the analysis of loadgen runs.
While doing this I had to write large files with PowerShell and was not impressed with the result.
I thought I’d share the problem and solution with the community.
There are at least four ways of writing files with PSH:
· use the ‘>>’' alias that calls into the out-file cmd-let
· use export-csv
· use .Net
So let’s say I want to write 10000 lines into a file.
Method 1: use the >> operator
$s=”some text”
1..10000 | % {
$s >> ".\test.txt"
}
This way you actually open and close the file 1000 times.
Method 2: use out-file to write an Array
1..10000 | % {
$a+=[Array] $s
}
$a | out-file ".\test.txt"
This way actually writes once, using powershell.
Method 3: use export-csv to write an Array
$a=@()
1..10000 | % {
$a+=[Array] $s
}
$a | export-csv "t.txt"
Export-csv can also write the array for you
Method 4: use .Net StreamWriter to write the file
$stream = [System.IO.StreamWriter] "t.txt"
1..10000 | % {
$stream.WriteLine($s)
}
$stream.close()
The StreamWriter object from .Net also does the work nicely.
Conclusion: how fast ?
I tried all methods on my laptop, and here is what I got:
Method |
Time to completion |
‘>>’ |
29 s |
Outfile and [Array] |
27 s |
export-csv |
22 s |
StreamWriter |
1.5 s |
Well results speak by themselves, if you are in a hurry, use .Net StreamWriter !
Cheers,
Next, check that with Powerhell v2!
Guillaume
Comments
Anonymous
January 01, 2003
$b = 1..10000 $b | Out-File C:Temptemp.txt 2.3 secondsAnonymous
January 01, 2003
Je viens de publier un petit post rapide sur la méthode d’écriture dans un fichier avec PSH : PowershellAnonymous
January 01, 2003
Thank you for this. I'm using this in a function to continuously write the output from a serial port to a fileAnonymous
May 06, 2009
1 s with old school language : vbscript ;-)Anonymous
September 09, 2009
Merci pour ce post, interessant surtout avec le temps d'execution, A+Anonymous
March 08, 2012
@dak > file.txt doesn't accomplish the same thingAnonymous
March 30, 2013
The comment has been removedAnonymous
May 09, 2013
In your example for Stream Writer it does not output anything to the text file... You should have $_ in place of your $s Example: $stream = [System.IO.StreamWriter] "t.txt" 1..10000 | % { $stream.WriteLine($_) } $stream.close()Anonymous
July 25, 2013
@Marcus -- The four snippets actually belong in one file, that is why you see reuse of the $s variable in all 4 examples with only one definition in the first. It would probably have been better to post the 4 examples as a single file, or post 4 complete files with all definitions needed included in each one.Anonymous
October 29, 2013
The comment has been removedAnonymous
May 20, 2014
Pingback from Powershell | PearltreesAnonymous
July 19, 2014
only 0.08 sec with the funny language BatchAnonymous
September 10, 2014
The comment has been removedAnonymous
May 28, 2015
Thank you, I was looking for a simple means of generating output (wasn't remembering out-file) and found something even better.Anonymous
June 11, 2015
I’ve been looking everywhere for this! Thank goodness I found it on Google. You have made my day!http://www.dietmythsandfacts.com/2013/10/old-school-new-body-user-reviews.htmlAnonymous
July 23, 2015
thanks Streamwriter solved my issue for writing large dataset into text file, it's quite efficient and file size also is very compactAnonymous
August 06, 2015
HIV ialah suatu jenis virus yang mengakibatkan sakit HIV. Virus AIDS kebiasaannya menyerang seseorang serta menyerang sistem imunitas seseorang, yang akhirnya mengakibatkan tubuh jadi lemah dalam melawan infeksi. Setiap orang yang terkena HIV kebiasaannya kekurangan sistem imun dalam tubuh.Anonymous
November 15, 2015
The comment has been removedAnonymous
February 11, 2016
I googled this while I was inpatient, Streamwriter rocks! Thanks:)Anonymous
April 08, 2016
The comment has been removed