Share via

Powershell and writing files (how fast can you write to a file? )

  • Article

Hi there,

I’ve been working for some time on a tool similar to PAL from mike lagase in order to automate the analysis of loadgen runs.

While doing this I had to write large files with PowerShell and was not impressed with the result.

I thought I’d share the problem and solution with the community.

There are at least four ways of writing files with PSH:

· use the ‘>>’' alias that calls into the out-file cmd-let

· use export-csv

· use .Net

So let’s say I want to write 10000 lines into a file.

Method 1: use the >> operator

$s=”some text”

1..10000 | % {
$s >> ".\test.txt"
}

This way you actually open and close the file 1000 times.

Method 2: use out-file to write an Array

1..10000 | % {
$a+=[Array] $s
}

$a | out-file ".\test.txt"

This way actually writes once, using powershell.

Method 3: use export-csv to write an Array

$a=@()

1..10000 | % {

      $a+=[Array] $s

}

$a | export-csv "t.txt"

Export-csv can also write the array for you

Method 4: use .Net StreamWriter to write the file

$stream = [System.IO.StreamWriter] "t.txt"

1..10000 | % {

      $stream.WriteLine($s)

}

$stream.close()

The StreamWriter object from .Net also does the work nicely.

Conclusion: how fast ?

I tried all methods on my laptop, and here is what I got:

 Method

Time to completion

‘>>’

29 s

Outfile and [Array]

27 s

export-csv

22 s

StreamWriter

1.5 s

Well results speak by themselves, if you are in a hurry, use .Net StreamWriter !

Cheers,

Next, check that with Powerhell v2!

Guillaume

Comments

  • Anonymous
    January 01, 2003
    $b = 1..10000 $b | Out-File C:Temptemp.txt 2.3 seconds

  • Anonymous
    January 01, 2003
    Je viens de publier un petit post rapide sur la méthode d’écriture dans un fichier avec PSH : Powershell

  • Anonymous
    January 01, 2003
    Thank you for this. I'm using this in a function to continuously write the output from a serial port to a file

  • Anonymous
    May 06, 2009
    1 s with old school language : vbscript ;-)

  • Anonymous
    September 09, 2009
    Merci pour ce post, interessant surtout avec le temps d'execution, A+

  • Anonymous
    March 08, 2012
    @dak > file.txt doesn't accomplish the same thing

  • Anonymous
    March 30, 2013
    The comment has been removed

  • Anonymous
    May 09, 2013
    In your example for Stream Writer it does not output anything to the text file... You should have $_ in place of your $s Example: $stream = [System.IO.StreamWriter] "t.txt" 1..10000 | % {      $stream.WriteLine($_) } $stream.close()

  • Anonymous
    July 25, 2013
    @Marcus -- The four snippets actually belong in one file, that is why you see reuse of the $s variable in all 4 examples with only one definition in the first.  It would probably have been better to post the 4 examples as a single file, or post 4 complete files with all definitions needed included in each one.

  • Anonymous
    October 29, 2013
    The comment has been removed

  • Anonymous
    May 20, 2014
    Pingback from Powershell | Pearltrees

  • Anonymous
    July 19, 2014
    only 0.08 sec with the funny language Batch

  • Anonymous
    September 10, 2014
    The comment has been removed

  • Anonymous
    May 28, 2015
    Thank you, I was looking for a simple means of generating output (wasn't remembering out-file) and found something even better.

  • Anonymous
    June 11, 2015
    I’ve been looking everywhere for this! Thank goodness I found it on Google. You have made my day!http://www.dietmythsandfacts.com/2013/10/old-school-new-body-user-reviews.html

  • Anonymous
    July 23, 2015
    thanks Streamwriter solved my issue for writing large dataset into text file, it's quite efficient and file size also is very compact

  • Anonymous
    August 06, 2015
    HIV ialah suatu jenis virus yang mengakibatkan sakit HIV. Virus AIDS kebiasaannya menyerang seseorang serta menyerang sistem imunitas seseorang, yang akhirnya mengakibatkan tubuh jadi lemah dalam melawan infeksi. Setiap orang yang terkena HIV kebiasaannya kekurangan sistem imun dalam tubuh.

  • Anonymous
    November 15, 2015
    The comment has been removed

  • Anonymous
    February 11, 2016
    I googled this while I was inpatient, Streamwriter rocks! Thanks:)

  • Anonymous
    April 08, 2016
    The comment has been removed