Share via

Big Downloads With PowerShell

  • Article

Personally I do not trust some of the popular 3rd party tools used for big downloads given the nepharious nature of the content people usually download with them.  That leads me to believe that the creators of such tools may not be entirely trustworthy, so I would prefer not to install those tools on my machines.  That's just my opinion, and I am open to be corrected.

That being said, I stumbled onto a blog post by the one-and-only Jeffrey Snover describing the ability for PowerShell to do large downloads with BITS.  It's a good post and worth the read.  Unfortunately his demo code was from a pre-release version, and some of the cmdlets have since been renamed in production.  The beauty of BITS is that it sips bandwidth while you're using the PC, throttles up when you're not using the PC, and resumes interrupted downloads.

Using his script as a base I have created an updated and rather robust PowerShell BITS script for large downloads, complete with download progress reporting.  Now you can kick off that big download before you go to bed, and in the morning your console will show you the start and end times of the download and the completion status.  This demo script uses the Windows Server 2008 R2 SP1 release candidate download as an example large file.

The comments in the code are straight forward.  Tweak the source and destination paths, and you're good to go.  Enjoy!

 # PowerShell Bits Transfer            
# Ashley McGlone            
# November, 2010            
#            
# Original PowerShell & BITS article by Jeffrey Snover here:            
#   https://blogs.msdn.com/b/powershell/archive/2009/01/11/transferring-large-files-using-bits.aspx            
# Mr. Snover's article was based on pre-release code.            
# This version uses the updated RTM syntax.            
# This version does a checking loop to see if the file has completed transfer            
# yet, and automatically completes the transfer if it is done.            
# It also displays progress of the download.            
            
$ErrorActionPreference = "Stop"            
            
Import-Module BitsTransfer            
# URL to your large download file here            
# EXAMPLE: Windows Server 2008 R2 SP1 Release Candidate            
$src  = "https://download.microsoft.com/download/0/A/F/0AFB5316-3062-494A-AB78-7FB0D4461357/windows6.1-KB976932-X86.exe"            
# Local file path here            
$dest = "C:\Users\username\Downloads\"            
            
# We make a unique display name for the transfer so that it can be uniquely            
# referenced by name and will not return an array of jobs if we have run the            
# script multiple times simultaneously.            
$displayName = "MyBitsTransfer " + (Get-Date)            
Start-BitsTransfer `
    -Source $src `
    -Destination $dest `
    -DisplayName $displayName `
    -Asynchronous            
$job = Get-BitsTransfer $displayName            
            
# Create a holding pattern while we wait for the connection to be established            
# and the transfer to actually begin.  Otherwise the next Do...While loop may            
# exit before the transfer even starts.  Print the job status as it changes            
# from Queued to Connecting to Transferring.            
# If the download fails, remove the bad job and exit the loop.            
$lastStatus = $job.JobState            
Do {            
    If ($lastStatus -ne $job.JobState) {            
        $lastStatus = $job.JobState            
        $job            
    }            
    If ($lastStatus -like "*Error*") {            
        Remove-BitsTransfer $job            
        Write-Host "Error connecting to download."            
        Exit            
    }            
}            
while ($lastStatus -ne "Transferring")            
$job            
            
# Print the transfer status as we go:            
#   Date & Time   BytesTransferred   BytesTotal   PercentComplete            
do {            
    Write-Host (Get-Date) $job.BytesTransferred $job.BytesTotal `
        ($job.BytesTransferred/$job.BytesTotal*100)            
    Start-Sleep -s 10            
}            
while ($job.BytesTransferred -lt $job.BytesTotal)            
            
# Print the final status once complete.            
Write-Host (Get-Date) $job.BytesTransferred $job.BytesTotal `
    ($job.BytesTransferred/$job.BytesTotal*100)            
            
Complete-BitsTransfer $job            
            
#   ><>

 

 

BitsTransfer.p-s-1.txt

Comments

  • Anonymous
    May 26, 2011
    Ashley, does BITS have the ability to perform a Dir on the remote host side? (similiar to an FTP Dir)? Thanks!

  • Anonymous
    May 26, 2011
    The comment has been removed

  • Anonymous
    September 23, 2015
    Ashley- I am a new powershell user and I appreciate your blog post. I am downloading large data files that contain bioinformatic sequence data and I am curious how to "loop" though a large number of files or URL's? I was able to get your code to work but given the level of complexity . . it is a bit out of reach for me to think of a way to loop. Can you provide an example where there are multiple $src inputs? Thanks!

  • Anonymous
    September 23, 2015
    Hello Dominick,

    I would recommend viewing another online code sample for multiple Bits transfers. Here is an example:

    https://gallery.technet.microsoft.com/scriptcenter/Script-to-SharePoint-2013-702e07df

    Hope this helps,
    GoateePFE
    Ashley

  • Anonymous
    September 28, 2016
    Will BITS work with the Build Artifact Download Url provided by Visual Studio Team Services?