Generate SharePoint 2010 Sitemap with Windows PowerShell

#Update: This script has been submitted to SharePoint category in TechNet Script Center Repository. 

If you are working on SEO project for your internet facing SharePoint site, one of the top thing would be sitemap generation. There’re quite a few sitemap generation tools on the market, including the sitemap tool in SEO Toolkit for IIS. But do you know you can achieve this just by Windows PowerShell script? It is quite flexible, and easy to customize and automate.

So here is something you can leverage….

New-SPSiteMap Script

 function New-SPSiteMap
{
    param($SavePath="C:\inetpub\wwwroot\wss\VirtualDirectories\80\SiteMap.xml", $Url="https://sharepoint")
    $web=Get-SPWeb $url
    $list = $web.Lists | ForEach-Object -Process {$_.Items}| ForEach-Object -Process {$_.url.Replace(" ","%20")}

     #excludes directories you don’t want in sitemap. you can put multiple lines here:

     $list=  $list | ? {$_ -notmatch "_catalogs"} 

     $list=  $list | ? {$_ -notmatch "Reporting%20Templates"} 

     $list=  $list | ? {$_ -notmatch "Reporting%20Metadata"} 

     $list | New-Xml -RootTag urlset -ItemTag url -ChildItems loc -SavePath $SavePath
}

function New-Xml
{
  
    param($RootTag="urlset",$ItemTag="url", $ChildItems="*", $SavePath="C:\SiteMap.xml")
    Begin {
        $xml="<?xml version=""1.0"" encoding=""UTF-8""?>
        <urlset xmlns=""https://www.sitemaps.org/schemas/sitemap/0.9"">"
        }
    Process {
        $xml += " <$ItemTag>"
        foreach ($child in $_){
        $Name = $child
        $xml += " <$ChildItems>$url/$child</$ChildItems>"
        }
        $xml += " </$ItemTag>"
        }
    End {
        $xml += "</$RootTag>"
        $xmltext=[xml]$xml
        $xmltext.Save($SavePath)
    }
} 

You can download the script here. This script does have some limitations – for example, I didn’t use http encode for the urls so if you have urls in Chinese, they may not be able to be accessed. But certainly one of the best practice of SEO is to avoid using these languages in the url which may cause such encoding. I didn’t implement <lastmod> either.

Usage: New-SPSiteMap –Url [sitename] –SavePath [yoursiteroot\sitemap.xml]

For example, I want to create a sitemap for a SharePoint demo site, so I first need to load the functions into my SharePoint Management Shell, then do the following:

New-SPSiteMap –Url https://www.sharepointisawesome.com –SavePath C:\inetpub\wwwroot\wss\VirtualDirectories\80\sitemap.xml

Robots.txt Customization

The next step is to create a robots.txt with the following content, replace yoursitename with domain name:

Sitemap: https://yoursitename/sitemap.xml

This is to make sure search engines can pick up your sitemap file directly when they read robots.txt. Save it to the same root directory of your SharePoint site.  For example: C:\inetpub\wwwroot\wss\VirtualDirectories\80\

Do a iisreset to ensure the new sitemap.xml and robots.txt can be loaded.  Done!

Search Engine Submission

You can wait for search engines to read your robots.txt and sitemap, or directly submit the sitemap to them.

Example: Submit a new site to Bing.com:

https://www.bing.com/webmaster/WebmasterAddSitesPage.aspx

Fill in web address and sitemap address, then follow the validation instructions.

Automate Sitemap Generation Process

If your site structure is updated frequently, you can also schedule the script to run everyday. This can be achieved by task scheduler. I will not go into the detail here, but don’t forget to use Add-PSSnapin Microsoft.SharePoint.PowerShell to load the snapin before you load the functions.

Jie.

Comments

• Anonymous
  July 29, 2010
  Jie, thanks for this script; it's awesome. We noticed that it only seemed to be generating a sitemap for the top SPWeb so I modified this to go through all webs in the site collection. I've posted an update on my blog here: www.glynblogs.com/.../generate-a-sitemap-for-sharepoint-2010-using-powershell.html Cheers!

• Anonymous
  August 02, 2010
  Simply awesome!

• Anonymous
  September 17, 2012
  Agreed, this is great but you need to go through the entire collection to get the full map. This script is fine for a single site but most public facing sites are not just the top level.