Troubleshooting Remote Blob Store (RBS) Install and Configuration issues
Summary
I worked on a long running RBS case where the customer followed the steps in our TechNet Article , which resulted in SharePoint throwing an the following error while uploading files to a document library, after enabling RBS.
w3wp.exe (0x0EF0) 0x24E4 SharePoint Foundation Runtime tkau Unexpected
System.Runtime.InteropServices.COMException: The URL 'Shared Documents/document1.docx' is invalid. It may refer to a nonexistent file or folder, or refer to a valid file or folder that is not in the current Web.
at Microsoft.SharePoint.Library.SPRequestInternalClass.PutFile(String bstrUrl, String bstrWebRelativeUrl, Object punkFile, Int64 cbFile, Object punkSPFileMgr, Object punkFFM, Int64 PutFileOpt, String bstrCreatedBy, String bstrModifiedBy, Int32 iCreatedByID, Int32 iModifiedByID, Object varTimeCreated, Object varTimeLastModified, Object varProperties, String bstrCheckinComment, String bstrLockIdMatch, String bstEtagToMatch, Int32 lockType, String lockId, Int32 minutes, Int32 fRefreshLock, Int32 bValidateReqFields, Guid gNewDocId, SPFileSaveParams sfsp, SPFileInfo& pFileProps, UInt32& pdwVirusCheckStatus, String& pVirusCheckMessage, String& pEtagReturn, Byte& piLevel, Int32& pbIgnoredReqProps)
at Microsoft.SharePoint.Library.SPRequest.PutFile(String bstrUrl, String bstrWebRelativeUrl, Object punkFile, Int64 cbFile, Object punkSPFileMgr, Object punkFFM, Int64 PutFileOpt, String bstrCreatedBy, String bstrModifiedBy, Int32 iCreatedByID, Int32 iModifiedByID, Object varTimeCreated, Object varTimeLastModified, Object varProperties, String bstrCheckinComment, String bstrLockIdMatch, String bstEtagToMatch, Int32 lockType, String lockId, Int32 minutes, Int32 fRefreshLock, Int32 bValidateReqFields, Guid gNewDocId, SPFileSaveParams sfsp, SPFileInfo& pFileProps, UInt32& pdwVirusCheckStatus, String& pVirusCheckMessage, String& pEtagReturn, Byte& piLevel, Int32& pbIgnoredReqProps)
While working on this issue, I learned many ways to troubleshoot RBS installation and configuration issues and I wanted to share.
Overview
First, there are many steps to install and configure RBS and the best Microsoft sources are the following articles.
Overview of RBS in SharePoint 2013
https://technet.microsoft.com/en-us/library/ee748649.aspx
Install and configure RBS with FILESTREAM in a SharePoint 2013 farm
https://technet.microsoft.com/en-us/library/ee748631.aspx
Install and configure RBS with SharePoint 2013 and SQL Server 2012
https://blogs.technet.microsoft.com/bogdang/2014/12/04/install-and-configure-rbs-with-sharepoint-2013-and-sql-server-2012
At a high level, the steps are:
- Enable FILESTREAM on the database server
- Provision a BLOB store for each content database
- Install the RBS client library on each Web server < - This is where my customer had problems.
- Enable RBS for each content database
- Assign db_owner permissions to the web application
Sounds easy enough? However, when things don't go as planned, here are a few tips.
Troubleshooting
1. Installing the correct RBS client is key. Ensure to install the RBS client that matches the SQL server version.
Here is a current list of SQL feature packs:
SQL 2008 R2
RTM: https://www.microsoft.com/en-us/download/details.aspx?id=16978
SP1: https://www.microsoft.com/en-us/download/details.aspx?id=26728
Sp2: https://www.microsoft.com/en-us/download/details.aspx?id=30440
Sp3: https://www.microsoft.com/en-us/download/details.aspx?id=44272
SQL 2012
RTM: https://www.microsoft.com/en-us/download/details.aspx?id=29065
SP1: https://www.microsoft.com/en-us/download/details.aspx?id=35580
SP2: https://www.microsoft.com/en-us/download/details.aspx?id=43339
SP3: https://www.microsoft.com/en-us/download/details.aspx?id=49999
SQL 2014
RTM: https://www.microsoft.com/en-us/download/details.aspx?id=42295
SP1: https://www.microsoft.com/en-us/download/details.aspx?id=46696
SP2: https://www.microsoft.com/en-us/download/details.aspx?id=53164
SQL 2016:
RTM: https://www.microsoft.com/en-us/download/details.aspx?id=52676
SP1: https://www.microsoft.com/en-us/download/details.aspx?id=54279
Note: You only need the RBS.MSI file (x86 | x64) from the feature pack.
2. You must install the RBS client with the command line, as it only installs what is needed and with a pre-defined configuration.
When installing RBS on the first client, this command only needs to be run once.
msiexec /qn /lvx* rbs_install_log.txt /i RBS.msi TRUSTSERVERCERTIFICATE=true FILEGROUP=PRIMARY DBNAME="WSS_Content" DBINSTANCE="DBInstanceName" FILESTREAMFILEGROUP=RBSFilestreamProvider FILESTREAMSTORENAME=FilestreamProvider_1
The remaining servers will use a slightly different command.
msiexec /qn /lvx* rbs_install_log.txt /i RBS.msi DBNAME="WSS_Content" DBINSTANCE="DBInstanceName" ADDLOCAL=Client,Docs,Maintainer,ServerScript,FilestreamClient,FilestreamServer
Note: The differences is that on the first server, you are creating the FileStreamFileGroup and FileStreamStore , the remaining WFEs will be clients that connect to the existing configuration.
3. After installing the client with command line, reading the install log is required. If there are any errors during the install, it needs to be addressed before moving on, and reading the log file is the only way to confirm if the install was successful.
The log will be in the same folder as the MSI is located.
Look for the following to indicate success:
MSI (s) (6C:A4) [15:45:50:262]: Windows Installer installed the product. Product Name: Microsoft SQL Server 2014 Remote BLOB Store. Product Version: 12.2.5000.0. Product Language: 1033. Manufacturer: Microsoft Corporation. Installation success or error status: 0.
4. When enabling RBS for each content database, you may receive the error "SQL remote blob storage must be installed on each web front end server and on the content database before it may be used.".
Command to enable:
$cdb = Get-SPContentDatabase <ContentDatabaseName>
$rbss = $cdb.RemoteBlobStorageSettings
$rbss.Installed()
$rbss.Enable()
$rbss.SetActiveProviderName($rbss.GetProviderNames()[0])
$rbss
Example of error:
Exception calling "Enable" with "0" argument(s): "SQL remote blob storage must
be installed on each web front end server and on the content database before
it may be used."
At line:1 char:1
+ $rbss.Enable()
+ ~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : InvalidOperationException
If this occurs, there was a problem with the client one or more WFES, or the server(s) needs to be rebooted.
Check the RBS client install log on all WFEs. If they look good, reboot them and try the enable again.
If you see errors in the install log, the client may need to be uninstalled and reinstalled (not updated).
5. Once everything is installed and "should be working" but it's not, check the "verbose" ULS Logs when attempting to upload a fie. In verbose mode, you will see log entries with the "Files" category detailing why the upload failed.
Example:
w3wp.exe (0x1518) 0x274C SharePoint Foundation Files aer55 Verbose
SPFileStream at ComputeStreamsToWrite: [Url=Shared Documents/Document1.docx] [StreamSize=1049038] [DocId=cc8bbe09-8ca3-481c-9a54-6bca8b061cf1] [Partition=0] [HistVer=0] [Level=Published] [ContentVer=0] [NextBSN=100] [SourceSchema=StreamBlobVirtual]
w3wp.exe (0x1518) 0x274C SharePoint Foundation Files 9j7o Medium
Exception thrown storing stream in new SqlRemoteBlob: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.Data.SqlRemoteBlobs.RemoteBlobConfigurationException: No provider of type <Filestream> found. Check the server configuration or install the provider on the client.
at Microsoft.Data.SqlRemoteBlobs.ProviderSession.Connect()
at Microsoft.Data.SqlRemoteBlobs.RemoteBlobCommand.ExecuteInternal()
at Microsoft.Data.SqlRemoteBlobs.RemoteBlobCommand.Execute()
at Microsoft.Data.SqlRemoteBlobs.SqlRemoteBlob.WriteFromStream(Stream inStream)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Microsoft.SharePoint.SqlRemoteBlobSession.StoreStreamInBlob(Int32 collectionId, String providerName, Stream stream)
- This is an RBS client issue or configuration issue.
- The client is not installed correctly
- The files steam was entered incorrectly while installing the first client.
- There are a few things to check to ensure the client in installed properly
6. Ensure the proper version is installed that matches the SQL version and service pack level.
- Ensure that the client assemblies were installed in the GAC properly. Ref: https://blogs.msdn.microsoft.com/sqlrbs/2010/09/23/fixing-rbsprovider-version-mismatch-issues
- Ensure that the proper machine.config has been updated with the following 2 sections for the RBS client.Note: If these entries are not present in the machine.config, the error "No provider of type <Filestream> found" will be present.
</sectionGroup>
<sectionGroup name="RemoteBlobStorage" type="System.Configuration.ConfigurationSectionGroup, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" >
<section name="InstalledVersions" type="System.Configuration.AppSettingsSection, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<section name="Logging" type="System.Configuration.AppSettingsSection, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<sectionGroup name="Providers" type="System.Configuration.ConfigurationSectionGroup, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" >
<section name="remotefilestream" type="System.Configuration.AppSettingsSection, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<section name="filestream" type="System.Configuration.AppSettingsSection, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</sectionGroup>
<sectionGroup name="ProviderSpecificConfigKeys" type="System.Configuration.ConfigurationSectionGroup, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" >
<section name="remotefilestream" type="System.Configuration.AppSettingsSection, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
<section name="filestream" type="System.Configuration.AppSettingsSection, System.Configuration, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
</sectionGroup>
</sectionGroup>
<RemoteBlobStorage>
<Logging>
<add key="EventViewerLog" value="4" />
<add key="FileLog" value="0" />
<add key="ConsoleLog" value="0" />
<add key="CircularLog" value="0" />
<add key="DatabaseTableLog" value="0" />
</Logging>
<InstalledVersions>
<add key="Count" value="2" />
</InstalledVersions>
<Providers>
<remotefilestream>
<add key="BlobStoreType" value="RemoteFilestream" />
<add key="DllFile" value="C:\Program Files\Microsoft SQL Remote Blob Storage 12.0\Provider Libraries\Filestream Provider\\Microsoft.Data.BlobStores.FilestreamBlobStore.dll" />
<add key="ClassName" value="Microsoft.Data.BlobStores.FilestreamBlobStore.RemoteFilestreamStoreLibrary" />
<add key="ProviderVersion" value="1.0.0.0" />
<add key="MinSupportedBackEndStoreVersion" value="1.0.0.0" />
<add key="BlobStoreAbstractClassVersion" value="10.50.0.0" />
<add key="EnumerationOptimizationLevelImplemented" value="2" />
<add key="ConsistencyCheckLevelImplemented" value="2" />
</remotefilestream>
<filestream>
<add key="BlobStoreType" value="Filestream" />
<add key="DllFile" value="C:\Program Files\Microsoft SQL Remote Blob Storage 12.0\Provider Libraries\Filestream Provider\\Microsoft.Data.BlobStores.FilestreamBlobStore.dll" />
<add key="ClassName" value="Microsoft.Data.BlobStores.FilestreamBlobStore.FilestreamStoreLibrary" />
<add key="ProviderVersion" value="1.0.0.0" />
<add key="MinSupportedBackEndStoreVersion" value="1.0.0.0" />
<add key="BlobStoreAbstractClassVersion" value="11.0.0.0" />
<add key="EnumerationOptimizationLevelImplemented" value="2" />
<add key="ConsistencyCheckLevelImplemented" value="2" />
</filestream>
</Providers>
<ProviderSpecificConfigKeys>
<remotefilestream>
<add key="schema_suffix" value="Name" />
<add key="max_size_inband_read" value="Number" />
<add key="tran_pool_size" value="Number" />
<add key="tran_pool_tran_lifetime" value="Duration" />
<add key="async_read_disabled" value="Boolean" />
<add key="filegroup_name" value="Name" />
<add key="data_filegroup_name" value="Name" />
<add key="max_pool_size" value="Number" />
<add key="max_size_inband_write" value="Number" />
<add key="max_size_inline_blob" value="Number" />
<add key="write_allocation_size" value="Number" />
<add key="async_write_disabled" value="Boolean" />
</remotefilestream>
<filestream>
<add key="schema_suffix" value="Name" />
<add key="max_size_inband_read" value="Number" />
<add key="tran_pool_size" value="Number" />
<add key="tran_pool_tran_lifetime" value="Duration" />
<add key="async_read_disabled" value="Boolean" />
<add key="filegroup_name" value="Name" />
<add key="data_filegroup_name" value="Name" />
<add key="max_pool_size" value="Number" />
<add key="max_size_inband_write" value="Number" />
<add key="max_size_inline_blob" value="Number" />
<add key="write_allocation_size" value="Number" />
<add key="async_write_disabled" value="Boolean" />
</filestream>
</ProviderSpecificConfigKeys>
</RemoteBlobStorage>
7. Lastly, read the Remote Blob Storage whitepaper, its excellent!
Remote Blob Storage
https://download.microsoft.com/download/9/5/2/9521D8DA-5D3C-4817-BB9D-B5B1BD293365/SQL_Server_2008_R2_Remote_Blob_Storage.docx