FAQ: SharePoint 2010 Remote BLOB Storage (RBS)
Recently on SharePointPro Connections I got a lot of questions for SharePoint RBS. To help people to get clear on this topic here’s a FAQ… For installation and other documentations please refer to my pervious blog post on this topic: SharePoint 2010 Beta with FILESTREAM RBS Provider. Todd Klindt also has a very nice step by step guide: Installing Remote Blob Store (RBS) on SharePoint 2010.
Q: What is RBS?
From SQL RBS Team Blog:
RBS(Remote Blob Store) is a set of standardized APIs that allow storage/retrieval of BLOBs outside of your main SQL database where a dedicated BLOB store is desirable for various reasons. This uses a provider model for plugging in any dedicated BLOB store that implements these RBS APIs.
Q: Which version of SQL Server can I use for SharePoint RBS?
SQL Server 2008 and SQL Server 2008 R2 both support RBS. A RBS Library needs to be downloaded and installed on SQL Server to enable the feature. All SQL editions (Express, Standard, Enterprise…) support RBS. Licensing requirement may involve depending on the scenarios.
Q: What is FILESTREAM?
FILESTREAM is a SQL Server 2008 feature to store BLOB content on to file system.
From FILESTREAM Overview:
FILESTREAM integrates the SQL Server Database Engine with an NTFS file system by storing varbinary(max) binary large object (BLOB) data as files on the file system. Transact-SQL statements can insert, update, query, search, and back up FILESTREAM data. Win32 file system interfaces provide streaming access to the data.
FILESTREAM uses the NT system cache for caching file data. This helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.
SQL FILESTREAM feature does not allow to store content on anything other than local storages. (SMB shares cannot be used for store BLOB content)
Q: Is there a comparison of FILESTREAM and RBS?
Please check SQL RBS team blog: SQL Server Remote BLOB Store and FILESTREAM feature comparison.
Q: What is RBS FILESTREAM Provider?
RBS FILESTREAM Provider is a free out-of-the-box provider shipped by Microsoft SQL RBS Team that allows a deployment to use a SQL Database (local or remote) as a dedicated BLOB store. This provider utilizes the FILESTREAM as the BLOB storage mechanism and ties the two technologies together.
Q: Any benefit for using RBS with SharePoint?
By using RBS for SharePoint, customer maybe able to leverage cheaper storage, improve performance, and enable better integration stories with 3rd party technology for their SharePoint databases. But be careful, the benefit is different case by case. You need to investigate your scenarios to see if RBS really fits you.
Q: Any benefit to use RBS FILESTREAM Provider?
Benefit for Windows SharePoint Services 3.0 with Windows Internal Database upgrade:
We no longer support Windows Internal Database (WID) with SharePoint Foundation 2010. However, SQL Server Express has a 4GB DB size limit. The direct upgrade of a content DB > 4gb from WID to SQL Server Express is not supported. To get around of this limit, customer can use RBS FILESTREAM Provider to store all the BLOB content on to file system so the DB size will still remain small.
Benefit for other scenarios:
We are currently recommending to use this technology with document archiving and media streaming scenarios. Tests show that with RBS FILESTREAM Provider large BLOB objects can have better I/O compared with putting everything in the database.
Q: Can RBS FILESTREAM Provider support SMB shares to store the content, for example a NAS device?
No. SQL FILESTREAM feature does not allow to store content on anything other than local storages. Therefore RBS FILESTREAM Provider has the same limitation. 3rd party RBS Providers do not have this limitation if they are not leveraging SQL FILESTREAM feature.
Q: How is the BLOB content stored on the file system with RBS FILESTREAM Provider? Is it encrypted? Does TDE helps with the encryption?
The BLOB objects are stored with GUID names to make sure they can be linked from the RBS tables. The content is not encrypted. TDE does not apply to the content in file stream.
Q: I heard there’s a remote feature in RBS FILESTREAM Provider, is that supported by SharePoint 2010?
This feature allows SQL RBS FILESTREAM Provider to store BLOBs on to a remote SQL Server DB/Instance. SharePoint 2010 will support this remote feature of RBS FILESTREAM Provider. However the test work will not be finished at the time of SharePoint 2010 RTM. We will announce the supportability of this remote feature later when the test is done.
Q: Is there a licensing requirement for SQL Server when I want to use RBS for SharePoint?
Yes, there is. If the BLOB objects are stored on local SQL Server file system then you can use any SQL Server SKU. For any other scenarios, local SQL Server needs to be Enterprise. The following table shows the license requirement for different scenarios.
| Scenario | Local SQL SKU | Remote SQL SKU |
| FILESTREAM Provider, BLOB stored on local SQL Server file system | Any SQL SKU | N/A |
| FILESTREAM Provider, BLOB stored on Remote SQL Server file system | SQL Enterprise | SQL Standard or above |
| 3rd Party RBS Provider | SQL Enterprise | N/A |
Q: Any 3rd party RBS provider I can use?
Currently StoragePoint (acquired by Metalogix recently) and EMC are providing RBS Providers for different storage options. NetApps, Opentext, Commvault are also working on it.
AvePoint also made their DocAve Extender free – a EBS provider for 2007, and a RBS provider for 2010. (thanks to Chris Musico for pointing out)
STEALTH Software also offers RBS providers. (Thanks Mark)
Q: Is EBS still supported?
EBS(External BLOB Storage API) is introduced by SharePoint 2007. It is still supported in this release (2010). Moving forward, in order to have a more unified storage solution, we will only support RBS in the next release.
Q: How should I backup the content if I’m using RBS?
By using SharePoint Backup and Restore, it doesn’t really matter where the BLOB objects are stored – this is taken care by SQL RBS. So when you use SharePoint Backup on a RBS enabled content database, it will still backup everything. You can restore the backup to a non-RBS enabled database too, the BLOB objects will be inside the database – SQL RBS will decide where to put them, not SharePoint.
If you are trying to backup the database by using SQL Backup, keep in mind this will not backup the BLOB objects. You need to manually backup the database first, then manually backup (XCOPY or other file backup tools) these BLOBs. When restore, using a reverse order to restore BLOBs first, then the databases. UPDATE: SQL Backup now can take care of FILESTREAM BLOB objects.
Q: Can RBS help to remove the 2GB file size limit in SharePoint?
No, using RBS with SharePoint will have the same file size limit.
Q: Can I enable RBS on a content database with content existing? How to migrate those content into the BLOB store? (Thanks to Chris Givens)
Yes, you can enable RBS on a content database and then migrate content into the external storage. There's a "Migrate()" method to handle this process. For example using Windows PowerShell, $cdb is the content db with RBS enabled, the following command will do the job:
$cdb.Migrate()
Q: Should I consider using RBS all the time?
No. RBS may be used when :
- You have huge content dbs for document archiving so you want to reduce storage cost (terabytes of data)
- You have large media files to stream to the audience
- You need to use RBS to integrate 3rd party storage/archiving solutions to SharePoint. (For example EMC Documentum)
For example, if you only have 100GB of data, seperated in several content databases, and most of the content are documents, go for RBS will not benefit your server farm.
Q: Are SQL Mirroring, Clustering and Log Shipping supported when using RBS FILESTREAM Provider?
Log Shipping and Clustering are supported. SQL FILESTREAM cannot be used together with SQL Mirroring.
If you have additional questions please leave them in the comment, I can add them into this list later.
Jie.
Comments
Anonymous
March 24, 2010
Awesome post Jie! Another question that should be addressed: How does one migrate to and from RBS for pre-existing data? Or another way to ask this, what happens when RBS is enabled on an existing content database? Does the data get moved over automatically and/or how? I know these answers, but for completeness for your blog post :)Anonymous
March 24, 2010
@chris, good question! I'll add that into the list... I know you have the answer:)Anonymous
March 24, 2010
Jie, Why do you note that 3rd Party RBS Providers require SQL Enterprise? We've done extensive testing with StoragePoint against Standard edition without issue. Thanks, RobAnonymous
March 24, 2010
Didn't say it in my first comment, but very good post. I think you do an excellent job of positioning the capability.Anonymous
March 24, 2010
@Rob, it is a licensing requirement from SQL team - I don't think there's something in code that blocks you from using 3rd party RBS providers on STD/Express.Anonymous
March 24, 2010
I'm not sure how Microsoft is going to justify that to customers that aren't running SQL Enterprise, especially ones that are currently using EBS-based solutions and want to upgrade to 2010 and will feel compelled to migrate to RBS due to EBS being deprecated. So if a customer builds their own RBS provider is that considered a 3rd party provider that requires SQL Enterprise?Anonymous
March 24, 2010
@Rob, actrually it does not matter if that's 3rd party or not. SQL team said any providers that is storing content outside local SQL Server would need a SQL Enterprise. Keep in mind this is only about using the provider with SharePoint. When you use RBS with ASP.net there's no licensing requirement.Anonymous
March 24, 2010
Hi Jie, I've taken a backup(using stsadm cmd) of a (sharepoint 2010)web application for which RBS is enabled(uploaded files are stored in both content db also in local disk successfully). The restoration is also successful to a new web application. The problem persists whenever I upload the documents, they are stored in content db only, but not in local disk. I tried this with and without enabling RBS for the new application. I need your help. Chandra.Anonymous
March 25, 2010
@veerachandra This is the expected behavior. You need to enable the new content db with RBS, not web application. If the content is already inside the db you should use Migrate() to move them to the disk.Anonymous
March 25, 2010
Hi Jie, I came around your blog. Good to read info on RBS. I just wanted to add that we- SYNERGY4 - deliver also a 3rd party provider called STEALTH Software Content Store for SharePoint from STEALTH software (http://www.stealthsoftware.eu) who is one of the few companies who have developed 3rd party RBS and EBS providers for SharePoint to connect with external storage like Caringo CAStor, ParaScale, Bizanga Store, Windows Azure, Amazon S3 and EMC Atmos and now also Dell™ DX Object Storage Solution (Dell announced this yesterday http://bit.ly/9iT0Kg). We are both Dutch companies and mainly operational in Europe now, so that's maybe why you haven't heard from us. Hope you can include us in further blogs about SharePoint connected to external storage via RBS and EBS. If you are interested in the STEALTH Software, let me know and I can send you some info. Kind regards, MarkAnonymous
March 25, 2010
Hi Jie, I'm novice to the sharepoint. Could you tell me what are the propable reasons behind the response given by sharepoint management shell?... -->> "$rbss.Installed() - False". The log file is configured successfully. Thanks, Chandra.Anonymous
March 26, 2010
@Mark Thanks, I will add that to the list. @veerachandra Can you check if your log file is larger than 1MB? The installer should run a large potion of SQL commands to create the RBS tables, and all these commands will be logged. If there're no such commands in the log, the rbs tables are not created. And Installed() will return false. The "successfully" in log sometimes does not mean success.Anonymous
March 28, 2010
Hi Jie, When a site collection is backed up (through stsadm.exe on web server)to a new, the phrase "Product: SQL Remote Blob Storage -- Configuration completed successfully." is shown(the log file size is 443kb only). RBS has not at all been installed. RBS is installed successfully when a new web application is used, also it works. Could you figure where I've gone wrong. Thanks, Jie.Anonymous
March 29, 2010
@veerachandra RBS is enabled per content db, not per web application.Anonymous
March 31, 2010
Hi Jie, Thank you for including us, AvePoint, in your informative blog post. We'd just like to make one small clarification -- our free DocAve Extender tool leverages both Microsoft’s External BLOB Store Provider and Remote BLOB Storage, instantly routing BLOBs to disk-based storage the moment an end-user uploads them to SharePoint. For more information, check out our announcement (http://sn.im/v6dzx) or visit our product page (http://sn.im/v6e0b). Don't hesitate to contact me if you have any further questions about DocAve Extender or DocAve's Storage Optimization Suite. Best regards, ChrisAnonymous
March 31, 2010
Hi Chris, I didn't noticed anything that allow me to choose which one to use, and this is not mentioned in any of your documentation. Is that a hidden feature?Anonymous
March 31, 2010
Does the 100 GB recommendation for the actual DB size still apply (http://technet.microsoft.com/en-us/library/cc262787.aspx)?Anonymous
March 31, 2010
@Chris, please ignore my previous comment - I just saw the updated Beta on Avepoint's Site. That one is using RBS. @mcodyw, in SharePoint 2010 the suggested logical size of a frequently accessed content DB is 200GB. A non frequently accessed content DB can be 1TB. For logical size, I mean the db size plus the blob store for RBS. This is mainly for backup and restore - SharePoint backup will still backup all the content, including blobs, no matter it is in a external blob store or not.Anonymous
April 22, 2010
Awesome, thanks for sharing Jie Li! Two more questions:
- Does search use RBS to crawl/index/query RBS data?
- Why does the RBS need installed on the SharePoint Application Servers?
- Anonymous
April 23, 2010
- When crawl SharePoint pages, Search grabs all the blob objects through API. It doesn't matter if that's RBS enabled or not. So the content will be indexed.
- That is to ensure the application server knows how to communicate with the RBS storage.
Anonymous
April 29, 2010
Hi Jie - Thanks for the great info. You mention the SPContentDatabase.Migrate() method to turn on RBS for existing content databases. However, the MSDN documentation ( http://msdn.microsoft.com/en-us/library/ms434076(office.14).aspx ) shows this method as being obsolete (and as doing something else). Reflector on the assembly confirms the MSDN docs are correct. Am I missing something obvious here? Thanks! ~JoshAnonymous
May 12, 2010
@Josh,I didn't mean SPContentDatabase.Migrate() in my blog post. What I talked about is SPContentDatabase.RemoteBlobStorageSettings.Migrate().Anonymous
May 26, 2010
what is the best method to reclaim the space in the content DB after data is moved out.. DBCC,AutoShrink, ShrinkFile or Export DB to new DB?Anonymous
June 16, 2010
Hi, you write that when doing SQL Backups of a content db using RBS the blobs are not backed up. What if I use Filestream with RBS? A SQL Backup of course backs up filestream data, as long as I do not exclude the filestream filegroup with partial backups, right? This is what books online of SQL Server says.....Anonymous
June 17, 2010
@Wolfgang It depends on the provider you use. Just like you said, FILESTREAM may be backup because it's a SQL feature. But for the rest of the providers SQL backup may or may not be able to trigger the backup of the objects.Anonymous
June 29, 2010
The comment has been removedAnonymous
July 22, 2010
How does RBS and filestream work with SQL failover clustering? I understand with SharePoint 2010 only local filestream will work - how does the cluster see local disk or does it need to be a local external storage array connected to both Sql servers in the cluster?Anonymous
August 25, 2010
If SQL Clustering is supported for RBS Filestream how can you use local storage when you SQL Cluster is in using shared storage - most likely SAN storage?Anonymous
September 16, 2010
What happens if the RBS store goes offline, will the database with structured data be online? Regards, Roshan.Anonymous
October 12, 2010
Hi, Curious if anyone has used RBS (or if it can be used) to facilitate document archiving. Ex: If file_date > age_limit, move file to RBS Is this possible via workflow or some other means?Anonymous
October 12, 2010
Great post to start with. I was using EBS on my application earlier ,now I want to migrate to RBS.Now earlier to retrieve blobs i used to query the dbo.AllDocStreams to get the content...but now i have a different coloumn named rbsId ...how do i retrieve my blobs.Another thing i found out was i need blobid to retrieve blobs.how does rbsid lead me to blobId.. Thanks in AdvanceAnonymous
October 20, 2010
Why we have to go for 3rd party providers RBS? Can't we use Filestream?Anonymous
October 25, 2010
@Kiran - You can use RBS FILESTREAM Provider shipped by SQL team.Anonymous
November 02, 2010
Hi, Can you please provide the clear steps for Installing & configuring Remote BLOB Storage (RBS) with the FILESTREAM provider (SharePoint Server 2010) FOR MORE THAN ONE CONTENT DATABASES (WEB-APPLICATIONS) I am able to successfully Install and Configure for one (1) Content Database smoothly by referring to technet.microsoft.com/.../ee663474.aspx But, repeating for the more than one (1) Content Databases like; repeating for the 2nd and 3rd...., is failing; Failing in the sense, when I run; msiexec /qn /i RBS_X64.msi REMOTEBLOBENABLE=1 FILESTREAMPROVIDERENABLE=1 DBNAME=”Content_Database_2” DBINSTANCE=”LOCALHOST” ADDLOCAL=EnableRBS,FilestreamRunScript FILESTREAMSTORENAME=FilestreamProvider_2 will simple open a msiexec window with some help; You can also see my thread @ social.msdn.microsoft.com/.../77355a32-e94a-4b37-82cb-3a00e6ced531 Can you please guide me completely; how to do for more than One Content Database(s). Thanks in Advance, Regards, Venkat.KLAnonymous
February 09, 2011
Set a content database to use Remote BLOB Storage technet.microsoft.com/.../ee748641.aspx Regards, Kazim RazaAnonymous
February 14, 2011
Jie, I am trying to get a definitive answer regarding backup/restore of BLOB stores. I refer to the following white paper: download.microsoft.com/.../SQL_Server_2008_R2_Remote_Blob_Storage.docx Referring to page 41 of the white paper, there is a note at the bottom of the page which reads: "if LOCAL RBS FILESTREAM Provider is being used, content database backup operations will also backup the BLOBs at the same time which simplifies the backup and restore process". However referring to your blog post this is not the case: "If you are trying to backup the database by using SQL Backup, keep in mind this will not backup the BLOB objects. You need to manually backup the database first, then manually backup (XCOPY or other file backup tools) these BLOBs. When restore, using a reverse order to restore BLOBs first, then the databases." Are you able to answer this question, or can you direct this question at the relevant person? Regards, Conrad GoodmanAnonymous
March 18, 2011
Great Post Jie... Very informative! @Tim, at AvePoint we have created a product called Archiver that does exacltly this-- you can set up arciving of content to RBS storage using criteria based around your business logic (Metadata column, last accessed time, content type etc)... ping me if you want more info...Anonymous
April 13, 2011
The comment has been removedAnonymous
April 25, 2011
Hi Jie, Thanks for your post. A question, there are lots of services in SharePoint 2010, so what kind of service needs BLOB provider installed?Anonymous
July 05, 2011
I have a question about the use of Migrate() method. In the given example the method is applied directly to a database object, but it seems to be supported on the rbs object instead. For example: $rbs.migrate(). There is a difference between the two examples or they obtain the same result?Anonymous
August 19, 2011
The comment has been removedAnonymous
September 01, 2011
"Great post to start with", indeed. How does SharePoint (using RBS) handle document versioning? Does every version of a document (minor & major) get a different guid? Are obsolete versions automatically deleted (if versions are limited to a specific number)? many thanksAnonymous
September 11, 2011
Q: Any 3rd party RBS provider I can use? I would like to add more in it, like other 3rd party tools StorageEdge also provides both EBS and RBS providers for SAN, NAS and Cloud storage and helps you reduce your SharePoint content database size up to 95%. StorageEdge gives you great control over the blob externalization where you can externalize blobs based on certain criteria and organize them in various folder structures. StorageEdge keeps the content database and the externalize content synchronized and provides full management for orphan content. It also helps you migrate from EBS to RBS. Furthermore, it allows for hierarchical multi-tiered storage such as local storage, NAS and SANs. More info can be found on www.alachisoft.com/.../storageedge-express.html Anywazz.. overall its a very nice entry :) MSDN especially is a nice source where we can easily find such information..Anonymous
November 01, 2011
There are many Intranets solutions on the market, however if you are looking for a solid SharePoint based Intranet solution, check out SharePoint Implementeds’ product. I think it’s the best solution out for the price. They seem to have put a lot of thought into usability and filling in gaps that you would not know exsist in sharepoint until you start your implementation. They offer a turnkey solution which provides a custom Home Site, Department Sites and Project Sites, installation, configuration and training all under $10,000 and even have a source code option. One thing that I would love to see that they dont have now is a hosted solution You can get more details at sharepointimplemented.com/AwesomeIntranetGorilla.htmlAnonymous
December 08, 2011
Great post, but what I'm missing is the theoretical reasoning behind using RBS. At what point regarding file sizes should you consider implement RBS?? Technet (post technet.microsoft.com/.../ff628569.aspx) has the following section:
Most optimal use of RBS Because RBS is a solution created for a specific set of conditions, there is an optimal use of RBS in which the benefits outweigh the costs. The optimal environment for using RBS is an environment where the following is true: You want to store fewer large BLOBs (256 KB or larger) for read-intensive or read-only access."
Is then 256 KB the best practice guidance on when to use RBS??
Anonymous
April 26, 2012
Awesome..!Very informative article...!Anonymous
May 21, 2012
Can anyone tell me how to deploy the custom provider library on sharepoint server?Anonymous
September 05, 2012
We have a scenario where we are having lacs of documents and images. We are using SharePoint 2010 Foundation with SQL Server Standard Edition as the backend. Whether it is fine to use RBS for storing the documents out of database. If we want to move the documents to production, how can we do that?Anonymous
November 21, 2012
Hi Jie, Thanks for very informative post. I am having certain questions regarding RBS implementation
- Can we use share drives or UNC paths for RBS and how?
- How can we configure the size above which a document will be sent to RBS?
- Is is possible to Configure RBS only for specific Document Type or document library?
Anonymous
August 27, 2013
I'm trying to test back and restore of content db with RBS enabled. Has anybody tried to restore a RBS enabled content database backup from one server to another server. I'm testing this by moving a RBS content database from my stage server to my dev server. My stage server is setup to store blob files in D: drive. However in my dev environment D: drive letter is utlized by the CD drive. I like to use the F: drive for blob files. How can I restore the database backup such that the blob files goto a folder in the F: drive.Anonymous
April 07, 2014
How does migration from SharePoint 2010 to 2013 work if you have RBS enabled, are there any gotchas to be aware of?Anonymous
July 28, 2014
I have a web application having 108 site collections in it. I'm using custom RBS provider from codeplex and its working as expected. Now, I need to enable RBS only to specific site collection and not for all site collections inside my WebApplication. Is there any way to achieve this ? Any immediate help will be highly appreciated.Anonymous
September 26, 2014
I have installed and configured RBS in SharePoint 2010 environment. I can see the uploaded files in the blob store but I see a duplicate file every time I update the metadata of the file in the SharePoint library. And also a duplicate file whenever a new version of file is added. Is this an issue or a default behavior of RBS?. If this is a default behavior, why do we see an additional file in the blob store with the same size of the file?Anonymous
April 08, 2015
Does RBS support SAN level Mirroring?