Synchronizing with Windows SharePoint Services, Part 1

Artikel
01/21/2008

Introduction

There may be occasions where you will need to synchronize third party programs with Windows SharePoint Services. GetListItemChangesSinceToken, introduced in Windows SharePoint Services version 3.0, makes this synchronization much more efficient.

This post explores using GetListItemChangesSinceToken to synchronize with Windows SharePoint Services version 3.0. The article details the queryOptions used by GetListItemChangesSinceToken.

The follow on post will discuss conflict detection issues; and document best practices for clients who want to synchronize with calendar items, documents, and tasks in Windows SharePoint Services version 3.0.

Please forgive the length of this entry, the various tables and descriptions I've included will be in future updates of the SDK.

Synchronization Using GetListItemChangesSinceToken

You can use GetList to download a list; however, the most efficient way to synchronize information is to download only those items that have changed since the last synchronization occurred. In Windows SharePoint Services version 3.0 this can be done by calling the GetListItemChangesSinceToken Web method.

Sending a GetListItemChangesSinceToken request without including a token returns the list schema, the full list contents and a token. The token represents the moment when those changes were requested. By including this token the next time you call GetListItemChangesSinceToken, the server returns only those changes that have occurred since the token was generated.

If the list schema itself has changed, GetListItemChangesSinceToken returns the entire list schema, the full list contents, and a token.

#GetListItemChangesSinceToken Request and Response

Here is a listing of the GetListItemChangesSinceToken Request and Response and an explanation of the very useful queryOptions parameter.

Request

public SoapXml.SoapXmlElement GetListItemChangesSinceToken(

string listName,

string viewName,

SoapXml.SoapXmlElement query,

SoapXml.SoapXmlElement viewFields,

string rowLimit,

SoapXml.SoapXmlElement queryOptions,

string changeToken,

SoapXml.SoapXmlElement contains)

Name	Description
listName	This can be the list title, but using the list id (GUID) results in better performance.
viewName	Not supported.
query	A CAML query similar to the CAML query used in GetListItems and SPQuery. Refer to GetListItems and SPQuery in the SDK for more information. Note Not intended to be used with the contains parameter.
viewFields	Set of field references for each field required in the response. <ViewFields /> returns all fields in the list. A Properties='true' attribute will separate the MetaInfo field into its separate decoded properties.
rowLimit	The maximum number of values to return. Used for paging.
queryOptions	Set of options related to the query (see details below).
changeToken	An opaque token used to determine the changes since the last call. This token should never be parsed or constructed, as its format may change in the future.
contains	A CAML filter applied to the query results. This is the Where clause in a SPQuery. Note Not intended to be used with the query parameter.

Query Options Parameter

The queryOptions element can contain a variety of tags which modify the query. For Boolean options the default is FALSE, only a value of TRUE (must be uppercase) enables them.

Available Query Option Tags

Query Option Tag	Description
<Paging ListItemCollection PositionNext="X" />	X is an opaque token used to determine the page of items to return. Like the changeToken value, this should never be parsed or constructed.
<IncludeMandatory Columns>	Ensures that fields defined as required are included, even if not specified in the viewFields. This option may be misleading because Windows SharePoint Services actually has a separate set of mandatory fields that are always returned independently of this option.
<RecurrenceOrderBy>	This is a requirement for some calendar programs. For each recurring series, the master item is returned first and then all exceptions. This is a special internal ordering that is applied ahead of any other ordering. Note This should not be used unless your program explicitly requires it. If the view has a field of type Recurrence, the list will be ordered by fields of reference type UID, EventType and StartDate in the definition of the recurrence field.
<ExpandUserField>	Special rendering for the user field values that makes them include the login name, email, SipAddress, and the title when present. This causes a user field to behave as a multi lookup field. The lookup fields used in the expansion are "Name", "EMail", "SipAddress" and "Title". The values are separated by ,#. Any commas in the lookup field name are encoded as ,,. These values occur in the normal field data for each item.

Examples of ExpandUserField

<ExpandUserField>FALSE</ExpandUserField> looks like:

ows_Author="1;#Admin AdminName"

<ExpandUserField>TRUE</ExpandUserField> looks like:

ows_Author="1;#Admin AdminName,#login\name,#email@address,#sip@address,#Admin AdminName "

More Query Option Tags

Description

< DateInUtc>

Date fields are returned in UTC format and zone.

UTC is in GMT time zone an ISO6801 format: 2006-10-04T10:00:00Z

Normal is in local (server) time zone and the same as above with the T and the Z replaced by spaces.

<ViewAttributes

foo=bar />

All attributes of this option will be used as view attributes of the view schema. The most commonly used is Scope="RecursiveAll" that defines the view as flat instead of scoped to a particular folder.

Set the root folder scope of the view. This is a server relative URL.

<Folder>Shared Documents/foldername</Folder>

Defines which meeting instance to sync with if this list is part of a meeting workspace. -1 should be used for all instances unless the client wants to filter for a particular instance.

One way a client can request individual item permissions.

Changes the value returned for the Attachments field from a Boolean to a list of full urls separated by ;#

<IncludeAttachment

Version>

Used in conjunction with IncludeAttachmentUrls, IncludeAttachmentVersion also returns the GUID and version number used for conflict detection on update.

<RecurrencePattern

XMLVersion>

</RecurrencePattern

XMLVersion>

Used to maintain backwards compatibility, RecurrencePatternXMLVersion changes the value of a RecurrenceData field to NOT return <V3RecurrencePattern /> when it contains elements only present on a version 3 pattern.

Without this tag, recurrence patterns that were not present in Windows SharePoint Services version 2 are sent as <V3RecurrencePattern />. Including this tag means that recurrence patterns new to Windows SharePoint Services are sent correctly.

1,4,23

</ExtraIds>

Request extra items to be included on the returned set regardless of whether they changed or not. The common use of ExtraIds is to specify the IDs of the folders you're syncing if you were in a doclib and chose "connect to..." on a folder rather than on the entire doclib. This way you get the folder name and can tell when it is renamed.

Note This should only be used with a change token.

This allows a client to sync to one or more folders and detect if any folder above the hierarchy was deleted or renamed.

Folder names are not returned unless some changes are done to the list and the query to fetch changed items also uses IDs.

The two values supported are:

ItemIds
FolderUrls

ItemIds is the default as long as a query or recurrence order is not requested and optimizes our SQL query with an ID order.

FolderUrls optimizes a sync filtered to the flat contents of one or more folders by optimizing the SQL query with a DirName, LeafName order.

<OptimizeFor>ItemIds</OptimizeFor>

Default Query Options

If the queryOptions parameter is absent, the following default options are used:

Note Clients should always specify a query option; failing to do so will negatively impact scale performance.

Response

A GetListItemChangesSinceToken response is delivered in the following format:

[Changes]

</Changes>

<rs:data ItemCount="N" [ListItemCollectionPositionNext="X"]>

[Data]

</rs:data>

</listitems>

Note X is an opaque token used to determine the page of items to return. Like the changeToken value, this should never be parsed or constructed.

Namespace declarations

Any XML namespaces used below are declared here.

List and global properties

These include configuration properties, alternate URL information, and list permissions.

List and global properties table

Property	Definition
MinTimeBetweenSyncs	A server parameter that represents the minimum amount of time between user-initiated or automatic synchronization. The value represents a time in minutes. Note Clients should respect this value even if the user initiates synchronization manually. That is, if this is set to 5 minutes, clients should only send one request per 5 minutes even if the user repeatedly clicks "send/receive".
RecommendedTime BetweenSyncs	The recommended minimum amount of time between synchronizations. This should specifically be respected for automatic syncs. Clients should never automatically synchronize more often than this. User-initiated synchronizations can override this interval.
MaxBulkDocumentSyncSize	The total size of content to be synchronized to the client. The default is 500 MB. You get the URL and metadata for each document when you call GetListItemChangesSinceToken, but you then need to do an HTTP GET to retrieve the actual document contents. Setting this value to high may degrade performance.
AlternateUrls	Alternate URLs are listed in the following zone order, delimited by commas: Intranet,Default,Extranet,Internet,Custom
EffectiveBasePermissions	The permissions on the list as returned by SPList.EffectiveBasePermissions.ToString().

Changes

Changes consist of a list of change events that need to be specially handled by the client.

These are taken from our internal change log when a change token is supplied. For a full synchronization (no change token) we still return the Changes tag with the current change token.

The limit on the number of updates returned from a change token is 100. The MoreChanges attribute indicates that the last change token is not current. More changes were done on the list and the client should call this method again with the new change token.

Change	Description
<List></List>	If the list schema has changed, or if no change token was provided, we return the list here. The format is the same as returned by GetList.
<Id ChangeType= "InvalidToken" />	The token is either invalid or old. You must do a full sync.
<Id ChangeType="Restore" />	The list has been restored from the recycle bin or from a backup. You should do a full sync.

In both of the cases above, the client should ignore other changes and do a full reconciliation of the list.

Change Type	Description
<Id ChangeType= "Delete">ID</Id>	This item is no longer present. Note that Delete changes are sent even if the item was filtered out by the query.
<Id ChangeType="MoveAway" AfterListId="ID" AfterItemId="ID">ID</Id>	Treat in the same manner as a delete.
<Id ChangeType="Restore"> ID </Id>	This item and any items beneath it were restored.
<Id ChangeType= "SystemUpdate">ID</Id>	Some clients may use hidden version, version history or modified time to determine whether to update an item. A SystemUpdate means Windows SharePoint Services has made changes and that you need to update all properties on that particular item.
<Id ChangeType="Rename">ID</Id>	Just like SystemUpdate, renamed items may retain hidden version information.

Data

The ItemCount attribute contains the number of items returned. Each Item is returned as a <z:row> tag.

A ListItemCollectionPositionNext attribute is returned only for a full sync (no change token) when a row limit parameter was used to limit the number of items returned in one call. This type of paging is blocked on an incremental sync by restricting the number of updates processed from the change log to the row limit if smaller than our internal maximum.

The contents of the items are returned as attributes with an "ows_" prefix and the internal name of the field. The values are encoded to be valid XML attribute values. A set of fields is always returned.

Some fields are marked as required and returned if the IncludeMandatoryColumns option is set. If <ViewFields /> is sent, all fields are returned.

When Properties="TRUE" is set in the ViewFields tag, the property bag (MetaInfo field) is separated into an attribute per property. These have a prefix of "ows_MetaInfo_".

Most of the column values are simply converted from their internal representation, while others are constructed specially for a client. Several attributes are not represented by a field in the list schema.

Table: List of attributes not represented by a field in the list schema.

Attribute	Description
MetaInfo	This is the property bag container, SPListItem.Properties. For more information, refer to the property bag in the SPListItem object model.
Fields of type "Attachments"	This is a bit column in the database, but query options modify it to return attachment data.
Recurrence Data	This is the XML definition of a recurrence.

See https://blogs.msdn.com/sharepoint/archive/2007/05/14/understanding-the-sharepoint-calendar-and-how-to-export-it-to-ical-format.aspx for more details on this XML

Acknowledgements

I would like to acknowledge the following persons for their gracious help in technical reviews for this article: Matt Swann (Microsoft Corporation), Bill Snead (Microsoft Corporation).

Comments

Anonymous
January 21, 2008
PingBack from http://clanad.musicnewsandviews.com/2008/01/21/synchronizing-with-windows-sharepoint-services-part-1/
Anonymous
January 21, 2008
There is an interesting post over at blogs.msdn.com
Anonymous
January 22, 2008
SharePoint Services 3.0 - Synchronization
Anonymous
February 01, 2008
Introduction In my last two posts I discussed how third party clients can most efficiently synchronize
Anonymous
February 01, 2008
Tools Master Page Commands for MOSS STSDEV: Simple Tools for SharePoint 2007 Development Access Checker
Anonymous
February 01, 2008
Tools Master Page Commands for MOSS STSDEV: Simple Tools for SharePoint 2007 Development Access Checker
Anonymous
March 13, 2008
How do you specify the row limit to return all rows? Currently if you enter: string rowLimit = "" //returns 100 items I need to return all the items so I coded: string rowLimit = "100000" //I know I know lists should not be more then 2000 items, but I am supporting a business and eventually they will listen to me. However, is there a way to specify return all rows?
Anonymous
June 05, 2009
Hi, When I was working to get powershell scripts from MSDN samples to get/set information, I created

Freigeben über