Condividi tramite


Protocol Test Suites (Part Two)

This is the second part of the two-part series on Open Specifications Protocol Test Suites. In the previous post, we gave an overview of Test Suites and their different components. Now let's go over the prerequisites of a Test Suite environment, the typical contents of a Test Suite package, cover the end-to-end process of configuring and running Test Suites, and analyze the results.

Setting up a Test Suite Environment

The following table lists the tools and software necessary to set up your environment for Exchange and SharePoint Test Suites.

Note: For information on setting up a test environment for a specific Test Suite, refer to the Test Suite Reference Guide for that Test Suite. For more information on the Reference Guide and other contents of a Test Suite folder, see the subsequent section.

Machine Test suite environment (Exchange) Test suite environment (SharePoint)
*SUT
  • Microsoft Exchange Server 2007
  • Microsoft Exchange Server 2010
  • Microsoft Exchange Server 2013
 
  • Microsoft Office SharePoint Server 2007
  • Microsoft Windows SharePoint Service 3.0
  • Microsoft SharePoint Server 2010
  • Microsoft SharePoint Server 2013
  • Microsoft SharePoint Foundation 2010
  • Microsoft SharePoint Foundation 2013
 Client  
  • Operating systems: 64-bit versions of Windows 2008 R2 SP1 or Windows 7 SP1
  • Microsoft Visual Studio 2010 Ultimate
  • Protocol Test Framework (PTF)
  • Spec Explorer (Only supports Visual Studio 2010 and Visual Studio 2012)
  • Message Analyzer
  • Microsoft Forefront Protection 2010
  • Microsoft SQL Server 2008 R2
  • Microsoft SQL Server 2008 R2 SP1

* SUT may be replaced in a non-Microsoft implementation

Using a Test Suite

When you download a Test Suite, the package is downloaded as a compressed folder in .zip format. When you unzip the folder, you will see several different folders and subfolders. It is helpful to understand what each one of these folders contains.

Folder structure and contents

A Test Suite folder mainly consists of configuration files (both client and server-side), test cases, and associated supporting files. These can include scripts, methods, properties, source code, technical specifications for protocols associated with the Test Suite, and end-user help documentation such as a Reference Guide, a Test Suite Deployment Guide, and a Test Suite Specification. The folder also contains a Requirements Specification spreadsheet (a list of normative requirements extracted from the technical document) for each protocol associated with the Test Suite.

The following diagram shows how the contents of a Test Suite folder are organized:

Figure 1: Test Suite Folder Structure

Note: For more information on each component in a specific Test Suite folder, refer to the Test Suite Deployment Guide for that Test Suite.

Running Test Suites

The following illustration shows the basic steps of running a Test Suite.

Note: Information for running a specific Test Suite is available in the Test Suite Deployment Guide for that Test Suite, found in the Test Suite folder.

Figure 2: Running Test Suites

Note: The Test Suite Deployment Guide contains detailed guidance on how to configure the client and the SUT, run test cases, and view debug reports.

Analyzing the Test Results

After a Test Suite has run successfully, log files and reports are generated. These files enable you to identify errors and determine what exactly failed in the validation. The following types of files are available for analysis:

  • TRX log: A log file produced by Visual Studio. This file contains information about how many test cases were run, and pass/fail results for each test case.
  • HTML report: Produced alongside the TRX log, this report contains the verification status (pass/fail) of the protocol requirements.
  • XML log: A log created by the Protocol Test Framework (PTF).
    Note: The Protocol Test Framework (PTF) is one of the client components of the test environment described in the first section of this post. The PTF is designed to provide support for the Microsoft Protocol Test Suites for Office and Windows Protocol Interoperability testing. PTF enables you to develop custom Test Suites for your implementation, and provides an interface called ITestSite to represent the environment for Test Case execution.

 

Figure 3: A sample TRX log file

Where to Get the Test Suites

You can download the Exchange, Web Application Open Platform Interface (WOPI), and SharePoint Test Suites from the Test Suites page on Microsoft Connect. This site is also where you can download the Protocol Test Framework (PTF) and Message Analyzer.

Currently, the following Test Suites are available for download.

Note: More Test Suites are always being added, so this list may become out of date. Visit the Microsoft Connect Test Suites page for the most current selection.

Test Suite Applicable protocols Protocol Name
Exchange ActiveSync (EAS) MS-ASAIRS Exchange ActiveSync: AirSyncBase Namespace Protocol
  MS-ASCAL Exchange ActiveSync: Calendar Class Protocol
  MS-ASCMD Exchange ActiveSync: Command Reference Protocol
  MS-ASCNTC Exchange ActiveSync: Contact Class Protocol
  MS-ASCON Exchange ActiveSync: Conversations Protocol
  MS-ASDOC Exchange ActiveSync:  Document Class Protocol
  MS-ASDTYPE Exchange ActiveSync: Data Types
  MS-ASEMAIL Exchange ActiveSync: Email Class Protocol
  MS-ASHTTP Exchange ActiveSync: HTTP Protocol
  MS-ASNOTE Exchange ActiveSync: Notes Class Protocol
  MS-ASPROV Exchange ActiveSync: Provisioning Protocol
  MS-ASRM Exchange ActiveSync: Right Management Protocol
  MS-ASTASK Exchange ActiveSync: Tasks Class Protocol
  MS-ASWBXML Exchange ActiveSync: WAP Binary XML (WBXML) Algorithm
Exchange Web Services (EWS) MS-OXWSATT Attachment Handling Web Service Protocol
  MS-OXWSBTRF  Bulk Transfer Web Service Protocol
  MS-OXWSCONT Contacts Web Service Protocol
  MS-OWXWSCORE Core Items Web Service Protocol
  MS-OXWSFOLD Folders and Folder Permissions Web Service Protocol
  MS-OXWSMG Email Message Types Web Service Protocol
  MS-OXWSMTGS Calendaring Web Service Protocol
  MS-OXWSSYNC Mailbox Contents Synchronization Web Service Protocol
  MS-OXWSTASK Tasks Web Service Protocol
Exchange Remote Procedure Call (RPC) MS-OXCFOLD Folder Object Protocol Specification
  MS-OXCFXICS Bulk Data Transfer Protocol Specification
  MS-OXCMSG Message and Attachment Object Protocol Specification
  MS-OXCPERM Exchange Access and Operation Permissions Protocol Specification
  MS-OXCPRPT Property and Stream Object Protocol Specification
  MS-OXCROPS Remote Operations (ROP) List and Encoding Protocol Specification
  MS-OXCRPC Wire Format Protocol Specification
  MS-OXCSTOR Store Object Protocol Specification
  MS-OXCTABL Table Object Protocol Specification
  MS-OXNSPI Exchange Server Name Service Provider Interface (NSPI) Protocol
Exchange MAPI MS-OXCMAPIHTTP Messaging Application Programming Interface (MAPI) Extensions for HTTP
  MS-OXCNOTIF Core Notification Protocol
  MS-OXORULE Email Rules Protocol
SharePoint Server Protocols MS-ADMINS Administration Web Service Protocol
  MS-AUTHWS Authentication Web Service Protocol
  MS-COPYS Copy Web Service Protocol
  MS-CPSWS SharePoint Claim Provider Web Service Protocol
  MS-DWSS Document Workspace Web Service Protocol
  MS-LISTSWS Lists Web Service Protocol
  MS-MEETS Meetings Web Service Protocol
  MS-OFFICIALFILE Official File Web Service Protocol
  MS-OUTSPS Lists Client Sync Protocol
  MS-SHDACCWS Shared Access Web Service Protocol
  MS-SITESS Sites Web Service Protocol
  MS-VERSS Versions Web Service Protocol
  MS-VIEWSS Views Web Service Protocol
  MS-WDVMODUU Office Document Update Utility Extensions
  MS-WEBSS Webs Web Service Protocol
  MS-WSSREST ListData Data Service Protocol
  MS-WWSP Workflow Web Service Protocol
SharePoint File Sync and WOPI Protocols MS-WOPI Web Application Open Platform Interface Protocol
  MS-FSSHTTP File Synchronization via SOAP over HTTP Protocol
  MS-FSSHTTPB Binary Requests for File Synchronization via SOAP Protocol

Resources and More Information

The following resources provide more information on Test Suites and Open Specifications.