Create a Simple Data Collector to Integrate Data into Load Test Analyzer (Tables View)
With a custom data collector, you can collect various diagnostic performance information from your Web applications. In my previous post, I have highlighted the key points of integrating custom collector data into Load Test Analyzer. The purpose of this post is to go through the procedure by using a simple collector.
One of the questions some forum users asked is ‘Did Load Test insert item records into my table?’ To find out this, I add a load test containing a Web test that purchases orders from a online store (Web application) to my test project. I create a SimpleRowCountDataCollector to collect the counts of the Order table before and after the Load test run. Then I enable the data collector in a .testsettings file, run the Web test in a Load test, and check the row counts after the run completes.
Since this sample data collector can be downloaded from https://teamtestplugins.codeplex.com/, I will only outline the steps in this post. If you would like more detailed instructions, read this page.
Step 1 – Create a Data Collector
1. Create a new class library
2. Add references to the following dlls
Microsoft.VisualStudio.QualityTools.ExecutionCommon
Microsoft.VisualStudio.QualityTools.Common.dll
3. Add RowCountDataCollector.cs
Specify the URI and a friendly name for the collector
[DataCollectorTypeUri("datacollector://MyCompany/SimpleRowCountDataCollector/1.0")]
[DataCollectorFriendlyName("RowCountDataCollector", false)]
[DataCollectorConfigurationEditor("configurationeditor://Microsoft/GenericEditor/1.0")]
public class SimpleRowCountDataCollector : DataCollector
Below are the brief descriptions of the major functions.
3.1 Initialize() function
This function does three thing
- Init data members
- Retrieve the collector configuration from the test settings file
- Register the TestCaseStart and TestCaseEnd events
// Required method called by the testing framework
public override void Initialize(
XmlElement configurationElement,
DataCollectionEvents events,
DataCollectionSink sink,
DataCollectionLogger logger,
DataCollectionEnvironmentContext environmentContext)
{
// 1. Init members
dataEvents = events; // The test events
dataLogger = logger; // The error and warning log
dataSink = sink; // Saves collected data
// Configuration from the test settings
configurationSettings = configurationElement;
// 2. Read collector configurations from the test settings
CollectorConfigurationManager configManager = new CollectorConfigurationManager(configurationSettings);
server = configManager["server"];
if (string.IsNullOrEmpty(server))
{
server = Environment.MachineName;
}
database = configManager["database"];
table = configManager["table"];
// 3. Register TestCaseStart and TestCaseEnd events for the data collector
dataEvents.TestCaseStart +=
new EventHandler<TestCaseStartEventArgs>(OnTestCaseStart);
dataEvents.TestCaseEnd +=
new EventHandler<TestCaseEndEventArgs>(OnTestCaseEnd);
}
3.2 OnTestCaseStart Event Handler
This function get the order count from the Order table right before the load test is started. As I mentioned in the previous post, you can log warning or error messages in the code. The message will show up in the Load Test Errors Table.
public void OnTestCaseStart(object sender, TestCaseEventArgs e)
{
// dataLogger.LogWarning(e.Context, "OnTestCaseStart. Time:" + DateTime.Now.ToString());
rowCountBeforeTestRun = SQLHelper.GetRowCount(server, database, table);
}
3.3 OnTestCaseEnd Event Handler
This function get the order count from the Order table after the load test is completed, write the tabular data in XML format, and send the file back to client (VS).
// Write the counts to a table in XML format
public void OnTestCaseEnd(object sender, TestCaseEndEventArgs e)
{
// Get the row count after run ends
rowCountAfterTestRun = SQLHelper.GetRowCount(server, database, table);
// create the table to be integrated into load test database
string filePath = Path.Combine(Path.GetTempPath(), "rowCounts.xml");
RowCountXMLWriter.CreateDataTable();
RowCountXMLWriter.AddDataRow("BeforeTestStart", rowCountBeforeTestRun);
RowCountXMLWriter.AddDataRow("AfterTestEnd", rowCountAfterTestRun);
RowCountXMLWriter.WriteRowCountsToXmlFile(filePath);
// Send the file back
dataSink.SendFileAsync(e.Context, filePath, false);
}
One important thing to keep in mind is that only files sent back in OnTestCaseStart and OnTestCaseEnd events can be added to Load Test Tables view in post run.
4. Add RowCountXMLWriter.cs file
This class creates a DataTable, add rows to the table, and write out the data in XML format
static DataTable m_dataTable;
public static void CreateDataTable()
{
m_dataTable = new DataTable();
m_dataTable.Locale = CultureInfo.InvariantCulture;
m_dataTable.Columns.Add(new DataColumn("Description", typeof(string)));
m_dataTable.Columns.Add(new DataColumn("RowCount", typeof(Int32)));
}
public static void AddDataRow(string description, int rowCount)
{
DataRow row = m_dataTable.NewRow();
row["Description"] = description;
row["RowCount"] = rowCount;
m_dataTable.Rows.Add(row);
}
public static void WriteRowCountsToXmlFile(string xmlFilePath)
{
DataSet dataSet = new DataSet();
dataSet.Locale = CultureInfo.InvariantCulture;
dataSet.Tables.Add(m_dataTable);
using (System.IO.StreamWriter xmlStreamWriter = new System.IO.StreamWriter(xmlFilePath))
{
dataSet.WriteXml(xmlStreamWriter, XmlWriteMode.WriteSchema);
xmlStreamWriter.Close();
}
dataSet.Dispose();
}
5. Add ConfigurationManager.cs
This class reads the data collector settings into a dictionary. Please note, it is written for the format described in Step 6, section <DefaultConfiguration>.
6. Add an App.config file
Note this data collector is configured to use a Generic XML editor. If you want to create your own UI editor, read this MSDN page.
[DataCollectorConfigurationEditor("configurationeditor://Microsoft/GenericEditor/1.0")]
public class SimpleRowCountDataCollector : DataCollector
[App.config]
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSection>
<section
name="DataCollectorConfiguration"
type="Microsoft.VisualStudio.QualityTools.Execution.DataCollectorConfigurationSection,
Microsoft.visualStudio.QualityTools.ExecutionCommon,
Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a" />
</configSection>
<DataCollectorConfiguration xmlns="https://microsoft.com/schemas/VisualStudio/TeamTest/2010">
<DataCollector typeUri="datacollector://MyCompany/SimpleRowCountDataCollector/1.0">
<DefaultConfiguration>
<Setting name="server" value="YourWebServer" />
<Setting name="database" value="Store" />
<Setting name="table" value="Order" />
</DefaultConfiguration>
</DataCollector>
</DataCollectorConfiguration>
</configuration>
This data collector config file allows you to specify three settings, Web server, database, and table name.
Step 2 – Install the Data Collector
You need to drop a copy of the data collector on every machine you want to collect data.
Setup Requirement: You must install Agent on these machines.
Files to drop: RowCountDataCollector.dll, RowCountDataCollector.dll.config
Drop location: %Program Files%\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\DataCollectors
An example of the setup
Machine1: VS2010 Ultimate, Controller, Agent
Machine2: Web Server, Agent, RowCountDataCollector.
For you to try out the sample collector, you can drop it on your local machine, configure it to point to a table in your local load test database (step 3), and then run the test(step 4).
Step 3 – Configure the Data Collector
Open the Testsetting file used by your load test run. Click on Data and Diagnostics. Enable the RowCountDataCollector.
Click on Configure, it will launch the default editor for you to modify the settings.
Step 4 – Run your load test and view the data
Run your load test view the before/after order counts in post run table.
Comments
Anonymous
April 20, 2010
Hi, It would be great if all VS test web and load could also be cross-posted in the Visual Studio Team Test blog. There are too many blogs to monitor. You should view all the test tools as one product (we use manual, coded UI, web/load testing portion from the tool, for those of us using VS Ultimate and doing multiple types of testing in our overall test strategy) BertrandAnonymous
October 26, 2010
Hi YuTong, First of all, thank you for this article. It's excellent. I have a question. Is it possible to use your custom data collector inside MTM (Microsoft Test Manager)? I need to add the new custom log file automatically in "Attachments" section right after the test was done along with other logs (Event Log and so on). Thank you in advance. Regards, GoranAnonymous
October 26, 2010
Hi YuTong, First of all, thank you for this article. It's excellent. I have a question. Is it possible to use your custom data collector inside MTM (Microsoft Test Manager)? I need to add the new custom log file automatically in "Attachments" section right after the test was done along with other logs (Event Log and so on). Thank you in advance. Regards, GoranAnonymous
October 26, 2010
The comment has been removedAnonymous
November 11, 2015
Hi Get I get example for section 5. Add ConfigurationManager.cs? Thanks