Bluetooth Performance Measurement - Part I

Introduction
Doing performance measurements on a regular basis is important while working on a Bluetooth transport driver. Poor performance can cause general slowness in certain applications (i.e. virtual serial connections) or cause other  profiles (i.e. A2DP) to become completely unusable.

The Bluetooth Performance test in the CETK measures throughput over RFCOMM, the protocol used by the Serial Port Profile. The test exercises L2CAP, the transport driver and the baseband as well, so problems in these areas will have a direct impact on the final results. Other protocols such as AVDTP (used in A2DP profile) and BNEP (used in PAN profile) are not exercised  at all and should be tested separately.  A diagram of the stack and related protocols can be found here.

The test is contained in one single tux dll (perf_bluetooth.dll) that must be launched on two separate CE devices  at the same time: the client on the device under test (DUT) and the server on an auxiliary device. 

The  test operates as follows:

1. DUT connects to server
2. DUT  sends a fixed amount of data multiple times, using a different buffer size each time
3. DUT receives a fixed amount of data multiple times, using a different buffer size each time
4. DUT disconnects
5. If several iterations were specified in the command line, goto (1) for the next iteration

For each basic operation such as data transfer or connection/disconnection, the elapsed interval in milliseconds will be recorded. The final results file will be stored in the DUT and will contain the following matrix of data points:

# datapoints = (Send, Receive) x (various buffer sizes (16, 512, 1024, 2048, random)) x iterations

One advantage of this methodology when compared to a simple file transfer using Obex or PAN is that we don’t exercise other components such as the file system, NDIS or other higher level protocols, so potentially unrelated bottlenecks can be eliminated.

Test execution
In order to run this test, you must be able to copy and run command line executables on a CE device (If you don’t know how to do this, please read the “Additional resources” section at the end of this post). I also assume that CETK is installed on your workstation machine and that you can locate its install directory (i.e. “C:\Program Files\Platform Builder for Windows Mobile\5.00\CEPB\wcetk”).

Here is the list of command line options recognized by Perf_bluetooth, and they are all required:

-b <bytes>: specifies the fixed number of bytes that will be transferred between client and server during a send or receive. The default is 163840 and there's generally no need to change it.

-i <number_of_iterations>: specifies the number of iterations. A performance report can be obtained with one single iteration, but a higher number of iterations will provide more data points.

-p <RFCOMM_channel_number>: specifies the RFCOMM channel number. You can pick any value between 1 and 30, but always verify that the selected channel is not in use by other applications. The same value must be used on both client and server.

-s <server_bluetooth_addr>:  specifies the server’s Bluetooth address and is used on the client side only.

With this information, we’re now ready to launch the test:

1) Verify that both the DUT and auxiliary device are in close range, and that Bluetooth is turned on and operational on both devices. There is no need to put the devices in discoverable mode, because the DUT will never need to perform an inquiry to find the server.

2) Locate the following files under the CETK install directory and copy them to your device (or to the flat release directory if you are using Platform Builder). Please verify that you are selecting the correct binaries for your CPU architecture.

Tux.exe
Tooltalk.dll
Kato.dll
Perflog.dll
Perf_bluetooth.dll

3) Use the following command line to launch the server on the auxiliary CE device :

tux -o -d perf_bluetooth -c"-i 10 -p 6 -b 163840"

Get the Bluetooth address of the server device by looking at the debug output generated.  There will be text similar to " localhost<080028effafa> listening on port 6(0x6)...".  The Bluetooth address is the number in <brackets>.

4) Now use your server’s Bluetooth address in the following command and run it on the client (DUT):

tux -o -d perf_bluetooth -c"-i 10 -p 6 -b 163840 -s 080028effafa "

This test will run 10 iterations  and will take several minutes to complete. Keep an eye on CPU usage while running it to ensure that the processor is not a bottleneck (If you don’t know how to do this, please read the “Additional resources” section at the end of this post). At the end of the test, the results can be found in the DUT ’s root folder (or \Release if connected via KITL). The file generated during my test run was  called  “perf_bluetooth_NOPLAT_ARMV4I_VER5_1_204.LOG” and yours should have a similar name.

We’ll see in the next post how to analyze the results on the desktop workstation.

Additional resources:
Remote File Viewer can be used to copy files to/from the target device.

Execution of command line applications on a CE device:
If you are using PB, the target control “s” command can be used to start a process on the device. 
You can also use the JShell tool (part of the Windows Mobile Developer Power Toys ) on Pocket PC or Smartphone. Some versions of CETK also include JShell in a Tools subdirectory.

CPU usage measurement:
Platform Builder comes with a tool called Remote Performance Monitor that can be used to measure CPU usage and other useful system parameters.
CETK includes a tool called CPU Monitor that shows you the CPU and memory usage of a CE device.
Both Remote Performance Monitor and CPU Monitor require a KITL or ActiveSync connection to the target device.

Complete Tux documentation can be found here.

Comments

• Anonymous
  July 10, 2006
  Realy helps .... Just wondering where I can find the more documents/information about the perf_bluetooh test itself.

• Anonymous
  July 13, 2006
  Thanks shyhlee. There is some basic documentation in the official help file, but this post actually includes more details. What kind of additional information would you like to see?

• Anonymous
  September 18, 2006
  The comment has been removed

• Anonymous
  September 19, 2006
  Constant: perf_bluetooth measures throughput in any environment that you setup. If you're only interested in SW related blottlenecks (i.e. drivers) then you might want to use an RF enclosure.
  

• Anonymous
  January 20, 2008
  I could not continuously test this case. It just only continuously test 2~4 times. How do I determine this case pass or not?

• Anonymous
  January 21, 2008
  terry, the test assumes that the lower transport is working properly, so a failure at that level automatically leads to an inconclusive performance result.

• Anonymous
  January 24, 2008
  Thanks. I manually test my throughput. TX is 50.8 KB and RX is 42KB under BT 2.0. The throuthput is not bad but I could not continuously run the test. Do you mean that transport is not stable to do continously test? Please give me some comments.

• Anonymous
  January 24, 2008
  you should be able to run the test continuosly. I suggest that you investigate the cause of failure in the bluetooth transport driver.

• Anonymous
  February 03, 2009
  The comment has been removed

• Anonymous
  February 04, 2009
  Tired of the broken links. Really tired. Just scan for them.