All About Load Test Planning (Part 3-Proper Use Cases and Scenarios)

项目
09/05/2014

PART 3 in a series of posts on Load Test Planning

As of this point, we should have well defined Goals, Objectives and success criteria (part 1) and we should have an idea of how we want to consider building our data (part 2). Now we want to lay out the basic scenarios and use cases. Before we start this, I want to give you my definitions for “Scenario” and “Use Case”. I have two versions for each term. The first applies to how we work with them in our test planning and the second applies to how they are used inside Visual Studio.

Term	Scope	Definition
Scenario	Real World	(test plan) – A Scenario here defines a grouping of different tasks that a user may perform on the system. I often define a separate scenario for each type of user role that will be part of the test (a school application might define teachers and students as two different roles. Each role will get a separate scenario in my test plan).
Scenario	Visual Studio	(load test) – A logical grouping of tests and test settings in a Visual Studio Load Test that tells the test rig how to generate load.
Use Case	Real World	This is a certain task that a user will perform using the application.
Use Case	Visual Studio	(web test) – The specific commands in a web or unit test that mimic the steps a user performs when executing a task (Real World Use Case).

Example 1 – “School’s Back In Session”

For this example, I will use a web site that a school uses to manage the classes offered. I will focus on two user roles; TEACHERS and STUDENTS. This will allow me to define two different “scenarios” in my test plan. Then, for each scenario, I will look at the work that a user from each role might do. The work will be broken down into specific tasks, and each task will become a “use case”:

Teacher:
- Login (I define this separately because I know it will be a test I want to perform so I get it out on the radar right away)
- Enter grades for tests already taken
- Submit updated homework assignments to public portal
Student:
- Login
- Look up test scores for current semester
- Download the latest homework assignments from the public portal

Example 2 – “Are You Ready For Some Football?”

My apologies up front to all of the readers who know soccer as “football”. I love soccer, but in this case, I am partial to what Americans call football and am referring to NCAAF and the NFL. Anyway, it’s time to hit the gridiron and I want to make sure my site can handle the load. So for this example, I will focus on a site that hosts both fantasy football and real football information, sales and game updates.

Fantasy Football Fan:
- Draft players onto the team
- Check the stats for the players throughout the season
Real Football Fan:
- Buy tickets for the game
- Follow the game in real time on the site.

Building the Test Plan Items

Now that we have our scenarios and use cases, we can incorporate them into our test plan. As we fill in this information, keep in mind that the data I am providing here has NOTHING to do with Visual Studio or any other testing tool. I am merely laying out the expected real world behavior that I wish to mimic when testing.

Example 1 – Our GOAL for this testing (just to make things easy to follow) is to ensure that the site will be able to handle the expected load and perform within the expected criteria (see the Introduction post for more info if needed).

Scenario	Use Case	# users	# times per user per week	Notes
Teachers	Login	150	5
Teachers	Enter grades	150	5	Assumes teacher has 5 classes, therefore, updates 1 test set per class per week.
Teachers	Update homework	150	25	Assumes teacher has 5 classes, therefore, updates all homework each day.
Students	Login	3000	5
Students	Check grades	3000	4	Assumes student has 4 classes.
Students	Get new homework	3000	20	Assumes student has 4 classes.

Of course, your table will be much more detailed and may contain caveats or other requirements here, but the idea is that we defined our use cases and numbers based on “expected load” and using “real world” examples.

Example 2 – Our GOAL for this testing is to determine the breaking point of the application, therefore we use a slightly different table to define our scenarios and use cases. Since we are finding a breaking point, we do not know what the final real world user load will be. However, we should be able to at least have a ratio of the amount of users and actions relative to all of the other use cases involved.

Scenario	Use Case	# users relative to other use cases	# times per user per week	Notes
Fantasy	Draft	100	0.1	Draft only happens once per season, so this number is way low
Fantasy	Check Stats	100	20
Real	Buy Tickets	1	1
Real	Follow Game	2000	180	This assumes that the users update their page every minute of a 3 hour game.

In the above table, the “# users” column represents a ratio (or multiplier) that I will use to increase load. If I decide that there will be 50 people buying tickets, then I will have 50 x 2000 (or 100,000 people following the game online). Also, we might want to be able to change the load for the “Real” scenario without changing the load for the “Fantasy” scenario. If that is the case, then we should call that out in our test plan. This is another example of “getting all of the necessary information up front.”

NOTE: I could easily split the “Real” scenario into two scenarios if I wanted to, but I chose to make it simpler. It doesn’t affect the way I will approach building the harness.

Conclusion

At this point, we should now have enough data to start building the test harness and start modeling the loads and profiles. We will cover that in the next post.

Comments

Anonymous
September 07, 2014
The comment has been removed
Anonymous
September 07, 2014
Glad to hear that. If this was helpful, then be sure to look for the next part in the series which will (hopefully) complete the portion of planning that maps the "plan" to the physical harness.
Anonymous
September 08, 2014
I like the use of the multiplier in the second example. We do it just like this as well with systems we're not sure of the final use pattern. Like a lot of groups, most of our volume is driven by peak times of day, i.e. 60,000 people logging in to check the schedule at 10pm with each person expecting a specific SLA. Which you covered in your first post. However it's fun to see where the system breaks and if you can handle that unforeseen critical edge case.
Anonymous
September 11, 2014
The comment has been removed
Anonymous
September 11, 2014
Bips, good question. I will add a new post to cover the details of this type of requirement, but (in keeping with my typical approach) I will ask you clarifying questions.
What actually "renders a row of data"? * is the row count a reflection of a single user generating a report and the count is based on the query results? * is the row count a reflection of a single user generating a report and the count is based on a value the user passes in ("top 200 records", etc.) * is the row count based on the number of times a user performs a different task in the application? Once I have that, then I can account for it in the mix. For now, I will assume that it is one of the first two items above. In that case, the business partners have clearly stated two different success criteria. Because of this, I would generate two different use cases in my mix and collect the results separately. I would not be able to tell you how I would handle the split without more info from you. I could see situations where each item could be a separate request, but both in the same web test, but I could also see where they might have to fall back into separate real-world scenarios (thus leading to separate V.S. scenarios).

通过