다음을 통해 공유

Tell me how I’m measured…and I’ll behave accordingly

  • 아티클

(This is the initial blog post from the MSCOM Service Management Team. This team is an essential resource for our group. They are responsible for on-boarding new customers to MSCOM, working with existing customers to provide guidance and smooth the way for releases, interfacing with our Data Center providers and provide a whole array of other functions.)

Without clear measures individual groups and/or individuals will decide for themselves what measures they use for making decisions as well as to gauge their success…leading to a disjointed organization and misaligned customer relationships. For example, your Operations group may measure an application’s availability on a 24/7 clock while the customer owning the application really only cares about the application’s availability during discrete time windows throughout the day. Without common and agreed to measures both parties will doubt each other’s data and blame the other for not doing their job while in the mean time make little or no progress on objectively addressing any existing issues. (That example isn’t fictitious either.)

Measures should be viewed as a process outcome, and fortunately most processes only have a handful of inputs that impact the outcome. Our ultimate goal is to identify and control those specific inputs using a dashboard with control charts so that our service monitoring is more proactive and less reactive. By understanding if critical inputs are in or out of control we can predict if the outcome will be in or out of control.

So how do you figure out what the right measures are? Understanding your customers and your organization’s goals and major processes sure helps, but here is a simple method to jumpstart defining the measures you and your group need.

1. Define the scope for your measures.   Is it for your CEO, Director, middle manager, worker bees, customers, etc?

2. Identify the questions the user needs an answer to. For us, being an IT Operations group there undoubtedly needs to be a mix of operationally focused and business focused questions.

3. Clarify why answering the question is important with bullet points or brief sentences. This will not only help validate the question but also help you understand if the question is at the right level for your target audience.

4. Identify the specific measure, its dimension/s, and target.

· Measures are the data point we’re interested in.

· Dimensions are ways in which you want to be able to slice and view the measure’s data.

· Targets are the goals that you have for that measure.

· Here’s an example showing how all 3 elements fit together. If the measure is “% of Incidents Resolved within SLA,” the dimensions you may want to see the measure by could include by customer, by application, by incident priority, and by time (quarter, month, week). The target would be pulled directly from your SLAs (Service Level Agreements).

5. Identify other questions that would be raised by this measure.   These questions can help you identify additional measures for your target audience as well as identifying linked measures for other audiences either up or down your company’s chain.

6. Determine what actions could be taken with this measure’s information.   If people don’t know how the information helps them make decisions or take action they’ll undoubtedly ignore the measure.

7. Identify potential data sources for the measure.

8. Identify the data owner.   Or in other words, who is responsible for making sure the data is collected consistently and remains up to date. The data owner isn’t necessarily also responsible for the good or bad news you’ll see with the measure.

9. Identify any potential data collection and reporting issues.   Maybe a tool exists to collect the data you need but it’s entered inconsistently so the process will have to be fixed before you can accurately report on the measure. Maybe the data you need isn’t currently collected at all. Clarifying issues such as these will help you determine what reporting can be built immediately and what reporting will require some additional planning and efforts to make available.

 

This linked table gives a generalized example of how all this information looks together. With a clear understanding of what measures you need and why, you’ll now be able to more easily explore options for effectively visualizing the information in charts, graphs, histograms, etc. If you define the picture before understanding the measures needed there’s a good chance of ending up with less than optimal reporting. If you put in the effort, your group and customers won’t doubt the data.

Comments

  • Anonymous
    January 01, 2003
    An example of a measured change In our first two articles we highlighted the importance of measurements

  • Anonymous
    January 01, 2003
    This is great stuff! I would like to point out however, that you should be very careful when using measurements to control behaviour. Very often (and this isn’t fictitious) You may find that people will perform exactly so that they measure properly, often not willing to perform better, even finding shortcuts to attaining the correct numbers, however not really fixing problems. (e.g.:"Please reboot your computer. if the problem still occurs call back") This makes for a short call (Often a something being measured), but rebooting a computer in this day and age rarely fixes problems.

  • Anonymous
    December 11, 2006
    Good feedback and examples.  I agree that people can manipulate any measure they want.  Measures cannot control behavior by themselves, but the "right" measures can provide behavior reinforcement.  When working with measures (or policies, culture, and the like) you need to think of the behavior to be reinforced while also considering and eliminating the reinforcements for conflicting behaviors.  As you've helped illustrate, identifying the right measures is not an easy task.

  • Anonymous
    January 18, 2007
    I'd reinforce what the previous commentors said about picking the right measures; sometimes "moving the goalposts" by changing the measures can alleviate some of this. BTW: the linked table image may be better as a PDF, and some of the cell content is truncated.