Share:
by Stacey Barr
Bernard Marr created the concept of “Key Performance Questions” to develop better Key Performance Indicators. But ask the wrong KPQ and you’ll get the wrong KPIs.
Do you agree that a KPI, or performance measure, is quantification that provides objective evidence of the degree to which a performance result is occurring over time? That’s the PuMP definition, and it’s served us very well in discerning KPIs and performance measures from the many other types of measures we use in business.
From this definition, it’s clear that good KPIs answer only one question: what is performance doing? Of course, that’s not the only question we need to answer to execute strategy, improve performance and reach goals. But it’s the first question we need to answer. This is why I haven’t become a supporter of Key Performance Questions, or KPQs.
What are Key Performance Questions?
Bernard Marr is an author, speaker and consultant in strategic performance, data strategy, KPIs and analytics. KPQs are his creation and he defines them this way:
“A KPQ is a management question that captures exactly what managers need to know when it comes to each of their strategic objectives. The rationale for KPQs is that they focus our attention on what actually needs to be discussed when we review performance and most importantly, they provide guidance for collecting meaningful performance indicators.”
Marr provides this example of how KPQs generate KPIs:
“For example, for [a] strategic goal around teacher development the organisation has identified a number of KPQs, including: ‘To what extent are we offering the teacher development people want,’ which in turn is measured by KPIs such as ‘teacher feedback’ and ‘school feedback’. A further KPQ is ‘To what extent are we training more teachers’” which is measured by the KPIs ‘number of teachers trained to teach IB courses,’ and ‘number of teachers trained per authorised program.'”
Three reasons why Key Performance Questions don’t reliably generate good KPIs.
There are a few problems that jump out of Marr’s own example of using KPQs to identify KPIs. And these problems are enough for me to conclude that KPQs are not a reliable way to select KPIs.
The first problem is that the strategic goal around “teacher development” is not a strategic goal. We need a clearly written strategic goal before we have any hope of working out how to measure it. “Teacher development” is too broad, too “business as usual”.
A second problem is that the strategic goal seems to be about the act of developing teachers. This is an action, not a result or goal. It’s a tactic to make some other result better than it has been. Maybe the engagement of students in learning activities, or the speed or ease of student learning, or something else. We don’t develop teachers for its own sake. Good strategic goals are results-oriented, not action-oriented. Action comes only after we know the result to create.
Because the strategic goal is not clear and specific, the KPQs triggered by the words “teacher development” have led to the wrong measures, which just count how much activity has happened (like number of teachers trained). Is tracking the number of teachers trained really going to give any insight into whether more students are engaged, or learning with more ease and speed?
The third problem is that the KPIs in the example are not real measures (see the PuMP definition at the start of this article). “Teacher feedback” is not a clear, quantitative measure. It’s just a type of data. Good KPIs and performance measures are written in a very specific way.
Key Performance Questions are not going to reliably lead us to great KPIs. There are more reliable ways to do that (make your goals measurable, then design measures that evidence those goals). If KPQs aren’t good for selecting KPIs, are they good for anything?
Key Performance Questions can drive meaningful analysis.
Marr says that good KPQs start with “To what extent…” and “How well…” which is certainly the direction we need to think in to quantify something. But, as we saw in how Marr’s own example above went wrong, this type of KPQ isn’t enough to find great KPIs.
Questioning is a fundamentally important tool in performance improvement. But they asking specific open-ended questions serve us best when they are about understanding what causes a result, not how the result is measured. Useful KPQs are answered by diagnostic data analysis, not KPIs.
KPQs don’t generate KPIs; KPIs generate KPQs.
Design your KPIs based on evidence of your performance goal. Get tangible and observable clarity about the evidence that will convince you performance is changing. Then find the most meaningful and feasible way to quantify that evidence. It’s the logical way to create KPIs or performance measures which can answer the question “what is performance doing?”
After you have meaningful KPIs or performance measures, then formulate KPQs that will answer a second question “why is performance doing that?” KPQs can help us explore the causes for why performance is where it is, and what the best changes are to make it more like we want it to be.
Bernard Marr’s Key Performance Questions are still a useful tool in strategic performance management. But don’t use them to find KPIs to monitor performance. Design your KPIs from evidence, not questions. Then use KPQs to discover ways to improve the performance your KPIs monitor.
Use KPIs to quantify the evidence of what performance is doing. Use KPQs to guide exploratory analysis to find out why performance is doing what’s it doing.
[tweet this]
TAKE ACTION:
Review your current KPIs. What proportion of them answer the question “what is performance doing?” versus “why is it doing that?” If your KPI answers the second question, it’s probably a diagnostic measure, not a KPI that monitors your goal.
