Automating smart charts/Culling KPIs

Posts

I have 20+ groups who are developing KPIs at my organization.  They will each come up with a few that are important to them.  This means our organization will likely end up with 50 – 75 KPIs.  That's a lot.  I have two questions regarding this. 

1. I have been working with our IT team to automate whatever parts of this process they can.  I don't want this to be TOO labor intensive, or it just won't happen.  And I can't do it all.  But while they can program looking at Smart Charts to see if there are 7 points above or below the trend line, I don't know that they can look at everything needed to adjust the trend line – short runs, 3 out of 4 points closer to the edge of an upper or lower range, etc.  So I'm thinking it is going to take a human looking at every one of these measures every month or whatever time period is appropriate.  Do you think this is true, or is there any way to automate any of the range limit adjustments, etc.?

 

2. I am working with our Finance folks to write my parts of the budget – primarily the section on Strategic Planning and Performance Measures. In past years we had about 25 organizational measures supporting 8 District goals.  We now have 5 Desired Results, which is fine, but I don't think all 50+ performance measures will be appropriate for inclusion in the budget.  How do I decide what makes sense?  We are just developing measures at a team (orange circle) level right now – nothing at the center of the bullseye.  And I don't know when/if we will do measures for the center results.  We haven't gotten to that stage yet.  Any suggestions?

–Cheryl Welch/TVWD/Oregon, USA

Hi Cheryl-  I work in the IT Department of our organization. As we are improving our dashboard, I have challenged our Business Intelligence/ Dashboard engineers to include Smart Charts. We're not far enough down the road, yet, where I can share any outcomes. But, I'll keep you posted. We've got a great team of engineers who like to be challenged by what I tell them might not be possible, and I am always amazed to see what creative innovations they come up with. 

Thanks, Scott.  I'd love to know about anything they come up with!  I'm still in the same spot with this that I was three months ago, so anything you/they find would be welcome!  –Cheryl

Cheryl- Our computer engineers were able to get something for us that seems to work really well. Basically, they said the key is to take individual time points that are each in their own row, and combine them to put them all on a single row. Then, you can validate if 7 in that single row are above or below the mean, and if they are, automate a recalculation of the central line and the upper and lower limits. Below is how it looks. We've added the Target Zone as an ongoing target to hit, rather than putting the target out into the future (which we may add at some point where it makes sense).

 

 

Also, on our dashboard, we have a link explaining how to understand and use XmR Charts; below is the text I've used for that if it's of any help.

Understanding XmR Charts

 If you timed your commute each day for a week, your minutes would vary; some days would be a bit shorter, some would be a bit longer. If one day were a little longer—or even a lot longer due to an accident—you probably wouldn’t change your course immediately; after all, some regular variation and even an unusual glitch is to be expected. However, if your commute consistently started taking longer than expected, or if you found a route that consistently saved you a few minutes, you would likely make changes.

The same natural variation happens in business processes. Unlike the commute example, however, leaders may respond prematurely to natural variations as if they were significant. An instance or two of poor performance may lead to executive action, when in fact it was part of the natural variation, and no change was needed. Alternatively, a few instances of good performance may lead to false confidence that things are improving, when in fact things are really just the same. To improve processes, leaders need to reliably separate the signals of real change from the noise of natural variation.

An XmR chart shows the natural variation that can be expected in any given process. XmR charts signal when changes have occurred that are higher (Upper Natural Process Limit) or lower (Lower Natural Process Limit) than what would normally be expected for that process. When 7 points in a row are above or below the Center Line (Average), a new Center Line is calculated, along with new Upper and Lower Natural Process Limits. This signals a “new normal” for the process. 

Actions to Take

Unstable/Chaotic – If values are all over the place and the Natural Process Limits are wide, focus on making the process more standard and predictable. Tighter limits indicate more predictability.

Outlier- If a single value or two show up outside the Natural Process Limits, look into it, but don’t make any changes until you give it time to see if it is the beginning of a new trend.

Trends- When 7 values in a row are above or below the Center Line, the graph will automatically calculate the new normal. If 7 values trend downward or upward across the Center Line, wait until the values meet the criteria of 7 above or below the Center Line, or level off, before taking any action. For short trends
of 3 or so, wait until the values level off before taking any action.

 

I hope this helps.

Scott, thanks so much for taking the time to post information on what your folks have done.  I like the way you have displayed the graph and the target zone.  As I said in my first post, our IT people did find a way to program the 7 points above or below the center line and recalculate the center line as needed.  But what about the short runs or 3 out of 4 points closer to the edge of the upper or lower range than to the center line?  Can your engineers incorporate that, as well?  It just seems like there are so many things to look for that it would be challenging to program something like that.  Although I suppose it would just be a series of OR statements – looking to see if any of the conditions were true.  

I appreciate what you have shared and do let me know if you ask them to delve any deeper or analyze the data based on other trends, too!  –Cheryl

Great description of the process, too. I love the commuting example (and plan to use that one myself when explaining it to others!).

 

Scott Anderson said:

Cheryl- Our computer engineers were able to get something for us that seems to work really well. Basically, they said the key is to take individual time points that are each in their own row, and combine them to put them all on a single row. Then, you can validate if 7 in that single row are above or below the mean, and if they are, automate a recalculation of the central line and the upper and lower limits. Below is how it looks. We've added the Target Zone as an ongoing target to hit, rather than putting the target out into the future (which we may add at some point where it makes sense).

Sorry I missed that about your engineers being able to do the 7 in a row. With the short runs (3 in a row) or 4 Points close to the edges, we decided not to include them in the graph based on two reasons: 

• First, the training materials from Stacey essentially say, “Don't do anything with the 3 or 4; wait until you have the 7 points before acting on it.” 6/7 points is a statistically safe point to say, “we have achieved a new normal,” while 3 or 4 says, “something may be happening, but not with the statistical certainty we need to change course.”
• Second, the XmR charts easily show where the 3 or 4 points are; our executives can see those. But, we don't want to alarm them prematurely by re-calculating anything. We talked about making those data points a different color to highlight them a little more, but decided that, for now, we won't worry about it, and will trust our executives to see those short trends.

The 3 or 4 points seem to be important to you. What value am I missing, or what am I not seeing by not highlighting them out on our charts? I may well have a blind spot here, and would appreciate anything you could point out.

No, Scott, I think I'm the one who had confused things.  I was confusing seeing a signal or trend (or the beginnings of such) with actually adjusting the central line.  So you're right, we don't need to actually program a change in until there are 7 points above or below the central line, but we can look at what's going on.  Thanks for your continued dialog.  Took me a while to understand what I was actually saying.  Good luck with your measures!  –Cheryl