Process Performance

Editor’s Note : The following is an extract from a  CrossTalk-July/August 2012 article.  The original  is  available at http://www.crosstalkonline.org/storage/issue- archives/2012/201207/201207-Childs.pdf  )

Words of Wisdom

Abstract .  Process performance forms the cornerstone of the high-maturity concepts in the CMMIR. High maturity generates great discussion in the CMMI-based process improvement world. However, understanding process performance provides benefits to an organization whether or not it adopts the CMMI. The CMMI provides a framework for an organization’s process improvement efforts. At its highest  levels, the CMMI describes how an organization can use process performance measures to understand and improve its business processes. While this article will mention the high-maturity process areas from the CMMI, it will primarily focus on the analysis of process performance to help an organization. Comments from wise men and women throughout history illustrate ideas related to process performance. In one of his songs, Jimmy Buffett said,  “Chasing illusions can get quite confusing.”

This is true of the use of process performance measures. To  understand how the use of process performance measures  affect an organization, it is good to look back at some words of  wisdom related to the concepts behind the use of performance  measures and results. This article will illustrate the practical  meaning and benefits of understanding process performance by  drawing connections to famous quotes.

Setting the Foundation

Managers often struggle for a clear understanding of what is  happening on their projects. They find themselves in the same  situation as Alexandre Ledru-Rollin when he said, “There go  my people. I must find out where they are going so that I can  lead them.”

Good use of process performance data depends on establishing  a foundation of measurement collection. In the CMMI, this  starts with the Measurement and Analysis (MA) process area. In MA, an organization identifies its information needs and measurement objectives. Managers are always looking for information to help them answer questions like Mr. Ledru- Rollin. By starting with information needs, an organization can  specify what is needed to answer some of those management  questions. It is important to define those measures clearly so  everyone is collecting the same data, the same way.  Operational definitions of measures are critical to  measurement success. For example, as a measure, a work hour  can represent many things. An organization should define what  it needs to collect. Is it a direct hour, an indirect hour, a  billable hour or a support hour? By clearly defining each  measure an organization sets itself up for more accurate and  meaningful reporting. Care should be taken to ensure that the  data is collected accurately and analyzed appropriately.  Managers should also communicate the results of the  measurement activities back to the people collecting the  measures. By doing so, the managers give the practitioners a  stake in the measurements. The measures will mean more to  the practitioners, which will lead to more accurate reporting.  Without this communication, an organization ends up boxed in  as Rowan D. Williams stated, “Bad human communication  leaves us less room to grow.”

Measurement establishes the foundation that grows into the  ability to use process performance data to help an organization  improve. Inaccurate reporting stifles that growth.

Dwight D. Eisenhower said, “Things are more like they are  now than they have ever been before.”

While that may seem obvious, in the world of process  performance, it cannot be taken for granted. In order to know  how things are now, an organization must measure the current  state of its process performance and compare it to historical  performance. Each of the high-maturity process areas in the  CMMI contains practices that look at historical results,  measure current performance, forecast future performance, and  look to make improvements. As Philip Crosby said, “Making a  wrong decision is understandable. Refusing to continually  search for learning is not.”

Organizational Process Performance

As mentioned above, an organization establishes measurement  goals based on information needs. As the organization  accumulates historical measurement data, it can begin to  predict process performance based on past results. In the  Organizational Process Performance (OPP) process area in the  CMMI, the organization refines those goals based on a  statistical analysis of historical data and business needs for  quality and process performance. This is important because as  Douglass Lurtan pointed out, “When you determine what you  want, you have made the most important decision of your life.  You have to know what you want in order to attain it.”

A statistical analysis of historical data is necessary to validate  these goals as attainable. Organizations should avoid setting  goals like, “We want to be a world-class provider of choice.”  No one knows what that means, but it sounds cool. People  relate to goals like, “We want to reduce customer found  defects by 25% in the next year.” Organizations should set  goals that are clear, measureable, realistic, and easy to  understand. After the organization sets its goals, it needs to  identify which processes contribute to achieving those goals.  Organizations should not reach for too much in analyzing  processes. It takes time and money to perform quantitative  analysis. Concentrate on those processes that are of concern or  that provide the most insight into the achievement of business  needs. There must be business reasons for choosing the  processes for analysis.

To determine if they can attain what they want, an organization  establishes process performance baselines to understand past  performance and process performance models to predict future  behavior. Keep in mind that these are just tools because as H.  Thiel said, “Models are to be used, not believed.”

These tools give insight into process performance. A process  performance baseline shows an organization its expected  rangeof performance based on past performance. By knowing  the expected range of performance, an organization understands whether or not a given process can meet its  performance goals. Bertrand Russell said, “The degree of  one’s emotion varies inversely with one’s knowledge of the  facts-the less you know, the hotter you get.”

Without the facts, managers can make reactive, emotional  decisions. Such decisions often lead an organization down the  wrong path. Understanding expected performance reduces  emotional decisions by giving managers an objective view and  reasonable performance expectations. Emotional reaction goes  away and objective decision making becomes possible.

Process performance models allow an organization to explore  the relationships between different pieces of their process. By  using past performance to understand how the different parts  of the process relate to one another, organizations can begin to  predict what will happen in later parts of the process based on  what happens in an earlier part of the process. This gives an  organization understanding of what it can do, not just what it  has done. John Wooden stressed, “Do not measure yourself by  what you have accomplished, but by what you should have  accomplished with your ability.”

Process performance models enable managers to understand their ability, and understand when actual results vary from that ability. In the CMMI, process performance models start with a controllable factor, like project size, and create predictive models based on the understanding of the effects of changes to that factor. For example, an organization knows that a size  increase of more than 10% during the design phase causes  increases in test defect rates. Such knowledge can be used to  determine if additional peer reviews or testers are needed to  accommodate the size change and prevent a significant  increase in test defects. Other models, while they may not be  considered process performance models in CMMI terms can  also help organizations understand and manage their projects.  For example, if an organization knows that finding a higher  than predicted rate of requirements review defects historically  means a reduction in test and customer-found defects, it can  anticipate performance results and make decisions related to  those future lifecycle phases.

Quantitative Project Management (QPM)

In QPM, the project managers within the organization select  the measures and techniques they will use to manage process  performance. Sharon Salzberg stated, “Each decision we make,  each action we take, is born out of intention.”

In QPM, the measures and techniques used in the project are  selected based on the objectives established in OPP and any  unique aspects of the project. If there is no connection to the  organization’s objectives, the organization goes back to  chasing illusions or, as Bob Seger offered, “Working on  mysteries without clues.”

Organizations should consider which items to include and  which to leave out. Joshua Schachter said it well when he  made the point, “Every decision has a cost. Do I make this  decision at all or can I move on to the next thing? What we  decided to leave out is almost as important as what we put in.”

The baselines and models that an organization creates give  insight into the quality and performance objectives set by the  organization. As Confucius said, “The expectations of life  depend on diligence; the mechanic that would perfect his work  must first sharpen his tools.”

Process performance baselines and models provide the tools,  which an organization sharpens over time as it gains an  understanding of its process performance. But, tools must be  used. As Debra Wilson explains, “People who do not use the  tools given to them only injure themselves.”

If an organization does not make use of QPM tools, it loses an  opportunity to meet business goals and improve performance.  QPM is where projects use the models and baselines that are  established in OPP to help manage their projects.

Process Performance and Decisions

Not every process is ripe for process performance measurement. An organization should concentrate on those  that directly address business and performance objectives.  Start with a small set and build from there. Once an  organization understands its past results, other areas of  opportunity present themselves. Organizations must start  somewhere, because as Washington Irving said, “One of the  greatest and simplest tools for learning more and growing is  doing more.”

When projects use the tools available to them, they gain insight  and make better management decisions. When the actual  performance, or prediction of performance, does not match expectations set by the baselines and models, managers should  ask questions and take action.

Lee Iococca said, “If I had to sum up in one word what makes  a good manager, I would say decisiveness. You can use the  fanciest computers to gather the numbers, but in the end you  have to set a timetable and act.”

Iococca correctly contends that numbers are not answers.  Numbers represent indicators that managers should use to ask  questions that lead to better decisions. By establishing  baselines and models, an organization sets its managers up to  make decisions based on an understanding of process  performance. Using our size example from earlier, if an  organization knows that an increase in project size of more  than 10% causes a corresponding increase in test defects, a  manager can adjust staff levels or increase test time to allow  for what it expects based on past performance.

Causal Analysis and Resolution

As Crosby pointed out in the earlier quote, organizations must continually learn by looking at their past mistakes and problems. That concept forms the basis for Causal Analysis and Resolution (CAR). Catherine Aird stated, “If you cannot be a good example, then you will just have to be a horrible warning.7”

Both good examples and horrible warnings should be looked at  when selecting outcomes for analysis in CAR. When analyzing  process performance, the organization should look at what has  worked well in addition to what needs improvement. Successes  should be leveraged across the organization and the root  causes of problems should be resolved to prevent the  recurrence of the problem. As Chuck Berry told us, “Do not let  the same dog bite you twice.”

Often organizations focus on symptoms rather than root  causes. Getting the right people in the room, which means  those involved in the process, helps identify root causes of  problems or successes. Organizations leverage their successes  by analyzing the causes behind them just as they fix problems  by analyzing the causes behind them. It may be true as Mark  Twain said, “Few things are harder to put up with than the  annoyance of a good example.”

However, successes can create peer pressure for others in an  organization to improve. By using CAR, an organization can  identify which annoying good examples are worth promulgating. On the flip side, it is also true that, “The best  way to escape from a problem is to solve it, “ as Alan Saporta  pointed out. CAR allows an organization to find root causes  and prevent problems from recurring again and again and  again and …

Organizational Performance Management

Organizational Performance Management (OPM) asks an organization to select the improvements it wants to make and to put structure in place to deploy and analyze improvement  proposals. The potential improvements can come from a  variety of sources. One source is when a project’s results  historically show that they cannot reach performance goals. For example, if the goal is to be within 10% of estimates and the project is always 25% to 40% off, the project is unlikely to  ever meet the goal without making a process change. Winston Churchill pointed to this when he said, “Success consists of  going from failure to failure without loss of enthusiasm.”

However, that success only comes from making change. The results of a CAR discussion can also be the source for potential improvements. For CAR groups to be successful, an organiza?tion must provide feedback that shows the results are consid?ered important and that the results are being used. As  Colin Powell said, “The day soldiers stop bringing you their problems is the day you have stopped leading them. They have either lost confidence that you can help them or concluded that you do not care. Either case is a failure of leadership.22” Another source comes from looking outside the organization for innovations. Organizations often become enamored with their own ideas and refuse to look outside of themselves. This is the trap Friedrich Nietzsche spoke of when he said, “Many are stubborn in pursuit of the path they have chosen, few in pursuit of the goal.23”

All available information and sources, internal and external to  the organization should be used to support improvement  initiatives. Jimmy Buffett summed this up when he told us, “I  have read dozens of books about heroes and crooks, and I have  learned much from both of their styles.”

Business goals should drive organizational improvements.  Outside ideas can be just as valid as those that come from  within.

Validation plays an important role in OPM. There are several  ways to validate if an improvement is successful. These  include piloting changes, modelling  behavior and simulating  results. To the extent possible, improvements driven by  process changes should be validated statistically to ensure that  observed changes are not random. In other words, a quantitative look should be taken to ensure that a significant  change has occurred. This prevents the pitfall that Mr. Spock  addressed when he said, “A difference that makes no  difference is no difference.”

Whatever method is chosen, organizations must find project  managers willing to take the first steps in trying out new  improvements. They should be willing to follow Frank Zappa’s  words, “I will do the stupid thing first and then you shy people  follow.”

Improvement proposals do not always work. An organization should not try to force an idea because it seems like it should work. Use the validation results to determine if the change is worth adopting. W.C. Fields explained that by saying, “If at first you do not succeed, try, try again. Then quit. There is no use being a damned fool about it.27”

However, if the organization determines that the improvement  was a success, then a plan should be put in place to deploy it.   Using Process Performance Measures to Manage Change   All improvements require change. Ideas may be easily  understood and accepted, but change always comes hard.  Charles Kettering explained, “The world hates change, yet it is  the only thing that has brought progress.”  While change is critical for improvement, change must be  managed. Changes deployed to the organization should be as timely as possible, but accomplished in an orderly fashion. As  John Wooden told his teams, “Be quick, but do not hurry.”

Unmanaged change creates chaos, but managed change brings benefits. As Francis Bacon pointed out, “Things alter for the worse spontaneously, if they are not altered for the best designedly.30”

Organizations have to deal with unplanned, spontaneous  change. Managed change is easier to accept and sets the  foundation for future improvement.

Using process performance measures greatly aids an organization in making improvements. However, organizations  should not blindly follow the numbers. Numbers can be  manipulated as Mark Twain said, “Get your facts first, and  then you can distort them as much as you please.”

As stated previously, numbers are just indicators. To create  useful indicators, organizations should clearly define the  analysis techniques that will be used and the rationale for using  them. Organizations must never lose sight of the fact that the  perception of the staff is just as important as the numbers.  George Santayana explained, “Those who speak most of  progress measure it by quantity and not by quality.”

Both the hard numbers and soft perceptions determine the  success of any improvement effort. If the numbers look good,  but the people have legitimate reasons for objection, the  organization must consider their viewpoint. Malcolm Gladwell  said it well when he explained the need for balance, “Truly  successful decision making relies on a balance between  deliberate and instinctive thinking.”

Organizations collect a lot of numbers, but real value comes  when they are used. Establishing a measurement foundation  enables the use of process performance measures once an  organization builds some historical data. The high-maturity  process areas in the CMMI provide guidance on how quantitative information and process performance measures  can be used to help an organization meet its business goals.  Remember as Hesiod stated around 800 BC, “Observe due  measure, for right timing is in all things the most important.”

The time is now for building a measurement program with the  vision for how performance measures will be used.  Understanding process performance can be perplexing. Others  weathered the storms of change in the past. In order to plan for  the future of process improvements and make meaningful  change, organizations should consider words of wisdom from  those who came before us.

(Editor’s Note : The original provides references for the  quotations and Disclaimer that  CMMIR is registered in the  U.S. Patent and Trademark Office by Carnegie Mellon  University .)

Editor’s Note : The following is an extract from a  CrossTalk-July/August 2012 article.  The original  is  available at http://www.crosstalkonline.org/storage/issue- archives/2012/201207/201207-Childs.pdf  )