Crosstalk Magazine July/August 2014 features some high profile software industry gurus with a range of topical, sometimes controversial, advice. There are also some interesting metrics I had not heard of before. This Crosstalk issue has the theme High Maturity Organizational Characteristics and is sponsored by The 309th Software Maintenance Group (309 SMXG), one of the first CMM/CMMI Level 5 (1998/2006) organizations in the US Department of Defense, but there are lessons here for organisations large and small.

The entire Jul/Aug 2014 Issue can be downloaded free (PDF 15MB) or flipped through online using the new Digital Flipbook Version. Individual articles can be downloaded from the Table of Contents.

“Newcomers to Agile often fail to see the connection between older thoughts about Agile and today’s Agile movement.”

The opening article, Agile and the Definition of Quality is by Gerald M. Weinberg, sometimes called the “Father of Agile.” Gerald notes that some Agile writers are now calling him “the grandfather of Agile”, which he takes as a compliment. In this article, Gerald gives a practical example of the relativity of quality and goes on to discuss “a few familiar but often conflicting ideas about what constitutes software quality.” Gerald also gives a rough estimate of the annual cost of buggy software to United States workers and suggests an answer to the question, “Why Is Improving Quality So Difficult?”

“Too many organizations have a single model of high maturity to which they try to fit all their projects.”

 

In High Maturity Is Not A Procrustean Bed, authors Barry Boehm, Richard Turner, Jo Ann Lane, and Supannika Koolmanojwong contribute an article based on material from their 2014 book, The Incremental Commitment Spiral Model: Principles and Practices for Successful Systems and Software. 

The authors note that traditional process and lifecycle models are not meeting current challenges and the Incremental Commitment Spiral Model (ICSM) is the result of their efforts to better integrate the hardware, software, and human factors aspects of systems, to provide value to the users as quickly as possible, and to handle the increasingly rapid pace of change.

The article provides interesting examples of “Procrustean” process consequences and also examples of how different risk patterns yield different processes. The authors go on to describe the ICSM criteria for determining which process or processes best fit a particular system and to illustrate how they have been successfully applied in various situations.

“Disciplined learning, or ‘learn early, learn often,’ updates naïve agile development and traditional risk management, and safely replaces the dreaded catch phrase, ‘fail early fail often.’”

Agile software development guru Dr. Alistair Cockburn contributes some new insights in his article Disciplined Learning: The Successor to Risk Management. He suggests that “Naïve agile development works remarkably well, given how simple it is” but is less than optimal and insufficient for many situations. Disciplined learning adds to agile and updates risk management by incorporating some of the principles of agile development.

Alistair warns that “Disciplined learning is neither obvious nor for the faint of heart, but it is in active use by top teams in many disciplines, who manage to deliver success in difficult circumstances.” The article goes on to illustrate the processes involved and the payoffs to be obtained from Disciplined Learning.

It is interesting to compare the above articles with the “short study” by Capers Jones which comes next.

“Software is the main operational component of every major organization in the world, but software quality is still not acceptable for many applications.”

In Achieving Software Excellence, Capers Jones uses data from about 600 companies of whom 150 are Fortune 500 companies. The data indicates that at the end of 2013, “Cancelled projects were about 35% in the 10,000 function point size range and about 5% of software outsource agreements ended up in court in litigation.” Capers Jones says that the major point of the article is: “High quality using a synergistic combination of defect prevention, pre-test inspections and static analysis is fast and cheap. Poor quality is expensive, slow, and unfortunately far too common…” He also notes that “The combination of low defect potentials and high defect removal efficiency (DRE) is what software excellence is all about.” He later adds that “Any company that does not measure and know their DRE is probably below 85% in DRE.”

For Capers Jones, “poor quality control is below 85% in defect removal efficiency (DRE) levels. In fact for cancelled projects or those that end up in litigation for poor quality, the DRE levels may drop below 80%, which is low enough to be considered professional malpractice.”

Caper Jones also draws conclusions on Agile development which are likely to stir up some debate. He notes that Agile development has proven to be effective for smaller applications but “does not scale up well and is not a top method for quality.” He sees ‘pair programming’, as “very expensive but only benefits quality by about 15% compared to single programmers. Pair programming is much less effective in finding bugs than formal inspections, which usually bring 3 to 5 personnel together to seek out bugs using formal methods.”

Caper Jones sees Agile as a definite improvement for quality compared to waterfall development, but “not as effective as the quality-strong methods of team software process (TSP) and the rational unified process (RUP).” He goes on to make a strong case for reuse of Certified Materials as the best long-term strategy for achieving consistent excellence at high speed.

The next article includes some internal metrics which were new to me, the Chidamber and Kemerer (CK) metrics, which can “assist users in understanding object oriented design complexity and in forecasting external software qualities for example software defects, testing, and maintenance effort”.

“Metrics are beneficial to an organization that supports a product from inception through product retirement and disposal.”

Improving Software through Metrics while Providing Cradle to Grave Support,

by Jennifer Walters, Northrop Grumman and Kevin MacG. Adams, Ph.D., NCSOSE explains how internal metrics can be collected early in the software development phase to predict the support required during the operations and maintenance phase. Those internal metrics can then be teamed with external metrics to predict external quality as well as the maintenance support needs of the final product, and to drive software development process improvements.

The authors note that “the metrics of concern in this paper are quality attributes because maintenance efforts are highly dependent on the overall quality of the software. In addition, quality attributes span both pre-delivery and post-delivery phases of the lifecycle…” The article provides a table of Metric Suites for Measuring Internal Quality Attributes, together with details of those internal metrics can be used in forecasting external software qualities and maintenance effort. Also provided is a Figure showing the relationships between internal and external metrics, together with discussion on the metrics which have been most important for reducing defects and for improving both development and maintenance processes to achieve overall software quality improvement “from the cradle to the grave.”

In the next article, there was a plaintive comment which echoed my own experience with many clients over the past 20 years: “Many engineers in industry and the DoD have never seen a detailed plan, let alone participated in making one.” The authors of this article share their experience as NAVAIR Internal Process Coaches, showing how far from reality the “by-the-book” approach to Continuous Process Improvement can sometimes be.

“The long-term goals of Process Improvement should be to introduce and sustain a culture of continuous process improvement.”

Adoption: Routes to Continuous Process Improvement by David Saint-Amand, Naval Air Systems Command and Mark Stockmyer, Naval Air Warfare Center, starts with the Abstract: This paper covers the different types of teams the authors have encountered as NAVAIR Internal Process Coaches and how they approached Process Improvement with them, with special emphasis on the curmudgeons (bad-tempered, difficult, or cantankerous persons).

The article notes that the authors have observed, participated, or assisted in numerous CPI initiatives in both private industry and United States Federal Government service, using methodologies which included Agile, CMM®-SW, CMMI®, TQM, HPO, Lean Six Sigma, PSPsm , RUP and the TSPsm. They ask “What can a process improvement coach do though when the proper preparations aren’t made and the people are not effectively prepared in advance?” and note that in some cases “intentionally or unintentionally, management itself is fostering an environment of chaos and will oppose attempts to introduce discipline.”

The authors go on to give examples of “Canon  vs.  Reality”, where the by-the-book approach was far removed from what they encountered, and of the answers they have found that are applicable to most process improvement initiatives.

The last article shows that even Level 5 CMMI Organisations are learning to question the accepted CMMI wisdom.

“How rocket scientists implement High Maturity.”

 

In High Maturity Heresy: Doing Level 5 Before Level 4 Without Data, Tom Lienhard, Sr. Principal Engineer at Raytheon Missile Systems and a Six Sigma Black Belt, starts by asking where does our knowledge regarding High Maturity of the CMMI® come from?…” What if all these sources were wrong or, at best, were only painting half the story?”

Tom notes that Raytheon Missile Systems (RMS) achieved SW CMM Level 5 in 2001 by statistically controlling defects detected in the software development process, which resulted in improvement and return on investment. However programs were still having the problem of being able to produce product at a price the customer was willing to pay. He asks, “So, what went wrong, RMS was Level 5?”

Tom goes on to give a colourful and entertaining explanation of the problem and to describe the ‘heretical’ solution.

in BackTalk, David A. Cook, Ph.D. Stephen F. Austin State University, contributes Schadenfreude and Mature Software Development, a humorous, rueful account of the guilty pleasure David takes in reading of major software disasters and “learning from the massive failures of others”. David cites a number of major disasters, sources and links, including a link to The Capability Im-Maturity Model (CIMM) – Nov 96, by then-Captain

Thomas Schorsch (now Retired Lt. Col, Ph.D.) This is a tongue-in-cheek article which extends the existing five CMMI levels downward from L1 Initial (Ad hoc and Chaotic) to include 0 and negative levels, of which David says “I am in awe of the cynicism, sarcasm, and validity of Tom’s research.”

 

Source:

Crosstalk July/August 2014 http://www.crosstalkonline.org