High Reliability DOD IT Acquisition Reform

As readers of this blog probably know by now, I am spending the summer of 2011 strengthening my doctoral seminar on High Reliability Organizations. The “Prologue” is a review of the now-legendary Berkeley Studies that were triggered by Admiral Tom Mercer’s brilliant recognition that he could be a better captain of the U.S.S. Carl Vinson by studying Pacific Gas & Electric (electrical grids) and the Federal Aviation Administration (air traffic control). The twelve contexts I’ve identified are military systems, naval systems, aviation systems, petrochemical plant systems, nuclear systems, safety and security systems, space exploration systems, firefighting systems, healthcare systems, medical systems, governmental systems, and national security systems. The “Epilogue” is a collection of miscellaneous HRO contexts that are likely to be developed more fully in the years ahead.

Like I did a couple of weeks ago when I wrote about the Tuscaloosa Tornadoes, today I’d like to drill down into one of the twelve large research domains in the syllabus (national security systems), and discuss one of the 144 more precise research domains (Department of Defense Information Technology Acquisition Reform).

Today, there are three helpful studies that I would like to recommend for DOD IT Acquisition Reformers.

The first HRO study of IT Systems that might help DOD IT Acquisitions Reformers think differently about the future is from my favorite renegade collection of HRO scholars – the participants in the annual conferences on Information Systems for Crisis Response and Management (ISCRAM). The U.S. bias in its IT systems historically seems to have been toward tighter coupling through centralization of control of the systems. In contrast, the ISCRAM people – who have a high degree of international representation -- are more likely to be focused on distributed cognition, local situational awareness, and the exploitation of inexpensive off-the-shelf mobile devices. One exemplary paper of this difference between big-budget centralized U.S. systems thinking and small-budget decentralized non-U.S. systems thinking was presented in Sweden by two Portuguese scholars, Claudio Sapatièro and Pedro Antunes. Perhaps instead of saying “More Cowbell,” DOD IT Acquisitions Reformers should be saying “More ISCRAM.”

Second, a 2010 University of Maryland dissertation by Catherine L. Anderson presents a helpful study of 153 “IT infrastructure failures” in 70 organizations in 9 different industries. Anderson’s study is a helpful read for Department of Defense and other national security organizations because it will help them understand things differently than they might do otherwise. First, Anderson shows that all new IT systems are going to fail because they are highly complex and highly coupled (a condition that creates the phenomenon Charles Perrow called “normal accidents” in 1984) – to wait for a perfect IT system is to never take delivery of an IT system. Second, Anderson finds that the magic fairy dust that helps transform imperfect IT systems into potentially useful IT systems is the concept of “collective mindfulness” – to adopt a new IT system into a broken organizational environment will invite failure, but adopting a new IT system into a healthy organizational environment might lead to success. Third, Anderson expands on Weick and Sutcliffe’s useful distinction between anticipation and containment – “mindful anticipation” before the beginnings of an IT failure can help the system be more robust; “mindful containment” after the beginnings of an IT failure can help the system be more resilient. (My personal preference here would be to distinguish between three types of HROs – High Robustness Organizations draws on James Reason and others, High Reliability Organizations draws on Karl Weick and Kathleen Sutcliffe and others, and High Resilience Organizations draws on Erik Hollnagel and others.) If DOD IT Acquisition people thought more along the lines of Anderson, they might get faster, better, and less expensive IT systems.

Finally, the best study I have seen that might be useful to DOD IT Acquisition Reformers is from researchers at Cranfield University, who have not focused on the failure of IT systems after installation, as Anderson did, but on the failure to implement the IT system. David Denyer, Elmar Kutsch, and Elizabeth (Liz) Lee-Kelley point out that IT system acquisition failures are highly frequent and highly expensive:

It appears that every year billions of pounds are wasted on new IS programmes such as the US Advanced Automation System project (Nelson, 2007), or the UK National Offender Management System implementation (National Audit Office, 2009). In 2004, only 29% of all IS programmes succeeded in meeting their time, budget and specification objectives (Johnson, 2006).

Denyer, Kutsch, and Lee-Kelley expand on the insights that Anderson presented, and propose four HRO-inspired research questions that I believe could help the Department of Defense IT Acquisition Reform process:

RQ1: To what extent do programmes find an appropriate balance and consensus between reliability and other performance goals (e.g. scope, timeframe and cost)?

RQ2: To what extent do programmes reconcile the tension between the need for centralisation (formal structures, hierarchical decision making and adherence to plans, procedures and processes) and the need for decentralisation (anticipation of problems followed by rapid, improvised and mindful responses)?

RQ3: To what extent is learning, particularly from errors, incidents and near misses, achieved in programmes?

RQ4: To what extent is redundancy (e.g. technological, human and time) created, fostered and used in programmes?

The U.S. national security system could be more effective at accomplishing national missions, for far less money that it currently spends, and with a much higher level of national safety, security, robustness, reliability, and resilience, if it can move from its 19th Century IT Acquisition process to a 21st Century IT Acquisition process. These three studies, which their authors have generously made available on the Web, could be helpful in that reform effort.