An October 23, 2015 article in The Conversation, Science is best when the data is an open book,
draws an interesting parallel between the 1986 NASA space shuttle Challenger disaster and a current need for more open data in the scientific community. The author describes the lack of open data as a “major deficiency in the system of science, an embarrassment really.”
In NASA’s case, physicist Richard Feynman revisited the NASA data and discovered how officials had “fooled themselves” into thinking that the shuttle was safe. Managers had ”exaggerated the reliability of the space shuttle to the point of fantasy.” Feynman and his colleagues on the Congressional commission recommended an independent oversight group. NASA needed input from people who didn’t have a stake in the shuttle being safe.
The October 23, 2015 article suggests that Individual scientists also need that kind of input. The author is Alex O. Holcombe, Associate Professor, School of Psychology, University of Sydney, who says “To me it’s clear: researchers should routinely examine others’ raw data. But in many fields today there is no opportunity to do so.” He notes that “Scientists communicate their findings to each other via journal articles. These articles provide summaries of the data, often with a good deal of detail, but in many fields the raw numbers aren’t shared. And the summaries can be artfully arranged to conceal contradictions and maximise the apparent support for the author’s theory.”
Alex also notes that while many scientific societies have a policy of requiring authors to provide the raw data on request, “Unfortunately, this policy has failed spectacularly, at least in some areas of science.” He advocates a new policy, already adopted by some journals, of requiring that data be posted online upon publication of the article. However adoption of this new data-posting policy has been slow because “Currently, researchers are rewarded – in the form of job promotions, and grants – for their articles announcing their findings, but not for the data behind the articles.”
Alex suggests “To fix science, we need to change these incentives: sharing data should be rewarded; providing a critical re-analysis of data should be rewarded; poking holes in others’ claims about a data set should be rewarded. If the returns of professional scepticism can be increased, science will waste less time pursuing false theories.”
As it happens, science is not the only field to be criticised lately for hiding data. “The bureaucracy is not very good at acknowledging when something isn’t working.” So said Finance secretary Jane Halton, speaking at a recent seminar organised by her department and the Australasian Evaluation Society. A September 28, 2015 article by Stephen Easton in The Mandarin, ‘We need to move away from our focus on quantitative KPIs’, reports the event. The Finance secretary notes that “The Enhanced Commonwealth Reporting Framework is forcing public servants to rebuild a culture of evaluation and develop the skills to measure the real impact of the policies and programs they implement, rather than just basic outputs.”
It also appears that another area where practitioners are often kidding themselves is public communication. An October 16, 2015 article in The Mandarin, All about listening: research highlights the lie public organisations tell themselves, quotes research which shows that “On average, 80% of organisational resources devoted to public communication are focussed on speaking instead of listening.” The article quotes from the latest research by Professor Jim Macnamara, the UTS professor of public communication who recently completed an international study of organisational listening involving 36 case studies in Australia, Britain and the United States. Apart from speaking instead of listening, Macnamara found that “When organisations did listen, it was largely to serve their own interests. Of the four main ways organisation listening occurred in the case studies — customer relations, research, social media monitoring and public consultation — the actual listening was often compromised.”
The article provides a link to a 75-page report which presents a summary of Macnamara’s findings, conclusions, and recommendations of the study. A detailed analysis will be available in a book to be released in December 2015.