1. Introduction
An accurate perception of the degree of scientific consensus
is an essential element to public support for climate policy
(Ding et al 2011). Communicating the scientific consensus
also increases people’s acceptance that climate change (CC)
is happening (Lewandowsky et al 2012). Despite numerous
indicators of a consensus, there is wide public perception
that climate scientists disagree over the fundamental cause
of global warming (GW; Leiserowitz et al 2012, Pew 2012).
In the most comprehensive analysis performed to date, we
have extended the analysis of peer-reviewed climate papers in
Oreskes (2004). We examined a large sample of the scientific
literature on global CC, published over a 21 year period,
in order to determine the level of scientific consensus that
human activity is very likely causing most of the current GW
(anthropogenic global warming, or AGW).
 |
Surveys of climate scientists have found strong agreement
(97–98%) regarding AGW amongst publishing climate
experts (Doran and Zimmerman 2009, Anderegg et al
2010). Repeated surveys of scientists found that scientific
agreement about AGW steadily increased from 1996 to 2009
(Bray 2010). This is reflected in the increasingly definitive
statements issued by the Intergovernmental Panel on Climate
Change on the attribution of recent GW (Houghton et al
1996, 2001, Solomon et al 2007).
The peer-reviewed scientific literature provides a groundlevel
assessment of the degree of consensus among publishing
scientists. An analysis of abstracts published from 1993–2003
matching the search ‘global climate change’ found that none
of 928 papers disagreed with the consensus position on AGW
(Oreskes 2004). This is consistent with an analysis of citation
networks that found a consensus on AGWforming in the early
1990s (Shwed and Bearman 2010).
Despite these independent indicators of a scientific
consensus, the perception of the US public is that the
scientific community still disagrees over the fundamental
cause of GW. From 1997 to 2007, public opinion polls have
indicated around 60% of the US public believes there is
significant disagreement among scientists about whether GW
was happening (Nisbet and Myers 2007). Similarly, 57% of
the US public either disagreed or were unaware that scientists
agree that the earth is very likely warming due to human
activity (Pew 2012).
Through analysis of climate-related papers published
from 1991 to 2011, this study provides the most comprehensive
analysis of its kind to date in order to quantify and
evaluate the level and evolution of consensus over the last two
decades.
2. Methodology
This letter was conceived as a ‘citizen science’ project
by volunteers contributing to the Skeptical Science website
(www.skepticalscience.com). In March 2012, we searched the
ISI Web of Science for papers published from 1991–2011
using topic searches for ‘global warming’ or ‘global climate
change’. Article type was restricted to ‘article’, excluding
books, discussions, proceedings papers and other document
types. The search was updated in May 2012 with papers added
to the Web of Science up to that date.
We classified each abstract according to the type of
research (category) and degree of endorsement. Written
criteria were provided to raters for category (Table 1)
and level of endorsement of AGW (Table 2). Explicit
endorsements were divided into non-quantified (e.g., humans
are contributing to global warming without quantifying the
contribution) and quantified (e.g., humans are contributing
more than 50% of global warming, consistent with the 2007
IPCC statement that most of the global warming since the
mid-20th century is very likely due to the observed increase
in anthropogenic greenhouse gas concentrations).
Abstracts were randomly distributed via a web-based
system to raters with only the title and abstract visible.
All other information such as author names and affiliations,
journal and publishing date were hidden. Each abstract was
categorized by two independent, anonymized raters. A team
of 12 individuals completed 97.4% (23 061) of the ratings; an
additional 12 contributed the remaining 2.6% (607). Initially,
27% of category ratings and 33% of endorsement ratings
disagreed. Raters were then allowed to compare and justify or
update their rating through the web system, while maintaining
anonymity. Following this, 11% of category ratings and 16%
of endorsement ratings disagreed; these were then resolved by
a third party.
|
Upon completion of the final ratings, a random sample
of 1000 ‘No Position’ category abstracts were re-examined
to differentiate those that did not express an opinion from
those that take the position that the cause of GW is uncertain.
An ‘Uncertain’ abstract explicitly states that the cause of
global warming is not yet determined (e.g., ‘. . . the extent of
human-induced global warming is inconclusive. . . ’) while a
‘No Position’ abstract makes no statement on AGW.
To complement the abstract analysis, email addresses for
8547 authors were collected, typically from the corresponding
author and/or first author. For each year, email addresses were
obtained for at least 60% of papers. Authors were emailed an
invitation to participate in a survey in which they rated their
own published papers (the entire content of the article, not just
the abstract) with the same criteria as used by the independent
rating team. Details of the survey text are provided in the
supplementary information (available at stacks.iop.org/ERL/
8/024024/mmedia).
3. Results
The ISI search generated 12,465 papers. Eliminating papers
that were not peer-reviewed (186), not climate-related (288) or
without an abstract (47) reduced the analysis to 11,944 papers
written by 29,083 authors and published in 1980 journals.
To simplify the analysis, ratings were consolidated into
three groups: endorsements (including implicit and explicit;
categories 1–3 in Table 2), no position (category 4) and
rejections (including implicit and explicit; categories 5–7).
We examined four metrics to quantify the level of
endorsement:
(1) The percentage of endorsements/rejections/undecideds
among all abstracts.
(2) The percentage of endorsements/rejections/undecideds
among only those abstracts expressing a position on AGW.
(3) The percentage of scientists authoring endorsement/
rejection abstracts among all scientists.
(4) The same percentage among only those scientists who
expressed a position on AGW (Table 3).
3.1. Endorsement percentages from abstract ratings
Among abstracts that expressed a position on AGW, 97.1%
endorsed the scientific consensus. Among scientists who
expressed a position on AGW in their abstract, 98.4%
endorsed the consensus.
The time series of each level of endorsement of
the consensus on AGW was analyzed in terms of the
number of abstracts (Figure 1(a)) and the percentage
of abstracts (Figure 1(b)). Over time, the no position
3.2. Endorsement percentages from self-ratings
We emailed 8547 authors an invitation to rate their own
papers and received 1200 responses (a 14% response rate).
After excluding papers that were not peer-reviewed, not
climate-related or had no abstract, 2142 papers received
self-ratings from 1189 authors. The self-rated levels of
endorsement are shown in Table 4. Among self-rated
papers that stated a position on AGW, 97.2% endorsed
the consensus. Among self-rated papers not expressing a
position on AGW in the abstract, 53.8% were self-rated as
endorsing the consensus. Among respondents who authored
a paper expressing a view on AGW, 96.4% endorsed the
consensus.
|
|
Figure 2(a) shows the level of self-rated endorsement in
terms of number of abstracts (the corollary to Figure 1(a))
and Figure 2(b) shows the percentage of abstracts (the
corollary to Figure 1(b)). The percentage of self-rated
rejection papers decreased (simple linear regression trend
-0.25% +/- 0.18% per yr, 95% CI, R2 = 0:28; p = 0:01,
Figure 2(b)). The time series of self-rated no position and
consensus endorsement papers both show no clear trend over
time.
A direct comparison of abstract rating versus self-rating
endorsement levels for the 2142 papers that received a
self-rating is shown in Table 5. More than half of the abstracts
that we rated as ‘No Position’ or ‘Undecided’ were rated
‘Endorse AGW’ by the paper’s authors.
Figure 3 compares the percentage of papers endorsing the
scientific consensus among all papers that express a position
endorsing or rejecting the consensus. The year-to-year
variability is larger in the self-ratings than in the abstract
ratings due to the smaller sample sizes in the early 1990s.
The percentage of AGWendorsements for both self-rating and
abstract-rated papers increase marginally over time (simple
linear regression trends 0.10 +/- 0.09% per yr, 95% CI, R2 =
0:20, p = 0:04 for abstracts, 0.35 +/- 0.26% per yr, 95%
CI, R2 = 0:26; p = 0:02 for self-ratings), with both series
approaching approximately 98% endorsements in 2011.
4. Discussion
Of note is the large proportion of abstracts that state no
position on AGW. This result is expected in consensus situations
where scientists ‘. . . generally focus their discussions
on questions that are still disputed or unanswered rather
than on matters about which everyone agrees’ (Oreskes 2007,
p 72). This explanation is also consistent with a description
of consensus as a ‘spiral trajectory’ in which ‘initially
intense contestation generates rapid settlement and induces
a spiral of new questions’ (Shwed and Bearman 2010);
the fundamental science of AGW is no longer controversial
among the publishing science community and the remaining
debate in the field has moved to other topics. This is supported
by the fact that more than half of the self-rated endorsement
papers did not express a position on AGW in their abstracts.
The self-ratings by the papers’ authors provide insight
into the nature of the scientific consensus amongst publishing
scientists. For both self-ratings and our abstract ratings,
the percentage of endorsements among papers expressing a
position on AGW marginally increased over time, consistent
with Bray (2010) in finding a strengthening consensus.
4.1. Sources of uncertainty
The process of determining the level of consensus in
the peer-reviewed literature contains several sources of
uncertainty, including the representativeness of the sample,
lack of clarity in the abstracts and subjectivity in rating the
abstracts.
We address the issue of representativeness by selecting
the largest sample to date for this type of literature analysis.
Nevertheless, 11 944 papers is only a fraction of the climate
literature. A Web of Science search for ‘climate change’
over the same period yields 43 548 papers, while a search
for ‘climate’ yields 128 440 papers. The crowd-sourcing
techniques employed in this analysis could be expanded
to include more papers. This could facilitate an approach
approximating the methods of Doran and Zimmerman (2009),
which measured the level of scientific consensus for varying
degrees of expertise in climate science. A similar approach
could analyze the level of consensus among climate papers
depending on their relevance to the attribution of GW.
Another potential area of uncertainty involved the text
of the abstracts themselves. In some cases, ambiguous
language made it difficult to ascertain the intended meaning
of the authors. Naturally, a short abstract could not be
expected to communicate all the details of the full paper. The
implementation of the author self-rating process allowed us to
look beyond the abstract. A comparison between self-ratings
and abstract ratings revealed that categorization based on the
abstract alone underestimates the percentage of papers taking
a position on AGW.
Lastly, some subjectivity is inherent in the abstract rating
process. While criteria for determining ratings were defined
prior to the rating period, some clarifications and amendments
were required as specific situations presented themselves. Two
sources of rating bias can be cited: first, given that the raters
themselves endorsed the scientific consensus on AGW, they
may have been more likely to classify papers as sharing
that endorsement. Second, scientific reticence (Hansen 2007)
or ‘erring on the side of least drama’ (ESLD; Brysse et al
2012) may have exerted an opposite effect by biasing raters
towards a ‘no position’ classification. These sources of bias
were partially addressed by the use of multiple independent
raters and by comparing abstract rating results to author
self-ratings. A comparison of author ratings of the full papers
and abstract ratings reveals a bias toward an under-counting of
endorsement papers in the abstract ratings (mean difference
0.6 in units of endorsement level). This mitigated concerns
about rater subjectivity, but suggests that scientific reticence
and ESLD remain possible biases in the abstract ratings
process. The potential impact of initial rating disagreements
was also calculated and found to have minimal impact on the
level of consensus (see supplemental information, section S1
available at stacks.iop.org/ERL/8/024024/mmedia).
4.2. Comparisons with previous studies
Our sample encompasses those surveyed by Oreskes (2004)
and Schulte (2008) and we can therefore directly compare
the results. Oreskes (2004) analyzed 928 papers from 1993 to
2003. Over the same period, we found 932 papers matching
the search phrase ‘global climate change’ (papers continue to
be added to the ISI database). From that subset we eliminated
38 papers that were not peer-reviewed, climate-related or
had no abstract. Of the remaining 894, none rejected
the consensus, consistent with Oreskes’ result. Oreskes
determined that 75% of papers endorsed the consensus, based
on the assumption that mitigation and impact papers implicitly
endorse the consensus. By comparison, we found that 28%
of the 894 abstracts endorsed AGW while 72% expressed no
position. Among the 71 papers that received self-ratings from
authors, 69% endorse AGW, comparable to Oreskes’ estimate
of 75% endorsements.
An analysis of 539 ‘global climate change’ abstracts
from the Web of Science database over January 2004
to mid-February 2007 found 45% endorsement and 6%
rejection (Schulte 2008). Our analysis over a similar period
(including all of February 2007) produced 529 papers—the
reason for this discrepancy is unclear as Schulte’s exact
methodology is not provided. Schulte estimated a higher
percentage of endorsements and rejections, possibly because
the strict methodology we adopted led to a greater number
of ‘No Position’ abstracts. Schulte also found a significantly
greater number of rejection papers, including 6 explicit
rejections compared to our 0 explicit rejections. See the
supplementary information (available at stacks.iop.org/ERL/
8/024024/mmedia) for a tabulated comparison of results.
Among 58 self-rated papers, only one (1.7%) rejected AGW
in this sample. Over the period of January 2004 to February
2007, among ‘global climate change’ papers that state a
position on AGW, we found 97% endorsements.
5. Conclusion
The public perception of a scientific consensus on AGW is a
necessary element in public support for climate policy (Ding
et al 2011). However, there is a significant gap between public
perception and reality, with 57% of the US public either
disagreeing or unaware that scientists overwhelmingly agree
that the earth is warming due to human activity (Pew 2012).
Contributing to this ‘consensus gap’ are campaigns
designed to confuse the public about the level of agreement
among climate scientists. In 1991, Western Fuels Association
conducted a $510 000 campaign whose primary goal was
to ‘reposition global warming as theory (not fact)’. A
key strategy involved constructing the impression of active
scientific debate using dissenting scientists as spokesmen
(Oreskes 2010). The situation is exacerbated by media
treatment of the climate issue, where the normative practice
of providing opposing sides with equal attention has allowed
a vocal minority to have their views amplified (Boykoff
and Boykoff 2004). While there are indications that the
situation has improved in the UK and USA prestige press
(Boykoff 2007), the UK tabloid press showed no indication
of improvement from 2000 to 2006 (Boykoff and Mansfield
2008).
The narrative presented by some dissenters is that
the scientific consensus is ‘. . . on the point of collapse’
(Oddie 2012) while ‘. . . the number of scientific “heretics”
is growing with each passing year’ (Allegre et al 2012). A
systematic, comprehensive review of the literature provides
quantitative evidence countering this assertion. The number
of papers rejecting AGW is a miniscule proportion of the
published research, with the percentage slightly decreasing
over time. Among papers expressing a position on AGW,
an overwhelming percentage (97.2% based on self-ratings,
97.1% based on abstract ratings) endorses the scientific
consensus on AGW.
Acknowledgments
Thanks to James Powell for his invaluable contribution
to this analysis, Stephan Lewandowsky for his comments
and to those who assisted with collecting email addresses
and rating abstracts: Ari Jokimaki, Riccardo Reitano, Rob
Honeycutt, Wendy Cook, Phil Scadden, Glenn Tamblyn,
Anne-Marie Blackburn, John Hartz, Steve Brown, George
Morrison, Alexander C Coulter, Martin B Stolpe (to name just
those who are not listed as (co-)author to this paper).
References
Allegre C et al 2012 No need to panic about global warming Wall
Street Journal (http://online.wsj.com/article/
SB10001424052970204301404577171531838421366.html,
accessed 14 September 2012)
Anderegg W R L, Prall J W, Harold J and Schneider S H 2010
Expert credibility in climate change Proc. Natl Acad. Sci. USA
107 12107–9
Boykoff M T 2007 Flogging a dead norm? Newspaper coverage of
anthropogenic climate change in the United States and United
Kingdom from 2003 to 2006 Area 39 470–81
Boykoff M T and Boykoff J M 2004 Balance as bias: global
warming and the US prestige press Glob. Environ. Change
14 125–36
Boykoff M T and Mansfield M 2008 ‘Ye Olde Hot Aire’: reporting
on human contributions to climate change in the UK tabloid
press Environ. Res. Lett. 3 024002
Bray D 2010 The scientific consensus of climate change revisited
Environ. Sci. Policy 13 340–50
Brysse K, Oreskes N, O’Reilly J and Oppenheimer M 2012 Climate
change prediction: erring on the side of least drama? Glob.
Environ. Change 23 327–37
Ding D, Maibach E W, Zhao X, Roser-Renouf C and
Leiserowitz A 2011 Support for climate policy and societal
action are linked to perceptions about scientific agreement
Nature Clim. Change 1 462–5
Doran P and Zimmerman M 2009 Examining the scientific
consensus on climate change EOS Trans. Am. Geophys. Union
90 22–3
Hansen J E 2007 Scientific reticence and sea level rise Environ. Res.
Lett. 2 024002
Houghton J T, Ding Y, Griggs D J, Noguer M, van der Linden P J,
Dai X, Maskell K and Johnson C A (ed) 2001 Climate Change
2001: The Scientific Basis. Contribution of Working Group I to
the Third Assessment Report of the Intergovernmental Panel on
Climate Change (Cambridge: Cambridge University Press)
(www.ipcc.ch/ipccreports/tar/)
Houghton J T, Meira Filho L G, Callander B A, Harris N,
Kattenberg A and Maskell K (ed) 1996 Climate Change 1995:
The Science of Climate Change. Contribution of Working
Group I to the Second Assessment Report of the
Intergovernmental Panel on Climate Change (Cambridge:
Cambridge University Press) (www.ipcc.ch/pdf/
climate-changes-1995/ipcc-2nd-assessment/
2nd-assessment-en.pdf)
Leiserowitz A, Maibach E, Roser-Renouf C, Feinberg G and
Howe P 2012 Climate change in the American mind:
Americans’ global warming beliefs and attitudes in September
2012 Yale Project on Climate Change Communication
(New Haven, CT: Yale University and George Mason
University) (http://environment.yale.edu/climate/files/
Climate-Beliefs-September-2012.pdf)
Lewandowsky S, Gilles G and Vaughan S 2012 The pivotal role of
perceived scientific consensus in acceptance of science Nature
Clim. Change 3 399–404
Nisbet M C and Myers T 2007 The polls—trends—twenty years of
public opinion about global warming Public Opin. Q.
71 444–70
Oddie W 2012 Is the ‘anthropogenic global warming’ consensus on
the point of collapse? If so, this is just the right time for Chris
Huhne to leave the Government Catholic Herald (www.
catholicherald.co.uk/commentandblogs/2012/02/06/is-the-%
E2%80%98anthropogenic-global-warming%E2%80%
99-consensus-on-the-point-of-collapse-if-so-this-is-just-theright-
time-for-chris-huhne-to-leave-the-government/, accessed
14 November 2012)
Oreskes N 2004 Beyond the ivory tower. The scientific consensus
on climate change Science 306 1686
Oreskes N 2007 The scientific consensus on climate change: how do
we know we’re not wrong? Climate Change: What It Means
for Us, Our Children, and Our Grandchildren (Cambridge,
MA: MIT Press) (www.lpl.arizona.edu/sites/default/files/
resources/globalwarming/oreskes-chapter-4.pdf)
Oreskes N 2010 My facts are better than your facts: spreading good
news about global warming How Do Facts Travel?
ed M S Morgan and P Howlett (Cambridge: Cambridge
University Press) pp 135–66 (http://ebooks.cambridge.org/
chapter.jsf?bid=CBO9780511762154&
cid=CBO9780511762154A016)
Pew 2012 More Say There is Solid Evidence of Global Warming
(Washington, DC: Pew Research Center for the People & the
Press) (www.people-press.org/files/legacy-pdf/10-15-12%
20Global%20Warming%20Release.pdf)
Schulte K-M 2008 Scientific consensus on climate change? Energy
Environ. 19 281–6
Shwed U and Bearman P S 2010 The temporal structure of scientific
consensus formation Am. Sociol. Rev. 75 817–40
Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt K B,
Tignor M and Miller H L (ed) 2007 Climate Change 2007: The
Physical Science Basis. Contribution of Working Group I to the
Fourth Assessment Report of the Intergovernmental Panel on
Climate Change (Cambridge: Cambridge University Press)
(www.ipcc.ch/publications and data/publications ipcc fourth
assessment report wg1 report the physical science basis.htm)
ABOUT THE AUTHORS
John Cook1,2,3, Dana Nuccitelli2,4, Sarah A. Green5, Mark Richardson6, Barbel Winkler2, Rob Painting2, Robert Way7, Peter Jacobs8 and Andrew Skuce2,9
1 Global Change Institute, University of Queensland, Australia
2 Skeptical Science, Brisbane, Queensland, Australia
3 School of Psychology, University of Western Australia, Australia
4 Tetra Tech, Incorporated, McClellan, CA, USA
5 Department of Chemistry, Michigan Technological University, USA
6 Department of Meteorology, University of Reading, UK
7 Department of Geography, Memorial University of Newfoundland, Canada
8 Department of Environmental Science and Policy, George Mason University, USA
9 Salt Spring Consulting Ltd, Salt Spring Island, BC, Canada
|
![[groups_small]](groups_small.gif)