SCOPE 27 - Climate Impact Assessment

 ‡ The National Center for Atmospheric Research is sponsored by the National Science Foundation.

22.1 INTRODUCTION

Integrated assessments, those that involve one or more connected links in the chain of climatesociety impacts, were undertaken with increasing frequency in the decade of the 1970s. The authors have selected five studies for the purpose of identifying their successes and their problem areas in the hope of assisting those who undertake similar assessments in the future. The readers should keep in mind that the review of these five assessments is meant to be suggestive and in no way is meant to provide definitive guidelines. The review is designed to:

1. identify the origins of, or reasons for, each study, 
2. evaluate the internal organizational factors, and
3. highlight the study's impacts (that is, how it was received). 

The studies chosen met totally or in part the following criteria: 

1. each was an attempt at an integrated climate-related impact assessment, 
2. each represented a multidisciplinary effort,
3. each had been requested by policy-makers,
4. the research activity was completed in the 1970s, and
5. each study was done at a high cost and/or received considerable coverage by the media and the attention of policy-makers.

Although we tried to deal with the studies systematically, we were limited by the varied amount and type of information available about each study. Our discussions are, for the most part, based on longer working papers on each of the impact assessments as well as on the discussions held with several of the principal investigators at the Workshop on Improving the Science of Climate-Related Impact Studies at Oak Ridge Associated Universities, Institute for Energy Analysis, in late June 1981. Given the limitations of time, space and resources, however, emphasis was placed on the important aspects and highlights of each study.

22.2 MASSACHUSETTS INSTITUTE OF TECHNOLOGY SUDAN-SAHEL STUDY

The US Agency for International Development (AID) gave $1 million to the Massachusetts Institute of Technology (MIT) Center for Policy Alternatives between September 1973 and September 1974 to prepare a framework for evaluating long-term strategies for the development of the Sahel-Sudan region of West Africa. The Center's final report consisted of 12 volumes, two of which were treated as the final report and distributed by AID (Matlock and Cockrum, 1974; Seifert and Kamrany, 1974), while the others, labeled `Annexes', were not widely distributed.

The report and its annexes discussed such topics as agricultural development; economic considerations; health, nutrition and population; industrial and urban development; sociopolitical factors; systems analysis of pastoralism; technology, education and institutional development; transportation; water resources; and energy and mineral resources.

22.2.1 Background

Most scholars as well as foreign assistance donors familiar with the region believed that this particular study was the direct result of concern about the plight of inhabitants in the West African Sahel caused by the cumulative effects of prolonged regional drought between 1968 and 1973. Internal AID documents, however, indicate that there was a nascent interest within AID as early as 1971 to address regional problems related to long-term, low-grade, but cumulative environmental degradation (AID, 1972, 1976a). The national and international news coverage of drought in West Africa, especially after March 1973, heightened public and congressional interest within the United States (Morentz, 1980) and presented AID with an opportunity to take a leading role in the long-term development of a region formerly within the French sphere of interest (Glantz, 1976).

At a United Nations meeting at Geneva held in June 1973, AID proposed the appointment of a major American university to develop a framework for evaluating mid- and long-term development programs for the Sahel-Sudan region of West Africa (Brown, 1973). The proposal was accepted, but not without skepticism voiced by European representatives about its chances for success (AID, 1976a).

Massachusetts Institute of Technology was the university selected by AID under `noncompetitive' procurement procedures because (according to M. J. Harvey, AID assistant administrator for legislative affairs)

... it had experience in systems approaches, especially with regard to water systems. It alone, of the institutions investigated, offered to devote the attention of senior people over long periods to the task, and it had the stature needed to attract French, African, and other U.S. academic cooperation in the task as well as to substantiate the validity of such an approach. (Harvey, 1974)

However, at the beginning of the project, none of the members of the MIT team had had experience with, or first-hand knowledge of, the Sahel-Sudan region. 

The 1-year period allowed for completion of the project, agreed to by MIT, was a direct result of AID's desire to meet a UN timetable for a series of meetings, a timetable to which UN eventually did not adhere (AID, 1976a).

22.2.2 Internal Factors

22.2.2.1 Design Approach

Nine disciplinary teams were established. A parallel design approach was adopted, with each team simultaneously researching a problem area (such as agriculture; hydrology; and the social, political and economic context for development) and each preparing a report. The researchers selected were associated primarily with MIT's Center for Policy Alternatives, but there were some participants from other universities as well. Many researchers assessed the problem using formal techniques such as linear programming, statistical analysis, and systems dynamics modeling, all of which, in retrospect, were too theoretical to be of direct value to policy-makers in their attempts to resolve the region's development problems (NAS, 1975a, 13; AID, 1976a).

Because several months were spent defining the problem and formulating a plan before additional members were recruited to initiate the component studies, some groups had only 6 months or less to complete their work. For example, although water resource projects had been given the highest priority by AID representatives, the hydrology group was among the last to be organized by project managers; thus, some of the potentially most relevant development alternatives were examined belatedly and in a cursory fashion (AID, 1976a).

22.2.2.2 Time Factor

Although MIT and AID officials settled on a 1-year time period for the project, both groups later agreed that it was inadequate for the task. Three of the major time-related problems were:

1. No work plan was prepared until long after the research effort was under way. According to AID, AID `should have insisted on a work plan earlier ... and a written explanation of methodology ... and managed the contract strictly against these measures of progress' (AID, 1976a, 27).

2. MIT researchers, unfamiliar with the region, were constantly in need of more information before they could conceptualize the problem (to the point of requiring an extension of the contract by 4 months).

3. Few senior scholars from MIT or elsewhere could be attracted to the project on such short notice.

22.2.2.3 Size of Research Staff

Of the 72 people listed as `personnel associated with the project', approximately one-third were students, a quarter were listed as consultants, and the remainder were professors, research assistants and advisors (the advisory committee met only once and was viewed by MIT as hostile to the project). While disciplinary subgroups were considered necessary because of the size and the scope of the study, they apparently contributed to the difficulty of integrating the components and, hence, to the isolation of disciplinary perspectives.

22.2.2.4 Integration of Component Studies

The lack of successful project conceptualization, in addition to time constraints, appears to have been a major obstacle to integrating the components of the MIT study, in spite of the fact that the study was supposed to have relied on systems analysis methodology. The study groups were allowed considerable freedom to define their own research approaches, and the project managers did not provide sufficiently strong leadership to bring them together effectively. The summary report, written by the project managers, was, in essence, a compilation of some of the primary findings from each component study, but was not a well-integrated document.

22.2.2.5 General Observations

The experience of the MIT Sahel-Sudan study suggests that an interdisciplinary systems analysis approach to climate impact assessment requires a well-defined focus, careful attention to project integration, and sufficient time to achieve the desired research objectives. The fact that the researchers, including the project managers, had little, if any, experience with the region was a major obstacle to successful project conceptualization and execution, even if it did, as the project director noted, permit MIT to approach the study `with no biases' (IEA, 1981).

22.2.3 Impact of the Study

When the study was completed in February 1975, AID requested only a few hundred copies of the final report. The report was generally ignored by the media. In its retrospective assessment, AID noted that

... the study did not live up to expectations (as well as to MIT's verbal and written commitments). Its reception has been reflective of that fact. Little reaction to it or demand for further effort by MIT have resulted once the report was distributed to AID, international organizations, and African governments in February 1975 (AID, 1976a,1).

AID representatives believed that MIT study managers were unable to conceptualize the Sahelian problem, despite repeated explanations by AID representatives (see Adams, 1973). For example, they continually treated the Sahel as six individual West African states, each with its own problems. AID, on the other hand, sought to develop new approaches to long-term development in West Africa that included coastal and Anglophone states as well as the six Francophone Sahelian states.

AID representatives expressed concern early that the MIT managers might not deliver what they expected: MIT did not convene a proposed senior think-tank retreat; the vital water systems study was not mentioned in MIT's interim report; and the two major alternative development strategies MIT focused on were viewed by AID as `unexciting' (AID, 1976a). Thus, none of the reasons suggested by AID for its selection of MIT's Center for Policy Alternatives proved to be valid. In late 1973 AID asked the National Academy of Sciences to appoint a panel to `advise AID with respect to the critical medium- and long-term natural resource management problems of the drought-stricken region of West Africa', and to critique the draft reports of the MIT group (NAS, 1978, 156; see also NAS, 1975b).

Midway through the project, AID stated its concerns in an internal memo (AID, 1974). Some changes were so critical to the project's success, the memo noted, that continuation of the contract to completion should be made conditional to implementing those changes. `The result is that MIT is not inspiring confidence and thus may do damage to future cooperation of Africans and other academics, and ultimately to the acceptability of the project', it concluded (p. 3). However, few substantive changes either in personnel or in project conception were forthcoming.

For their part, MIT study managers continue to believe that AID did not properly advise them during the project (IEA, 1981) and viewed AID's attempts to keep the research effort on track as interference that hindered the research process (Holloman, 1974, 3).

AID representatives constantly sought to salvage some parts of the MIT effort to present to the international development assistance community, but they felt that only the first two volumes (Summary Report and Agricultural Development Report) would serve to some extent. MIT has questioned this view, however, noting that many excerpts from the entire 1974 MIT report were used in an AID proposal to the US Congress for a long-term comprehensive development program for the Sahel (AID, 1976b, passim). In private correspondence, W, W. Seifert, project manager, stated that the MIT study team felt that `AID panned us but then presented our results to the Congress without reference [to the MIT study]'.

22.3 NATIONAL DEFENSE UNIVERSITY: CLIMATE CHANGE TO THE YEAR 2000

The National Defense University (NDU) study on Climate Change to the Year 2000 (NDU, 1978a), the first of a three-part assessment of the impacts of climate change on agriculture, began in the fall of 1976. The final report was released to the public in February 1978. The second part of this study, Crop Yields and Climate Change to the Year 2000 (NDU, 1980), was published in late 1980 and distributed in mid-1981. A progress report of the crop yields study had been issued in August 1978 (NDU, 1978b). The third part, The World Grain Economy and Climate Change to the Year 2000: Implications for Policy, was released for limited circulation in June 1981 in manuscript form; the final version of the third part was distributed in the spring of 1983, bringing to completion this project (Johnson, 1983).

The project represented the first attempt to quantify in a comprehensive way perceptions about climatic change. The task of Climate Change to the Year 2000 was `to define and estimate the likelihood of changes in climate during the next twenty-five years, and to construct climate scenarios for the year 2000' (NDU, 1978a, vii). Crop Yields and Climate Change indicated a broadened view of the task: `A secondary goal of this interdisciplinary effort was to advance the art of making climate impact assessments' (NDU, 1980), iii).

The entire study was supported by the US Department of Defense through the Defense Advanced Research Projects Agency. An estimated $100,000 was given, primarily to the Institute for the Future, to assist with the study design. This excluded approximately nine person-years of contributed research by a small, multidisciplinary staff detailed from the Department of Defense (DOD), the US Department of Agriculture (USDA), and the National Oceanic and Atmospheric Administration (NOAA), as well as the contributions of numerous expert panelists and advisers who received nominal honoraria.

22.3.1 Background

The pressure by US policy-makers for information about climate change began in the early 1970s with the debate among climatologists about whether the earth's atmosphere would become warmer or cooler during the next few decades. The debate was fueled by weather anomalies in 1972 that adversely affected food production and availability in some regions of the world, as well as by the publication of such popular articles as `Ominous changes in the world's weather' (Alexander, 1974), `What's happening to our climate?' (Matthews, 1976), and the Central Intelligence Agency reports on climate and agricultural production (CIA, 1974a,b, 1976). These publications have been explicitly acknowledged as having stimulated government agencies to consider the climate factor in planning (Gasser, 1981).

In light of this debate, USDA was criticized for not having considered possible future climate scenarios and their effects on US and global grain production and trade (Shapley, 1976). This led a research fellow from USDA to recommend to the NDU research directorate that a climate change study be undertaken, noting that it would relate to strategic planning and to the management of resources for national security (Gasser, 1981; Johnson, 1983, v).

22.3.2 Internal Factors

22.3.2.1 Design Approach

NDU study managers undertook three separate substudies, referred to as `tasks', on climate change, climate-crop yields, and policy implications. They used a sequential approach, with the output of one substudy consistent with the input needs of the next one. The objective of Task I was to `seek from those who were thoroughly familiar with the state of research and knowledge subjective probability judgments about the likelihood of occurrence of certain well-defined climatic events in the future', using a questionnaire survey. The responses of 24 climate experts from seven countries were used by the NDU team to develop five scenarios of climate change to the year 2000. The scenarios were designated as large cooling, moderate cooling, same as the last 30 years, moderate warming, and large warming.

In the second substudy, 35 agriculturalists were surveyed to quantify their expectations about how various combinations of changes in annual temperature and precipitation might affect crop yields in 15 countrycrop combinations. (For example, how might a 2 °C warming combined with a 10 percent increase in precipitation affect wheat yields in Australia? How might a 1 °C cooling and a 20 percent decrease in precipitation affect US corn yields?) Responses in this survey were used with information from the previously developed climate scenarios to create climate and yield scenarios. The climate-response model devised for the study predicted not only changes in average yield but also changes in the interannual variability of yields. In the third substudy, the climate and yield scenarios were used to drive a USDA econometric model of international agricultural demand, production, and trade in order to generate information on the potential implications of climate change for international grain trade and agricultural policies.

22.3.2.2 Time Factor

Because the NDU study managers reported to NDU officials and not to an outside agency, the project had only self-imposed deadlines (IEA, 1981). However, personnel attrition at NDU and the return of all but one resident study manager to their parent organizations by 1979 apparently affected the degree of internal support for the project, contributing to the long delay between the publication of the first report (February 1978) and the third one (May 1983).

22.3.2.3 Size of Research Staff

The size of the core study group was relatively small, with apparently well-established lines of communication among the staff. The second of the study's reports commented on those involved in the NDU research effort:

The focal point of the endeavor was a small, interdisciplinary staff drawn from the several branches of the Government'. Assisted by the Institute for the Future, the resident staff conducted a brokerage operation, planning the study around futuristic techniques for the solicitation and analysis of nonexistent information [sic], and orchestrating advice, `data' and insights from a host of volunteers. (NDU, 1980, xv)

22.3.2.4 Integration of Component Studies

The sequential approach facilitated the integration of the component studies. The first study produced climate scenarios; the second integrated those scenarios with information on crop yield responses to produce climate-crop scenarios; and the third study used the latter scenarios to produce projections on the global dynamics of the agricultural sector.

22.3.2.5 General Observations

The NDU study was well defined in its scope and survey methodology. It was, limited in general to the agricultural effects of climate change to the year 2000 or, more specifically, it assessed those effects only on 15 countrycrop combinations.* Adaptive measures for countering adverse impacts or for capitalizing on favorable ones were excluded, as were societal or ecological effects. With respect to their survey methodology, T. Stewart, in an internal NCAR memo (1981), expressed concern about the lack of safeguards to protect against possible judgmental bias and inconsistency and about the pooling procedure which ignored individual differences among experts.

22.3.3 Impact of the Study

Climate Change to the Year 2000 was released at the 1978 annual meeting of the American Association for the Advancement of Science (AAAS) in Washington, DC. Later, insight about the intended audience was gained from the preface to the second part: `In drafting this report we envisioned an inhomogeneous audience of meteorologists, climatologists, agronomists, economists, futurists, model builders, and policy-makers, to name a few' (NDU, 1980, xvii). The findings of the report were presented to the press at the AAAS meeting and were subsequently reported by the media in a descriptive way, with little or no analysis of the report, its methodology or its conclusions (see Kraemer, 1978; Science News, 1978; UNFAO, 1979; Sellers, 1979). During the World Climate Conference convened by the World Meteorological Organization in Geneva in February 1979, the NDU report was one of the very few reports unofficially available in large quantity to the conference participants, who represented developing as well as developed countries.

The impact of the first part of the study was evident in the Council on Environmental Quality's (CEQ) The Global 2000 Report to the President: Entering the Twenty-First Century (1980), in which the two climate-related chapters were heavily dependent on the NDU report. The Global 2000 Report has been republished in five languages and its sales have totalled well over half a million, hence it has greatly extended the circulation of the NDU study's findings. The foreword to Part III of this assessment offers additional insight into the impact that the NDU expected it to make on policy-makers:

The team was pleased to note that the Carter administration's responses to the Soviet invasion of Afghanistan took cognizance of the conclusions of draft copies of the reports sent to high-level policy-makers in key agencies. NDU is equally pleased that a draft of this concluding report is being integrated into the report of the Reagan administration's Global Issues Working Group. (Johnson, 1983, v)

Soviet scientists and policy-makers apparently took interest in the first report of the NDU study and included it as part of a survey article on the influence of climate on man's economic activity (Gruza, 1979).

*The agriculture panelists were also asked to forecast the influence of technological changes on yield trends, assuming no change in climate. The responses to this inquiry enabled the staff to assess the potential importance of technological change for crop yields relative to the estimated effects of a range of climatic change.

In our opinion, four aspects of the NDU study may have affected its usefulness and, therefore, its long-term impact.

1. The report concludes that `the most likely event will be a climate which resembles the average of the past thirty years ...' (NDU, 1978a, xix). Statements such as these, among other factors, seemingly diminished interest in the final reports of Parts II and III. This view was reiterated in Part III: `The significance of this study is that the United States can consider its proper role in the world food situation without great concern that climatic changes during the rest of this century will upset its calculations' (Johnson, 1983, 4). Yet, the negative conclusion about the agricultural implications of midrange climate change may also be a useful finding for policy-makers. One of the study's principal conclusions is that `technology, rather than climate, is likely to be the chief determinant of most crop yields in the last quarter of the twentieth century' (NDU, 1980, 2).

2. The NDU study has been referred to as `science by consensus' (NDU, 1978a). Study managers feel that this accusation is unjustified, although this perception continues to exist. In their report they noted

The experts' aggregated subjective probabilities do not reflect a consensus on any narrowly defined climatic issue, but a large majority of the climate panelists were in broad agreement, for example, that the average global temperature is not likely to change  more than half a degree Celsius by the year 2000. (NDU, 1978a, xix)

3. The relatively long time lag between publication of the first and the successive two parts of the study suggests a possible change in interest at the NDU or Department of Defense for this research activity, thereby detracting from the value of the entire three-part study. Changes in personnel involved with the study (as people retired or were reassigned) as well as apparent changes in concern by policy-makers about climate (that is, consideration of climatic factors, such as interannual variability, as being of more immediate concern than climatic change) have perhaps made the reports of the last two parts of the study less important to policy-makers (except in the use of the methodology) than might have been the case had they been published closer together.

4. 4. Discussions of the NDU research effort inevitably include comments on the `packaging' of this scientific study. Some people have noted (IEA, 1981) that the cover graphics of the first volume were designed to attract attention to the report but were not suitable for a serious scholarly report. The `packaging' of the report has had a positive aspect, however. Most people recall the study, many have retained it for their library (partly because of its content and partly due to the way it is packaged), and those who have saved it know where it is on their library shelf. This may not be the case for other climate-related impact studies of similar length, no matter how important their research findings.

22.4 INTERNATIONAL FEDERATION OF INSTITUTES FOR ADVANCED STUDY: DROUGHT AND MAN: THE 1972 CASE HISTORY

The `Drought and Man' study was sponsored by the International Federation of Institutes for Advance Study (IFIAS) in Stockholm, Sweden, and was carried out under the auspices of the director of the Food and Climate Program of the Aspen Institute for Humanistic Studies (United States). It was funded by UNEP (Kenya) and three private foundations at a cost of approximately $380,000, with an estimated $750,000 in contributed research.

The study resulted in a three-volume report. The manuscripts for all three volumes were completed in 1979. Volume 1, Nature Pleads Not Guilty, was published in August 1981(García,1981); Volume 2, The Constant Catastrophe: Malnutrition, Famines, and Drought (García and Escudero, 1982) was published the following year; and Volume 3, Case Studies, is scheduled for future publication.

The study examined in depth the impact of the climate anomalies of 1972specifically drought in many parts of the worldand their impacts on the production and availability of food, encompassing both social and physical aspects of the situation. The year 1972 was selected for assessment because in that year droughts adversely affected food production in the Soviet Union, China, Eastern Europe, Latin America, and sub-Saharan Africa.

22.4.1 Background

The underlying reasons for this study were similar to those for the National Defense University study, and to some extent for the MIT study: with the occurrence of the climate anomalies of 1972 which resulted in the first major decline in total food production since 1945, climatic factors in food and energy issues took on a greater importance than had previously been the case.

In 1972 a new interdisciplinary, international, nongovernmental organization, IFIAS, was created for the purpose of ensuring that science be used to improve the quality of human life. IFIAS established a 'climate and the quality of life' project under the leadership of W. O. Roberts, the representative of one of its member institutes, the Aspen Institute for Humanistic Studies (AIHS). Its mandate was clear:

The project is not to be viewed as a contribution to physical climate research. The project instead will focus on its main taskthe social, ethical, and humanistic implications of changes in global or regional climate. (IFIAS, 1974, 2)

The `Drought and Man' study was one of the three projects undertaken. With the selection of Rolando García as the senior study author, the study officially began in the spring of 1976.

22.4.2 Internal Factors 

22.4.2.1 Design Approach

García was given total independence for conducting the study, which included reconceptualizing the research problem several months after the project began. This was necessary, according to García (IEA, 1981), because it was not clear to him that the `official version' of the direct cause-and-effect relationship on which his investigations were expected to have been based (between drought and famine, and drought and malnutrition) was, in fact, valid. García not only redefined the research problem but, after establishing research groups in Latin America, Africa and South Asia, he also coordinated the research activities, and finally integrated all of the project's research results. Separate case studies of specific aspects of the 1972 droughts were also undertaken by individuals in the United States and Europe.

The study groups were encouraged to follow what García termed `a structural approach', which entailed research at three levels of analysis (García, 1981, Chapter 6). Basic to this approach were the interdisciplinary assessments of the events of 1972, particularly in terms of atmospheric anomalies, soil systems, agricultural systems, and social and economic systems. At the second level, researchers identified processes perceived to be responsible for the observed events, including such trends as global increase of meat and cash crop production, urbanization, and industrialization. The third level of analysis involved identifying the causes of the processes observed at the second level.

22.4.2.2 Time Factor

García has stated that time was not a serious constraint in carrying out the study (IEA, 1981), However, there were two extensions to the study: the first from August to December 1978 to allow for the completion of the regional studies, and the second to March 1979. The time element of importance with regard to the IFIAS study was the delay in the publication of its results (see Verstraete, 1984).

22.4.2.3 Size of Research Staff

The project involved a small core group under the leadership of García and several regional study groups headed by local study authors who were to submit their reports to García. The regional study groups were spread over several continents, making communication and the exchange of information between them difficult and often delayed.

22.4.2.4 Integration of Component Studies

Intellectually, the study was laid out so that component studies formed pieces of an integrated whole. Logistically, however, the project was difficult to integrate, because some regional groups were late in producing their reports or produced reports inconsistent with the study's lines of inquiry, and the geographical separation of contributors handicapped the effective integration of component studies.

22.4.3 Impact of the Study

The first volume of the study, Nature Pleads Not Guilty, was published in August 1981. To date, there have been reviews of the project based on the earliest working manuscripts (Cusack, 1981; Ruttenberg, 1981) as well as formal reviews of the final reports (Palutikof, 1983; Verstraete, 1984; Talbot and Olorunsola, 1984).

García has prepared reports on various topics stemming from the `Drought and Man' study (García 1977, 1978). In spite of these presentations, it appears that his strategy was to publish all of the findings at the completion of the research instead of piecemeal throughout the project as the conclusions arose. As a result, other authors, undertaking independent research on topics related to those in the IFIAS study, have in a sense preempted (but also reinforced) some of the conclusions drawn earlier by the IFIAS study, conclusions that had not been published until 1981.

The IFIAS study had at least two important impacts:

1. it considered and discarded the hypothesis that we are now witnessing a period of profound climatic changes;

2. it raised doubts about the validity of what García referred to as the ^,official' or generally accepted version of the sequence of events to explain the crisis of 1972 and thereafter.

That version maintained that, as a result of the decline in their agricultural production, the Soviet Union purchased large quantities of grain from the United States, which led to the depletion of US grain reserves, to shortages in the international marketplace, and to the exceptional price increases of grains and other foodstuffs. García asserted that in addition to an assessment of the impact on society of `natural forces' such as climate fluctuations, `human structures' (societal, political, economical) must be examined, and their interactions studied to reveal the actual forces at play. It is this set of forces, perhaps triggered by a physical disaster, that determines in the end what will be the effects on man and his structures (García, 1981, 4). In fact, one reviewer noted that `the influence of climate in general is largely discounted' (Palutikof, 1983, 635).

Such challenges to the `official' version often become labeled as radical or iconoclastic assessments and become part of the polarized debate over whether people (as policy-makers) or nature (as climate anomalies) represent the primary cause for worldwide food crises or shortages that are usually perceived as climate-related.

22.5 DOT CLIMATE IMPACT ASSESSMENT PROGRAM

The Department of Transportation Climatic Impact Assessment Program (CIAP) was authorized by the US Congress in 1971 and was funded at a total cost of about $20 million, with an unofficially estimated $40 million of contributed research (Dotto and Schiff, 1978). The study was a 3-year, multidisciplinary effort `to determine the regulatory constraints necessary to safeguard that future flights in the stratosphere do not result in adverse environmental effects' (Grobecker, 1974, 179). It was to assess the potential impact of fuel emissions from a large fleet of high-flying supersonic transports (SST) on stratospheric ozone concentrations and of the hypothesized effects of the resulting ozone depletion on the incidence of skin cancer as well as on climate. The US Department of Transportation had responsibility for the entire program and provided the principal investigator and project manager.

CIAP publications consist of six monographs (Bauer, 1974; Caldwell, 1974; Daly, 1974;, Hidalgo, 1974a, b; Oliver, 1974) and a Report of Findings, (Grobecker et al., 1974) which summarizes the monographs. An executive summary of the Report of Findings and a press release (DOT, 1975) were issued at a press conference in mid-January 1975, the Report of Findings in March 1975, and the monographs from September to December 1975. Additional supporting CIAP material was issued after the final CIAP conference in February 1975 (Hard and Broderick, 1976).

22.5.1 Background

In the early 1960s, public controversy developed concerning the potential value to America of supersonic aircraft. The debate on whether to develop an American SST centered primarily around the economic and political costs and benefits (Primack and von Hippel, 1972) and around two environmental issuesnoise pollution in the vicinity of airports and sonic booms (Shurcliff, 1970). At the end of the 1960s, environmental concern expanded to include the possibility of stratospheric ozone depletion (SCEP, 1970), caused not only by American SSTs but also by the European Concorde and the Soviet Tupolev. Scientists raised the possibility that trace gases (at first water vapor and later oxides of nitrogen) from fuel emissions during SST flights in the stratosphere could reduce the concentration of stratospheric ozone (Crutzen, 1971; Johnston, 1971), thereby reducing its effectiveness in shielding the ground from biologically damaging ultraviolet radiation (UV-B). It was also suggested that there was a link between increased UV-B and the incidence of skin cancer (a 1 percent reduction in stratospheric ozone was predicted to cause a 2 percent increase in skin cancer). *

In March 1971, Congress refused to fund the Boeing SST prototype program, despite European and Soviet intentions to develop theirs. The ozone depletion aspect of the SST debate appears to have been an `eleventh hour' consideration, used by opponents of the SST to block its development. In 1971 Congress established CIAP to investigate more closely the potential impact of SSTs on the stratosphere. Although the American SST debate had essentially been concluded before the CIAP program began, US decision-makers were still in need of more information on SST emissions and stratospheric ozone because they had to contend eventually with a decision about landing rights in the United States for the BritishFrench Concorde (DOT, 1976).

From the beginning of the study, CIAP managers considered this an international research activity and invited hundreds of scientists from more than ten countries (but predominantly from the United States) to participate in CIAP conferences to discuss research activities and findings.

22.5.2 Internal Factors 

22.5.2.1 Design Approach 

CIAP project managers embarked on a `crash' program to combine basic research done in the past with the stimulation of new research and with a climate impact assessment. A parallel approach was taken for the basic research activities in the areas of atmospheric science, aircraft propulsion and biological science, and in modeling activities. A sequential approach was used later in analyzing the monograph data to assess the impact of climatic changes resulting from propulsion effluents of vehicles in the stratosphere, as projected to 1990. Concurrent analyses, for which all data collected by CIAP were made available, were undertaken by the National Academy of Sciences (NAS, 1975c), and by groups in France and Belgium (COVOS, 1976), England (COMESA, 1975), USSR (Budyko and Karol, 1976), and Canada (Atmosphere, 1976).

* At the conclusion of the project, CIAP's project manager cited three instrumental factors that in his view led to the development of CIAP. `During the discussion of the U.S. supersonic transport project in 1970, the question was raised, notably by (a) James McDonald (who linked ozone depletion to skin cancer), (b) the SCEP (Study of Critical Environmental Problems), and (c) Harold Johnston (who linked a trace gas in SST fuel emissions to ozone depletion), whether impurities resulting from aircraft flight high in the stratosphere could alter the proportions of atmospheric trace constituents, with harmful results to the earth's environment' (Grobecker, 1976,1).

22.5.2.2 Time Factor

CIAP managers were given 3 years to produce their final report. During this period, much basic research was required, particularly on various aspects of stratospheric chemistry and on biological responses to increased UV-B.

Allocation of time among the six research topics encompassed by each of the monographs favored atmospheric research. The first three monographs were given several months more time than the last three. Two years for designing, conducting the research for, and compiling the results of the biospheric and economic components (monographs 5 and 6, respectively) proved too short for the task.

Time became an important factor also at the end of the study; pressures of a congressional deadline caused the project managers to release the final version of the executive summary without peer review, 2 months before issuing the full Report of Findings. The ensuing controversy about the timing and the wording of the executive summary overshadowed the impact of the report itself, and, in fact, led to congressional hearings on the matter (Carter, 1975; US Congress, 1976).

22.5.2.3 Size of Research Staff

Because large amounts of information had to be gathered from a great number of sources, the CIAP study involved hundreds of scientists, organized into disciplinary subgroups and sub-subgroups, which tended to complicate the task of integrating component studies, thereby contributing to isolation of disciplinary perspectives.

22.5.2.4 Integration of Component Studies

CIAP considered two types of activity: data gathering and integrative analysis of the data. The data gathering was accomplished by representatives of six groups working in parallel to assemble all data that seemed relevant to their particular task. Preparation of each monograph began with a conference attended by 25100 scientists. Existing information was assembled and suggestions for additional research were made. Coherence of the contributions within each major group was achieved under the editorial direction of the group leader, who was responsible for preparing the monograph. The monographs were further refined and expanded through an iterative process that involved the circulation of monograph drafts and the convening of annual workshops.

The integrative analysis of the monograph data, accomplished in the final year of the study, consisted of several steps, some under the direction of the responsible monograph chairmen, and others by the CIAP staff. Summaries of each monograph and a complete description of the analysis were compiled into a single volume, Report of Findings.

22.5.2.5 General Observations

Although initially viewed with great skepticism, CIAP's atmospheric research has since received general praise. The community of atmospheric scientists needed relatively little incentive to bridge the gaps between various atmospheric science-related disciplines, as tropospheric scientists crossed disciplinary lines to work on stratospheric research and scientists who researched the unperturbed stratosphere also undertook research on the perturbed atmosphere. In addition, interdisciplinary communication was aided by the fact that study structure cut across the deepest disciplinary line within the atmospheric sciencesthat between dynamicists and chemists.

The biospheric and economic component studies were less successful in achieving interdisciplinary coordination or in developing a viable set of research results, in part because both of these studies were under relatively severe time constraintshaving only 2 years to design the studies, conduct the required research and compile the research results. Since both studies took place simultaneously, the economists were unable to receive the biologists' research results until late in their research schedule. Perhaps more important was that the biologists could not provide the information that the economists most neededthe magnitude of the effect of UV radiation on crop plants.

In summary, CIAP's use of different study designs apparently worked well within the atmospheric sciences, where there was a higher degree of initial momentum, enthusiasm among scientists, spontaneous communication among those working in some component studies, and a relatively generous allocation of research time and funds. The mix of design approaches was less effective, however, in the biological and social science studies because:

1. studies were undertaken in parallel under circumstances where one study required the other's results,

2. little communication developed between biologists and social scientists, and

3. those scientists involved in both studies were required (expected) to conduct basic research but had been allocated insufficient time to complete their research task.

22.5.3 Impact of the Study

In January 1975, a press conference was held to announce the completion of the Climatic Impact Assessment Program and to issue an executive summary of the project. The contents of the summary were immediately challenged by some of the scientists who had worked on various aspects of the CIAP reports (for example, Donahue, 1975; Johnston, 1975a). They felt that the tone* of the summary disagreed with the tone of earlier drafts of the executive summary and with the tone of the complete Report of Findings† in that it appeared to indicate that the report found that the effects of existing SSTs (that is, the effects of a small number of lower-flying Concordes and Tupolevs) were negligable and those of a large, hypothetical fleet were technologically preventable. Also, the `scientific conclusions' listed in the summary were described in words different from those of the Report of Findings and were listed in an order which seemed, again, to bring out an up-beat (favorable) opinion regarding the SST. In addition, some topics that were identified as important in the Report of Findings (such as skin cancer effects) received little or no mention in the summary. Many of the scientists who participated in CIAP activities felt that their credibility had been permanently damaged; they cited news headlines and editorials that appeared after the release of the executive summary, such as `Scientists Clear the SST', (Christian Science Monitor, February 5, 1975) and `World SST Fleets Said Not To Damage the Ozone Blanket' (New York Times, January 21, 1975). One scientist reacted by issuing his own executive summary and principal scientific conclusions (Johnston, 1975b).

CIAP study managers declared that they had been misquoted in local editorials based on an Associated Press dispatch issued as a result of the January 1975 press conference. They argued that their summary simply emphasized the positive aspects of technological improvement, such as engine design changes to lower trace gas emissions. They contended that their conclusions were consistent with those in the National Academy of Sciences report on the environmental impact of the SST, issued in late March (NAS, 1975c), and with those of a number of other studies abroad (COMESA, 1975; COVOS, 1976), After the congressional hearing, a review of the issue in Science (Carter, 1975) concluded that the summary of conclusions was what may have been expected from a study by engineers who anticipate the best (rather than the worst) that technology might offer for the future.

According to CIAP's project manager (IEA, 1981), the controversy over the executive summary could have been avoided if time had been taken to distribute the final version to scientists for comment (as had been done with other parts of the study and earlier drafts of the summary).

The CIAP scientific findings, as an assessment of knowledge at that time and as presented in the Report of Findings, have been generally praised (except monograph 6) as a good summary of the state of the art, even by most scientists who vehemently criticized the executive summary. In fact, an outgrowth of CIAP, the Lawrence Livermore Laboratory's High Altitude Pollution Program (HAPP), has been supported since 1975 by the Federal Aviation Administration (Luther, 1980; Wuebbles, 1981; Knox et al., 1983). CIAP also appears to have stimulated research on the effects that trace gases from sources other than SSTs (for example, chlorofluorocarbons from aerosol cans, refrigerants, etc., oxides of nitrogen from agricultural fertilizers) might have on the concentration of stratospheric ozone or on climate (Mormino et al., 1975).

As a corollary to the CIAP studies, DOT requested the US National Academy of Sciences to issue an independent report, Environmental Impact of Stratospheric Flight (NAS, 1975c). Thus, in spite of the controversy surrounding the executive summary, CIAP mobilized researchers in an area of atmospheric sciences that had been relatively neglectedthe stratosphere (Hoffert and Stewart, 1975; MacDonald, 1976; Bastian, 1982).

* On the question of tone, tenor, or mood of scientific reports in general, and their potential impact on the effectiveness of a report, see Martin (1979).

† For example, the August 20, 1974, version of the summary (`Executive Conspectus', US Department of Transportation, Climatic Assessment Program, Review Draft) was the last one sent to Harold Johnston for comment (private communication). Johnston was one of those scientists who strongly criticized the final version of the summary.

22.6 NAS STUDIES ON THE EFFECTS OF CHLOROFLUOROCARBON RELEASES ON STRATOSPHERIC OZONE

The US National Academy of Sciences (NAS) Panel on Atmospheric Chemistry was appointed in March 1975 to assess `the extent to which man-made halocarbons, particularly chlorofluoromethanes (CFMs), and potential emissions from the space shuttle might inadvertently modify the stratosphere' (NAS, 1976a, vii). The panel issued its report, Halocarbons: Effects on Stratospheric Ozone, in 1976 (after a delay of some months caused by the panel's need to evaluate newly developed scientific information). The panel's parent Committee on Impacts of Stratospheric Change (CISC) undertook a study to address `the question of biological and climatic effects of ozone reduction and the appropriate policy consequences of both our present knowledge and the knowledge we are likely to have in the future' (NAS, 1976b, viii). CISC issued its report, Halocarbons: Environmental Effects of Chlorofluoromethane Release, at the same time as the Atmospheric Chemistry panel issued its report. The two reports were produced at a total cost of more than $300,000 and were funded by the National Aeronautics and Space Administration (NASA), National Science Foundation (NSF), Environmental Protection Agency (EPA) and National Oceanic and Atmospheric Administration (NOAA). Computer studies were supported by the Federal Aviation Administration (FAA).

22.6.1 Background

Fluorocarbons, Freon 11 (CFC1₃) and Freon 12 (CF₂Cl₂), are manufactured worldwide for use in aerosol spray cans, refrigerators, air conditioners, certain types of plastic foams, solvents and cleaning agents, among other uses. As late as the mid-1970s, about 50 percent of the fluorocarbons in the United States had been used as aerosol propellants and up to 90 percent of those were used in such products as hairsprays, deodorants and antiperspirants.

In the early 1970s, scientific research activities that would eventually tie the CFCs closely to stratospheric ozone depletion were under way in different places and in different disciplines, such as the activities of James Lovelock, a scientist in England interested in developing highly sensitive instrumentation to measure fluorocarbons in the atmosphere. A few atmospheric chemists (for example, Molina and Rowland, 1974; Stolarski and Cicerone, 1974), working independently focused attention on chlorine in the stratosphere. Molina and Rowland suggested that manmade chlorofluorocarbons, while inert in the lower atmosphere, were diffusing upward into the stratosphere where they would photodissociate, releasing free chlorine to react catalytically with ozone, thereby significantly depleting the ozone layer. They suggested that, given the long time lag between the emission of CFCs in the atmosphere and their diffusion to the stratosphere, there had already been a serious atmospheric build-up of CFCs. Estimates from different sources of eventual ozone depletion resulting from CFC releases ranged from 5 to 30 percent, based on projected rates of CFC production.

Interest in the effects of trace gases on stratospheric ozone was heightened by the CIAP research efforts and by concern about fuel emissions from the American space shuttle. In the summer of 1974, while involved in the preparation of the final report for CIAP, the Climatic Impacts Committee determined that the impact of photodissociated chlorine atoms released from CFCs was potentially a more serious threat to the ozone layer than space shuttle exhaust. This concern led to the establishment of an ad hoc panel to ascertain whether the chlorine issue was important and worthy of a more serious investigation. In 1975 NAS charged CISC with the task of investigating the impact of CFCs on stratospheric ozone depletion and the effects of ozone depletion on the lower atmosphere and at ground level.

22.6.2 Internal Factors 

22.6.2.1 Design Approach

The two studies (NAS, 1976a, b) assessed the following scientific contentions: 

1. CFC releases to the atmosphere can lead to ozone depletion; and

2. ozone depletion might have serious, deleterious effects on life on earth. 

The studies used both a sequential and a parallel approach. During the first few months of the studies, a 13member panel on atmospheric chemistry sorted through evidence on the possible links between manmade CFC releases and stratospheric ozone depletion. Once this panel was working, a multidisciplinary committee, also with 13 members, representing various physical (but no social) science disciplines was established to investigate the broader problems related to ozone depletion.

The procedures used in these studies were informal and ad hoc. Panel and committee members were chosen because they had a neutral viewpoint, a high standing in the scientific community, and access to the latest information on questions of concern to the study. Monthly and bimonthly meetings were held to update and exchange information and to work toward a consensus. The committee chairman wrote the main body of the report, which was then reviewed by all committee members as well as by an outside body of scholars.

In 1977, the US Congress called on the Academy to conduct an additional study, covering not only the physical sciences, including biology, but also the `health and welfare effects', as well as `methods for control ... including alternatives, costs, feasibility, and timing' (US Congress, 1977). For these CFC studies (NAS, 1979), the research for the physical and biological sciences section was conducted by CISC, the same committee that had conducted the previous Academy study of CFCs. A second committee, the Committee on Alternatives for the Reduction of Chlorofluorocarbon Emissions (CARCE), was formed to look at the regulatory aspects of the CFC issue. CISC in its physical science studies used an approach similar to the one used in its 1976 study. Stratospheric chemistry was first assessed, followed by an interdisciplinary assessment of climatic, biological and human health impacts. Small panels (fewer than five) conducted disciplinary substudies, reported them back to the full committee, and each contributed one chapter to the committee's report (NAS, 1979).

The societal impact component (undertaken by CARCE) concerned possible options for CFC emission control. This component proved to be unavoidably political. Rather than attempt to establish a committee of ^,unbiased' scientific experts (as it had done with the 1976 report), the Academy sought to include a balance of viewpoints. Committee membership included representatives from corporations, conservation, labor and consumer groupsin addition to university professors with backgrounds in engineering, economics and law. CARCE divided its inquiry into topics related to industrial technology and socioeconomic impacts, and panels were selected to investigate each topic.

Both CARCE and CISC worked closely together through informal communication links and a number of joint meetings. Their respective chairmen maintained regular communication, and their executive secretaries shared an office at the Academy. The committee issued a joint report, Protection Against Depletion of Stratospheric Ozone by Chlorofluorocarbons (NAS, 1979) that represented a consensus of both committees and was reviewed by an outside panel in accordance with the Academy's review procedure.

The Academy's approach to these studies apparently balanced the treatment of the complex physical aspects of the CFC issue with an assessment of possible options for CFC control, as well as with an assessment of the societal impacts that included discussion of the potential effectiveness of different control options. While covering both policy and physical science considerations, the approach left large gaps between these two areas of research. For example, the paucity of information on the effects of UV radiation on plant growth meant that CISC would be unable to assess meaningfully the non-human biological consequences of ozone depletion. As for CARCE, no attempt was made to assess the economic and social costs of ozone depletion. While the US Environmental Protection Agency wanted the Academy to undertake a full cost-benefit analysis, the Academy chose instead to review critically three studies on costs and benefits of CFC emissions regulation.

22.6.2.2 Time Factor

The NAS studies are different from the other studies considered in that they are part of a series of assessments. Time constraints do not appear to have affected the effectiveness of these studies. While it does have deadlines to meet, the Academy's prestige and the potential authority of its reports gives it an additional degree of flexibility. As a further step in this ongoing assessment (biennial review), two other Academy studies related to the CFC-ozone depletion issue were published in early 1982 (NAS 1982a, b).

22.6.2.3 Size of Staff and Integration of Component Studies

The NAS deliberately keeps its committees small (15 members or fewer) to increase the flexibility of its studies and the ease of achieving integrated findings. Where specialized disciplinary study is needed, it forms panels that report back to the parent committee. This format seems to aid multidisciplinary exchange. Integration was achieved through cooperation of those responsible for the study, both committee and panel members. The general report of each NAS study on the CFC issue was produced by a multidisciplinary group as a consensus position.

22.6.2.4 General Observations

In early 1976, a few months before the report was published, the resiliency of the NAS study process as well as the potential importance of its reports was tested; new scientific questions arose, which needed to be accounted for in the study's findings. The time needed to evaluate these new inputs delayed the release of the NAS report by several months (Handler, 1976). The panel's ability to assess new scientific information and to secure agreement on delaying the release of the final report shows a high degree of flexibility afforded to NAS studies that might be difficult to find in most other climate-related impact assessments.

The NAS recruitment policies for the panel gave an image of integrity and impartiality to the panel and the final reports. As noted earlier, the report represented a consensus and was carefully worded to avoid a strong position favoring any side of the CFC dispute. The report asserted that CFCs might pose a serious hazard, but also suggested that regulations on some uses and releases of CFCs be delayed for up to 2 years, based on the view that `the impact on the world of waiting a couple of years before deciding whether or not to regulate the uses and releases of Freon 11 and Freon 12 is small although we are uncertain just how small' (NAS, 1976b, 9).

22.6.3 Impact of the Studies

The Academy reports appear to have had a major impact in policy-making circles. Interest in them was evident even before the reports were issued. For example, Bastian noted that the recommendations of the interagency task force on the Inadvertent Modification of the Stratosphere (IMOS, 1975) were tied to the NAS study: `[IMOS] set forth a timetable for decision-making by the Executive Branch ... [calling for] initiation of rulemaking for some type of restrictions on fluorocarbon use following issuance of the NAS report' (Bastian, 1976, 12). The study was expected to have significant influence on regulations of CFC emissions. Bastian noted that agencies such as the Consumer Product Safety Commission and the Food and Drug Administration `have been petitioned by the Natural Resources Defense Council and several states to take some action, and have not done so, waiting for the results of the NAS study to be available' and that `the various industries potentially affected by any regulation are aware of the importance of the NAS study' (Bastian, 1976,2).*

The NAS study confirmed the concern of atmospheric scientists, who since 1975 had been looking into the possible effects on climate of fluorocarbons and other infra-red-absorbing gases in the troposphere through the greenhouse effect (Ramanathan, 1975, 1980; Dickinson and Chervin, 1979; Ramanathan and Dickinson, 1979).

Although scientists recognize that there is a large degree of scientific uncertainty surrounding the interactions of CFCs with stratospheric ozone and that there is still no general agreement on the scientific issue (UK Department of Environment, 1976, 1979; Harris and Kosovich, 1981; UNEP, 1981), worldwide research and regulatory activity has been steadily developing since the mid-1970s. The Environment Committee of the Organization for Economic Cooperation and Development (OECD) agrees that `CFCs constitute a potentially serious problem which should be reviewed periodically' (Harris and Kosovich, 1981, 45) and is preparing reports on the scientific, economic and legal aspects of this issue.

* As a later example of the importance of NAS reports on this issue, Don Clay, Director of the Office of Toxic Substances of the EPA, testifying to Congress, noted with trespect to the 1982 NAS reports that `EPA is awaiting the NAS assessment, the NASA/WMO report and other pertinent scientific information, and is reviewing and analyzing the issues raised .... I can assure you that further EPA action would be based upon reasonable scientific evidence and the most sound economic analysis available.' (Hearings of the House Subcommittee on Health and Environment of the Committee on Energy and Commerce, November 5, 1981)

All major CFC-producing nations and many users (including the ten member nations of the European Economic Community, Canada, Japan and several other nations) have taken actions to control CFC emissions, by reducing or banning aerosol propellant uses of CFC-11 and -12 and/or limiting production capacity for all uses of those CFCs to present levels. Many of these nations are considering further emission controls for various CFC uses and are working with industry on the technology required to do so. In 1977, the United Nations Environment Programme (UNEP) established a Coordinating Committee on the Ozone Layer (CCOL) to assess the significance of research and other information relating to stratospheric ozone protection and to recommend further efforts. The UNEP Governing Council has also initiated work on a global framework convention for the protection of stratospheric ozone. Studies have been completed within the World Meteorological Organization Ozone Project on the possible impact of ozone variability on climate (UNEP, 1981; WMO, 1982).

In the United States, the first phase of CFC regulatory activity ended in 1978 when the EPA ordered the termination of the manufacture and use of most CFC-propelled aerosols. A second phase in CFC regulations is presently under consideration as the US Congress and Federal agencies debate the need for additional regulatory action on nonaerosol uses of CFCs (Nautilus, 1981).

22.7 CONCLUSIONS

Discussions on ways to improve the science of climate-related impact studies took place at a workshop convened by the Institute for Energy Analysis, Oak Ridge Associated Universities, 2930 June 1981, in which the principal investigators of four of the five studies participated (no representative of the NAS chlorofluorocarbon studies was able to attend). In addition to the successes and problem areas that become evident from the studies reviewed, it was suggested that these studies could serve to highlight important aspects of conducting climate-related impact assessments, from project conceptualization to dissemination of the research findings. The reader should keep in mind that the following are suggestions and are not meant to be definitive guidelines. 

22.7.1 MIT Sudan-Sahel Study

1. All five studies faced delays either in project completion or in the dissemination of results, but for the MIT study the time factor proved to be an insurmountable problem. A year after the project had been completed, AID acknowledged that `... in one year, no group could have been expected to produce a once-and-for-all conceptual framework for the entire region. The task is much more a gradual one, a learning process systematically pursued over a much longer period' (AID, 1976a, 9). Time constraints were even more critical in this specific case because MIT personnel were unfamiliar with the region they were researching, no matter how capable they may have been in their own disciplines. These and other time constraint problems, precipitated by the eagerness of AID to acquire research results and the eagerness of MIT to undertake the research project and to comply with its own 1year time frame, could have been minimized had MIT convened the retreat of senior scholars and policy-makers requested by AID. Another facet of this problem was that those negotiating the contract with AID were not directly involved in the project and had no precise knowledge of what research efforts and how much time might be required to complete the project successfully.

2. Under certain circumstances researchers not directly familiar with the topic to be investigated will be selected for involvement in a project. While this may be a valuable tactic (i.e., no preconceived biases, a chance for `new thinking'), measures need to be taken to assure that additional researchers familiar with the topic are associated with the project from the outset and in a significant way; for example, authoritative (as opposed to nominal) advisers on an advisory council that is to be convened early in the project, one that meets often and to whose advice serious consideration is given.

3. At the outset of the research effort, study objectives, problem conceptualization and design approach must be made explicit and agreed to by the contractor and the group undertaking the research activity. As the project develops, the need may arise to modify the original research agenda including, possibly, the methodological approach or the underlying assumptions. Such re-evaluation some months into the project can be healthy and there must be enough flexibility (of time, of funds, of perceptions of principal investigators) to accommodate such changes in research design if they are deemed necessary.

4. The selection of the principal investigator is of major importance. The principal investigator will be, directly or indirectly, the driving force for the study. For example, he/she will be instrumental in the selection of other project managers and researchers, as well as in the conceptualization of the research problem and in determining the research design.

22.7.2 NDU: Climate Change to the Year 2000

1.  The NDU study's methodological approach and the packaging of its final report have attracted much attention. Its methodological approach has been viewed as useful because policy-makers often have to make decisions relying on expert judgment, even though uncertainties surrounding a particular issue may be very large. With the methodology chosen, the NDU staff sought to identify and quantify the subjective views of selected climate experts on climate-related issues. In their third and final volume they suggest that the methodology developed in this study could be applied to CO₂ impact assessments and to other similar problems (Willett, 1981, vi). Yet the methodological approach did also generate criticism that the study represented ^,science by consensus' rather than evaluative judgment of climate experts concerning future climate scenarios.

As for the packaging of the NDU report, few can argue that the report stands out visually from the others; for groups without well-established reputations there may be some value in packaging their results so as to attract the attention of their target audiences.

2. The strengths and weaknesses of the chosen methodological approaches need to be explicitly identified so that the research findings can be judged appropriately. Once a study's methodological underpinnings have been questioned, the credibility of its conclusions, no matter how valid they maybe, will be subject to challenge.

3. While good packaging cannot replace `good science', it can assist in the distribution of `good science'.

22.7.3 IFIAS: Drought and Man

1. The IFIAS_. project highlights at least two potential problems that project managers might encounter during these types of studies: a need to redefine the study after it has begun and a delay in publication of the study's findings. Delays of only a few months in the publication of the 1975 NAS chlorofluorocarbon reports caused concern among those awaiting the panel's findings. Policy-makers awaited the NAS reports and had to accept the delay. Such may not be the case for policy-makers awaiting the results of other climate-related impact assessments.

2. It is important to identify and make explicit the underlying assumptions on which a research project is developed. If some of those assumptions are proven not to be valid, they can be corrected early in the research process. This, too, reinforces the need for flexibility of the research timetable, as well as of the research team, to readjust activities part way through the project.

3. Reports need to be published in a timely fashion to avoid long delays between submission of the manuscript and publication and distribution of the final project. Likewise, the lag time between publication of multiple volumes from the same research project should be minimized if they are to be useful for policy-makers.

22.7.4 DOT Climatic Assessment Program

CIAP's problem with its executive summary highlights in general the importance of executive summaries. It was with respect to this final stage of the CIAP process (that is, the preparation and dissemination of the executive summary) that charges of political interference in the scientific aspects of CIAP arose.

Other reports, too, had problems with their executive summaries. For example, the MIT Summary Report (Volume 1) provides less information on the West African Sahel than already existed (for example, UNFAO, 1962). The executive summary of the NAS report on CFCs and stratospheric ozone depletion was challenged by industry representatives on the ground that it omitted necessary caveats, thereby giving the appearance of a higher degree of scientific certainty about the chlorofluorocarbon issue than was warranted. In contrast, the British reports on stratospheric ozone depletion provided conclusions but not executive summaries. As for the IFIAS study, García noted (IEA, 1981) that he did not produce an executive summary in order to avoid what he termed misinterpretation of his report's findings.

Executive summaries are often more important to policy-makers and the public than the reports that they summarize. They are often designed for the busy decision-makers who have little time (and little expertise) to delve into the lengthier technical report from which they would otherwise have to draw their own, probably less informed, conclusions. These summaries must accurately reflect the tone and the content of the longer report as well as the degree of uncertainty that surrounds its conclusions.

22.7.5 NAS Chlorofluorocarbon Studies

The NAS studies on stratospheric ozone raised the issue of `term' (that is, one-time) studies as opposed to ongoing assessments. It is important to qualify what might be viewed as an ongoing assessment. In the case of the NAS ozone depletion studies, although several studies have been done since 1970 in which the general topic of ozone depletion has been addressed, the NAS staff members as well as the scientists involved in the preparation of the individual reports changed from one assessment to the next. The NAS assessment process, however, does seem to have more flexibility than the one-time impact assessment. One might argue that an ongoing climate-related impact assessment program can follow up on its earlier findings by taking advantage of new developments in instrumentation, data analysis techniques and enhanced scientific understanding.

`Term' or one-time projects that are expected to present definitive results within a specific period of time may not have this degree of research flexibility. On the other hand, it could be argued that once important issues and areas for concern have been identified in a one-time impact assessment, it is possible to focus on those few issues and to establish smaller but more specific follow-up assessments, such as was done by establishing the High Altitude Pollution Program as a result of the CIAP effort.

The objectives for undertaking an assessment must match the way the assessment will be conducted. Certain research objectives (for example, to determine the state of the art with respect to a climate-related problem) might require a one-time study, while others might call for an ongoing assessment (for example, if the range of uncertainties surrounding our knowledge were so great that periodic assessments could serve to monitor the state of knowledge as it changed over time). While either approach is valid, both types should be considered and matched to a study's objectives.

22.7.6 Final Observations

There is much to learn from past experience of previous climate impact assessments. In fact, a few reviews of such programs now exist (Mormino et al., 1975; AID, 1976a; Gasser, 1981), in addition to reviews of sets of environmental studies (Mar, 1977; McHale, 1981). It would be valuable for managers and funding agencies of future climate impact assessment studies if retrospective reviews of each project were to become a required component in a formal climate-related impact study process. A Chinese proverb is appropriate: To know the road ahead, ask those coming back.

ACKNOWLEDGMENTS

The authors of this paper wish to acknowledge the support, financial and moral, of the scientists and staff at the Oak Ridge Associated Universities Institute for Energy Analysis for the NCAR/IEA workshop on `Improving the Science of Climate-related Impact Studies'. They would also like to thank the principal investigators of the impact studies reviewed as well as the scientists and staff at the National Center for Atmospheric Research for their support and encouragement. Special appreciation is due to the many scientists who took the time to critique the several drafts of this paper, without whose comments this paper could not have been written.

An earlier version of this paper, which was commissioned for the SCOPE project, appeared in W. C. Clark (Ed.) (1982) Carbon Dioxide Review: 1982, Oxford University Press.

REFERENCES

Adams, S. C. Jr. (1973). Letter to Director, Center for Policy Alternatives, MIT, from Adams, AID's Assistant Administrator for Africa (November 9).

AID (Agency for International Development) (1972). Development and Management of the Steppe and Brush-Grass Savannah Zone Immediately South of the Sahara. AID In-House Report, Washington, DC: 161 pages.

AID (Agency for International Development) (1974). Internal AID Review of MIT Interim Report: Memo (March). Washington, DC.

AID (Agency for International Development) (1976a). Retrospective Assessment of the MIT Study. AID internal memo (P. Lyman). Washington, DC.

AID (Agency for International Development) (1976b). A Report to Congress: Proposal for a Long-Term Comprehensive Development Program for the Sahel. Part II, Technical Background Papers. Washington, DC.

Alexander, T. (1974). Ominous changes in the world's weather. Fortune (February), 90-95, 142, 146, 150-152.

Atmosphere (Journal of the Canadian Meteorological Society) (1976). Entire Vol. 14, No. 3 deals with findings of the Canadian program.

Bastian, C. (1976). National Science Foundation memo to files (August 2, 1976). 

Bastian, C. (1982). The formulation of federal policy. In Ward, R. (Ed.) Stratospheric Ozone and Man. CRC Press, Boca Raton, Florida.

Bauer, E. (1974). CIAP Monograph 1: The Natural Stratosphere of 1974. US Dept. of Transportation, Climatic Impact Assessment Program, Washington, DC.

Brown, D. S. (1973). U.S.A. statement. In Final Report on the Meeting of the Sudano-Sahelian Mid- and Long-Term Programme, 28-29 June 1973, Geneva. UN Special Sahelian Office, New York.

Budyko, M. I., and Karol, I. L. (1976). A study of CIAP. Meteorologiya: Gidrologiya (Soviet Meteorology and Hydrology), No. 9, 103-111 (in Russian). Translated by Allerton Press Journal Program, New York, 1977, pp. 82-91.

Caldwell, M. M. (1974). CIAP Monograph 5: Impacts of Climatic Change on the Biosphere. US Dept. of Transportation, Climatic Impact Assessment Program, Washington, DC.

Carter, L. J. (1975). Deception charged in presentation of SST study. Science, 190 (November 28), 861.

CEQ (Council on Environmental Quality) (1980). The Global 2000 Report to the President: Entering the Twenty-First Century. US Government Printing Office, Washington, DC.

CIA (Central Intelligence Agency) (1974a). A Study of Climatological Research as it Pertains to Intelligence Problems (August). Washington, DC.

CIA (Central Intellingence Agency) (1974b). Potential Implications of Trends in World Population, Food Production, and Climate (August, OPR-401). Washington, DC. 

CIA (Central Intellingence Agency) (1976). USSR: The Impact of Recent Climate Change on Grain Production (October, ER 76-10577 U). Washington, DC. 

COMESA (1975). The Report of the Committee on Meteorological Effects of Stratospheric Aircraft, 1972-75. Meteorological Office, United Kingdom.

COVOS (1976). Comité d' Études sur les Conséquences des Vols Stratosphériques. Meteorological Society of France, Boulogne, France.

Crutzen, P. J. (1971). Ozone Production rates in an oxygen-hydrogen-nitrogen atmosphere. Journal of Geophysical Research, 76 (30), 7311.

Cusack, D. (1981). Variabilidad climática y el hambre mundial: Solución técnica o solución política? Interciencia, 6 (July-August), 288-291.

Daly, G. (1974). CIAP Monograph 6: Economic and Social Measures of Biologic and Climatic Change. US Department of Transportation, Climatic Impact Assessment Program, Washington, DC.

Dickinson, R., and Chervin, R. M. (1979). Sensitivity of a general circulation model to changes in infrared cooling due to chlorofluoromethanes with and without prescribed zonal ocean surface temperature change. Journal of the Atmospheric Sciences, 36 (December), 2304-2319.

Donahue, T. M. (1975). The SST and ozone depletion: Letter. Science, 187 (March 28),1142-1143.

DOT (Dept. of Transportation) (1975). Press release, Office of the Secretary (January 21). Washington, DC.

DOT (Dept. of Transportation) (1976). The Secretary's Decision on Concorde Supersonic Transport (February 4). Washington, DC.

Dotto., L., and Schiff, H. (1978). The Ozone War. Doubleday & Co., New York. 

García, R. (1977). Drought, Desertification and the Structural Stability of EcosystemsThe Case of Latin America. Paper prepared for UNEP's Desertification Secretariat. Geneva: 40 pages.

García, R. (1978). Climate impacts and socioeconomic conditions. In National Academy of Sciences, International Perspectives on the Study of Climate and Society, pp. 43-47. NAS, Washington, DC.

García, R. (1981). Drought and Man: The 1972 Case History. Vol. 1: Nature Pleads Not Guilty. Pergamon, New York.

García, R., and Escudero, J. C. (1982). Drought and Man: The 1972 Case History. Vol. 2: The Constant Catastrophe: Malnutrition, Famines, and Drought. Pergamon, New York.

Gasser, W. R. (1981). Climate change to the year 2000 and possible impacts on world agriculture: A review of the National Defense University Study. Paper presented to the Institute for Energy Analysis/NCAR Workshop on Improving the Science of Climate-Related Impact Studies, June 30-July 2, 1981, Oak Ridge, Tennessee.

Glantz, M. H. (Ed.) (1976). Politics of a Natural Disaster: The Case of the Sahel Drought. Praeger, New York.

Grobecker, A. J. (1974). Research program for assessment of stratospheric pollution. Acta Astronautica, 1, 179-224.

Grobecker, A. J. (1976). The CIAP report of findings: The effects of stratospheric pollution by aircraft. In Hard, T. M., and Broderick, A. J. (Eds.) Proceedings of the Fourth Conference on the Climatic Impact Assessment Program. NTIS, US Dept. of Commerce, Springfield, Virginia.

Grobecker, A. J., Coroniti, S. C., and Cannon, R. H., Jr. (1974). Report of Findings: The Effects of Stratospheric Pollution by Aircraft. US Dept.. of Transportation, Climatic Impact Assessment Program, Washington, DC.

Gruza, G. V. (1979). Fluctuations of climate and man's economic activity. Hydrometeorology, Vol. 3 (translation from the Russian by the National Science Foundation). 

Handler, P. (1976). Letter to H. G. Stever, National Academy of Sciences (April 16). 

Hard, T. M., and Broderick, A. J. (Eds.) (1976). Proceedings of the Fourth Conference on the Climatic Impact Assessment Program. NTIS, US Dept. of Commerce, Springfield, Virginia.

Harris, F. A., and Kosovich, J. (1981). International action to protect the ozone layer. Paper presented to the Air Pollution Control Association Annual Meeting, June 21-26, Philadelphia, Pennsylvania.

Harvey, M. J. (1974). Letter to US Congressman Diggs from Harvey, AID'S Assistant Administrator for Legislative Affairs (April 23).

Hidalgo, H. (1974a). CIAP Monograph 3: The Stratosphere Perturbed by Propulsion Effluents. US Dept. of Transportation, Climatic Impact Assessment Program, Washington, DC.

Hidalgo, H. (1974b). CIAP Monograph 4: The Natural and Radiatively Perturbed Troposhere. US Dept. of Transportation, Climatic Impact Assessment Program, Washington, DC.

Hoffert, M. I, and Stewart, R. W. (1975). Stratospheric ozonefragile shield? Astronautics and Aeronautics (October), 42-55.

Holloman, J. H. (1974). Letter to Adams, AID'S Assistant Administrator for Africa (February 14).

IEA (Institute for Energy Analysis) (1981). Notes from discussions, Workshop on Improving the Science of Climate-Related Impact Studies. IEA, Oak Ridge, Tennessee.

IFIAS (1974). The Impact on Man of Climate Change: Report of an IFIAS Project Workshop, Meteorological Institute, University of Bonn, May 6-10, 1974. International Federation of Institutes for Advanced Study, Stockholm, Sweden.

IMOS (1975). Fluorocarbons and the Environment: Report of Federal Task Force on Inadvertent Modification of the Stratosphere (IMOS) by the Council on Environmental Quality (CEQ) and Federal Council for Science and Technology (FCST). US Government Printing Office, Washington, DC.

Johnson, D. G. (1983). The World Grain Economy and Climate Change to the Year 2000: Implications for Policy. National Defense University Press, Washington, DC.

Johnston, H. S. (1971). Reduction of stratospheric ozone by nitrogen oxide catalysts from supersonic transport exhaust. Science, 173 (August 6), 517-522.

Johnston, H. S. (1975a). Supersonic transports: Letter. Chemical and Engineering News (April 25), 5.

Johnston, H. S. (1975b). Panel discussion. In Hard, T. M., and Broderick, A. J. (Eds.) Proceedings of the Fourth Conference on the Climatic Impact Assessment Program, p. 35. NTIS, US Dept. of Commerce, Springfield, Virginia.

Knox, J. B., MacCracken, M. C., Dickerson, M. H., Gresho, P. M., Luther, F. M., and Orphan, R. C. (1983). Program Report for FY 1982 Atmospheric and Geophysical Sciences Division of the Physics Department. Lawrence Livermore National Laboratory, University of California, Livermore, California.

Kraemer, R. S. (1978). Meeting reviews: Session on climatic futures at the annual meeting of the AAAS, 17 February 1978. Bulletin of the American Meteorological Society, 59 (7), 822-823.

Luther, F. M. (1980). Annual Report of Lawrence Livermore National Laboratory to the FAA on the High Altitude Pollution Program1980. University of California, Livermore, California.

MacDonald, G. J. (1976). Panel discussion of CIAP report of findings. In Hard, T. M., and Broderick, A. J. (Eds.) Proceedings of the Fourth Conference on CIAP. NTIS, US Dept. of Commerce, Springfield, Virginia.

Mar, B. (Principal Investigator) (1977). Regional Environmental Systems: Assessment of RANN Projects. University of Washington, Dept. of Civil Engineering, Seattle, Washington.

Martin, B. (1979). The Bias of Science. Society for Social Responsibility in Science, O'Connor, Australia.

Matlock, W. G., and Cockrum, E. L. (1974). A Framework for Evaluating Long-Term Strategies for the Development of the Sahel-Sudan Region; Vol. 2, A Framework for Agricultural Development Planning. MIT Center for Policy Alternatives, Cambridge, Massachusetts.

Matthews, S. W. (1976). What's happening to our climate? National Geographic, 50 (5), 576-615.

McHale, M. C. (1981). Ominous Trends and Valid Hopes: A Comparison of Five World Reports. Hubert H. Humphrey Institute of Public Affairs, University of Minnesota, Minneapolis, Minnesota.

Molina, M. J., and Rowland, F. S. (1974). Stratospheric sink for chlorofluoromethanes: Chlorine atomic-catalyzed destruction of ozone. Nature, 249, 810-812.

 Morentz, J. W. (1980). Communication in the Sahel drought: Comparing the mass media with other channels of international communication. In National Academy of Sciences, Disasters and the Mass Media, pp. 158-186. National Academy of Sciences, Washington, DC.

Mormino, J., Sola, D., and Patten, C. (1975). Climatic Impact Assessment Program: Development and Accomplishments, 197175. US Dept. of Transportation, Washington, DC: 206 pages.

NAS (National Academy of Sciences) (1975a). Arid Lands of Sub-Saharan Africa: Staff Final Report, July 1974December 1974. NAS, Washington, DC.

NAS (National Academy of Sciences) (1975b). Arid Lands of Sub-Saharan Africa: Staff Progress Report, September 1973 June 1974. NAS, Washington, DC.

NAS (National Academy of Sciences) (1975c). Environmental Impact of Stratospheric Flight. NAS, Washington, DC.

NAS (National Academy of Sciences) (1976a). Panel on Atmospheric Chemistry. Halocarbons: Effects on Stratospheric Ozone. NAS, Washington, DC.

NAS (National Academy of Sciences) (1976b). Committee on Impacts of Stratospheric Change. Halocarbons: Environmental Effects of Chlorofluoromethane Release. NAS, Washington, DC.

NAS (National Academy of Sciences) (1978). Programs of the Board on Science and Technology for International Development (BOSTID): Summary of Activities, 1970-78. NAS, Washington, DC.

NAS (National Academy of Sciences) (1979). Protection Against Depletion of Stratospheric Ozone by Chlorofluorocarbons. NAS, Washington, DC.

NAS (National Academy of Sciences) (1982a). Biological Effects of the Increased Solar UV Radiation. NAS, Washington, DC.

NAS (National Academy of Sciences) (1982b). Chemistry and Physics of Stratospheric Ozone Depletion. NAS, Washington, DC.

Nautilus (1981). A decision to further regulate CFCs hinges on a value judgment. Weather and Climate Report, 4 (8), 3.

NCAR (National Center for Atmospheric Research) (1981). Internal memo, T. Stewart. 

NDU (National Defense University) (1978a). Climate Change to the Year 2000: A Survey of Expert Opinion. NDU, Washington, DC.

NDU (National Defense University) (1978b). Crop Yields and Climate Change: The Year 2000Progress Report. NDU, Washington, DC.

NDU (National Defense University) (1980). Crop Yields and Climate Change to the Year 2000, Vol. 1. NDU, Washington, DC.

Oliver, R. C. (1974). CIAP Monograph 2: Propulsion Effluents in the Stratosphere. US Dept. of Transportation, Climatic Impact Assessment Program, Washington, DC. 

Palutikof, J. (1983). Drought without water. Nature, 203, 635.

Primack, J., and von Hippel, F. (1972). Scientists, politics and SST: A critical review. Bulletin of the Atomic Scientists (April), 24-30.

Ramanathan, V. (1975). Greenhouse effect due to chlorofluorocarbons: Climatic implications. Science, 190 (October 3), 50-52.

Ramanathan, V. (1980). Climatic effects of anthropogenic trace gases. In Bach, W., et al. (Eds.) Interactions of Energy and Climate, pp. 269-280. D. Reidel Publishing Co., Netherlands.

Ramanathan, V., and Dickinson, R. (1979). The role of stratospheric ozone in the zonal and seasonal radiative energy balance of the earth-troposhere system. Journal of the Atmospheric Sciences, 36 (6), 1084-1104.

Ruttenberg, S. (1981). Climate, food and society. In Slater, L. E., and Levin, S. K. (Eds. ) Climate's Impact on Food Supplies, pp. 23-37. Westview Press, Boulder, Colorado. SCEP (Study of Critical Environmental Problems) (1970). Man's Impact on the Global Ehvironment. MIT Press, Cambridge, Massachusetts.

Science News (1978). The 25-year forecast: Group prediction. 113 (8),116.

Seifert, W. W., and Kamrany, N. M. (1974). A Framework for Evaluating Long-Term Strategies for the Development of the Sahel-Sudan Region; Vol. 1, Summary Report: Project Objectives, Methodologies, and Major Findings. MIT Center for Policy Alternatives, Cambridge, Massachusetts.

Sellers, W. D. (1979). Climate change to the year 2000: A book review. Journal of Applied Meteorology, 60 (6), 686.

Shapley, D. (1976). Crops and climatic change: USDA's forecasts criticized. Science, 193 (September 24), 1222-1224.

Shurcliff, W. A. (1970). SST and Sonic Boom Handbook. Ballantine Books, New York.

Stolarski, R. S., and Cicerone, R. J. (1974). Stratospheric chlorine: A possible sink for ozone. Canadian Journal of Chemistry, 52 (8) (Part 2), 1610-1615.

Talbot, R.. B., and Olorunsola, V. A. (1984). An essay prompted by Rolando García's edited work, Nature Pleads Not Guilty. Climatic Change, 6, forthcoming.

UK Dept. of Environment (1976). Chlorofluorocarbons and Their Effect on Stratospheric Ozone (Pollution Paper No. 5). Her Majesty's Stationery Office, London. 

UK Dept. of Environment (1979). Chlorofluorocarbons and Their Effect on Stratospheric Ozone (Pollution Paper No. 15). Her Majesty's Stationery Office, London. 

UNEP (United Nations Environment Programme) (1981). Environmental Assessment of Ozone Layer Depletion and Its Impact. Bulletin No. 6. Nairobi, Kenya.

UNFAO (United Nations Food and Agriculture Organization (1962). Africa Survey. FAO, Rome.

UNFAO (United Nations Food and Agriculture Organization) (1979). Scanning the future for climatic change. CERES (January-February), 7.

US Congress (1976). FAA Certification of the SST Concorde. Hearings before the Committee on Government Operations (HR: 94th Congress, 1st and 2nd Sessions; November 13, 1975). US Government Printing Office, Washington, DC.

US Congress (1977). Clean Air Act Amendment of 1977. PL 95-95, Section 153 (d). US Government Printing Office, Washington, DC.

Verstraete, M. M. (1984). Review of Drought and Man: The 1972 Case History; Vol. 1, Nature Pleads Not Guilty, by Rolando García. Climatic Change, 6 (forthcoming). 

Willett, J. W. (1981). Preface to Climate Change and the World Grain Economy to the Year 2000: Some Implications for Domestic and International Agricultural PolicyReport on the Third Phase of the [NDU] Climate Impact Assessment (D. G. Johnson). Draft manuscript dated April 1979, cleared for release June 1981. National Defense University, Washington, DC.

WMO (World Meteorological Organization) (1982). Report of the Meeting of Experts on Potential Climatic Effects of Ozone and Other Minor Trace Gases, Boulder, Colorado, September 13-17, 1982. WMO Report 14. Geneva.

Wuebbles, D. J. (1981). Chlorocarbon Emission Scenarios: Potential Impact on Stratospheric Ozone. Lawrence Livermore National Laboratory, University of California, Livermore, California.