Introduction
On March 6 and 7, 1991, an invitational workshop was conducted in Boston to peer review the 1990 Office of Management and Budget report, CURRENT REGULATORY ISSUES IN RISK ASSESSMENT AND RISK MANAGEMENT IN REGULATORY PROGRAM OF THE UNITED STATES GOVERNMENT, APRIL 1, 1990 - MARCH 31, 1991 (OMB Report or Report). The Report was written by several economists who have had significant experience reviewing proposed and final rules issued by federal regulatory agencies, and the purpose of the Workshop was to subject it to rigorous scrutiny by experts in the fields of risk assessment and management.
The OMB Report is significant because the views expressed therein represent the official position of the Executive Office of the President. Although the Report was the subject of internal editorial review, it was not peer reviewed prior to publication and was produced without significant consultation with scientists and risk assessors in academia or the federal government. Much of the Report was based on criticisms of agency practice that have appeared previously in both published literature and regulatory proceedings.
The Workshop included a panel of 15 invited experts and 35 additional participants who expressed an interest.1 Each invited expert was asked to critique a particular portion of the OMB Report. The range of disciplines and affiliations represented gave rise to a rich diversity of viewpoints expressed during two days of discussion. Key OMB staff also participated.
The following summarizes the result of deliberations and highlights the discussion of specific topics. Although it was reviewed by invited Workshop panelists and participants, this is not a consensus document. Indeed, the Workshop was designed to elicit a range of views.
Workshop Summary
Risk managers at federal regulatory agencies are seeking to achieve multiple objectives. They include:
* protection of public health from widespread exposure to toxic agents;
* protection of highly exposed and/or sensitive groups from toxic agents on the basis of equity or fairness, even when exposure is not widespread;
* protection of the natural environment and ecosystems from the adverse effects of toxic agents on behalf of both current and future generations;
* responsiveness to public concerns about human health and environmental risks, even when risk assessors are skeptical about the magnitude of the risks posed by toxic agents; and
* economic efficiency by adopting protective regulations when the marginal social benefits of risk management exceed the marginal social costs of risk management (at least where an agency's legislative mandate does not prohibit consideration of economics).
An adversarial relationship between OMB and the federal regulatory agencies has developed over the years, in part because it is usually impossible to achieve all of these objectives simultaneously. The parties in this adversarial relationship appear to assign differing degrees of importance to the achievement of the various regulatory objectives. Given the different objectives, it should be expected that OMB and the agencies might have different ideas about what is a good risk assessment and what is a good risk management decision.
Several types of risk management decisions are made by the government. At the broadest level, the federal government decides, through a political process involving Congress, the executive branch, and interest groups, how much money and staff will be available at each federal agency to engage in risk management activities. Within each federal agency, decisions are also made about how much money and staff will be allocated to specific risk management programs. Although these resource allocation decisions are of critical national importance, they are not always recognized explicitly as risk management decisions.
Within a particular agency, decisions are made about what rules should be issued and how they should be written. Federal rules address diverse matters such as exposure limits for toxic chemicals in the workplace, tolerance levels for food additives, permissible levels of contaminants in drinking water, and cleanup standards for hazardous waste sites. In addition to national rules, critical decisions are made by regional, state, and local regulatory officials. Cleanup decisions at each hazardous waste site, for example, are made on the basis of feasibility studies, risk assessments, and guidelines established by Environmental Protection Agency (EPA) headquarters and regional offices.
Federal rules are adopted based on legislative mandates that provide risk managers with varying degrees of discretion. When discretion is restricted, it may reflect congressional determination to achieve specific regulatory objectives. Some statues (e.g. the Toxic Substances Control Act) authorize risk managers to weigh the risks, costs, and benefits of alternative courses of action -- even though such factors are not always considered. Other statutes (e.g., the famous Delaney Clause covering carcinogenic food additives and the National Primary Ambient Air Quality Standards under the Clean air Act) compel the federal government to base decisions exclusively on health considerations. Still other statutes order risk managers to reduce human health risk to the maximum extent that is technically and economically feasible (e.g., the permissible exposure limits designed to protect worker health under the Occupational Safety and Health Act).
Although regulatory statutes differ markedly, they rarely specify how agencies are to perform risk assessments of human exposures to potentially harmful agents. Risk assessment practice has therefore evolved at federal agencies and become routinized through informal and formal agency guidelines. While statutory mandates and risk assessment practices differ among agencies, the various regulatory cultures share a common policy viewpoint; that risk managers should err on the side of safety when making regulatory decisions in the face of scientific uncertainty.
OMB's Role
Within this complex administrative process, OMB requires each federal agency to prepare regulatory impact analyses. Working as an agent of the President, it sees the regulatory review process as a device to assure some degree of analytic rigor and economic efficiency in the rulemaking activities of federal agencies. OMB has taken an increasing interest in the technical and policy aspects or risk assessment because risk assessment plays a critical role in rulemaking activities.
White House review of regulatory proposals has been a significant function of the Executive Office of the President since the Ford Administration. Both Democratic and Republican Presidents have seen value in OMB regulatory review, even though such reviews inevitably create an adversarial relationship between regulatory agencies and the Executive Office of the President. While some observers see OMB's regulatory review function as critical to a President's ability to carry out his Constitutional responsibilities and achieve economic and social objectives, others see OMB review as a pernicious barrier to the achievement of the human health and environmental objectives established by the U.S. Congress. Hence, disputes about OMB's role reflect the tensions between Congressional and Presidential authority.
Risk Assessment as a Regulatory Tool
Risk assessment is an analytic tool rather than an end in itself. Since the tool is used for a wide variety of risk management decisions, it is critical that risk assessors and managers forge a constructive collaboration. Risk assessments should address the needs of risk managers in a rigorous, objective, and timely fashion. Some assessments need to be more refined than others, depending on the importance of the decision.
For chemicals that are known or suspected to cause cancer, federal agencies have adopted standard risk assessment procedures. Specific "default" assumptions are considered appropriate when data are insufficient to complete an assessment. The default assumptions are designed to err on the side of safety in the absence of scientific knowledge. In contrast to more refined risk assessments, these standard assessments do not require extensive case-specific data.
Despite their scientific limitations, standard risk assessments of carcinogens are a useful screening tool (i.e., they help identify potential problems and indicate exposures that are not worthy of further concern). They may also provide a basis for regulatory decisions in cases where risks are potentially significant and the estimated costs of risk reduction are too small to justify a more refined risk assessment.
Refined risk assessments should be tailored to the needs of risk managers as the stakes in risk management decisions increase. Federal agencies are striving to develop refined health risk estimates for human exposures to those chemicals (e.g., benzene, dioxin, and formaldehyde) that cannot be eliminated without imposing substantial economic burdens. Refined risk assessments can be especially helpful when pertinent data are available to replace some of the default assumptions used in standard risk assessment.
Risk managers are often faced with the difficult questions of whether to regulate carcinogens on the basis of a standard risk assessment or await a more refined risk assessment. One of the dangers of excessive reliance on refined risk assessments is that regulatory paralysis can result as regulators await additional data. Interestingly, EPA has recently decided to refine its risk assessment of dioxin. In the interim, EPA has decided to continue its current regulatory programs that are designed to reduce human exposure to dioxin.
Congress and federal agencies are now considering new approaches to risk assessment. Dr. Bernard Goldstein is chairing the Committee on Risk Assessment Methodology of the National Academy of Sciences (NAS), which is looking into ways to improve the risk assessment process. Another NAS Committee on risk assessment is being formed in response to the mandates of the Clean Air Act Amendments of 1990. The President's Science Advisor, Dr. D. Allan Bromley, has launched an interagency committee to explore improvements in risk assessment practice while several federal agencies have internal groups working toward the same goal. The EPA is considering revision of its risk assessment guidelines and the EPA Science Advisory Board recently expressed its intent to offer comments aimed at accelerating EPA's review of the guidelines. Hearings on risk assessment were recently held by a subcommittee of the U.S. House of Representatives, which is another indication of the growing interest in reform of the risk assessment practices.
OMB's Major Observations
The three major observations in the OMB Report are quoted below:
1. The continued reliance on conservative (worst-case) assumptions (by federal agencies) distorts risk assessment, yielding estimates that may overstate likely risks by several orders of magnitude.
2. Conservative biases embedded in risk assessment impart a substantial "margin of safety." The choice of an appropriate margin of safety should remain the province of responsible risk-management officials, and should not be preempted through biased risk assessment.2
3. Conservatism in risk assessment distorts the regulatory priorities of the Federal Government, directing societal resources to reduce what are often trivial carcinogenic risks while failing to address more substantial threats to life and health.
Strengths of the OMB Report
The Workshop discussion indicated that the OMB Report correctly identified some important deficiencies in the risk assessment practices of federal regulatory agencies.
1. Federal agencies do not adequately communicate the scientific uncertainties in the cancer risk estimates that are used to justify regulatory decisions. While many actors in the regulatory process recognize the uncertainties in cancer risk assessment, others are not aware of these uncertainties. By neglecting to characterize these uncertainties, federal agencies provide a misleading picture (i.e., false precision) of what is known about cancer risk to regulators, journalists, Congress, and the American people. Key policy judgments about the treatment of uncertainty are often embedded in the risk assessment itself rather than being presented for resolution to the accountable regulatory officials.
2. The cancer risk estimates reported by federal agencies, while often based on uniform assumptions and procedures, contain hidden and nonuniform margins of safety. For example, the use of the linearized multistage model for low-dose extrapolation may generate risk estimates which are more protective for some chemicals than others. These inconsistencies arise because the default procedures are scientifically more appropriate for some chemical carcinogens than for others. While in some cases scientists have clues about these inconsistencies, in other cases scientists do not know which chemicals are the best candidates for departing from standard procedures.
3. Deficiencies in cancer risk assessment may distort the regulatory priorities of the federal government. For example, if standard risk assessments for carcinogens are more protective than those for various noncancer health and safety effects, then they induce the nation to devote too many resources to the control of exposure to selected chemical carcinogens and too few resources to other health problems. Priorities among chemical carcinogens may also be misordered since standard cancer risk estimates are less protective for some carcinogens than for others.
In light of these observations, the OMB Report has raised a red flag about whether America's scarce resources for public health and safety protection are being allocated in the best way.
Weaknesses of the OMB Report
The Workshop discussion exposed several important deficiencies in the OMB Report that should be acknowledged.
1. The OMB Report neglected to mention a variety of factors that may cause cancer risks to be underestimated and underregulated. By neglecting to mention these factors, the OMB Report provides an incomplete and imbalanced account of the biases and scientific uncertainties in risk assessment. The regulation of carcinogens is severely limited by the amount of laboratory animal data and epidemiology. Numerous compounds in widespread use have not been adequately tested for carcinogenicity, even though short-term tests and other data may suggest cause for concern. OMB neglected to note that some carcinogenic agents are underregulated simply because they have not been adequately tested.
2. In its discussion of the biological and statistical issues in risk assessment, the OMB Report makes several misleading and incorrect statements. While these problems are not always highly significant, their cumulative impact is to lessen the scientific quality of the Report. Such errors may have been avoided if the document had been peer reviewed prior to publication. Readers should consult more authoritative references on the key issues in risk assessment.3
3. The major findings of the OMB Report are not based on a systematic review of a random sample of agency risk assessments and regulations. Instead, the Report cites selected examples of agency practice in an ad hoc fashion to buttress its main points. Workshop participants noted several examples where agency practice was different from the impression given in the OMB Report. Some federal agencies and programs appear to have done a better job than others at the difficult tasks of expressing scientific uncertainty and incorporating new scientific information into risk estimates. Workshop participants noted that the federal government has several efforts underway to improve the risk assessment process.
4. The OMB Report would have been stronger if it had contained a comprehensive set of recommendations to correct the deficiencies that were identified. OMB does urge more quantification of uncertainty and greater use of expected values in cancer risk management. The Report does not, however, recommend specific scientific research programs to reduce uncertainties. Nor does OMB recommend programs to develop credible methods for quantifying uncertainty and calculating expected values of risk. More importantly, the OMB Report does not highlight the specific public health and safety risks that have been neglected due to the alleged distortion of priorities in favor of cancer risk management.
Future Directions
In the process of peer reviewing the OMB Report, Workshop participants proposed a variety of steps that might be taken in the future to improve risk assessment and management. Although the Workshop did not attempt to achieve consensus on appropriate steps, they are summarized here for consideration by readers.
* Reporting risk distributions. When risk management decisions are important enough to justify refined risk assessment, analysts should avoid the "tyranny of the single number" by reporting more complete "risk distributions." In some cases numerical distributions may simply reflect scientific uncertainty while in other cases they may reflect known variability in human exposures or sensitivity to toxic agents. These two kinds of distributions should be distinguished and reported separately.
When scientists are unsure about the extent of human exposure or the shape of dose-response curves at low doses, probability distributions should be employed to indicate the extent of uncertainty, thereby taking into account sources of possible conservatism and nonconservatism. The reporting of probability distributions over risk would minimize the hidden policy judgments in risk assessment while forcing risk managers to make explicit policy judgments about what margins of safety are appropriate in risk management. Distributions that highlight uncertainty can also build the case for more scientific research and data collection to reduce uncertainty.
Numerical risk distributions can also be used to elucidate variability in human exposure and sensitivity to toxic agents. Historically, many cancer risk assessments have focused solely on a hypothetical maximally exposed individual. Risk managers should be informed about the full range of human exposures and sensitivities unless there is a compelling policy reason to do otherwise.
In the short run, reporting risk distributions may complicate the tasks of risk managers because it is easier to base decisions on protective point estimates -- especially when the manager's political mandate calls for conservatism in regulatory choice. In the long run, however, numerical risk distributions will better inform everyone about what is at stake in these decisions and thus facilitate more informed political deliberations.
An important limitation of risk distributions is that subjective scientific judgments will be required to quantify at least some of the critical input values and uncertainties (e.g., the shape of the distributions of uncertain biological quantities). Often data will not exist to verify or refute these subjective judgments. There may be situations where subjective judgments are too speculative or polarized to report credible risk distributions. In cases where risk distributions are reported, risk managers and journalists will require training to interpret these distributions properly. Federal agencies have already made some important steps forward in this direction.
* Strategic program of research and data collection. The U.S. should consider implementing an expanded, strategic program of research and data collection to identify and, where possible, reduce uncertainties in risk assessment. A strategic program would focus research resources on the assumptions in risk assessment that have the biggest impact on uncertainty and are resolvable through research and data collection. In the short run, significant payoffs may result from increased application of modern techniques of exposure assessment. In the longer run, an expanded research program on the biological underpinnings of cancer risk assessment is promising. In order to multiply resources and improve the credibility of research, both government and industry should consider making expanded research investment.
* Criteria for departing from default assumptions. Federal agencies should consider adopting a more explicit process for the acceptance of new types of data in risk assessment because some new data are more relevant and valid than others. While current agency guidelines permit departure from default assumptions when warranted by improved science, criteria have not been developed to assess when new information is reliable enough to replace standard assumptions.
It may be easier for agencies to utilize the best science if risk estimates are reported as probability distributions, since probability distributions can be adjusted by scientists to reflect degrees of confidence in new scientific findings. The process of departing from default assumptions should allow for extensive public and scientific comment.
* Noncancer risk assessment. The development of risk assessment methods for noncancer endpoints (e.g., neurological effects and aquatic effects) is critical to making sound regulatory decisions. Reporting noncancer risk estimates may also make it easier for certain cancer risk estimates to be revised downward when new scientific evidence is reassuring. As long as risk managers are presented only cancer risk estimates, there may be a tendency for risk assessors -- consciously or subconsciously -- to retain certain conservative assumptions in cancer risk assessment in order to capture concerns about other environmental damages and human health effects.
* Scrutinize the estimated costs of risk management. In conjunction with efforts to improve estimates of regulatory benefits due to reduced cancer risk, OMB and federal agencies should also consider uncertainty in the estimated costs of risk management decisions. Although some attempts have been made to quantify the total costs of environmental regulation, less information is available on the specific costs of regulations designed to reduce cancer risk from chemical exposure. More serious consideration of the indirect economic benefits of environmental regulation is also required.
Risk managers also need to know whether the projected costs of regulatory compliance, made at the point of a regulatory decision, are unbiased estimates of the actual costs of compliance incurred several years after the implementation of a rule. Legitimate concerns have been raised about deliberate overestimation and underestimation of regulatory costs. The scrutiny of regulatory costs should extend beyond simple compliance costs and include indirect consequences (both positive and negative) for the productivity, degree of innovation, and competitiveness of American industry.
* Improve risk communication. Efforts to improve risk assessment and management should be undertaken with recognition of the daunting challenge of effective risk communication. More refined risk assessments may make it more difficult for risk assessors and managers to communicate risk estimates to nontechnical audiences. At the same time, more effort needs to be made to understand the concerns of communities that may be at risk because of their proximity to hazardous facilities and to make sure that they are addressed by risk assessors and managers. These communities are often populated by poor and minority groups.3
-----
Notes
* The workshop was funded and organized by the Center for Risk Analysis of the Harvard School of Public Health. A full report containing several appendices, including a list of invited panelists and participants is available from the Center.
The authors express special thanks to the invited experts and participants for a stimulating and memorable experience. The moderator, Dr. Paul Deisler, did a fine job of guiding the discussion in productive ways while giving each person adequate time to articulate his or her point of view.
They also appreciate helpful comments on the summary from Sarah Spedden, Doctoral Candidate, Department of Environmental Health.
** Each author is affiliated with the Center for Risk Analysis. Dr. Evans is Associate Professor of Environmental Health; Dr. Graham, Associate Professor of Policy and Decision Sciences; Dr. Gray, Postdoctoral Fellow in Environmental Science and Public Policy; Dr. Hollis, Postdoctoral Fellow in Environmental Science and Public Policy; Dr. Ryan, Associate Professor of Environmental Health; Mr. A. Smith, Doctoral Candidate in Environmental Health; Dr. M. Smith, Postdoctoral Fellow in Environmental Science and Public Policy, and Ms. Taylor, Doctoral Candidate in Environmental Health.
1 Invited panelists were: Melvin Anderson, Paul Anderson, Michael Baram, Barbara Beck, Maureen Cropper, W. Gary Flamm, Dale Hattis, Thomas Hopkins, Paul Lioy, Thomas McKone, Franklin Mirer, Colin Park, Robert Sielken, Thomas Walton, and Lauren Zeise. Their affiliations and the names and affiliation of other participants appear in Appendix A of the full report mentioned above.
2 See, e.g., Appendix B of the full report.
3 The OMB Report and Workshop discussion identified significant flaws in the federal government's approach to risk assessment and management. Although such flaws were apparent, solutions are not so easy to identify. It is time to move beyond criticisms and begin to propose solutions. The various groups working on improvement of the risk assessment process may want to consider and further develop the future directions that were discussed at the Workshop.