Formative Evaluation of CEPA 1999

1.0 Introduction

1.1 Background

The purpose of this formative evaluation was to assess whether the obligations of the Canadian Environmental Protection Act, 1999 (CEPA 1999) are being fulfilled and whether Environment Canada has undertaken the required actions to meet the Act's intent.

A renewed CEPA 1999 received Royal Assent on September 14, 1999, and was proclaimed in force on March 31, 2000. This Act is the federal government's primary piece of environmental protection legislation. It promotes pollution prevention and the protection of the environment and human health in order to contribute to sustainable development. This Act is more than double the length of the previous Canadian Environmental Protection Act (CEPA 1988), increasing from 149 sections to 356 sections. It provided new authorities to allow the Minister of the Environment (the Minister) flexibility in achieving environmental results. It also established requirements, many with specified time frames, that the Minister must meet.

Section 343 of CEPA 1999 requires a review of the Act to be conducted every five years by a committee of one or both of the Houses of Parliament. Such a review is expected in 2005. In support of the upcoming Parliamentary Review, the Minister of the Environment and the Minister of Health are expected to table a formal joint submission to Parliament in advance of the committee's work. This formative evaluation complements the submission of the Ministers. It provides an evidence-based, independent evaluation of progress to date in meeting the Act's obligations and the Department's expected outcomes forCEPA 1999 and its various Parts.

This document contains the results of the "Formative Evaluation of CEPA 1999." It is organized into four main sections:

• Section 1 outlines the purpose and objectives of the evaluation, provides an introduction to the Act and includes a summary of the evaluation methodology.
• Section 2 provides a summary of the detailed findings for each Part and sub-Part of the Act.
• Section 3 documents the evaluation findings from the perspective of the overall Act.
• Section 4 provides conclusions and recommendations.

In addition, the report includes a number of appendices containing more detailed information on key departmental outputs during the evaluation period. Finally, a summary of the views of stakeholders interviewed in support of this evaluation has been included as Annex 1 to the report.

1.2 Evaluation Objectives

The objectives of the evaluation were to:

• examine the degree to which Environment Canada (the Department) is fulfilling its mandatory requirements under CEPA 1999 (listed in Appendix I);
• examine the degree to which the Department has initiated identified priority actions to improve the efficiency and effectiveness of the delivery of CEPA 1999 (listed in Appendix III);
• identify progress to date in meeting the Department's expected outcomes for CEPA 1999 and its various Parts^iv (listed in Appendix IV);
• examine and make recommendations concerning the Department's governance structures, processes and procedures that have been put in place to implement the Act;
• make conclusions and recommendations concerning the above questions; and
• identify any changes that may be required to CEPA 1999 to improve the effectiveness and efficiency of departmental processes.^v

1.3 Overview of CEPA 1999

CEPA 1999 is a federal law jointly administered by Environment Canada and Health Canada with the primary purpose of protecting the environment and contributing to sustainable development through pollution prevention. The Act aims to integrate environmental factors into all decision-making by government and private entities.

The Act commits the Government of Canada to implementing pollution prevention as a national goal and as the priority approach to environmental protection. The Act provides the federal government with new tools to protect the environment and human health and provides a framework for protecting Canadians from pollution caused by "toxic" substances.^vi The Act ensures that the potential risks posed by chemical substances and biotechnology products are properly assessed, establishes strict deadlines for controlling certain toxic substances and requires the virtual elimination of inherently toxic substances that are bioaccumulative, persistent and result primarily from human activity and that are not naturally occurring.

Through CEPA 1999, the Government of Canada expects to demonstrate national leadership, work to minimize overlap and duplication and increase harmonization across and within Canadian jurisdictions.

Box 1 -CEPA 1999 Guiding Principles

• Sustainable Development -- a clean, healthy environment and a strong, healthy economy that meets the needs of the present generation without compromising the ability of future generations to meet their own needs.
• Pollution Prevention --"the use of processes, practices, materials, products, substances or energy that avoid or minimize the creation of pollutants and waste and reduce the overall risk to the environment or human health."
• Virtual Elimination --of releases of substances that are persistent, bioaccumulative, toxic and primarily the result of human activities.
• Ecosystem Approach -- Based on natural geographic units rather than political boundaries, and that considers environmental, social and economic elements that affect the environment as a whole.
• Precautionary Principle --which states that "where there are threats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost-effective measures to prevent environmental degradation."
• Intergovernmental Cooperation --directs the federal government to endeavour to act in cooperation with governments in Canada to ensure that federal actions are complementary to and avoid duplication with other governments.
• National Standards --by providing for the creation of science-based, national environmental standards.
• Polluter Pays Principle --users and producers of pollutants and wastes should bear the responsibility for their actions. Companies or people that pollute should pay the costs they impose on society.
• Science-based Decision-Making --the integral role of science and traditional aboriginal knowledge (where available) in decision-making and that social, economic and technical issues are to be considered in the risk management process.

CEPA 1999 is described as being an "enabling" Act. However, it also imposes a significant number of obligations on Environment Canada, while maintaining many of the obligations that had been established forCEPA 1988 (see Appendix I). Many of these obligations are unique among Canadian jurisdictions. For example, the Act requires the Ministers of the Environment and Health to consider the risks posed by all substances included on the Domestic Substances List and, where appropriate, mandates the Department to propose and finalize risk management measures and tools within specified timelines.

CEPA 1999 is organized into 12 major Parts:

• Part 1 - Administration
• Part 2 - Public Participation
• Part 3 - Information Gathering, Objectives, Guidelines and Codes of Practice
• Part 4 - Pollution Prevention
• Part 5 - Controlling Toxic Substances
• Part 6 - Animate Products of Biotechnology
• Part 7 - Controlling Pollution and Managing Wastes
  • Division 1: Nutrients
  • Division 2: Protection of the Marine Environment from Land-based Sources of Pollution
  • Division 3: Disposal at Sea
  • Division 4: Fuels
  • Division 5: Vehicle, Engine and Equipment Emissions
  • Division 6: International Air Pollution
  • Division 7: International Water Pollution
  • Division 8: Control of Movement of Hazardous Waste and Hazardous Recyclable Material and of Prescribed Non-Hazardous Waste for Final Disposal
• Part 8 - Environmental Matters Related to Emergencies
• Part 9 - Government Operations and Federal and Aboriginal Land
• Part 10 - Enforcement
• Part 11 - Miscellaneous Matters
• Part 12 - Consequential Amendments, Repeal, Transitional Provision and Coming into Force

1.4 Roles and Responsibilities

To achieve the objectives of CEPA 1999, Environment Canada works in partnership with Health Canada, other departments, the provinces and territories, Aboriginal governments, industry and the public.

The governance of CEPA 1999 is the responsibility of a number of parties:

• The Minister of the Environment must administer and implement the Act.
• The Minister of Health is expected to assist the Minister of the Environment and also has specific obligations under the Act.
• The Ministers are advised by a National Advisory Committee, whose representation and mandate are outlined in Part 1 of the Act.
• The Act allows the Minister of the Environment to enter into equivalency and administrative agreements with the provinces and territories and agreements respecting administration with Aboriginal governments and peoples, allowing these other orders of government to implement or administer some aspects of the Act.
• The Governor in Council (Cabinet) responds to the Ministers' recommendations and issue orders and make regulations, where appropriate.
• Parliament must review the Act every five years and has responsibilities for final approval of any changes or amendments to the Act.

The Department has established accountabilities for delivering on the Ministerial obligations and intended outcomes associated with each Part of the Act. These are documented in Appendix II.

1.5 Scope of Evaluation

The evaluation assessed departmental progress in achieving the Department's expected outcomes for CEPA 1999 and its various Parts over the period from its entry into force on March 31, 2000, through December 31, 2004.^viiThe evaluation includes all the programs and activities established under CEPA 1999, including all the programs previously under CEPA 1988 that were continued under CEPA 1999. The evaluation focused on the first 11 substantive Parts of the Act.^viii

During the evaluation period, the administration ofCEPA 1999 fell under the Department's Clean Environment Business Line umbrella (one of four business lines in the Department at the time). The evaluation includes all programs and activities conducted under the Clean Environment Business Line's Air Result and Toxics Result,^ix with the exception of environmental assessment.^x Relevant to CEPA 1999, this Clean Environment Business Line had the following expected outcomes:

• Strategic Outcome: Protection from domestic and global sources of pollution
• Key Result (Air): Improved air quality and minimized human impact on the atmosphere
  • air quality improved
  • emissions and deposition of hazardous air pollutants reduced
  • acid rain deposition reduced
  • stratospheric ozone layer protected and recovered
  • Canadian environmental priorities advanced through partnerships
  • compliance
• Key Result (Toxics): Reduced risk from toxic substances and other substances of concern
  • risks identified and understood for toxics or substances of concern
  • risks managed through strategies, tools and communication
  • pollutants directly managed
  • compliance
  • environmental conditions and impacts monitored, tracked and reported by the Department

Obligations and activities undertaken by Health Canada are excluded from the evaluation, as that department is conducting its own evaluation. The Act's obligations with respect to the role and activities of the Governor in Council (Cabinet) are also excluded.CEPA 1999 also places a significant number of obligations on Persons (e.g., no unauthorized use or disposal of identified substances or wastes). The evaluation undertook an indirect examination of the extent to which these obligations are being met by considering the effectiveness of departmental programs and procedures, including its compliance and enforcement activities. No independent assessment of the actual compliance of Persons with the various obligations was undertaken, however.

Finally, the evaluation did not directly assess the actions and performance of other jurisdictions with equivalency agreements or agreements respecting administration with Aboriginal governments and peoples. Instead, the evaluation focused on the information- sharing and verification and assurance mechanisms instituted by the Department to ensure that the Act's obligations and objectives are satisfied.

1.6 Evaluation Approach

CEPA 1999 is an enabling tool to protect human and environmental health. However, the "use" of the Act alone is not an effective measure of its achievements. In some cases, various Parts of the Act have not yet been used because there is no rationale at this point to apply the particular provisions. In other cases, different Acts, tools and/or instruments can be more effectively used to meet the intent of the CEPA 1999 provisions (e.g., the use of the Fisheries Act or Canada-wide Standards). As a result, the evaluation does not assess whether all Parts of the Act are used and/or used equally, but rather whether the obligations in the Act are being fulfilled and whether the Department is organized in a manner that will enable progress on meeting the intended objectives of the Act.

The evaluation is formative in nature. It measures progress towards achieving results, such as whether systems and procedures are in place to implement the Act and whether the Department is on track to eventually achieve the intended outcomes. It does not evaluate actual environmental outcomes. The Act has not been in place long enough to fully evaluate its impact in terms of environmental results. Actual environmental improvements are noted where evident, however.

Responsibilities for delivering on the various obligations underCEPA 1999 are distributed across the Department, as noted above. Through the 2001-02 CEPA Operational Review exercise, the Department conducted an examination of implementation progress, using an approach that corresponded to the Department's program structure. For the purposes of the upcoming Parliamentary Review, however, the decision was taken to organize the evaluation framework in a manner that corresponds directly to the individual Parts of the Act, as Members of the Parliamentary Committee may be unfamiliar with the Department's organization and likely will want to know the impact of the Act itself. The evaluation focuses on the degree to whichCEPA 1999 has enabled the achievement of the expected outcomes ofCEPA 1999 (as documented in Appendix IV) rather than on the use of various Parts or instruments.

1.7 Evaluation Methodology

This evaluation is "evidence-based." That is, its conclusions and recommendations are based on objective, quantitative and documented evidence to the fullest extent possible. The evaluation was conducted in accordance with the work plan described in the Evaluation Plan prepared by the Department's Audit and Evaluation Branch. The major project Phases included:

• Phase I: Evaluation Planning;
• Phase II: Data Collection and Review;
• Phase III: Analysis and Development of Findings;
• Phase IV: Debriefing; and
• Phase V: Reporting.

Phase I: Evaluation Planning

During Phase I, a project initiation meeting was held with the Department's Evaluation Committee to review and confirm the project's scope and objectives; clarify roles and responsibilities; and finalize the evaluation work plan. A brief examination of the available documentation^xi was conducted to gain a better understanding of the range of documentation available to support the evaluation and to identify any shortcomings. A series of evaluation instruments were then developed, including:

• an evidence collection template to capture and record evidence during the documentation review and analysis phase (see Appendix V);
• a summary evidence template that included the evaluation criteria, and a related guidance document to assist the evaluation team in completing the template in a consistent manner; the a summary evidence template (see Appendix VI) was applied to the analysis of all Parts and related sub-Parts of the Act and provided the vehicle by which the evaluation team reached conclusions on evaluation issues; and
• an interview guide to support the conduct of stakeholder interviews (Appendix VII).

The completed evidence collection templates, summary evidence templates, interview notes and other working notes were submitted to the Audit and Evaluation Directorate upon completion of the evaluation.

Phase II: Data Collection and Review

In Phase II, the evaluation instruments were applied to a review of the documentation made available to the evaluation team. Gaps in the evidence base were documented, and interviews were scheduled with the Department's Accountable Leads (see Appendix II). The Leads were provided with a summary of the gaps in the documentation for their areas of accountability in advance of the interview. The initial analysis was then updated to incorporate any additional documentation or information made available through the interview process.

Forty-five stakeholders were contacted through a parallel process and invited to provide input to the evaluation. Of those contacted, 35 accepted the invitation and agreed to be interviewed. (Appendix VIII includes a list of the individuals and organizations interviewed.) As stakeholder views provided limited hard "evidence" in support of the evaluation, these views have been summarized and presented separately as Annex 1 to this report.

Phase III: Analysis and Development of Findings

In Phase III, the evidence within each area of relevance was analyzed, and preliminary findings were developed. The preliminary findings provided an assessment for each of the evaluation criteria within each Part and sub-Part of the Act. The preliminary findings were presented to the Evaluation Committee.

Presentations on the preliminary findings that were relevant to their individual areas of accountability were then made to the Department's Accountable Leads. The Leads were asked to identify errors and omissions and were requested to provide additional sources of evidence in instances where the preliminary findings were considered to be in error. Additional evidence received was analyzed accordingly, and the preliminary findings were updated.

Draft evaluation findings were then prepared. These draft findings consisted of three presentation decks:

• stakeholder interview results;
• Parts-based findings (the updated preliminary findings); and
• evaluation-wide findings, which integrated the Parts-based and stakeholder interview findings and provided cross-cutting analysis and conclusions concerning implementation of CEPA 1999 over the evaluation period.

Phase IV: Debriefing

In Phase IV, the Evaluation Committee was briefed on the draft findings of the evaluation. Subsequent debriefs were conducted with:

• the Assistant Deputy Minister, Environmental Protection Service;
• the Environmental Protection Service Executive Committee;
• the Deputy Minister;
• Health Canada representatives; and
• the Office of the Minister of the Environment.

The individuals briefed were able to provide comment and feedback on the draft findings. Additional evidence received was analyzed, and the draft findings were updated.

Phase V: Reporting

In Phase V, the draft evaluation findings were documented as a Draft Report, which was submitted to the Evaluation Committee, Accountable Leads and Regional Directors General. Reviewers were asked to provide written comments. The Draft Report was adjusted, where appropriate, and a Final Evaluation Report was prepared and submitted.

1.8 Limitations of the Methodology

There are a number of limitations associated with the methodology used in the evaluation. These include the following:

• The evaluation's reliance on documentary evidence means that the summation of activities and outputs over the evaluation period as documented in this report represents a minimal accounting of what has been done. For example, documentation on additional activities and outputs may not have been made available to the evaluation team, or documentation may not exist for other activities and outputs.
• Documentation provided was not independently verified as being factual and accurate. Much of the documentation was provided in draft form, and the evaluation team needed to assess whether these documents provided clear evidence of actions being planned and/or executed or whether they simply documented a summary of available options.
• CEPA 1999 is jointly administered by Environment Canada and Health Canada. This evaluation, however, excluded actions, outputs and accomplishments of Health Canada. Therefore, the evaluation cannot reach conclusions on the degree to which the relationship between the two departments is functional and effective.
• Although tested with and agreed to by Accountable Leads, the expected outcomes developed and documented for the purposes of assisting this evaluation do not necessarily reflect Government of Canada or department policy and may not have full support from all relevant program authorities.
• The evaluation was expected to consider issues of cost-effectiveness. However, with the exception of annual allocations to CEPA 1999, no information was made available to allow a determination of whether the Department's CEPA-related activities are being executed in a cost-effective manner relative to experiences under CEPA 1988 or to the period prior to the 2001-02 CEPA Operational Review.

^iv The Department has not articulated CEPA-specific outcomes. Nevertheless, a set of expected outcomes has been developed, based on a review of the expected results under the Clean Environment Business Line, the Treasury Board Submission 2003 concerningCEPA implementation, the Act itself and the Government of Canada's response, entitled "Environmental Protection Legislation Designed for the Future - A Renewed CEPA," to the 1995 Report of the Standing Committee responsible for reviewing CEPA 1988. These expected outcomes are documented in Appendix IV.

^v The Department is currently undertaking a parallel, supporting exercise to provide departmental advice for the five-year review (Ministerial Advice Paper). It is this study, and not the evaluation, that will primarily identify and analyze potential themes and questions with respect to provisions of the Act and areas where it could be adjusted to enhance its effectiveness.

^vi UnderCEPA 1999, substances are defined as "toxic" if they enter or may enter the environment in amounts, concentrations, or conditions that:

• have an immediate or long-term effect on the environment or its biological diversity;
• endanger the environment upon which life depends (e.g., chlorofluorocarbons, or CFCs, damage the stratospheric ozone layer, increasing exposure to ultraviolet rays, thereby increasing the risk of skin cancer); or
• endanger human life or health (e.g., lead can hinder development of the human nervous system).

^vii Major activities and outputs that were finalized from January 1 through March 31, 2005, such as newly published regulations in theCanada Gazette, Part II, were also included within the scope of the evaluation.

^viii Part 12 (Consequential amendments, repeal, transitional provision and coming into force) is administrative in nature and not considered relevant to this evaluation.

^ix The Clean Environment Business Line also includes a Climate Change Result. This result, and all climate change-related programs and activities, are excluded from this evaluation.

^x Authorities for environmental assessment activities are provided under theCanadian Environmental Assessment Act, not CEPA 1999.

^xi Documentation was provided to the evaluation team prior to the launch of the evaluation.